Map data monitoring and verification method and component-based map data business cloud platform
By calculating the normal distribution and extreme value analysis of the map data change rate, outliers are identified and output, solving the problems of redundant data and detection vulnerabilities in map version updates and achieving efficient outlier detection.
Patent Information
- Application Number
- CN202110470378.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-04-28
- Publication Date
- 2025-09-30
- Estimated Expiration
- 2041-04-28
AI Technical Summary
When updating the map version, the existing technology has the problem of redundant data increasing the workload of data analysis and missing some abnormal changes with large data base, resulting in detection loopholes.
By calculating the rate of change of data volume, performing normal distribution analysis and extreme value analysis, identifying and outputting the first and second outliers, and combining correlation analysis, generating abnormal monitoring results and analysis reports.
It effectively avoids the impact of data cardinality on outlier detection, reduces the workload of data analysis, and improves the accuracy and efficiency of outlier detection.
Smart Images

Figure CN115248773B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of navigation map data analysis, and in particular to a map data monitoring and verification method and a component-based map data service cloud platform. Background Art
[0002] Currently, each version of the map contains a huge amount of data, with more than 2,000 data elements counted. Based on the current 33 provinces in the country, each version will count more than 70,000 records.
[0003] When updating each major map version, all data content and elements must be compared against the previous version's data to identify any unusual changes and ensure data quality. This means that each version of the map will require analysis for 70,000 records. Previous methods for differential analysis of data versions compared the percentage change between the two versions. When the percentage change exceeded a specified range, the difference was extracted for further analysis.
[0004] When using the above method to perform data version analysis, there are many differences due to different data elements. When the cardinality of some element data is small and there are small changes, the percentage will be relatively large; when the cardinality of some element data is large and there are a lot of changes, the percentage will be relatively small. In this case, the abnormal analysis data extracted from each version of the map contains a lot of redundant data, which increases the workload of data analysis. At the same time, some abnormal changes in element data with a large cardinality are omitted, resulting in control loopholes. Summary of the Invention
[0005] The present disclosure mainly provides a map data monitoring and verification method and system, and a component-based map data business cloud platform to solve the problem that redundant data increases the data analysis workload and data base when the map version is updated, thereby affecting the outlier detection.
[0006] To this end, according to a first aspect of the present disclosure, a map data monitoring and verification method is disclosed, which includes the following process:
[0007] Obtain the data volume of the target data element in the electronic map update version database;
[0008] Calculate the corresponding data volume change rate based on the data volume of the same target data element of the updated version to be tested and the comparison version, and form a change rate set corresponding to the data volume change rate by combining the data volume change rates of different versions of the same target data element within the target administrative area;
[0009] Performing a normal distribution analysis on the change rate set to obtain a first change rate set that conforms to the normal distribution and a second change rate set that does not conform to the normal distribution in the change rate set;
[0010] In the normal distribution corresponding to the first change rate set, outputting a data amount change rate in the first change rate set that is greater than a preset threshold as a first abnormal value;
[0011] performing extreme value analysis on the data volume change rates in the second change rate set, and outputting the data volume change rates in the second change rate set that exceed a preset threshold range as second abnormal values;
[0012] Output abnormality monitoring results and analysis reports based on the first abnormal value and / or the second abnormal value.
[0013] As an optional implementation, the process of determining the corresponding data volume change rate according to the data volume and obtaining a change rate set corresponding to the data volume change rate further includes:
[0014] Since the version to be tested is updated, the data volume change rate between the data volumes of corresponding data elements between two adjacent versions is calculated to obtain an inter-version change rate set of inter-version data volume change rates; and / or,
[0015] The data volume change rate between the current data volume corresponding to each data element in the updated tested version under each regional unit and the corresponding data volume in the previous map version is calculated to obtain an inter-regional change rate set of the data volume change rate between regional units.
[0016] As an optional implementation, the process of performing extreme value analysis on the data volume change rate in the second change rate set further includes:
[0017] Obtaining a historical change extreme value corresponding to the target data element, and setting the preset threshold range based on the historical change extreme value corresponding to each data volume change rate in the second change rate set plus or minus the corresponding historical change average value;
[0018] The data amount change rate in the second change rate set that exceeds the preset threshold range is output as the second abnormal value.
[0019] As an optional implementation method, the data volume of the data elements in the electronic map update test version database is obtained as follows:
[0020] Entering the data volume corresponding to each data element into the electronic map update test version database, and calculating the data volume change rate corresponding to the data element;
[0021] If there is a gap in the data volume change rate, the missing data volume change rate is supplemented according to the data volume change rate of the corresponding data element in the updated version to be tested;
[0022] If there is an error in the data volume change rate, the erroneous data volume change rate is corrected according to the historical average value of the data volume change rate.
[0023] As an optional implementation, the map data monitoring and verification method may further include:
[0024] Classifying the data volume change rates of data elements in at least one historical version according to the categories of the data elements, calculating the correlation coefficient between the data volume change rates within each category, and when the correlation coefficient is greater than a strong correlation threshold, the two data elements corresponding to the correlation coefficient constitute a data element group; based on the IDs of the two data elements in the data element group, extracting the data volume change rates of the target data elements corresponding to the data element IDs in each map version from the electronic map update version database to generate a correlation change rate set;
[0025] Performing a fitting calculation on the data volume change rate of the strongly correlated target data element according to the correlation change rate set, and determining a confidence interval according to the result of the fitting calculation and a preset confidence level;
[0026] It is determined whether the data volume change rate of the target data element extracted in the updated version to be tested exceeds the confidence interval, and the data volume change rate exceeding the confidence interval is output as a third abnormal value.
[0027] As an optional implementation, the map data monitoring and verification method may further include:
[0028] Performing an anomaly analysis based on the first anomaly value, the second anomaly value, and / or the third anomaly value to obtain an anomaly result and an analysis report, and updating and verifying the electronic map to-be-tested version database; further comprising:
[0029] The abnormality analysis of the first abnormal value and / or the second abnormal value includes: if the cause of the first abnormal value and / or the second abnormal value is a data problem, re-counting the data volume of the data element corresponding to the first abnormal value and / or the second abnormal value, updating the first abnormal value and / or the second abnormal value in the electronic map test version database according to the calculated data volume change rate, and outputting the first abnormal value and / or the second abnormal value and the corresponding abnormality cause analysis report, wherein the data problem includes an abnormal data volume change rate problem caused by a workflow abnormality; if the cause of the first abnormal value and / or the second abnormal value is not a data problem, updating the first abnormal value and / or the second abnormal value in the electronic map update test version database according to the average value of the historical records corresponding to the first abnormal value and / or the second abnormal value, and outputting the first abnormal value and / or the second abnormal value and the corresponding abnormality cause analysis report, wherein the non-data problem includes an abnormal data volume change rate problem caused by a workflow change;
[0030] The abnormality analysis of the third abnormal value includes: if the cause of the third abnormal value is a data problem, the data volume of the data element corresponding to the third abnormal value is re-counted, and the third abnormal value in the electronic map update version database to be tested is updated by the calculated data volume change rate, and the third abnormal value and the corresponding abnormality cause analysis report are output; if the cause of the third abnormal value is not a data problem, the third abnormal value in the electronic map update version database to be tested is updated with the average value of the historical records corresponding to the third abnormal value, and the third abnormal value and the corresponding abnormality cause analysis report are output.
[0031] According to a second aspect of the present disclosure, a map data monitoring and verification system is also disclosed, the map data monitoring and verification system comprising the following components:
[0032] A data import module is used to obtain the data volume of target data elements in the database of the electronic map update version to be tested;
[0033] a differential statistics module, configured to calculate a corresponding data volume change rate based on the data volume of the same target data element of the updated version to be tested and the comparison version, and to form a change rate set corresponding to the data volume change rate by combining the data volume change rates of different versions of the same target data element within the target administrative area;
[0034] a first analysis module, configured to perform a normal distribution analysis on the set of change rates to obtain a first set of change rates that conforms to the normal distribution and a second set of change rates that do not conform to the normal distribution;
[0035] an anomaly monitoring module, configured to, in a normal distribution corresponding to the first change rate set, identify a data volume change rate in the first change rate set that is greater than a preset threshold as a first anomaly value; and to perform extreme value analysis on the data volume change rates in the second change rate set, identifying a data volume change rate in the second change rate set that exceeds a preset threshold as a second anomaly value;
[0036] The output verification module is used to output an abnormality monitoring result and an analysis report according to the first abnormal value and / or the second abnormal value, and to update and verify the electronic map version database to be tested.
[0037] According to a third aspect of the present disclosure, a component-based map data service cloud platform is also disclosed, which includes the following components:
[0038] User interaction components are used to provide access to the platform through Web, API, or SDK;
[0039] Unified authentication component for identity authentication, functional permission management, access control and / or dynamic configuration;
[0040] A database component for storing electronic map data, coordinates of key charging points, various high-speed line units pre-divided according to the coordinates of the key charging points, and tolls corresponding to the high-speed line units;
[0041] a product editing component configured with an independent data editing space for generating customized data products based on editing, converting, fusing, checking, differencing, batch processing of data in the database component, and / or combining with imported reference data;
[0042] Product release components, used for data and business version management, permission management, incremental data release, real-time push, service launch, traffic control, and service shutdown;
[0043] A business customization component is used to provide customizable map data products and services based on user needs, including matching the travel path with the expressway main line unit, extracting the billing unit corresponding to each expressway main line unit corresponding to the travel path, and outputting it;
[0044] A statistical analysis component is used to execute the map data monitoring and verification method disclosed in any of the aforementioned solutions, as well as to perform usage statistics, hotspot data analysis, operation and maintenance monitoring, and early warning.
[0045] As an optional implementation, the unified authentication component further includes:
[0046] Load balancer, which monitors network ports and dynamically configures network resources;
[0047] API gateway module, used for API routing and access control;
[0048] The authentication module is used to authenticate users and manage their functional permissions.
[0049] According to a fourth aspect of the present disclosure, a computer device is further disclosed, comprising:
[0050] at least one processor; and
[0051] a memory communicatively coupled to the at least one processor;
[0052] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor executes the map data monitoring and verification method disclosed in any of the aforementioned solutions.
[0053] Compared with the prior art, the technical solution of the present disclosure can achieve the following beneficial effects:
[0054] By implementing the technical solution disclosed in the present invention, not only can the influence of data cardinality on outlier detection be avoided, but also data redundancy can be avoided and the workload of data analysis can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS
[0055] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0056] Figure 1 1 is a flowchart of a map data monitoring and verification method provided in the first embodiment of the present disclosure;
[0057] Figure 2 is a schematic diagram of an example provided in the first embodiment of the present disclosure;
[0058] Figure 3 This is a schematic diagram of calculating the standard deviation of a set of inter-region change rates in the first embodiment of the present disclosure;
[0059] Figure 4 This is a schematic diagram of calculating the standard deviation of a set of inter-version change rates in the first embodiment of the present disclosure;
[0060] Figure 5 Schematic diagram of the process of map data monitoring and verification method provided in the second embodiment of the present disclosure;
[0061] Figure 6is a schematic diagram of the confidence interval in the map data monitoring and verification method provided in the second embodiment of the present disclosure;
[0062] Figure 7 A schematic diagram of the map data monitoring and verification system disclosed in an embodiment of the present disclosure;
[0063] Figure 8 A schematic diagram of the composition of the component-based map data business cloud platform disclosed in an embodiment of the present disclosure.
[0064] The above drawings illustrate specific embodiments of the present disclosure, which will be described in more detail below. These drawings and textual descriptions are not intended to limit the scope of the present disclosure in any way, but rather to illustrate the concepts of the present disclosure to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION
[0065] The preferred embodiments of the present disclosure are described in detail below in conjunction with the accompanying drawings so that the advantages and features of the present disclosure can be more easily understood by those skilled in the art, thereby making a clearer and more precise definition of the protection scope of the present disclosure.
[0066] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.
[0067] The map data monitoring and verification method disclosed herein is applicable to the following scenarios: Based on the differences in update elements across map data product lines and varying product update cycles, this technical method can be used for anomaly analysis of provincial data elements, anomaly analysis of national element data, anomaly analysis of single element updates, and anomaly analysis during updates of cloud-based service products. By employing the same technical method to extract different model parameters for different service objects and based on update changes, it can support diverse map element update scenarios and demonstrates universal applicability.
[0068] When each map version is updated, the existing technology needs to combine the data elements of the previous map version and obtain the difference data elements by comparing the data change percentages between the two map versions. When the data cardinality of some elements is small and there are small changes, the percentage will be relatively large; when the data cardinality of some elements is large and there are a lot of changes, the percentage will be relatively small; in this case, the abnormal analysis data extracted from each version will have a lot of redundant data, which increases the workload of data analysis; at the same time, some abnormal changes in the data of elements with a large cardinality are omitted, resulting in detection loopholes.
[0069] Method Example
[0070] In response to the above technical problems, the present disclosure provides a map data monitoring and verification method that can not only avoid the impact of data cardinality on outlier detection, but also avoid data redundancy and reduce the workload of data analysis.
[0071] The map data monitoring and verification method comprises the following steps:
[0072] S1: Obtain the data volume of the target data element in the electronic map update version database;
[0073] S2: Calculate the corresponding data volume change rate based on the data volume of the same target data element in the updated version to be tested and the comparison version, and combine the data volume change rates of the same target data element between different versions within the target administrative area into a change rate set corresponding to the data volume change rate;
[0074] S3: performing normal distribution analysis on the change rate set to obtain a first change rate set that conforms to the normal distribution and a second change rate set that does not conform to the normal distribution.
[0075] S4: In the normal distribution corresponding to the first change rate set, outputting the data amount change rate in the first change rate set that is greater than a preset threshold as a first abnormal value;
[0076] S5: performing extreme value analysis on the data volume change rates in the second change rate set, and outputting the data volume change rates in the second change rate set that exceed a preset threshold range as second abnormal values;
[0077] S6: Outputting abnormality monitoring results and analysis reports based on the first abnormal value and / or the second abnormal value.
[0078] This disclosure uses the data volume change rate as a reference, and simultaneously performs normal distribution analysis and correlation analysis on the data elements in the electronic map update version database. Data elements that do not conform to the normal distribution analysis are subjected to extreme value analysis. When outliers are detected in these three different dimensions, multi-dimensional labeling is performed, and subsequent analysis is conducted to determine whether the anomalies in different dimensions are reasonable. The map data monitoring and verification method provided by this disclosure mainly includes obtaining the data volume of data elements in the electronic map update version database; performing statistics on the data in the data elements in the map database to obtain the data volume of the data elements, and entering the data volume of the data elements in advance into the electronic map update version database. The corresponding data volume change rate is determined based on the data volume, and a corresponding change rate set is obtained. A normal distribution analysis is performed on the change rate set to obtain a first change rate set that conforms to the normal distribution and a second change rate set that does not conform to the normal distribution. Within the normal distribution corresponding to the first change rate set, the data volume change rates in the first change rate set that are greater than a preset threshold are output as first outliers. The data volume change rate corresponding to the data volume is calculated in the electronic map update version database to be tested, and then missing data volume change rates in the database are supplemented and incorrect data volume change rates are corrected. An extreme value analysis is performed on the data volume change rates of the second change rate set that do not conform to the normal distribution analysis, and the data volume change rates in the second change rate set that exceed the preset threshold are output as second outliers. The data volume change rate is used to obtain a change rate set, and the first and second outliers are obtained through normal distribution analysis and extreme value analysis. A correlation analysis is performed on the data volume change rates of the data elements in the electronic map update version database to be tested, and a third outlier is obtained by obtaining a confidence interval. Outliers obtained in three different dimensions are also subjected to anomaly analysis to obtain anomaly results and an analysis report.
[0079] The following further embodiments describe in detail the technical solutions of the present disclosure and how they solve the above-mentioned technical problems. These further embodiments may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments. The following embodiments of the present disclosure are described in conjunction with the accompanying drawings.
[0080] Example 1
[0081] Reference Figure 1 , which is a flowchart of an embodiment of a map data monitoring and verification method disclosed herein, the method may include the following steps:
[0082] Step S101: obtaining the data volume of target data elements in the electronic map update version database.
[0083] Step S102, calculate the corresponding data volume change rate based on the data volume of the same target data element of the updated version to be tested and the comparison version, and form a change rate set corresponding to the data volume change rate by combining the data volume change rates of different versions of the same target data element within the target administrative area.
[0084] Step S103 , performing normal distribution analysis on the change rate set to obtain a first change rate set that conforms to the normal distribution and a second change rate set that does not conform to the normal distribution.
[0085] Step S104: In the normal distribution corresponding to the first change rate set, the data amount change rate in the first change rate set that is greater than a preset threshold is output as a first abnormal value.
[0086] In this example, the same data element, in the same region, has only one record value in the same map version. Data elements vary significantly between regions, so analysis based on their record values can result in significant errors. Since the rate of change in data updates exhibits a relatively stable pattern in historical cyclical changes, the rate of change in the amount of data for the same data element between versions is used as the standard variable for analysis. The rate of change is calculated by dividing the difference between the amount of data for the same data element in the current map version and the amount of data in the previous map version, and then multiplying the difference by a percentage.
[0087] It should be noted that data refers to the actual map data, including all map data elements and related attribute information; data volume is a statistical analysis of map data, which facilitates understanding of the number of data attributes of data elements in each map; data volume change rate reflects the change in the number of attributes or data elements in the map data between adjacent versions. Figure 2 For the data element "door", the statistics of the attribute "vehicle door" in the two map versions are 12655 (current update to be tested version) and 12505 (previous historical version). Based on the data volume, the change rate of vehicle door can be calculated as (12655-12505) / 12505*100%.
[0088] In an optional embodiment, the process of determining the corresponding data volume change rate based on the data volume and obtaining a change rate set corresponding to the data volume change rate further includes: starting from updating the version to be tested, calculating the data volume change rate between the data volumes of corresponding data elements between two adjacent versions, and obtaining an inter-version change rate set of inter-version data volume change rates; and / or calculating the data volume change rate between the current data volume corresponding to each data element in the updated version to be tested under each regional unit and the corresponding data volume in the previous map version, and obtaining an inter-regional change rate set of data volume change rates between regional units.
[0089] In this optional embodiment, provinces are used to represent regional units based on the geographical conditions within the target range. This means that the inter-regional change rate set is the inter-provincial change rate set. Using the data volume change rate describing data updates as the change rate set helps understand the patterns of data changes during historical periodic changes and facilitates locating anomalies.
[0090] In an optional embodiment, the regional units are represented by provinces, and the inter-province change rate set of the inter-province data volume change rate corresponding to each data element and the inter-version change rate set of the corresponding inter-version data volume change rate are each subjected to normal distribution analysis. After normalizing the first change rate set corresponding to the change rate set that conforms to the normal distribution, the standard deviation of the inter-province change rate set in the first change rate set is calculated, and the data volume change rate in the inter-province change rate set that is greater than three times the standard deviation is output as the first outlier, and the first outlier detected in this dimension is marked as 1_1; the standard deviation of the inter-version change rate set in the first change rate set is then calculated, and the data volume change rate in the inter-version change rate set that is greater than three times the standard deviation is output as the first outlier, and the first outlier detected in this dimension is marked as 1_2.
[0091] Optionally, the preset threshold may be three times the standard deviation corresponding to different sets of change rates.
[0092] In an optional embodiment, the rate of change of the data volume of the same data element in two map versions between different provinces can be used as the rate of change set of the rate of change of the data volume between provinces, such as Figure 3 As shown in ; between different historical map versions, the rate of change of the amount of data in the data elements of the two previous map versions is taken as the rate of change of the amount of data between versions, as shown in Figure 4 As shown, normal distributions are plotted for each of these two sets of change rates. Based on the data updates within the same data element, manual verification is required to confirm that the data elements satisfy a normal distribution. For data elements that satisfy a normal distribution, real-world data updates are relatively regular, such as roads, backgrounds, and related road types, speed limits, traffic restrictions, vehicle information, and directional signs. Further data analysis and parameter selection are required for these data elements. Here, triple the standard deviation can be used as an indicator to measure whether the data volume exceeds the preset threshold. The data volume change rate that exceeds the threshold is output as the first outlier.
[0093] It should be noted that based on the data updates between different versions of the same data element, the data elements corresponding to the set of change rates that satisfy the normal distribution can be confirmed through computer programs or manual verification. Since the update characteristic of data elements is the positive addition of data, the normal distribution of the set of change rates corresponding to the data volume change rate of the same data element tends to extend infinitely toward one end.
[0094] In an optional embodiment, a rule module is set according to the importance of the data element to perform outlier detection on the data volume change rate in the map version to be tested. For data elements with high importance and wide impact, an S-level rule module is set. For example, when the data volume change rate of a data element in the map version to be tested is less than three times its standard deviation, and the data volume change rate of the same data element in the historical map version also exceeds its corresponding three times the standard deviation, the S1_1 rule module is set; when the data reduction of the same data element in the historical map version exceeds three times the standard deviation of the data volume change rate recorded in the historical map version, the S1_2 rule module is set; when the data volume of more than n provinces in the map version to be tested has decreased, and the data volume of no province has increased, the S2_1 rule module is set; when the data volume of a certain type of data element has decreased, the S2_2 rule module is set. For other data elements, set A-level rule modules. For example, when there is no change in the data elements in the historical map version, but the data volume change rate of the data elements in the map version to be tested changes, set the A1_1 rule module; when the data volume change rate of the data elements in the historical map version changes, and the change range exceeds three times the standard deviation of the data volume change rate of the data elements in the historical map version, set the A1_2 rule module; when the data volume of the data elements in more than two historical map versions increases and does not decrease, and the data volume of the data elements in the map version to be tested decreases, set the A2_1 rule module; when the data volume change rate of the data elements in the historical map version changes, and the change range of the data volume change rate of the data elements in the map version to be tested exceeds the extreme point of the historical change, set the A2_2 rule module.
[0095] Step S105 , performing extreme value analysis on the data volume change rates in the second change rate set, and outputting the data volume change rates in the second change rate set that exceed a preset threshold range as second abnormal values.
[0096] Step S106: outputting abnormality monitoring results and analysis reports based on the first abnormal value and / or the second abnormal value.
[0097] In an optional embodiment, the process of performing extreme value analysis on the data volume change rate in the second change rate set that does not conform to the normal distribution analysis includes setting a preset threshold range based on the historical change extreme value corresponding to each data volume change rate in the second change rate set and the historical change average value corresponding to the data volume change rate, and outputting the data volume change rate in the second change rate set that exceeds the preset threshold range as a second outlier.
[0098] In this embodiment, for the data volume change rates within the same second change rate set, the sum of the corresponding historical maximum change value and its historical average change value serves as the upper limit of the preset threshold range, while the difference between the corresponding historical minimum change value and its historical average change value serves as the lower limit of the preset threshold range. In the map version being tested, data volume change rates within the corresponding second change rate set that exceed the preset threshold range are output as second anomalies. By defining data element anomalies using historical extreme change values and historical average change values, it is easier to identify anomalies within unstable second change rate sets.
[0099] In an optional embodiment, the changes of certain data elements in reality are unstable, that is, they do not satisfy the normal distribution, such as specific types of POIs, agency stores, etc.; this method calculates the historical change extreme points of the data volume change rate of these data elements and adds or subtracts their corresponding historical change average values as the preset threshold range, and outputs the data volume change rate of the corresponding data elements in the map version to be tested that exceeds the preset threshold range as the second outlier, and marks the second outlier detected in this dimension as 2_1, so as to facilitate subsequent multi-dimensional anomaly analysis of the outliers.
[0100] As an optional implementation, the map data monitoring and verification method may further include:
[0101] Classifying the data volume change rates of data elements in at least one historical version according to the categories of the data elements, calculating the correlation coefficient between the data volume change rates within each category, and when the correlation coefficient is greater than a strong correlation threshold, the two data elements corresponding to the correlation coefficient constitute a data element group; based on the IDs of the two data elements in the data element group, extracting the data volume change rates of the target data elements corresponding to the data element IDs in each map version from the electronic map update version database to generate a correlation change rate set;
[0102] Performing a fitting calculation on the data volume change rate of the strongly correlated target data element according to the correlation change rate set, and determining a confidence interval according to the result of the fitting calculation and a preset confidence level;
[0103] It is determined whether the data volume change rate of the target data element extracted in the updated version to be tested exceeds the confidence interval, and the data volume change rate exceeding the confidence interval is output as a third abnormal value.
[0104] As an optional implementation method, the above-mentioned map data monitoring and verification method may also include: performing anomaly analysis based on the first anomaly value, the second anomaly value, and / or the third anomaly value, obtaining anomaly results and an analysis report, and updating and verifying the electronic map test version database.
[0105] The above embodiment is described below:
[0106] Example 2
[0107] Reference Figure 5 , which shows a flowchart of the map data monitoring and verification method provided in the second embodiment of the present disclosure. The method may also include the following steps.
[0108] Step S500, obtaining the data volume of the data elements in the electronic map update version to be tested database, including entering the data volume of the data elements of each map version into the electronic map update version to be tested database, calculating and obtaining the data volume change rate of the data volume; if there is a gap in the data volume change rate of the data element in the historical map version, then the missing data volume change rate is supplemented according to the data volume change rate of the corresponding data element in the updated version to be tested; if there is an error in the data volume change rate of the data element in the historical map version, then the erroneous data volume change rate is corrected according to the historical average value of the data volume change rate.
[0109] In this embodiment, the data preprocessing operation maintains the integrity of the data, does not eliminate abnormal data volume change rates, and uses correct value replacement or mean smoothing as much as possible for noise data.
[0110] In an optional embodiment, the data preprocessing operation is first performed by warehousing the data volume, in which the original data volume in each map version is processed into the data volume to obtain the data volume change rate; then, the missing data volume change rate is supplemented. Since the statistics of each data element in the historical map version are not exactly the same, the data volume change rate in the data element in the historical map version needs to be supplemented, and no processing is performed on the data volume change rate that needs to be deleted; finally, the historical outlier cleaning operation of the data volume change rate is performed, in which the problematic data volume change rate that has appeared in the historical map version is cleaned, and replaced with the correct value or mean smoothed to reduce the impact on the average value of the data volume change rate in the data element.
[0111] In an optional embodiment, the process of supplementing the data volume change rate of the historical version based on the data volume change rate in the map version to be tested further includes comparing the data volume change rates of all data elements in each historical map version with the data volume change rates of the corresponding data elements in the map version to be tested based on the map version to be tested, and only supplementing the missing data volume change rates in the data elements in each historical map version, and not deleting the redundant data volume change rates.
[0112] In this optional embodiment, the process of supplementing the historical version data based on the data in the map version to be tested may further include supplementing the data volume change rate of the data elements in the previous map version based on the map version to be tested, and then supplementing the data volume change rate of the data elements in the previous map version based on the supplemented previous map version. For example, if there are three map versions, the third version is the map update version to be tested, and the first and second versions are historical map versions, the data volume change rate of the data elements in the second version is supplemented based on the third version, and the data volume change rate of the data elements in the first version is supplemented based on the supplemented second version.
[0113] Step 501: Perform normal distribution analysis on the data volume change rate in the first change rate set.
[0114] Step 502: Perform extreme value analysis on the data volume change rate in the second change rate set.
[0115] In this embodiment, the further implementation process and technical principles of steps S501 to S502 can be found in Figure 1 The relevant descriptions of steps S101 to S106 in the method shown are not repeated here.
[0116] Step S503, based on the correlation analysis, extracting the correlation change rate set further includes, based on the data element ID of each data element in the predetermined data element group with correlation, extracting the data volume change rate of the data element corresponding to the data element ID in each map version in the electronic map update test version database as the correlation change rate set; performing fitting calculation on the data volume change rate in the correlation change rate set, and determining the confidence interval based on the result of the fitting calculation and a preset confidence level; and outputting the data volume change rate in the updated test version that exceeds the confidence interval as a third outlier.
[0117] In an optional embodiment, the data volume change rates of data elements in at least one historical version are classified according to the categories of the data elements to obtain a third change rate set containing data volume change rates of different categories; the correlation coefficients between the data volume change rates in each category in the third change rate set are calculated, and when the correlation coefficient is greater than the correlation threshold, the two data elements corresponding to the correlation coefficient constitute a data element group.
[0118] In this embodiment, the data volume change rates of data elements in at least one historical version are used to identify strongly correlated data elements. The attribute IDs of these strongly correlated data elements are then used to extract the corresponding data elements from each map version. A linear fit is then performed on the data volume change rates of these data elements to determine confidence intervals and detect outliers. First, a third set of data volume change rates is generated based on the categories of the data elements. The correlation coefficients between the data volume change rates within each category in the third set are calculated. When these correlation coefficients exceed a correlation threshold, a group of correlated data elements is generated. Within the third set of data volume change rates, the data volume change rates of the corresponding data elements in the map version under test and the historical map versions are extracted based on the IDs of the two data elements in each data volume change rate group. This creates a set of correlated change rates, which are then linearly fitted. A confidence interval is determined based on the fitting result and a preset confidence level. Data volume change rates in the updated version under test that fall outside the confidence interval are output as third outliers.
[0119] In this embodiment, the categories of data elements mainly include map elements and attribute data, such as road mileage of various levels and functional levels, number of intersections, mileage of construction roads, number of Daoxiangcun index points, lake area, green space area, etc. Classifying data elements according to their categories and calculating the correlation between two data elements of the same category is more accurate; based on the data element ID, the set of correlation change rates corresponding to the data element group is extracted, and the output data element group is manually verified and judged to exclude data element groups that are actually not correlated, making the detection of outliers more accurate.
[0120] In an optional embodiment, the process of outputting the data volume change rate that exceeds the confidence interval as the third outlier further includes, in turn, determining whether the corresponding data volume change rate in each data element group in the map version to be tested is within the confidence interval, outputting the data volume change rate that is not within the confidence interval as the third outlier, and marking the third outlier detected in this dimension as 3_1, to facilitate subsequent multi-dimensional anomaly analysis based on the outlier.
[0121] In an optional embodiment, the data elements in the database of the electronic map update to be tested are first divided into several major categories according to roads, backgrounds, and POIs; the correlation coefficients of the data volume change rates of the data elements in each category are calculated pairwise, and the corresponding two data elements with strong correlation are output. For example, there are 600 groups of data elements corresponding to correlation coefficients greater than the correlation threshold. The output data element groups are manually verified and judged to exclude combinations that do not actually have strong correlations; data elements with strong correlations have strong associations in the map data, such as intersections-limits, intersections-traffic lights, functional types of POI-background surfaces, and the fitting straight line is calculated based on the data volume change rates of the two data elements in the data element group, that is, y=ax+b, and the confidence interval is confirmed by adjusting the variable values a and b. When the data volume change rate of the data elements in the map version to be tested exceeds the confidence interval, an abnormal value is reported. According to the characteristics of the data changes in the map data elements, the change intervals of some data elements are non-negative, such as Figure 6 As shown, the actual confidence interval is the light grey area.
[0122] In this optional embodiment, the confidence interval is confirmed based on the corresponding data in the data element group in the historical map version, the corresponding data volume change rate in each data element group in the map version to be tested is substituted into the confidence interval, and the data volume change rate that is not within the confidence interval is output as an abnormal value.
[0123] Optionally, the present disclosure uses the Pearson correlation coefficient to obtain a confidence interval with a preset confidence level of 95%, and the correlation coefficient threshold is set to a further value greater than 80%.
[0124] In an optional embodiment, after the data volume of each map version is entered into the electronic map update test version database for completion and replacement processing, a differential operation is performed on the data volume change rate, which includes calculating the difference and percentage, and then analyzing based on the differential data. For the data elements for establishing correlation analysis, first obtain the data volume change rate of the historical map version of data element a and data element b, set the obtained data volume change rate in data element a as the x-axis data, and set the obtained data volume change rate in data element b as the y-axis data, and then calculate the correlation coefficients of the data volume change rate in data element a and the data volume change rate in data element b respectively, including but not limited to the Pearson correlation coefficient, Spear rank correlation coefficient and Kendall correlation coefficient. On the premise that data element a and data element b have a strong correlation, fit a straight line according to the points corresponding to the x-axis data and the y-axis data respectively, and calculate the confidence interval based on the three standard deviations corresponding to data element a and data element b. Then, determine in turn whether the data volume change rate in data element a and the data volume change rate in data element b in the map version to be tested are within the confidence interval, and the data volume change rate that is not within the confidence interval is output as the third outlier.
[0125] In the above embodiment, the first data on the x-axis and the first data on the y-axis correspond to one point, the second data on the x-axis and the second data on the y-axis correspond to one point, and so on. The data on the x-axis and the data on the y-axis correspond to multiple points, and straight line fitting is performed on these points.
[0126] Example 3
[0127] The present embodiment discloses a map data monitoring and verification method, which includes obtaining the data volume of data elements in a database of a to-be-tested version of an electronic map update; determining a corresponding data volume change rate based on the data volume, and obtaining a change rate set corresponding to the data volume change rate; performing a normal distribution analysis on the change rate set to obtain a first change rate set that conforms to the normal distribution and a second change rate set that does not conform to the normal distribution in the change rate set; in the normal distribution corresponding to the first change rate set, taking the data volume change rate in the first change rate set that is greater than a preset threshold as a first abnormal value, performing an extreme value analysis on the data volume change rate in the second change rate set, and taking the data volume change rate in the second change rate set that exceeds the preset threshold range as a second abnormal value; performing an abnormal analysis on the first abnormal value and / or the second abnormal value, if the cause of the first abnormal value and / or the second abnormal value is If it is a data problem, the data volume of the data elements corresponding to the first outlier value and / or the second outlier value will be re-counted and the electronic map will be updated using the calculated data volume change rate to update the first outlier value and / or the second outlier value in the tested version database, and the first outlier value and / or the second outlier value and the corresponding abnormal cause analysis report will be output, wherein the data problem includes the problem of abnormal data volume change rate caused by abnormal workflow; if the cause of the first outlier value and / or the second outlier value is not a data problem, the electronic map will be updated with the average value of the historical records corresponding to the first outlier value and / or the second outlier value to update the first outlier value and / or the second outlier value in the tested version database, and the first outlier value and / or the second outlier value and the corresponding abnormal cause analysis report will be output, wherein the non-data problem includes the problem of abnormal data volume change rate caused by workflow changes.
[0128] In this embodiment, workflow changes include changes in data collection planning schemes, data production process changes, etc., among which sudden increases and decreases in abnormal values may be related to data collection planning schemes, data production process changes, etc., which are normal data changes and not data problems. If it is not a data problem, it is generally replaced with the historical average value of the data volume change rate, and then the electronic map update test version database is updated; workflow anomalies include operational errors, among which the change in abnormal values does not comply with factors such as planning schemes and production process changes, but is caused by incorrect operations in a certain link (human or program), etc. This is a data problem. If it is a data problem, the data elements of the actual map of the test version will be corrected, and the corrected statistical values and the data volume change rate obtained by calculation are normal values, and are updated to the electronic map update test version database.
[0129] In an optional embodiment, based on the data element ID of each data element in a predetermined group of data elements with correlation, the data volume change rate of the data element corresponding to the data element ID in each map version is extracted from the electronic map update test version database as a correlation change rate set, a fitting calculation is performed on the data volume change rate in the correlation change rate set, and a confidence interval is determined based on the result of the fitting calculation and a preset confidence level, and the data volume change rate in the update test version that exceeds the confidence interval is taken as a third outlier; an outlier analysis is performed on the third outlier, and if the cause of the third outlier is a data problem, the data volume of the data element corresponding to the third outlier is re-counted and the third outlier in the electronic map update test version database is updated using the calculated data volume change rate, and the third outlier and the corresponding outlier cause analysis report are output; if the cause of the third outlier is not a data problem, the third outlier in the electronic map update test version database is updated using the average value of the historical records corresponding to the third outlier, and the third outlier and the corresponding outlier cause analysis report are output.
[0130] In an optional embodiment, the map data monitoring and verification method further includes performing a correlation analysis on data elements in the electronic map update version database to obtain a third outlier. The data elements in the electronic map update version database to be tested are classified according to their elemental meanings to obtain a third change rate set representing data volume change rates of different categories. A correlation coefficient is calculated between the data volume change rates within each category in the third change rate set. When the correlation coefficient is greater than a correlation threshold, the data elements in the third change rate set that are correlated are grouped as a data element group. Based on the IDs of the two data elements in a data element group, the data volume change rates of the corresponding data elements in the map version to be tested and the historical map version are extracted to obtain a correlation change rate set. A straight line fitting calculation is performed on the data elements. A confidence interval is determined based on the fitting calculation result and a preset confidence level. The data volume change rate in the updated version to be tested that exceeds the confidence interval is output as the third outlier.
[0131] In an optional embodiment, after analyzing the data volume change rate of data elements in the electronic map update version database to be tested, a first outlier value, a second outlier value, and / or a third outlier value are obtained, and each of these values is manually analyzed. If the cause of the anomaly is determined to be a data issue, the data volume and the corresponding data volume change rate in the electronic map update version database to be tested are updated, statistics are re-calculated for the data elements corresponding to the outlier value to obtain the data volume, and a new data volume change rate is calculated as a normal value, which is used to replace the corresponding outlier value in the electronic map update version database to be tested. If the cause of the anomaly is determined not to be a data issue, the corresponding outlier value in the electronic map update version database to be tested is replaced with the historical average of the data volume change rate of the data element corresponding to the outlier value. For example, the first outlier value is manually analyzed. If the cause of the anomaly is determined to be a data issue, the corresponding first outlier value in the database is updated with the normal value corresponding to the first outlier value. If the cause of the anomaly is determined not to be a data issue, the corresponding first outlier value in the database is updated with the historical average of the data volume change rate of the data element corresponding to the outlier value. The second outlier is manually analyzed. If the cause of the second outlier is confirmed to be a data issue, the corresponding second outlier in the database is updated with the normal value corresponding to the second outlier. If the cause of the second outlier is confirmed to be non-data issue, the corresponding second outlier in the database is updated with the average value of the historical records corresponding to the second outlier. The third outlier is manually analyzed. If the cause of the third outlier is confirmed to be a data issue, the corresponding third outlier in the database is updated with the normal value corresponding to the third outlier. If the cause of the third outlier is confirmed to be non-data issue, the corresponding third outlier in the database is updated with the average value of the historical records corresponding to the third outlier.
[0132] Here, the implementation process of the above method is further illustrated with the following examples:
[0133] 1. Data preprocessing
[0134] (1) Data storage: storing the original data of each version into the database;
[0135] (2) Data completion: The statistics of various elements in the historical version of data are not exactly the same, so the historical data needs to be completed.
[0136] (3) Cleaning of historical outliers: Clean the problematic record values that appeared in the historical versions and replace them with correct values to reduce the impact on the average value.
[0137] 2. Select analysis variables
[0138] Data elements vary significantly between provinces, so analysis based on the number of records can result in significant errors. However, the rate of change in data updates exhibits a relatively stable pattern in historical cyclical changes, so the rate of change between versions of the same element is used as the standard variable. Rate of change = (current version - previous version) / previous version * 100%;
[0139] 3. Select data analysis dimensions
[0140] (1) Calculate the standard deviation of normal distribution
[0141] We collected data on the change rates of the same feature across different provinces between two versions and plotted its normal distribution. Based on the data updates, we manually verified which features met the normal distribution criteria. Due to the nature of map feature updates, data is generally added in a positive direction, resulting in a normal distribution that tends to extend infinitely toward one end.
[0142] Data elements that meet the normal distribution are updated more regularly in the real world, such as roads, backgrounds and related road types, speed limits, traffic limits, vehicle information, direction signs, etc. For these elements, further data analysis and parameter selection are carried out. Here, we use three times the standard deviation as an indicator to measure whether the standard change is exceeded, and records that exceed the indicator are output as abnormal records.
[0143] The standard deviation can be calculated using this formula:
[0144] The measure of abnormal changes can be calculated using the formula:
[0145] When Z = 3, it is three times the standard deviation, which is used as an indicator to measure whether the standard variation is exceeded.
[0146] The method for calculating the standard deviation of historical data and the standard deviation of inter-provincial data can refer to the above embodiments and will not be repeated here.
[0147] (2) Calculate confidence interval based on data correlation
[0148] In this step, the elements in the database are divided into several major categories based on roads, backgrounds, and POIs; the correlation coefficients of the elements in each category are calculated pairwise, and the elements with strong correlations are output. There are more than 600 groups of elements with correlations above 80%.
[0149] The output feature groups are manually verified and judged to exclude combinations that are not actually related; strongly related features have strong associations in the map data, such as (intersection-traffic limit, intersection-traffic light, functional category POI-background surface).
[0150] The fitted straight line y=ax+b is calculated based on the two element data of the element group, and the confidence interval is confirmed by adjusting the variable values a and b.
[0151] When the confidence interval is exceeded, an abnormal record is reported. Based on the characteristics of map element data changes and the correlation between attributes, the change interval of some elements is non-negative. As shown in the figure, the actual confidence interval is the yellow area.
[0152] (3) Find abnormal records based on the historical extreme values of data elements
[0153] For some factors, the actual changes are unstable and do not conform to the normal distribution, such as certain types of POIs and agency stores. We calculate the historical change extreme points of these factors plus or minus the average value of the historical changes of the factors as the standard change range, and output out of the range as abnormality.
[0154] Product Examples
[0155] To implement the technical solution of the above method embodiment, this embodiment discloses a map data monitoring and verification system, such as Figure 7 As shown, the map data monitoring and verification system includes the following components:
[0156] A data import module is used to obtain the data volume of target data elements in the database of the electronic map update version to be tested;
[0157] The differential statistics module is used to calculate the corresponding data volume change rate based on the data volume of the same target data element of the updated version to be tested and the comparison version, and to form a change rate set corresponding to the data volume change rate by combining the data volume change rates of different versions of the same target data element within the target administrative area;
[0158] A first analysis module is configured to perform a normal distribution analysis on the change rate set to obtain a first change rate set that conforms to the normal distribution and a second change rate set that does not conform to the normal distribution.
[0159] an anomaly monitoring module configured to, in a normal distribution corresponding to the first change rate set, identify a data volume change rate in the first change rate set that is greater than a preset threshold as a first anomaly value; and to perform extreme value analysis on the data volume change rate in the second change rate set, identifying a data volume change rate in the second change rate set that exceeds a preset threshold as a second anomaly value;
[0160] The output verification module is used to output the abnormality monitoring result and analysis report according to the first abnormal value and / or the second abnormal value, and to update and verify the electronic map version database to be tested.
[0161] In this embodiment, based on the differences in update elements across map data product lines and varying product update cycles, anomaly analysis can be performed for provincial data elements, national element data, single element updates, and updates of cloud-based service products. Using the same technical approach to extract different model parameters based on update changes for different service objects, this approach supports diverse map element update scenarios, demonstrating universal applicability.
[0162] Accordingly, this embodiment also discloses a component-based map data service cloud platform, such as Figure 8 As shown, the component-based map data business cloud platform includes the following components:
[0163] User interaction components are used to provide access to the platform through Web, API, or SDK;
[0164] Unified authentication component for identity authentication, functional permission management, access control and / or dynamic configuration;
[0165] A database component for storing electronic map data, coordinates of key charging points, various high-speed line units pre-divided according to the coordinates of the key charging points, and tolls corresponding to the high-speed line units;
[0166] A product editing component, configured with an independent data editing space, is used to generate customized data products based on editing, transforming, fusing, checking, differencing, batch processing data in the database component, and / or combining with imported reference data;
[0167] Product release components, used for data and business version management, permission management, incremental data release, real-time push, service launch, traffic control, and service shutdown;
[0168] A business customization component is used to provide customizable map data products and services based on user needs. This includes matching travel paths with expressway units, extracting and outputting the billing units corresponding to each expressway unit along the travel path.
[0169] A statistical analysis component is used to execute the map data monitoring and verification method disclosed in any of the aforementioned solutions, as well as to perform usage statistics, hotspot data analysis, operation and maintenance monitoring, and early warning.
[0170] As an optional implementation, the unified authentication component further includes:
[0171] Load balancer, which monitors network ports and dynamically configures network resources;
[0172] API gateway module, used for API routing and access control;
[0173] The authentication module is used to authenticate users and manage their functional permissions.
[0174] According to a fourth aspect of the present disclosure, a computer device is further disclosed, comprising:
[0175] at least one processor; and
[0176] a memory communicatively coupled to the at least one processor;
[0177] The memory stores a computer program that can be executed by at least one processor, and the computer program is executed by at least one processor so that the at least one processor executes the map data monitoring and verification method disclosed in any of the aforementioned solutions.
[0178] The present disclosure provides a map data monitoring and verification system, a component-based map data business cloud platform, and a computer device, which can be used to execute the map data monitoring and verification method described in any of the above embodiments. The implementation principles and technical effects are similar and will not be repeated here.
[0179] In an optional embodiment, each functional unit in a data statistics module of the present disclosure may be directly in hardware, in a software module executed by a processor, or in a combination of the two.
[0180] The software modules may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such that the processor can read information from and write information to the storage medium.
[0181] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any combination thereof. A general-purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP core, or any other such configuration. In the alternative, the storage medium may be integral to the processor. The processor and storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In the alternative, the processor and storage medium may reside as discrete components in the user terminal.
[0182] Example 5
[0183] Reference Figure 8 , which shows the structure of the map cloud platform, which mainly includes:
[0184] A map element database, which is used to store data in the data elements of each map version;
[0185] The data management platform includes the map data monitoring and verification system described in the above embodiment, which is used to process the data in the map element database.
[0186] The present disclosure provides a map cloud platform that can be used to execute the map data monitoring and verification method described in any of the above embodiments. Its implementation principles and technical effects are similar and will not be repeated here.
[0187] In the several embodiments provided in the present disclosure, it should be understood that the disclosed modules and methods can be implemented in other ways. For example, the module embodiments described above are merely illustrative. For example, the division of units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of modules or units, which can be electrical, mechanical or other forms.
[0188] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.
[0189] The above description is merely an embodiment of the present disclosure and does not limit the patent scope of the present disclosure. Any equivalent structural transformation made using the contents of the present disclosure and the drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present disclosure.
Claims
1. A map data monitoring and verification method, characterized in that: include: Obtain the data volume of the target data element in the electronic map update version database; Calculate the corresponding data volume change rate based on the data volume of the same target data element of the updated version to be tested and the comparison version, and form a change rate set corresponding to the data volume change rate by combining the data volume change rates of different versions of the same target data element within the target administrative area; Performing a normal distribution analysis on the change rate set to obtain a first change rate set that conforms to the normal distribution and a second change rate set that does not conform to the normal distribution in the change rate set; In the normal distribution corresponding to the first change rate set, outputting a data amount change rate in the first change rate set that is greater than a preset threshold as a first abnormal value; Performing an extreme value analysis on the data volume change rates in the second change rate set, and outputting the data volume change rates in the second change rate set that exceed a preset threshold range as second abnormal values, wherein the process of performing an extreme value analysis on the data volume change rates in the second change rate set further includes: obtaining historical change extreme values corresponding to the target data element, and setting the preset threshold range based on the historical change extreme value corresponding to each data volume change rate in the second change rate set plus or minus the corresponding historical change average value; outputting the data volume change rates in the second change rate set that exceed the preset threshold range as the second abnormal value; Outputting abnormality monitoring results and analysis reports according to the first abnormal value and / or the second abnormal value; Classify the data volume change rates of data elements in at least one historical version according to the categories of the data elements, calculate the correlation coefficient between the data volume change rates within each category, and when the correlation coefficient is greater than a strong correlation threshold, the two data elements corresponding to the correlation coefficient constitute a data element group; According to the two data element IDs in the data element group, extracting the data volume change rates of the target data elements corresponding to the data element IDs in each map version in the electronic map update version database to generate a correlation change rate set; Performing a fitting calculation on the data volume change rate of the strongly correlated target data elements based on the correlation change rate set, and determining a confidence interval based on the result of the fitting calculation and a preset confidence level; It is determined whether the data volume change rate of the target data element extracted in the updated version to be tested exceeds the confidence interval, and the data volume change rate exceeding the confidence interval is output as a third abnormal value.
2. The map data monitoring and verification method according to claim 1, wherein: The process of determining the corresponding data volume change rate according to the data volume and obtaining a change rate set corresponding to the data volume change rate further includes: Since the version to be tested is updated, the data volume change rate between the data volumes of corresponding data elements between two adjacent versions is calculated to obtain an inter-version change rate set of inter-version data volume change rates; and / or, The data volume change rate between the current data volume corresponding to each data element in the updated tested version under each regional unit and the corresponding data volume in the previous map version is calculated to obtain an inter-regional change rate set of the data volume change rate between regional units.
3. The map data monitoring and verification method according to claim 1, wherein: The method of obtaining the amount of data elements in the electronic map update test version database includes: Entering the data volume corresponding to each data element into the electronic map update version database to be tested, and calculating the data volume change rate corresponding to the data element; If there is a gap in the data volume change rate, the missing data volume change rate is supplemented according to the data volume change rate of the corresponding data element in the updated version to be tested; If there is an error in the data volume change rate, the erroneous data volume change rate is corrected according to the historical average value of the data volume change rate.
4. The map data monitoring and verification method according to any one of claims 1 to 3, characterized in that: Also includes: Performing an anomaly analysis based on the first anomaly value, the second anomaly value, and / or the third anomaly value to obtain an anomaly result and an analysis report, and updating and verifying the electronic map to-be-tested version database; further comprising: The abnormality analysis of the first abnormal value and / or the second abnormal value includes: if the cause of the first abnormal value and / or the second abnormal value is a data problem, re-counting the data volume of the data element corresponding to the first abnormal value and / or the second abnormal value, updating the first abnormal value and / or the second abnormal value in the electronic map test version database according to the calculated data volume change rate, and outputting the first abnormal value and / or the second abnormal value and the corresponding abnormality cause analysis report, wherein the data problem includes an abnormal data volume change rate problem caused by a workflow abnormality; if the cause of the first abnormal value and / or the second abnormal value is not a data problem, updating the first abnormal value and / or the second abnormal value in the electronic map update test version database according to the average value of the historical records corresponding to the first abnormal value and / or the second abnormal value, and outputting the first abnormal value and / or the second abnormal value and the corresponding abnormality cause analysis report, wherein the non-data problem includes an abnormal data volume change rate problem caused by a workflow change; The abnormality analysis of the third abnormal value includes: if the cause of the third abnormal value is a data problem, the data volume of the data element corresponding to the third abnormal value is re-counted, and the third abnormal value in the electronic map update version database to be tested is updated by the calculated data volume change rate, and the third abnormal value and the corresponding abnormality cause analysis report are output; if the cause of the third abnormal value is not a data problem, the third abnormal value in the electronic map update version database to be tested is updated with the average value of the historical records corresponding to the third abnormal value, and the third abnormal value and the corresponding abnormality cause analysis report are output.
5. A map data monitoring and verification system, characterized in that: include: A data import module is used to obtain the data volume of target data elements in the database of the electronic map update version to be tested; a differential statistics module, configured to calculate a corresponding data volume change rate based on the data volume of the same target data element of the updated version to be tested and the comparison version, and to form a change rate set corresponding to the data volume change rate by combining the data volume change rates of different versions of the same target data element within the target administrative area; a first analysis module, configured to perform a normal distribution analysis on the set of change rates to obtain a first set of change rates that conforms to the normal distribution and a second set of change rates that do not conform to the normal distribution; The abnormality monitoring module is used to, in the normal distribution corresponding to the first change rate set, take the data volume change rate in the first change rate set that is greater than a preset threshold as the first abnormal value; and to perform extreme value analysis on the data volume change rate in the second change rate set, and take the data volume change rate in the second change rate set that exceeds the preset threshold range as the second abnormal value, wherein the process of performing extreme value analysis on the data volume change rate in the second change rate set further includes: obtaining the historical change extreme value corresponding to the target data element, and setting the preset threshold range according to the historical change extreme value corresponding to each data volume change rate in the second change rate set plus or minus the corresponding historical change average value; outputting the data volume change rate in the second change rate set that exceeds the preset threshold range as the second abnormal value, classifying the data volume change rate of data elements in at least one historical version according to the category of the data element, and calculating the correlation coefficient between the data volume change rates in each category. When the correlation coefficient is greater than the strong correlation threshold, the two data elements corresponding to the correlation coefficient constitute a data element group; according to the two data element IDs in the data element group, in the electronic map update test version database, Extracting the data volume change rate of the target data element corresponding to the data element ID in each map version to generate a set of correlation change rates; performing a fitting calculation on the data volume change rate of the strongly correlated target data element based on the set of correlation change rates, and determining a confidence interval based on the result of the fitting calculation and a preset confidence level; determining whether the data volume change rate of the target data element extracted in the updated version to be tested exceeds the confidence interval, and outputting the data volume change rate that exceeds the confidence interval as a third outlier; The output verification module is used to output an abnormality monitoring result and an analysis report according to the first abnormal value and / or the second abnormal value, and to update and verify the electronic map version database to be tested.
6. A component-based map data business cloud platform, characterized by: include: User interaction components are used to provide access to the platform through Web, API, or SDK; Unified authentication component for identity authentication, functional permission management, access control and / or dynamic configuration; A database component for storing electronic map data, coordinates of key charging points, various high-speed line units pre-divided according to the coordinates of the key charging points, and tolls corresponding to the high-speed line units; a product editing component configured with an independent data editing space for generating customized data products based on editing, converting, fusing, checking, differencing, batch processing of data in the database component, and / or combining with imported reference data; Product release components, used for data and business version management, permission management, incremental data release, real-time push, service launch, traffic control, and service shutdown; A business customization component is used to provide customizable map data products and services based on user needs, including matching the travel path with the expressway main line unit, extracting the billing unit corresponding to each expressway main line unit corresponding to the travel path, and outputting it; A statistical analysis component for executing the map data monitoring and verification method as described in any one of claims 1 to 4, and for performing usage statistics, hotspot data analysis, operation and maintenance monitoring, and early warning.
7. The component-based map data service cloud platform according to claim 6, characterized in that: The unified authentication component further includes: Load balancer, which monitors network ports and dynamically configures network resources; API gateway module, used for API routing and access control; The authentication module is used to authenticate users and manage their functional permissions.
8. A computer device, characterized in that: include: at least one processor; as well as a memory communicatively coupled to the at least one processor; The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor executes the map data monitoring and verification method according to any one of claims 1 to 4.
Citation Information
Patent Citations
Testing method and device for electronic map data
CN104156357A
Method and device for detecting data access exception
CN106815255A
Geographical provincial situation monitoring database management system and method, and database
CN110704569A