Method for processing urbanized big data based on geographic information data

Abstract

The invention discloses a method for processing urbanized big data based on geographic information data, and relates to the technical field of electric digital data processing. The method for processing urbanized big data based on geographic information data comprises the following steps: collecting and processing data, obtaining a time resolution coefficient and an adjustment index, obtaining a comparison result, and obtaining a regulation and control scheme. According to the method, the first time data is obtained by obtaining the time data and performing data preprocessing on the time data, then the time resolution coefficient is obtained based on the first time data, and meanwhile the regulation and control index is obtained according to the first time data and the reference time data; and comparing the obtained time resolution threshold with the time resolution coefficient to obtain a comparison result, and finally evaluating the time resolution based on the comparison result and the regulation index to obtain a regulation scheme, thereby achieving the effect of improving the accuracy of data acquisition frequency regulation when processing urbanized big data. The problem that in the prior art, when urbanization big data is processed, data collection regulation and control are not accurate is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of electronic digital data processing, and in particular to a method for processing urbanization big data based on geographic information data. Background Art

[0002] As the global urbanization process accelerates, a large amount of data related to urban development continues to emerge. These data come from a wide range of sources, including remote sensing images, GPS (Global Positioning System) data, demographics, traffic flow, environmental monitoring, etc. Geographic Information System (GIS) technology has unique advantages in processing and analyzing these multi-source heterogeneous data. GIS can effectively integrate, manage and visualize spatial data, providing accurate geographic positioning and multi-dimensional analysis functions. Through advanced technologies such as spatial databases, data mining and machine learning, GIS can extract valuable information from massive data and identify key trends and patterns in the urbanization process. In the processing of urbanization big data, remote sensing technology and drone telemetry equipment can quickly collect high-resolution geographic information over a large area and monitor land use changes, urban expansion and environmental changes in real time. Big data analysis methods such as cluster analysis, regression analysis and spatial analysis can be used to identify urban growth hotspots, predict future development trends and evaluate urban sustainability.

[0003] Existing technologies can integrate and analyze data from different sources, such as remote sensing images, GPS data, demographic data, and environmental monitoring data, through GIS systems, and achieve accurate monitoring and analysis of key urbanization indicators such as urban spatial pattern, land use change, population distribution, and migration; through data mining, machine learning, and big data analysis, existing systems can extract valuable information from massive data and conduct intelligent prediction and trend analysis. For example, cluster analysis can identify hot spots in urban development, regression analysis can predict future population growth trends, and spatial analysis can evaluate urban sustainability and resource allocation efficiency. These intelligent analysis and prediction functions provide strong decision-making support for urban planning and management, achieve accurate monitoring, real-time data collection, intelligent prediction, and optimal resource allocation, and provide solid technical support for scientific urban decision-making and sustainable development.

[0004] For example, the invention patent with announcement number: CN116756263B announces a method for processing urbanization big data based on geographic information data, including: first, obtaining the geographic information data of the target area for each statistical year, determining multiple living circles, and generating a data warehouse in a three-dimensional coordinate system according to the urbanization big data classification indicators; then obtaining the ODS source data corresponding to each data block through crawler technology, cleaning and standardizing the ODS source data to obtain DW detail layer data; finally, correcting and normalizing the DW detail layer data based on the effective supply rate calculated by the living circle to obtain DA application layer data; this urbanization big data processing method combines geographic information data, and improves data quality and accuracy while realizing effective data management.

[0005] For example, the patent application with publication number: CN113434558A discloses a data collection system of a geographic information ecological chain based on big data, including: a cloud server, a data acquisition unit, a data processing unit and a monitoring control unit. The cloud server is connected to the Internet, and the cloud server is used to provide big data information of the geographic ecological chain; the present invention can search for ecological chain big data in a set area, remove repeated redundant information, reduce the amount of data calculation, model and analyze the data, simulate the ecological chain, and generate simulation results. It can exercise the prediction results of the current ecological chain. At the same time, it matches the corresponding incentives and the corresponding solutions, and transmits the solutions to managers. Managers can understand the ecological chain situation in the area and strengthen the management of the area. Managers can also set the collection area to obtain big data of the corresponding area and realize the acquisition of big data in different areas.

[0006] However, in the process of implementing the technical solution of the invention in the embodiments of the present application, the present application found that the above technology has at least the following technical problems:

[0007] In the existing technology, geographic information data is collected and integrated, and GIS technology is used to perform spatial analysis and data mining on geographic information data. At the same time, cloud computing and big data analysis tools are combined to achieve real-time monitoring, dynamic analysis and intelligent decision-making support for the urbanization process. However, when capturing subtle temporal dynamics such as urban expansion and land use changes to analyze the urbanization process and trends, the processed urbanization big data may be incomplete. Summary of the invention

[0008] The embodiment of the present application solves the problem of inaccurate data collection and control when processing urbanization big data in the prior art by providing a method for processing urbanization big data based on geographic information data, and improves the accuracy of data collection and control when processing urbanization big data.

[0009] The embodiment of the present application provides a method for processing urbanization big data based on geographic information data, including the following steps: S1, real-time acquisition of time data when processing urbanization big data through a GIS system within a preset time period, and data preprocessing of the time data to obtain first time data; S2, obtaining a time resolution coefficient of the preset time period according to the first time data, and obtaining a control index according to the first time data and the reference time data, the time resolution coefficient is used to quantify the time collection density of the preset time period, and the control index is used to quantify the degree of matching between the time resolution and data collection; S3, comparing the obtained time resolution threshold with the time resolution coefficient to obtain a comparison result; S4, evaluating the time resolution of the preset time period based on the comparison result and the control index to obtain a control plan.

[0010] Furthermore, the specific process of processing urbanization big data through the GIS system is as follows: obtaining urbanization-related geographic information data from urban data sources of the urban area to be monitored through the GIS system and uniformly storing them in the GIS system, wherein the urban data sources include satellite remote sensing, spatial databases, and data acquisition sensors; preprocessing the geographic information data, wherein the preprocessing is used to ensure the quality and consistency of the geographic information data; based on the preprocessed geographic information data, using spatial statistical methods to construct a spatial model of urbanization in the urban area to be monitored, wherein the spatial statistical methods include spatial autocorrelation analysis, kernel density analysis, and spatial interpolation, wherein the spatial model is used to predict the development trend and spatial pattern of urbanization in the urban area to be monitored; and visually presenting the spatial model in the form of a map through the GIS system, wherein the map is used to display spatial information, hot spots, and changing trends in the urbanization process.

[0011] Furthermore, the specific process of acquiring the time data is as follows: the number of bytes and the transmission speed of the geographic information data packet are acquired through the GIS system, and the time delay is obtained based on the acquired data, and the time delay is stored in the GIS system, the geographic information data packet represents the data packet obtained by encapsulating the geographic information data, and the time delay represents the data transmission time of the urban data source transmitting the geographic information data to the GIS system; the collection interval of the urban data source is acquired in real time, and the collection interval is transmitted to the GIS system in real time to obtain the time interval, and the time interval represents the length of time between two geographic information data collections.

[0012] Furthermore, the specific process of the data preprocessing is as follows: performing data cleaning on the acquired time data, the data cleaning includes deleting duplicate data and removing outliers, the outliers represent the abnormal data caused by the failure of the town data source or the transmission error; classifying and integrating the time data according to the type of the town data source, and standardizing the time data to obtain the first time data, the classification and integration represents the unified storage of the time data when the town data source collects geographic information data into the GIS system according to the different types of the town data source; removing outliers represents the use of interpolation methods to fill in the outliers in the time data, the interpolation methods include linear interpolation and spline interpolation.

[0013] Furthermore, the method for obtaining the time resolution coefficient is as follows: obtaining the first time data and the time correction factor, the time correction factor is used to correct the time delay difference caused by the difference in data volume when the town data source transmits the geographic information data to the GIS system; obtaining the average time interval from the GIS system, the average time interval represents the average value based on the historical time interval statistics; numbering the preset time periods, and combining the first time data and the time correction factor to obtain the time resolution coefficient, the time resolution coefficient is calculated using the following formula:

[0014]

[0015] Where g represents the number of the preset time period, g = 1, 2, ..., G, G represents the total number of preset time periods, TS g represents the time interval of the g-th preset time period, represents the average time interval, ε represents the time correction factor, TD g Indicates the time delay of the g-th preset time period, TRF g represents the time resolution coefficient of the g-th preset time period, and e represents a natural constant.

[0016] Furthermore, the steps for obtaining the time correction factor are as follows: Step 1, obtaining model data through the GIS system, and dividing the model data into first model data and second model data according to a preset ratio, wherein the model data includes the number of bytes of the geographic information data packet and the corresponding time delay; Step 2, substituting the first model data into a linear regression equation to obtain an initial model of the time correction factor, wherein the initial model of the time correction factor is used to establish a linear relationship between the time delay and the number of bytes of the geographic information data packet; Step 3, obtaining a first time delay according to the established initial model of the time correction factor, and drawing a time delay image through the first time delay and the second time delay, and at the same time according to The time delay image is drawn to obtain the time correction factor, the first time delay represents the time delay corresponding to the geographic information data packet, the second time delay represents the time delay recorded in the GIS, the time delay image is used to visualize the change trend between the first time delay and the second time delay, and the time correction factor is the slope of the curve of the drawn image; step four, substitute the second model data into the time correction factor initial model to obtain the predicted time delay, and derive the mean square error based on the predicted time delay and the second time delay, if the mean square error belongs to the reference error range, use the current time correction factor, otherwise execute steps one to four until the obtained mean square error belongs to the reference error range.

[0017] Furthermore, the method for obtaining the comparison result is as follows: obtain a time resolution threshold, where the time resolution threshold represents an average time resolution index for data collection in the GIS system; compare the time resolution threshold with the time resolution coefficient: if the time resolution coefficient is less than the time resolution threshold, the comparison result is that the time resolution does not meet the standard; if the time resolution coefficient is not less than the time resolution threshold, otherwise the comparison result is that the time resolution meets the standard.

[0018] Furthermore, the specific method for obtaining the control index is as follows: obtaining the time interval and the reference time interval in the first time data through the GIS system, the reference time interval is the reference time length between two data collections when the urban data source collects geographic information data, and the reference time interval is obtained through statistics of the GIS system database; obtaining the reference control range of the time interval, the reference control range represents the time interval interval that meets the data collection density; obtaining the control index according to the time interval in the first time data, the reference time interval and the reference control range, and the control index is calculated using the following formula:

[0019]

[0020] Where g represents the number of the preset time period, g = 1, 2, ..., G, G represents the total number of preset time periods, TS grepresents the time interval of the g-th preset time period, TS0 represents the reference time interval, δ TS Indicates the reference control range of the time interval, CI g+1 Indicates the control index of the g+1th preset time period.

[0021] Furthermore, the process of determining the control scheme is as follows: if the comparison result is that the time resolution does not meet the standard, it indicates that the time interval for collecting data from the urban data source in the current preset time period cannot meet the collection requirements of geographic information data, and the first control scheme is adopted; if the comparison result is that the time resolution meets the standard, it indicates that the time interval for collecting data from the urban data source in the current preset time period meets the collection requirements of geographic information data, and the second control scheme is adopted.

[0022] Furthermore, the first control scheme indicates that the time interval for collecting geographic data in the next preset time period is the first time interval; the first time interval is the product of the time interval of the current preset time period and a first control index, and the first control index represents a control index calculated when the absolute value of the difference between the time interval and the reference time interval is greater than a reference control range of the time interval; the second control scheme indicates that the time interval for collecting geographic data in the next preset time period is the second time interval, and the second time interval is the product of the time interval of the current preset time period and a second control index, and the second control index represents a control index calculated when the absolute value of the difference between the time interval and the reference time interval is not greater than the reference control range of the time interval.

[0023] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0024] 1. The time data when processing urbanization big data is processed by the GIS system to obtain the first time data and the time resolution coefficient. At the same time, the control index is obtained according to the first time data and the reference time data. Then, the obtained time resolution threshold is compared with the time resolution coefficient to obtain a comparison result. Finally, based on the comparison result and the control index, the time resolution of the preset time period is evaluated to obtain a control scheme, thereby achieving more accurate time intervals for collecting data, thereby achieving improved accuracy of data collection and control when processing urbanization big data, and effectively solving the problem of inaccurate data collection and control when processing urbanization big data in the prior art.

[0025] 2. The GIS system is used to obtain the number of bytes and transmission speed of geographic information data packets, and the time delay is obtained based on the obtained data. The time delay is stored in the GIS system. At the same time, the collection interval of the urban data source is obtained in real time, and the collection interval is transmitted to the GIS system in real time to obtain the time interval, thereby obtaining accurate time data and realizing the quantification of time collection density.

[0026] 3. The time interval and reference time interval in the first time data are obtained through the GIS system, and then the reference control range of the time interval is obtained. According to the time interval, reference time interval and reference control range in the first time data, the frequency of data collection is regulated, thereby obtaining more comprehensive geographic information data. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A flow chart of a method for processing urbanization big data based on geographic information data provided in an embodiment of the present application;

[0028] Figure 2 A time resolution coefficient variation diagram provided for an embodiment of the present application. DETAILED DESCRIPTION

[0029] The embodiment of the present application solves the problem of inaccurate data collection and control when processing urbanization big data in the prior art by providing a method for processing urbanization big data based on geographic information data. The method obtains time data in a preset time period in real time when processing urbanization big data using a GIS system, performs data preprocessing on the time data to obtain first time data, and then obtains a time resolution coefficient for the preset time period based on the first time data. At the same time, the time interval and the reference time interval in the first time data are obtained through the GIS system, and a reference control range for the time interval is obtained. A control index is obtained based on the time interval, the reference time interval and the reference control range in the first time data. The obtained time resolution threshold is then compared with the time resolution coefficient to obtain a comparison result. Finally, the time resolution of the preset time period is evaluated based on the comparison result and the control index to obtain a control plan, thereby improving the accuracy of data collection and control when processing urbanization big data.

[0030] The technical solution in the embodiment of the present application is to solve the problem of inaccurate data collection and control when processing urbanization big data. The overall idea is as follows:

[0031] The first time data and the time resolution coefficient are obtained by preprocessing the time data acquired in real time, and the control index is obtained by combining the reference time data. Then, the acquired time resolution threshold is compared with the time resolution coefficient to obtain a comparison result. Finally, a control plan is obtained based on the comparison result and the control index, thereby achieving the effect of improving the accuracy of data collection and control when processing urbanization big data.

[0032] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0033] like Figure 1As shown, it is a flow chart of a method for processing urbanization big data based on geographic information data provided by an embodiment of the present application, and the method includes the following steps: S1, real-time acquisition of time data when processing urbanization big data through a GIS system within a preset time period, and data preprocessing of the time data to obtain first time data, the time data includes time intervals and time delays, the data preprocessing includes data cleaning, data interpolation and data standardization, and the time lengths of the preset time periods are equal; S2, obtaining a time resolution coefficient of the preset time period according to the first time data, and obtaining a control index according to the first time data and the reference time data, the time resolution coefficient is used to quantify the time acquisition density of the preset time period, the reference time data is used to make the time interval density higher, and the control index is used to quantify the degree of matching between the time resolution and data acquisition; S3, comparing the obtained time resolution threshold with the time resolution coefficient to obtain a comparison result, the time resolution threshold is used to determine the time resolution coefficient, and the comparison result includes time resolution that meets the standard and time resolution that does not meet the standard; S4, based on the comparison result and the control index, the time resolution of the preset time period is evaluated to obtain a control scheme, the control scheme includes a first control scheme and a second control scheme, and the control scheme is used to continuously improve the degree of satisfaction of the time resolution with the geographic information data acquisition until the comparison result of the time resolution is that the time resolution meets the standard.

[0034] In this embodiment, a time data processing flow based on a GIS system is described, the purpose of which is to improve the degree of satisfaction of time resolution for geographic information data collection, obtain processed first time data, including time intervals and time delays, and the lengths of preset time periods are equal to ensure data consistency and comparability; calculate the time resolution coefficient based on the first time data to quantify the time collection density and frequency of data in the preset time period, then calculate the control index, and comprehensively evaluate the degree of satisfaction of time resolution for data collection effect to guide subsequent optimization strategies; then compare the calculated time resolution coefficient with the preset time resolution threshold to determine whether the time resolution meets the expected data collection density and frequency requirements; formulate a specific control plan based on the time resolution evaluation results and the control index, the purpose of which is to continuously improve and enhance the time resolution to ensure the efficiency and quality of geographic information data collection; through this systematic process, starting from data acquisition and preprocessing, to time resolution calculation and evaluation, and finally to the formulation of the control plan, the time resolution of geographic information data is optimized to achieve the improvement of data collection control accuracy when processing urbanization big data.

[0035] Furthermore, the specific process of processing urbanization big data through the GIS system is as follows: obtain urbanization-related geographic information data from urban data sources in the urban areas to be monitored through the GIS system and store them in the GIS system in a unified manner. The urban data sources include satellite remote sensing, spatial databases and data acquisition sensors. The geographic information data include urban boundary data, land use data, road network data and population distribution data; preprocess the geographic information data, which includes data cleaning, denoising, missing value filling and data formatting. The preprocessing is used to ensure the quality and consistency of the geographic information data; based on the preprocessed geographic information data, a spatial model of urbanization in the urban areas to be monitored is constructed using spatial statistical methods. The spatial statistical methods include spatial autocorrelation analysis, kernel density analysis and spatial interpolation. The spatial model is used to predict the development trend and spatial pattern of urbanization in the urban areas to be monitored; the spatial model is visualized in the form of a map through the GIS system. The map is used to display spatial information, hot spots and changing trends in the urbanization process.

[0036] In this embodiment, the GIS system is a computing system for storing, analyzing and displaying geographic data. It combines map and database technologies to manage and analyze spatial data. The sources of urbanization-related data are specifically to obtain geographic information from satellite images using satellite remote sensing, store and manage geographic data using spatial databases, and collect geographic information data in real time using data acquisition sensors. Geographic information data refers to data related to geographic location, such as town boundary data (defining the boundaries of town areas), land use data (describing the uses of different lands), road network data (describing road structures and connections), and population distribution data (describing the distribution of population in geographic space). The specific steps of using spatial statistical methods to construct a spatial model of urbanization in the town area to be monitored are as follows: first, collect urbanization-related data, such as population density, building area, land use type, etc., and ensure that the data contains spatial references, such as latitude and longitude coordinates or administrative divisions, etc.; then, clean and convert the data to ensure the integrity and consistency of the data and determine the spatial resolution and unit of the data; use spatial autocorrelation analysis (such as Moran's I index) is used to explore the spatial distribution pattern of urbanization data. The steps include calculating the variable value of each region and the spatial weight between neighboring regions, and then calculating the spatial autocorrelation index to evaluate the strength and direction of spatial correlation; the kernel density estimation method is then used to evaluate the spatial concentration and distribution density of urbanization phenomena. Kernel density analysis can help identify hot spots and sparse areas of urbanization, and use point data to generate a smooth density surface map; in addition, spatial interpolation methods (such as Kriging interpolation and inverse distance weighted interpolation) are used to fill in data gaps and generate continuous urbanization surfaces. The interpolation method infers the values ​​of unknown areas through the values ​​of surrounding known data points in order to more comprehensively understand the distribution and changes of urbanization phenomena; finally, based on the previous analysis results, a suitable spatial statistical model (such as spatial regression model, geographically weighted regression model, etc.) is selected to establish a spatial model of urbanization, taking the urbanization variable as the dependent variable, considering its relationship with other geographical elements (such as terrain, transportation network, etc.), and considering the influence of spatial autocorrelation and heterogeneity; finally, G The IS system displays the analysis results in the form of a map to more intuitively understand the spatial information, hot spots and change trends in the urbanization process. Hot spots are generally concentrated in terms of population density, construction activities, and land use changes. For example, a color gradient (such as from blue to red) is used in a heat map to represent urbanization activities of different intensities. Red indicates a high-intensity hot spot, and blue indicates a low-intensity area. Through the above process, the GIS system can effectively integrate, process and analyze urbanization big data, provide accurate predictions and visualization results, and provide a scientific basis for urban planning and management.

[0037] Furthermore, the specific process of acquiring time data is as follows: the number of bytes and transmission speed of the geographic information data packet are acquired through the GIS system, and the time delay is obtained based on the acquired data, and the time delay is stored in the GIS system. The geographic information data packet represents the data packet obtained by encapsulating the geographic information data, and the time delay represents the data transmission time for the urban data source to transmit the geographic information data to the GIS system; the collection interval of the urban data source is acquired in real time, and the collection interval is transmitted to the GIS system in real time to obtain the time interval, and the time interval represents the length of time between two geographic information data collections.

[0038] In this embodiment, a process of obtaining the number of bytes and transmission speed of a geographic information data packet through a GIS system is described, wherein the number of bytes and the transmission speed of the geographic information data packet are compared to obtain the corresponding time delay, and the general steps of encapsulating the geographic information data into a data packet are specifically as follows: first, the structure and format of the geographic information data are determined, which includes determining the fields contained in each data (such as longitude, latitude, timestamp, data type, etc.), and the encoding format of the data (such as JSON, XML, CSV, etc.); then, the geographic information data is encoded according to the defined data structure, which usually includes serialization of the data fields (converted into a format that can be transmitted on the network, such as: JavaScript Object Notation format, Extendible Markup Language format, Comma-Separated Values ​​format), in addition, the data can be selectively compressed to reduce bandwidth occupancy during transmission; information about the data itself (metadata) is added to the data packet, such as the data source, data generation time, data quality indicators, etc. This information helps to verify and interpret the data at the receiving end; then the protocol header information is added to the beginning of the data packet, including the length, type, checksum, etc. of the data packet to ensure the integrity and correctness of the data, and then the encapsulated data packet is transmitted to the target system or receiver through the network. During the transmission process, the network protocol will route and process the data packet according to the protocol header information; finally, the receiving end parses the data packet according to the protocol header, extracts the geographic information data and its related metadata, and then performs subsequent data processing, storage or display; this process highlights the communication performance between the data source and the GIS system that should be considered in the real-time data acquisition process, and these factors directly affect the timeliness and accuracy of the data; the time delay is stored in the GIS system to represent the transmission time from the town data source to the GIS system, emphasizing the importance of time delay and time interval in data management and analysis, especially in application scenarios that require real-time or near real-time data, emphasizing how to ensure the real-time, integrity and accuracy of the data during the geographic information data collection process. This information is a key factor in evaluating the temporal resolution to support the subsequent spatial analysis of geographic information data using GIS technology.

[0039] Furthermore, the specific process of data preprocessing is as follows: data cleaning is performed on the acquired time data, and data cleaning includes deleting duplicate data and removing outliers. Outliers represent abnormal data caused by failure of urban data sources or transmission errors; the time data are classified and integrated according to the type of urban data sources, and the time data are standardized to obtain the first time data. Classification and integration means that the time data when the urban data source collects geographic information data is uniformly stored in the GIS system according to the different types of urban data sources. Standardization is used to make the time data consistent in unit, dimension and range; removing outliers means using interpolation methods to fill in outliers in the time data. Interpolation methods include linear interpolation and spline interpolation.

[0040] In this embodiment, data cleaning is the process of processing and cleaning raw data, with the purpose of improving the quality and reliability of data, including deleting duplicate data (removing the same data recorded multiple times) and removing outliers (eliminating abnormal data caused by collection or transmission errors); outliers refer to data points that deviate significantly from the normal range, which may be caused by equipment failure, data collection error or transmission error; classification integration is to classify data according to different types of data sources, and store and manage these classified data in a unified manner, for example, data from satellite remote sensing and data from sensors are stored and processed separately; standardization is the process of converting data from different sources and types into data with consistent units, dimensions and ranges, ensuring that different data can be compared and integrated with each other. The data after standardization is called first-time data; interpolation method is a technology to fill missing data or correct abnormal data. Common interpolation methods include linear interpolation (linear estimation between known data points) and spline interpolation (using more complex mathematical functions for estimation). Specifically, linear interpolation is a simple interpolation method that estimates the data of unknown points by drawing a straight line between two known data points, determining the abscissa of the point to be interpolated between the two abscissas, and then using the linear interpolation formula to interpolate the corresponding ordinate; spline interpolation is a method of interpolation using a polynomial function, which is usually smoother than linear interpolation and more suitable for processing complex data. This method first requires determining the known points, then constructing a cubic spline function to determine which coefficients are needed and based on the interpolation conditions and The continuity condition is used to establish a system of equations, and then the system of equations is solved to obtain the coefficients to be determined. Finally, the value of the horizontal axis of the point to be inserted is substituted into the corresponding equation to calculate the value of the vertical axis of the point to be inserted, and the value of the interpolation point is obtained accordingly. Through the above data preprocessing process, the quality and consistency of the time data are ensured, providing a reliable data basis for subsequent analysis and modeling. Data cleaning, classification integration and standardization processing improve the accuracy and comparability of the data. The interpolation method fills the outliers in the data and ensures the integrity of the data. These steps are crucial for the effective processing and analysis of urbanization big data, and help improve the accuracy and reliability of data analysis results.

[0041] Furthermore, the method for obtaining the time resolution coefficient is as follows: obtaining the first time data and the time correction factor, the time correction factor is used to correct the time delay difference caused by the difference in data volume when the urban data source transmits the geographic information data to the GIS system; obtaining the average time interval from the GIS system, the average time interval represents the average value based on the historical time interval statistics; numbering the preset time periods, and combining the first time data and the time correction factor to obtain the time resolution coefficient, the time resolution coefficient is calculated using the following formula:

[0042]

[0043] Where g represents the number of the preset time period, g = 1, 2, ..., G, G represents the total number of preset time periods, TS g represents the time interval of the g-th preset time period, represents the average time interval, ε represents the time correction factor, TD g Indicates the time delay of the g-th preset time period, TRF g represents the time resolution coefficient of the g-th preset time period, and e represents a natural constant.

[0044] The formula of this embodiment combines the time interval, time delay, average time interval and time correction factor to obtain the dependent variable time resolution coefficient TRF through comprehensive analysis. g With the independent variable time interval TS g With independent variable time delay TD g It increases and decreases, indicating that different time intervals and time delays have different degrees of trend influence on the time resolution coefficient. Therefore, this trend change is visualized by drawing a graph and analyzed in detail.

[0045] Specifically, Figure 2 The figure shows the change diagram of the time resolution coefficient provided by the present embodiment. From the figure, it can be seen that the influence of the time interval and the time delay on the time resolution coefficient. As the time interval increases, the height of the surface gradually decreases, indicating that the time resolution coefficient decreases. As the time delay increases, the height of the surface also decreases, but the decrease is relatively small. The range of the time resolution coefficient is from 0 to 1. The overall trend is that as the time interval and the time delay increase, the time resolution coefficient decreases. The time interval has a significant effect on the time resolution coefficient, while the effect of the time delay on the time resolution coefficient is relatively small, but still exists; and there is an interactive effect between the two. When the time interval is small (close to the starting point of the X-axis), the time delay has a greater effect on the time resolution coefficient; when the time interval is large (close to the end of the X-axis), the downward trend of the time resolution coefficient is more obvious, that is, the influence of the time interval on the resolution coefficient is more significant; in general, the time interval is the main factor affecting the time resolution. The increase in the time interval will significantly reduce the time resolution coefficient. At the same time, the time delay also has a certain effect, especially when the time interval is small. By controlling the time interval and the time delay, the time resolution of geographic information data acquisition can be optimized, so that the higher the acquisition density, the more comprehensive the spatial coverage data.

[0046] In this embodiment, the average time interval is set is 5, the time correction factor ε is 0.1, and the time resolution coefficient change data table is shown in Table 1:

[0047] Table 1 Time resolution coefficient change data table

[0048] <![CDATA[Time interval TS g > <![CDATA[Time delay TD g > <![CDATA[Time Resolution Factor TRF g > 3.255293 4.625564 0.763529 3.672908 2.317768 0.872066 3.781599 1.994638 0.889497 6.490146 2.241251 0.872005 8.925324 4.447383 0.696794 9.690922 2.605241 0.741506 9.885498 4.480137 0.658601 ... ... ...

[0049] It can be seen from Table 1 that when the time interval is small (such as the time interval of the first set of data is 3.255293), the corresponding time resolution coefficient is high (such as the time resolution coefficient of the first set of data is 0.763529). As the time interval increases (such as the time interval of the fifth set of data is 8.925324), the time resolution coefficient decreases (such as the time resolution coefficient of the fifth set of data is 0.696794). It can be inferred that there is an inverse relationship between the time interval and the time resolution coefficient, that is, the larger the time interval, the smaller the time resolution coefficient. As the time interval increases, the time resolution coefficient shows a downward trend. This trend shows that a longer time interval leads to a decrease in the density of data collection. The time delay in the table varies from 1.994638 to 4.625564, and the corresponding time resolution coefficient also varies from 0.658601 to 0.889497. It is found from the observation data that a smaller time delay (such as the third The time resolution coefficient corresponding to the time delay of the first group of data is 1.994638 (for example, the time resolution coefficient of the third group of data is 0.889497), the time resolution coefficient corresponding to the larger time delay (for example, the time delay of the first group of data is 4.625564) is lower (for example, the time resolution coefficient of the first group of data is 0.763529), and the smaller time delay usually corresponds to a higher time resolution coefficient. However, this relationship is not as significant as the relationship between the time interval and the time resolution coefficient, which may be affected by the time correction factor ε. In general, the time interval and the time resolution coefficient show an obvious inverse relationship, and there is a certain negative correlation between the time delay and the time resolution coefficient, but it is not as significant as the influence of the time interval. The time interval and the time delay jointly affect the time resolution coefficient. The time resolution coefficient reliably reflects the influence of the time interval and the time delay on the data acquisition frequency, which is helpful for the subsequent regulation of the data acquisition frequency.

[0050] Furthermore, the steps for obtaining the time correction factor are as follows: step 1, obtaining model data through the GIS system, and dividing the model data into first model data and second model data according to a preset ratio, the model data including the number of bytes of the geographic information data packet and the corresponding time delay; step 2, substituting the first model data into the linear regression equation to obtain an initial model of the time correction factor, the initial model of the time correction factor is used to establish a linear relationship between the time delay and the number of bytes of the geographic information data packet; step 3, obtaining a first time delay according to the established initial model of the time correction factor, and drawing a time delay image through the first time delay and the second time delay, and obtaining the time correction factor according to the drawn time delay image, the first time delay representing the time delay corresponding to the geographic information data packet, and the second time delay representing the GIS data packet. The time delay recorded by IS, the time delay image is used to visualize the change trend between the first time delay and the second time delay, and the time correction factor is the slope of the curve of the drawn image; step four, substituting the second model data into the initial model of the time correction factor to obtain the predicted time delay, and deriving the mean square error according to the predicted time delay and the second time delay, if the mean square error belongs to the reference error range, the current time correction factor is used, otherwise, steps one to four are executed until the obtained mean square error belongs to the reference error range, the reference error range is used to verify the time correction factor to improve the accuracy and reliability of the time correction factor, and the reference error range is set according to historical experience.

[0051] In this embodiment, the linear regression equation is a statistical method used to establish a linear relationship between two variables. In this embodiment, it is used to establish the relationship between the number of bytes of the geographic information data packet and the time delay; the mean square error is a statistical indicator for evaluating the error between the predicted value and the actual value, and is used to measure the accuracy of the model prediction.

[0052] Specifically, assuming that the first model data (500, 50), (1000, 90), (1500, 135) and (2000, 180) and the second model data (2500, 230) are obtained according to the ratio of 4:1, wherein the abscissa represents the number of bytes of the data packet and the ordinate represents the time delay, the first model data is substituted into the linear regression equation y=ax+b, and the coefficient a and the intercept b are calculated by using the least squares method:

[0053]

[0054]

[0055] Where n is the number of data points, ∑(xy) is the sum of the products of the horizontal and vertical coordinates of all data points, ∑x is the sum of the horizontal coordinates of all data points, ∑y is the sum of the vertical coordinates of all data points, ∑(x 2) is the sum of the squares of the horizontal coordinates of all data points, and it is calculated that: a≈0.09, b≈0, therefore, the initial model of the time correction factor is: y=0.09x; then the horizontal coordinate of the second model data (2500, 230) is substituted into the initial model of the time correction factor, and the first time delay is calculated to be 225, and then the time delay image is drawn to show the relationship between the actual time delay (230) and the predicted time delay (225) and obtain the image slope, that is, the time correction factor is about 0.002, and finally, the second model data is substituted into the initial model of the time correction factor to obtain the predicted time delay of 225, and the mean square error (MSE) is calculated by the following formula:

[0056]

[0057] Where n is the total number of data points, y i is the ith actual value, y^ i is the i-th prediction value. Assuming that the reference error range is 15 to 20, if the mean square error is not within the reference error range, adjust the time correction factor and repeat the steps: adjust the time correction factor and recalculate the predicted time delay and mean square error until the mean square error belongs to the reference error range; through the above steps, the method of obtaining the time correction factor ensures the accuracy and reliability of the model. The process of data division, initial model establishment, time delay analysis and model verification and optimization helps to accurately determine the relationship between time delay and the number of bytes of geographic information data packets, and improve the accuracy and application effect of the time correction factor. This process is of great significance to the GIS system in dealing with the time delay problem in urbanization big data.

[0058] Furthermore, the comparison result is obtained as follows: obtain the time resolution threshold, which represents the average time resolution index of GIS system data acquisition; compare the time resolution threshold with the time resolution coefficient: if the time resolution coefficient is less than the time resolution threshold, the comparison result is that the time resolution does not meet the standard; if the time resolution coefficient is not less than the time resolution threshold, otherwise the comparison result is that the time resolution meets the standard.

[0059] In this embodiment, the time resolution threshold is an average time resolution index, which represents the standard time resolution of the GIS system during the data acquisition process. It is used to evaluate the time accuracy of data acquisition and is obtained by calculating the average value of the time resolution of historical data in the database; the time resolution coefficient is an index used to actually evaluate the time resolution of a specific data set. It is compared with the time resolution threshold to determine whether the time resolution meets the standard; the time resolution refers to the time accuracy of data acquisition, that is, the degree of detail of the time information recorded by the GIS system; through the above comparison method, it is possible to effectively evaluate whether the time resolution of the GIS system during the data acquisition process meets the standard. This process ensures that the time accuracy of data acquisition meets the expected standard, which is of great significance for improving data quality and reliability.

[0060] Furthermore, the specific method for obtaining the regulation index is as follows: Obtain the time interval in the first time data and the reference time interval through the GIS system. The reference time interval is the reference time length between two data acquisitions when the urban data source collects geographic information data. The reference time interval is obtained through statistics in the GIS system database; obtain the reference regulation range of the time interval. The reference regulation range represents the time interval range that meets the data acquisition density; obtain the regulation index based on the time interval, reference time interval, and reference regulation range in the first time data. The regulation index is calculated using the following formula:

[0061]

[0062] In the formula, g represents the number of the preset time period, g = 1, 2,..., G, where G represents the total number of preset time periods, TS g represents the time interval of the g-th preset time period, TS0 represents the reference time interval, and δ TS represents the reference regulation range of the time interval, and CI g+1 represents the regulation index of the (g + 1)-th preset time period.

[0063] In this embodiment, the formula combines the time interval, reference time interval, and reference regulation range to comprehensively analyze and obtain the regulation index CI g+1 , when TS g > TS0 and the absolute value of the difference between them does not belong to the reference regulation range, then as the time interval TS g increases, the absolute value of the difference between them also increases, and correspondingly, it decreases with the increase of the independent variable time interval TS g ; when TS g < TS0 and the absolute value of the difference between them does not belong to the reference regulation range, then as the time interval TS g decreases, the absolute value of the difference between them increases, and correspondingly, it decreases with the decrease of the independent variable time interval TSg This indicates that the time interval has a significant impact on the regulation index. g -TS0|≤δ TS When the regulation index CI g+1 =1, which means that when the current time interval TS g Close to the reference time interval TS0 and the difference is within the reference control range δ TS When the control index is within 1, it means no adjustment is needed; when |TS g -TS0|>δ TS When the control index needs to be substituted into the formula for specific calculation, this means that the current time interval TS g The difference from the reference time interval TS0 exceeds the reference control range δ TS When the control index is adjusted according to the difference in time intervals, the cubic root and inverse tangent function are used to adjust the index, making the change of the control index smoother. Among them, the time interval with the highest frequency of data collection in the statistical GIS system that meets the data collection requirements is used as the reference time interval, and the reference control range is set according to different data collection requirements and historical data. The application of the control index makes the urban data source stable at different time intervals during data collection. g and the permissible deviation range δ TS , which can ensure that the system's time interval changes will not exceed the expected range, thereby avoiding system instability caused by time intervals that are too large or too small.

[0064] Furthermore, the process of determining the control plan is as follows: if the comparison result is that the time resolution does not meet the standard, it indicates that the time interval for collecting data from the urban data source in the current preset time period cannot meet the collection requirements of geographic information data, and the first control plan is adopted; if the comparison result is that the time resolution meets the standard, it indicates that the time interval for collecting data from the urban data source in the current preset time period meets the collection requirements of geographic information data, and the second control plan is adopted.

[0065] In this embodiment, the control scheme is a specific measure to adjust the data collection time interval and method according to the time resolution comparison result. When the time resolution does not meet the standard, the adjustment measures are taken to improve the time accuracy of data collection and shorten the time interval to meet the demand; when the time resolution meets the standard, the measures taken to maintain or optimize the existing data collection scheme are to ensure that the time accuracy of data collection continues to meet the demand; through the process of determining the control scheme, the time interval of data collection can be flexibly adjusted according to the comparison result of the time resolution to ensure that the time accuracy of data collection meets the preset requirements. When the time resolution does not meet the standard, the first control scheme is adopted to adjust the data collection time interval; when the time resolution meets the standard, the second control scheme is adopted to maintain or optimize the existing data collection method. This process is of great significance to improving the quality and reliability of data collection from urban data sources.

[0066] Furthermore, the first control scheme indicates that the time interval for collecting geographic data in the next preset time period is the first time interval; the first time interval is the product of the time interval of the current preset time period and the first control index, and the first control index represents the control index calculated when the absolute value of the difference between the time interval and the reference time interval is greater than the reference control range of the time interval; the second control scheme indicates that the time interval for collecting geographic data in the next preset time period is the second time interval, and the second time interval is the product of the time interval of the current preset time period and the second control index, and the second control index represents the control index calculated when the absolute value of the difference between the time interval and the reference time interval is not greater than the reference control range of the time interval.

[0067] In this embodiment, the reference time interval is a preset standard time interval, which is used to evaluate whether the current time interval needs to be adjusted. The reference control range is a preset allowable time interval error range, which is used to determine whether the time interval needs to be adjusted. Through a clear control index calculation and time interval adjustment method, the time interval for data collection can be flexibly adjusted to ensure that the time accuracy of data collection meets the preset requirements. If the current time interval deviates too much from the reference time interval, the time interval is adjusted through the first control scheme to shorten the data collection time interval to meet the standard. If the current time interval is within the allowable range, the time interval is maintained or optimized through the second control scheme to ensure the time accuracy and stability of data collection.

[0068] To summarize, the embodiment of the present application obtains first time data and a time resolution coefficient by using the time data of urbanization big data processed by the GIS system, and obtains a control index based on the first time data and the reference time data, and then compares the obtained time resolution threshold with the time resolution coefficient to obtain a comparison result. Finally, based on the comparison result and the control index, the time resolution of the preset time period is evaluated to obtain a control scheme, thereby achieving more accurate time intervals for collecting data, and further achieving improved accuracy of data collection and control when processing urbanization big data, effectively solving the problem of inaccurate data collection and control when processing urbanization big data in the prior art.

[0069] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0070] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0071] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0072] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0073] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0074] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for processing urbanization big data based on geographic information data, characterized in that: The following steps are involved: S1, real-time acquisition of time data when urbanization big data is processed by the GIS system within a preset time period, and data preprocessing of the time data to obtain first time data; S2, obtaining a time resolution coefficient of a preset time period according to the first time data, and obtaining a control index according to the first time data and the reference time data, wherein the time resolution coefficient is used to quantify the time acquisition density of the preset time period, and the control index is used to quantify the matching degree between the time resolution and the data acquisition; S3, comparing the obtained time resolution threshold with the time resolution coefficient to obtain a comparison result; S4, evaluating the time resolution of the preset time period based on the comparison result and the control index to obtain a control plan.

2. The method for processing urbanization big data based on geographic information data as claimed in claim 1, characterized in that: The specific process of processing urbanization big data through the GIS system is as follows: Acquire the urban data sources related to urbanization in the urban area to be monitored through the GIS system and store them in the GIS system; the urban data sources include satellite remote sensing, spatial database and data acquisition sensors; Preprocessing the geographic information data, wherein the preprocessing is used to ensure the quality and consistency of the geographic information data; Based on the preprocessed geographic information data, a spatial model of urbanization in the urban area to be monitored is constructed using a spatial statistical method, wherein the spatial statistical method includes spatial autocorrelation analysis, kernel density analysis and spatial interpolation, and the spatial model is used to predict the development trend and spatial pattern of urbanization in the urban area to be monitored; The spatial model is visualized in the form of a map through the GIS system, and the map is used to display the spatial information, hot spots and changing trends in the urbanization process.

3. The method for processing urbanization big data based on geographic information data as claimed in claim 2, characterized in that: The specific process of obtaining the time data is as follows: The GIS system obtains the number of bytes and the transmission speed of the geographic information data packet and obtains the time delay according to the obtained data, and stores the time delay in the GIS system, wherein the geographic information data packet represents the data packet obtained by encapsulating the geographic information data, and the time delay represents the data transmission time of the town data source transmitting the geographic information data to the GIS system; The collection interval of the town data source is obtained in real time, and the collection interval is transmitted to the GIS system in real time to obtain the time interval, which represents the time length between two geographic information data collections.

4. The method for processing urbanization big data based on geographic information data as claimed in claim 1, characterized in that: The specific process of data preprocessing is as follows: Performing data cleaning on the acquired time data, wherein the data cleaning includes deleting duplicate data and removing outliers, wherein the outliers represent abnormal data caused by a failure of a town data source or a transmission error; The time data is classified and integrated according to the type of town data source, and the time data is standardized to obtain the first time data. The classification and integration means that the time data when the town data source collects geographic information data is uniformly stored in the GIS system according to the different types of town data sources; The removing of outliers means filling the outliers in the time data using an interpolation method, wherein the interpolation method includes linear interpolation and spline interpolation.

5. The method for processing urbanization big data based on geographic information data as claimed in claim 4, characterized in that: The method for obtaining the time resolution coefficient is specifically as follows: Acquire first time data and a time correction factor, wherein the time correction factor is used to correct time delay differences caused by data volume differences when the town data source transmits geographic information data to the GIS system; Obtain an average time interval from the GIS system, where the average time interval represents an average value based on historical time interval statistics; The preset time periods are numbered, and a time resolution coefficient is obtained by combining the first time data and the time correction factor. The time resolution coefficient is calculated using the following formula: Where g represents the number of the preset time period, g = 1, 2, ..., G, G represents the total number of preset time periods, TS g represents the time interval of the g-th preset time period, represents the average time interval, ε represents the time correction factor, TD g Indicates the time delay of the g-th preset time period, TRF g represents the time resolution coefficient of the g-th preset time period, and e represents a natural constant.

6. The method for processing urbanization big data based on geographic information data as claimed in claim 5, characterized in that: The steps for obtaining the time correction factor are as follows: Step 1, obtaining model data through a GIS system, and dividing the model data into first model data and second model data according to a preset ratio, wherein the model data includes the number of bytes of a geographic information data packet and a corresponding time delay; Step 2: Substituting the first model data into a linear regression equation to obtain an initial model of a time correction factor, wherein the initial model of the time correction factor is used to establish a linear relationship between the time delay and the number of bytes of the geographic information data packet; Step three, obtaining a first time delay according to the established time correction factor initial model, and drawing a time delay image through the first time delay and the second time delay, and obtaining a time correction factor according to the drawn time delay image, wherein the first time delay represents the time delay corresponding to the geographic information data packet, and the second time delay represents the time delay recorded by the GIS, and the time delay image is used to visualize the change trend between the first time delay and the second time delay, and the time correction factor is the slope of the curve of the drawn image; Step 4: Substitute the second model data into the time correction factor initial model to obtain the predicted time delay, and obtain the mean square error based on the predicted time delay and the second time delay. If the mean square error belongs to the reference error range, the current time correction factor is used. Otherwise, execute steps 1 to 4 until the obtained mean square error belongs to the reference error range.

7. The method for processing urbanization big data based on geographic information data as claimed in claim 1, characterized in that: The method for obtaining the comparison result is specifically as follows: Obtaining a time resolution threshold, wherein the time resolution threshold represents an average time resolution index of GIS system data collection; Compare the temporal resolution threshold to the temporal resolution factor: If the time resolution coefficient is less than the time resolution threshold, the comparison result is that the time resolution does not meet the standard; If the time resolution coefficient is not less than the time resolution threshold, otherwise the comparison result is that the time resolution meets the standard.

8. The method for processing urbanization big data based on geographic information data as claimed in claim 1, characterized in that: The specific method for obtaining the regulation index is as follows: Acquire the time interval and the reference time interval in the first time data through the GIS system, wherein the reference time interval is the reference time length between two data collections when the town data source collects geographic information data, and the reference time interval is obtained through statistics of the GIS system database; Acquire a reference control range of the time interval, wherein the reference control range represents a time interval that satisfies the data collection density; The control index is obtained according to the time interval in the first time data, the reference time interval and the reference control range, and the control index is calculated using the following formula: Where g represents the number of the preset time period, g = 1, 2, ..., G, G represents the total number of preset time periods, TS g represents the time interval of the g-th preset time period, TS0 represents the reference time interval, δ TS Indicates the reference control range of the time interval, CI g+1 Indicates the control index of the g+1th preset time period.

9. The method for processing urbanization big data based on geographic information data as claimed in claim 1, characterized in that: The process of determining the control scheme is as follows: If the comparison result is that the time resolution does not meet the standard, it means that the time interval for collecting data from the urban data source in the current preset time period cannot meet the collection requirements of geographic information data, and the first control scheme is adopted; If the comparison result shows that the time resolution meets the standard, it means that the time interval for collecting data from the urban data source in the current preset time period meets the collection requirements of geographic information data, and the second control scheme is adopted.

10. The method for processing urbanization big data based on geographic information data as claimed in claim 9, characterized in that: The first control scheme indicates that the time interval for collecting geographic data in the next preset time period is the first time interval; The first time interval is the product of the time interval of the current preset time period and the first control index, and the first control index represents the control index calculated when the absolute value of the difference between the time interval and the reference time interval is greater than the reference control range of the time interval; The second control scheme indicates that the time interval for collecting geographic data in the next preset time period is the second time interval, the second time interval is the product of the time interval of the current preset time period and the second control index, and the second control index represents the control index calculated when the absolute value of the difference between the time interval and the reference time interval is not greater than the reference control range of the time interval.

Images