Data optimization acquisition method and system applied to urban planning

Abstract

The invention relates to the technical field of data processing, and provides a data optimization acquisition method and system applied to urban planning, and the method comprises the steps: obtaining related data for urban planning, and recording the type of each related data, and the geographic position distribution, storage space and priority of each node in a platform; analyzing an access relationship, and determining an access correlation factor of each node to each type of related data in combination with the type of each related data; analyzing the size relationship between the related data and the storage space of each node to obtain a storage adaptation factor of each related data to each node; analyzing the priority and the access related factor of each node, and determining the storage relationship between each related data and each node according to the geographic position distribution of each node; related data are collected and then stored in a distributed mode. The invention aims to solve the problem that the calling of related data is affected due to the fact that the urban planning data cannot be effectively stored in the acquisition process due to huge data volume.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a data optimization collection method and system applied to urban planning. Background Art

[0002] Collecting urban planning-related data to carry out modern urban development planning has become an important means of urban construction planning. Urban planning-related data include but are not limited to basic geographic information data, such as remote sensing image data, terrain elevation data, ecological planning data, business audit data, government data, network open source data, etc.; by integrating and analyzing urban planning data, more comprehensive and accurate information support can be provided for urban development planning.

[0003] Since urban planning related data contains many types of data and complex data styles, including but not limited to big data, image data, three-dimensional image data, document reports and even code data, usually multiple types of data of the same urban planning exist in the same urban planning data, and the data packet size will be large. In the process of collecting urban planning data, its storage process is subject to certain tests. If the urban planning related data is collected and stored in the storage center, the user call process will be affected by transmission delay and storage space. The collection process will reduce the storage efficiency due to the storage of data packets one by one, and seriously reduce the user's reading efficiency of urban planning data. Therefore, distributed storage can be used to realize data storage for urban planning related data; however, there are storage space limitations on nodes in the distributed storage process. At the same time, different nodes have different requirements for urban planning related data. If storage is only allocated based on the collection process and access frequency, the geographical location relationship and access relationship between urban planning data and each node will be ignored, and the storage efficiency after collection cannot be effectively improved for users of each node. Summary of the invention

[0004] The present invention provides a data optimization collection method and system for urban planning to solve the problem that the existing urban planning data cannot be effectively stored during the collection process due to the huge amount of data, which affects the call of related data. The technical solution adopted is as follows: The present invention proposes a data optimization collection method for urban planning, which comprises the following steps: Obtain relevant data for urban planning through the urban planning management platform, record the type of each relevant data, the geographical location distribution, storage space and priority of each node in the platform, and obtain the access relationship between each node and each relevant data; Analyze the access relationship between each node and each related data, and determine the access correlation factor of each node to each type of related data in combination with the type of each related data; analyze the size relationship between each related data and the storage space of each node according to the collection process of each related data, and obtain the storage adaptation factor of each related data for each node; Analyze the priority of each node and its access-related factors to each type of related data, and determine the storage relationship between each related data and each node based on the geographical location distribution of each node and the storage adaptability factor; Based on the storage relationship between each relevant data and each node, each relevant data is collected and distributedly stored.

[0005] Optionally, the analyzing the access relationship between each node and each related data, and determining the access correlation factor of each node to each type of related data in combination with the type of each related data, includes the following specific methods: According to the access frequency of each node to each relevant data in each week and the type of each relevant data, the total access frequency of each node to each type of relevant data in each week is obtained; The ratio of the total access frequency of any node to any type of related data in any week to the sum of the access frequencies of the node to all related data in that week is taken as the relative frequency of the node's access to the related data of that type in that week; Based on the ratio of the mean to the standard deviation of the relative frequency of the node's access to this type of related data each week, combined with the similarity relationship between the total frequency of the node's access to this type of related data each week and the sum of the node's access frequencies to all related data each week, the node's access correlation factor for this type of related data is obtained.

[0006] Optionally, the specific method of analyzing the size relationship between each relevant data and the storage space of each node according to the collection process of each relevant data to obtain the storage adaptation factor of each relevant data for each node includes: The most recently collected relevant data is recorded as the current relevant data, and the type corresponding to the current relevant data is recorded as the current type relevant data; The ratio of the mean access frequency of each node to each related data under the current type of related data to the mean total access frequency of all nodes to each related data under the current type of related data is taken as the first The access frequency of each node to the data related to the current type; The first The ratio of the access frequency of a node to any related data under the current type of related data to the total access frequency of all nodes to the related data under the current type of related data is taken as the first The access frequency of each node to the related data under the current type of related data; According to the access frequency of each node to each type of related data or its related data, combined with the geographical location distribution and storage space of the nodes, the storage adaptation factor of each related data for each node is obtained.

[0007] Optionally, the method of obtaining the storage adaptability factor of each relevant data for each node based on the access frequency of each node to each type of relevant data or the relevant data thereunder, in combination with the geographical location distribution and storage space of the node, includes the following specific methods: The current relevant data for Storage Adaptability Factor of Nodes The calculation method is: in, Indicates The access frequency of nodes to data related to the current type, Indicates The geographical distance between each node and the node where the current relevant data is collected, Indicates the current data packet size of the relevant data, Indicates The storage space size of each node, represents an exponential function with a natural constant as base; No. The relevant data for Storage Adaptability Factor of Nodes The calculation method is: in, Indicates Node pair The access frequency of relevant data, Indicates The node and The geographical location distance of the nodes where the relevant data is collected, Indicates The data packet size of the relevant data.

[0008] Optionally, the analysis of the priority of each node and its access-related factors to each type of related data, and determining the storage relationship between each related data and each node based on the geographical location distribution of each node and the storage adaptability factor, includes the following specific methods: Based on the access-related factors of each node to each type of related data and the priority of each node, the storage priority of each node to each type of related data is obtained; According to the storage priority of each node for each type of related data, the delay influence of different nodes on the transmission of related data is analyzed, and the storage adaptation factor is adjusted to obtain the storage non-essential factor of each related data for each node; Based on the storage non-essential factors and in combination with the size relationship between the relevant data and the storage space of the node, a storage model of each node is constructed; By optimizing the storage model of each node, the storage relationship between each related data and each node is determined.

[0009] Optionally, the storage priority of each node for each type of related data is analyzed, the delay influence of different nodes on the transmission of related data is adjusted, and the storage adaptation factor is obtained to obtain the storage non-essential factor of each related data for each node, including the specific method of: For any node and other nodes, based on the storage priority of the current type of related data, the storage priority difference between the node and other nodes in the current type of related data is obtained; Based on the geographical location distance between different nodes and the data packet size of the current relevant data, the transmission interference factor of the current relevant data between different nodes is obtained by data transmission speed and signal transmission speed, combined with the storage priority difference of the current type of relevant data of different nodes, and the transmission interference factor of the current relevant data between different nodes is negatively correlated with the storage priority difference; Based on the transmission interference factor of the current relevant data between different nodes, the storage adaptation factor of the current relevant data for each node is adjusted to obtain the storage non-essential factor of the current relevant data for each node.

[0010] Optionally, the method of obtaining the non-essential factors of storage of the current relevant data for each node includes: Subtract 1 from the current relevant data for the The difference between the storage adaptation factor of the node and the current relevant data for the The product of the transmission interference factors of the nodes is used as the current relevant data for the The storage of each node is not a necessary factor.

[0011] Optionally, the storage model of each node is constructed based on the storage non-essential factor in combination with the size relationship between the relevant data and the storage space of the node, and the specific method includes: For nodes, and sort the relevant data according to their The storage non-essential factors of the nodes are arranged from small to large to form the The data call sequence of nodes; The storage model of each node is as follows: in, For the The output value of the storage model objective function of each node, Expressing the The data call sequence of nodes is traversed to the The storage space occupied by the relevant data is Expressing the The data call sequence of nodes is traversed to the The number of related data traversed when traversing related data, Indicates The first node in the data call sequence Is the relevant data stored in The parameters in the node, Indicates The first node in the data call sequence The data packet size of the relevant data, Indicates The first node in the data call sequence The relevant data for The storage of nodes is not a necessary factor. Indicates The storage space size of each node; Indicates the preset storage ratio. is a hyperparameter.

[0012] Optionally, the storage relationship between each relevant data and each node is determined by optimizing the storage model of each node, including the specific method of: Arrange the nodes in ascending order according to their priority to form a node traversal sequence. The first node in the node traversal sequence is recorded as the current node. Optimize the objective function of the storage model of the current node, the optimization goal is to maximize the function output value, and obtain a number of related data that need to be stored in the current node; All relevant data not stored in the current node before the last relevant data stored in the current node in the data call sequence of the current node are recorded as unstored relevant data of the current node; In the process of calculating the transmission interference factor of each unstored related data for the current node, the node corresponding to the transmission interference factor determined by each unstored related data is used as the neighboring broadcast node of the current node, and the corresponding unstored related data is stored in the neighboring broadcast node.

[0013] The present invention also proposes a data optimization collection system for urban planning, which includes a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, the steps of the above method are implemented.

[0014] The beneficial effects of the present invention are as follows: the present invention distributes and stores the relevant data of urban planning after collection, and at the same time considers the broadcast effect between adjacent nodes in the management platform, and selectively stores the relevant data according to the access relationship during the storage process of each node, so that the storage space of each node is effectively utilized, and the efficiency of users calling and reading the relevant data of urban planning through the node is improved, thereby ensuring the efficiency of the relevant data for urban planning in the collection process, thereby realizing the optimized collection of urban planning data; wherein, by analyzing the time series changes of the access relationship of the node to the same type of relevant data, the access correlation factor of the node to each type of relevant data is quantified, reflecting the importance of the node in the access relationship of the corresponding type of relevant data, and providing a priority correction for the subsequent node to store the corresponding type of relevant data after collection. It provides a basis; at the same time, according to the access relationship of the node to the relevant data, combined with the collection process of the relevant data and the size relationship between the relevant data and the node storage space, the impact of the transmission delay on the high-frequency access related data is reduced, thereby obtaining the access correlation factor; through the access correlation factor, the node's storage priority for each type of related data is further optimized, and the broadcast effect between adjacent nodes is analyzed, so as to adjust the storage adaptation factor of the related data for the node, thereby effectively utilizing the broadcast effect between nodes and the storage space of each node, and then by constructing a node storage model, while ensuring the effective use of storage space and storing more necessary related data, the efficiency of storage after the collection of relevant data for urban planning is improved, and at the same time, the impact of centralized storage or direct distributed storage on the transmission delay when calling and reading related data is reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0016] Figure 1 A schematic flow chart of a data optimization collection method for urban planning provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] See also Figure 1 , which shows a flow chart of a data optimization collection method for urban planning provided by an embodiment of the present invention, the method comprising the following steps: Step S001: Obtain relevant data for urban planning through the urban planning management platform, record the type of each relevant data, the geographical location distribution, storage space and priority of each node in the platform, and obtain the access relationship between each node and each relevant data.

[0019] The purpose of this embodiment is to comprehensively consider the geographical location distribution, storage space and priority of the nodes in the platform during the process of collecting and storing relevant data for urban planning, so as to make full use of the broadcast effect between adjacent nodes in the platform, that is, the rapid call between data packets stored in adjacent nodes; and combine the access relationship between nodes and related data to effectively utilize the storage space of each node, and finally achieve high efficiency in the storage of urban planning data after collection.

[0020] Specifically, various types of relevant data for urban planning are obtained through the urban planning management platform, including basic geographic information data, ecological planning data, business audit data, government data and network open source data. Each relevant data is collected through each node in the platform, and the type of each relevant data and its data packet size are recorded.

[0021] Furthermore, the urban planning management platform is built based on the digital twin model. The storage center serves as the central node in the model to store all relevant data for urban planning. The central node and other nodes are mapped to the digital twin model according to their real geographical location. The geographical distance between each node and the central node can be obtained. At the same time, in the model construction process, each node corresponds to the functional departments of different cities, and the priority is given to each node according to the government affairs level (the higher the administrative level of the city, the higher the priority). At the same time, the size of the storage space of the server corresponding to each node is recorded.

[0022] Furthermore, since the establishment of the urban planning management platform, each time each node accesses each relevant data, a record is made, and the access relationship between each node and each relevant data is obtained.

[0023] Step S002: Analyze the access relationship between each node and each related data, and determine the access correlation factor of each node to each type of related data in combination with the type of each related data; based on the collection process of each related data, analyze the size relationship between each related data and the storage space of each node, and obtain the storage adaptation factor of each related data for each node.

[0024] Preferably, in one embodiment of the present invention, the access relationship between each node and each related data is analyzed, and the access correlation factor of each node to each type of related data is determined in combination with the type of each related data, including the specific method of: It should be noted that different nodes have different access relationships to related data, which is specifically reflected in the differences in the changes in the frequency of access to different types of related data in a temporal manner. A node quantifies the access correlation between the node and the relevant data type by performing a temporal analysis on the historical access to various types of related data. In the subsequent process of selective storage based on priority, further limitations are made based on the priority to ensure that the urban planning-related data stored in the subsequent distributed storage process is more consistent with the access and reading of the node.

[0025] Specifically, for any node, the access frequency of the node to each relevant data in each week is obtained since the establishment of the urban planning management platform; it is particularly noted that if the current time is less than a week, the nearest few days remaining after screening each week will be taken as a week for subsequent processing; the sum of the access frequencies of the node to the same type of relevant data in the same week is taken as the total access frequency of the node to the relevant data of this type in that week; the ratio of the total access frequency to the sum of the access frequencies of the node to all relevant data in that week is taken as the relative frequency of the node's access to the relevant data of this type in that week; then Node pair Access-related factors for type-related data The calculation method is: in, Indicates All weekly pairs of nodes The average relative frequency of access to data of different types, Indicates All weekly pairs of nodes The standard deviation of the relative frequency of access to data of each type, Indicates Nodes are checked weekly The total access frequency of data of each type is a sequence composed of time sequence, Indicates The sum of the access frequencies of all relevant data by each node per week is a sequence formed in time order. It means to calculate the cosine similarity of two sequences. To avoid hyperparameters with a denominator of 0, this embodiment uses Give a narrative.

[0026] It should be noted that the relative frequency of access reflects the proportion of the node's access frequency to the corresponding type of related data in each week. The larger the mean, the larger the proportion in each week. At the same time, the smaller the standard deviation, the more similar the access frequency proportions between weeks are, and the overall trend is that the node accesses the relevant data of this type more frequently. At the same time, the closer the change relationship between the total access frequency of each week and the sum of the access frequencies is, that is, the larger the cosine similarity, the more easily the change in the node's access relationship to the relevant data is affected by the relevant data of this type, and the larger the access correlation factor between the two.

[0027] Preferably, in one embodiment of the present invention, according to the collection process of each relevant data, the size relationship between each relevant data and the storage space of each node is analyzed to obtain the storage adaptation factor of each relevant data for each node, including the specific method of: It should be noted that the storage space in the server of each node is limited. At the same time, affected by the geographical distribution of the nodes, the relevant data of urban planning are collected through the nodes. The storage after collection needs to give priority to the node's collection process of the relevant data to avoid the overall transmission and then storing it in the storage center, which will generate a lot of transmission time, thereby affecting the collection efficiency; at the same time, since the relevant data is collected through the corresponding nodes, the corresponding nodes will access the relevant data frequently. If it is stored in the storage center, the subsequent high-frequency call of the relevant data will be affected by the transmission delay, which will seriously reduce the call and reading efficiency of the relevant data.

[0028] It should be further explained that, based on the collection process, the greater the proportion of the node's access frequency to the same type of related data in the total access frequency of the corresponding type of related data, the greater the possibility that the corresponding related data is accessed frequently by the corresponding node; at the same time, the relationship between the data packet size of the related data and the storage space of the node server is considered to ensure that high-frequency calls are less affected by transmission delays, and at the same time, the storage space is effectively utilized for distributed storage.

[0029] Specifically, the most recently collected relevant data is recorded as the current relevant data, and the type corresponding to the current relevant data is recorded as the current type relevant data; The ratio of the mean access frequency of each node to each related data under the current type of related data to the mean total access frequency of all nodes to each related data under the current type of related data is taken as the first The access frequency of the nodes to the data related to the current type; The ratio of the access frequency of a node to any related data under the current type of related data to the total access frequency of all nodes to the related data under the current type of related data is taken as the first The access frequency of each node to the related data under the current type of related data.

[0030] It should be noted that the current relevant data has just been collected and has not been accessed by any node. It needs to be analyzed based on the access frequency of the corresponding type of relevant data. The relevant data obtained from the platform has recorded the collected nodes and the access relationship of each node to it. In the process of calculating the storage adaptation factor, it is directly quantified by its access frequency.

[0031] Specifically, the current relevant data is for Storage Adaptability Factor of Nodes The calculation method is: in, Indicates The access frequency of nodes to data related to the current type, Indicates The geographical distance between each node and the node where the current relevant data is collected, Indicates the current data packet size of the relevant data, Indicates The storage space size of each node, represents an exponential function with a natural constant as the base; in particular, if the node where the current relevant data is collected is the If there are nodes, the actual geographical distance is 0, but it is not calculated as 0. Some of them are directly replaced with 1, and the following formulas adopt the same operation.

[0032] Similarly, The relevant data for Storage Adaptability Factor of Nodes The calculation method is: in, Indicates Node pair The access frequency of relevant data. It should be noted that The relevant data is not necessarily the relevant data of the current type, it can be any type of relevant data, and the access frequency of the node to it is obtained according to the above method; Indicates The node and The geographical location distance of the nodes where the relevant data is collected, Indicates The data packet size of the relevant data.

[0033] It should be noted that if there is no access relationship between the newly collected relevant data and the node, the proportion of the access frequency of the same type of relevant data in the node is used for analysis, while other relevant data are directly analyzed based on the overall proportion of the node's access frequency. The greater the proportion of the node's access frequency to the overall access frequency of the relevant data, the more necessary it is for the relevant data to be stored in the corresponding node for subsequent calls and reading; at the same time, the greater the geographical distribution distance between the collection node of the relevant data and the node, the more it should be stored in the node in the case of frequent calls to reduce the impact of transmission delay, and if the node is the collection node, the necessity of storing the relevant data is maximized, that is, the corresponding parameter is adjusted to 1; and the smaller the storage space of the node occupied by the relevant data packet, the smaller the burden on the node server to store the relevant data, and storage under frequent access will not interfere with the normal operation of the node server.

[0034] At this point, by analyzing the temporal changes in the access relationship of nodes to the same type of related data, the access correlation factors of nodes to each type of related data are quantified, reflecting the importance of the node in the access relationship of the corresponding type of related data, and providing a basis for the priority correction of the corresponding type of related data stored after subsequent node collection; at the same time, based on the access relationship of the node to the related data, combined with the collection process of the related data and the size relationship between the related data and the node storage space, the impact of transmission delay on high-frequency access related data is reduced, thereby obtaining the access correlation factor.

[0035] Step S003: Analyze the priority of each node and its access-related factors to each type of related data, and determine the storage relationship between each related data and each node based on the geographical location distribution of each node and the storage adaptability factor.

[0036] Preferably, in one embodiment of the present invention, the specific method included in this step is: Based on the access-related factors of each node to each type of related data and the priority of each node, the storage priority of each node to each type of related data is obtained; According to the storage priority of each node for each type of related data, the delay influence of different nodes on the transmission of related data is analyzed, and the storage adaptation factor is adjusted to obtain the storage non-essential factor of each related data for each node; Based on the storage non-essential factors and in combination with the size relationship between the relevant data and the storage space of the node, a storage model of each node is constructed; By optimizing the storage model of each node, the storage relationship between each related data and each node is determined.

[0037] It should be noted that the priority of the node itself is the priority of the city where the node is located in the urban planning management platform, which is determined by administrative, geographical and other factors. For example, the priority of provincial administrative units will be higher than that of municipal administrative units, and the relevant data stored and called will contain more urban planning-related data of more cities than actually exist. On this basis, the types of relevant data collected by each node itself will be different, and it is necessary to correct the storage priority of the acquisition node for type-related data by accessing relevant factors.

[0038] As an example, based on the access-related factors of each node to each type of related data and in combination with the priority of each node, the storage priority of each node to each type of related data is obtained, including the specific method as follows: Specifically, The node and For example, the higher the node priority is, the greater the priority value is. The priority of the node is Node pair The product of the access correlation factors of the type-related data is taken as the Node pair The storage priority of data of related types.

[0039] It should be noted that after determining the storage priority, the storage adaptation factor needs to be adjusted based on the storage priority and the geographical distribution between the nodes. That is, if there are nodes of the same type with higher storage priority within a relatively close distance to the node, the corresponding related data can be stored in the node with higher storage priority, thereby effectively utilizing the storage space of the node.

[0040] As an example, according to the storage priority of each node for each type of related data, the delay influence of different nodes on the transmission of related data is analyzed, the storage adaptation factor is adjusted, and the storage non-essential factor of each related data for each node is obtained, including the specific method of: Specifically, taking the current related data and its corresponding current type related data as an example, the storage priority of each node for the current type related data is obtained, and the storage priority of other nodes for the current type related data is calculated by subtracting the storage priority of the first node from the storage priority of the second node. The node obtains several differences in the storage priority of the current type of related data, and linearly normalizes all the differences. The result is used as the The storage priority difference between the node and the corresponding node in the current type of related data.

[0041] Furthermore, the data transmission speed between nodes in the urban planning management platform is known, and the signal transmission speed is known, so the current relevant data is The calculation method of the transmission interference factor of a node is: in, Indicates that the current relevant data is in The node and The transmission interference factor between nodes is Indicates The node and The storage priority of the nodes in the current type of related data is different. Indicates the data transmission speed. Indicates the signal transmission speed. Indicates the current data packet size of the relevant data, Indicates The node and The geographical distance of the nodes, Indicates the maximum size of a packet of all related data. Indicates the maximum geographical distance between all nodes; the denominator is added with 1 to avoid the denominator being 0 affecting the calculation result; according to the above method, the current relevant data is obtained in the The transmission interference factor between the node and other nodes is taken as the minimum value of the current relevant data. The transmission interference factor of each node.

[0042] Further, subtract 1 from the current relevant data for the first The difference between the storage adaptation factor of the node and the current relevant data for the The product of the transmission interference factors of the nodes is used as the current relevant data for the The storage of each node is not a necessary factor.

[0043] It should be noted that the larger the storage adaptability factor is, the more necessary it is to store at the node, and the smaller the storage non-essential factor is; if the storage is not performed at the corresponding node, the broadcast effect between nodes needs to be considered, that is, the data transmission between adjacent nodes. The greater the difference in storage priority, the higher the storage priority of the corresponding node for the current type of related data. The smaller the transmission delay, the stronger the broadcast effect, and the smaller the corresponding transmission interference factor. During the transmission process, it is necessary to send a signal to confirm the transmission, and then transmit the data packet to comprehensively obtain the transmission delay. The smaller the transmission interference factor, the corresponding node can rely on the broadcast effect of the adjacent node to call and read the related data, thereby reducing the necessity of storage and increasing the storage non-essential factor.

[0044] It should be noted that the storage non-essential factor reflects the necessity of storing the relevant data in the corresponding node. Therefore, the relevant data are traversed one by one in the order of the storage non-essential factors from small to large, and the proportion of the stored relevant data to the node storage space is determined at the same time, so as to construct a storage model.

[0045] As an example, based on the storage non-essential factor and in combination with the size relationship between the relevant data and the storage space of the node, a storage model of each node is constructed, including the following specific methods: Specifically, for nodes, and sort the relevant data according to their The storage non-essential factors of the nodes are arranged from small to large to form the The data call sequence of nodes; The storage model of each node is as follows: in, For the The output value of the storage model objective function of each node, Expressing the The data call sequence of nodes is traversed to the The storage space occupied by the relevant data is Expressing the The data call sequence of nodes is traversed to the The number of related data traversed when traversing related data, Indicates The first node in the data call sequence Is the relevant data stored in The parameters in the nodes are ,otherwise ; Indicates The first node in the data call sequence The data packet size of the relevant data, Indicates The first node in the data call sequence The relevant data for The storage of nodes is not a necessary factor. Indicates The storage space size of each node; Indicates the preset storage ratio. This embodiment adopts to give a narrative; To avoid hyperparameters with a denominator of 0, this embodiment uses Give a narrative.

[0046] Furthermore, the objective function of the storage model is obtained for each node according to the above method.

[0047] It should be noted that under the objective function of the node, it is necessary to ensure that the storage necessity of the relevant data stored in the corresponding node is large enough, and at the same time, the storage space is fully utilized. In order to ensure the normal operation of the node server, the storage ratio is preset to construct the objective function of the storage model and perform subsequent optimization.

[0048] It should be further explained that after the storage model is constructed, it is necessary to traverse it according to the priority of the node itself, and optimize the storage model one by one, so as to finally obtain the storage relationship between all relevant data and each node.

[0049] As an example, by optimizing the storage model of each node, the storage relationship between each related data and each node is determined, including the specific method of: Specifically, the nodes are arranged in ascending order according to their own priorities to form a node traversal sequence, and the first node in the node traversal sequence is recorded as the current node; then the objective function of the storage model of the current node is optimized, and the optimization goal is to maximize the function output value, so as to obtain a number of related data that need to be stored in the current node; at the same time, all related data not stored in the current node before the last related data stored in the current node in the data call sequence of the current node are recorded as unstored related data of the current node; in the process of calculating the transmission interference factor of each unstored related data for the current node, the node corresponding to the transmission interference factor determined by each unstored related data (that is, the transmission interference factor is determined by the minimum value of the transmission interference factors of other nodes, and the minimum value corresponds to a node) is used as the neighboring broadcast node of the current node, and the corresponding unstored related data is stored in the neighboring broadcast node.

[0050] Furthermore, the objective function of the storage model is optimized for the next node in the node traversal sequence. If the node is an adjacent broadcast node of the previous node in the sequence, the objective function of the storage model and its corresponding data call sequence will no longer calculate the unstored related data of the previous node in the sequence stored by the adjacent broadcast node, that is, the corresponding unstored related data is fixedly stored in the node, and the situation of not storing the unstored related data is not considered in the storage model optimization process, and the calculation of non-essential factors needs to be involved.

[0051] By analogy, the relevant data that needs to be stored in each node is finally obtained, and the storage relationship between each relevant data and each node is obtained.

[0052] Furthermore, after collecting new relevant data, the non-essential factors for storage of the new relevant data for each node are obtained according to the above method, starting from the node with the lowest storage priority for the relevant data of the type to which the new relevant data belongs, on the premise that the storage model objective function of the node has been optimized, whether adding the new relevant data will increase the objective function, if it increases, it will be stored, otherwise it will not be stored, and so on, the storage relationship of the new relevant data at each node is determined.

[0053] At this point, the storage priority of nodes for various types of related data is further optimized by accessing related factors, and the broadcast effect between neighboring nodes is analyzed to adjust the storage adaptation factor of related data for nodes, thereby effectively utilizing the broadcast effect between nodes and the storage space of each node. Then, by constructing a node storage model, while ensuring the effective use of storage space and storing more necessary related data, the efficiency of storage after collection of related data for urban planning is improved, while reducing the impact of centralized storage or direct distributed storage on the transmission delay when calling and reading related data.

[0054] Step S004: Based on the storage relationship between each relevant data and each node, each relevant data is collected and distributedly stored to achieve data collection optimization for urban planning.

[0055] It should be noted that each relevant data is collected through different nodes during the collection process, and after analyzing the access relationship between each type of relevant data and each node, the storage relationship between each relevant data and each node is obtained, which can comprehensively consider the storage space occupied by the relevant data for the node, and the impact of the transmission delay of the node on the relevant data access process after collection, thereby ensuring the storage efficiency after collection while improving the calling and reading efficiency of each node for the relevant data.

[0056] Specifically, after any node in the platform collects relevant data of any urban planning, the storage relationship between it and each node in the platform is quantified, that is, whether it needs to be stored in the corresponding node. If it needs to be stored, the relevant data will be stored in the corresponding node server through transmission; if it does not need to be stored, the corresponding node will not store the relevant data, so as to realize distributed storage of each relevant data after collection, thereby optimizing the collection process of relevant data for urban planning.

[0057] At this point, the relevant data of urban planning are distributedly stored after collection, and the broadcast effect between adjacent nodes in the management platform is considered. In the process of storage at each node, relevant data is selectively stored based on the access relationship, so that the storage space of each node is effectively utilized, and the efficiency of users calling and reading urban planning related data through the node is improved, thereby ensuring the efficiency of relevant data for urban planning in the collection process, thereby realizing the optimized collection of urban planning data.

[0058] It should be noted that this embodiment adopts Model to present inverse proportional relationship and normalization, is an exponential function with a natural constant as base, As the input of the model, the implementer can set the inverse proportional function and normalization function according to the actual situation.

[0059] Another embodiment of the present invention provides a data optimization collection system for urban planning, the system comprising a memory, a processor, and a computer program stored in the memory and running on the processor, and when the processor executes the computer program, the above method steps S001 to S004 are implemented.

[0060] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A data optimization collection method for urban planning, characterized in that: The method comprises the following steps: Obtain relevant data for urban planning through the urban planning management platform, record the type of each relevant data, the geographical location distribution, storage space and priority of each node in the platform, and obtain the access relationship between each node and each relevant data; Analyze the access relationship between each node and each related data, and determine the access correlation factor of each node to each type of related data in combination with the type of each related data; analyze the size relationship between each related data and the storage space of each node according to the collection process of each related data, and obtain the storage adaptation factor of each related data for each node; Analyze the priority of each node and its access-related factors to each type of related data, and determine the storage relationship between each related data and each node based on the geographical location distribution of each node and the storage adaptability factor; Based on the storage relationship between each relevant data and each node, each relevant data is collected and distributedly stored.

2. The data optimization collection method for urban planning according to claim 1, characterized in that: The specific method of analyzing the access relationship between each node and each related data and determining the access correlation factor of each node to each type of related data in combination with the type of each related data is as follows: According to the access frequency of each node to each relevant data in each week and the type of each relevant data, the total access frequency of each node to each type of relevant data in each week is obtained; The ratio of the total access frequency of any node to any type of related data in any week to the sum of the access frequencies of the node to all related data in that week is taken as the relative frequency of the node's access to the related data of that type in that week; Based on the ratio of the mean to the standard deviation of the relative frequency of the node's access to this type of related data each week, combined with the similarity relationship between the total frequency of the node's access to this type of related data each week and the sum of the node's access frequencies to all related data each week, the node's access correlation factor for this type of related data is obtained.

3. The data optimization collection method for urban planning according to claim 2 is characterized in that: According to the collection process of each relevant data, the size relationship between each relevant data and the storage space of each node is analyzed to obtain the storage adaptation factor of each relevant data for each node, including the specific method of: The most recently collected relevant data is recorded as the current relevant data, and the type corresponding to the current relevant data is recorded as the current type relevant data; The ratio of the mean access frequency of each node to each related data under the current type of related data to the mean total access frequency of all nodes to each related data under the current type of related data is taken as the first The access frequency of each node to the data related to the current type; The first The ratio of the access frequency of a node to any related data under the current type of related data to the total access frequency of all nodes to the related data under the current type of related data is taken as the first The access frequency of each node to the related data under the current type of related data; According to the access frequency of each node to each type of related data or its related data, combined with the geographical location distribution and storage space of the nodes, the storage adaptation factor of each related data for each node is obtained.

4. The data optimization collection method for urban planning according to claim 3 is characterized in that: The method of obtaining the storage adaptation factor of each node for each related data according to the access frequency of each node to each type of related data or the related data thereunder, combined with the geographical location distribution and storage space of the node, includes the following specific methods: The current relevant data for Storage Adaptability Factor of Nodes The calculation method is: in, Indicates The access frequency of nodes to data related to the current type, Indicates The geographical distance between each node and the node where the current relevant data is collected, Indicates the current data packet size of the relevant data, Indicates The storage space size of each node, represents an exponential function with a natural constant as base; No. The relevant data for Storage Adaptability Factor of Nodes The calculation method is: in, Indicates Node pair The access frequency of relevant data, Indicates The node and The geographical location distance of the nodes where the relevant data is collected, Indicates The data packet size of the relevant data.

5. The data optimization collection method for urban planning according to claim 3 is characterized in that: The analysis of the priority of each node and its access-related factors to each type of related data determines the storage relationship between each related data and each node based on the geographical location distribution of each node and the storage adaptability factor, including the specific method of: Based on the access-related factors of each node to each type of related data and the priority of each node, the storage priority of each node to each type of related data is obtained; According to the storage priority of each node for each type of related data, the delay influence of different nodes on the transmission of related data is analyzed, and the storage adaptation factor is adjusted to obtain the storage non-essential factor of each related data for each node; Based on the storage non-essential factors and in combination with the size relationship between the relevant data and the storage space of the node, a storage model of each node is constructed; By optimizing the storage model of each node, the storage relationship between each related data and each node is determined.

6. The data optimization collection method for urban planning according to claim 5, characterized in that: The method of analyzing the delay influence of different nodes on the transmission of relevant data according to the storage priority of each node for each type of relevant data, adjusting the storage adaptation factor, and obtaining the storage non-essential factor of each relevant data for each node includes the following specific methods: For any node and other nodes, based on the storage priority of the current type of related data, the storage priority difference between the node and other nodes in the current type of related data is obtained; Based on the geographical location distance between different nodes and the data packet size of the current relevant data, the transmission interference factor of the current relevant data between different nodes is obtained by data transmission speed and signal transmission speed, combined with the storage priority difference of the current type of relevant data of different nodes, and the transmission interference factor of the current relevant data between different nodes is negatively correlated with the storage priority difference; Based on the transmission interference factor of the current relevant data between different nodes, the storage adaptation factor of the current relevant data for each node is adjusted to obtain the storage non-essential factor of the current relevant data for each node.

7. The data optimization collection method for urban planning according to claim 6, characterized in that: The specific method of obtaining the non-essential factors of the storage of the current relevant data for each node includes: Subtract 1 from the current relevant data for the The difference between the storage adaptation factor of the node and the current relevant data for the The product of the transmission interference factors of the nodes is used as the current relevant data for the The storage of each node is not a necessary factor.

8. The data optimization collection method for urban planning according to claim 5, characterized in that: The storage model of each node is constructed based on the non-essential storage factor and in combination with the size relationship between the relevant data and the storage space of the node, and the specific method includes: For nodes, and sort the relevant data according to their The storage non-essential factors of the nodes are arranged from small to large to form the The data call sequence of nodes; The storage model of each node is as follows: in, For the The output value of the storage model objective function of each node, Expressing the The data call sequence of nodes is traversed to the The storage space occupied by the relevant data is Expressing the The data call sequence of nodes is traversed to the The number of related data traversed when traversing related data, Indicates The first node in the data call sequence Is the relevant data stored in The parameters in the node, Indicates The first node in the data call sequence The data packet size of the relevant data, Indicates The first node in the data call sequence The relevant data for The storage of nodes is not a necessary factor. Indicates The storage space size of each node; Indicates the preset storage ratio. is a hyperparameter.

9. The data optimization collection method for urban planning according to claim 8, characterized in that: The storage relationship between each relevant data and each node is determined by optimizing the storage model of each node, and the specific method includes: Arrange the nodes in ascending order according to their priority to form a node traversal sequence. The first node in the node traversal sequence is recorded as the current node. Optimize the objective function of the storage model of the current node, the optimization goal is to maximize the function output value, and obtain a number of related data that need to be stored in the current node; All relevant data not stored in the current node before the last relevant data stored in the current node in the data call sequence of the current node are recorded as unstored relevant data of the current node; In the process of calculating the transmission interference factor of each unstored related data for the current node, the node corresponding to the transmission interference factor determined by each unstored related data is used as the neighboring broadcast node of the current node, and the corresponding unstored related data is stored in the neighboring broadcast node.

10. A data optimization collection system for urban planning, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of a data optimization collection method applied to urban planning as described in any one of claims 1 to 9 are implemented.

Images