A grid partitioning method and device for improving data availability

Abstract

The present application discloses a grid division method and device for improving data availability. The present application trains a prediction model based on the population of the area to be studied and the data of points of interest to predict the population of each first grid; the first grid is obtained by dividing the area to be studied according to a preset first grid size; the optimal unified offset is searched through a first optimization strategy, and all the first grids are offset according to the optimal unified offset, so that the number of first grids whose first population is greater than or equal to a first threshold is maximized; the optimal merging scheme is searched through a second optimization strategy, and the first grids are merged according to the optimal merging scheme, so that the first population of each first grid after merging and the number of first grids are both minimized and greater than or equal to the first threshold. Through the present application, the number of grids can be reduced while improving the grid accuracy, thereby improving the efficiency of subsequent data processing.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a grid partitioning method and device for improving data availability. Background Art

[0002] When purchasing population data from Internet big data producers, due to the contradiction between data desensitization requirements and grid design, while pursuing data accuracy in the grid, the population in some grid areas is too low, further triggering the desensitization mechanism, causing the population data in the grid area to be blanked, where population data includes a series of data such as population portrait (age, education, occupation, income, etc.), population migration, and transportation. Therefore, if too many grids are blanked, a large amount of population data will be lost, thereby reducing the availability of data. If the grids are not blanked while ensuring the accuracy of the data in the grid, there will be too many grids, further reducing the efficiency of subsequent processing of grid data.

[0003] Therefore, how to ensure the data accuracy in the grid while reducing the number of grids to improve the efficiency of subsequent data processing is a technical problem that needs to be solved at present. Summary of the invention

[0004] The present application provides a grid partitioning method for improving data availability, so as to solve the technical problem of how to reduce the number of grids while ensuring the accuracy of data in the grids.

[0005] In order to solve the above technical problems, in a first aspect, an embodiment of the present application provides a grid partitioning method for improving data availability, comprising:

[0006] Acquire the latest population and points of interest data of the area to be studied, and predict the first population of each first grid in the area to be studied based on the population and the points of interest data; the first grid is obtained by dividing the area to be studied according to a preset first grid size;

[0007] searching for an optimal uniform offset through a first optimization strategy, and offsetting all the first grids according to the optimal uniform offset, so that the number of first grids whose first population number meets a first threshold reaches a maximum;

[0008] An optimal merging solution is searched through a second optimization strategy, and the first grids are merged according to the optimal merging solution, so that the first population number and the number of first grids of each first grid after the merger are minimized at the same time and the first threshold is met.

[0009] Compared with the prior art, the embodiments of the present application have the following beneficial effects: first, based on the relationship between the point of interest data and the population in each grid, the population in each first grid is predicted to provide accurate population distribution for subsequent grid division; further, through the first optimization strategy, the overall offset of the first grid is optimized so that the population in the offset first grid is as large as possible and equal to the first threshold, that is, the desensitization requirement; finally, through the second optimization strategy, the first grids are merged to eliminate the first grids whose population does not meet the first threshold so as to reduce the number of first grids. At the same time, when merging, the data accuracy in the first grid is considered to ensure that the population data in the merged first grid is as small as possible on the premise that it is greater than or equal to the first threshold.

[0010] In some embodiments of the first aspect of the present application, predicting the first population number of each first grid in the area to be studied according to the population number and the point of interest data includes:

[0011] Input the first point of interest data corresponding to each second grid into a preset prediction model to obtain the second population of each second grid; the second grid is the smallest grid allowed when dividing the grid; the preset prediction model is trained and obtained according to the population of the area to be studied and the point of interest data;

[0012] All the second grids are superimposed on all the first grids to obtain the first population quantity of each first grid.

[0013] Compared with the prior art, the above embodiment has the following beneficial effects: through minimum grid division, the population data within the minimum grid is predicted, so as to obtain accurate population distribution, improve the accuracy of the population data when the first grid is subsequently obtained, and solve the data distortion problem caused by fixed grid design in the traditional grid division method.

[0014] In some embodiments of the first aspect of the present application, the preset prediction model is obtained by training according to the population of the area to be studied and the data of the points of interest, and includes:

[0015] sequentially calculating the cosine distance between the first interest point data and the current old second interest point data in each of the second grids, deleting the second grids whose cosine distance is greater than the second threshold, and acquiring a first grid set;

[0016] Filter the second grids in the first grid set, whose second population number and the first point of interest data are respectively greater than or equal to the third threshold and the fourth threshold, to obtain a second grid set;

[0017] The preset prediction model is trained according to the second population quantity corresponding to each second grid in the second grid set and the first point of interest data.

[0018] Compared with the prior art, the above embodiment has the following beneficial effects: in order to improve the accuracy of subsequent prediction model training, when generating a training data set, a data consistency check is performed on the data in the second grid, and second grids with too large differences between the old and new POI data corresponding to the second grid are excluded, and then second grids with too small a population and too few POI data are eliminated to improve the prediction accuracy of the prediction model for valuable grid data.

[0019] In some embodiments of the first aspect of the present application, training the preset prediction model according to the second population number corresponding to each second grid in the second grid set and the first point of interest data includes:

[0020] The preset prediction model includes: a plurality of classification regression trees; the first point of interest data includes: a plurality of categories of points of interest and the number corresponding to the category of each point of interest;

[0021] Repeatedly extracting a number of interest points of different categories from all the interest points to form corresponding feature sets until the number of feature sets is the same as the number of the classification and regression trees;

[0022] The number of interest points corresponding to each category in the feature set in each second grid is used as an independent variable, the second population number corresponding to the second grid is used as a target variable, and the corresponding classification regression tree is trained.

[0023] Compared with the prior art, the above embodiment has the following beneficial effects: by sampling the types of points of interest multiple times, different feature sets are generated, and different classification and regression trees are trained using these feature sets, thereby effectively reducing the deviation and variance of a single model through the combination of multiple classification and regression trees, thereby improving the generalization ability of the model, and the combination of multiple regression trees can better capture the nonlinear relationship between population and points of interest, and reduce the prediction error of the model in different regions. In addition, the classification and regression tree has high computational efficiency, and can achieve rapid prediction and improve data processing efficiency when facing large-scale grid data processing.

[0024] In some embodiments of the first aspect of the present application, searching for the optimal unified offset by using the first optimization strategy includes:

[0025] Randomly generate an initial offset within a first preset range by a random search algorithm;

[0026] The initial offset is iteratively optimized until an iteration termination condition is met, thereby obtaining the optimal unified offset.

[0027] Compared with the prior art, the above embodiment has the following beneficial effects: since the first grid is generated on the basis of the second grid of the smallest size, the offset of the first grid is optimized, and compared with the traditional fixed offset division method, multiple initial offsets can be flexibly generated within a preset range through a random search algorithm, thereby ensuring that the data after the first grid is divided is available, avoiding a large number of invalid grids, and for edge grids, reasonable division can be obtained through appropriate offsets to improve data accuracy.

[0028] In some embodiments of the first aspect of the present application, searching for an optimal merging solution through a second optimization strategy includes:

[0029] Generate a plurality of Thiessen polygons according to all the first meshes; each of the Thiessen polygons includes a plurality of the first meshes;

[0030] Through a recursive search algorithm combined with a pruning strategy, the first optimal merging solution of the first grid in each of the Thiessen polygons is searched in turn.

[0031] Compared with the prior art, the above embodiment has the following beneficial effects: Thiessen polygons can generate accurate spatial area divisions according to the distribution of data points, which conforms to the geometric characteristics of natural geographic space, thereby ensuring the rationality of subsequent grid division and optimization of spatial structure, and will not destroy the coherence or logic of the region due to merging. Moreover, through the cooperation of recursive search algorithm and pruning strategy, not only can accurate optimization of grid division be achieved, but also the efficiency of searching for the first optimal merging solution can be improved.

[0032] In some embodiments of the first aspect of the present application, generating a plurality of Thiessen polygons according to all the first meshes includes:

[0033] Selecting a third grid whose first population is greater than or equal to the first threshold from all the first grids;

[0034] Generate Thiessen polygons corresponding to each of the third grids according to the center points of all the third grids;

[0035] All the Thiessen polygons are superimposed on all the first meshes to determine the first mesh included in each of the Thiessen polygons.

[0036] Compared with the prior art, the above embodiment has the following beneficial effects: the method of generating Thiessen polygons according to the center of the first grid that meets the first threshold can take into account the distribution characteristics of the population, accurately divide each densely populated area, and perform separate grid division inside each Thiessen polygon, thereby ensuring the rationality of subsequent grid division and optimization of spatial structure. The grid division task of the entire area to be studied is divided into several sub-problems of grid division tasks, effectively improving the efficiency of searching for the first optimal merging solution.

[0037] In some embodiments of the first aspect of the present application, the recursive search algorithm combined with the pruning strategy sequentially searches for the first optimal merging solution of the first grid in each of the Thiessen polygons, including:

[0038] Taking the first grid with the largest number of the first population in the currently searched Thiessen polygon as the initial recursive node, preferentially traversing the first grid with the largest number of the first population among all adjacent grids corresponding to the first grid of the current recursive node;

[0039] If the number of the first grids traversed by the recursive path corresponding to the current recursive node exceeds the number of grids merged in any currently recorded feasible solution, the current recursive path is pruned and backtracked to the previous recursive node; if the recursive path corresponding to the current recursive node is not pruned, and the sum of the first population numbers of all the first grids in the recursive path is greater than or equal to the first threshold, the current recursive path is determined to be a feasible solution, and backtracked to the previous recursive node;

[0040] Repeat traversing all recursive paths several times, and select the optimal merged path from the feasible solutions obtained by traversing all recursive paths each time, until any first grid whose first population number is not zero in the current Thiessen polygon is included in one of the obtained optimal merged paths.

[0041] Compared with the prior art, the above embodiment has the following beneficial effects: in order to improve the optimization efficiency of the first grid merging scheme in each Thiessen polygon, if the number of merged grids corresponding to the current recursive path is too large, the accuracy of the grid data corresponding to the recursive path will be reduced. Therefore, it is judged whether the current recursive path is worth continuing to recurse according to the number of merged grids, thereby improving the optimization efficiency; at the same time, during the current grid merging process, if only the first grid with a first population of zero is left in the Thiessen polygon, the iteration is terminated to prevent a large number of first grids with a first population of zero from being merged, thereby avoiding a large amount of invalid data in the grid data.

[0042] In some embodiments of the first aspect of the present application, the step of repeatedly traversing all recursive paths for several times and selecting an optimal merged path from feasible solutions obtained by traversing all recursive paths each time, until any first grid whose first population number is not zero in the current Thiessen polygon is included in one of the obtained optimal merged paths, comprises:

[0043] Before traversing all recursive paths each time, remove all recursive nodes of the previously obtained optimal merged paths from this traversal process;

[0044] After traversing all recursive paths each time, from all the feasible solutions obtained by currently traversing all recursive paths, the feasible solution with the least number of fourth grids and the smallest population after grid merging is selected as the optimal merged path; the fourth grid is the first grid whose first population is zero.

[0045] Compared with the prior art, the above embodiment has the following beneficial effects: in order to prevent the same first grid from being repeatedly selected in different optimal merge paths, it is necessary to remove the recursive nodes corresponding to all previously acquired optimal merge paths from the current traversal process before traversing all recursive paths each time. In order to avoid merging too many first grids with zero populations during the generation of the optimal merge path, thereby causing a large amount of invalid data in the final merged grid, when the optimal merge path is finally selected, the feasible solution with the least number of fourth grids is selected. At the same time, in order to ensure the accuracy of the final merged grid data, when the population of the merged grid is greater than or equal to the first threshold, the feasible solution with the smallest population is selected, thereby improving the accuracy of the grid data.

[0046] In a second aspect, an embodiment of the present application further provides a grid division device for improving data availability, comprising: a prediction module, a first grid shift optimization module, and a first grid merging optimization module;

[0047] The prediction module is used to obtain the latest population and interest point data of the area to be studied, and predict the first population of each first grid in the area to be studied based on the population and interest point data; the first grid is obtained by dividing the area to be studied according to the first grid size;

[0048] The first grid offset optimization module is used to search for an optimal unified offset through a first optimization strategy, and offset all the first grids according to the optimal unified offset, so that the number of first grids whose first population is greater than or equal to a first threshold reaches a maximum;

[0049] The first grid merging optimization module is used to search for an optimal merging solution through a second optimization strategy, and merge the first grids according to the optimal merging solution, so that the first population number and the number of first grids of each first grid after the merger are both minimized and greater than or equal to the first threshold. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A schematic flow chart of a grid partitioning method for improving data availability provided in some embodiments of the present application;

[0051] Figure 2 A schematic diagram of a flow chart of a recursive search algorithm provided in some embodiments of the present application;

[0052] Figure 3 A schematic diagram of the structure of a grid partitioning device for improving data availability provided in some embodiments of the present application. DETAILED DESCRIPTION

[0053] When purchasing population data from Internet big data producers, due to the contradiction between data desensitization requirements and grid design, while pursuing data accuracy in the grid, the population in some grid areas is too low, further triggering the desensitization mechanism, causing the population in the grid area to be blanked. If too many grids are blanked, a large amount of population data will be lost, thereby reducing the availability of data. If the grids are not blanked while ensuring the accuracy of the data in the grid, there will be too many grids, further reducing the efficiency of subsequent processing of grid data.

[0054] In order to solve the above problems, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0055] Embodiment 1

[0056] Please refer to Figure 1 , a grid partitioning method for improving data availability provided in an embodiment of the present application, including S10 to S30, specifically:

[0057] S10: Obtain the population number and interest point data of the area to be studied, and predict the first population number of each first grid in the area to be studied based on the population number and the interest point data; the first grid is obtained by dividing the area to be studied according to a preset first grid size.

[0058] Preferably, in some embodiments of the present application, the above-mentioned population data can be selected from WorldPop population data, and the Point of Interest (POI) data can be obtained or purchased through map providers. For example, the Point of Interest data of Amap include: automobile services, automobile sales, automobile repairs, motorcycle services, catering services, shopping services, life services, sports and leisure services, medical care services, accommodation services, scenic spots, commercial residences, government agencies and social groups, science, education and culture services, transportation facilities services, financial insurance services, companies and enterprises, road ancillary facilities, place name and address information, public facilities and other major categories. Furthermore, each major category will also include secondary and tertiary subcategories.

[0059] Further, in some embodiments of the present application, predicting the first population number of each first grid in the area to be studied according to the population number and the point of interest data includes:

[0060] Input the first point of interest data corresponding to each second grid into a preset prediction model to obtain the second population of each second grid; the second grid is the smallest grid allowed when dividing the grid; the preset prediction model is trained and obtained according to the population of the area to be studied and the point of interest data;

[0061] All the second grids are superimposed on all the first grids to obtain the first population quantity of each first grid.

[0062] Through the minimum grid division, the population data within the minimum grid is predicted to obtain accurate population distribution, which improves the accuracy of the population data when the first grid is obtained later and solves the data distortion problem caused by fixed grid design in the traditional grid division method.

[0063] Furthermore, in some embodiments of the present application, the preset prediction model is obtained by training according to the population of the area to be studied and the data of the points of interest, including:

[0064] sequentially calculating the cosine distance between the first interest point data and the current old second interest point data in each of the second grids, deleting the second grids whose cosine distance is greater than the second threshold, and acquiring a first grid set;

[0065] Filter the second grids in the first grid set, whose second population number and the first point of interest data are respectively greater than or equal to the third threshold and the fourth threshold, to obtain a second grid set;

[0066] The preset prediction model is trained according to the second population quantity corresponding to each second grid in the second grid set and the first point of interest data.

[0067] In order to improve the accuracy of subsequent prediction model training, when generating the training data set, the data in the second grid is checked for data consistency. According to the old and new POI data corresponding to the second grid, the second grid with too large difference between the previous and the next grid is excluded. Then, the second grid with too small population and too few POI data is eliminated to improve the prediction accuracy of the prediction model for valuable grid data.

[0068] Preferably, in some embodiments of the present application, the first grid set can be obtained through the following preferred implementation: on the basis of the minimum grid size, by calculating the cosine distance, check the consistency of the old POI data and the new POI data of each second grid; when the cosine distance is greater than or equal to a certain threshold, eliminate the second grid, and use all the remaining second grids as the first grid set.

[0069] The calculation formula of cosine distance is as follows:

[0070]

[0071] in, is the cosine distance; , is the first The number of POI-like points; , is the first The number of POI-like points.

[0072] If the second grid is too small and there are fewer POI points in the second grid, kernel density analysis can be performed on the POI points to increase the search radius of the second grid. The sum of the kernel densities of the POIs in the grid multiplied by the grid area is taken as the number of POI points of the corresponding category. The calculation formula of the kernel density is:

[0073]

[0074] in, is the kernel density of a certain category of POI data corresponding to the second grid; is the search radius In, the number of POI data points of the corresponding category; For the The field value of a POI data point, here it is 1, indicating that this is a POI data point; For the The distance between a POI data point and the center of the current second grid.

[0075] Preferably, in some embodiments of the present application, in some embodiments of the present application, the second grid set can be obtained by the following preferred implementation: the POI data and population data in each second grid in the first grid set are traversed in sequence, and when the population number in the second grid is less than the third threshold and the number of POI categories is less than the fourth threshold, the corresponding second grid is eliminated, and the remaining second grids are formed into a second grid set.

[0076] Further, in some embodiments of the present application, training the preset prediction model according to the second population number corresponding to each second grid in the second grid set and the first point of interest data includes:

[0077] The preset prediction model includes: a plurality of classification regression trees; the first point of interest data includes: a plurality of categories of points of interest and the number corresponding to the category of each point of interest;

[0078] Repeatedly extracting a number of interest points of different categories from all the interest points to form corresponding feature sets until the number of feature sets is the same as the number of the classification and regression trees;

[0079] The number of interest points corresponding to each category in the feature set in each second grid is used as an independent variable, the second population number corresponding to the second grid is used as a target variable, and the corresponding classification regression tree is trained.

[0080] By sampling the types of points of interest multiple times, different feature sets are generated, and these feature sets are used to train different classification and regression trees. The combination of multiple classification and regression trees can effectively reduce the bias and variance of a single model, thereby improving the generalization ability of the model. The combination of multiple regression trees can better capture the nonlinear relationship between population and points of interest, and reduce the prediction error of the model in different regions. In addition, the classification and regression tree has high computational efficiency. When facing large-scale grid data processing, it can achieve fast prediction and improve data processing efficiency.

[0081] Preferably, in some embodiments of the present application, the preset prediction model is a random forest model. Further, the preset prediction model can be obtained by training each classification regression tree in the random forest model through the following method:

[0082] S11: Data splitting: splitting the data in the second grid set into a training set and a test set according to a preset ratio, which are used for training the model and evaluating the model effect respectively. In the preferred embodiment, the preset ratio is 7:3;

[0083] S12: Feature selection: Randomly extract M features from all categories of POI data in the current study area, take the number of POI data points corresponding to each POI category in the second grid as the input variable, and the population number corresponding to the second grid as the target variable, and build N classification and regression trees; for example, if there are 20 POI categories in the current study area, and 10 categories are randomly selected as features each time, then a maximum of Classification and regression trees, where the number of classification and regression trees can be adaptively adjusted according to demand;

[0084] S13: Model training: training each classification and regression tree by using the POI data and population in each second grid in the training set, wherein the input feature of each classification and regression tree is the feature extracted when constructing the classification and regression tree;

[0085] S14: Model tuning: Tune each classification and regression tree based on methods such as Mean Squared Error (MSE), Root Mean Squared Error (RMSE) and Coefficient of Determination (R-squared);

[0086] S15: Grid size tuning: If each classification regression tree cannot achieve the required accuracy, the size of each second grid can be adjusted at the same time. After changing the POI data distribution characteristics of the second grid, the second grid set acquisition step and steps S11 to S14 are re-executed until the training of the random forest regression model is completed.

[0087] After the random forest regression model is trained, the latest POI data in each second grid is input into each classification regression tree of the random forest regression model, and the average value of the results of all classification regression trees is taken as the prediction result.

[0088] Preferably, in some embodiments of the present application, the superposition of the first grid and the second grid can be performed by the following process: starting from a certain corner of the circumscribed rectangle of the area to be studied, a first grid with a side length of L is generated, and the side length L of the first grid can be adaptively set according to the user's accuracy requirements for the grid data. The first grid is superimposed with the second grid containing the population distribution data, and the number of first grids that meet the minimum population number (i.e., greater than or equal to the first threshold) is counted, wherein the population number of the first grid is the sum of the corresponding population numbers in all the second grids included in the first grid.

[0089] Preferably, in some embodiments of the present application, after the first grid and the second grid are superimposed, since the optimal grid size used in the POI population regression analysis is not necessarily the size we need, it is necessary to interpolate the population size in the first grid to obtain more detailed population distribution data, such as interpolation through spatial interpolation method.

[0090] S20: searching for an optimal unified offset through a first optimization strategy, and offsetting all the first grids according to the optimal unified offset, so that the number of first grids whose first population is greater than or equal to a first threshold reaches a maximum.

[0091] Further, in some embodiments of the present application, searching for the optimal unified offset by using the first optimization strategy includes:

[0092] Randomly generate an initial offset within a first preset range by a random search algorithm;

[0093] The initial offset is iteratively optimized until an iteration termination condition is met, thereby obtaining the optimal unified offset.

[0094] Since the first grid is generated based on the second grid with the smallest size, the offset of the first grid is optimized. Compared with the traditional fixed offset division method, the random search algorithm can flexibly generate multiple initial offsets within a preset range, thereby ensuring that the data after the first grid division is available and avoiding a large number of invalid grids. For edge grids, reasonable division can also be obtained through appropriate offsets to improve data accuracy.

[0095] Preferably, in some embodiments of the present application, the optimal unified offset can be obtained by any random search algorithm, such as a genetic algorithm, a simulated annealing algorithm, or a particle swarm algorithm, etc., to randomly search within a first preset range. The present application does not limit the random search algorithm used. The iteration termination condition can be adaptively set according to the number of iterations, the difference between the results of the previous and next iterations, or the conventional iteration termination condition corresponding to the random search algorithm used. The present application does not limit the setting of the iteration termination condition.

[0096] S30: searching for an optimal merging solution through a second optimization strategy, and merging the first grids according to the optimal merging solution, so that the first population number and the number of first grids of each of the merged first grids are both minimized and greater than or equal to the first threshold.

[0097] Furthermore, in some embodiments of the present application, searching for the optimal merging solution through the second optimization strategy includes:

[0098] Generate a plurality of Thiessen polygons according to all the first meshes; each of the Thiessen polygons includes a plurality of the first meshes;

[0099] Through a recursive search algorithm combined with a pruning strategy, the first optimal merging solution of the first grid in each of the Thiessen polygons is searched in turn.

[0100] Thiessen polygons can generate accurate spatial area divisions based on the distribution of data points, which conforms to the geometric characteristics of natural geographic space, thereby ensuring the rationality of subsequent grid division and optimization of spatial structure, and will not destroy the coherence or logic of the region due to merging. Moreover, through the coordination of recursive search algorithms and pruning strategies, not only can accurate optimization of grid division be achieved, but also the efficiency of searching for the first optimal merging solution can be improved.

[0101] Furthermore, in some embodiments of the present application, generating a plurality of Thiessen polygons according to all the first meshes includes:

[0102] Selecting a third grid whose first population is greater than or equal to the first threshold from all the first grids;

[0103] Generate Thiessen polygons corresponding to each of the third grids according to the center points of all the third grids;

[0104] All the Thiessen polygons are superimposed on all the first meshes to determine the first mesh included in each of the Thiessen polygons.

[0105] The method of generating Thiessen polygons based on the center of the first grid that meets the first threshold can take into account the distribution characteristics of the population, accurately divide each densely populated area, and perform separate grid division inside each Thiessen polygon, thereby ensuring the rationality of subsequent grid division and the optimization of spatial structure. The grid division task of the entire area to be studied is divided into several sub-problems of grid division tasks, effectively improving the efficiency of searching for the first optimal merging solution.

[0106] Further, in some embodiments of the present application, the recursive search algorithm combined with the pruning strategy sequentially searches for the first optimal merging solution of the first grid in each of the Thiessen polygons, including:

[0107] Taking the first grid with the largest number of the first population in the currently searched Thiessen polygon as the initial recursive node, preferentially traversing the first grid with the largest number of the first population among all adjacent grids corresponding to the first grid of the current recursive node;

[0108] If the number of the first grids traversed by the recursive path corresponding to the current recursive node exceeds the number of grids merged in any currently recorded feasible solution, the current recursive path is pruned and backtracked to the previous recursive node; if the recursive path corresponding to the current recursive node is not pruned, and the sum of the first population numbers of all the first grids in the recursive path is greater than or equal to the first threshold, the current recursive path is determined to be a feasible solution, and backtracked to the previous recursive node;

[0109] Repeat traversing all recursive paths several times, and select the optimal merged path from the feasible solutions obtained by traversing all recursive paths each time, until any first grid whose first population number is not zero in the current Thiessen polygon is included in one of the obtained optimal merged paths.

[0110] In order to improve the optimization efficiency of the first grid merging scheme in each Thiessen polygon, if the number of merged grids corresponding to the current recursive path is too large, the accuracy of the grid data corresponding to the recursive path will be reduced. Therefore, it is judged whether the current recursive path is worth continuing to recurse based on the number of grid mergers, thereby improving the optimization efficiency; at the same time, during the current grid merging process, if only the first grid with a first population of zero is left in the Thiessen polygon, the iteration is terminated to prevent a large number of first grids with a first population of zero from being merged, thereby avoiding a large amount of invalid data in the grid data.

[0111] Further, in some embodiments of the present application, the step of repeatedly traversing all recursive paths for several times, and selecting an optimal merged path from the feasible solutions obtained by traversing all recursive paths each time, until any first grid whose first population number is not zero in the current Thiessen polygon is included in one of the obtained optimal merged paths, comprises:

[0112] Before traversing all recursive paths each time, remove all recursive nodes of the previously obtained optimal merged paths from this traversal process;

[0113] After traversing all recursive paths each time, from all the feasible solutions obtained by currently traversing all recursive paths, the feasible solution with the least number of fourth grids and the smallest population after grid merging is selected as the optimal merged path; the fourth grid is the first grid whose first population is zero.

[0114] In order to prevent the same first grid from being repeatedly selected in different optimal merge paths, it is necessary to remove the recursive nodes corresponding to all previously obtained optimal merge paths from the current traversal process before traversing all recursive paths each time. In order to avoid merging too many first grids with zero populations during the generation of the optimal merge path, resulting in a large amount of invalid data in the final merged grid, when the optimal merge path is finally selected, the feasible solution with the least number of fourth grids is selected. At the same time, in order to ensure the accuracy of the final merged grid data, when the population of the merged grid is greater than or equal to the first threshold, the feasible solution with the smallest population is selected, thereby improving the accuracy of the grid data.

[0115] Preferably, reference Figure 2 , a flow chart of a recursive search algorithm provided in some embodiments of the present application, next combined with Figure 2 The above-mentioned S30 is described in detail.

[0116] S31: First, extract the center points of the first grids whose population is greater than or equal to the first threshold from all the first grids, input these center points into the Thiessen polygon Coverage tool, and generate the same number of Thiessen polygons as the first grids that meet the above conditions;

[0117] S32: The set of all first meshes in a single Thiessen polygon is represented as g, and an adjacency table representing the adjacency relationship of each first mesh is generated. , the elements recorded in the adjacency list are information about each first grid and all first grids adjacent to the first grid;

[0118] S33: Through the depth-first search algorithm, combined with the pruning strategy, merge the first grid in g corresponding to each Thiessen polygon. For details on the merging process, refer to Figure 2 ;

[0119] S331: Input g and , and create an empty stack Stack and a feasible solution set Path at the same time;

[0120] S332: Determine whether the current g is empty or only the grid with a population of 0 is left in g; if not, execute step S333; otherwise, output the optimal feasible solution in Path;

[0121] S333: Find the grid with the largest population in g as the initial recursive node, denoted as ;

[0122] S334: Judgment Does it exist in If so, execute step S335; otherwise, execute step S3310;

[0123] S335: From Recorded in Among the adjacent grids of , select the grid with the largest population, denoted as , and exist Marked as merged;

[0124] S336: Determine whether the number of merged grids in the current recursive path is greater than the number of merged grids corresponding to at least one feasible solution in the Path; if not, execute S337, otherwise execute S3310;

[0125] S337: Determine whether the total population of the current recursive path is greater than or equal to the desensitized population threshold (i.e., the first threshold); if so, execute S338; otherwise, execute S339;

[0126] S338: Add the merged grid route corresponding to the current path as a feasible solution to the Path, and execute S334;

[0127] S339: Push it into the Stack and Mark as , execute S334;

[0128] S3310: Determine whether the Stack is empty; if so, execute S3311; otherwise, execute S3312;

[0129] S3311: Find the feasible solution with the least number of grids containing 0 population from Path, and and g to generate the grid corresponding to the feasible solution, clear Path, and The remaining grids are marked as unmerged, and S332 is executed;

[0130] S3312: Pop a grid from the Stack and mark it as , execute S334.

[0131] From the above steps S331 to S3312, it can be seen that in order to improve the optimization efficiency, when the number of merged grids in the current state exceeds the number of merged grids in the previous feasible solution, the path is pruned because: continuing the search cannot be better than the previous feasible solution; at the same time, in order to ensure the accuracy of the grid data, when the total population of the merged grid is greater than the set threshold, the merge solution with the least number of grids with a population of 0 is preferred. If there are multiple solutions that meet the conditions, the solution with a smaller total population is preferred. Further, by setting up a Stack, the backtracking operation in the depth-first search process is realized to find the optimal merge solution in each Thiessen polygon.

[0132] In summary, the grid division method for improving data availability provided in the embodiment of the present application has the following beneficial effects: first, based on the relationship between the point of interest data and the population in each grid, the population in each first grid is predicted to provide accurate population distribution for subsequent grid division; further, through the first optimization strategy, the overall offset of the first grid is optimized so that the population in the offset first grid is as large as possible and equal to the first threshold, that is, the desensitization requirement; finally, through the second optimization strategy, the first grids are merged to eliminate the first grids whose first population does not meet the first threshold so as to reduce the number of first grids. At the same time, when merging, the data accuracy in the first grid is considered to ensure that the population data in the merged first grid is as small as possible on the premise that it is greater than or equal to the first threshold.

[0133] Embodiment 2

[0134] refer to Figure 3 , a grid division device for improving data availability provided in an embodiment of the present application, includes: a prediction module 11, a first grid offset optimization module 12 and a first grid merging optimization module 13.

[0135] Furthermore, in some embodiments of the present application, the prediction module 11 is used to obtain the latest population number and point of interest data of the area to be studied, and predict the first population number of each first grid in the area to be studied based on the population number and the point of interest data; the first grid is obtained by dividing the area to be studied according to the first grid size; the first grid offset optimization module 12 is used to search for the optimal unified offset through a first optimization strategy, and offset all the first grids according to the optimal unified offset, so that the number of first grids whose first population number is greater than or equal to the first threshold is maximized; the first grid merging optimization module 13 is used to search for the optimal merging scheme through a second optimization strategy, and merge the first grids according to the optimal merging scheme, so that the first population number and the number of first grids after the merger are both minimized and greater than or equal to the first threshold.

[0136] Furthermore, in some embodiments of the present application, the prediction module 11 is used to predict the first population number of each first grid in the area to be studied based on the population number and the point of interest data, including: inputting the first point of interest data corresponding to each second grid into a preset prediction model to obtain the second population number of each second grid; the second grid is the minimum grid allowed when dividing the grid; the preset prediction model is trained and obtained based on the population number and the point of interest data of the area to be studied; all second grids are superimposed on all first grids to obtain the first population number of each first grid.

[0137] Furthermore, in some embodiments of the present application, the preset prediction model is trained and acquired based on the population size and the interest point data of the area to be studied, including: calculating the cosine distance between the first interest point data and the current old second interest point data in each of the second grids in turn, deleting the second grids whose cosine distance is greater than the second threshold, and acquiring a first grid set; screening the second grids in the first grid set whose second population size and the first interest point data are respectively greater than or equal to a third threshold and a fourth threshold, and acquiring a second grid set; training the preset prediction model based on the second population size and the first interest point data corresponding to each second grid in the second grid set.

[0138] Furthermore, in some embodiments of the present application, the preset prediction model is trained according to the second population quantity corresponding to each second grid in the second grid set and the first interest point data, including: the preset prediction model includes: a plurality of classification regression trees; the first interest point data includes: a plurality of interest point categories, and the quantity corresponding to each interest point category; repeatedly extracting a plurality of interest points of different categories from all the interest points to form a corresponding feature set until the number of feature sets is the same as the number of the classification regression trees; taking the number of interest points corresponding to each category in the feature set in each second grid as an independent variable, and taking the second population quantity of the corresponding second grid as a target variable, and training the corresponding classification regression tree.

[0139] Furthermore, in some embodiments of the present application, the first grid offset optimization module 12 is used to search for an optimal unified offset through a first optimization strategy, including: randomly generating an initial offset within a first preset range through a random search algorithm; iteratively optimizing the initial offset until an iteration termination condition is met, thereby obtaining the optimal unified offset.

[0140] Furthermore, in some embodiments of the present application, the first mesh merging optimization module 13 is used to search for an optimal merging solution through a second optimization strategy, including: generating a number of Thiessen polygons based on all the first meshes; each of the Thiessen polygons includes a number of the first meshes; and searching for the first optimal merging solution of the first meshes in each of the Thiessen polygons in turn through a recursive search algorithm combined with a pruning strategy.

[0141] Furthermore, in some embodiments of the present application, generating a plurality of Thiessen polygons based on all the first grids includes: screening, from all the first grids, third grids whose first population number is greater than or equal to the first threshold; generating Thiessen polygons corresponding to each of the third grids based on the center points of all the third grids; and superimposing all the Thiessen polygons with all the first grids to determine the first grids included in each of the Thiessen polygons.

[0142] Further, in some embodiments of the present application, the recursive search algorithm is combined with a pruning strategy to sequentially search for the first optimal merging solution of the first grid in each of the Thiessen polygons, including: taking the first grid with the largest number of the first population in the currently searched Thiessen polygon as the initial recursive node, and preferentially traversing the first grid with the largest number of the first population among all adjacent grids corresponding to the first grid of the current recursive node; if the number of the first grids traversed by the recursive path corresponding to the current recursive node exceeds the number of grid mergers of any currently recorded feasible solution, clipping the current recursive path and backtracking to the previous recursive node; if the recursive path corresponding to the current recursive node is not clipped, and the sum of the first population numbers of all the first grids in the recursive path is greater than or equal to the first threshold, determining that the current recursive path is a feasible solution, and backtracking to the previous recursive node; repeatedly traversing all recursive paths several times, and selecting the optimal merging path from the feasible solutions obtained by traversing all recursive paths each time, until any first grid with a non-zero first population number in the current Thiessen polygon is included in one of the obtained optimal merging paths.

[0143] Furthermore, in some embodiments of the present application, the traversal of all recursive paths is repeated several times, and the optimal merged path is screened from the feasible solutions obtained from each traversal of all recursive paths, until any first grid in the current Thiessen polygon whose first population is not zero is included in one of the obtained optimal merged paths, including: before each traversal of all recursive paths, the recursive nodes of all previously obtained optimal merged paths are removed from the current traversal process; after each traversal of all recursive paths, from all the feasible solutions obtained from the current traversal of all recursive paths, the feasible solution with the least fourth grid number and the smallest population number after grid merging is selected as the optimal merged path; the fourth grid is the first grid whose first population is zero.

[0144] It can be understood that the above-mentioned device item embodiments correspond to the method item embodiments of the present invention. A grid division device for improving data availability provided by the embodiments of the present invention can implement any method item embodiment of the present invention, that is, the grid division method for improving data availability provided in Embodiment 1.

[0145] Embodiment 3

[0146] Based on the above-mentioned embodiments of the grid partitioning method for improving data availability, another embodiment of the present application provides a grid partitioning terminal device, which includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, the grid partitioning method for improving data availability of any embodiment of the present application is implemented.

[0147] Exemplarily, in this embodiment, the computer program may be divided into one or more modules, which are stored in the memory and executed by the processor to complete the present application. The one or more modules may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the meshing device.

[0148] The grid partitioning device may be a computing device such as a desktop computer, a notebook, a palm computer, a cloud server, etc. The grid partitioning terminal device may include, but is not limited to, a processor and a memory.

[0149] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the meshing device, and uses various interfaces and lines to connect the various parts of the entire meshing device. The memory may be used to store the computer program and / or module, and the processor implements various functions of the meshing device by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the mobile phone, etc. In addition, the memory may include high-speed random access memory and may also include non-volatile memory, such as a hard disk, an internal memory, a plug-in hard disk, a smart memory card (SmartMediaCard, SMC), a secure digital (SecureDigital, SD) card, a flash card (FlashCard), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0150] Embodiment 4

[0151] Based on the above-mentioned embodiments of the grid partitioning method for improving data availability, another embodiment of the present application provides a storage medium, wherein the storage medium includes a stored computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute the grid partitioning method for improving data availability of any embodiment of the present application.

[0152] In this embodiment, the storage medium is a computer-readable storage medium, and the computer program includes computer program code, which may be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0153] The specific embodiments described above further describe the purpose, technical solutions and beneficial effects of the present application in detail. It should be understood that the above description is only a specific embodiment of the present application and is not intended to limit the scope of protection of the present application. It is particularly pointed out that for those skilled in the art, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of protection of the present application.

Claims

1. A grid partitioning method for improving data availability, characterized in that: include: Obtaining the population number and the data of points of interest in the area to be studied, and predicting the first population number of each first grid in the area to be studied according to the population number and the data of the points of interest; the first grid is obtained by dividing the area to be studied according to a preset first grid size; Searching for an optimal unified offset through a first optimization strategy, and offsetting all the first grids according to the optimal unified offset, so that the number of first grids whose first population is greater than or equal to a first threshold reaches a maximum; Searching for an optimal merging solution through a second optimization strategy, and merging the first grids according to the optimal merging solution, so that the first population number and the number of first grids of each first grid after merging are both minimized and greater than or equal to the first threshold; The predicting the first population number of each first grid in the area to be studied according to the population number and the point of interest data includes: Input the first point of interest data corresponding to each second grid into a preset prediction model to obtain the second population of each second grid; the second grid is the smallest grid allowed when dividing the grid; the preset prediction model is trained and obtained according to the population of the area to be studied and the point of interest data; Superimpose all the second grids with all the first grids to obtain the first population quantity of each first grid; The preset prediction model is obtained by training according to the population of the area to be studied and the data of the points of interest, and includes: sequentially calculating the cosine distance between the first interest point data and the current old second interest point data in each of the second grids, deleting the second grids whose cosine distance is greater than the second threshold, and acquiring a first grid set; Filter the second grids in the first grid set, whose second population number and the first point of interest data are respectively greater than or equal to the third threshold and the fourth threshold, to obtain a second grid set; training the preset prediction model according to the second population number corresponding to each second grid in the second grid set and the first point of interest data; The training of the preset prediction model according to the second population number corresponding to each second grid in the second grid set and the first point of interest data comprises: The preset prediction model includes: a plurality of classification regression trees; the first point of interest data includes: a plurality of categories of points of interest and the number corresponding to the category of each point of interest; Repeatedly extracting a number of interest points of different categories from all the interest points to form corresponding feature sets until the number of feature sets is the same as the number of the classification and regression trees; The number of interest points corresponding to each category in the feature set in each second grid is used as an independent variable, the second population number corresponding to the second grid is used as a target variable, and the corresponding classification regression tree is trained.

2. A grid partitioning method for improving data availability as claimed in claim 1, characterized in that: The searching for the optimal unified offset by the first optimization strategy includes: Randomly generate an initial offset within a first preset range by a random search algorithm; The initial offset is iteratively optimized until an iteration termination condition is met, thereby obtaining the optimal unified offset.

3. A grid partitioning method for improving data availability as claimed in claim 1, characterized in that: The searching for the optimal merging solution through the second optimization strategy includes: Generate a plurality of Thiessen polygons according to all the first meshes; each of the Thiessen polygons includes a plurality of the first meshes; Through a recursive search algorithm combined with a pruning strategy, the first optimal merging solution of the first grid in each of the Thiessen polygons is searched in turn.

4. A grid partitioning method for improving data availability as claimed in claim 3, characterized in that: According to all the first meshes, a plurality of Thiessen polygons are generated, including: Selecting a third grid whose first population is greater than or equal to the first threshold from all the first grids; Generate Thiessen polygons corresponding to each of the third grids according to the center points of all the third grids; All the Thiessen polygons are superimposed on all the first meshes to determine the first mesh included in each of the Thiessen polygons.

5. A grid partitioning method for improving data availability as claimed in claim 4, characterized in that: The recursive search algorithm is combined with a pruning strategy to sequentially search for a first optimal merging solution of the first grid in each of the Thiessen polygons, including: Taking the first grid with the largest number of the first population in the currently searched Thiessen polygon as the initial recursive node, preferentially traversing the first grid with the largest number of the first population among all adjacent grids corresponding to the first grid of the current recursive node; If the number of the first grids traversed by the recursive path corresponding to the current recursive node exceeds the number of grids merged in any currently recorded feasible solution, the current recursive path is pruned and backtracked to the previous recursive node; if the recursive path corresponding to the current recursive node is not pruned, and the sum of the first population numbers of all the first grids in the recursive path is greater than or equal to the first threshold, the current recursive path is determined to be a feasible solution, and backtracked to the previous recursive node; Repeat traversing all recursive paths several times, and select the optimal merged path from the feasible solutions obtained by traversing all recursive paths each time, until any first grid whose first population number is not zero in the current Thiessen polygon is included in one of the obtained optimal merged paths.

6. A grid partitioning method for improving data availability as claimed in claim 5, characterized in that: Repeating traversing all recursive paths for several times, selecting an optimal merged path from feasible solutions obtained by traversing all recursive paths each time, until any first grid whose first population number is not zero in the current Thiessen polygon is included in one of the obtained optimal merged paths, including: Before traversing all recursive paths each time, remove all recursive nodes of the previously obtained optimal merged paths from this traversal process; After traversing all recursive paths each time, from all the feasible solutions obtained by currently traversing all recursive paths, the feasible solution with the least number of fourth grids and the smallest population after grid merging is selected as the optimal merged path; the fourth grid is the first grid whose first population is zero.

7. A grid partitioning device for improving data availability, characterized in that: include: A prediction module, a first grid shift optimization module, and a first grid merging optimization module; The prediction module is used to obtain the latest population and interest point data of the area to be studied, and predict the first population of each first grid in the area to be studied based on the population and interest point data; the first grid is obtained by dividing the area to be studied according to the first grid size; The first grid offset optimization module is used to search for an optimal unified offset through a first optimization strategy, and offset all the first grids according to the optimal unified offset, so that the number of first grids whose first population is greater than or equal to a first threshold reaches a maximum; The first grid merging optimization module is used to search for an optimal merging solution through a second optimization strategy, and merge the first grids according to the optimal merging solution, so that the first population number and the number of first grids of each first grid after merging are both minimized and greater than or equal to the first threshold; The prediction module is used to predict the first population of each first grid in the area to be studied according to the population and the point of interest data, including: Input the first point of interest data corresponding to each second grid into a preset prediction model to obtain the second population of each second grid; the second grid is the smallest grid allowed when dividing the grid; the preset prediction model is trained and obtained according to the population of the area to be studied and the point of interest data; Superimpose all the second grids with all the first grids to obtain the first population quantity of each first grid; The preset prediction model is obtained by training according to the population of the area to be studied and the data of the points of interest, and includes: sequentially calculating the cosine distance between the first interest point data and the current old second interest point data in each of the second grids, deleting the second grids whose cosine distance is greater than the second threshold, and acquiring a first grid set; Filter the second grids in the first grid set, whose second population number and the first point of interest data are respectively greater than or equal to the third threshold and the fourth threshold, to obtain a second grid set; training the preset prediction model according to the second population number corresponding to each second grid in the second grid set and the first point of interest data; The training of the preset prediction model according to the second population number corresponding to each second grid in the second grid set and the first point of interest data comprises: The preset prediction model includes: a plurality of classification regression trees; the first point of interest data includes: a plurality of categories of points of interest and the number corresponding to the category of each point of interest; Repeatedly extracting a number of interest points of different categories from all the interest points to form corresponding feature sets until the number of feature sets is the same as the number of the classification and regression trees; The number of interest points corresponding to each category in the feature set in each second grid is used as an independent variable, the second population number corresponding to the second grid is used as a target variable, and the corresponding classification regression tree is trained.

Images