Method and system for large-batch data aggregation on-graph point scattering

Abstract

The invention provides a method and system for large-batch data aggregation on-graph point scattering, and the method comprises the steps: a data governance step: carrying out the data collection, access governance, analysis and storage of geographic information data through a big data governance platform; an intelligent grid division step: dividing grids by adopting at least one of the following modes: dividing the grids according to the boundary of an area to which the grids belong, and customizing the grids; a data aggregation step: on the basis of grid division, performing data aggregation by adopting parallel computing and distributed processing technologies and combining an aggregation condition configured by a user in a user-defined manner; and a visual display step: the front end adopts an advanced visualization technology to display the aggregation result on a map in point or other forms, the size, color and transparency of the points are adjusted according to the number, density and position of the aggregation points, and resource mapping custom pop-up window configuration is realized according to different data types. And processing and display of mass geographic data are realized under the conditions of low cost, low computing power and less time consumption.

Description

Technical Field

[0001] The present invention belongs to the technical field of big data processing, and in particular relates to a method and system for aggregating large amounts of data and scattering them on a graph. Background Art

[0002] With the rapid development of Internet technology and big data, the acquisition and processing of geographic information data has become increasingly important and complex, and the processing and visualization of large amounts of data has become an important requirement.

[0003] In the field of large-scale data analysis and map visualization, the method of aggregating large amounts of data and scattering points on the map has been widely used. This method mainly divides the map into multiple grids, aggregates the data points falling in each grid, and finally displays the aggregation results on the map in the form of points or other forms. Although the existing technology has achieved certain results in processing large amounts of data by aggregating and scattering points on the map according to grids, it still has the following shortcomings and deficiencies:

[0004] Complex parameter settings: The size and shape of the grid have a great impact on the aggregation results, but how to set these parameters often requires multiple experiments and continuous parameter adjustment and optimization according to specific application scenarios, which increases the complexity of use, cost, time and other resource investment;

[0005] Accuracy problem: Since the grid division is discrete, there may be a situation where originally adjacent points are assigned to different grids, resulting in inaccurate aggregation results. This problem is particularly prominent in areas where data points are dense or unevenly distributed;

[0006] Performance bottleneck: When the amount of data is very large, existing technologies may face performance bottlenecks. For example, when users perform frequent operations such as fast zooming and dragging, the client may experience freezes, time-consuming, poor rendering effects, and sometimes screen overlaps and backlogs. On the other hand, during the data aggregation stage, the backend service needs to traverse each data point and calculate its position, which may consume a lot of computing resources and time.

[0007] In view of this, it is very meaningful to propose a method and system for aggregating large amounts of data. Summary of the invention

[0008] In order to solve the problems existing in the above-mentioned prior art, the present invention provides a method and system for aggregating large quantities of data on a map and scattering points, which has a more efficient, accurate and flexible method for aggregating large quantities of data on a map and scattering points, and realizes the processing and display of large quantities of geographic data at low cost, low computing power and less time consumption. It makes the use of geographic data more reasonable, closer to practicality, and the display effect is richer, which can better adapt to different map application scenarios and meet different types of needs.

[0009] In a first aspect, the present invention proposes a method for aggregating large amounts of data and scattering points on a graph, the method comprising the following steps:

[0010] Data governance step: using the big data governance platform to collect, access, manage, analyze and store geographic information data. The governance includes data cleaning, jurisdiction boundary data management, longitude and latitude conversion, data association integration and field standard specification;

[0011] The smart grid division step divides the grid by at least one of the following methods, including: dividing the grid according to the boundaries of the jurisdiction, rendering the outline of the corresponding area on the map according to the boundaries of the jurisdiction of data governance; custom grid, the user builds thematic scenes according to actual needs, and performs custom area division and collection on the map;

[0012] The data aggregation step uses parallel computing and distributed processing technology based on grid division and combines user-defined aggregation conditions to perform data aggregation;

[0013] In the visualization step, the front end uses advanced visualization technology to display the aggregation results on the map in the form of points or other forms. The size, color and transparency of the points are adjusted according to the number, density and location of the aggregation points, and the custom pop-up window configuration of the resource map is implemented according to different data types.

[0014] Preferably, in the data aggregation step, the aggregation conditions include the map level to be aggregated, the dynamic threshold of the map element aggregation data volume, the dynamic threshold of the grid distance, and a grid shortest distance aggregation algorithm is adopted. The aggregation algorithm includes converting longitude and latitude into two-dimensional coordinates, using the quadtree structure QuadTree to convert the defined Item object into a QuadItem object and save it in the QuadTree tree, and determining the aggregated cluster by iterating the QuadItem.

[0015] Further preferably, the aggregation algorithm also includes: providing a ClusterItem interface for generating a marker point on the map, and subsequently adding the object of the marker point to the management container of the aggregated map to realize the point aggregation function, wherein the marker point contains the latitude and longitude coordinates, center point, icon, color and other properties that need to be displayed on the map; ClusterManager interface for monitoring map status changes, such as performing point aggregation operations when zooming and dragging the map, and monitoring click events of the marker point to render the display effect.

[0016] Further preferably, in the data aggregation step, the main steps for realizing the point aggregation function include: defining the coordinate points of the current screen visualization area: sw is the southwest coordinate of the visible area, and ne is the northeast coordinate of the visible area; judging whether the data needs to be aggregated: customizing the map level that needs to be aggregated, the dynamic threshold of the amount of map element aggregation data, and the dynamic threshold of the grid distance, that is, when the map is zoomed to above this level, it will no longer be aggregated, or it will be aggregated only when the same longitude and latitude are used, or it is assumed that no aggregation will be performed when the number is less than n; determining the aggregation algorithm: the grid shortest distance aggregation algorithm is mainly used.

[0017] Further preferably, in the data aggregation step, the method of converting longitude and latitude into two-dimensional coordinates includes: mapping longitude from [-180, 180] to [0, 1] to calculate the X coordinate, double x = lngLat.getLongitude() / 360+0.5; converting the latitude into radians and then converting the latitude into a sine value using a trigonometric function, and then converting the latitude into a sine value using a trigonometric function: double sinY = Math.sin(Math.toRadians(lngLat.getLatitude())), and then using a Mercator projection or a similar projection to map the latitude to [0, 1] to calculate the Y coordinate, double y=0.5*Math.log((1+sinY) / (1-sinY)) / -(2*Math.PI)+0.5);Finally, create an object containing the X and Y coordinates and multiply them by the world width to get the coordinates in the world coordinate system, QuadTreeItem.Point(x*mWorldWidth,y*mWorldWidth, where the world width is calculated by the formula 256*(zoom^2)=worldWidth, where zoom is the current zoom level of the map.

[0018] Further preferably, in the data aggregation step, the implementation method of determining whether a point is within the current grid or polygon area includes: defining the current point as Point2D.Double point; defining the polygonal point as List<Point2D.Double> pts; the tolerance when comparing with 0 during floating point calculations is double precision = 2e-10; adjacent boundary points are Point2D.Double p1, p2; first, assume that p, p1, and p2 are all point objects with x and y attributes, and pts is a set containing points, N is the number of points in the set, and if a point is located on a vertex or edge of a polygon, it is also considered to be inside the polygon, and true is returned directly; in the ClusterManager, a quadtree structure QuadTree is introduced, that is, all defined Item objects are converted into QuadItem objects, and then saved in the QuadTree tree; next, iterate over all QuadItems, draw a frame with this QuadItem as the center, and the points in the frame are considered a cluster, that is, QuadItems that can be aggregated into a point, and the size of the frame is the defined distance for aggregation; finally, a cluster set is returned, that is, the final aggregation result is required.

[0019] Further preferably, in the data aggregation step, executing the cluster set operation includes: first defining a method for executing aggregation runClustering, which accepts the following parameters: features, a list of entities of FeatureDO type; clusterAlgoKey, a string representing the key value of the clustering algorithm; zoom, an integer representing the current map zoom level; hasVisibleBounds, a Boolean value representing whether there are visible area coordinates; the return type is a Cluster <clusteritem>Type collection; use StrUtil.blankToDefault method to convert clusterAlgoKey to a non-empty string, which defaults to the algorithm name in clusterProperties, and the result is stored in clusterAlgoName; use ClusterAlgoFactory.getAlgorithm method to get the ClusterAlgorithm corresponding to clusterAlgoName <clusteritem>Instance, stored in clusterAlgo; filter and transform the features list, first filter out null elements and FeatureDO objects whose type is not POINT, and then perform the following operations on each remaining element: If hasVisibleBounds is true, that is, the coordinates of the visualization area are passed in, then get the coordinates of the feature, that is, the longitude and latitude, from the featureCache, otherwise the longitude and latitude are set to the default value (0,0), create a new Map, and put the feature into the Map cache as the value of CLUSTER_MAP_KEY_FEATURE; create a new RegionItem object, using the longitude and latitude, Map data, and feature weight obtained in the previous step; collect the converted RegionItem objects into a list; add the list obtained in the previous step to the items of clusterAlgo; call the getCluster method of clusterAlgo, pass in the zoom level, and get and return the clustering result.

[0020] Further preferably, in the data aggregation step, implementing the add method of QuadItem includes: initializing the visitedCandidates collection for storing the visited QuadItem <t>Object; initialize the results collection to store the final Cluster <t>Object; initializes the distanceToCluster map to store QuadItem <t>Mapping relationship to other nearest Cluster distances; initialize itemToCluster mapping for storing QuadItem <t>To its owning SimpleCluster <t>Mapping relationship; first traverse each QuadItem in mItems <t>Candidate object candidate, if candidate is already in visitedCandidates, skip the current loop iteration; if the above condition is not true, retrieve the QuadItem that intersects retrieveBounds from mQuadTree <t>The collection clusterItems, and then perform the following operations on the collection: If the size of the clusterItems collection is 1, then add the candidate directly to the results collection, and add the candidate to the visitedCandidates collection, and finally map the candidate to the distance 0 in the distanceToCluster mapping; if the size of the clusterItems collection is not 1, that is, clusterItems contains multiple elements, then you need to create a new SimpleCluster <t>Object cluster, used to add cluster to QuadItem <t>Collection clusterItems; then continue to traverse each item and calculate the distance from each item point to the candidate point as squaredDistance.

[0021] Preferably, in the intelligent grid division step, dividing the grid according to the boundaries of the jurisdiction also includes automatically switching to the corresponding jurisdiction according to the current map level when the map is zoomed and displaying the data displayed in an aggregated manner in the area, supporting the jurisdiction penetration function; in the custom grid, custom area division and collection are performed on the map, and the collection of squares, rectangles, polygons, circles and irregular shapes is also included, or the jurisdiction range is drawn based on the route on the map, and the default display center point and level are set, and then data points are collected in the area or data is connected to the map for display.

[0022] Preferably, the visualization step also includes calling a map service as a base map, adding a grid overlay layer and annotations on the base map, obtaining the data to be displayed from the server and displaying it on the map according to different data categories, obtaining the aggregated data to be displayed in the visible area of ​​the current screen through the above, locating all data to their corresponding grid longitudes and latitudes and rendering them into the map, thereby achieving the purpose of data aggregation display; the supported map services include pgis maps, tiandimap, baidu maps, google maps and 3D satellite maps.

[0023] In a second aspect, an embodiment of the present invention provides a system for aggregating large amounts of data and scattering them on a graph, including:

[0024] A data governance module is configured to collect, access, manage, analyze and store geographic information data using a big data governance platform, including data cleaning, jurisdiction boundary data management, longitude and latitude conversion, data association integration and field standard specification;

[0025] The intelligent grid division module is configured to divide the grid in at least one of the following ways, including: dividing the grid according to the boundaries of the jurisdiction, rendering the corresponding area outline on the map according to the boundaries of the jurisdiction of data governance; custom grid, the user builds thematic scenes according to actual needs, and performs custom area division and collection on the map;

[0026] The data aggregation module is configured to aggregate data based on grid division, using parallel computing and distributed processing technology combined with user-defined aggregation conditions;

[0027] The visualization display module is configured to use advanced visualization technology on the front end to display the aggregation results on the map in the form of points or other forms. The size, color and transparency of the points are adjusted according to the number, density and location of the aggregation points, and the custom pop-up window configuration of the resource map is implemented according to different data types.

[0028] Preferably, in the intelligent grid division module, the division of the grid according to the boundaries of the jurisdiction also includes automatically switching to the corresponding jurisdiction according to the current map level when the map is zoomed and displaying the data displayed in the area in an aggregated manner, supporting the jurisdiction penetration function; the custom area division and collection on the map in the custom grid also includes the collection of squares, rectangles, polygons, circles and irregular shapes, or drawing the jurisdiction range based on the route on the map, and setting the default display center point and level, and then collecting data points in the area or accessing the data for display on the map.

[0029] Preferably, the visualization display module also includes calling a map service as a base map, adding a grid overlay layer and annotations on the base map, obtaining the data to be displayed from the server and displaying it on the map according to different data categories, obtaining the aggregated data to be displayed in the visible area of ​​the current screen through the above, locating all data to their corresponding grid longitudes and latitudes and rendering them into the map, thereby achieving the purpose of data aggregation display; the supported map services include pgis maps, tiandimap, baidu maps, google maps and 3D satellite maps.

[0030] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner in the first aspect.

[0031] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any implementation manner in the first aspect.

[0032] Compared with the prior art, the beneficial results of the present invention are:

[0033] The present invention proposes a more efficient, accurate and flexible method and system for aggregating large quantities of data and scattering them on a map. The method realizes the processing and display of large quantities of geographic data at low cost, low computing power and less time consumption. The method makes the use of geographic data more reasonable and more practical, and the display effect is richer, so that it can better adapt to different map application scenarios and meet different types of needs. For example, it can be applied to the field of video security, the geographical distribution of camera equipment, the trajectory and path analysis of faces and vehicles, the analysis and planning of surrounding resources, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The accompanying drawings are included to provide a further understanding of the embodiments and are incorporated into and constitute a part of this specification. The accompanying drawings illustrate the embodiments and are used together with the description to explain the principles of the present invention. It will be easy to recognize other embodiments and many expected advantages of the embodiments because they become better understood by reference to the following detailed description. The elements of the drawings are not necessarily to scale with each other. The same reference numerals refer to corresponding similar parts.

[0035] Figure 1 A schematic diagram of a process of a method for aggregating large amounts of data in the above figure according to an embodiment of the present invention;

[0036] Figure 2 It is a schematic diagram of the process of data access management in the method of aggregating large amounts of data in the above figure according to an embodiment of the present invention;

[0037] Figure 3 A schematic diagram of the process of grid division in the method of aggregating large amounts of data in the above figure according to an embodiment of the present invention;

[0038] Figure 4 This is a schematic diagram of the overall structure of the aggregation algorithm in the method for aggregating large amounts of data in an embodiment of the present invention;

[0039] Figure 5 It is a schematic diagram of a process of calculating the distance between two points in the method of aggregating large amounts of data and scattering points in the figure in an embodiment of the present invention;

[0040] Figure 6 A schematic diagram of performing cluster set operations in the method for aggregating large amounts of data in the above figure according to an embodiment of the present invention;

[0041] Figure 7 A schematic diagram of a front-end visualization display process in a method for aggregating large amounts of data in an embodiment of the present invention;

[0042] Figure 8 This is a schematic diagram of customizing grid area collection in the method of aggregating large amounts of data in the above figure according to an embodiment of the present invention;

[0043] Fig. 9 This is an effect diagram of data aggregation and scattering points in the method for aggregating large amounts of data on a graph according to an embodiment of the present invention;

[0044] Fig.10 This is a schematic diagram of the architecture of a system for aggregating large amounts of data in an embodiment of the present invention;

[0045] Fig.11 It is a schematic diagram of the structure of a computer device suitable for implementing an electronic device of an embodiment of the present invention. DETAILED DESCRIPTION

[0046] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It is also necessary to explain that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.

[0047] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0048] With the rapid development of Internet technology and big data, the acquisition and processing of geographic information data has become increasingly important and complex, and the processing and visualization of large amounts of data has become an important requirement.

[0049] The existing product structure usually consists of the following key components:

[0050] Data source: including various geographic location data, such as mobile phone positioning points, landmarks, scenic spots, schools, parks, etc. Grid division algorithm: responsible for dividing the map into multiple grids, which can be squares, rectangles or other shapes. The size and number of grids depend on the specific application scenario and accuracy requirements. Data aggregation module: aggregates the data points that fall within the same grid and calculates the coordinates of the aggregation points, such as the center point and other information of the aggregated data. Map display system: displays the aggregated data on the map in a visual form.

[0051] The working principle of the prior art mainly includes the following steps:

[0052] Data access: access data from the data source, and perform necessary data cleaning and latitude and longitude conversion. Grid division: divide the map into multiple grids according to the preset grid size. Data aggregation: traverse each data point, assign it to the corresponding grid according to its location, and aggregate the data in each grid. Result display: display the aggregated data in a visual form on the map for users to view and analyze.

[0053] Although the existing technology has achieved certain results in processing large amounts of data by grid aggregation and scattering points on the map, it still has the following shortcomings and deficiencies:

[0054] 1. Complex parameter settings: The size and shape of the grid have a great influence on the aggregation results, but how to set these parameters often requires multiple experiments and continuous parameter adjustment and optimization according to the specific application scenarios, which increases the complexity of use, cost, time and other resource investment.

[0055] 2. Accuracy problem: Since the grid division is discrete, there may be a situation where originally adjacent points are assigned to different grids, resulting in inaccurate aggregation results. This problem is particularly prominent in areas where data points are dense or unevenly distributed.

[0056] 3. Performance bottleneck: When the amount of data is very large, existing technologies may face performance bottleneck problems. For example, when users perform frequent operations such as fast zooming and dragging, the client may experience freezes, time-consuming, poor rendering effects, and sometimes screen overlaps and backlogs. On the other hand, during the data aggregation stage, the backend service needs to traverse each data point and calculate its position, which may consume a lot of computing resources and time.

[0057] In view of the shortcomings and deficiencies of the prior art, and in order to solve the problems existing in the above-mentioned prior art, the embodiments of the present invention disclose a more efficient, accurate and flexible method and system for aggregating large quantities of data and scattering points on a map. The processing and display of large quantities of geographic data can be achieved with low cost, low computing power and less time consumption. The use of geographic data is made more reasonable, closer to practicality, and the display effect is richer, which can better adapt to different map application scenarios and meet different types of needs. For example, it can be applied to the field of video security, the geographical distribution of camera equipment, face, vehicle trajectory and path analysis, and the analysis and planning of surrounding resources.

[0058] First, the embodiment of the present invention discloses a method and system for aggregating large amounts of data and scattering points on a graph. The main technical key points are as follows:

[0059] 1. Data governance capability: Access various geographic information data that need to be displayed on the map through big data technology, and perform data cleaning, coordinate conversion, data association, field standards, etc. to ensure data quality before aggregation.

[0060] 2. Aggregation algorithm: The aggregation algorithm proposed in the present invention can realize dynamic and efficient aggregation of data in an environment with less memory and less computing power resources. It not only improves efficiency and solves performance bottlenecks, but also makes the use of data more reasonable.

[0061] 3. Visual display of front-end maps: Solve the problems of freezing, time-consuming and poor visual display when rendering map data, optimize interactive functions such as zooming, panning and clicking, and ensure that a large number of aggregation points can be displayed smoothly on the map through reasonable layer management and rendering strategies. At the same time, it also proposes more abundant usage scenarios, which are more flexible and applicable, improve the readability and clarity of data visualization, and make the data display more intuitive and easy to understand.

[0062] Secondly, the embodiments of the present invention aim to protect the following aspects:

[0063] 1. Data privacy protection: During data processing and transmission, encryption, desensitization and other measures need to be taken to protect user privacy and data security. Comply with relevant laws, regulations and industry standards to ensure the legality and compliance of data use.

[0064] 2. System stability: Design a reasonable system architecture and disaster recovery backup solution to ensure system stability and availability under high concurrency and large data volumes. Perform system maintenance and upgrades regularly to fix vulnerabilities and defects in a timely manner.

[0065] 3. User experience optimization: Pay attention to user feedback and changes in demand, continuously optimize front-end display and interactive design, and enhance user experience.

[0066] 4. Performance monitoring and optimization: Real-time monitoring of system performance indicators (such as response time, throughput, etc.) to promptly identify and resolve problems. Performance tuning is performed based on monitoring data to ensure that the system can maintain good performance under high load.

[0067] 5. Data accuracy: Ensuring data accuracy during the aggregation process is key. Reasonable aggregation algorithms and strategies need to be adopted to accurately calculate and display data. At the same time, possible errors and deviations should be corrected and adjusted.

[0068] Again, the specific embodiments of the present invention are described in detail below.

[0069] First, Figure 1 It shows that the embodiment of the present invention discloses a method for aggregating large amounts of data on a graph. Figure 1 As shown, the method comprises the following steps:

[0070] S1. Data governance step: using the big data governance platform to collect, access, manage, analyze and store geographic information data. The governance includes data cleaning, jurisdiction boundary data management, longitude and latitude conversion, data association integration and field standard specification;

[0071] The specific process of this step is as follows Figure 2 After the data source is connected, data cleaning, coordinate conversion, and standardization are performed before data storage and storage in the resource library.

[0072] Among them, the preprocessed data is sent to Kafka, which is a distributed stream processing platform used to process and store real-time data streams; the data obtained from Kafka is stored in the MySQL database, which is a relational database management system used to store structured data; the data is also sent to Elasticsearch, which is a search and analysis engine used to quickly search and analyze large amounts of data; the data is also stored in MongoDB, which is a non-relational database used to store semi-structured data, such as JSON documents.

[0073] S2, intelligent grid division step, using at least one of the following methods to divide the grid, including: dividing the grid according to the boundaries of the jurisdiction, rendering the corresponding area outline on the map according to the boundaries of the jurisdiction of data governance; custom grid, the user builds thematic scenes according to actual needs, and performs custom area division and collection on the map;

[0074] Specifically, Figure 3 As shown in the figure, the grid division in this step is mainly based on a map visualization operation within the current visualization area window. The following grid division methods can be supported:

[0075] One is to divide the grid according to the boundaries of the jurisdiction. According to the boundaries of the jurisdiction of data governance, the corresponding area outline can be quickly rendered on the map. And when the map is zoomed, it can automatically switch to the corresponding jurisdiction according to the current map level, and display the data in the area in an aggregated manner. This aggregation method will be described in detail later. Click the corresponding area to realize the jurisdiction penetration function, go down to the next level or return to the previous level.

[0076] One is a custom grid. Users can build a thematic scene according to their actual needs, and then divide and collect data in a custom area on the map, including the collection of squares, rectangles, polygons, circles, and various irregular shapes, or draw a jurisdiction based on the route on the map according to business needs, and set the default display center point and level. Then collect data points in the area or access data to display on the above map.

[0077] S3, data aggregation step, based on grid division, uses parallel computing and distributed processing technology, combined with user-defined aggregation conditions to perform data aggregation;

[0078] Specifically, in this step, the main steps to achieve the point aggregation function are divided into three parts. The following are some definitions and specific implementation plans of the aggregation method:

[0079] S31. Define the coordinate points of the current screen visualization area: sw is the southwest coordinate of the visualization area, and ne is the northeast coordinate of the visualization area.

[0080] S32, determine whether data needs to be aggregated: Here, users can customize the map level that needs to be aggregated, the dynamic threshold of the amount of map element aggregation data, and the dynamic threshold of the grid distance. That is, when the map is zoomed to a level above this level, it will no longer be aggregated, or it will be aggregated only when the same longitude and latitude are used, or it is assumed that no aggregation will be performed when the number is less than n.

[0081] S33, determine the aggregation algorithm: in this embodiment, the grid shortest distance aggregation algorithm is mainly used, and the implementation of the aggregation can be divided into two parts.

[0082] The specific implementation method is as follows Figure 4 The details are as follows:

[0083] The first one is the ClusterItem interface, which requires us to provide an implementation class, which is used to generate a marker point on the map, and then add the objects of these marker points to the management container of the aggregate map to implement the point aggregation function. This marker point contains the latitude and longitude coordinates, center point, icon, color and other properties that need to be shown in the figure above.

[0084] The other is ClusterManager, which has two main functions: one is to monitor changes in map status, such as performing point aggregation operations when zooming and dragging the map, and the other is to monitor click events of marker points and render the display effect.

[0085] In the background service, we will start a thread to execute the algorithm according to the parameters determined in steps 31, 32, and 33 above. First, the entire earth is unfolded according to a plane. After unfolding, we can calculate the width of the earth plane, that is, the world width. The calculation formula is as follows:

[0086] 256*(zoom^2)==worldWidth

[0087] Zoom is the current zoom level of the map, so that the latitude and longitude positions that are not conducive to calculation can be converted into the two-dimensional coordinate points we are familiar with for calculation. In a two-dimensional coordinate system, there will be several points, and we will group these points according to a given distance. Of course, if we have high requirements for the speed of aggregation, and are limited by memory and computing power, we do not need to establish a coordinate system for the entire world, but only need to obtain the range within the current screen visualization area for processing.

[0088] Furthermore, the method to convert position into point is as follows:

[0089] Assume that QuadTreeItem.Point is a defined class that contains x and y coordinates, and mWorldWidth is a defined member variable that represents the width of the world.

[0090] Calculate the X coordinate, mapping longitude from [-180,180] to [0,1]:

[0091] double x=lngLat.getLongitude() / 360+0.5;

[0092] To calculate the y coordinate, first convert the latitude to radians, then use trigonometric functions to convert the latitude to its sine:

[0093] double sinY=Math.sin(Math.toRadians(lngLat.getLatitude()));

[0094] Use the Mercator projection or similar to map latitude to [0,1]:

[0095] double y=0.5*Math.log((1+sinY) / (1-sinY)) / -(2*Math.PI)+0.5;

[0096] Finally, create a QuadTreeItem.Point object and return the x and y coordinates, which need to be multiplied by mWorldWidth to get the final coordinates in the world coordinate system:

[0097] QuadTreeItem.Point(x*mWorldWidth,y*mWorldWidth).

[0098] like Figure 5 As shown, the distance between two points is calculated as follows:

[0099] First, define the function distanceSquared(a,b);

[0100] Input: point a: QuadTreeItem.Point type; point b: QuadTreeItem.Point type;

[0101] Output: double type, representing the square of the distance between point a and point b.

[0102] Furthermore, all ClusterItems are then grouped according to a given distance range, and a cluster set is returned. Each cluster in this set contains several ClusterItems, and each cluster meets the requirements for aggregation operations. The returned cluster set is then processed by a Renderer class. This class implements the ClusterRenderer class interface and is mainly responsible for processing the data of each marker point and then providing it for display on the front-end map. The implementation method for determining whether a point is within the current grid or polygon area is as follows:

[0103] Definition: The current point is Point2D.Double point; the fixed points of the polygon are List<Point2D.Double> pts; the tolerance double precision when comparing with 0 during floating-point type calculations is 2e-10; the adjacent boundary fixed points are Point2D.Double p1, p2;

[0104] First, we assume that p, p1, and p2 are all point objects with x and y attributes, and pts is a set containing points, and N is the number of points in the set. If the point is located at the vertex or on the side of the polygon, it is also considered to be inside the polygon, and true is directly returned. The calculation method is as follows:

[0105] if(p == p1)

[0106] return boundOrVertex

[0107] p2 = pts(i % N)

[0108] if((p.x < the minimum of p1.x, p2.x) || (p.x > the maximum of p1.x, p2.x))

[0109] p1 = p2

[0110] continue;

[0111] if(p.y <= the maximum of p1.y, p2.y)

[0112] if(p1.x == p2.x && p.y >= the minimum of p1.y, p2.y)

[0113] return boundOrVertex

[0114] if(p1.y == p2.y)

[0115] if(p1.y == p.y)

[0116] return boundOrVertex

[0117] else

[0118] intersectCount increases by 1

[0119] else

[0120] Calculate the y coordinates of the intersection points xinters

[0121] if(py-xinters <precision)

[0122] return boundOrVertex

[0123] if(py <xinters)

[0124] intersectCount increases by 1

[0125] In ClusterManager, we introduce a quadtree structure QuadTree, which means we will convert all defined Item objects into QuadItem objects and save them in the QuadTree tree. The next step is to iterate all QuadItems and draw a box with this QuadItem as the center. The points in the box are considered a cluster, that is, QuadItems that can be aggregated into a point. The size of the box is the distance we define for aggregation. Finally, a cluster set is returned, which is the final aggregation result we want.

[0126] Next, we will introduce the execution of cluster set operations, such as Figure 6 As shown:

[0127] First, define a method called runClustering that performs aggregation and accepts the following parameters:

[0128] features: a list of entities of type FeatureDO;

[0129] clusterAlgoKey: a string representing the key value of the clustering algorithm;

[0130] zoom: an integer indicating the current map zoom level;

[0131] hasVisibleBounds: A Boolean value indicating whether there are visible area coordinates;

[0132] Return type: a Cluster <clusteritem>A collection of types.

[0133] The execution process of the method is as follows:

[0134] 1. Use the StrUtil.blankToDefault method to convert clusterAlgoKey to a non-empty string, which defaults to the algorithm name in clusterProperties and is stored in clusterAlgoName.

[0135] 2. Get the ClusterAlgorithm corresponding to clusterAlgoName through the ClusterAlgoFactory.getAlgorithm method <clusteritem>Instances, stored in clusterAlgo.

[0136] 3. Filter and transform the features list. First, filter out null elements and FeatureDO objects whose type is not POINT. Then, for each remaining element, perform the following operations:

[0137] If hasVisibleBounds is true, that is, the coordinates of the visualization area are passed in, the coordinates (longitude and latitude) of the feature are obtained from the featureCache, otherwise the longitude and latitude are set to the default value (0,0).

[0138] Create a new Map and cache the feature as the value of CLUSTER_MAP_KEY_FEATURE.

[0139] Create a new RegionItem object, using the longitude and latitude, Map data, and feature weight obtained in the previous step.

[0140] Collect the converted RegionItem objects into a list.

[0141] 4. Add the list obtained in the previous step to the items of clusterAlgo.

[0142] 5. Call the getCluster method of clusterAlgo, pass in the zoom level, obtain and return the clustering result.

[0143] To further introduce the partial implementation process of addItem, the add method of QuadItem is implemented through the following steps:

[0144] 1. Initialize the visitedCandidates collection to store the visited QuadItem <t>object.

[0145] 2. Initialize the results collection to store the final Cluster <t>object.

[0146] 3. Initialize the distanceToCluster map to store QuadItem <t>The mapping relationship to other nearest Cluster distances.

[0147] 4. Initialize itemToCluster mapping to store QuadItem <t>To its own SimpleCluster <t>The mapping relationship.

[0148] First, iterate over each QuadItem in mItems <t>The candidate object candidate. If candidate is already in visitedCandidates, skip the current loop iteration.

[0149] If the above conditions are not met, retrieve the QuadItem that intersects retrieveBounds from mQuadTree <t>The collection is clusterItems. Then perform the following operations on the collection:

[0150] If the size of the clusterItems collection is 1, then the candidate is added directly to the results collection, and the candidate is added to the visitedCandidates collection. Finally, in the distanceToCluster map, the candidate is mapped to a distance of 0.

[0151] If the size of the clusterItems collection is not 1, that is, clusterItems contains multiple elements, then we need to create a new SimpleCluster <t>Object cluster, used to add cluster to QuadItem <t>Collection clusterItems.

[0152] Then continue to traverse each item and calculate the distance from each item point to the candidate point as squaredDistance.

[0153] Finally, this embodiment also introduces a cache mechanism to store the above aggregated results, thereby improving the client's access speed to data and increasing the efficiency of page rendering.

[0154] Through the above algorithm implementation, the present invention enhances the flexibility of flexible data mapping from multiple aspects, including dynamic parameter adjustment mechanism, etc., so that the method of the present invention can adapt to the needs of different data scales and application scenarios, with convenient access and fast mapping.

[0155] S4, visualization step. The front end uses advanced visualization technology to display the aggregation results on the map in the form of points or other forms. The size, color and transparency of the points are adjusted according to the number, density and location of the aggregation points. Custom pop-up window configuration of resources on the map is implemented according to different data types.

[0156] Specifically, this step mainly involves the visualization and rendering of various processed geographic information data on the front-end map page. Advanced visualization technology is used to display the aggregation results on the map in the form of points or other forms. Figure 7 As shown, there are mainly the following implementation steps:

[0157] Step 1: Use any map as the base map.

[0158] Step 2: Add a grid overlay layer, annotations, etc. to the called map.

[0159] Step 3: Get the data to be displayed from the server and display it as shown above according to different data categories.

[0160] Step 4: Obtain the aggregated data to be displayed in the visible area of ​​the current screen through the first three steps, locate all data to their corresponding grid longitude and latitude and render them into the map, thus achieving the purpose of data aggregation display.

[0161] When the map level changes, that is, when the map is zoomed in, zoomed out, or dragged, just repeat steps 2 to 4. The map services currently supported in step 1 are pgis map, Tiandi map, Baidu map, Google map, and 3D satellite map.

[0162] The front-end visualization display module uses OpenLayers to scatter points on the data map, and can adjust the size, color and transparency of the points according to the number, density and position of the aggregation points. At the same time, it can also implement customized pop-up window configuration on the resource map according to different data types to improve the visualization effect and make the data display more intuitive and easy to understand.

[0163] Figure 8 A schematic diagram of custom grid area acquisition in this embodiment is shown; Fig. 9 The figure shows the effect of scattering points on the data aggregation in this embodiment.

[0164] The beneficial effects brought about by the technical solution disclosed in the embodiments of the present invention are mainly reflected in the following aspects:

[0165] 1. Improve performance and efficiency

[0166] Reduce rendering pressure: By aggregating large amounts of data first and then scattering the aggregated data points on the map, the amount of data required for map rendering can be significantly reduced, thereby reducing the rendering pressure of the browser and improving the loading speed and response speed of the map.

[0167] Optimize user experience: reduce waiting time, avoid long periods of white screen or freeze during map loading, and improve the overall user experience.

[0168] 2. Improve the readability and clarity of data visualization

[0169] Reduce visual clutter: A large number of data points scattered directly on the map often leads to visual clutter and difficulty in distinguishing important information. Through aggregation processing, similar data points can be combined into a larger point or area, making the information on the map clearer and more organized.

[0170] Highlight key information: The aggregated data points can be displayed differently according to the amount of data, such as size, color depth, etc., to highlight data-intensive areas or important data points.

[0171] 3. Facilitate data analysis and decision-making

[0172] Data Overview: By aggregating the points in the above figure, users can quickly obtain the overall distribution of data and understand the density and changing trends of data in different regions.

[0173] Decision support: Based on the aggregated data, further operations such as spatial analysis and hot spot analysis can be performed to provide strong data support for decision-making.

[0174] 4. Support large-scale data processing

[0175] Relying on the big data governance platform, this method can handle data volumes ranging from thousands to millions, while also supporting various types of data access and has strong scalability.

[0176] 5. Rich application scenarios and high flexibility

[0177] This method has a wide range of application scenarios, and the use and display of data is more reasonable, more practical, and more adaptable to the various needs of current users without repeated custom development. For example, it can be applied to the field of video security, the geographical distribution of camera equipment, face, vehicle trajectory and path analysis, and surrounding resource analysis and planning.

[0178] Further references Fig.10 As an implementation of the methods shown in the above figures, the present application provides an embodiment of a system, and the device embodiment is Figure 1 Corresponding to the method embodiment shown, the system can be specifically applied to various electronic devices.

[0179] In a second aspect, the embodiment of the present invention also discloses a system for aggregating large amounts of data and scattering points on a graph, such as Fig.10 As shown, it includes: a data governance module 101, an intelligent grid division module 102, a data aggregation module 103 and a visualization display module 104.

[0180] In a specific embodiment, the data governance module 101 is configured to use the big data governance platform to collect, access, manage, analyze and store geographic information data, and the governance includes data cleaning, jurisdiction boundary data management, longitude and latitude conversion, data association integration and field standard specification; the intelligent grid division module 102 is configured to divide the grid in at least one of the following ways, including: dividing the grid according to the jurisdiction boundary, rendering the corresponding area outline on the map according to the jurisdiction boundary of the data management; custom grid, the user builds thematic scenes according to actual needs, and performs custom area division and collection on the map; the data aggregation module 103 is configured to use parallel computing and distributed processing technology on the basis of grid division, combined with user-defined aggregation conditions for data aggregation; the visualization display module 104 is configured for the front end to use advanced visualization technology to display the aggregation results on the map in the form of points or other forms, adjust the size, color and transparency of the points according to the number, density and position of the aggregation points, and implement custom pop-up window configuration of resources on the map according to different data types.

[0181] The functions of the above modules correspond to the methods and will not be repeated here.

[0182] Reference below Fig.11 , which shows a schematic structural diagram of a computer device 600 of an electronic device suitable for implementing an embodiment of the present invention. Fig.11 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0183] like Fig.11 As shown, the computer device 600 includes a central processing unit (CPU) 601 and a graphics processing unit (GPU) 602, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 603 or a program loaded from a storage part 609 to a random access memory (RAM) 604. In the RAM 604, various programs and data required for the operation of the device 600 are also stored. The CPU 601, GPU 602, ROM 603 and RAM 604 are connected to each other through a bus 605. An input / output (I / O) interface 606 is also connected to the bus 605.

[0184] The following components are connected to the I / O interface 606: an input section 607 including a keyboard, a mouse, etc.; an output section 608 including a display such as a liquid crystal display (LCD), etc. and a speaker, etc.; a storage section 609 including a hard disk, etc.; and a communication section 610 including a network interface card such as a LAN card, a modem, etc. The communication section 610 performs communication processing via a network such as the Internet. A drive 611 may also be connected to the I / O interface 606 as needed. A removable medium 612, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 611 as needed so that a computer program read therefrom is installed into the storage section 609 as needed.

[0185] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 610, and / or installed from the removable medium 612. When the computer program is executed by the central processing unit (CPU) 601 and the graphics processing unit (GPU) 602, the above-mentioned functions defined in the method of the present invention are executed.

[0186] It should be noted that the computer-readable medium described in the present invention may be a computer-readable signal medium or a computer-readable medium or any combination of the above two. The computer-readable medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor device, apparatus or device, or any combination of the above. More specific examples of computer-readable media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution device, apparatus or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable medium other than a computer-readable medium that can send, propagate or transmit a program for use by or in conjunction with an instruction execution device, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0187] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0188] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the device, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based device that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0189] The modules involved in the embodiments of the present invention may be implemented in software or hardware, and the modules described may also be arranged in a processor.

[0190] As another aspect, the present invention further provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by the electronic device, the electronic device executes the method steps described in the first aspect.

[0191] The above description is only a preferred embodiment of the present invention and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present invention is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present invention (but not limited to) to form a technical solution.< / t> < / t> < / t> < / t> < / t> < / t> < / t> < / t> < / t> < / clusteritem> < / clusteritem> < / t> < / t> < / t> < / t> < / t> < / t> < / t> < / t> < / t> < / clusteritem> < / clusteritem>

Claims

1. A method for scattering points on a large-scale data aggregation graph, characterized in that: The method comprises the following steps: Data governance step: using the big data governance platform to collect, access, manage, analyze and store geographic information data. The governance includes data cleaning, jurisdiction boundary data management, longitude and latitude conversion, data association integration and field standard specification; The smart grid division step divides the grid by at least one of the following methods, including: dividing the grid according to the boundaries of the jurisdiction, rendering the outline of the corresponding area on the map according to the boundaries of the jurisdiction of data governance; custom grid, the user builds thematic scenes according to actual needs, and performs custom area division and collection on the map; The data aggregation step uses parallel computing and distributed processing technology based on grid division and combines user-defined aggregation conditions to perform data aggregation; In the visualization step, the front end uses advanced visualization technology to display the aggregation results on the map in the form of points or other forms. The size, color and transparency of the points are adjusted according to the number, density and location of the aggregation points, and the custom pop-up window configuration of the resource map is implemented according to different data types.

2. The method for scattering points on a large-scale data aggregation map according to claim 1 is characterized in that: In the data aggregation step, the aggregation conditions include the map level to be aggregated, the dynamic threshold of the map element aggregation data volume, and the dynamic threshold of the grid distance, and a grid shortest distance aggregation algorithm is adopted. The aggregation algorithm includes converting longitude and latitude into two-dimensional coordinates, using the quadtree structure QuadTree to convert the defined Item object into a QuadItem object and save it in the QuadTree tree, and determining the aggregated cluster by iterating the QuadItem.

3. The method for scattering points on a large-scale data aggregation map according to claim 2 is characterized in that: The aggregation algorithm also includes: Provide a ClusterItem interface to generate a marker point on the map. The marker point object is then added to the management container of the aggregate map to implement the point aggregation function. The marker point contains the latitude and longitude coordinates, center point, icon, color and other attributes that need to be displayed on the map. The ClusterManager interface is used to monitor changes in map status, such as performing point aggregation operations when zooming and dragging the map, and monitoring click events of marker points to render display effects.

4. The method for scattering points on a large-scale data aggregation map according to claim 3 is characterized in that: In the data aggregation step, the main steps of implementing the point aggregation function include: Define the coordinate points of the current screen visualization area: sw is the southwest coordinate of the visual area, ne is the northeast coordinate of the visual area; Determine whether data needs to be aggregated: Customize the map level that needs to be aggregated, the dynamic threshold of the amount of map element aggregation data, and the dynamic threshold of the grid distance. That is, when the map is zoomed to a level above this level, it will no longer be aggregated, or it will be aggregated only when the same longitude and latitude are used, or it is assumed that no aggregation will be performed when the number is less than n; Determine the aggregation algorithm: the main one used is the grid shortest distance aggregation algorithm.

5. The method for scattering points on a large-scale data aggregation map according to claim 4 is characterized in that: In the data aggregation step, the method of converting longitude and latitude into two-dimensional coordinates includes: Map longitude from [-180,180] to [0,1] to calculate the X coordinate, double x = lngLat.getLongitude() / 360+0.5; After converting the latitude to radians, use trigonometric functions to convert the latitude to sine values, and then use trigonometric functions to convert the latitude to sine values: double sinY = Math.sin(Math.toRadians(lngLat.getLatitude())), and then use Mercator projection or similar projection to map the latitude to [0,1] to calculate the Y coordinate, double y = 0.5*Math.log((1+sinY) / (1-sinY)) / -(2*Math.PI)+0.5; Finally, create an object containing the X and Y coordinates and multiply them by the world width to get the coordinates in the world coordinate system, QuadTreeItem.Point(x*mWorldWidth, y*mWorldWidth, where the world width is calculated by the formula 256*(zoom^2)=worldWidth, where zoom is the current zoom level of the map.

6. The method for scattering points on a large-scale data aggregation map according to claim 3 is characterized in that: In the data aggregation step, the implementation method of determining whether a point is within the current grid or polygon area includes: Define the current point as Point2D.Double point; the fixed point of the polygon is List<Point2D.Double> pts; the tolerance when comparing floating point type with 0 is double precision = 2e-10; the adjacent boundary points are Point2D.Double p1, p2; First, assume that p, p1, and p2 are all point objects with x and y attributes, and pts is a set containing points. N is the number of points in the set. If a point is located on a vertex or edge of a polygon, it is considered to be inside the polygon and true is returned directly. In the ClusterManager, a quadtree structure QuadTree is introduced, that is, all defined Item objects are converted into QuadItem objects and then saved in the QuadTree tree; Next, iterate over all QuadItems and draw a box with this QuadItem as the center. The points in the box are considered a cluster, that is, QuadItems that can be aggregated into a point. The size of the box is the defined distance for aggregation. Finally, a cluster collection is returned, which is the final aggregation result.

7. The method for scattering points on a large-scale data aggregation map according to claim 6 is characterized in that: In the data aggregation step, performing cluster set operations includes: First, define a method to perform aggregation, runClustering, which accepts the following parameters: features, a list of entities of type FeatureDO; clusterAlgoKey, a string representing the key value of the clustering algorithm; zoom, an integer representing the current map zoom level; hasVisibleBounds, a Boolean value indicating whether there are visible area coordinates; the return type is a Cluster <clusteritem> A collection of types;< / clusteritem> Use the StrUtil.blankToDefault method to convert clusterAlgoKey to a non-empty string. The default value is the algorithm name in clusterProperties. The result is stored in clusterAlgoName. Get the ClusterAlgorithm corresponding to clusterAlgoName through the ClusterAlgoFactory.getAlgorithm method <clusteritem> Instances, stored in clusterAlgo;< / clusteritem> Filter and convert the features list. First, filter out null elements and FeatureDO objects whose type is not POINT. Then, for each remaining element, perform the following operations: If hasVisibleBounds is true, that is, the coordinates of the visualization area are passed in, then get the coordinates of the feature, that is, the longitude and latitude, from the featureCache. Otherwise, the longitude and latitude are set to the default value (0,0). Create a new Map and put the feature into the Map cache as the value of CLUSTER_MAP_KEY_FEATURE. Create a new RegionItem object and use the longitude and latitude, Map data, and feature weight obtained in the previous step. Collect the converted RegionItem objects into a list. Add the list obtained in the previous step to the items of clusterAlgo; Call the getCluster method of clusterAlgo, pass in the zoom level, and obtain and return the clustering result.

8. The method for scattering points on a large-scale data aggregation map according to claim 7 is characterized in that: In the data aggregation step, implementing the add method of QuadItem includes: Initialize the visitedCandidates collection to store the visited QuadItems <t>Object; initialize the results collection to store the final Cluster <t>Object; initializes the distanceToCluster map to store QuadItem <t>Mapping relationship to other nearest Cluster distances; initialize itemToCluster mapping for storing QuadItem <t>To its own SimpleCluster <t> The mapping relationship of< / t> < / t> < / t> < / t> < / t> First, iterate through each QuadItem in mItems <t> Candidate object candidate, if candidate is already in visitedCandidates, skip the current loop iteration;< / t> If the above conditions are not met, retrieve the QuadItem that intersects retrieveBounds from mQuadTree <t> The clusterItems collection is then used to perform the following operations on the collection:< / t> If the size of the clusterItems collection is 1, then the candidate is added directly to the results collection, and the candidate is added to the visitedCandidates collection, and finally in the distanceToCluster mapping, the candidate is mapped to a distance of 0; If the size of the clusterItems collection is not 1, that is, clusterItems contains multiple elements, you need to create a new SimpleCluster <t>Object cluster, used to add cluster to QuadItem <t> collection clusterItems;< / t> < / t> Then continue to traverse each item and calculate the distance from each item point to the candidate point as squaredDistance.

9. The method for scattering points on a large-scale data aggregation map according to claim 1 is characterized in that: In the intelligent grid division step, the grid division according to the boundaries of the jurisdiction also includes automatically switching to the corresponding jurisdiction according to the current map level when the map is zoomed and displaying the data displayed in an aggregated manner in the area, supporting the jurisdiction penetration function; In the custom grid, you can divide and collect custom areas on the map, including the collection of squares, rectangles, polygons, circles and irregular shapes, or draw the jurisdiction based on the route on the map, set the default display center point and level, and then collect data points in the area or access the data to display on the map.

10. The method for scattering points on a large-scale data aggregation map according to claim 1 is characterized in that: In the visualization display step, it also includes calling a map service as a base map, adding a grid overlay layer and annotations on the base map, obtaining the data to be displayed from the server and displaying it on the map according to different data categories, obtaining the aggregated data to be displayed in the visible area of ​​the current screen through the above, locating all data to their corresponding grid longitudes and latitudes and rendering them into the map, thereby achieving the purpose of data aggregation display; the supported map services include pgis maps, tiandimap, baidu maps, google maps and 3D satellite maps.

11. A system for mass data aggregation and scattering, characterized in that: include: A data governance module is configured to collect, access, manage, analyze and store geographic information data using a big data governance platform, including data cleaning, jurisdiction boundary data management, longitude and latitude conversion, data association integration and field standard specification; The intelligent grid division module is configured to divide the grid in at least one of the following ways, including: dividing the grid according to the boundaries of the jurisdiction, rendering the corresponding area outline on the map according to the boundaries of the jurisdiction of data governance; custom grid, the user builds thematic scenes according to actual needs, and performs custom area division and collection on the map; The data aggregation module is configured to aggregate data based on grid division, using parallel computing and distributed processing technology combined with user-defined aggregation conditions; The visualization display module is configured to use advanced visualization technology on the front end to display the aggregation results on the map in the form of points or other forms. The size, color and transparency of the points are adjusted according to the number, density and location of the aggregation points, and the custom pop-up window configuration of the resource map is implemented according to different data types.

12. The system for mass data aggregation and graph scattering according to claim 11, characterized in that: In the data aggregation module, the aggregation conditions include the map level that needs to be aggregated, the dynamic threshold of the map element aggregation data volume, and the dynamic threshold of the grid distance, and a grid shortest distance aggregation algorithm is adopted. The aggregation algorithm includes converting longitude and latitude into two-dimensional coordinates, using the quadtree structure QuadTree to convert the defined Item object into a QuadItem object and save it in the QuadTree tree, and determining the aggregated cluster by iterating the QuadItem.

13. The system for mass data aggregation and graph scattering according to claim 11, characterized in that: In the intelligent grid division module, the grid division according to the boundaries of the jurisdiction also includes automatically switching to the corresponding jurisdiction according to the current map level when the map is zoomed and displaying the data displayed in an aggregated manner in the area, supporting the jurisdiction penetration function; In the custom grid, you can divide and collect custom areas on the map, including the collection of squares, rectangles, polygons, circles and irregular shapes, or draw the jurisdiction based on the route on the map, set the default display center point and level, and then collect data points in the area or access the data to display on the map.

14. The system for mass data aggregation and graph scattering according to claim 11, characterized in that: In the visualization display module, it also includes calling a map service as a base map, adding a grid overlay layer and annotations on the base map, obtaining the data to be displayed from the server and displaying it on the map according to different data categories, obtaining the aggregated data to be displayed in the visible area of ​​the current screen, locating all data to their corresponding grid longitudes and latitudes and rendering them into the map, thereby achieving the purpose of data aggregation display; the supported map services include pgis maps, tiandimap, baidu maps, google maps and 3D satellite maps.

15. An electronic device, comprising: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 10.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

Images