Method and system for rendering GIS (Geographic Information System) mass data of electric power metering equipment

Abstract

The invention discloses a rendering method and system for GIS mass data of electric power metering equipment. The method comprises the steps that a background server regularly generates an equipment GIS full-amount snapshot data binary file; the client side directly obtains the equipment GIS full-amount snapshot data binary file from the file server; the client analyzes the binary file and renders the binary file; acquiring longitude and latitude coordinate data of state change from the binary file generation time to the access time, and performing incremental updating; and when the data of the equipment changes, data updating is actively carried out on the background server. According to the method, the bottleneck problem of GIS visualization performance in real-time position monitoring of massive equipment such as a power concentrator and a meter is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of electric power, and in particular to a rendering method and system for GIS mass data of electric power metering equipment. BACKGROUND

[0002] In the scenario of electric power system equipment monitoring, the Web GIS (Geographic Information System) platform faces severe performance challenges in rendering massive spatial data of metering equipment (such as concentrators and meters) and directly affects the efficiency of real-time monitoring and operation and maintenance of the power grid. The main technical bottlenecks are as follows:

[0003] (1) Data transmission bottleneck: The real-time location and state data of the massive metering equipment in the transformer area (usually up to tens of thousands of devices) are huge, and the traditional HTTP protocol cannot support high-frequency data transmission, resulting in significant communication delay and bandwidth pressure, which affects the timeliness of real-time monitoring of the power grid.

[0004] (2) Performance bottleneck of rendering engine: The Canvas rendering used in traditional power grid GIS systems has a frame rate generally lower than 10 FPS in the scenario of tens of thousands of device point elements. When using DOM element type markers (such as Leaflet's CircleMarker), the memory usage of 5000+ device instances exceeds 2 GB, causing browser lag and even crashes in the dispatching system.

[0005] (3) Real-time update and memory management bottleneck: Metering equipment data needs to be updated at a minute or even second level (such as fault positioning scenarios), and frequent state calculation and coordinate update will significantly increase CPU load. At the same time, the memory residence of device historical trajectory data will further exacerbate resource consumption, affecting the operation experience of dispatch personnel.

[0006] Current GIS solutions in the electric power industry mainly rely on WMTS / WMS map servers:

[0007] (1) WMTS (Web Map Tile Service) server: Pre-generate vector tiles with spatial coordinates for device data, suitable for displaying device distribution heat maps.

[0008] (2) WMS (Web Map Service) server: Render real-time device status as a raster image superimposed on the power grid topology map, suitable for device status monitoring.

[0009] The above solutions have the following obvious defects:

[0010] 1) Multiple high-performance servers must be deployed to form a cluster, which is costly.

[0011] 2) A single ordinary server cannot support real-time updates of 10,000 devices.

[0012] 3) Low resource utilization (less than 30%) resulting in waste.

[0013] 4) Requires a professional team for maintenance, and operation and maintenance is complex.

[0014] 5) When device data is updated frequently, the server is under heavy computing pressure and visualization is delayed.

[0015] Especially in the county grid and other budget limited scenarios, the existing solution is difficult to achieve device second-level update and smooth display in a low-cost single machine environment. SUMMARY

[0016] The technical problem to be solved by the present application is to provide a method for efficient update and rendering of GIS data for single machine deployment of GIS visualization in real-time location monitoring of a large number of devices such as power concentrators and meters in the electric power system.

[0017] According to an aspect of the present application, a rendering method for GIS massive data of electric power metering devices is provided, comprising the following steps:

[0018] The background server generates device GIS full snapshot data binary files at regular intervals, and stores the binary files in a file server;

[0019] The client directly obtains the binary files from the file server;

[0020] The client parses the binary files and performs rendering;

[0021] Obtain the latitude and longitude coordinate data that has changed between the binary file generation time and the access time, and perform incremental update, the incremental update comprising: the client sends the binary file generation time and the access time to the backend service; the backend service queries the latitude and longitude coordinate data that has changed between the binary file generation time and the client access time; the backend service removes duplicate data; the latitude and longitude coordinate data after removing duplicate data is returned to the client; the client updates and renders the latitude and longitude coordinate data;

[0022] If the device itself data changes, the device actively updates the data to the background server.

[0023] According to some embodiments,

[0024] The data elements in the GIS full snapshot data binary file include:

[0025] Latitude and longitude data and / or device online data.

[0026] According to some embodiments,

[0027] Each piece of data in the GIS full snapshot data binary file occupies 9 bytes;

[0028] The latitude and longitude data are floating point type data, each occupying 4 bytes;

[0029] The device online data is integer type data, occupying 1 byte.

[0030] According to some embodiments,

[0031] The device GIS full snapshot data binary file generation task is configured by using a distributed task scheduling framework.

[0032] According to some embodiments,

[0033] The file name of the GIS full snapshot data binary file is time stamped.

[0034] According to some embodiments,

[0035] The file name further includes a data set slice index.

[0036] According to another aspect of the present application, a power metering device GIS massive data rendering system is also provided, which is used to implement the power metering device GIS massive data rendering method described above, and the system comprises:

[0037] A background server is used to provide a backend service, including generating a device GIS full snapshot data binary file at a regular time;

[0038] A file server is used to provide a file service, including storing the device GIS full snapshot data binary file;

[0039] The nginx has the functions of static resource service, reverse proxy, load balancing, and HTTP cache;

[0040] A client is used for user interaction, request initiation, data processing and display, and the client includes a mobile phone, a tablet computer, a notebook computer, and a desktop computer.

[0041] According to another aspect of the present application, a computer readable storage medium is also provided, which comprises a stored computer program, wherein when the computer program is run by a processor, the device where the storage medium is located is controlled to perform the power metering device GIS massive data rendering method described above.

[0042] The present application has the following beneficial effects:

[0043] The technical scheme of the application realizes real-time monitoring and smooth display of millions of power equipment (concentrators, meters, etc.) in a low-cost single server environment through a "binary snapshot + incremental update" mechanism, effectively solving the three core problems existing in traditional solutions:

[0044] ① Breakthrough cluster dependence: only a single ordinary configuration server can support efficient processing and rendering of massive device data;

[0045] ② Optimize resource utilization: through binary snapshot compression and incremental transmission technology, the server resource utilization rate is improved to more than 80%;

[0046] ③ Reduce the threshold of operation and maintenance: use the automatic snapshot generation and update mechanism to complete system maintenance without professional GIS team.

[0047] The technical scheme of the application is particularly suitable for small power distribution network transformation and other budget-limited power application scenarios, which can greatly reduce the system construction and operation and maintenance cost while ensuring device data update.

[0048] The technical scheme of the application has the following beneficial technical effects:

[0049] 1) Significantly reduce server load: the client mainly obtains static binary snapshots from the file server, avoiding high-frequency direct access to the core business server and database, significantly reducing server pressure.

[0050] 2) Significantly save network bandwidth: store full data in binary format, which is much smaller than text format; subsequently, only incremental data of state changes are transmitted through the interface, greatly reducing network transmission volume.

[0051] 3) Significantly improve client performance: binary file parsing speed is fast; after initial rendering based on snapshots, only incremental updates of changed devices are required, making map rendering more smooth and efficient, and user experience better.

[0052] 4) Enhance system scalability: file servers (such as CDN) naturally support high concurrency downloads, combined with server decompression, making the system easily support real-time location monitoring of massive devices. BRIEF DESCRIPTION OF DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description. Obviously, the drawings in the following description are only some embodiments of the application, and for those skilled in the art, other drawings can be obtained without creative labor based on these drawings.

[0054] Figure 1A rendering system architecture diagram of power metering device GIS mass data according to an example embodiment is shown.

[0055] Figure 2 A rendering flow chart of power metering device GIS mass data according to an example embodiment is shown. DETAILED DESCRIPTION

[0056] The embodiments of the present application will be described in detail below with reference to the drawings. It should be noted that the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of the present application.

[0057] Those skilled in the art should know that the following specific embodiments or specific embodiments are a series of optimized settings listed by the present application to further explain the specific application content, and these settings can be used together or in association with each other, unless the present application explicitly states that some or a specific embodiment or embodiment cannot be associated with other embodiments or embodiments. At the same time, the following specific embodiments or embodiments are only the most optimized setting, not as an understanding of the protection scope of the present application.

[0058] Example 1

[0059] Figure 1 A rendering system architecture diagram of power metering device GIS mass data according to an example embodiment is shown.

[0060] As shown in Figure 1 A rendering system of power metering device GIS mass data includes a background JAVA server for providing backend services, including generating device GIS full snapshot data binary files at regular intervals, receiving data actively reported by the device to the service; a file server for providing file services, including storing device GIS full snapshot data binary files; nginx, with static resource service, reverse proxy, load balancing, HTTP cache function, including receiving customer end data request, reverse proxy JAVA service, etc.; a client for user interaction, request initiation, data processing and display, the client includes mobile phones, tablet computers, notebook computers, desktop computers.

[0061] The main process of the system of the present embodiment to realize the rendering method of power metering device GIS mass data is as follows:

[0062] 1) The background server generates device GIS full snapshot data binary files (data elements in binary files: longitude and latitude, whether the device is online) with a timestamp in the file name.

[0063] According to the business situation and server performance, a distributed task scheduling framework (such as XXL-JOB) is used to configure snapshot generation tasks, and the task is set through a cron expression (such as: 0 * / 2 * * *; generate once every two hours), and the CPU of the single-click server will occupy a high running memory when generating the task, and the time setting needs to not affect the running of the main business.

[0064] Among them, the device data generation binary code example is as follows (nodeJs):

[0065] / / Generate binary data function

[0066] function generateDeviceBinaryData(deviceList) {

[0067] const buffer = new ArrayBuffer(deviceList.length * 9); / / Each data is 9 bytes

[0068] const dataView = new DataView(buffer);

[0069] / / Fill in the access device data

[0070] for (let i = 0; i < deviceList.length; i++) {

[0071] const offset = i * 9;

[0072] / / Longitude (-180~180)

[0073] dataView.setFloat32(offset, deviceList[i].long, true);

[0074] / / Latitude (-90~90)

[0075] dataView.setFloat32(offset + 4, deviceList[i].lat, true);

[0076] / / Status (0 / 1)

[0077] dataView.setUint8(offset + 8, deviceList[i].status);

[0078] }

[0079] return buffer;

[0080] };

[0081] In each piece of data, the longitude and latitude use Float32 floating-point data type, each data occupies four bytes, and the device status uses integer type, occupying 1 byte.

[0082] Use the zlib third-party library for high-performance data compression to reduce file size.

[0083] const compressed = zlib.deflateSync(Buffer.from(binaryData));

[0084] Compose the file name (e.g., 1751609957861_1.bin) by timestamp and data set slice index (assuming 1 million device data, 10,000 slices, then 10 files will be generated, which can be set according to the actual business scenario).

[0085] const filename = `${new Date().getTime()}_${index}`;

[0086] fs.writeFileSync(filename, binaryData);

[0087] 2) The client directly obtains the device GIS full snapshot data binary file from the file server through nginx proxy; parses and renders the file, and configures the file service to cache the file on the client side, improving access efficiency next time.

[0088] The client gets binary data through an HTTP request, parses it, and then renders it on the client side.

[0089] The following is an example of rendering code:

[0090] import { ScatterplotLayer} from 'deck.gl';

[0091] / / Parse data from binary

[0092] const dataView = new DataView(binaryData);

[0093] const numDevices = binaryData.byteLength / 9;

[0094] const layer = new ScatterplotLayer({

[0095] data: {

[0096] length: numDevices,

[0097] attributes: {

[0098] getPosition: {

[0099] value: new Float32Array(binaryData, 0, numDevices * 2), / / Longitude + Latitude

[0100] size: 2

[0101] },

[0102] getFillColor: {

[0103] value: new Uint8Array(binaryData, 8, numDevices), / / State

[0104] size: 1,

[0105] normalized: true / / Normalizes uint8[0,255] to [0,1]

[0106] }

[0107] }

[0108] },

[0109] getPosition: d => [d[0], d[1]],

[0110] getFillColor: d => [

[0111] d[2] === 0 ? 255 : 0, / / Offline = red, Online = green

[0112] d[2] === 1 ? 255 : 0,

[0113] 0,

[0114] 200 / / transparency

[0115] ],

[0116] radiusMinPixels: 2,

[0117] opacity: 0.8

[0118] });

[0119] If the latitude and longitude are stored as integers, you need to convert them to floating-point data types when rendering. For example, if you want to store the latitude and longitude with six decimal places, you need to multiply the latitude by 10^6 when storing it as an integer, and divide it by 10^6 when rendering.

[0120] Client-side caching for binary files. For example, the Nginx caching code is as follows:

[0121] http {

[0122] # Define cache path and parameters

[0123] proxy_cache_path / var / cache / nginx / html / geo levels=1:2 keys_zone=my_cache:100m inactive=2h use_temp_path=off;

[0124] server {

[0125] listen 80;

[0126] server_name example.com;

[0127] location / geoFile {

[0128] # Enable caching

[0129] proxy_cache my_cache;

[0130] proxy_cache_valid 200 302 2h; # Cache HTTP 200 / 302 responses for 2 hours

[0131] proxy_cache_valid 404 1m; # Cache 404 for 1 minute

[0132] # Other cache control headers (optional)

[0133] add_header X-Proxy-Cache $upstream_cache_status;

[0134] proxy_cache_key "$scheme$request_method$host$request_uri";

[0135] }

[0136] }

[0137] }

[0138] 3) The client obtains the latitude and longitude coordinate data of the state change from file generation time to access time in the form of an interface through nginx proxy for incremental update.

[0139] The differential update process is as follows:

[0140] The client sends the file generation time and access time to the backend service through the interface, the backend queries the device state change data of the file generation time and the current access time, and removes the data (for example, a device frequently changes at this time point, and the last state is consistent with the file generation state, remove this data and return), and then returns the data to the client. The client updates the rendered device data through latitude and longitude.

[0141] The code example is as follows:

[0142] const localUpdateTime = localStorage.getItem(“updateTimestamp”);

[0143] const updateFileTime = filename.split(“_”)[0];

[0144] let updateTime = “”;

[0145] if (localUpdateTime < updateTime){

[0146] updateTime = updateFileTime;

[0147] localStorage.setItem(“updateTimestamp”, `${ updateFileTime}`);

[0148] }else{

[0149] updateTime = localUpdateTime

[0150] };

[0151] When the latest data is clicked to refresh in the GIS interface, the refreshing can start from the time when the interface is entered (i.e., the differentiated access time is called when the interface is entered, and the current time when the refresh button is clicked), and the data in this time period is obtained through the differentiated interface.

[0152] In this scenario, one latitude and longitude is mainly solved for one device, and the device cannot be moved. The latitude and longitude is used as the unique value of the device to associate the data. When the mouse moves to the point of the device, that is, the icon, the latitude and longitude value is used as a parameter to query the device data through the backend service interface and tooltip prompt (such as: showing the running time of the device, whether it is online, the device number, the manufacturer information, etc.).

[0153] 4) A certain device data changes itself and actively updates and stores the data to the background service.

[0154] Example 2

[0155] Figure 2 A rendering flowchart of power metering device GIS massive data according to an example embodiment is shown

[0156] As shown in Figure 2 , the process of the power metering device GIS massive data rendering method implemented in the client is as follows:

[0157] 1) The client accesses the GIS interface and initiates a data request to the server.

[0158] 2) The server first returns the binary file name generated at regular intervals.

[0159] 3) After obtaining the binary file name, the client obtains the full device snapshot data from the file server and performs rendering.

[0160] 4) Through the timestamp of the binary file name and the current access time, the client obtains the incremental data in the time period through the interface.

[0161] 5) After obtaining the incremental data, the changed device data is replaced, and the changed data is parsed and rendered.

[0162] 6) The client GIS interface displays all device states.

[0163] The specific embodiments of the application also provide a computer readable storage medium having a program stored thereon, which, when executed by a processor, implements the rendering method of the power metering device GIS massive data in the above embodiments.

[0164] The computer readable storage medium can be an internal storage unit of any of the aforementioned devices with data processing capability, such as a hard disk or a memory. The computer readable storage medium can also be an external storage device of any of the aforementioned devices with data processing capability, such as a plug-in hard disk, a Smart Media Card (SMC), an SD card, a Flash Card, etc. Further, the computer readable storage medium can include both an internal storage unit and an external storage device of any of the aforementioned devices with data processing capability. The computer readable storage medium is used to store the computer program and other programs and data required by the aforementioned devices with data processing capability, and can also be used to temporarily store data that has been output or will be output.

[0165] The above merely provides the specific implementation of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of the changes or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for rendering massive GIS data from power metering equipment, characterized in that, Includes the following steps: The background server periodically generates binary files of full snapshot data of the device GIS and stores the binary files in the file server; The client obtains the binary file directly from the file server; The client parses the binary file and renders it. The system acquires latitude and longitude coordinate data showing state changes between the binary file's generation time and access time, and performs incremental updates. The incremental update includes: the client sending the binary file's generation time and access time to the backend service; the backend service querying latitude and longitude coordinate data showing device state changes between the binary file's generation time and the client's access time; the backend service removing duplicate data; returning the deduplicated latitude and longitude coordinate data to the client; and the client updating and rendering the latitude and longitude coordinate data. If the device's own data changes, the device will proactively update the data with the backend server.

2. The method for rendering massive GIS data of power metering equipment according to claim 1, characterized in that, The data elements in the binary file of the GIS full snapshot data include: Latitude and longitude data and / or whether the device is online.

3. The method for rendering massive GIS data of power metering equipment according to claim 2, characterized in that, In the binary file of the full GIS snapshot data, each data entry occupies 9 bytes; The latitude and longitude data are floating-point data, each occupying 4 bytes; The device online status data is an integer, occupying 1 byte.

4. The method for rendering massive GIS data of power metering equipment according to claim 1, characterized in that, A distributed task scheduling framework is used to configure the task of generating binary files of the full snapshot data of the device's GIS.

5. The method for rendering massive GIS data of power metering equipment according to claim 1, characterized in that, The filenames of the binary files containing the full GIS snapshot data are timestamped.

6. The method for rendering massive GIS data of power metering equipment according to claim 5, characterized in that, The filename also includes a dataset slice index.

7. A rendering system for massive GIS data of power metering equipment, used to implement the method of any one of claims 1-6, characterized in that, include: The backend server is used to provide backend services, including the periodic generation of binary files containing full snapshot data of the device's GIS. A file server is used to provide file services, including storing binary files of full snapshot data of GIS on storage devices; Nginx provides static resource services, reverse proxy, load balancing, and HTTP caching. A client is used for user interaction, request initiation, data processing and display. The client includes mobile phones, tablets, laptops and desktop computers.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein the computer program, when executed by a processor, controls the device containing the storage medium to perform a rendering method for massive GIS data of power metering equipment as described in any one of claims 1-6.

Images