A paging retrieval method for power grid model data
Through paging retrieval method and read-write lock synchronization technology, the problem of insufficient memory in power grid model data processing is solved, and efficient display and stable operation are achieved.
Patent Information
- Application Number
- CN202310011116.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-01-05
- Publication Date
- 2025-08-15
- Estimated Expiration
- 2043-01-05
AI Technical Summary
When the prior art processes power grid model data of a scale of more than one million, it leads to insufficient memory in the workstation, affecting the data refresh speed and endangering the normal operation of other components.
The pagination search method is used to generate table parameter requests through the client, and the server passes back data. The client uses a subcontracting algorithm to generate multiple table data requests. It uses read and write locks to synchronize threads and map container objects, store data in a temporary file, and load and display data in memory.
It significantly improves the display speed of power grid model data, reduces client memory usage, ensures the stable operation of the system, and shortens the time for maintenance personnel to view model data.
Smart Images
Figure CN116070044B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a paging retrieval method for power grid model data, and belongs to the technical field of smart grid data processing. Background Art
[0002] In power grid dispatching and control systems, to ensure reliable and efficient system operation, engineers and operation and maintenance personnel need to frequently view and modify system model data.
[0003] Currently, grid model data is obtained by sending a request from a client on a workstation to a microservice running on the server. The microservice then compresses the data obtained from its own real-time database and returns it to the requesting client. The client caches the obtained data in computer memory and displays it in a table.
[0004] Because data is cached in memory, real-time display and filtering operations are smooth. However, as the system runs for a long time, massive amounts of model data are generated in the power system. The number of records in a single table may grow to millions or even tens of millions. If this data is also cached in computer memory, the data refresh speed of a single workstation node will be reduced due to insufficient memory, making it impossible to view model data normally, and even seriously affecting the operation of other components of the node.
[0005] Therefore, a more efficient and intelligent way is needed to process these power grid model data with a scale of more than one million. Summary of the Invention
[0006] Objective: To overcome the deficiencies in the prior art, the present invention provides a paging retrieval method for power grid model data, which can not only improve the display speed of model data when acquiring ultra-large-scale data, but also control the memory usage within a very small range.
[0007] Technical solution: To solve the above technical problems, the technical solution adopted by the present invention is:
[0008] A paging retrieval method for power grid model data includes the following steps:
[0009] Step 1: When a user needs to open a table on the client of a workstation, the client generates a table parameter request containing the table number and its category, and sends the table parameter request to the microservice on the server through the service bus.
[0010] Step 2: After receiving the table parameter request from the workstation client, the server-side microservice obtains the total number of records in the table from the real-time database based on the category and table number in the request, packages it into a table parameter response data, and sends it back to the workstation client.
[0011] Step 3: After receiving the table parameter response data, the client reads the local power grid model configuration file to obtain the number of first-screen records and the number of non-first-screen records. Based on the total number of records in the table parameter response data, the client uses a subpackaging algorithm to generate the difference between the end row number and the start row number in the last table data request (Num). last The number of table data requests in the list is Count, based on the difference Num last and the number Count to obtain a list consisting of multiple table data requests.
[0012] Step 4: The client starts multiple child threads to send each request in the table data request list to the server, and receives the table data response from the server in each child thread.
[0013] Step 5: After receiving the table data response from the server, the client stores the power grid model data contained therein into different temporary files, and simultaneously creates a "position-file" mapping container object and a "position-lock" mapping container object on the heap.
[0014] Step 6: When the client retrieves the power grid model data, it obtains the corresponding temporary file path from the "position-file" mapping container object according to the row number of the data currently to be retrieved, and obtains the write lock of the row from the "position-lock" mapping container object. It then performs a locking operation and loads the temporary file content corresponding to the row number into the memory. After loading is complete, it unlocks the write lock of the row.
[0015] Step 7: As the corresponding temporary file contents are loaded into memory, the client begins displaying the data for the row number. It obtains a read lock for the row from the "position-lock" mapping container object and performs a lock operation. If the lock fails, the current power grid model data will display "Writing"; if the lock succeeds, the actual power grid model data value will be displayed.
[0016] As a preferred solution, each table data request includes location information of the power grid model data, that is, the starting row number and the ending row number.
[0017] As a preferred solution, the specific steps of the subcontracting algorithm are as follows:
[0018] Calculate the difference between the end row number and the start row number in the last table data request in the list based on the number of first screen records, the number of non-first screen records, and the total number of records. last , the calculation formula is as follows:
[0019]
[0020] Calculate the number of table data requests in the list based on the number of first-screen records, the number of non-first-screen records, and the total number of records. The calculation formula is as follows:
[0021]
[0022] Among them, Num first Is the number of records on the first screen, Num other The maximum number of records outside the first screen, Num all is the total number of records, % represents the modulo function, Represents the floor function.
[0023] As a preferred solution, the table data response includes the end row number, start row number and value of the power grid model data.
[0024] As a preferred solution, the temporary file name format includes the category, table number, starting row number and ending row number.
[0025] As a preferred solution, read lock operations will not affect write lock operations, but write lock operations will affect read lock operations.
[0026] As a preferred solution, a write lock operation indicates that the data in the table cannot be updated or edited, and a write lock unlock operation indicates that the data in the table can be updated or edited.
[0027] As a preferred solution, when the client displays the power grid model data value, the table appearance of each row of records is set to a uniform row height.
[0028] Beneficial Effects: The present invention provides a paging retrieval method for power grid model data. On the one hand, because the client interface only processes the model data required for display, uses read-write locks to synchronize threads, and maintains a uniform display size for model data, it can significantly improve the display speed of power grid model data, reduce the time required for operation and maintenance personnel to view model data, and improve the efficiency of the entire system. On the other hand, by storing data on disk in the form of small segmented files, the memory space occupied by the client can be greatly reduced without reducing the delay in obtaining model data, thereby ensuring the safe operation of other components under the node. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] Figure 1 This is an architecture diagram of a client obtaining power grid model data provided by an embodiment of the present invention.
[0030] Figure 2 This is the mapping relationship between the read-write lock and model data, and the temporary file and model data provided by the embodiment of the present invention. DETAILED DESCRIPTION
[0031] The present invention will be further described below with reference to specific embodiments.
[0032] A paging retrieval method for power grid model data includes the following steps:
[0033] Step 1: When a user needs to open a table on the client of a workstation, the client generates a table parameter request containing the table number and its category, and sends the table parameter request to the microservice on the server through the service bus.
[0034] Step 2: After receiving the table parameter request from the workstation client, the server-side microservice obtains the total number of records in the table from the real-time database based on the category and table number in the request, packages it into a table parameter response data, and sends it back to the workstation client.
[0035] Step 3: After receiving the table parameter response data, the client reads the local power grid model configuration file to obtain the number of first-screen records and the number of non-first-screen records. Based on the total number of records in the table parameter response data, the client uses a subpackaging algorithm to generate the difference between the end row number and the start row number in the last table data request (Num). last The number of table data requests in the list is Count, based on the difference Num last and the number Count to obtain a list consisting of multiple table data requests.
[0036] Each table data request includes the location information of the power grid model data, that is, the starting row number and the ending row number.
[0037] Among them, the specific method of the subcontracting algorithm is:
[0038] Calculate the difference between the end row number and the start row number in the last table data request in the list based on the number of first screen records, the number of non-first screen records, and the total number of records. last , the calculation formula is as follows:
[0039]
[0040] Calculate the number of table data requests in the list based on the number of first-screen records, the number of non-first-screen records, and the total number of records. The calculation formula is as follows:
[0041]
[0042] Among them, Num first Is the number of records on the first screen, Num other The maximum number of records outside the first screen, Num all is the total number of records, % represents the modulo function, Represents the floor function.
[0043] Step 4: The client starts multiple child threads to send each request in the table data request list to the server, and receives the table data response from the server in each child thread.
[0044] The table data response includes the end row number, start row number, and value of the power grid model data.
[0045] Step 5: After receiving the table data response from the server, the client stores the power grid model data contained therein into different temporary files, and simultaneously creates a "position-file" mapping container object and a "position-lock" mapping container object on the heap.
[0046] The temporary file name format is fixed, including the category, table number, starting row number, and ending row number.
[0047] Step 6: When the client retrieves the power grid model data, it obtains the corresponding temporary file path from the "position-file" mapping container object according to the row number of the data currently to be retrieved, and obtains the write lock of the row from the "position-lock" mapping container object. It then performs a locking operation and loads the temporary file content corresponding to the row number into the memory. After loading is complete, it unlocks the write lock of the row.
[0048] Step 7: As the corresponding temporary file contents are loaded into memory, the client begins displaying the data for the row number. It obtains a read lock for the row from the "position-lock" mapping container object and performs a lock operation. If the lock fails, the current power grid model data will display "Writing"; if the lock succeeds, the actual power grid model data value will be displayed.
[0049] When the client displays the power grid model data value, the table appearance of each row of records is set to a uniform row height.
[0050] Example:
[0051] like Figure 1 As shown, the present invention provides a paging retrieval method for power grid model data, comprising the following steps:
[0052] Step 1. The user opens table 112 under the scada category on the client of the workstation. The client generates a RequestTablePara table parameter request and sends the request to the microservice on the server through the service bus.
[0053] The data structure of the table parameter request is:
[0054]
[0055] scn is the category to which the table belongs, and tableno is the table number.
[0056] Step 2. After the server-side microservice receives the table parameter request from the workstation client, it obtains a total of 10,000 records from the real-time database based on the category and table number in the request, packages them into a table parameter response data, and sends it back to the workstation client.
[0057] Step 3. After receiving the table parameter response data, the client reads the local power grid model configuration file to obtain the number of first screen records (100) and the number of non-first screen records (5000). Based on the total number of records (10000) in the table parameter response data, the client uses a subpacketization algorithm to obtain the difference between the end row number and the start row number in the last table data request (Num). last The value is 4900 and the number Count is 3.
[0058] Request Num based on the last table data of the difference last The value is 4900, and the number of the last table data request is 3. A list consisting of three table data requests is generated. The starting and ending row numbers of each table data request are:
[0059] The starting line number is 0 and the ending line number is 99. The number of records on the first screen is 100.
[0060] The starting row number is 100 and the ending row number is 5099. The number of records outside the first screen is 5000.
[0061] The starting row number is 5100 and the ending row number is 10000. The difference between the ending row number and the starting row number of the last table data request is 4900.
[0062] Step 4. The client starts three sub-threads and sends three table data requests to the server respectively. In each sub-thread, the client receives a table data response from the server, including the end row number, start row number, and value of the three power grid model data.
[0063] Step 5. After receiving the table data response from the server, the client stores the power grid model data contained therein into three temporary files, and simultaneously creates a "position-file" mapping container object map1 and a "position-lock" mapping container object map2 on the heap.
[0064] The three temporary file names are:
[0065] "scada_112_0_99.data";
[0066] "scada_112_100_5099.data";
[0067] "scada_112_5100_10000.data";
[0068] Step 6. Assume that the client needs to retrieve the data on row 10. It obtains the temporary file path "scada_112_0_99.data" from the "position-file" mapping container object map1. At the same time, based on the row number 10, it obtains the write lock of the current row from the "position-lock" mapping container object map2, locks it, and then loads the file contents into memory. After loading, it unlocks the write lock.
[0069] Step 7. While the file contents are being loaded into memory, the client interface will synchronously display the data on row 10. Based on the row number 10, the client obtains a read lock for the current row from the "position-lock" mapping container object map2 and applies the lock. If the data on row 10 is currently being updated and the lock fails, the current power grid model data will display "Writing." If the data on row 10 has already been updated and the lock succeeds, the actual power grid model data value will be displayed.
[0070] The present invention stores the power grid model data in different temporary files and loads the contents of the corresponding temporary files into the memory according to the retrieval needs, which can effectively reduce the client memory occupancy rate and ensure the safe operation of other components under the node; at the same time, the required data is read and written with a lock operation when retrieving data, which significantly improves the display speed of the power grid model data, shortens the time for operation and maintenance personnel to view the model data, and improves the efficiency of the entire system.
[0071] The working principle related to the beneficial effects of the present invention is further explained as follows:
[0072] 1. Reducing memory usage: This method distributes 10,000 data items into multiple files. When searching for a row of data, the corresponding file content is loaded into memory. Existing methods, on the other hand, do not store data in multiple files and require all 10,000 data items to be loaded into memory. Loading all 10 million data items into memory would be very memory-intensive and would also affect the memory usage of other components.
[0073] 2. Regarding improving display speed: This invention uses a "read-write lock" operation. A "read lock" operation is performed when reading data, and a "write lock" operation is performed when updating or editing data. The "read lock" operation does not affect the "write lock" operation, while the "write lock" operation affects the "read lock" operation.
[0074] The "read-write lock" operation is performed simultaneously when loading into memory and displaying data. For example, if the user is viewing the data on row 10 and clicks the "Refresh" button, the program will do the following: lock the "write lock" -> re-acquire the data -> save to a temporary file -> load the temporary file into memory -> unlock the "write lock". The data is displayed while loading into memory. The "read lock" is used to display the data. If the area where the row of data is located is being updated (for example, rows 0 to 99 are being updated), the "read lock" operation will fail. At this time, the interface will display "Writing" to remind the user of the status. Conversely, the "read lock" operation will succeed, indicating that the current data has been loaded and is up to date. At this time, the actual value can be displayed.
[0075] The existing method does not have a "read-write lock" and will not refresh the latest value of the current interface until all data is loaded. If the amount of data is large, the interface will be stuck during the refresh. When the interface is stuck, the user will not know what is happening and may think that the program is abnormal. However, the present invention prompts "writing" during the update to indicate the data refresh status.
[0076] The present invention can not only improve the display speed of power grid model data when acquiring ultra-large-scale data, but also control the memory usage within a very small range, thereby achieving maximum safety benefits.
[0077] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. A paging retrieval method for power grid model data, characterized by: The steps include: Step 1: When a user needs to open a table on the client side of a workstation, the client generates a table parameter request containing the table number and its category, and sends the table parameter request to the microservice on the server side through the service bus; Step 2: After receiving the table parameter request from the workstation client, the server-side microservice obtains the total number of records in the table from the real-time database based on the category and table number in the request, packages it into a table parameter response data, and sends it back to the workstation client; Step 3: After receiving the table parameter response data, the client reads the local power grid model configuration file to obtain the number of first-screen records and the number of non-first-screen records. Based on the total number of records in the table parameter response data, the client uses a subpackaging algorithm to generate the difference between the end row number and the start row number in the last table data request. and the number of table data requests in the list , according to the difference and number Get a list consisting of multiple table data requests; Step 4: The client starts multiple child threads to send each request in the table data request list to the server, and receives the table data response from the server in each child thread; Step 5: After receiving the table data response from the server, the client stores the grid model data contained therein into different temporary files and simultaneously creates a "position-file" mapping container object and a "position-lock" mapping container object on the heap; Step 6: When the client retrieves the power grid model data, it obtains the corresponding temporary file path from the "Location-File" mapping container object based on the row number of the data currently being retrieved. It also obtains the write lock for the row from the "Location-Lock" mapping container object, locks the file, and loads the temporary file content corresponding to the row number into memory. After loading, it unlocks the write lock for the row. Step 7: As the corresponding temporary file content is loaded into memory, the client begins displaying the data at the corresponding row number, obtains a read lock for the row from the "position-lock" mapping container object, and performs a locking operation. If the lock fails, the current power grid model data is displayed as "writing". If the lock is successful, the actual grid model data value will be displayed.
2. The paging retrieval method for power grid model data according to claim 1, characterized in that: Each table data request includes the location information of the power grid model data, that is, the starting row number and the ending row number.
3. The paging retrieval method for power grid model data according to claim 1, characterized in that: The specific steps of the subpackaging algorithm are as follows: Calculate the difference between the ending row number and the starting row number in the last table data request in the list based on the number of first-screen records, the number of non-first-screen records, and the total number of records. , the calculation formula is as follows: Calculate the number of table data requests in the list based on the number of first-screen records, the number of non-first-screen records, and the total number of records , the calculation formula is as follows: in, is the number of records on the first screen, is the maximum number of records off the first screen, is the total number of records, represents the modulo function, Represents the floor function.
4. The paging retrieval method for power grid model data according to claim 1, characterized in that: The table data response contains the end row number, start row number, and value of the power grid model data.
5. The paging retrieval method for power grid model data according to claim 1, characterized in that: The temporary file name format includes the category, table number, starting row number, and ending row number.
6. The paging retrieval method for power grid model data according to claim 1, characterized in that: Read lock operations do not affect write lock operations, but write lock operations do affect read lock operations.
7. The paging retrieval method for power grid model data according to claim 1, characterized in that: A write lock operation indicates that the data in the table cannot be updated or edited, and a write lock unlock operation indicates that the data in the table can be updated or edited.
8. The paging retrieval method for power grid model data according to claim 1, characterized in that: When the client displays the power grid model data values, the table appearance of each row of records is set to a uniform row height.
Citation Information
Patent Citations
Method for browsing decoded ticket file
CN103699699A
Decentralized database optimizations
US20190288850A1