Cache optimization method and device of power data system, computer equipment, readable storage medium and program product

Abstract

The invention relates to the technical field of computer storage systems, and provides a cache optimization method and device for a power data system, computer equipment, a readable storage medium and a program product. The method comprises the following steps: determining target cache data in a fixed period access scene according to a historical access rule of a power data system; according to a historical access rule of the power data system, predicting the current cache time of the target cache data; according to the current cache time of the target cache data, prolonging the retention period of the target cache data in the cache region; and when a data generation request related to target cache data is received, reading the target cache data from the cache region, and generating data according to the target cache data. By adopting the method, the loading time of the target cache data can be shortened, and the data processing efficiency of the power data system can be remarkably improved.

Description

Technical Field

[0001] The present application relates to the technical field of computer storage systems, and in particular to a cache optimization method, apparatus, computer equipment, computer-readable storage medium, and computer program product for a power data system. Background Art

[0002] As the number of users increases year by year, the amount of data processed by the power data system increases, and the performance requirements of the power data system are also getting higher and higher. In order to improve the performance of the power data system, the cache method of the power data system can be optimized for fixed-cycle access scenarios.

[0003] Traditional methods use Elasticsearch (ES) and recursive dictionaries (Redis) to cache user data, but this results in low cache hit rates and severe resource waste, leading to low data processing efficiency and heavy database load in power data systems. Summary of the Invention

[0004] Based on this, it is necessary to provide a cache optimization method, device, computer equipment, computer-readable storage medium and computer program product for a power data system to address the above technical problems.

[0005] In a first aspect, the present application provides a cache optimization method for a power data system, comprising:

[0006] Determine the target cache data in a fixed-cycle access scenario based on the historical access patterns of the power data system;

[0007] Predicting a current cache time of the target cache data based on a historical access pattern of the power data system;

[0008] Extending the retention period of the target cache data in the cache area according to the current cache time of the target cache data;

[0009] When a data generation request related to the target cache data is received, the target cache data is read from the cache area, and data is generated according to the target cache data.

[0010] In one embodiment, the method further comprises:

[0011] The power data is written into the storage layer of the power data system using a sequential write operation.

[0012] In one embodiment, the method further comprises:

[0013] Each time before writing power data to the storage layer of the power data system, a pre-write log is first recorded on the disk; the pre-write log includes a record of power data changes.

[0014] In one embodiment, the method further comprises:

[0015] The power data is sharded according to the hash value of the user identifier to store the power data in different nodes of the power data system.

[0016] In one embodiment, the method further comprises:

[0017] Monitor the node load of the power data system in real time and route requests to idle nodes first.

[0018] In one embodiment, the method further comprises:

[0019] When a new node is added to the power data system, some slot data is migrated to the new node through the migration tool.

[0020] In a second aspect, the present application further provides a cache optimization device for a power data system, comprising:

[0021] A target cache data determination module is used to determine the target cache data in a fixed-period access scenario based on the historical access pattern of the power data system;

[0022] a current cache time prediction module, configured to predict the current cache time of the target cache data based on a historical access pattern of the power data system;

[0023] A retention period extension module, configured to extend the retention period of the target cache data in the cache area according to the current cache time of the target cache data;

[0024] The data generation module is configured to read the target cache data from the cache area and generate data according to the target cache data when receiving a data generation request related to the target cache data.

[0025] In a third aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program and the processor executes the above method.

[0026] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is used by a processor to execute the above method.

[0027] In a fifth aspect, the present application further provides a computer program product, wherein the computer program product includes a computer program, and the computer program is executed by a processor to execute the above method.

[0028] The cache optimization method, device, computer equipment, computer-readable storage medium and computer program product of the above-mentioned electric power data system determine the target cache data in a fixed-period access scenario based on the historical access pattern of the electric power data system; predict the current cache time of the target cache data based on the historical access pattern of the electric power data system; extend the retention period of the target cache data in the cache area based on the current cache time of the target cache data; when a data generation request related to the target cache data is received, read the target cache data from the cache area, and generate data based on the target cache data. The present application predicts the current cache time of the target cache data based on the historical access pattern of the electric power data system to extend the retention period of the target cache data in the cache area. Even if the target cache data is not accessed in the intermediate time and is not eliminated in the cache area, when generating data related to the target cache data, the target cache data can be directly read from the cache area, reducing the loading time of the target cache data, which can significantly improve the data processing efficiency of the electric power data system and reduce the database load of the electric power data system. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments of the present application or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying any creative work.

[0030] Figure 1 A diagram illustrating an application environment of a cache optimization method for a power data system according to an embodiment;

[0031] Figure 2 1 is a flow chart of a cache optimization method for a power data system according to an embodiment;

[0032] Figure 3 A schematic diagram of a database structure of a power data system in one embodiment;

[0033] Figure 4 A structural block diagram of a cache optimization device for a power data system according to an embodiment;

[0034] Figure 5 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0036] The cache optimization method of the power data system provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. The terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The terminal 102 can extend the retention period of the target cache data in the cache area according to the historical access rules of the power data system, and when a data generation request related to the target cache data is received, the target cache data is read from the cache area, and data is generated according to the target cache data. The terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablets, Internet of Things devices and portable wearable devices. The server 104 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services.

[0037] In an exemplary embodiment, Figure 2 As shown, a cache optimization method for a power data system is provided, and the method is applied to Figure 1 The terminal 102 in the example is used as an example to illustrate the process, including the following steps S201 to S204.

[0038] Step S201 : determining target cache data in a fixed-period access scenario based on historical access patterns of the power data system.

[0039] Fixed-period access scenarios may include billing report tasks, scheduled batch tasks, and other business scenarios with strong periodicity and high single-access costs.

[0040] The historical access patterns of users to the power data in the power data system can be obtained from the access log of the power data system, where the power data system can be a power marketing system.

[0041] According to the historical access pattern of users to the power data in the power data system, the power data in a fixed period access scenario may be determined as the target cache data.

[0042] Step S202 : predicting the current cache time of the target cache data based on the historical access pattern of the power data system.

[0043] The periodicity of access to the target cache data can be obtained based on the historical access pattern of the power data system; and the current cache time of the target cache data can be predicted based on the periodicity of access to the target cache data.

[0044] Step S203 : extending the retention period of the target cache data in the cache area according to the current cache time of the target cache data.

[0045] The extension time of the target cache data in the cache area can be determined based on the current cache time of the target cache data, so that the retention period of the target cache data in the cache area can be extended according to the extension time, thereby ensuring that when a data generation request related to the target cache data is received, the target cache data is not deleted in the cache area and the target cache data can be directly read from the cache area.

[0046] Step S204 : when a data generation request related to the target cache data is received, the target cache data is read from the cache area, and data is generated according to the target cache data.

[0047] When a data generation request related to target cache data is received, the target cache data can be directly read from the cache area, reducing the loading time of the target cache data; and data can be generated based on the target cache data. This can significantly improve the data processing efficiency of the power data system while reducing the database load of the power data system. In the cache optimization method of the above-mentioned power data system, a periodic enhancement strategy is adopted to predict the current cache time of the target cache data based on the historical access rules of the power data system to extend the retention period of the target cache data in the cache area. Even if the target cache data is not accessed in the intermediate time and is not eliminated in the cache area, when generating data related to the target cache data, the target cache data can be directly read from the cache area, reducing the loading time of the target cache data, which can significantly improve the data processing efficiency of the power data system while reducing the database load of the power data system.

[0048] In one embodiment, the method provided in the present application further includes: writing the power data into a storage layer of the power data system using a sequential write operation.

[0049] The power data system can be constructed based on the Pika database, where the database structure of the power data system is as follows: Figure 3 As shown in the figure, the Pika database uses the Log-Structured Merge-Tree (LSM-Tree) engine of the RocksDB disk storage engine at the bottom layer. This engine converts random write operations into sequential write operations, also known as append-only log operations. Write performance is unaffected by data volume. Therefore, sequential write operations can be used to write power data to the storage layer of the power data system.

[0050] For example, user points changes (INCRBY commands) are written directly to the power data system's storage layer's in-memory table (‌MemTable). Once a threshold is reached, data is flushed to disk in batches (ordered string table files), avoiding the append-only file (AOF) rewrite bottleneck in the Remote Dictionary Server (Redis) system. The full name of the ordered string table is Sorted String Table, or SSTable for short.

[0051] In this embodiment, a sequential write operation is adopted to write the power data into the storage layer of the power data system, and the write performance is not affected by the data volume.

[0052] In one embodiment, the method provided by the present application further includes: recording a write-ahead log to a disk before writing power data to a storage layer of a power data system each time; the write-ahead log includes a record of power data changes.

[0053] Before each power data is written to the storage layer of the power data system, a write-ahead log (WAL) is first recorded on disk. Even in the event of a system outage, the power data can be restored through the WAL. This ensures the atomicity and durability of integration operations, and guarantees the data security and persistence of the power data system. The WAL includes a record of power data changes.

[0054] In one embodiment, the method provided by the present application further includes: sharding the power data according to the hash value of the user identifier to store the power data in different nodes of the power data system.

[0055] The Combined DNA Index System (Codis) or a self-developed proxy layer can be used to shard power data according to the hash value of the user identifier (Identity Document, ID) (e.g., 1024 virtual slots), with each shard bound to a physical node in the power data system.

[0056] For example, if user A’s identifier hashes to slot 200, user A’s points data is always processed by node 2, avoiding the migration overhead of the remote dictionary service system cluster (Redis Cluster).

[0057] The above-mentioned power data can be sharded according to the hash value of the user identifier to store the power data in different nodes of the power data system, which is called a consistent hash sharding strategy.

[0058] In this embodiment, the power data is sharded according to the hash value of the user identifier so that the power data is stored in different nodes of the power data system. This can avoid migration overhead, significantly improve the data processing efficiency of the power data system, and reduce the database load of the power data system.

[0059] In one embodiment, the method provided by the present application further includes: monitoring the node load of the power data system in real time, and preferentially routing requests to idle nodes.

[0060] It can monitor node loads in the power data system in real time and automatically route requests to idle nodes. If a node fails (e.g., a disk is damaged), the replica will immediately take over, ensuring the continuity of the power data system service.

[0061] In this embodiment, the node load of the power data system is monitored in real time, and requests are preferentially routed to idle nodes, which can ensure the continuity of the power data system service, significantly improve the data processing efficiency of the power data system, and reduce the database load of the power data system.

[0062] In one embodiment, the method provided by the present application further includes: when a new node is added to the power data system, migrating part of the slot data to the new node through a migration tool.

[0063] When a new node is added to the power data system, some slot data can be migrated to the new node using a migration tool (such as the Pika migration tool, pika-migrate). This process supports hot migration (without user awareness).

[0064] For example, if the power data system is expanded from 3 nodes to 6 nodes, the throughput of the power data system can be linearly increased by nearly 2 times.

[0065] In this embodiment, when a new node is added to the power data system, part of the slot data is migrated to the new node through the migration tool, which can improve the throughput of the power data system, significantly improve the data processing efficiency of the power data system, and reduce the database load of the power data system.

[0066] In order to better understand the above method, an application embodiment of the cache optimization method of the power data system of the present application is described in detail below.

[0067] As the number of users increases annually, the amount of data processed increases, placing increasing demands on the performance of power data systems. In power data systems, such as marketing and customer service systems, to improve retrieval of massive user data volumes at scales exceeding 120 million, a least recently used (LRU) page replacement algorithm is used, combined with the time characteristics of user access to the system for electricity bill payments, to optimize the cache access strategy for user data and index data. This improves the hit rate of valid cached data, significantly enhances the performance of retrieving customer profile information, and significantly improves the performance of users checking and paying electricity bills using the power marketing system and applications (apps).

[0068] Traditional methods utilize Elasticsearch (ES) and recursive dictionaries (Redis) to cache user data. Without a built-in, efficient least recently used (LRU) algorithm like a recursive dictionary, caching systems face challenges such as implementation difficulties, poor performance, and difficulty adapting to dynamically changing workloads. These issues not only limit the efficiency of power data systems but also pose a threat to security and reliability. Therefore, developing a more efficient and easily integrated LRU optimization algorithm to improve the caching mechanism for data management in power data systems has significant practical significance and technical value.

[0069] In power data systems, such as marketing and customer service systems, as service operations continue to unfold, the existing technical architecture of the points center is unable to support large-scale user participation. This often results in long response times, service interruptions, and unavailability during activities, leading to a poor user experience and even user complaints, seriously impacting the business operations of China Southern Power Grid's digital channels. At the same time, some users seek improper benefits through illegal means, making real-time user behavior monitoring and anti-fraud measures urgent. However, real-time user behavior monitoring and anti-fraud measures are complex processes that require a highly sensitive, accurate, and efficient system. This process involves extensive data processing and analysis, and therefore presents a variety of system performance issues.

[0070] In order to effectively avoid the occurrence of poor user experience, low satisfaction, and impact on activity operation results caused by performance issues, it is necessary to solve the performance bottleneck of the points center application under the premise of fully considering Internet channels, Internet technologies and business development trends.

[0071] To address these issues, further optimization of the digital service technology architecture can be achieved to ensure high stability, high availability, and high performance of the platform in high-concurrency scenarios, and to support the continuous and uninterrupted operation of platform services with hundreds of millions of users. Hotspot services can be separated to allocate and optimize resources specifically for high-traffic, high-concurrency services, ensuring that these services receive sufficient resource support during peak hours. This effectively reduces the system resource usage of non-hotspot services, lowering the overall system load and improving system stability and reliability.

[0072] The following is a detailed description of the purpose of the core threads in the Pika multi-threaded model:

[0073] (1) Main thread (PikaServer): responsible for global coordination, managing the life cycle of other threads, and handling system-level tasks such as startup / shutdown.

[0074] (2) Dispatch Thread: listens to the specified port, receives client connection requests, and assigns new connections to idle client worker threads for processing.

[0075] (3) Client Worker: Multiple threads, each of which manages multiple client connections, parses and executes commands, processes read and write operations, and appends write operations to the binary log (binlog).

[0076] (4) Trysync: Responsible for establishing the initial synchronization connection with the master node (master). If the connection is disconnected, it will automatically try to reconnect to ensure that the replication link is restored.

[0077] (5) ReplicaSender: Dynamically created, one for each slave node, which sends incremental data from the specified position in the binary log according to the slave node's request.

[0078] (6) Replica Receiver: A single thread that receives synchronization data from the master node, writes commands to the local binary log, and distributes them to the binary log background worker thread for execution by hash.

[0079] (7) Binary log background worker thread (BinlogBGWorker): Multiple background threads that execute commands assigned by replica receivers to ensure concurrent processing of replicated data.

[0080] (8) Slave server heartbeat detection (SlavePing): The slave node regularly sends heartbeat packets to the master node to detect whether the master node is alive and maintain the activity of the replication connection.

[0081] (9) Heartbeat: The master node listens to the heartbeat requests of all slave nodes, responds to the survival status, detects whether the slave node is online, and maintains the connection list.

[0082] (10) Background Save (bgsave): Executes background data snapshots (such as generating persistent files, RDB files) to avoid blocking the main thread. It is used for persistence or full replication.

[0083] (11) Scan: Scans the key space in the background and performs periodic tasks such as deleting expired keys, collecting statistics, or migrating data.

[0084] (12) Purge: Regularly purge expired binary log files to free up disk space and maintain storage efficiency.

[0085] In the service operation points activities of the marketing customer service system, when the amount of points user data is too large, traditional in-memory databases (such as the recursive dictionary Redis) will lead to cost surges, poor scalability (single-node memory bottleneck), low persistence efficiency (time-consuming snapshots / recovery), and sudden performance drops under high concurrency (insufficient memory triggering elimination strategies or memory overflow (Out Of Memory, OOM)) due to memory capacity limitations. Pika uses the disk storage engine (RocksDB) to achieve low-cost storage of massive data, uses distributed sharding to horizontally expand capacity and throughput, combines asynchronous multi-threading and memory caching to accelerate performance and stability, and is compatible with the recursive dictionary (Redis) protocol for seamless migration, thus efficiently solving the problems of high-concurrency writing, long-term storage and elastic expansion in large-capacity points scenarios.

[0086] The following is a solution to solve the problem of excessive points user data through Pika database:

[0087] (1) Using a Log-Structured Merge-Tree (LSM-Tree) data structure: The Pika database uses the Log-Structured Merge-Tree engine of the disk storage engine (RocksDB) at the bottom layer, converting random writes into sequential append (Append-Only Log) operations. Write performance is not affected by the amount of data. For example, user points changes (auto-increment command, INCRBY) operations are directly written to the memory table (‌MemTable) in the storage layer of the power data system. After reaching the threshold, the data is flushed to disk in batches (ordered string table files), avoiding the append-only file (AOF) rewrite bottleneck of the remote dictionary server (Redis) system. The full name of the ordered string table is SortedString Table, abbreviated as SSTable in English.

[0088] (2) Hierarchical storage and compression: Ordered string table (SSTable) files are organized hierarchically (Level 0 to N). A background merge (compaction) thread merges redundant data and compresses it (supporting the Snappy / Zstandard (ZSTD) algorithms), achieving a compression ratio of 5:1. For example, after compression, the disk usage of the user points log (list structure) is reduced by 80%, while maintaining efficient query efficiency.

[0089] (3) Write-ahead logging (WAL) is used to ensure persistence: each write is first recorded in the write-ahead log to the disk. Even if there is a crash, the power data can be restored through the write-ahead log, which can ensure the atomicity and persistence of the integration operation.

[0090] (4) Using a consistent hashing sharding strategy: Pika uses the Combined DNA Index System (Codis) or a self-developed proxy layer to shard data by the hash value of the user's identity document (ID) (e.g., 1024 virtual slots). Each shard is bound to a physical node in the power data system. For example, if user A's identifier is hashed to slot 200, user A's points data will always be processed by node 2, avoiding the migration overhead of the remote dictionary service system cluster (Redis Cluster).

[0091] (5) Dynamic scaling: When adding a new node, some slot data is migrated to the new node using a migration tool (such as the Pika migration tool, pika-migrate). This process supports hot migration (without user awareness). For example, if the power data system is expanded from 3 nodes to 6 nodes, the throughput of the power data system can be linearly increased by nearly 2 times.

[0092] (6) Load balancing and fault isolation: The proxy layer monitors node load in real time and automatically routes requests to idle nodes. If a node fails (e.g., a disk is damaged), the replica immediately takes over to ensure the continuity of the points service.

[0093] (7) Multi-threaded concurrency model: Pika uses a multi-threaded division of labor (such as input / output threads, worker threads, and background threads). Input / output is called Input / Output in English, and is abbreviated as IO in English. After the main thread receives a request, it distributes it to the worker thread for processing through a lock-free queue. For example, a 16-core server can be configured with 24 worker threads, fully utilizing the central processing unit (CPU) resources, and the number of queries per second (QPS) can reach over 100,000.

[0094] (8) Adopt asynchronous design:

[0095] Asynchronous disk flushing: The disk storage engine (RocksDB) memory table (‌MemTable) flushing and merging operations are performed in the background thread and do not block foreground requests.

[0096] Asynchronous master-slave replication: The master node asynchronously synchronizes data to the slave node through the binary log (binlog). The master-slave node latency is controlled at the millisecond level, avoiding the performance loss of synchronous replication (such as Redis's wait command).

[0097] (9) In-memory acceleration: The disk storage engine (RocksDB) caches hot data (such as user points balances) in a block cache (configurable to GB level) and manages it using a least recently used policy. For example, when the cache hit rate is 90%, read performance approaches that of a pure in-memory database.

[0098] (10) Adopting transparent adaptation at the protocol layer: Pika implements a recursive dictionary (Redis) protocol parser at the network layer, supporting all mainstream data structure commands, such as the Get (GET) / Set (SET) commands for string types, the Hash Increment By (HINCRBY) command for hash types, and the Sorted Set Add (ZADD) command for ordered set types (Zset). For example, the deduction of electricity points can directly reuse the Redis hash increment command without modifying the business code.

[0099] (11) Using hybrid storage to support complex scenarios:

[0100] String type: stores the user's current points balance (high-frequency reading and writing).

[0101] List: stores the flow of points changes (append writes, merge operation optimization).

[0102] Ordered set type (Zset): implements a score ranking list (leveraging the ordered iteration characteristics of the disk storage engine).

[0103] (12) Use a smooth migration tool chain: Provide a port (pika-port) tool to listen to the Append Only File (AOF) log of the recursive dictionary (Redis) instance, synchronize data to the Pika cluster in real time, and the business will not be aware of the migration process.

[0104] (13) Use a hybrid storage system of Solid State Drive (SSD) and Hard Disk Drive (HDD): cold data (such as points flow from three months ago) is automatically downgraded to the HDD, while hot data is retained on the SSD. The storage cost is only 1 / 10 of that of the recursive dictionary (Redis).

[0105] (14) Adopting adaptive compression strategy:

[0106] Snappy: Default compression algorithm, balancing speed and compression ratio (compression time < 1ms / GB).

[0107] ZSTD: Optional high-level compression, compression ratio increased by 30% (suitable for historical data archiving).

[0108] (15) Adopt refined monitoring: Prometheus is used to collect indicators such as shard load, cache hit rate, and merge operation pressure, and combined with Grafana dashboards to provide real-time warnings. For example, when the merge operation thread continues to be under high load, it will automatically trigger expansion.

[0109] The technical solution provided in this embodiment adopts a periodic enhancement strategy to predict the current cache time of the target cache data based on the historical access patterns of the power data system, so as to extend the retention period of the target cache data in the cache area. Even if the target cache data is not accessed in the intermediate time and is not eliminated in the cache area, when the data related to the target cache data is generated, the target cache data can be read directly from the cache area, reducing the loading time of the target cache data, which can significantly improve the data processing efficiency of the power data system and reduce the database load of the power data system. In addition, the Pika database used by the power data system uses a disk storage engine (RocksDB) to achieve low-cost storage of massive data, uses distributed sharding to horizontally expand capacity and throughput, combines asynchronous multi-threading and memory cache acceleration to balance performance and stability, and is compatible with the recursive dictionary (Redis) protocol for seamless migration, thereby efficiently solving the problems of high-concurrency writing, long-term storage and elastic expansion in large-capacity integration scenarios.

[0110] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0111] Based on the same inventive concept, embodiments of the present application also provide a cache optimization device for a power data system, which is used to implement the cache optimization method for a power data system. The solution provided by this device is similar to the solution described in the method described above. Therefore, the specific limitations of one or more embodiments of the cache optimization device for a power data system provided below can be found in the limitations of the cache optimization method for a power data system described above and will not be further elaborated here.

[0112] In an exemplary embodiment, Figure 4 As shown, a cache optimization device for a power data system is provided, wherein:

[0113] The target cache data determination module 401 is used to determine the target cache data in a fixed period access scenario according to the historical access pattern of the power data system;

[0114] The current cache time prediction module 402 is used to predict the current cache time of the target cache data based on the historical access pattern of the power data system;

[0115] A retention period extension module 403 is configured to extend the retention period of the target cache data in the cache area according to the current cache time of the target cache data;

[0116] The data generation module 404 is configured to read the target cache data from the cache area upon receiving a data generation request related to the target cache data, and generate data according to the target cache data.

[0117] In one embodiment, the device further includes a data writing module, configured to write the power data into a storage layer of the power data system using a sequential write operation.

[0118] In one embodiment, the device further includes a log recording module configured to record a write-ahead log to a disk before writing power data to a storage layer of the power data system each time; the write-ahead log includes a record of power data changes.

[0119] In one embodiment, the device further includes a data sharding module, configured to shard the power data according to hash values of user identifiers, so as to store the power data in different nodes of the power data system.

[0120] In one embodiment, the device further includes a node allocation module, configured to monitor node loads of the power data system in real time and preferentially route requests to idle nodes.

[0121] In one embodiment, the device further includes a data migration module, which is used to: when a new node is added to the power data system, migrate part of the slot data to the new node through a migration tool.

[0122] Each module in the cache optimization device for the power data system described above can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0123] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 5As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store data of an embodiment of the cache optimization method of the power data system. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a cache optimization method of the power data system is implemented.

[0124] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0125] In one embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0126] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0127] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0128] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0129] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), quantum computing-based data processing logic devices, artificial intelligence (AI) processors, and the like.

[0130] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0131] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A cache optimization method for a power data system, characterized in that: The method comprises: Determine the target cache data in a fixed-cycle access scenario based on the historical access patterns of the power data system; Predicting a current cache time of the target cache data based on a historical access pattern of the power data system; Extending the retention period of the target cache data in the cache area according to the current cache time of the target cache data; When a data generation request related to the target cache data is received, the target cache data is read from the cache area, and data is generated according to the target cache data.

2. The method according to claim 1, characterized in that The method further comprises: The power data is written into the storage layer of the power data system using a sequential write operation.

3. The method according to claim 1, characterized in that The method further comprises: Each time before writing power data to the storage layer of the power data system, a pre-write log is first recorded on the disk; the pre-write log includes a record of power data changes.

4. The method according to claim 1, wherein The method further comprises: The power data is sharded according to the hash value of the user identifier to store the power data in different nodes of the power data system.

5. The method according to claim 1, wherein The method further comprises: Monitor the node load of the power data system in real time and route requests to idle nodes first.

6. The method according to claim 1, characterized in that The method further comprises: When a new node is added to the power data system, some slot data is migrated to the new node through the migration tool.

7. A cache optimization device for a power data system, characterized in that: The device comprises: A target cache data determination module is used to determine the target cache data in a fixed-period access scenario based on the historical access pattern of the power data system; a current cache time prediction module, configured to predict the current cache time of the target cache data based on a historical access pattern of the power data system; A retention period extension module, configured to extend the retention period of the target cache data in the cache area according to the current cache time of the target cache data; The data generation module is configured to read the target cache data from the cache area and generate data according to the target cache data when receiving a data generation request related to the target cache data.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images