Photovoltaic grid-connected data storage method and device and electronic equipment

Abstract

The invention belongs to the technical field of data processing, and particularly relates to a photovoltaic grid-connected data storage method and device and electronic equipment, and the method comprises the steps: obtaining a plurality of groups of photovoltaic grid-connected data records describing the same data entity; for any group of photovoltaic grid-connected data records in the plurality of groups of photovoltaic grid-connected data records, performing key-value processing on the photovoltaic grid-connected data records, mapping the photovoltaic grid-connected data records into key-value pairs, matching keys of the key-value pairs with unique IDs of any group of photovoltaic grid-connected data records, and matching values of the key-value pairs with a photovoltaic grid-connected metadata set of any group of photovoltaic grid-connected data records; performing Hash operation on the key value pair to obtain a corresponding Hash value, and determining a photovoltaic grid-connected metadata storage task for storing any group of photovoltaic grid-connected data records in a target database based on the Hash value; and any group of photovoltaic grid-connected data is recorded and stored in the target database based on the data task, so that dynamic and semi-structured photovoltaic grid-connected metadata storage is realized.

Description

Technical Field

[0001] The present invention belongs to the field of data processing technology, and in particular to a photovoltaic grid-connected data storage method, device and electronic equipment. Background Art With the transformation of the global energy structure and the rapid development of renewable energy technology, photovoltaic energy has become an important part of energy supply. However, with the large-scale grid-connection of photovoltaic systems, it has brought many challenges to the safe and stable operation of the power grid. The power quality problems generated by photovoltaic grid-connected systems, such as voltage fluctuations and harmonic pollution, have a significant impact on the stability and power supply quality of the power grid. Therefore, improving the "observable, measurable, controllable, and adjustable" level of photovoltaic stations and enhancing photovoltaic safety management and control capabilities have become urgent issues to be solved in the current power industry.

[0002] In the research of photovoltaic grid-connected systems, intelligent edge modules play a key role. Intelligent edge modules can realize real-time collection, processing and storage of photovoltaic grid-connected data, providing strong guarantee for the safe and stable operation of the power grid.

[0003] However, the amount of photovoltaic grid-connected data generated by photovoltaic grid-connected systems is huge and complex, including various types of metadata such as voltage, current, power, etc. Existing photovoltaic grid-connected data storage methods often find it difficult to effectively manage these complex data, resulting in low data retrieval efficiency and failure to meet real-time requirements. Summary of the invention

[0004] The object of the present invention is to provide a photovoltaic grid-connected data storage method, device and electronic equipment, which are used to solve or alleviate the technical problems existing in the prior art.

[0005] In order to achieve the above object, the present invention adopts the following technical solution: In a first aspect, the present invention provides a photovoltaic grid-connected data storage method, comprising: Acquire multiple groups of photovoltaic grid-connected data records describing the same data entity, wherein any group of photovoltaic grid-connected data records includes at least one photovoltaic grid-connected metadata set, and the photovoltaic grid-connected metadata set includes a plurality of metadata; For any group of photovoltaic grid-connected data records among the multiple groups of photovoltaic grid-connected data records, key-value them to map them into a key-value pair, wherein the key of the key-value pair matches the unique ID of the any group of photovoltaic grid-connected data records, and the value of the key-value pair matches the photovoltaic grid-connected metadata set of the any group of photovoltaic grid-connected data records; Performing a hash operation on the key-value pair to obtain a corresponding hash value, so as to determine, based on the hash value, a photovoltaic grid-connected metadata storage task for storing any group of photovoltaic grid-connected data records in a target database; Based on the data task, any group of photovoltaic grid-connected data records are stored in the target database.

[0006] In a second aspect of the present invention, there is provided a photovoltaic grid-connected data storage device, comprising: A photovoltaic grid-connected data record acquisition unit, used to acquire multiple groups of photovoltaic grid-connected data records describing the same data entity, any group of photovoltaic grid-connected data records includes at least one photovoltaic grid-connected metadata set, and the photovoltaic grid-connected metadata set includes a plurality of metadata; A keying unit, for keying any one of the plurality of photovoltaic grid-connected data records to map it into a key-value pair, wherein the key of the key-value pair matches the unique ID of the any one of the photovoltaic grid-connected data records, and the value of the key-value pair matches the photovoltaic grid-connected metadata set of the any one of the photovoltaic grid-connected data records; A task allocation unit, configured to perform a hash operation on the key-value pair to obtain a corresponding hash value, so as to determine, based on the hash value, a photovoltaic grid-connected metadata storage task for storing any group of photovoltaic grid-connected data records in a target database; A data storage unit is used to store any group of photovoltaic grid-connected data records into the target database based on the data task.

[0007] According to a third aspect of the present invention, there is provided an electronic device, comprising: one or more processors; A computer readable medium configured to store one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method described in the embodiments of the present application.

[0008] Compared with the prior art, the beneficial effects of the present invention are as follows: In this application, by obtaining multiple sets of photovoltaic grid-connected data records describing the same data entity and key-value processing of these data records, the method allows complex data structures to be simplified into key-value pairs, which is convenient for unified management and storage. The scheme also takes into account the various metadata types contained in the photovoltaic grid-connected metadata set. To this end, different types of metadata can be uniformly mapped to the values ​​of key-value pairs through the key-value method, thereby achieving effective management of multiple metadata types. Furthermore, the scheme uses a hash operation to generate a hash value of a fixed length, which can be used as an index of the storage location of the data in the database. Through the hash value, the location of the data in the database can be quickly located, greatly improving the efficiency of data retrieval. Moreover, the speed of the hash operation is very fast, and a hash value can be quickly generated for the data and its storage location can be determined. In addition, once the data is hashed and stored in the database, the retrieval based on the hash value is also very fast. This fast data processing and retrieval capability enables the scheme to meet the real-time requirements of the photovoltaic grid-connected system. Finally, the scheme determines the data storage location through hash values, and can be flexibly stored in any database with sufficient space without pre-defining a fixed data structure or pattern. This flexibility enables the scheme to adapt to various complex and changeable data generated by photovoltaic grid-connected systems. Moreover, each photovoltaic grid-connected data record is assigned a unique ID and serves as the key of the key-value pair. This unique ID ensures the uniqueness of the data and also allows the data to be accurately tracked and retrieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The drawings constituting a part of the present application are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings: Figure 1 A schematic diagram of a photovoltaic grid-connected data storage method according to an embodiment of the present application.

[0010] Figure 2 A schematic diagram of a photovoltaic grid-connected data storage method according to an embodiment of the present application.

[0011] Figure 3 This is a schematic structural diagram of a photovoltaic grid-connected data storage device according to an embodiment of the present application.

[0012] Figure 4 Schematic diagram of the structure of the electronic device of this embodiment.

[0013] Figure 5 Detailed description of the hardware structure of the electronic device of this embodiment. DETAILED DESCRIPTION

[0014] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.

[0015] The following detailed description is an exemplary description, which is intended to provide further detailed description of the present invention. Unless otherwise specified, all technical terms used in the present invention have the same meaning as those generally understood by those skilled in the art to which the present application belongs. The terms used in the present invention are only for describing specific embodiments, and are not intended to limit exemplary embodiments according to the present invention.

[0016] In this application, by obtaining multiple sets of photovoltaic grid-connected data records describing the same data entity and key-value processing of these data records, the method allows complex data structures to be simplified into key-value pairs, which is convenient for unified management and storage. The scheme also takes into account the various metadata types contained in the photovoltaic grid-connected metadata set. To this end, different types of metadata can be uniformly mapped to the values ​​of key-value pairs through the key-value method, thereby achieving effective management of multiple metadata types. Furthermore, the scheme uses a hash operation to generate a hash value of a fixed length, which can be used as an index of the storage location of the data in the database. Through the hash value, the location of the data in the database can be quickly located, greatly improving the efficiency of data retrieval. Moreover, the speed of the hash operation is very fast, and a hash value can be quickly generated for the data and its storage location can be determined. In addition, once the data is hashed and stored in the database, the retrieval based on the hash value is also very fast. This fast data processing and retrieval capability enables the scheme to meet the real-time requirements of the photovoltaic grid-connected system. Finally, the scheme determines the data storage location through hash values, and can be flexibly stored in any database with sufficient space without pre-defining a fixed data structure or pattern. This flexibility enables the scheme to adapt to various complex and changeable data generated by photovoltaic grid-connected systems. Moreover, each photovoltaic grid-connected data record is assigned a unique ID and serves as the key of the key-value pair. This unique ID ensures the uniqueness of the data and also allows the data to be accurately tracked and retrieved.

[0017] Figure 1 This is a schematic diagram of a photovoltaic grid-connected data storage method according to an embodiment of the present application. Figure 1 As shown, it includes: S101, acquiring multiple groups of photovoltaic grid-connected data records describing the same data entity, wherein any group of photovoltaic grid-connected data records includes at least one photovoltaic grid-connected metadata set, and the photovoltaic grid-connected metadata set includes a plurality of metadata; Optionally, in this embodiment, for example, it is applied to obtain photovoltaic grid-connected data records from a source database to store them in a target database.

[0018] In this embodiment, the forms of the source database and the target database are not limited.

[0019] In this embodiment, when obtaining multiple sets of photovoltaic grid-connected data records describing the same data entity, step S101 may specifically include the following steps: Obtaining a table structure related to the data entity storage in a source database, and parsing the table structure to determine fields in the table structure; According to the table structure and the fields, a data query statement is written to obtain multiple groups of photovoltaic grid-connected data records describing the same data entity from the source database based on the table structure of the source database.

[0020] It should be noted here that in order to adapt to different types of source databases, such as relational databases and NoSQL data, the data query statement can be executed with the help of a data query tool configured in the source database to obtain multiple groups of photovoltaic grid-connected data records describing the same data entity from the source database based on the table structure of the source database.

[0021] In this embodiment, by parsing the table structure of the source database, accurate understanding of the data can be ensured. The data query statement written according to the table structure and fields can accurately extract the required data from the source database, ensuring the accuracy and consistency of the data. Since the query statement is written according to the table structure and fields, this method has high flexibility. Even if the table structure of the source database changes, as long as the table structure is re-parsed and the query statement is adjusted, the required data can continue to be obtained. Furthermore, writing clear data query statements improves the readability and maintainability of the code, which is very helpful for subsequent data processing and management, because it can be clearly seen how the data is extracted from the source database. The optimized query statement written according to the table structure and fields can improve the performance of data retrieval. By avoiding unnecessary table scans, selecting the correct index, etc., the time for data retrieval can be significantly reduced and the response speed of the system can be improved. By directly obtaining data from the source database, the integrity of the data can be ensured. All photovoltaic grid-connected data records are based on the original data in the source database without any intermediate processing or modification, so the data has a high degree of credibility. This method is not only applicable to the current photovoltaic grid-connected system, but can also be easily extended to other similar data acquisition scenarios. As long as the table structure and fields of the source database meet certain specifications, the same method can be used to obtain data.

[0022] In addition, based on the analysis of the table structure, the valid fields are further analyzed, and on this basis, the access to the source database is realized by writing query statements, for example, it can be realized by writing scripts, thereby reducing manual intervention and improving the efficiency of source database access. On this basis, since it can be realized by scripts, the above data processing and acquisition processes can be reused and executed, which can ensure timely access to the source database, thereby ensuring the timeliness of the data.

[0023] In this embodiment, after the photovoltaic grid-connected data records are obtained from the source database, in order to ensure the quality of the photovoltaic grid-connected data records, the photovoltaic grid-connected data records can be cleaned and completed, and the cleaning includes deduplication and data merging, and the completion includes, for example, adding missing fields, checking missing values ​​and abnormal values ​​and correcting them, etc., thereby improving the quality of the data to be involved in subsequent technical processing steps.

[0024] Data cleaning and conversion: Clean and convert data as needed to ensure data consistency and accuracy. This may involve operations such as data format conversion, data type conversion, data deduplication, and data merging.

[0025] S102, keying any one of the multiple photovoltaic grid-connected data records to map it into a key-value pair, wherein the key of the key-value pair matches the unique ID of the any one of the photovoltaic grid-connected data records, and the value of the key-value pair matches the photovoltaic grid-connected metadata set of the any one of the photovoltaic grid-connected data records; Optionally, in an embodiment of the present application, keying any one of the multiple sets of photovoltaic grid-connected data records to map them into key-value pairs includes: A unique ID is assigned to any group of photovoltaic grid-connected data records in the multiple groups of photovoltaic grid-connected data records, and the any group of photovoltaic grid-connected data records is key-valued to obtain a parent key-value pair, wherein the key in the key-value pair is the unique ID, and the value is the photovoltaic grid-connected metadata set in the group of photovoltaic grid-connected data records.

[0026] To this end, a unique ID is assigned to each PV grid-connected data record to ensure the uniqueness of each set of data in the system. This is crucial when retrieving, comparing or merging data, avoiding problems caused by data duplication or confusion. Through key-value, complex data structures are simplified into simple key-value pairs, greatly improving the efficiency of data retrieval. The system can quickly locate the corresponding data record based on the unique ID without traversing the entire data set, thus significantly reducing the retrieval time. The key-value data structure has good scalability and can easily add new metadata or modify existing metadata. When the PV grid-connected system needs to process new data types or data formats, it only needs to simply update the value in the key-value pair without making large-scale modifications to the entire system. The introduction of unique ID can be used as a data security measure. By controlling access rights to the unique ID, access and modification to the data can be restricted to ensure data security and integrity. The key-value data structure makes the system easier to maintain and debug. When a problem occurs, developers can quickly locate the data record where the problem is and take appropriate measures to repair it. Since each PV grid-connected data record has a unique ID, it is more convenient in the subsequent data integration and aggregation process. The system can aggregate data records with the same characteristics or attributes as needed to facilitate deeper analysis and mining.

[0027] Optionally, in an embodiment of the present application, keying any one of the multiple sets of photovoltaic grid-connected data records to map them into key-value pairs may also include: The parent key-value pair corresponding to any group of photovoltaic grid-connected data records is subjected to fission processing to obtain several child key-value pairs, the keys of the child key-value pairs reuse the keys of the parent key values, and the values ​​of the child key-value pairs are part of the metadata in the photovoltaic grid-connected metadata set.

[0028] To this end, the detailed processing of photovoltaic grid-connected data is achieved by splitting the parent key-value pair into multiple child key-value pairs. Each child key-value pair contains a part of the metadata in the photovoltaic grid-connected metadata collection, making data management and query more refined. The child key-value pairs after fission processing allow users to perform precise retrieval based on specific metadata. Since the value of the child key-value pair is more specific, the child key-value pair containing the required metadata can be directly located during retrieval, thereby significantly improving the retrieval efficiency. The structure of the child key-value pair allows users to build complex query conditions. By combining multiple child key-value pairs, users can query data records that meet multiple metadata conditions and achieve higher-level data analysis. Fission processing helps optimize data storage. By dispersing metadata into multiple child key-value pairs, the storage size of each key-value pair can be reduced, reducing storage costs. At the same time, since the value of the child key-value pair is more specific, the storage of redundant data can be reduced. The child key-value pairs after fission processing have higher flexibility. When the photovoltaic grid-connected system needs to add new metadata or modify existing metadata, it can easily create a new child key-value pair or update the value of an existing child key-value pair without large-scale modification of the entire parent key-value pair. This technical approach can be easily extended to a wider range of application scenarios. Whether it is a photovoltaic grid-connected system or other types of data management systems, as long as complex data structures need to be refined and efficiently retrieved, similar key-value pair fission strategies can be adopted.

[0029] Optionally, in the present embodiment, assigning a unique ID to any one of the multiple groups of photovoltaic grid-connected data records may include, for example: texting the multiple groups of photovoltaic grid-connected data records so that one group of photovoltaic grid-connected data records is located in one line of the text. To this end, the line number may be directly used as the unique ID of the corresponding group of photovoltaic grid-connected data records, thereby reducing the difficulty of algorithm implementation and improving the efficiency of algorithm execution.

[0030] To this end, the process of generating unique IDs is simplified by texting the photovoltaic grid-connected data records and using the row number as the unique ID. This method does not require complex algorithms or additional data structures to generate and manage unique IDs, which reduces the difficulty of algorithm implementation. The row number as a unique ID has extremely high generation efficiency because the row number is naturally generated during the texting process without the need for additional calculation or processing. This efficient unique ID generation method makes the entire data processing process faster and improves the efficiency of algorithm execution. Since the row numbers have natural order and continuity, they make the management and maintenance of photovoltaic grid-connected data records easier. For example, specific data records can be quickly located by row number to search, modify or delete data. Using row numbers as unique IDs does not require additional storage space to save these IDs. Compared with traditional unique ID generation methods, this method can significantly reduce storage space usage and storage costs. In the textual photovoltaic grid-connected data records, the row number as a unique ID can be directly used for data retrieval. By specifying the row number, the corresponding data record can be quickly located and subsequent data processing and analysis can be performed. This method does not rely on a specific data structure or file format, so it has strong adaptability. Regardless of the form in which the PV grid-connected data records are stored (such as CSV files, database tables, etc.), this method can be used to assign a unique ID.

[0031] Or, alternatively, assigning a unique ID to any one group of photovoltaic grid-connected data records among multiple groups of photovoltaic grid-connected data records may include, for example: including the timestamp of any one group of photovoltaic grid-connected data records, converting it into a UTC timestamp; performing a hash operation on the server address of the target database to obtain a hash value; generating at least one random number based on a set random function; and concatenating the UTC timestamp, the hash value, and the random number to obtain a unique ID assigned to any one group of photovoltaic grid-connected data records.

[0032] To this end, a unique ID generated by combining UTC timestamp, server address hash value and random number can ensure that each set of photovoltaic grid-connected data records has a unique and non-repeating ID in a distributed system or across multiple time periods. This uniqueness ensures the accuracy and consistency of the data in subsequent processing. Because it contains the hash value of the server address, this unique ID can achieve global uniqueness between multiple servers or data centers. Even in a distributed system, it can ensure that data records on different servers will not have the same ID. As part of the ID, the UTC timestamp makes the data record time-sensitive. By parsing the timestamp in the ID, you can quickly understand the generation time of the data record, which is very useful in data analysis and problem tracking. By introducing random numbers, the randomness and complexity of the ID are further enhanced, which helps prevent malicious users or systems from accessing or tampering with data by guessing or forging IDs. The algorithm for generating unique IDs in this method is simple and easy to implement, and the verification of the ID is also very convenient. By re-executing the same algorithm, it is easy to verify whether an ID is valid and unique. Since the ID generation algorithm is based on the standard UTC timestamp, hash function and random number generator, this method has good scalability and flexibility. As the system expands or the amount of data grows, more unique IDs can be easily generated.

[0033] S103, performing a hash operation on the key-value pair to obtain a corresponding hash value, so as to determine, based on the hash value, a photovoltaic grid-connected metadata storage task for storing any group of photovoltaic grid-connected data records in a target database; Optionally, in an embodiment of the present application, in step S103, performing a hash operation on the key-value pair to obtain a corresponding hash value, so as to determine, based on the hash value, a photovoltaic grid-connected metadata storage task for storing any group of photovoltaic grid-connected data records in a target database, including: Performing a hash operation on any sub-key-value pair to obtain a corresponding hash value, and dividing the hash value by the number of groups of photovoltaic grid-connected data records to obtain a remainder value; According to the remainder value, a photovoltaic grid-connected metadata storage task is determined to store the metadata represented by the value in any one of the sub-key-value pairs into a target database.

[0034] To this end, the remainder value obtained by dividing the hash value by the number of groups of photovoltaic grid-connected data records is used as the basis for allocating storage tasks, so that the data can be evenly distributed. This method can ensure that each storage unit or partition in the target database receives approximately equal amounts of data, thereby avoiding overloading of a storage unit and improving the load balancing capability of the entire system. As the number of photovoltaic grid-connected data records increases, the system can expand capacity by simply adding storage units or partitions. Due to the characteristics of hash operations, newly added data records will be automatically allocated to each storage unit without manual intervention or adjustment of the allocation strategy. Due to the rapidity and determinism of hash operations, the efficiency of data retrieval is also improved. When a specific metadata needs to be retrieved, it is only necessary to calculate its corresponding hash value, and then quickly locate the storage unit or partition storing the metadata according to the hash value, thereby improving the retrieval speed. The hash operation itself is irreversible, that is, the original data cannot be deduced from the hash value. This enhances the security of the data to a certain extent. Even if the storage unit is illegally accessed, the attacker cannot directly obtain the original photovoltaic grid-connected data record. In a distributed system, due to the determinism of hash operations, when a storage unit fails, the data on the failed unit can be quickly migrated to other healthy storage units. At the same time, due to the global uniqueness of the hash operation, the migrated data can still maintain its original uniqueness and consistency, thus ensuring the system's fault tolerance and fault recovery capabilities.

[0035] Optionally, in this embodiment, a hash operation can be directly performed on the key in any sub-key value pair, that is, a hash operation can be performed on the unique ID to obtain a corresponding hash value, so as to reduce the amount of data in the hash operation and improve the efficiency of data processing.

[0036] To this end, since only the key (unique ID) in the sub-key-value pair is hashed instead of the entire sub-key-value pair or the value therein, the amount of data that needs to be processed is significantly reduced, which helps to reduce the consumption of computing resources and improve the efficiency of hash operations. By reducing the amount of data for hash operations, the overall efficiency of data processing is improved. The system can generate hash values ​​more quickly and determine the specific location to store photovoltaic grid-connected data records in the target database based on the hash values, which helps to speed up the storage and retrieval of data and improve the responsiveness of the system. Hashing only the key simplifies the implementation process of the algorithm. Developers do not need to write complex algorithms to process the entire sub-key-value pair or the value therein, but only need to hash the unique ID. This reduces the difficulty of development, reduces the possibility of errors, and improves the readability and maintainability of the code. Although only the key is hashed, as long as the generation of the unique ID is sufficiently random and unique, the result of the hash operation can still maintain the uniform distribution of data. This means that the system can evenly distribute photovoltaic grid-connected data records to various storage units or partitions in the target database, avoiding the problems of data skew and overload of certain storage units.

[0037] Optionally, in this embodiment, considering that there is no absolute dependency between different groups of photovoltaic grid-connected data records, separate parallel photovoltaic grid-connected metadata storage tasks can be assigned for processing different groups of photovoltaic grid-connected data records, thereby improving the efficiency of data processing. To this end, in this example, multiple data storage tasks can be provided for selection in the form of a task pool, each data storage task is assigned a task ID, and when selecting that data storage task, the remainder value obtained by dividing the hash value by the number of groups of photovoltaic grid-connected data records is directly matched with the data task ID (keeping the similarity ratio as 1), thereby allocating the photovoltaic grid-connected metadata storage for the same group of photovoltaic grid-connected data records to multiple data storage tasks for parallel processing, thereby improving the efficiency of data processing.

[0038] Optionally, in one embodiment of the present application, considering that a data storage task may be overloaded and cause a task crash, after determining to store any group of photovoltaic grid-connected data records in the photovoltaic grid-connected metadata storage task in the target database, the method further includes: Counting the task load of the data storage task, and determining the position of the task load in the task load queue of all data storage tasks in a full sorting order from large to small; If the task load of the data storage task does not exceed the set position boundary threshold in the full sorting position of the task load queue, the data storage task is determined as an actual data storage task to store any group of photovoltaic grid-connected data records in the target database.

[0039] To this end, by evaluating the task load of the data storage task and comparing it with other tasks in the queue, it can be ensured that high-load tasks are not concentrated on one or a few servers, thereby avoiding overloading these servers. This approach helps to achieve balanced utilization of system resources and improve overall performance. Setting the location demarcation threshold can exclude data storage tasks with too high task load from the tasks actually executed. In this way, even if the system faces a large number of concurrent requests or a surge in data volume, system crashes or data loss caused by task overload can be avoided. By limiting the execution of high-load tasks, the system can maintain a stable operating state. This is crucial to ensure the integrity and accuracy of photovoltaic grid-connected data records, especially in scenarios with high requirements for real-time and accuracy. The location demarcation threshold can be adjusted according to the actual operation and needs of the system. As the system scale expands or the amount of data grows, the threshold can be appropriately adjusted to adapt to the new workload. This flexibility enables the system to adapt to different application scenarios and changes in demand. By excluding high-load tasks, the system can use more resources to perform other tasks, which helps to improve resource utilization, reduce resource waste, and improve the operating efficiency of the entire system.

[0040] Optionally, in an embodiment of the present application, the method further includes: Establish a task load structure tree based on the task load queues of all data storage tasks in a fully sorted order from large to small; According to the branch height of the task load of the data storage task in the load structure tree, it is determined whether the full sorting position of the task load of the data storage task in the task load queue exceeds a set position boundary threshold.

[0041] To this end, the task load structure tree intuitively displays the task load of all data storage tasks in a tree structure, making the load comparison between tasks clearer. According to the branch height of the data storage task in the load structure tree, the full sort position of its task load in the queue can be accurately evaluated. This method is more efficient than simple full sorting because it uses the hierarchical relationship of the tree structure to quickly locate the position of the task. Through the task load structure tree, the position boundary threshold can be set more flexibly. The administrator can adjust the height of the threshold according to the actual situation and needs of the system to adapt to different workload scenarios. Based on the evaluation results of the task load structure tree, the system can schedule data storage tasks more intelligently. For tasks with lower load, they can be executed first; for tasks with higher load, they can be temporarily delayed or assigned to servers with lower load to achieve load balancing. By accurately evaluating the task load and optimizing task scheduling, the method of this embodiment helps to improve the stability of the system. It can effectively avoid system crashes or data loss problems caused by task overload, and ensure the safe storage and efficient processing of photovoltaic grid-connected data.

[0042] Optionally, in an embodiment of the present application, the method further includes: If the task load of the data storage task exceeds the set position boundary threshold in the full sort position of the task load queue, then based on the task load structure tree, the photovoltaic grid-connected metadata storage task with the smallest task load is selected as the actual data storage task to store any group of photovoltaic grid-connected data records in the target database, and the task load structure tree is updated.

[0043] To this end, the system can achieve dynamic load balancing by evaluating the task load in real time and selecting tasks based on the load structure tree. Even in high-load scenarios, the system can avoid task overload and ensure stable operation of the system. Selecting the task with the smallest load as the actual execution task can ensure more efficient use of system resources, help improve the overall throughput of the system, and reduce the delay time of data processing. By replacing high-load tasks with low-load tasks in a timely manner, the system can avoid system bottlenecks caused by overloading of individual tasks, which helps to improve the reliability and availability of the system. The task selection strategy based on the task load structure tree makes task scheduling more intelligent and efficient. The system can dynamically adjust the order of task execution according to the real-time load situation to optimize the overall performance. During the task execution process, the system updates the task load structure tree in real time to reflect the current task load status. This enables the system to continuously monitor and adjust the task load to ensure that the system always runs in the best state.

[0044] Optionally, in one embodiment of the present application, a task load structure tree is established according to the task load queues of all data storage tasks in a fully sorted position from large to small, and may include, for example: All data storage tasks are sorted from largest to smallest according to task load, so as to select the photovoltaic grid-connected metadata storage task with the largest task load; Create an empty task load structure tree, and add the photovoltaic grid-connected metadata storage task with the largest task load as the root node to the task load structure tree; In this way, the remaining photovoltaic grid-connected metadata storage tasks are traversed and added to the task load structure tree as follows: Starting from the root node, compare the task load of the current data storage task with the initial task load of each child node.

[0045] If the task load of the current data storage task is greater than the initial task load of a child node, the current data storage task is inserted into the position of the child node.

[0046] If the task load of the current data storage task is less than the initial task load of all child nodes, the current data storage task is added as a new child node to the task load structure tree.

[0047] To this end, the task load structure tree displays the load distribution of data storage tasks in the form of a tree structure. Based on the task load structure tree, the system can intelligently select tasks with smaller loads for execution, thereby achieving load balancing and avoiding the crash of certain servers or resources due to overload. By dynamically adjusting task scheduling, the system is ensured to always run in the best state, reducing the risk of system crash and data loss caused by task overload, and improving the stability and reliability of the system. The task load structure tree enables the system to more accurately evaluate the resource requirements of each task and allocate resources according to actual conditions, thereby improving resource utilization and overall system performance. When a new data storage task is added to the system, it can be easily added to the task load structure tree, and sorted and adjusted according to its load to adapt to changes in the system. Through the task load structure tree, the system can quickly locate tasks with smaller loads, reducing the search time of task scheduling and improving the efficiency of task processing.

[0048] S104. Based on the data task, store any group of photovoltaic grid-connected data records into the target database.

[0049] Optionally, in an embodiment of the present application, storing any group of photovoltaic grid-connected data records in the target database based on the data task includes: Based on the data storage task, the metadata represented by the value in any sub-key-value pair is stored in the target database, and based on the key of any sub-key-value pair, the metadata belonging to the same group of photovoltaic grid-connected data records are merged into the same target file directory to be restored into the corresponding photovoltaic grid-connected metadata set.

[0050] To this end, the integrity of the data is ensured by splitting the photovoltaic grid-connected data records into multiple sub-key-value pairs and storing the values ​​(i.e., metadata) in these sub-key-value pairs in the target database. Each sub-key-value pair contains one or more important parts of the data record, making the data less likely to be lost or damaged during storage. Since the metadata is stored in the target database, the system can retrieve this data through an efficient database query mechanism, greatly improving the efficiency and accuracy of data retrieval, so that users can obtain the required photovoltaic grid-connected data more quickly. Based on the key of the sub-key-value pair, the metadata belonging to the same group of photovoltaic grid-connected data records are merged into the same target file directory. This step ensures the recoverability and comprehensibility of the data. By fusing the metadata, the system can restore the complete set of photovoltaic grid-connected metadata, so that users can understand and use the data more conveniently. By storing the metadata in the form of key-value pairs and combining it with a database management system, more optimized storage management can be achieved. The system can dynamically adjust the storage space as needed, improve storage efficiency, and reduce storage costs. Since the metadata is split into multiple sub-key-value pairs and stored separately, the system can flexibly process this data. For example, specific metadata can be encrypted, compressed, or converted to meet different data processing requirements.

[0051] Figure 2 This is a schematic diagram of a photovoltaic grid-connected data storage method according to an embodiment of the present application. Figure 2 As shown, it includes: S201, acquiring multiple groups of photovoltaic grid-connected data records describing the same data entity, wherein any group of photovoltaic grid-connected data records includes at least one photovoltaic grid-connected metadata set, and the photovoltaic grid-connected metadata set includes a plurality of metadata; S202, keying any one of the multiple photovoltaic grid-connected data records to map it into a key-value pair, wherein the key of the key-value pair matches the unique ID of the any one of the photovoltaic grid-connected data records, and the value of the key-value pair matches the photovoltaic grid-connected metadata set of the any one of the photovoltaic grid-connected data records; S203. Store any group of photovoltaic grid-connected data records into the target database based on the key-value pair.

[0052] Optionally, in an embodiment of the present application, step S203 includes: Performing a hash operation on the key-value pair to obtain a corresponding hash value, so as to determine, based on the hash value, a photovoltaic grid-connected metadata storage task for storing any group of photovoltaic grid-connected data records in a target database; Based on the data task, any group of photovoltaic grid-connected data records are stored in the target database.

[0053] Figure 2 The exemplary explanation of each step in the embodiment can be found in the above Figure 1 The description of the embodiments will not be repeated here.

[0054] Figure 3 A schematic diagram of the structure of a photovoltaic grid-connected data storage device provided in an embodiment of the present application. Figure 3 As shown, it includes: The photovoltaic grid-connected data record acquisition unit 301 is used to acquire multiple groups of photovoltaic grid-connected data records describing the same data entity, and any group of photovoltaic grid-connected data records includes at least one photovoltaic grid-connected metadata set, and the photovoltaic grid-connected metadata set includes a plurality of metadata; A keying unit 302 is used to key-value any one of the multiple photovoltaic grid-connected data records to map it into a key-value pair, wherein the key of the key-value pair matches the unique ID of the any one of the photovoltaic grid-connected data records, and the value of the key-value pair matches the photovoltaic grid-connected metadata set of the any one of the photovoltaic grid-connected data records; The task allocation unit 303 is used to perform a hash operation on the key-value pair to obtain a corresponding hash value, so as to determine a photovoltaic grid-connected metadata storage task for storing any group of photovoltaic grid-connected data records in a target database based on the hash value; The data storage unit 304 is used to store any group of photovoltaic grid-connected data records into the target database based on the data task.

[0055] Figure 3 In the illustrated embodiment, when each structural unit performs its corresponding specific technical steps, the exemplary explanation can be referred to above. Figure 1 The records are not repeated here.

[0056] Figure 4 Schematic diagram of the structure of the electronic device of this embodiment; the electronic device may include: One or more processors 401; The computer readable medium 402 may be configured to store one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the methods described in the above embodiments.

[0057] Figure 5is the hardware structure of the electronic device of this embodiment; Figure 5 As shown, the hardware structure of the electronic device may include: a processor 501, a communication interface 502, a computer readable medium 503 and a communication bus 504; The processor 501, the communication interface 502, and the computer readable medium 503 communicate with each other via a communication bus 504; Optionally, the communication interface 502 may be an interface of a communication module, such as an interface of a GSM module; The processor 501 can be specifically configured to: obtain multiple groups of photovoltaic grid-connected data records describing the same data entity, any group of photovoltaic grid-connected data records includes at least one photovoltaic grid-connected metadata set, and the photovoltaic grid-connected metadata set includes a number of metadata; for any group of photovoltaic grid-connected data records in the multiple groups of photovoltaic grid-connected data records, key-value them to map them into key-value pairs, the key of the key-value pair matches the unique ID of the any group of photovoltaic grid-connected data records, and the value of the key-value pair matches the photovoltaic grid-connected metadata set of the any group of photovoltaic grid-connected data records; perform hash operation on the key-value pair to obtain a corresponding hash value, so as to determine a photovoltaic grid-connected metadata storage task for storing the any group of photovoltaic grid-connected data records in a target database based on the hash value; store the any group of photovoltaic grid-connected data records in the target database based on the data task, thereby realizing dynamic and semi-structured photovoltaic grid-connected metadata storage.

[0058] The processor 501 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0059] The computer readable medium 503 can be, but is not limited to, a random access memory medium (RAM), a read-only memory medium (ROM), a programmable read-only memory medium (PROM), an erasable programmable read-only memory medium (EPROM), an electrically erasable read-only memory medium (EEPROM), etc.

[0060] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains a program code configured to execute the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication part, and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), the above functions defined in the method of the present application are executed. It should be noted that the computer-readable medium described in the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more conductors, a portable computer disk, a hard disk, a random access storage medium (RAM), a read-only storage medium (ROM), an erasable programmable read-only storage medium (EPROM or flash memory), an optical fiber, a portable compact disk read-only storage medium (CD-ROM), an optical storage medium, a magnetic storage medium, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, an apparatus or a device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program configured to be used by or in combination with an instruction execution system, an apparatus or a device. The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0061] Computer program code configured to perform the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0062] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment, or a part of a code, and the module, program segment, or a part of the code contains one or more executable instructions configured to implement the specified logical function. There are specific sequential relationships in the above-mentioned specific embodiments, but these sequential relationships are only exemplary. When the specific implementation is implemented, these steps may be less, more, or the execution order may be adjusted. That is, in some implementations as replacements, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0063] As another aspect, the present application further provides a computer-readable medium having a computer program stored thereon, which implements the method described in the above embodiment when the program is executed by a processor.

[0064] As another aspect, the present application further provides a computer-readable medium, which may be included in the device described in the above embodiment; or may exist independently without being assembled into the device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the device, the device: obtains multiple groups of photovoltaic grid-connected data records describing the same data entity, any group of photovoltaic grid-connected data records includes at least one photovoltaic grid-connected metadata set, and the photovoltaic grid-connected metadata set includes a plurality of metadata; for any group of photovoltaic grid-connected data records in the multiple groups of photovoltaic grid-connected data records, key-values ​​them to map them into key-value pairs, the key of the key-value pair matches the unique ID of the any group of photovoltaic grid-connected data records, and the value of the key-value pair matches the photovoltaic grid-connected metadata set of the any group of photovoltaic grid-connected data records; performs a hash operation on the key-value pair to obtain a corresponding hash value, so as to determine a photovoltaic grid-connected metadata storage task for storing the any group of photovoltaic grid-connected data records in a target database based on the hash value; stores the any group of photovoltaic grid-connected data records in the target database based on the data task, thereby realizing dynamic and semi-structured photovoltaic grid-connected metadata storage.

[0065] The expressions "first", "second", "the first" or "the second" used in various embodiments of the present disclosure may modify various components regardless of order and / or importance, but these expressions do not limit the corresponding components. The above expressions are only configured for the purpose of distinguishing an element from other elements. For example, a first user device and a second user device represent different user devices, although both are user devices. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element without departing from the scope of the present disclosure.

[0066] When one element (e.g., a first element) is referred to as being "(operably or communicatively) coupled" or "(operably or communicatively) coupled to" or "connected to" another element (e.g., a second element), it is understood that the one element is directly connected to the other element or the one element is indirectly connected to the other element via yet another element (e.g., a third element). Conversely, it is understood that when an element (e.g., a first element) is referred to as being "directly connected" or "directly coupled" to another element (the second element), no element (e.g., a third element) is interposed between the two.

[0067] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above invention concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.

Claims

1. A photovoltaic grid-connected data storage method, characterized in that: include: Acquire multiple groups of photovoltaic grid-connected data records describing the same data entity, wherein any group of photovoltaic grid-connected data records includes at least one photovoltaic grid-connected metadata set, and the photovoltaic grid-connected metadata set includes a plurality of metadata; For any group of photovoltaic grid-connected data records among the multiple groups of photovoltaic grid-connected data records, key-value them to map them into a key-value pair, wherein the key of the key-value pair matches the unique ID of the any group of photovoltaic grid-connected data records, and the value of the key-value pair matches the photovoltaic grid-connected metadata set of the any group of photovoltaic grid-connected data records; Performing a hash operation on the key-value pair to obtain a corresponding hash value, so as to determine, based on the hash value, a photovoltaic grid-connected metadata storage task for storing any group of photovoltaic grid-connected data records in a target database; Based on the data task, any group of photovoltaic grid-connected data records are stored in the target database.

2. The method according to claim 1, characterized in that: The keying of any one of the plurality of photovoltaic grid-connected data records to map them into key-value pairs includes: A unique ID is assigned to any group of photovoltaic grid-connected data records in the multiple groups of photovoltaic grid-connected data records, and the any group of photovoltaic grid-connected data records is key-valued to obtain a parent key-value pair, wherein the key in the key-value pair is the unique ID, and the value is the photovoltaic grid-connected metadata set in the group of photovoltaic grid-connected data records.

3. The method according to claim 2, characterized in that: The keying of any one of the plurality of photovoltaic grid-connected data records to map them into key-value pairs also includes: The parent key-value pair corresponding to any group of photovoltaic grid-connected data records is subjected to fission processing to obtain several child key-value pairs, the keys of the child key-value pairs reuse the keys of the parent key values, and the values ​​of the child key-value pairs are part of the metadata in the photovoltaic grid-connected metadata set.

4. The method according to claim 3, characterized in that: The performing hash operation on the key-value pair to obtain a corresponding hash value, and determining, based on the hash value, a photovoltaic grid-connected metadata storage task to store any group of photovoltaic grid-connected data records in a target database, comprises: Performing a hash operation on any sub-key-value pair to obtain a corresponding hash value, and dividing the hash value by the number of groups of photovoltaic grid-connected data records to obtain a remainder value; According to the remainder value, a photovoltaic grid-connected metadata storage task is determined to store the metadata represented by the value in any sub-key-value pair into the target database.

5. The method according to claim 4, characterized in that: The storing any group of photovoltaic grid-connected data records into the target database based on the data task includes: Based on the data storage task, the metadata represented by the value in any sub-key-value pair is stored in the target database, and based on the key of any sub-key-value pair, the metadata belonging to the same group of photovoltaic grid-connected data records are merged into the same target file directory to be restored into the corresponding photovoltaic grid-connected metadata set.

6. The method according to any one of claims 1 to 5, characterized in that: After determining to store any group of photovoltaic grid-connected data records into the photovoltaic grid-connected metadata storage task in the target database, the method further includes: Counting the task load of the data storage task, and determining the position of the task load in the task load queue of all data storage tasks in a full sorting order from large to small; If the task load of the data storage task does not exceed the set position boundary threshold in the full sorting position of the task load queue, the data storage task is determined as an actual data storage task to store any group of photovoltaic grid-connected data records in the target database.

7. The method according to claim 6, characterized in that The method further comprises: Establish a task load structure tree based on the task load queues of all data storage tasks in a fully sorted order from large to small; According to the branch height of the task load of the data storage task in the load structure tree, it is determined whether the full sorting position of the task load of the data storage task in the task load queue exceeds a set position boundary threshold.

8. The method according to claim 7, characterized in that The method further comprises: If the task load of the data storage task exceeds the set position boundary threshold in the full sort position of the task load queue, then based on the task load structure tree, the photovoltaic grid-connected metadata storage task with the smallest task load is selected as the actual data storage task to store any group of photovoltaic grid-connected data records in the target database, and the task load structure tree is updated.

9. A photovoltaic grid-connected data storage device, characterized in that: include: A photovoltaic grid-connected data record acquisition unit, used to acquire multiple groups of photovoltaic grid-connected data records describing the same data entity, any group of photovoltaic grid-connected data records includes at least one photovoltaic grid-connected metadata set, and the photovoltaic grid-connected metadata set includes a plurality of metadata; A keying unit, for keying any one of the plurality of photovoltaic grid-connected data records to map it into a key-value pair, wherein the key of the key-value pair matches the unique ID of the any one of the photovoltaic grid-connected data records, and the value of the key-value pair matches the photovoltaic grid-connected metadata set of the any one of the photovoltaic grid-connected data records; A task allocation unit, configured to perform a hash operation on the key-value pair to obtain a corresponding hash value, so as to determine, based on the hash value, a photovoltaic grid-connected metadata storage task for storing any group of photovoltaic grid-connected data records in a target database; A data storage unit is used to store any group of photovoltaic grid-connected data records into the target database based on the data task.

10. An electronic device, characterized in that: include: one or more processors; A computer readable medium configured to store one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 8.

Images