A data storage method based on composite allocation algorithm

Abstract

The present application relates to the technical field of photovoltaic power generation data storage, and specifically, to a data storage method based on a composite allocation algorithm. The data storage method includes: establishing a database according to the years, and establishing different types of database engine tables in the database, wherein the database engine table includes: acquisition time and sequence ID; performing allocation calculations on the data to be stored based on a preset composite allocation algorithm, and determining the configuration rules corresponding to the data to be stored; wherein the composite allocation algorithm includes: precise allocation algorithm, range allocation algorithm, and compound allocation algorithm including precise allocation algorithm and range allocation algorithm; according to the configuration rules, the data to be stored is allocated to the corresponding type of database engine table to realize data storage; to solve the problems of reading and writing difficulties, access timeouts, maintenance difficulties, and thus data storage difficulties in the prior art.

Description

Technical Field

[0001] The present application relates to the technical field of photovoltaic power generation data storage, and in particular to a data storage method based on a composite allocation algorithm. Background Art

[0002] Photovoltaic power generation is a technology that uses the photovoltaic effect of semiconductor interfaces to directly convert light energy into electrical energy. As a new energy source, the installed capacity of photovoltaic power generation is constantly increasing. Some small disturbances will affect the normal operation of photovoltaic power generation, so it is necessary to always grasp the stable operation status of photovoltaic power station equipment. However, obtaining information about the stable operation of photovoltaic power station equipment requires analyzing and processing the information data of each point of power equipment. However, photovoltaic power stations have many equipment points and a large amount of data.

[0003] At present, MySQL database, Oracle database and other databases are usually used to store the data of photovoltaic power station equipment. However, due to the large number of photovoltaic power station equipment points and large data volume, such databases will have difficulties in data reading and writing, access timeouts, and maintenance due to the engine's own bottleneck problems, which will lead to difficulties in allocating and storing photovoltaic power station equipment point data. For example, when the data volume of a single table of MySQL database exceeds 10 million, the data reading and writing performance is poor and the access time is long. Summary of the invention

[0004] The present application provides a data storage method based on a composite allocation algorithm to solve the problems of reading and writing difficulties, access timeouts, maintenance difficulties, and further data storage difficulties in the prior art.

[0005] The embodiment of the present application is implemented as follows:

[0006] The embodiment of the present application provides a data storage method based on a composite allocation algorithm, including: establishing a database according to years, and establishing different types of database engine tables in the database, wherein the database engine table includes: collection time and sequence ID; performing allocation calculation on the data to be stored based on a preset composite allocation algorithm, and determining a configuration rule corresponding to the data to be stored, wherein the composite allocation algorithm includes: a precise allocation algorithm, a range allocation algorithm, and a compound allocation algorithm including a precise allocation algorithm and a range allocation algorithm; according to the configuration rule, allocating the data to be stored to the corresponding type of database engine table to realize data storage.

[0007] In some embodiments, the specific steps of establishing different types of database engine tables in the database include: designing the structure of different types of database engine tables, and establishing corresponding database engine table scripts based on the structures of the different types of database engine tables; establishing different types of database engine tables based on the database engine table scripts; wherein the sequence ID is generated according to the snowflake algorithm and is unique.

[0008] In some embodiments, the specific steps of determining the configuration rule corresponding to the data to be stored include: performing an allocation algorithm analysis on the data to be stored to obtain the allocation algorithm corresponding to the data to be stored; if the allocation algorithm corresponding to the data to be stored is a precise allocation algorithm, performing allocation calculation on the data to be stored by using the precise allocation algorithm, and determining that the configuration rule corresponding to the data to be stored is a precise allocation rule; if the allocation algorithm corresponding to the data to be stored is a range allocation algorithm, performing allocation calculation on the data to be stored by using the range allocation algorithm, and determining that the configuration rule corresponding to the data to be stored is a range allocation rule; if the allocation algorithm corresponding to the data to be stored is a complex allocation algorithm, performing allocation calculation on the data to be stored by using the complex allocation algorithm, and determining that the configuration rule corresponding to the data to be stored is a complex allocation rule.

[0009] In some embodiments, the specific steps of allocating the data to be stored to the corresponding type of database engine table according to the configuration rule include: if the configuration rule corresponding to the data to be stored is a precise allocation rule, the data to be stored is accurately allocated according to the time category according to the precise allocation rule to obtain the time rule corresponding to the data to be stored, and the data to be stored is allocated to the database engine table under the time rule; wherein, the time category includes year category, month category, day category, and hour category, the time rule corresponding to the year category is the database date year rule, the time rule corresponding to the month category is the database date month rule, the time rule corresponding to the day category is the database date day rule, and the time rule corresponding to the hour category is the database date hour rule.

[0010] The specific steps of allocating the data to be stored to the corresponding type of database engine table according to the configuration rule include: if the configuration rule corresponding to the data to be stored is a range allocation rule, according to the range allocation rule, the data to be stored is range allocated according to the time category to obtain the time limit range corresponding to the data to be stored; according to the time limit range, the data to be stored is allocated to the database engine table within the time limit range.

[0011] The method further comprises: performing range allocation on the data to be stored according to time categories according to the range allocation rule, and further comprising: a range configuration switch for setting range allocation limits; and setting the upper limit and the lower limit of the range allocation according to the range configuration switch.

[0012] The beneficial effect of the present application is that the present application provides a data storage method based on a composite allocation algorithm, establishes a database according to years, and establishes different types of database engine tables in the database, wherein the database engine table includes: acquisition time and sequence ID; performs allocation calculation on the data to be stored based on a preset composite allocation algorithm, and determines the configuration rules corresponding to the data to be stored; according to the configuration rules, the data to be stored is allocated to the corresponding type of database engine table to realize data storage, so that large-scale photovoltaic power stations can perform data storage at the lowest cost, provide data storage support for intelligent analysis of photovoltaic equipment, and solve the problems of high data storage cost of photovoltaic power station operating equipment, difficulty in reading and writing data storage, and long access time. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Specifically, in order to more clearly illustrate the technical solution of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0014] Figure 1 A schematic diagram of a data storage method based on a composite allocation algorithm according to an embodiment of the present application is shown;

[0015] Figure 2 A schematic diagram showing an embodiment of the present application for determining data configuration rules based on a preset composite allocation algorithm. DETAILED DESCRIPTION

[0016] Certain exemplary embodiments will now be described to understand the principles of the structure, function, manufacture and use of the apparatus and methods disclosed herein as a whole. One or more examples of these embodiments have been shown in the accompanying drawings. It will be appreciated by those of ordinary skill in the art that the apparatus and methods specifically described herein and shown in the accompanying drawings are non-restrictive exemplary embodiments, and the scope of the multiple embodiments of the present application is limited only by the claims. The features shown or described in conjunction with an exemplary embodiment may be combined with the features of other embodiments. Such modifications and variations are intended to be included within the scope of the present application.

[0017] References throughout this specification to "multiple embodiments", "some embodiments", "one embodiment" or "an embodiment" etc. mean that the specific features, structures or characteristics described in conjunction with the embodiment are included in at least one embodiment. Therefore, phrases such as "in multiple embodiments", "in some embodiments", "in at least another embodiment" or "in an embodiment" etc. that appear throughout this specification do not necessarily all refer to the same embodiment. In addition, in one or more embodiments, specific features, structures or characteristics may be combined in any suitable manner. Therefore, without limitation, specific features, structures or characteristics shown or described in conjunction with one embodiment may be combined in whole or in part with features, structures or characteristics of one or more other embodiments. Such modifications and variations are intended to be included within the scope of this application.

[0018] Flowcharts are used in the present application to illustrate the operations performed by the system according to some embodiments of the present application. It should be clearly understood that the operations of the flow charts may not be accurately performed in order. On the contrary, these operations may be performed in reverse order or simultaneously. Moreover, one or more other operations may be added to the flow chart. One or more operations may be removed from the flow chart.

[0019] Figure 1 A schematic flow chart of a data storage method based on a composite allocation algorithm in an embodiment of the present application is shown.

[0020] In step 101, a database is established according to the years, and different types of database engine tables are established in the database, wherein the database engine tables include: acquisition time and sequence ID.

[0021] In some embodiments, the specific steps of establishing different types of database engine tables in the database include: designing the structure of different types of database engine tables, and establishing corresponding database engine table scripts based on the structures of the different types of database engine tables; establishing different types of database engine tables based on the database engine table scripts; wherein the sequence ID is generated according to the snowflake algorithm and is unique.

[0022] Before designing the structure of different types of database engine tables, two fields, sequence ID and collection time, are included. The sequence ID field data is generated by the snowflake algorithm and is unique. The collection time is consistent with the snowflake algorithm time series. It is necessary to manually establish a database, write a script to establish a database engine table, upload the database engine table script to the program through the program, and establish different types of database engine tables.

[0023] Figure 2 A schematic diagram showing an embodiment of the present application for determining data configuration rules based on a preset composite allocation algorithm.

[0024] In step 102, allocation calculation is performed on the data to be stored based on a preset composite allocation algorithm to determine the configuration rules corresponding to the data to be stored; wherein the composite allocation algorithm includes: a precise allocation algorithm, a range allocation algorithm, and a compound allocation algorithm including the precise allocation algorithm and the range allocation algorithm.

[0025] In some embodiments, the specific steps of determining the configuration rule corresponding to the data to be stored include: performing an allocation algorithm analysis on the data to be stored to obtain the allocation algorithm corresponding to the data to be stored; if the allocation algorithm corresponding to the data to be stored is a precise allocation algorithm, performing allocation calculation on the data to be stored by using the precise allocation algorithm, and determining that the configuration rule corresponding to the data to be stored is a precise allocation rule; if the allocation algorithm corresponding to the data to be stored is a range allocation algorithm, performing allocation calculation on the data to be stored by using the range allocation algorithm, and determining that the configuration rule corresponding to the data to be stored is a range allocation rule; if the allocation algorithm corresponding to the data to be stored is a complex allocation algorithm, performing allocation calculation on the data to be stored by using the complex allocation algorithm, and determining that the configuration rule corresponding to the data to be stored is a complex allocation rule.

[0026] Among them, the data to be stored is reversely parsed based on the acquisition time and the sequence ID, and the data to be stored is allocated and calculated to determine the configuration rules corresponding to the data to be stored. Different storage modes are used according to the configuration rules, and the storage mode is selected according to the amount of data in different scenarios. Different configuration rules are used for data storage; wherein, the reverse parsing of the sequence ID can obtain time information consistent with the acquisition time.

[0027] In some embodiments, the specific steps of allocating the data to be stored to the corresponding type of database engine table according to the configuration rule include: if the configuration rule corresponding to the data to be stored is a precise allocation rule, the data to be stored is accurately allocated according to the time category according to the precise allocation rule to obtain the time rule corresponding to the data to be stored, and the data to be stored is allocated to the database engine table under the time rule; wherein, the time category includes year category, month category, day category, and hour category, the time rule corresponding to the year category is the database date year rule, the time rule corresponding to the month category is the database date month rule, the time rule corresponding to the day category is the database date day rule, and the time rule corresponding to the hour category is the database date hour rule.

[0028] Among them, according to the sequence ID and the collection time, the stored data is accurately allocated to the year, month, day and hour, and according to the precise allocation rule, the stored data is accurately allocated to the time rule containing year, month, day and hour, and the field data is obtained and accurately matched to the database engine table under the time rule.

[0029] For example, if the allocation is by day, the data will eventually be accurately allocated to the database engine table according to the daily time rule based on the collection time or sequence ID; the precise allocation algorithm supports the use of sequence ID and collection time separately, or supports compatible modification, and can also be used separately, and can be upgraded to a combined mode, but the table allocation rule cannot be upgraded or downgraded after it is selected. If you must modify the rule, return to re-initialize the table by deleting the table, and then return to the corresponding configuration mode.

[0030] In some embodiments, the field content value is obtained through the database year equivalence judgment method, or the data table date month rule equivalence judgment method, or the data table date day rule equivalence judgment method, or the data table date hour rule equivalence judgment method, the sequence ID is reverse parsed, the time is matched after conversion, the time field is compatible with the data format, the long type timestamp, and the string type time content are uniformly converted to Date, and the util tool is used to perform value judgment after conversion.

[0031] In some embodiments, the specific steps of allocating the data to be stored to the corresponding type of database engine table according to the configuration rule include: if the configuration rule corresponding to the data to be stored is a range allocation rule, according to the range allocation rule, the data to be stored is range allocated according to time category to obtain the time limit range corresponding to the data to be stored; according to the time limit range, the data to be stored is allocated to the database engine table within the time limit range.

[0032] Among them, according to the sequence ID and the collection time, a certain range allocation rule is used to allocate the year, month, day and hour range to the stored data, and the time range span is allocated to the time category containing year, month, day and hour. In theory, it can span a year, but it is not recommended to use it in this way, which will cause excessive use of resources and have a greater impact on server resources. Using different configuration rules, you can allocate the range according to the range allocation rule to all related database engine tables covered by the range. Note: Sequence and collection time can also be used separately.

[0033] In some embodiments, the data to be stored is range-allocated according to time categories according to the range allocation rule, and further includes: a range configuration switch for setting range allocation limits; and setting the upper and lower limits of the range allocation according to the range configuration switch.

[0034] Among them, when the range of the stored data is allocated, a range configuration switch is provided to set the range allocation size. If >, <, >=, <= are used, the specific value used by the range configuration switch at this time marks the upper and lower limits (i.e., the highest limit and the lowest limit). At this time, all tables in the database engine will not be scanned, but the upper and lower limits will be isolated according to the specific value in the range configuration range switch.

[0035] For example, if the day allocation rule is used to allocate the range of the data to be stored, and the value specified in the range configuration switch is 5, then > 2021-11-14 is used, then the time range allocated at this time is: all database engine tables included from 2021-11-14 to 2021-11-19, and vice versa. The specific use depends on the device type, the number of points, the amount of data, and the specific scenario.

[0036] In some embodiments, the range allocation rules include a data date boundary determination method and a database engine table date range determination method for obtaining field content values.

[0037] Among them, the data date boundary judgment method uses the date division boundary configured to make range rule judgments. All range rules use the same configuration. Regardless of which rule is used, the boundary judgment method is processed uniformly. Single rule and single configuration are not supported. After obtaining the boundary configuration value, the upper and lower boundaries of the stored data are restricted: if >>=, a + boundary value operation is performed, and if <<=, a - boundary value operation is performed.

[0038] For example, if the critical value is set to 5, the critical value calculation for the monthly range is: >>=, get the data content of the current configuration field, perform month+5, return the current month and all database engine tables within +5, perform query, write and other operations, and return; <<=, get the data content of the current configuration field, perform month-5, return the current month and all database engine tables within -5, perform query, write and other operations, and return. Calculation of the critical value for the day range: >>=, get the data content of the current configuration field, perform day+5 calculation, return the current date and all database engine tables within +5, perform query, write and other operations, and return; <<=, get the data content of the current configuration field, perform day-5 calculation, return the current date and all database engine tables within -5, perform query, write and other operations, and return. Calculation of critical values ​​in hour range: >>=, get the data content of the current configuration field, perform hour+5, return all database engine tables in the current hour and +5 range, perform query, write and other operations, and return; <<=, get the data content of the current configuration field, perform hour-5, return all database engine tables in the current hour and -5 range, perform query, write and other operations, and return.

[0039] In some embodiments, the database engine table date range judgment method performs time range judgment based on the data to be stored according to the configuration rules, and reads the device configuration usage rules for matching, including two situations: a. If there are clear boundaries, the method returns all data within all boundaries; b. If there are no clear boundaries, it returns to call the boundary judgment method, obtains the boundaries, and returns all data within the boundaries.

[0040] In some embodiments, the specific step of allocating the data to be stored to the corresponding type of database engine table according to the configuration rule includes: if the configuration rule corresponding to the data to be stored is a multiple allocation rule, according to the multiple allocation rule, the data to be stored is allocated to the corresponding type of database engine table to realize data storage.

[0041] Among them, precise allocation and range allocation are compound compatible, and the above two algorithm rules are compatible, and the storage function is more powerful. If more complex business scenarios are required, it is recommended to use this algorithm rule. The compound allocation algorithm is suitable for more complex upstream and downstream business scenarios and supports compatibility with other tool components.

[0042] The beneficial effect of the present application is that the present application provides a data storage method based on a composite allocation algorithm, establishes a database according to years, and establishes different types of database engine tables in the database, wherein the database engine table includes: acquisition time and sequence ID; performs allocation calculation on the data to be stored based on a preset composite allocation algorithm, and determines the configuration rules corresponding to the data to be stored; according to the configuration rules, the data to be stored is allocated to the corresponding type of database engine table to realize data storage, so that large-scale photovoltaic power stations can perform data storage at the lowest cost, provide data storage support for intelligent analysis of photovoltaic equipment, and solve the problems of high data storage cost of photovoltaic power station operating equipment, difficulty in reading and writing data storage, and long access time.

[0043] In addition, it will be appreciated by those skilled in the art that various aspects of the present application may be illustrated and described by a number of patentable categories or situations, including any new and useful process, machine, product or combination of substances, or any new and useful improvements thereto. Accordingly, various aspects of the present application may be performed entirely by hardware, entirely by software (including firmware, resident software, microcode, etc.), or by a combination of hardware and software. The above hardware or software may all be referred to as "data blocks", "modules", "engines", "units", "components" or "systems". In addition, various aspects of the present application may be represented as a computer product located in one or more computer-readable media, which includes computer-readable program code.

[0044] A computer storage medium may include a propagated data signal containing computer program code, for example, in baseband or as part of a carrier wave. The propagated signal may be in a variety of forms, including electromagnetic, optical, etc., or a suitable combination. A computer storage medium may be any computer-readable medium other than a computer-readable storage medium, which can be connected to an instruction execution system, device or apparatus to communicate, propagate or transmit the program for use. The program code on the computer storage medium may be transmitted via any suitable medium, including radio, cable, fiber optic cable, RF, or similar media, or any combination of the above media.

[0045] The computer program coding required for the operation of each part of the application can be written in any one or more programming languages, including object-oriented programming languages ​​such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C#, VB.NET, Python, etc., conventional procedural programming languages ​​such as C language, Visual Basic, Fortran 2003, Perl, COBOL 2002, PHP, ABAP, dynamic programming languages ​​such as Python, Ruby and Groovy, or other programming languages, etc. The program coding can be run completely on the user's computer, or run on the user's computer as an independent software package, or run partly on the user's computer and partly on the remote computer, or run completely on the remote computer or server. In the latter case, the remote computer can be connected to the user's computer through any network form, such as a local area network (LAN) or a wide area network (WAN), or connected to an external computer (e.g., via the Internet), or in a cloud computing environment, or used as a service such as software as a service (SaaS).

[0046] In addition, unless explicitly stated in the claims, the order of the processing elements and sequences described in this application, the use of alphanumeric characters, or the use of other names are not intended to limit the order of the processes and methods of this application. Although the above disclosure discusses some invention embodiments that are currently considered useful through various examples, it should be understood that such details are only for illustrative purposes, and the attached claims are not limited to the disclosed embodiments. On the contrary, the claims are intended to cover all modifications and equivalent combinations that are consistent with the essence and scope of the embodiments of this application. For example, although the system components described above can be implemented by hardware devices, they can also be implemented only by software solutions, such as installing the described system on an existing server or mobile device.

[0047] Similarly, it should be noted that in order to simplify the description of the disclosure of this application and thus help understand one or more embodiments of the invention, in the above description of the embodiments of this application, multiple features are sometimes combined into one embodiment, figure or description thereof. However, this disclosure method does not mean that the features required by the object of this application are more than the features mentioned in the claims. In fact, the features of the embodiments are less than all the features of the single embodiment disclosed above.

[0048] Each patent, patent application, patent application disclosure, and other materials, such as articles, books, instructions, publications, documents, etc., cited in this application are hereby incorporated by reference in their entirety. Except for application history documents that are inconsistent with or conflicting with the content of this application, documents that limit the broadest scope of the claims of this application (currently or later attached to this application) are also excluded. It should be noted that if the descriptions, definitions, and / or use of terms in the attached materials of this application are inconsistent or conflicting with the content described in this application, the descriptions, definitions, and / or use of terms in this application shall prevail.

Claims

1. A data storage method based on a composite allocation algorithm, characterized in that: include: Establish a database according to the years, and establish different types of database engine tables in the database, wherein the database engine table includes: acquisition time and sequence ID; Performing allocation calculation on the data to be stored based on a preset composite allocation algorithm to determine the configuration rules corresponding to the data to be stored; wherein the composite allocation algorithm includes: a precise allocation algorithm, a range allocation algorithm, and a compound allocation algorithm including the precise allocation algorithm and the range allocation algorithm; The specific steps of determining the configuration rules corresponding to the data to be stored include: Performing allocation algorithm analysis on the data to be stored to obtain the allocation algorithm corresponding to the data to be stored; if the allocation algorithm corresponding to the data to be stored is a precise allocation algorithm, performing allocation calculation on the data to be stored by using the precise allocation algorithm, and determining that the configuration rule corresponding to the data to be stored is a precise allocation rule; if the allocation algorithm corresponding to the data to be stored is a range allocation algorithm, performing allocation calculation on the data to be stored by using the range allocation algorithm, and determining that the configuration rule corresponding to the data to be stored is a range allocation rule; if the allocation algorithm corresponding to the data to be stored is a complex allocation algorithm, performing allocation calculation on the data to be stored by using the complex allocation algorithm, and determining that the configuration rule corresponding to the data to be stored is a complex allocation rule; According to the configuration rules, the data to be stored is allocated to a database engine table of a corresponding type to implement data storage, including: If the configuration rule corresponding to the data to be stored is a precise allocation rule, the data to be stored is precisely allocated according to the time category according to the precise allocation rule to obtain the time rule corresponding to the data to be stored, and the data to be stored is allocated to the database engine table under the time rule; wherein the time category includes year category, month category, day category, and hour category, the time rule corresponding to the year category is the database date year rule, the time rule corresponding to the month category is the database date month rule, the time rule corresponding to the day category is the database date day rule, and the time rule corresponding to the hour category is the database date hour rule; Also included: a range configuration switch for setting range allocation limits; and setting the upper limit and the lower limit of the range allocation according to the range configuration switch; The specific steps of allocating the data to be stored to the corresponding type of database engine table according to the configuration rule include: If the configuration rule corresponding to the data to be stored is a range allocation rule, the data to be stored is range allocated according to the time category according to the range allocation rule to obtain the time limit range corresponding to the data to be stored; according to the time limit range, the data to be stored is allocated to the database engine table under the time limit range.

2. A data storage method based on a composite allocation algorithm according to claim 1, characterized in that: The specific steps of establishing different types of database engine tables in the database include: designing the structures of different types of database engine tables, and establishing corresponding database engine table scripts according to the structures of the different types of database engine tables; establishing different types of database engine tables according to the database engine table scripts; wherein the sequence ID is generated according to the snowflake algorithm and is unique.

Images