Data storage and reading method, processing device, electronic terminal and storage medium

Abstract

The invention provides a data storage and reading method, a processing device, an electronic terminal and a storage medium, and the data storage method provided by the invention comprises the following steps: receiving to-be-stored data, and writing the to-be-stored data into a real-time partition table; in response to a preset condition, converting the real-time partition table into a historical partition table; wherein the storage mode of the real-time partition table is line storage; the storage mode of the historical partition table is column type storage. When the data is written, the to-be-stored data is written into the real-time partition table in a row storage manner, so that the data writing performance can be improved; the real-time partition table is converted from row type storage to column type storage (historical partition table), data can be compressed at the same time, and the storage cost of the data is saved.

Description

Technical Field

[0001] This invention relates to the field of database technology, and in particular to a data storage and retrieval method, processing device, electronic terminal, and storage medium. Background Technology

[0002] Greenplum is a high-performance, massively parallel distributed database designed for big data analytics and high concurrency. Its core feature is extending PostgreSQL functionality to support large-scale data warehousing and analytical processing in a distributed database system. It supports partitioned tables, which can split large tables into sub-partitioned tables to improve query performance and manage historical data.

[0003] Traditional partitioned tables use a fixed storage format, which cannot simultaneously meet the requirements of high real-time data write performance and small storage space. Summary of the Invention

[0004] This invention mainly provides a data storage method, device, electronic terminal, and storage medium, which can improve data writing performance and storage while reducing the space occupied.

[0005] To solve the above-mentioned technical problems, the first technical solution adopted by the present invention is: to provide a data storage method, comprising: Receive the data to be stored and write it into the real-time partition table; In response to preset conditions, the real-time partition table is converted into a historical partition table; the real-time partition table is stored in row-oriented storage, while the historical partition table is stored in column-oriented storage.

[0006] In one embodiment, converting a real-time partition table into a historical partition table in response to preset conditions includes: In response to the real-time partition table not storing new data to be stored within a preset time, the real-time partition table is converted into a historical partition table; and / or If no data to be stored is received within a preset time, the real-time partition table is converted into a historical partition table.

[0007] In one embodiment, converting a real-time partition table into a historical partition table includes: Convert the table structure corresponding to the real-time partition table to the table structure corresponding to the historical partition table to obtain a temporary storage table; Insert the data stored in the real-time partition table into a temporary storage table; The real-time partition table is replaced with a temporary storage table, thereby converting the real-time partition table into a historical partition table.

[0008] In one embodiment, after converting the real-time partition table into a historical partition table, the method further includes: Create a new real-time partition table.

[0009] In one embodiment, receiving data to be stored and writing the data to be stored into a real-time partition table includes: In response to the fact that the data to be stored is real-time data, the data to be stored is written into the real-time partition table.

[0010] The method further includes: In response to the fact that the data to be stored is historical data, the data to be stored is written into the historical partition table.

[0011] To solve the above-mentioned technical problems, the second technical solution adopted by the present invention is: to provide a data reading method, comprising: Receive data read command; The corresponding data is read from the real-time partition table or the historical partition table according to the data read instruction; the real-time partition table is stored in row-oriented storage; the historical partition table is stored in column-oriented storage.

[0012] In one embodiment, reading corresponding data from a real-time partition table or a historical partition table according to a data read instruction includes: The corresponding data is read from the real-time partition table or the historical partition table based on the time information of the data to be read as represented by the data read instruction.

[0013] To solve the above-mentioned technical problems, the third technical solution adopted by the present invention is: to provide a data processing device, comprising: The data storage module is used to receive data to be stored, write the data to be stored into a real-time partition table; convert the real-time partition table into a historical partition table in response to preset conditions; and / or The data reading module is used to receive data reading instructions and read the corresponding data from the real-time partition table or the historical partition table according to the data reading instructions. The real-time partition table is stored in row-based storage, while the historical partition table is stored in column-based storage.

[0014] To solve the above-mentioned technical problems, the fourth technical solution adopted by the present invention is: to provide an electronic terminal, the electronic terminal including a memory and a processor coupled to each other, the processor being used to execute program instructions stored in the memory, and the processor being used to execute program data to implement the steps in the method described above.

[0015] To solve the above-mentioned technical problems, the fifth technical solution adopted by the present invention is to provide a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps in the method described above.

[0016] The beneficial effects of this invention are as follows: Unlike existing technologies, the data storage method provided in this application includes: receiving data to be stored and writing the data to be stored into a real-time partition table; and converting the real-time partition table into a historical partition table in response to preset conditions; wherein the real-time partition table is stored in row-based storage, and the historical partition table is stored in column-based storage. This application improves data writing performance by using row-based storage to write the data to be stored into the real-time partition table; and by converting the real-time partition table from row-based storage to column-based storage (historical partition table), data compression can be performed simultaneously, saving data storage costs. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating the first embodiment of the data storage method of the present invention; Figure 2 This is a flowchart illustrating an embodiment of the data storage method of the present invention for converting a real-time partition table into a historical partition table; Figure 3 This is a flowchart illustrating the second embodiment of the data storage method of the present invention; Figure 4 This is a flowchart illustrating an embodiment of the data reading method of the present invention; Figure 5 This is a schematic diagram of the structure of an embodiment of the data processing device of the present invention; Figure 6 This is a schematic diagram of the framework of an embodiment of the electronic terminal of the present invention; Figure 7 This is a schematic diagram of a framework of an embodiment of the computer-readable storage medium of the present invention. Detailed Implementation

[0019] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0020] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.

[0021] In this article, the term "and / or" simply describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "more" in this article means two or more objects.

[0022] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0023] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0024] The data storage and retrieval methods provided in this application embodiment can be implemented by a server or terminal alone, or by a server and terminal working together. In some embodiments, the terminal or server can implement the data storage and retrieval methods provided in this application embodiment by running a computer program. For example, the computer program can be a native program or software module in an operating system; it can be a native application (APP), i.e., a program that needs to be installed in the operating system to run, such as a client that supports virtual scenes, such as a game APP; it can also be a mini-program, i.e., a program that only needs to be downloaded to a browser environment to run; or it can be a mini-program that can be embedded in any APP. In short, the above-mentioned computer program can be any form of application, module, or plugin.

[0025] Greenplum is a high-performance, massively parallel distributed database designed for big data analytics and high concurrency. Its core feature is extending PostgreSQL functionality to support large-scale data warehousing and analytical processing. It supports partitioned tables, allowing large tables to be split into sub-partitions to improve query performance and manage historical data. Greenplum supports both row-based and column-based storage. Row-based storage is suitable for frequent writes and offers higher write performance compared to column-based storage. However, it cannot compress written data, resulting in higher storage space consumption and wasted storage space. Column-based storage allows for data compression with a high compression ratio, saving storage costs and improving memory caching efficiency. However, its write performance is lower than row-based storage, and frequent small-batch data writes (especially with indexes) can cause column-based tables to expand rapidly, leading to wasted storage space.

[0026] The existing database Greenplum stores data using a single columnar or row-based storage, which cannot simultaneously meet the requirements of high real-time data write performance and small storage space.

[0027] In view of this, this application provides a data storage method, comprising: receiving data to be stored and writing the data to be stored into a real-time partition table; and converting the real-time partition table into a historical partition table in response to preset conditions; wherein the real-time partition table is stored in row-oriented storage and the historical partition table is stored in column-oriented storage. This data storage method combines row-oriented and column-oriented storage, writing the received real-time data into a row-oriented real-time partition table to improve real-time data writing performance, and subsequently converting the real-time partition table into a column-oriented historical partition table to achieve compression, thereby reducing the data storage space.

[0028] To enable those skilled in the art to better understand the technical solution of the present invention, a data storage method provided by the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0029] Please see Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the data storage method of the present invention. The data storage method of this embodiment includes: Step S11: Receive the data to be stored and write it into the real-time partition table.

[0030] Greenplum databases typically consist of a parent table and partitioned tables. The parent table serves as a template or container for the partitioned tables, defining the common table structure for all partitions, such as column definitions, data types, and constraints, but it does not store the actual data. The parent table acts as the unified entry point for partitions; all partitioned tables inherit the structure from it, and data in all partitioned tables can be accessed by querying the parent table. Partitioned tables are child tables that inherit the structure from the parent table and actually store the data. Each partitioned table is an independent table, but logically part of the parent table. Partitioned tables are used to split data from large tables into different partitions according to preset rules (such as time ranges, regional lists, etc.), reducing the amount of data scanned during queries and improving performance. Each partitioned table has its own storage location and physical attributes and can be maintained independently (such as backups and deletion of old partitions). When data is inserted, it is automatically allocated to the corresponding partitioned table according to the partitioning rules defined in the parent table. During queries, Greenplum automatically filters out irrelevant partitioned tables, improving efficiency.

[0031] In one embodiment, during data storage, data to be stored is received and written to a real-time partition table, wherein the real-time partition table is stored in row-based storage. Since row-based storage is suitable for frequent writes and has higher write performance, this embodiment utilizes row-based storage to write the data to be stored to the real-time partition table, thereby improving data write performance.

[0032] Step S12: In response to preset conditions, convert the real-time partition table into a historical partition table.

[0033] Specifically, when preset conditions are met, the real-time partition table is converted into a historical partition table, which is stored in columnar format. The advantage of columnar storage is that it allows for compression of written data, achieving a very high compression ratio and saving storage costs. In this embodiment, the real-time partition table is converted into a historical partition table. It is understandable that, since the historical partition table is stored in columnar format, it can itself compress written data. Therefore, converting the real-time partition table from row-based storage to columnar storage (historical partition table) simultaneously compresses the data, saving on data storage costs.

[0034] In one embodiment, in response to the real-time partition table not storing new data to be stored within a preset time, the real-time partition table is converted into a historical partition table.

[0035] In another embodiment, in response to the failure to receive data to be stored within a preset time, the real-time partition table is converted into a historical partition table.

[0036] In another embodiment, in response to the real-time partition table not storing new data to be stored within a preset time, and in response to the data to be stored not being received within a preset time, the real-time partition table is converted into a historical partition table.

[0037] Specifically, a monitoring module can be set up to monitor the data in the real-time partition table. If no new data to be stored is written to the real-time partition table within a preset time, the real-time partition table is converted into a historical partition table. Alternatively, the monitoring module can monitor the data to be stored; if no data to be stored is received within a preset time, the real-time partition table is converted into a historical partition table.

[0038] In one embodiment, it is assumed that there are multiple real-time partition tables, for example, three. These three real-time partition tables are designated as Real-time Partition Table 1, Real-time Partition Table 2, and Real-time Partition Table 3 according to preset rules (such as time range, region list, etc.). When data to be stored is received, it can be stored in any one of these three real-time partition tables based on the data's tag. The monitoring module monitors these three real-time partition tables. If no new data to be stored is written to Real-time Partition Table 1 within a preset time, it can be converted into a historical partition table. The processing method for Real-time Partition Table 2 and Real-time Partition Table 3 is the same and will not be described further. If the monitoring module detects that no data to be stored has been received for a long time (i.e., the preset time), it converts Real-time Partition Table 1, Real-time Partition Table 2, and Real-time Partition Table 3 into historical partition tables.

[0039] In another embodiment, assuming there is one real-time partition table, when data to be stored is received, it is stored in the real-time partition table. The monitoring module monitors this real-time partition table. If no new data to be stored is written to the real-time partition table within a preset time, the real-time partition table can be converted into a historical partition table. Alternatively, the data to be stored can also be monitored; if no data to be stored is received within a preset time, the real-time partition table can be converted into a historical partition table.

[0040] It's important to note that the real-time partition table stores real-time data, while the historical partition table stores historical data. Understandably, when data to be stored is received, it is stored in the real-time partition table. Under preset conditions (indicating that some time has passed), the data stored in the real-time partition table at this point is considered old data, i.e., historical data, in terms of time series.

[0041] In one embodiment of this application, when receiving data to be stored, the time information corresponding to the data to be stored can also be determined, and the data to be stored can be written into a real-time partition table or a historical partition table according to the time information. Specifically, in response to the data to be stored being real-time data, the data to be stored is written into the real-time partition table. In response to the data to be stored being historical data, the data to be stored is written into the historical partition table.

[0042] Specifically, the data to be stored includes a timestamp. The timestamp determines whether the data is real-time or historical. If the timestamp indicates the data is new (real-time), it is stored in the real-time partition table. If the timestamp indicates the data is old (historical), it is stored in the historical partition table. For example, if the current time is January 1, 2025, and the timestamp of the received data indicates it occurred on January 1, 2025, it is considered real-time and stored in the real-time partition table. If the timestamp indicates it occurred on December 1, 2024, it is considered historical and stored in the historical partition table. In essence, data stored within the same day as the current time can be defined as real-time, while data before that day can be defined as historical; alternatively, data stored within the same week can be defined as real-time, while data before that week can be defined as historical. The specific definition is not limited.

[0043] In one embodiment, combined with Figure 2 , Figure 2 This is a flowchart illustrating an embodiment of the data storage method of the present invention for converting a real-time partition table into a historical partition table, specifically including: Step S21: Convert the table structure corresponding to the real-time partition table into the table structure corresponding to the historical partition table to obtain a temporary storage table.

[0044] Specifically, the table creation statement (i.e., SQL statement) corresponding to the real-time partitioned table is obtained. At this time, the table creation statement is a row-oriented table creation statement. The row-oriented table creation statement is modified by keyword replacement to become a column-oriented table creation statement. The column-oriented table creation statement is used to obtain the column-oriented table structure, that is, the table structure corresponding to the historical partitioned table, and it is named, for example, it is called temporary storage table A. At this time, temporary storage table A only includes the table structure corresponding to the real-time partitioned table, and does not include the specific data content of the real-time partitioned table.

[0045] Step S22: Insert the data stored in the real-time partition table into the temporary storage table.

[0046] The data stored in the real-time partition table is inserted into the temporary storage table A to obtain the temporary storage table B. At this time, the temporary storage table B contains the specific data content of the real-time partition table.

[0047] In one embodiment, an execution statement is created to query data in the real-time partition table, and each piece of data in the query result is inserted into temporary storage table A, thereby obtaining temporary storage table B.

[0048] Step S23: Replace the real-time partition table with a temporary storage table, thereby converting the real-time partition table into a historical partition table.

[0049] At this point, temporary storage table B contains the specific data content of the real-time partition table and has completed the conversion from row-based storage to column-based storage. Temporary storage table A is used to replace the real-time partition table, thereby converting the real-time partition table into a historical partition table.

[0050] Specifically, the partition swap execution statement can be used to replace the live partition table with the temporary storage table B. Partition swapping is a very powerful operation; it can remove a partition from one table and replace it with another, not just move data. The partition swapping function is used to swap the temporary storage table B with the live partition table. It should be noted that the swap execution statement includes the name of the parent table corresponding to the partition table to be swapped, and the name of the partition table to be swapped (including the name of the live partition table and the name of the temporary storage table B).

[0051] After the partitions are swapped, the real-time partition table is converted to columnar storage (at this time, it is called temporary storage table B). When the partitions are swapped, the data is inserted into the temporary storage table A all at once, so there will be no bloat caused by frequent writes. Furthermore, the temporary storage table B is stored in columnar storage mode, which can compress the inserted data and achieve efficient storage.

[0052] After the partition swap is performed, temporary storage table B replaces the real-time partition table, and the real-time partition table that was swapped is then deleted.

[0053] The data storage method of this application employs a hybrid row-column storage design. Real-time data is written to a row-based storage partition table, achieving high-efficiency write performance. Historical data is converted from row to column and stored in a column-based storage partition table, leveraging the high compression ratio of column-based storage for efficient storage. A monitoring module monitors the real-time partitions and automatically converts the row-based storage partition table to a column-based storage partition table, achieving automatic partition conversion.

[0054] See Figure 3 , Figure 3 This is a flowchart illustrating the second embodiment of the data storage method of the present invention. Steps S31 and S32 in this embodiment are the same as those described above. Figure 1 Steps S11 and S12 in the first embodiment shown are the same, except that this embodiment further includes the following after step S32: Step S33: Create a new real-time partition table.

[0055] In order not to affect the real-time writing of data, after converting the real-time partition table into a historical partition table, a new real-time partition table is created so that new data to be stored can be written.

[0056] Specifically, if a real-time partition table includes multiple real-time partition tables, a new real-time partition table can be created only after all real-time partition tables have been converted to historical partition tables. Alternatively, if multiple real-time partition tables are partitioned according to preset rules, a new real-time partition table with the same rules should be created immediately after the corresponding real-time partition table is converted to a historical partition table to avoid affecting data storage. If there is only one real-time partition table, the corresponding real-time partition table should be created immediately after the real-time partition table is converted to a historical partition table to avoid affecting data storage.

[0057] Understandably, during the initial database runtime, it only has a real-time partition table. When storing data to be stored, it uses row-based storage to store the data in the real-time partition table. When preset conditions are met, the real-time partition table is converted into a column-based historical partition table, at which point the partition table includes the column-based historical partition table. After converting the real-time partition table into a historical partition table, a new real-time partition table is created to allow new data to be stored to be written. At this point, the partition table includes both the row-based real-time partition table and the column-based historical partition table.

[0058] In one embodiment, a timing module can also be set to add new real-time partition tables according to the partitioning rules of the real-time partition table, without waiting for the real-time partition table to be converted into a historical partition table before adding new real-time partition tables.

[0059] The data storage method of this application employs a hybrid row-column storage design. Real-time data is written to a row-based storage partition table, achieving high-efficiency write performance. Historical data is converted from row to column and stored in a column-based storage partition table, leveraging the high compression ratio of column-based storage for efficient storage. A monitoring module monitors the real-time partitions and automatically converts the row-based storage partition table to a column-based storage partition table, achieving automatic partition conversion.

[0060] See Figure 4 , Figure 4 This is a flowchart illustrating an embodiment of the data reading method of the present invention. The data reading method of this embodiment includes: Step S41: Receive data read command.

[0061] When reading data, a data read command is generated, and the database receives the data read command.

[0062] Step S42: Read the corresponding data from the real-time partition table or the historical partition table according to the data reading instruction.

[0063] The real-time partition table is stored in row-oriented format; the historical partition table is stored in column-oriented format. Data is read from either the real-time or historical partition table according to a data read instruction. In one embodiment, the corresponding data is read from either the real-time or historical partition table based on the time information of the data to be read, as indicated by the data read instruction.

[0064] Assuming the current time is January 1, 2025, and the received data read instruction indicates that the time information of the data to be read is January 1, 2025, then the corresponding data is read from the real-time partition table. If the received data read instruction indicates that the time information of the data to be read is December 1, 2024, then the corresponding data is read from the historical partition table.

[0065] It should be noted that because the historical partition table uses columnar storage and the data is compressed, if you are reading data from the historical partition table, you need to decompress the data.

[0066] See Figure 5 , Figure 5 This is a schematic diagram of the structure of an embodiment of the data processing device of the present invention, specifically including: a data storage module 51 and / or a data reading module 52.

[0067] The data storage module 51 receives data to be stored and writes it into a real-time partition table; in response to preset conditions, it converts the real-time partition table into a historical partition table. The real-time partition table is stored in row-oriented format, while the historical partition table is stored in column-oriented format.

[0068] In one embodiment, the data storage module 51 is configured to convert the real-time partition table into a historical partition table in response to the real-time partition table not storing new data to be stored within a preset time; or to convert the real-time partition table into a historical partition table in response to the real-time partition table not receiving data to be stored within a preset time.

[0069] In one embodiment, the data storage module 51 is used to convert the table structure corresponding to the real-time partition table into the table structure corresponding to the historical partition table to obtain a temporary storage table; insert the data stored in the real-time partition table into the temporary storage table; and replace the real-time partition table with the temporary storage table, thereby converting the real-time partition table into a historical partition table.

[0070] In one embodiment, the data storage module 51 is used to create a new real-time partition table.

[0071] In one embodiment, the data storage module 51 is used to determine the time information corresponding to the data to be stored, and write the data to be stored into a real-time partition table or a historical partition table according to the time information.

[0072] The data reading module 52 is used to receive data reading instructions; and read the corresponding data from the real-time partition table or the historical partition table according to the data reading instructions; wherein, the real-time partition table is stored in row-oriented storage and the historical partition table is stored in column-oriented storage.

[0073] In one embodiment, the data reading module 52 is used to read the corresponding data from the real-time partition table or the historical partition table according to the time information of the data to be read represented by the data reading instruction.

[0074] This invention's data processing device employs a hybrid row-column storage design. Real-time data is written to a row-based storage partition table, achieving high-efficiency real-time data writing performance. Historical data is converted from row to column and stored in a column-based storage partition table, leveraging the high compression ratio of column-based storage for efficient storage. A monitoring module monitors the real-time partitions and automatically converts the row-based storage partition table to a column-based storage partition table, achieving automatic partition conversion.

[0075] Please see Figure 6 , Figure 6 This is a schematic diagram of the framework of an embodiment of the electronic terminal of the present invention. The electronic terminal 80 includes a memory 81 and a processor 82 coupled to each other. The processor 82 is used to execute program instructions stored in the memory 81 to implement the steps of any of the above-described data storage method and data reading method embodiments. In a specific implementation scenario, the electronic terminal 80 may include, but is not limited to, a microcomputer or a server. In addition, the electronic terminal 80 may also include mobile devices such as laptops and tablets, which are not limited here.

[0076] Specifically, processor 82 controls itself and memory 81 to implement the steps of any of the above-described data storage and data retrieval method embodiments. Processor 82 may also be referred to as a CPU (Central Processing Unit). Processor 82 may be an integrated circuit chip with signal processing capabilities. Processor 82 may also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component. A general-purpose processor may be a microprocessor or any conventional processor. Furthermore, processor 82 may be implemented using integrated circuit chips.

[0077] Please see Figure 7 , Figure 7This is a schematic diagram of a framework of an embodiment of the computer-readable storage medium of the present invention. The computer-readable storage medium 90 stores program instructions 901 that can be executed by a processor. The program instructions 901 are used to implement the steps of any of the above-described data storage method and data reading method embodiments.

[0078] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0079] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0080] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0081] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0082] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0083] The above are merely embodiments of the present invention and do not limit the scope of patent protection of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A data storage method, characterized in that, include: Receive the data to be stored and write the data to be stored into the real-time partition table; In response to preset conditions, the real-time partition table is converted into a historical partition table; wherein the real-time partition table is stored in row-oriented storage and the historical partition table is stored in column-oriented storage.

2. The data storage method according to claim 1, characterized in that, In response to preset conditions, the real-time partition table is converted into a historical partition table, including: In response to the real-time partition table not storing new data to be stored within a preset time, the real-time partition table is converted into a historical partition table; and / or If the data to be stored is not received within a preset time, the real-time partition table is converted into a historical partition table.

3. The data storage method according to claim 1, characterized in that, Converting the real-time partition table into a historical partition table includes: The table structure corresponding to the real-time partition table is converted into the table structure corresponding to the historical partition table to obtain a temporary storage table. Insert the data stored in the real-time partition table into the temporary storage table; The real-time partition table is replaced with the temporary storage table, thereby converting the real-time partition table into a historical partition table.

4. The data storage method according to claim 1, characterized in that, After converting the real-time partition table into a historical partition table, the process also includes: Create a new real-time partition table.

5. The data storage method according to claim 1, characterized in that, Receiving data to be stored and writing the data to be stored into a real-time partition table includes: In response to the fact that the data to be stored is real-time data, the data to be stored is written into the real-time partition table; The method further includes: In response to the fact that the data to be stored is historical data, the data to be stored is written into the historical partition table.

6. A data reading method, characterized in that, include: Receive data read command; The corresponding data is read from the real-time partition table or the historical partition table according to the data read instruction; wherein the real-time partition table is stored in row-oriented storage and the historical partition table is stored in column-oriented storage.

7. The data reading method according to claim 6, characterized in that, According to the data read instruction, the corresponding data is read from the real-time partition table or the historical partition table, including: The corresponding data is read from the real-time partition table or the historical partition table according to the time information of the data to be read represented by the data reading instruction.

8. A data processing apparatus, characterized in that, include: The data storage module is used to receive data to be stored and write the data to be stored into a real-time partition table; In response to preset conditions, the real-time partition table is converted into a historical partition table; and / or The data reading module is used to receive data reading instructions and read corresponding data from the real-time partition table or the historical partition table according to the data reading instructions. The real-time partition table is stored in row-based storage, while the historical partition table is stored in column-based storage.

9. An electronic terminal, characterized in that, The electronic terminal includes a memory and a processor coupled to each other, the processor being used to execute program instructions stored in the memory, and the processor being used to execute program data to implement the steps in the method as described in any one of claims 1 to 5 or 6 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method as described in any one of claims 1 to 5 or 6 to 7.

Images