Memory optimization method, device, equipment and computer storage medium

Abstract

The application discloses a memory optimization method, device and equipment and a computer storage medium, relates to the technical field of memory optimization, and comprises the following steps: obtaining original table information, and determining filtering sub-table information according to the original table information and an input filtering rule; performing triple conversion based on the filtering sub-table information to obtain triple information; determining a triple storage array according to the triple information, and performing optimized storage on the filtering sub-table information based on the triple storage array. The application reduces the memory occupation of a generated table.

Description

Technical Field

[0001] This application relates to the field of memory optimization technology, and in particular to a memory optimization method, apparatus, device, and computer storage medium. Background Technology

[0002] With the development of OLAP (On-Line Analytical Processing) database engines, users have increasingly higher requirements for OLAP storage, especially for ROLAP (Relational OLAP, an OLAP implementation based on relational databases) database engines.

[0003] Traditional storage methods involve directly storing the generated tables during crosstab analysis. This method has significant drawbacks, including high memory consumption due to direct storage, which puts pressure on the engine server's memory usage. In other words, this storage method results in a large memory footprint for storing the generated tables. Summary of the Invention

[0004] The main objective of this application is to provide a memory optimization method, apparatus, device, and storage medium, addressing the technical problem of how to optimize the memory usage of generated tables.

[0005] To achieve the above objectives, this application provides a memory optimization method, the steps of which include:

[0006] Obtain the original table information, and determine the filtered sub-table information based on the original table information and the input filtering rules;

[0007] Triple information is obtained by transforming the filtered sub-table information into triples.

[0008] A triplet storage array is determined based on the triplet information, and the filtering sub-table information is stored in an optimized manner based on the triplet storage array.

[0009] Optionally, the step of transforming triples based on the filtered sub-table information to obtain triple information includes:

[0010] The table row value information in the filtered sub-table information is determined sequentially, and the dimension parameters corresponding to the table row value information are determined.

[0011] The triplet information is obtained by performing triplet transformation based on the stated dimension parameters.

[0012] Optionally, the dimension parameters include a first dimension, a second dimension, and a target dimension located in different dimensions, and the step of performing triplet transformation based on the dimension parameters to obtain triplet information includes:

[0013] The target numerical information in the target dimension is determined based on the first dimension information in the first dimension and the second dimension information in the second dimension;

[0014] Triple information is obtained by performing triplet transformation based on the first dimension information, the second dimension information, and the target numerical information.

[0015] Optionally, after the step of performing triplet transformation based on the first dimension information, the second dimension information, and the target value to obtain triplet information, the method further includes:

[0016] Determine the completed value corresponding to the table row value information, determine the required value corresponding to the filter sub-table information, and check whether the completed value is equal to the required value; wherein, the completed value includes the value of completing the triplet transformation, and the required value includes the value of the triplet transformation that needs to be completed.

[0017] If the completed value is equal to the required value, then the step of determining the triplet storage array based on the triplet information is executed.

[0018] Optionally, the step of determining the triplet storage array based on the triplet information includes:

[0019] All triples in the triple information are determined, and each triple is stored as an array to determine the triple storage array.

[0020] Optionally, the step of determining the filtered sub-table information based on the original table information and the input filtering rules includes:

[0021] Determine the dimension filtering criteria in the input filtering rules, and determine all target information in the original table information that matches the dimension filtering criteria;

[0022] Based on the summarization of the target information, a filtering sub-table is generated.

[0023] Optionally, after the step of determining a triplet storage array based on the triplet information, and optimizing the storage of the filtered sub-table information based on the triplet storage array, the method further includes:

[0024] In response to the input expand instruction, all array elements in the triplet storage array are determined, and the storage dimension corresponding to the triplet storage array is determined;

[0025] A table is generated based on the storage dimension and each of the array elements to obtain optimized table information.

[0026] Furthermore, to achieve the above objectives, the present invention also provides a memory optimization apparatus, the memory optimization apparatus comprising:

[0027] The filtering module is used to obtain the original table information and determine the filtering sub-table information based on the original table information and the input filtering rules.

[0028] The conversion module is used to convert triples based on the information in the filtered sub-table to obtain triple information;

[0029] The storage module is used to determine a triplet storage array based on the triplet information, so as to optimize the storage of the filter sub-table information based on the triplet storage array.

[0030] This application also provides a memory optimization device, the memory optimization device comprising: a memory, a processor, and a program of the memory optimization method stored on the memory and executable on the processor, wherein when the program of the memory optimization method is executed by the processor, it can implement the steps of the memory optimization method as described above.

[0031] This application also provides a computer storage medium on which a program implementing a memory optimization method is stored, the program implementing the memory optimization method being executed by a processor to implement the steps of the memory optimization method as described above.

[0032] The technical solution of this application obtains original table information and determines filtering sub-table information based on the original table information and input filtering rules; performs triple transformation on the filtering sub-table information to obtain triple information; and determines a triple storage array based on the triple information to optimize the storage of the filtering sub-table information based on the triple storage array. By performing triple transformation on the filtering sub-table information to obtain triple information, and then optimizing the storage of the information in the filtering sub-table information using the triple storage array corresponding to the triple information, the large memory consumption caused by direct storage can be avoided, thus avoiding the memory pressure on the engine server. The memory optimization method of this application can reduce the memory consumption of the generated table by optimizing the storage of the information in the filtering sub-table information using the triple storage array corresponding to the triple information. Attached Figure Description

[0033] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0034] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is a schematic diagram of the memory optimization device structure of the hardware operating environment involved in the embodiments of the present invention;

[0036] Figure 2 This is a flowchart illustrating the first embodiment of the memory optimization method of this application;

[0037] Figure 3 This is a flowchart illustrating the second embodiment of the memory optimization method of this application;

[0038] Figure 4 This is a schematic diagram of the memory optimization device module of this application;

[0039] Figure 5 This is a flowchart illustrating the memory optimization method of this application.

[0040] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0041] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0042] Reference Figure 1 , Figure 1 This is a schematic diagram of the memory optimization device structure of the hardware operating environment involved in the embodiments of the present invention.

[0043] like Figure 1As shown, the memory optimization device may include: a processor 0003, such as a central processing unit (CPU), a communication bus 0001, an acquisition interface 0002, a processing interface 0004, and a memory 0005. The communication bus 0001 is used to establish communication between these components. The acquisition interface 0002 may include an information acquisition device or acquisition unit, such as a computer; optionally, the acquisition interface 0002 may also include a standard wired interface or a wireless interface. The processing interface 0004 may optionally include a standard wired interface or a wireless interface. The memory 0005 may be a high-speed random access memory (RAM) or a stable non-volatile memory (NVM), such as a disk storage device. Optionally, the memory 0005 may also be a storage device independent of the aforementioned processor 0003.

[0044] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the memory optimization device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0045] like Figure 1 As shown, the memory 0005, which serves as a storage medium, may include an operating device, an acquisition interface module, an execution interface module, and a memory optimization program.

[0046] exist Figure 1 In the memory optimization device shown, the communication bus 0001 is mainly used to realize the connection and communication between components; the acquisition interface 0002 is mainly used to connect to the backend server and communicate data with the backend server; the processing interface 0004 is mainly used to connect to the deployment end (user end) and communicate data with the deployment end; the processor 0003 and the memory 0005 in the memory optimization device of the present invention can be set in the memory optimization device. The memory optimization device calls the memory optimization program stored in the memory 0005 through the processor 0003 and executes the memory optimization method provided in the embodiment of the present invention.

[0047] To ensure clarity and conciseness in the description of the following embodiments, a brief introduction to the implementation of a memory optimization method is given first:

[0048] In enterprise data analysis, it is often necessary to generate cross-tabs from multi-dimensional data and present them in a two-dimensional format. For example, OLAP database engines, especially ROLAP (an OLAP implementation based on relational databases), generate significant memory usage by directly storing the generated tables during cross-tab analysis, putting pressure on the engine server's memory consumption. Optimizing the memory usage of multi-dimensional cross-tabs has become an urgent problem to solve. Traditional solutions focus on compressing memory usage by building "key-value pairs and node trees" in non-relational databases without addressing specific scenarios. They also do not specify memory optimization methods for relational databases, especially ROLAP. Based on the above issues, this application proposes a technical solution that focuses on compressing memory usage for multi-dimensional cross-tab queries using relational database triples.

[0049] This application employs a memory optimization method. It involves obtaining original table information and determining filtering sub-table information based on the original table information and input filtering rules. The method then performs a triplet transformation on the filtering sub-table information to obtain triplet information. Finally, it determines a triplet storage array based on the triplet information, and optimizes the storage of the filtering sub-table information based on this triplet storage array. By transforming the filtering sub-table information into triplet information and then optimizing the storage of the information within the filtering sub-table information using the corresponding triplet storage array, the method avoids the excessive memory consumption and memory pressure on the engine server caused by direct storage. This memory optimization method reduces the memory usage of the generated table by optimizing the storage of the information in the filtering sub-table information using the corresponding triplet storage array.

[0050] Based on the above hardware structure, an embodiment of the memory optimization method of the present invention is proposed.

[0051] This invention provides a memory optimization method, referring to... Figure 2 , Figure 2 This is a flowchart illustrating a first embodiment of a memory optimization method according to the present invention. The memory optimization method includes:

[0052] Step S10: Obtain the original table information and determine the filtering sub-table information based on the original table information and the input filtering rules;

[0053] Step S20: Based on the information in the filtered sub-table, perform triplet transformation to obtain triplet information;

[0054] Step S30: Determine a triplet storage array based on the triplet information, and optimize the storage of the filtering sub-table information based on the triplet storage array.

[0055] In this embodiment, after obtaining the original table information, the filtering sub-table information is determined based on the original table information and the input filtering rules. The filtered sub-table information is then transformed into triplet information using triplet transformation, and a corresponding triplet storage array is determined for this triplet information. Finally, the filtered sub-table information is optimized and stored based on the triplet storage array. This avoids the phenomenon of directly storing filtered sub-table information when processing relational database OLAP engines in existing technologies, ensuring that memory consumption is not caused by null values ​​in the filtered sub-table information. Here, a brief explanation of relational databases is provided: a relational database refers to tables with fixed formats and corresponding column values, where there are relationships between tables, such as a table of employee names and employee performance. Existing technologies generally optimize by determining the null values ​​in the entire table. The original table information refers to the initial table to be filtered. Since the initial table does not occupy much memory, it does not need to be stored as triples. The filtering rules refer to the filtering conditions, such as filtering for male employees or employees over 30 years old in the original table. The filtered sub-table information refers to the table obtained after filtering the initial table, such as a table containing only the names of male employees or employees over 30 years old. Triple information refers to information consisting of three elements, such as a triple combination of age, gender, and performance. The triple storage array refers to an array composed of multiple triple information. Finally, storing based on arrays ensures that the entire memory usage is based on triples, avoiding the storage of null values ​​from the filtered sub-table information, which would otherwise waste memory.

[0056] It is worth noting that this embodiment aims to address the memory optimization problem of OLAP database engines in a specific common scenario (ROLAP database engine scenario, i.e., a specific optimization method for memory usage of cross-table queries in relational database OLAP engines). Common techniques, on the other hand, focus on general optimization methods for memory usage of non-relational database OLAP engines, and these optimization methods are not applicable to relational databases, especially relational database OLAP database engine scenarios (i.e., ROLAP scenarios).

[0057] Furthermore, this embodiment also provides a flowchart of a memory optimization method, referring to... Figure 5In this embodiment, the original table storage within the OLAP does not occupy excessive storage space; memory is only consumed after the original table is filtered. This application addresses memory optimization issues of the OLAP database engine in specific common scenarios (specific scenarios refer to ROLAP scenarios). Therefore, the memory optimization scheme of this embodiment is executed only after the filtering operation. Alternatively, the original table can be stored after triplet transformation, or triplet transformation can be performed before filtering, depending on requirements. After filtering the original table to obtain a table, each row of the table is transformed, i.e., each row is transformed sequentially using triplet transformation, checking whether each row has been transformed. If not, the transformation continues to the next row. When the transformation is complete, all transformed triplets are concatenated into a triplet sequential list array, which is stored as an array. When displaying, each element of the triplet sequential list array is expanded row by row. This avoids the phenomenon of storing a large number of null values ​​when the OLAP database engine stores the filtered table, thereby reducing memory usage and optimizing the memory consumption of the ROLAP database engine in specific scenarios.

[0058] This embodiment obtains the original table information and determines the filtering sub-table information based on the original table information and the input filtering rules; it then performs a triplet transformation based on the filtering sub-table information to obtain triplet information; and finally determines a triplet storage array based on the triplet information to optimize the storage of the filtering sub-table information. By performing a triplet transformation on the filtering sub-table information to obtain triplet information, and then optimizing the storage of the information in the filtering sub-table information using the triplet storage array corresponding to the triplet information, it is possible to avoid the phenomenon of large memory occupation caused by direct storage, which puts pressure on the memory usage of the engine server. The memory optimization method of this application can reduce the memory occupation of the generated table by optimizing the storage of the information in the filtering sub-table information using the triplet storage array corresponding to the triplet information.

[0059] Furthermore, based on the first embodiment of the memory optimization method of the present invention, a second embodiment of the memory optimization method of the present invention is proposed, referring to... Figure 3 , Figure 3 This is a flowchart illustrating the second embodiment of the memory optimization method of this application. The memory optimization method includes:

[0060] Further, after the step of determining the triplet storage array based on the triplet information, and optimizing the storage of the filtered sub-table information based on the triplet storage array, the method includes:

[0061] Step S40: In response to the input expand instruction, determine all array elements in the triplet storage array and determine the storage dimension corresponding to the triplet storage array;

[0062] Step S50: Generate a table based on the storage dimension and each of the array elements to obtain optimized table information.

[0063] In this embodiment, after optimized storage based on steps S10-S30, when an input expand instruction is received, all array elements in the triplet storage array are determined, and the array elements are expanded row by row to obtain optimized table information. Specifically, the storage dimension corresponding to the triplet storage array is determined, and then a table is generated based on the storage dimension and all array elements to obtain optimized table information. Here, the expand instruction refers to a user-inputted or internally generated instruction to expand the array; the array element refers to the internally stored element; the optimized table information refers to the table obtained by expanding the optimized storage through steps S10-S30; and the storage dimension refers to the storage dimension of the array element, such as time, location, and value. Information corresponding to different dimensions can be separated using commas within the array element, and can be expanded sequentially or partially according to the actual situation. For example, the table corresponding to the filtered sub-table information is shown in Table 1:

[0064] line number Province city Year moon Sales revenue (ten thousand yuan) 1 Zhejiang Province Hangzhou 2022 1 1 2 Zhejiang Province Ningbo City 2022 1 1 3 Zhejiang Province Huzhou City 2022 1 2 4 Zhejiang Province Huzhou City 2022 2 1

[0065] Table 1

[0066] Based on the optimized storage obtained from steps S10-S30 above, the triplet storage array is S = [(Zhejiang Province - Hangzhou City, 2022-1, 1), (Zhejiang Province - Ningbo City, 2022-1, 1), (Zhejiang Province - Huzhou City, 2022-1, 2), (Zhejiang Province - Huzhou City, 2022-2, 1)]. Upon receiving the expand instruction, the first row of the triplet storage array S will be expanded sequentially based on "location (first dimension information)" and "time (second dimension information)" as "rows" and "columns," respectively, as shown in Table 2.

[0067] Location / Time 2022-1 Zhejiang Province - Hangzhou City 1

[0068] Table 2

[0069] Finally, by expanding all rows in sequence, we obtain all the array elements in the triplet storage array, as shown in Table 3:

[0070]

[0071]

[0072] Table 3

[0073] In contrast to traditional storage methods, the expanded table is shown in Table 4:

[0074] Location / Time 2022-1 2022-2 2022-3 …… 2022-12 Zhejiang Province - Hangzhou City 1 Zhejiang Province - Ningbo City 1 Huzhou City, Zhejiang Province 2 1 Taizhou City, Zhejiang Province …… Lishui City, Zhejiang Province

[0075] Table 4

[0076] Based on the comparison between Table 4 and Table 3, it is clear that Table 3 contains more useful information and is more user-friendly in terms of its expanded format. Table 4, on the other hand, is generated by enumerating all possible province-city combinations and year-month combinations within the province for that year and directly storing the entire table. This results in a large number of null values ​​being stored in memory, significantly increasing storage pressure. Therefore, this embodiment is implemented in a specific scenario based on relational database OLAP, thereby reducing the memory usage of the generated table (i.e., the table corresponding to the filtered sub-table information).

[0077] Furthermore, based on the first and second embodiments of the memory optimization method of the present invention, a third embodiment of the memory optimization method of the present invention is proposed, the memory optimization method comprising:

[0078] Furthermore, the step of transforming the triples based on the filtered sub-table information to obtain triple information includes:

[0079] Step a: Sequentially determine the table row value information in the filtered sub-table information, and determine the dimension parameters corresponding to the table row value information;

[0080] Step b: Perform triplet transformation based on the dimensional parameters to obtain triplet information.

[0081] In this embodiment, when performing triplet transformation on the information in the filtered sub-table, the process mainly involves determining the table row values ​​in the filtered sub-table information, then determining the corresponding dimension parameters for these row values, and finally performing triplet transformation based on the obtained dimension parameters to obtain triplet information. The table row values ​​refer to the information in each row of the table. For example, referring to Table 1, the first row's table row values ​​include province, city, year, month, and sales amount, along with corresponding information. Based on these row values, the corresponding dimensions can be determined, such as location and time, or other dimension determination methods, which are not limited here. After determining the dimensions, triplet transformation is performed to obtain the triplet information corresponding to the first row. Then, the dimension parameters corresponding to the table row values ​​in the second row are determined, and triplet transformation is performed to obtain the triplet information corresponding to the second row. Finally, the triplet information from all rows after triplet transformation is summarized to obtain the triplet information for the entire table. Dimension parameters refer to the dimensions of each row, such as time and location. The dimensions of each row are consistent, but the information corresponding to the corresponding dimensions is different. For example, the location dimension in Table 1 can include three types of information: Zhejiang Province-Hangzhou City, Zhejiang Province-Ningbo City, and Zhejiang Province-Huzhou City. Therefore, after transforming the information of each row into triples, the triple information of the entire table is obtained, which provides a storage basis for subsequent storage and optimizes the memory usage in the ROLAP scenario through the triple storage method.

[0082] In this embodiment, by sequentially determining the table row value information in the filtered sub-table information and determining the dimension parameters corresponding to the table row value information, and performing triplet transformation based on the dimension parameters to obtain triplet information, the progress of the entire storage can be guaranteed by sequentially performing triplet transformation on the table row value information of each row of the entire table based on the corresponding dimension parameters, providing a basis for memory-optimized storage.

[0083] Furthermore, the dimension parameters include a first dimension, a second dimension, and a target dimension located in different dimensions. The triplet information obtained by performing triplet transformation based on the dimension parameters includes:

[0084] Step c: Determine the target numerical information in the target dimension based on the first dimension information in the first dimension and the second dimension information in the second dimension;

[0085] Step d: Based on the first dimension information, the second dimension information, and the target numerical information, perform triplet transformation to obtain triplet information.

[0086] In this embodiment, after determining the dimension parameters of each row, the first dimension information corresponding to the first dimension and the second dimension information corresponding to the second dimension in each row are determined. Simultaneously, the target numerical information of the first and second dimension information in the target dimension is determined. Finally, a triplet transformation is performed based on the first dimension information, the second dimension information, and the target numerical information to obtain triplet information. Here, the first dimension refers to one dimension of the table, such as time; the second dimension refers to another dimension of the table, such as location; and the target dimension refers to the third dimension of the table, such as a numerical value. Assuming the dimension parameters include the first dimension (time) and the second dimension (location), the first dimension information refers to the time corresponding to the time dimension in each row, the second dimension information refers to the location corresponding to the location dimension in each row, and the target numerical information refers to the unique numerical value corresponding to the two dimensions in each row. Table 1 is used as an example to illustrate this.

[0087] First, take the first row, merge the location dimension into K1, merge the time dimension into K2, and record the value as V. This can be merged into a triple T, i.e., (K1, K2, V). Referring to Table 5, take the first row from Table 1 as shown:

[0088] line number Province city Year moon Sales revenue (ten thousand yuan) 1 Zhejiang Province Hangzhou 2022 1 1

[0089] Table 5

[0090] By determining that the first dimension (time) in the first row of dimension parameters is January 2022, and the second dimension (location) is Hangzhou City, Zhejiang Province, the target value information in the target dimension (sales revenue) is 10,000 yuan. The first and second dimension information can be interchanged; the correspondence is not limited. The dimensions within each axis are joined using the separator "-". The separator can be any symbol; this embodiment will use "-" for all examples, resulting in the table shown in Table 6.

[0091] line number Place time value 1 Zhejiang Province - Hangzhou City 2022-1 1

[0092] Table 6

[0093] The information in Table 6 is converted into triples to obtain the triple information in the first row, as shown in Table 7:

[0094] line number Triple group 1 (Hangzhou, Zhejiang Province, 2022-1, 1)

[0095] Table 7

[0096] Based on the above process, each row in the table corresponding to the filtered sub-table information can be converted into the form of Table 6, as shown in Table 8 below:

[0097] line number Place time value 1 Zhejiang Province - Hangzhou City 2022-1 1 2 Zhejiang Province - Ningbo City 2022-1 1 3 Huzhou City, Zhejiang Province 2022-1 2 4 Huzhou City, Zhejiang Province 2022-2 1

[0098] Table 8

[0099] Finally, based on the triple transformations in Tables 6 and 7 above, the overall triples of the tables corresponding to the filtered sub-table information can be obtained as the triplet information, as shown in Table 9:

[0100] line number Triple group 1 (Hangzhou, Zhejiang Province, 2022-1, 1) 2 (Ningbo City, Zhejiang Province, 2022-1, 1) 3 (Huzhou City, Zhejiang Province, 2022-1, 2) 4 (Huzhou City, Zhejiang Province, February 1, 2022)

[0101] Table 9

[0102] By performing the above triplet transformation operation on the table corresponding to the information in the filter sub-table, the accuracy of the storage basis for subsequent processing can be guaranteed.

[0103] In this embodiment, the target numerical information in the target dimension is determined based on the first dimension information in the first dimension and the second dimension information in the second dimension. Triple information is obtained by performing triple transformation based on the first dimension information, the second dimension information and the target numerical information. Triple information is obtained by performing triple transformation based on the second dimension information and the target numerical information, thereby realizing the triple transformation of the entire table, which can provide a basis for the subsequent optimized storage of the entire table.

[0104] Further, after the step of performing triplet transformation based on the first dimension information, the second dimension information, and the target value to obtain triplet information, the process includes:

[0105] Step e: Determine the completed value corresponding to the table row value information, determine the required value corresponding to the filtered sub-table information, and check whether the completed value is equal to the required value; wherein, the completed value includes the value for completing the triplet transformation, and the required value includes the value for which the triplet transformation needs to be completed;

[0106] Step f: If the completed value is equal to the required value, then the step of determining the triplet storage array based on the triplet information is executed.

[0107] In this embodiment, when performing triplet conversion on the entire table, each row of the table needs to be converted sequentially, thus requiring a progress monitoring process. By determining the completed value corresponding to the table row values, the required value corresponding to the filtering sub-table information is determined. The completed value is checked against the required value. If they are equal, the step of determining the triplet storage array based on the triplet information is executed; otherwise, the next row value in the table row values ​​is retrieved and triplet conversion is performed, until the conversion is complete. The completed value refers to the number of rows whose triplet conversion has been completed, and the required value refers to the number of rows whose triplet conversion needs to be completed, generally based on the row number or the actual number of rows in the table corresponding to the filtering sub-table information. For example, referring to Table 1, the required value should be 4 based on the row number. When performing triplet conversion to determine the table row values, the completed value is incremented by one as each row's triplet conversion is completed, until the completed value equals the required value, at which point the next step is executed. Other detection methods can also be used, such as performing triplet transformations on each row of table row value information sequentially. When the current row to be triplet transformed is the last row, the next step is executed after the triplet transformation is completed here. Other detection methods can also be used, which are not limited here.

[0108] In this embodiment, by determining the completed value corresponding to the table row value information and the required value corresponding to the filtered sub-table information, and detecting whether the completed value is equal to the required value, wherein the completed value includes the value for which triplet transformation has been completed, and the required value includes the value for which triplet transformation needs to be completed; if the completed value is equal to the required value, then the step of determining the triplet storage array based on the triplet information is executed. By detecting the triplet transformation of the entire table, the accuracy of triplet transformation can be guaranteed, and a storage basis can be provided for subsequent storage.

[0109] Furthermore, the step of determining the triplet storage array based on the triplet information includes:

[0110] Step g: Determine all triples in the triple information and store each triple as an array, thus determining the triple storage array.

[0111] In this embodiment, after determining the triplet information corresponding to each row based on the above method, the triplet information of each row in the entire table, that is, all the triplets in the triplet information, are stored as an array, and the triplet storage array is determined. The triplet storage array refers to all triplets stored as an array. Therefore, based on triplet transformations, the phenomenon of storing null values ​​can be avoided in specific ROLAP scenarios, thereby optimizing memory usage in specific scenarios. For example, the triples of each row in the concatenated table 9 are a triple sequential list array S = [(Zhejiang Province-Hangzhou City, 2022-1, 1), (Zhejiang Province-Ningbo City, 2022-1, 1), (Zhejiang Province-Huzhou City, 2022-1, 2), (Zhejiang Province-Huzhou City, 2022-2, 1)]. This array storage method combines the tables corresponding to the filtered sub-table information into an array of triples. Null values ​​are not stored in the array, which can effectively reduce memory usage. The array does not store null values ​​generated by row and column intersections, avoiding the phenomenon of storing null values ​​that occurs when storing the entire result table in the traditional method. This can greatly improve memory optimization in the ROLAP scenario.

[0112] In this embodiment, by determining all triples in the triplet information and storing each triplet as an array, a triplet storage array is determined. By storing the triplet as an array, null values ​​are not stored. This improves memory utilization and greatly increases the user's storage processing speed by avoiding the occupation of null values ​​and expansion time.

[0113] Furthermore, based on the first, second, and third embodiments of the memory optimization method of the present invention, a fourth embodiment of the memory optimization method of the present invention is proposed, the memory optimization method comprising:

[0114] Furthermore, the step of determining the filtered sub-table information based on the original table information and the input filtering rules includes:

[0115] Step m: Determine the dimension filtering criteria in the input filtering rules, and determine all target information in the original table information that matches the dimension filtering criteria;

[0116] Step n: Generate a filtering sub-table based on the summaries of the target information.

[0117] In this embodiment, since the original table information does not consume excessive memory, it does not need to be stored based on a triplet storage array. The original table information can be filtered after triplet transformation. That is, by defining the dimension filtering criteria in the input filtering rules, all target information in the original table information after triplet transformation that matches the dimension filtering criteria is determined, and finally, filtered sub-table information is generated based on the target information. Here, the dimension filtering criteria refers to the basis for filtering, such as using employees over 30 years old as the filtering dimension criterion. Target information refers to the information in the original table information after triplet transformation that meets the dimension filtering criteria. Filtering based on the original table information after triplet transformation ensures that the filtered sub-table information does not contain redundant null values, thus ensuring the simplicity of the filtered sub-table information. The following example uses a multi-dimensional cross-tabulation of "time, location, and sales." The time axis can be defined as columns, the location axis as rows, and sales as values. The time axis dimension can be "year" or "month," the location axis dimension can be "province" or "city," and the value is sales. For example, suppose the total sales of an online store in each province and city for each month and year are shown in Table 10:

[0118]

[0119]

[0120] Table 10

[0121] Based on Table 10 above, each single row record is transformed into a triplet. No storage is required. The goal of the transformation is to avoid filtering out too many null values. For example, performing a triplet transformation on the first row of Table 10 merges the location dimension into K1, the time dimension into K2, and the value record into V, forming a triplet T, i.e., (K1, K2, V). The delimiter "-" can be used to concatenate the dimensions within each axis. The delimiter can be any symbol; this embodiment will use "-" for all examples. The first row is transformed into triplets as shown in Table 11, and all the triplets corresponding to Table 10 are shown in Table 12.

[0122] line number Triple group 1 (Hangzhou, Zhejiang Province, 2022-1, 1)

[0123] Table 11

[0124] line number Triple group 1 (Hangzhou, Zhejiang Province, 2022-1, 1) 2 (Ningbo City, Zhejiang Province, 2022-1, 1) 3 (Huzhou City, Zhejiang Province, 2022-1, 2) 4 (Huzhou City, Zhejiang Province, February 1, 2022) 5 (Huzhou City, Zhejiang Province, February 1, 2023) 6 (Guangdong Province - Guangzhou City, March 3, 2023)

[0125] Table 12

[0126] After obtaining Table 12, a filtering query operation will be performed, which involves filtering the triplet table corresponding to the original detailed record table and returning the query results. The query results are the tables corresponding to the information in the filtered sub-tables. For example, if the filtering conditions in this query are "Province = Zhejiang Province" and "Year = 2022", then the result returned after filtering Table 11 is shown in Table 13.

[0127] line number Province city Year moon Sales revenue (ten thousand yuan) 1 Zhejiang Province Hangzhou 2022 1 1 2 Zhejiang Province Ningbo City 2022 1 1 3 Zhejiang Province Huzhou City 2022 1 2 4 Zhejiang Province Huzhou City 2022 2 1

[0128] Table 13

[0129] In this embodiment, by determining the dimension filtering criteria in the input filtering rules and identifying all target information in the original table information that matches the dimension filtering criteria, filtering sub-table information is generated based on the sum of each target information. This ensures the simplicity of the filtering sub-table information and avoids the phenomenon of a large number of null values ​​in Table 4. This also ensures the efficiency of storage processing, as it avoids the need to handle triple transformations with a large number of null values. This optimizes memory usage and improves the efficiency of subsequent triple transformations. Instead of globally enumerating all possibilities, it only enumerates the row and column combinations after the current query is filtered, avoiding the phenomenon of enumerating all possible row and column combinations that occurs in traditional methods.

[0130] The present invention also provides a memory optimization device, with reference to Figure 4 The memory optimization device includes:

[0131] The filtering module A01 is used to obtain the original table information and determine the filtering sub-table information according to the original table information and the input filtering rules.

[0132] The conversion module A02 is used to convert triples based on the information in the filtered sub-table to obtain triple information;

[0133] Storage module A03 is used to determine a triplet storage array based on the triplet information, so as to optimize the storage of the filter sub-table information based on the triplet storage array.

[0134] Optionally, the conversion module A02 is further configured to:

[0135] The table row value information in the filtered sub-table information is determined sequentially, and the dimension parameters corresponding to the table row value information are determined.

[0136] The triplet information is obtained by performing triplet transformation based on the stated dimension parameters.

[0137] Optionally, the conversion module A02 is further configured to:

[0138] The target numerical information in the target dimension is determined based on the first dimension information in the first dimension and the second dimension information in the second dimension;

[0139] Triple information is obtained by performing triplet transformation based on the first dimension information, the second dimension information, and the target numerical information.

[0140] Optionally, the conversion module A02 is further configured to:

[0141] The system determines the completed value corresponding to the row value information in the table, and determines the required value corresponding to the filtered sub-table information, and checks whether the completed value is equal to the required value; wherein, the completed value includes the value for completing the triplet transformation, and the required value includes the value for which the triplet transformation needs to be completed;

[0142] If the completed value is equal to the required value, then the step of determining the triplet storage array based on the triplet information is executed.

[0143] Optionally, the storage module A03 is further configured to:

[0144] All triples in the triple information are determined, and each triple is stored as an array to determine the triple storage array.

[0145] Optionally, the filtering module A01 is further configured to:

[0146] Determine the dimension filtering criteria in the input filtering rules, and determine all target information in the original table information that matches the dimension filtering criteria;

[0147] Based on the summarization of the target information, a filtering sub-table is generated.

[0148] Optionally, the storage module A03 is further configured to:

[0149] In response to the input expand instruction, all array elements in the triplet storage array are determined, and the storage dimension corresponding to the triplet storage array is determined;

[0150] A table is generated based on the storage dimension and each of the array elements to obtain optimized table information.

[0151] The methods executed by the above-mentioned program modules can be referred to in the various embodiments of the memory optimization method of the present invention, and will not be repeated here.

[0152] The present invention also provides a memory optimization device.

[0153] The device of the present invention includes: a memory, a processor, and a memory optimization program stored on the memory and executable on the processor, wherein the memory optimization program, when executed by the processor, implements the steps of the memory optimization method as described above.

[0154] The present invention also provides a storage medium.

[0155] The present invention stores a memory optimization program on a storage medium, which, when executed by a processor, implements the steps of the memory optimization method described above.

[0156] The method implemented when the memory optimization program running on the processor is executed can be referred to in various embodiments of the memory optimization method of the present invention, and will not be repeated here.

[0157] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0158] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0159] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0160] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A memory optimization method, characterized in that, The memory optimization method is applied to the ROLAP database engine scenario, and the steps of the memory optimization method include: Obtain original table information and determine filter sub-table information based on the original table information and input filtering rules. The step of determining filter sub-table information based on the original table information and input filtering rules includes: determining the dimension filtering criteria in the input filtering rules and determining all target information in the original table information that matches the dimension filtering criteria; and generating filter sub-table information based on the summaries of each target information. Based on the information in the filtered sub-table, triplet transformation is performed to obtain triplet information, wherein the triplet information includes triplet combinations that are not null. The triplet storage array is determined based on the triplet information, and the filtering sub-table information is optimized and stored based on the triplet storage array. Following the step of determining the triplet storage array based on the triplet information and optimizing the storage of the filtering sub-table information based on the triplet storage array, the process includes: determining all array elements in the triplet storage array in response to an input expand instruction, and determining the storage dimension corresponding to the triplet storage array; generating a table based on the storage dimension and each array element to obtain optimized table information.

2. The memory optimization method as described in claim 1, characterized in that, The step of transforming triplets based on the information in the filtered sub-table to obtain triplet information includes: The table row value information in the filtered sub-table information is determined sequentially, and the dimension parameters corresponding to the table row value information are determined. The triplet information is obtained by performing triplet transformation based on the stated dimension parameters.

3. The memory optimization method as described in claim 2, characterized in that, The dimension parameters include a first dimension, a second dimension, and a target dimension, which are located in different dimensions. The process of performing triplet transformation based on the dimension parameters to obtain triplet information includes: The target numerical information in the target dimension is determined based on the first dimension information in the first dimension and the second dimension information in the second dimension; Triple information is obtained by performing triplet transformation based on the first dimension information, the second dimension information, and the target numerical information.

4. The memory optimization method as described in claim 3, characterized in that, After the step of performing triplet transformation based on the first dimension information, the second dimension information, and the target numerical information to obtain triplet information, the following steps are included: Determine the completed value corresponding to the table row value information, determine the required value corresponding to the filter sub-table information, and check whether the completed value is equal to the required value; wherein, the completed value includes the value of completing the triplet transformation, and the required value includes the value of the triplet transformation that needs to be completed. If the completed value is equal to the required value, then the step of determining the triplet storage array based on the triplet information is executed.

5. The memory optimization method as described in claim 1, characterized in that, The step of determining the triplet storage array based on the triplet information includes: All triples in the triple information are determined, and each triple is stored as an array to determine the triple storage array.

6. A memory optimization device, characterized in that, The memory optimization device is applied to scenarios using the ROLAP database engine, and the memory optimization device includes: A filtering module is used to obtain original table information and determine filtering sub-table information based on the original table information and input filtering rules. The step of determining filtering sub-table information based on the original table information and input filtering rules includes: determining the dimension filtering criteria in the input filtering rules, and determining all target information in the original table information that matches the dimension filtering criteria; and generating filtering sub-table information based on the summaries of each target information. The conversion module is used to perform triplet conversion based on the information in the filtered sub-table to obtain triplet information, wherein the triplet information includes triplet combinations that are not null. A storage module is configured to determine a triplet storage array based on the triplet information, so as to optimize the storage of the filter sub-table information based on the triplet storage array. After the step of determining the triplet storage array based on the triplet information and optimizing the storage of the filter sub-table information based on the triplet storage array, the module further includes: determining all array elements in the triplet storage array in response to an input expand instruction, and determining the storage dimension corresponding to the triplet storage array; generating a table based on the storage dimension and each array element to obtain optimized table information.

7. A memory optimization device, characterized in that, The memory optimization device includes: a memory, a processor, and a memory optimization program stored on the memory and running on the processor, wherein when the memory optimization program is executed by the processor, it implements the steps of the memory optimization method as described in any one of claims 1 to 5.

8. A computer storage medium, characterized in that, The computer storage medium stores a program that implements a memory optimization method, which is executed by a processor to implement the steps of the memory optimization method as described in any one of claims 1 to 5.

Images