A method and apparatus for storing

Abstract

Embodiments of the present application provide a kind of stored method and device, applied to big data technical field, comprising: obtaining multiple target query statements;Based on the matching relationship between each target query statement and N business types, determine the query statement proportion corresponding to N business types respectively;Based on the query statement proportion of N business types and the corresponding execution cost when N business types are stored under M storage models respectively, determine the execution efficiency corresponding to M storage models respectively, based on the execution efficiency corresponding to M storage models respectively and the storage space occupied by M storage models respectively corresponding, determine at least one target storage model for storing the business data of N business types from M storage models. According to the execution efficiency corresponding to M storage models respectively and the storage space occupied by M storage models respectively, so as to realize the storage of target storage model in limited storage space, realize the reasonable use of storage space.

Description

Technical Field

[0001] This invention relates to the field of big data technology, and in particular to a storage method and apparatus. Background Technology

[0002] With the continuous development of technology, the amount of business data generated for different business operations is also increasing. When business data is stored in different databases, the corresponding data models are also different. For example, if the data source is stored in a relational database, the corresponding data model is a two-dimensional table model; if the data source is stored in a key-value database, the corresponding data model is a key-value model; and if the data source is stored in a document-oriented database, the corresponding data model is a full-text search model.

[0003] Currently, business data is stored simultaneously in each data model, enabling the data to handle multiple business processes concurrently. However, due to limited storage resources, this leads to inefficient use of storage space.

[0004] In summary, how to achieve the rational use of storage space is a technical problem that urgently needs to be solved. Summary of the Invention

[0005] This invention provides a storage method and apparatus to solve the problem of inefficient use of storage space in the prior art.

[0006] In a first aspect, embodiments of the present invention provide a storage method, comprising: acquiring a plurality of target query statements; determining the proportion of query statements corresponding to each of the N business types based on the matching relationship between each target query statement and N business types; determining the execution efficiency of each of the M storage models based on the proportion of query statements of the N business types and the execution cost corresponding to each of the N business types when stored under M storage models, wherein N and M are positive integers; and determining at least one target storage model from the M storage models for storing business data of the N business types based on the execution efficiency corresponding to each of the M storage models and the storage space occupied by each of the M storage models.

[0007] In this embodiment of the invention, the execution efficiency of each of the M storage models is determined more accurately based on the target query statement. Considering the storage space occupied by each storage model, at least one target storage model for storing business data of N business types can be determined more accurately from the M storage models based on the execution efficiency and storage space occupied by each of the M storage models. Thus, at least one target storage model can be selected from the limited storage space, so as to achieve reasonable use of storage space.

[0008] Optionally, determining at least one target storage model for storing business data of the N business types from the M storage models based on the execution efficiency and storage space occupied by each of the M storage models includes: determining the storage cost-effectiveness of each storage model based on the execution efficiency and storage space occupied by each storage model; and determining at least one target storage model for storing business data of the N business types from the M storage models based on the storage cost-effectiveness of each of the M storage models.

[0009] In this embodiment of the invention, by determining the storage cost-effectiveness of each of the M storage models, it is possible to accurately determine at least one target storage model from the M storage models for storing business data of N business types, thereby achieving a balance between storage and performance within a limited storage space and realizing the rational use of storage space.

[0010] Optionally, determining at least one target storage model for storing the business data of the N business types from the M storage models based on their respective storage cost-effectiveness includes: sorting the obtained storage cost-effectiveness ratios to obtain a target sorting result; sequentially selecting target storage models from the M storage models according to the target sorting result and saving them to a preset storage area until the remaining storage space in the preset storage area is less than the storage space occupied by the next added target storage model; and selecting at least one target storage model already added to the preset storage area to store the business data of the N business types.

[0011] In this embodiment of the invention, since the storage space of the preset storage area is limited, it is necessary to sort the multiple storage cost-effectiveness ratios obtained, and then select the target storage model from M storage models according to the sorting results, so as to achieve a balance between storage and performance, and thus achieve reasonable use of storage space.

[0012] Optionally, obtaining multiple target query statements includes: obtaining multiple historical query statements; converting each historical query statement into a corresponding candidate query statement using a target template; and selecting multiple target query statements from the multiple candidate query statements based on their respective access frequencies.

[0013] In this embodiment of the invention, since the number of historical query statements is large and their formats are inconsistent, it is first necessary to use a target template to convert the historical query statements into corresponding candidate query statements, thereby unifying the format of the historical query statements and facilitating the subsequent determination of the target storage model. Then, based on the access frequency of each of the multiple candidate query statements, multiple target query statements are selected, thereby avoiding the problem of complex subsequent processing due to the large number of historical query statements, and thus improving the efficiency of the subsequent determination of the target storage model.

[0014] Optionally, the step of selecting multiple target query statements from the multiple candidate query statements based on their respective access frequencies includes: performing a clustering operation on the multiple candidate query statements based on their respective access frequencies to obtain multiple query statement clusters; and selecting multiple target query statements from the multiple query statement clusters.

[0015] In this embodiment of the invention, clustering is performed based on the access frequency of each of the multiple candidate query statements to filter out multiple target query statements, thereby enabling the target storage model to be determined more accurately based on the multiple target query statements.

[0016] Optionally, the step of selecting multiple target query statements from the multiple candidate query statements based on their respective access frequencies includes: inputting each candidate query statement into a classification model to determine the access frequency type of each candidate query statement, wherein the training samples of the classification model include: a set of multiple query statements obtained by clustering based on historical access frequencies; and selecting multiple target query statements from the multiple candidate query statements based on their access frequency types.

[0017] In this embodiment of the invention, by inputting candidate query statements into the classification model, the access frequency type of the candidate query statements can be quickly determined. By filtering multiple target query statements from multiple candidate query statements according to their access frequency type, the efficiency of determining target query statements can be improved.

[0018] Optionally, determining the proportion of query statements corresponding to each of the multiple business types based on the matching relationship between each target query statement and multiple business types includes: determining the target business type of each target query statement from the multiple business types according to the parse tree features of each target query statement; and determining the proportion of query statements corresponding to each of the multiple business types based on the target business types corresponding to each of the multiple target query statements.

[0019] In this embodiment of the invention, clustering is performed based on the parse tree features of each target query statement, thereby enabling the determination of the target business type of each target query statement and the proportion of query statements corresponding to each of the multiple business types from multiple business types. This facilitates the subsequent accurate determination of the target storage model based on the proportion of query statements corresponding to each of the multiple business types.

[0020] Optionally, determining the target business type of each target query statement from multiple business types based on the parse tree features of each target query statement includes: performing secondary clustering on the multiple target query statements based on the parse tree features of the multiple target query statements, and determining the target business type of each target query statement from multiple business types based on the clustering results.

[0021] In this embodiment of the invention, multiple target query statements are clustered in a secondary manner based on the parse tree features of multiple target query statements, thereby determining the target business type of multiple target query statements. This facilitates the subsequent determination of the proportion of query statements corresponding to the target business type, and thus makes it easier to determine the target storage model more accurately.

[0022] Secondly, embodiments of the present invention also provide a storage apparatus, comprising: an acquisition unit for acquiring a plurality of target query statements; a processing unit for determining the proportion of query statements corresponding to each of the N business types based on the matching relationship between each target query statement and N business types; determining the execution efficiency of each of the M storage models based on the proportion of query statements of the N business types and the execution cost corresponding to each of the N business types when stored under M storage models, wherein N and M are positive integers; and determining at least one target storage model from the M storage models for storing business data of the N business types based on the execution efficiency of each of the M storage models and the storage space occupied by each of the M storage models.

[0023] Optionally, based on the execution efficiency and storage space occupied by each of the M storage models, at least one target storage model for storing business data of N business types is determined from the M storage models, including: determining the storage cost-effectiveness of each storage model based on the execution efficiency and storage space occupied by each storage model; and determining at least one target storage model for storing business data of N business types from the M storage models based on the storage cost-effectiveness of each of the M storage models.

[0024] Optionally, the processing unit is specifically used to: sort the obtained multiple storage cost-effectiveness ratios to obtain a target sorting result; select target storage models from M storage models in sequence according to the target sorting result and save them to a preset storage area until the remaining storage space in the preset storage area is less than the storage space occupied by the next added target storage model; and select at least one target storage model that has been added to the preset storage area to store business data of N business types.

[0025] Optionally, the acquisition unit is specifically used to: acquire multiple historical query statements; convert each historical query statement into a corresponding candidate query statement using a target template; and select multiple target query statements from the multiple candidate query statements based on their respective access frequencies.

[0026] Optionally, the acquisition unit is specifically used to: cluster the multiple candidate query statements based on their respective access frequencies to obtain multiple query statement clusters; and select multiple target query statements from the multiple query statement clusters.

[0027] Optionally, the acquisition unit is specifically used to: input each candidate query statement into the classification model to determine the access frequency type of each candidate query statement, wherein the training samples of the classification model include: a set of multiple query statements obtained by clustering based on historical access frequencies; and select multiple target query statements from the multiple candidate query statements based on the access frequency type of the multiple candidate query statements.

[0028] Optionally, the processing unit is specifically used to: determine the target business type of each target query statement from multiple business types based on the parse tree characteristics of each target query statement; and determine the proportion of query statements corresponding to each of the multiple business types based on the target business types corresponding to each of the multiple target query statements.

[0029] Optionally, the processing unit is specifically used to: perform secondary clustering on multiple target query statements based on the parse tree features of multiple target query statements, and determine the target business type of each target query statement from multiple business types based on the clustering results.

[0030] Thirdly, embodiments of the present invention also provide an electronic device, including at least one processor and at least one memory, wherein the memory stores a computer program, and when the program is executed by the processor, the processor performs the storage method described in the first aspect.

[0031] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a program that, when run on a computer, causes the computer to perform the storage method described in the first aspect. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 A flowchart of a storage method is provided as an embodiment of the present invention;

[0034] Figure 2 A flowchart illustrating a method for obtaining multiple target query statements provided in an embodiment of the present invention;

[0035] Figure 3 A flowchart illustrating a method for determining the proportion of query statements corresponding to multiple business types, provided in an embodiment of the present invention;

[0036] Figure 4 A flowchart illustrating a method for determining a target storage model according to an embodiment of the present invention;

[0037] Figure 5 A schematic diagram of a storage device provided in an embodiment of the present invention;

[0038] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0040] The following explanations of certain parts of this application are intended to provide general clarification for those skilled in the art, and do not limit the terminology used in this application.

[0041] 1. Database: A database is a collection of interconnected data stored together in a specific organizational structure. The data in a database is not scattered or isolated, but rather organized according to a certain data model.

[0042] 2. Data Model: Since the data in a database is structured, and these structures reflect the relationships between things, the description of this structure is called a data model, which is a method of representing the relationships between data.

[0043] In one possible scenario, as more and more enterprises increase their demand for digitalization, this promotes the development and promotion of many new applications. During the development of these new applications, a single piece of data often needs to be represented by multiple different data sources for joint analysis. When business data is stored in different databases, the corresponding data models also differ. For example, there might be three data models: a two-dimensional table model, a key-value (KV) model, and a full-text search model. Two-dimensional table models are typically stored in relational databases like MySQL, KV models are typically stored in in-memory databases like Redis, and full-text search models are typically stored in open-source systems like Elasticsearch / Solar.

[0044] For example, if the business data consists of transaction data, it is usually stored in a relational database. However, if full-text search is needed on one or more columns of the transaction data, these columns need to be stored separately in Elasticsearch for word segmentation, retrieval, and other related functions. If high-performance point lookups are required, the transaction data needs to be loaded into Redis. In summary, this transaction data requires three types of business processing capabilities: relational database model capabilities, full-text search model capabilities, and key-value model capabilities.

[0045] To enable the transaction data in the above example to possess these three business data capabilities, the transaction data is typically imported simultaneously into the aforementioned relational database, Elasticsearch (ES), and Redis. However, since the storage space in the preset storage area is limited, importing the transaction data into these three databases simultaneously will result in insufficient storage space, leading to inefficient use of storage space. For ease of understanding, it should be noted that the databases mentioned above represent the storage model in this solution.

[0046] Therefore, this invention provides a storage method to achieve reasonable use of storage space.

[0047] like Figure 1 The diagram shown is a flowchart of a storage method provided by an embodiment of the present invention. The method includes the following steps:

[0048] Step 101: Obtain multiple target query statements.

[0049] In this embodiment of the invention, business data can be categorized into multiple types based on business type. For example, there can be five types of business data: point query, batch processing, search scenarios, complex Online Analytical Processing (OLAP), and simple OLAP. Specifically, if the business data type is point query, running it in MySQL or Redis performs well; if the business data type is search, running it in MongoDB or Elasticsearch performs well; if the business data type is simple OLAP, running it in Greenplum performs well; if the business data type is complex OLAP, running it in Spark SQL performs well; and if the business data type is batch processing, running it in Hadoop performs well. Therefore, if multiple business data can correspond to five business types, in an ideal state, to enable the business data to simultaneously possess multiple business processing capabilities, storing multiple business types in the aforementioned storage model would allow the business data to simultaneously possess multiple business processing capabilities. However, in reality, due to the limited storage space of the preset storage area, it is impossible to store all business data in the storage model. Therefore, it is necessary to select a target storage model from multiple storage models to achieve reasonable use of storage space within the limited storage space. First, multiple target query statements need to be obtained to facilitate the subsequent accurate selection of the target storage model from multiple storage models based on the relationship between the multiple target query statements and the business types.

[0050] Step 102: Based on the matching relationship between each target query statement and N business types, determine the proportion of query statements corresponding to each of the N business types.

[0051] In this embodiment of the invention, based on the matching relationship between each target query statement and N business types, the number of query statements corresponding to each of the N business types can be determined. Then, the proportion of query statements corresponding to each of the N business types can be determined based on the number of query statements corresponding to each of the N business types. For example, if there are 10 target query statements and 5 business types: point query, batch processing, search scenario, complex OLAP, and simple OLAP, where point query has 2 query statements, batch processing has 2 query statements, search has 2 query statements, scenario has 2 query statements, complex OLAP has 2 query statements, and simple OLAP has 2 query statements, then the proportion of query statements corresponding to point query is 1 / 5, the proportion of query statements corresponding to batch processing is 1 / 5, the proportion of query statements corresponding to search scenario is 1 / 5, the proportion of query statements corresponding to complex OLAP is 1 / 5, and the proportion of query statements corresponding to simple OLAP is 1 / 5.

[0052] Step 103: Based on the proportion of query statements for N business types and the execution cost of each of the N business types when stored under M storage models, determine the execution efficiency of each of the M storage models.

[0053] In this embodiment of the invention, firstly, the cost matrix of the execution cost corresponding to each of the N business types when stored under M storage models can be represented by Formula 1:

[0054]

[0055] Then, based on the proportion of query statements for each of the N business types and the execution cost of each of the N business types when stored under the M storage models, the execution efficiency of each of the M storage models can be determined. The execution efficiency of each of the M storage models is determined by Formula 2:

[0056]

[0057] Wherein, Kn represents the proportion of query statements for the business type.

[0058] Step 104: Based on the execution efficiency of each of the M storage models and the storage space occupied by each of the M storage models, determine at least one target storage model from the M storage models for storing business data of N business types.

[0059] In this embodiment of the invention, since the storage model also occupies space even when no business data is stored, and the storage space of the preset storage area is limited, in order to accurately determine the target storage model from the M storage models, it is necessary to determine at least one target storage model from the M storage models for storing business data of N business types based on the execution efficiency and storage space occupied by each of the M storage models.

[0060] As can be seen from steps 101 to 104 above, by determining the execution efficiency of each of the M storage models based on the target query statement, and considering the storage space occupied by each storage model, it is possible to accurately determine at least one target storage model from the M storage models for storing business data of N business types, based on the execution efficiency and storage space occupied by each of the M storage models. This allows for the selection of at least one target storage model within the limited storage space, thus enabling the reasonable use of storage space.

[0061] To accurately determine at least one target storage model from M storage models for storing business data of N business types, it is first necessary to determine how multiple target query statements are obtained in step 101. For ease of understanding of this solution, the method for obtaining multiple target query statements is described below.

[0062] like Figure 2 The diagram shows a flowchart of a method for obtaining multiple target query statements according to an embodiment of the present invention. The method includes the following steps:

[0063] Step 201: Obtain multiple historical query statements.

[0064] In this embodiment of the invention, since a large number of historical query statements are obtained, and these historical query statements can be divided into two categories: one category consists of query statements with high reference value, which facilitates the subsequent accurate determination of the target storage model; the other category consists of query statements with low reference value, which have little impact on the accuracy of the subsequent determination of the target storage model, but may even slow down the determination process. Therefore, after obtaining multiple historical query statements, it is first necessary to filter them to obtain the target query statement. This allows for the accurate determination of at least one target storage model from M storage models for storing N business types of business data, based on the target query statement.

[0065] Step 202: Use the target template to convert each historical query statement into a corresponding candidate query statement.

[0066] In this embodiment of the invention, since the formats of historical query statements vary widely, it is not conducive to subsequently determining the target storage model. Therefore, by adopting a target template, each historical query statement is converted into a corresponding candidate query statement, thereby unifying the format of each historical query statement. This facilitates subsequent statistics on the access frequency of historical query statements, and thus facilitates a more accurate determination of the target storage model. The target template can be an SQL template or other templates, and is not limited thereto.

[0067] Step 203: Based on the access frequency of each of the multiple candidate query statements, select multiple target query statements from the multiple candidate query statements.

[0068] In this embodiment of the invention, multiple candidate query statements can be obtained through step 202. However, since the number of candidate query statements is large, multiple target query statements can be selected from the multiple candidate query statements based on their respective access frequencies, thereby improving the accuracy of subsequent determination of the target storage model. It can be understood that, based on the access frequencies of the multiple candidate query statements, the multiple candidate query statements are clustered to obtain multiple query statement clusters; from these multiple query statement clusters, multiple target query statements are selected. Specifically, clustering is performed based on the access frequencies of the multiple candidate query statements. The clustering algorithm can be the DBSCAN algorithm or other algorithms, which are not limited here. The specific steps of the standard DBSCAN algorithm are as follows: First, the distance between all nodes and the point to be clustered is calculated to select the cluster center. This scheme only calculates the distance between the point to be clustered and the existing cluster center. The cluster center represents the entire class used for predicting workload and can reduce computational overhead, making it more suitable for online clustering scenarios. For any new SQL template, calculate the distance between its historical access frequency and existing cluster centers, and determine if it exceeds a threshold; check the distance between already clustered SQL templates and their cluster centers; if the similarity is below the threshold, remove it from the current cluster; calculate the distance between cluster centers; if the distance exceeds the threshold, merge the two clusters. This clustering process ensures that the online clustering algorithm is adaptive to the data and guarantees that the SQL templates within a cluster are always similar to their cluster centers.

[0069] As can be seen from steps 201 to 203 above, by using a target template to convert each historical query statement into a corresponding candidate query statement, the format of the historical query statements is first standardized, which facilitates the subsequent quick and accurate determination of the target query statement based on the candidate query statements. Since there are many candidate query statements, those with an access frequency greater than a first threshold are identified as target query statements, while those with an access frequency below the first threshold are deleted. This reduces the computational load for determining the target storage model based on the query statements and improves the accuracy of the subsequent determination.

[0070] In this embodiment of the invention, after obtaining the target query statement, it is necessary to determine the proportion of query statements corresponding to each of the multiple business types based on the matching relationship between the target query statement and multiple business types. This allows for a more accurate determination of the target storage model based on the query statements corresponding to each of the multiple business types. To facilitate understanding of this solution, the following describes how to determine the proportion of query statements corresponding to each of the multiple business types.

[0071] like Figure 3 The diagram shows a flowchart of a method for determining the proportion of query statements corresponding to multiple business types according to an embodiment of the present invention. The method includes the following steps:

[0072] Step 301: Based on the parse tree characteristics of each target query statement, determine the target business type of each target query statement from multiple business types.

[0073] In this embodiment of the invention, clustering is performed based on the parse tree features of each target query statement, which can determine the target business type of each target query statement from multiple business types.

[0074] Step 302: Based on the target business type corresponding to each of the multiple target query statements, determine the proportion of query statements corresponding to each of the multiple business types.

[0075] In this embodiment of the invention, based on the target business type corresponding to each of the multiple target query statements, the proportion of query statements corresponding to each of the multiple business types can be determined. For details, please refer to step 102 above, which will not be repeated here.

[0076] As can be seen from steps 301 to 302 above, by clustering based on the parse tree features of each target query statement, it is possible to determine the target business type of each target query statement and the proportion of query statements corresponding to each of the multiple business types from multiple business types. This facilitates the subsequent accurate determination of the target storage model based on the proportion of query statements corresponding to each of the multiple business types.

[0077] In this embodiment of the invention, after determining the proportion of query statements corresponding to each of the multiple business types, the execution efficiency of each of the M storage models can be determined relatively accurately based on the proportion of query statements corresponding to each of the multiple business types and the execution cost corresponding to each of the N business types when stored under the M storage models. Then, based on the execution efficiency of each of the M storage models, at least one target storage model for storing business data of the N business types can be determined relatively accurately from the M storage models. To facilitate understanding of this solution, the following describes how to determine at least one target storage model for storing business data of the N business types from the M storage models.

[0078] like Figure 4 The diagram shown is a flowchart of a method for determining a target storage model according to an embodiment of the present invention. The method includes the following steps:

[0079] Step 401: Determine the storage cost-effectiveness of each storage model based on its execution efficiency and storage space usage.

[0080] In this embodiment of the invention, since each storage model occupies a portion of storage space even when it is not storing any business data, the storage space occupied by the data information of each storage model is defined as the storage space occupied by each storage model for ease of understanding. To improve the accuracy of the storage cost-effectiveness of each storage model, the storage space occupied by each storage model needs to be considered. Specifically, by considering the execution efficiency and storage space occupied by each storage model, the storage cost-effectiveness of each storage model can be determined more accurately, as expressed by Formula 3:

[0081] C = sort(Lm / Sm) Formula 3

[0082] Where C represents the storage cost-effectiveness; Lm represents the execution efficiency corresponding to the storage model; and Sm represents the space occupied by the storage model.

[0083] Step 402: Sort the obtained storage cost-effectiveness ratios to obtain the target sorting result.

[0084] In this embodiment of the invention, the storage cost-effectiveness of each storage model can be determined through step 401 above. Then, the obtained storage cost-effectiveness values ​​are sorted to obtain the target sorting result. For example, since the storage space is limited, the total storage space is 10G. The storage models are divided into storage model A, storage model B, storage model C, and storage model D. The storage space of storage model A is 2G, the storage space of storage model B is 2G, the storage space of storage model C is 3G, and the storage space of storage model D is 3G. The service types are divided into service type E, service type F, service type G, service type H, and service type L. Service type E has better performance in storage model A, but its performance is worse than storage model A in other storage models. Service type F has better performance in storage model B, but its performance is worse than storage model B in other storage models. Service types G and H have better performance in storage model C, but their performance is worse than storage model C in other storage models. Service type L has better performance in storage model D, but its performance is worse than storage model D in other storage models. Ideally, to ensure that the business data corresponding to business types E, F, G, H, and L can be stored in a high-performance storage model, these business types would need to be stored simultaneously in storage models A, B, C, and D. However, the total storage space is limited to 10GB, with each model having 2GB, 2GB, 3GB, and 3GB of storage space. If the required storage space for the business data across multiple business types is 3GB, then storage models A and B cannot accommodate all the data. Therefore, while considering performance, the limited storage space must also be taken into account. First, the various storage models need to be sorted based on their cost-effectiveness. Then, a target sorting result is obtained. This target sorting result facilitates the determination of at least one target storage model from among the multiple storage models for storing the business data of various business types, thereby achieving reasonable use of storage space.

[0085] Step 403: Select target storage models from the M types of storage models according to the target sorting results and save them to the preset storage area until the remaining storage space in the preset storage area is less than the storage space occupied by the next added target storage model.

[0086] In this embodiment of the invention, since the storage space of the preset storage area is limited, the target storage model is first selected from M storage models according to the target sorting result and saved to the preset storage area. For example, if the storage space of the preset storage area is 10G, and there are 4 storage models, namely storage model A, storage model B, storage model C, and storage model D, storage model A has a storage space of 4G, storage model B has a storage space of 6G, storage model C has a storage space of 5G, and storage model D has a storage space of 3G. The target sorting result of these four storage models is storage model A, storage model B, storage model C, and storage model D. It can be understood that storage model A has the highest cost performance, followed by storage model B, which has a lower cost performance than storage model A, storage model C, which has a lower cost performance than storage model B, and storage model D, which has a lower cost performance than storage model C. Therefore, storage model A will first be saved to the preset storage area. The remaining storage space in the preset storage area is 10G - 4G = 6G. Since the remaining storage space in the preset storage area is the same as that of storage model B, storage model B can be saved to the preset storage area. The remaining storage space in the preset storage area is then 6G - 6G = 0G. Because the remaining storage space in the preset storage area is less than that of storage model C, both storage models A and B are stored in the preset storage area. Since the storage space in the preset storage area is limited, by determining the cost-effectiveness of the M storage models, the storage model with the higher cost-effectiveness is prioritized for storage in the preset storage area. This maximizes the performance of business data corresponding to the business type within the limited storage space, thereby achieving rational use of storage space.

[0087] Step 404: Select at least one target storage model that has been added to the preset storage area to store the business data of N business types.

[0088] In this embodiment of the invention, at least one target storage model can be determined through step 403. Then, by selecting at least one target storage model that has been added to the preset storage area to store business data of N business types, a storage model with a high cost-performance ratio is selected as the target storage model within a limited storage space. This maximizes the performance of business data corresponding to the business type in the storage model while considering storage space. Ultimately, this achieves a balance between storage and performance, enabling the rational use of storage space.

[0089] As can be seen from steps 401 to 404, by determining at least one target storage model from the M storage models based on the cost-effectiveness of the storage model, the relationship between storage and performance can be balanced within a limited storage space, thereby achieving reasonable use of storage space.

[0090] In this embodiment of the invention, to facilitate the rapid selection of multiple target query statements from multiple candidate query statements and improve the speed of determining target query statements, training samples can first be input into a classification model. This allows the training model to accurately determine the access frequency type of candidate query statements. The training samples for the classification model include a set of multiple query statements obtained through clustering based on historical access frequencies. Then, each candidate query statement is input into the classification model to determine its access frequency type. Finally, based on the access frequency types of the multiple candidate query statements, multiple target query statements are selected from the candidate query statements.

[0091] In this embodiment of the invention, if a new candidate query statement exists within the first time threshold, it will be input into the classification model to determine its access frequency type. There are two scenarios: first, the access frequency type of the new candidate query statement is a known access frequency type in the classification model; second, the access frequency type of the new candidate query statement is an unknown access frequency type in the classification model. The following sections will describe how these two different scenarios are used to accurately determine the target query statement.

[0092] First scenario

[0093] The newly added candidate query statements are input into the classification model to determine the access frequency type of the newly added candidate query statements. Then, based on the access frequency type of the newly added candidate query statements, multiple target query statements are selected from the newly added candidate query statements.

[0094] The second scenario

[0095] The newly added candidate query statement is input into the classification model. Since the access frequency type of the newly added candidate query statement is unknown in the classification model, the classification model cannot determine the access frequency type of the newly added candidate query statement. The newly added candidate query statement needs to go through steps 202 to 203 to determine multiple target query statements. For details, please refer to steps 202 and 203, which will not be repeated here.

[0096] Based on the same technical concept described above, embodiments of the present invention also provide a storage device, such as... Figure 5As shown, the device 500 includes: an acquisition unit 501, configured to acquire multiple target query statements; and a processing unit 502, configured to: determine the proportion of query statements corresponding to each of the N business types based on the matching relationship between each target query statement and N business types; determine the execution efficiency of each of the M storage models based on the proportion of query statements of the N business types and the execution cost of each of the N business types when stored under M storage models, where N and M are positive integers; and determine at least one target storage model from the M storage models for storing business data of the N business types based on the execution efficiency of each of the M storage models and the storage space occupied by each of the M storage models.

[0097] Optionally, based on the execution efficiency and storage space occupied by each of the M storage models, at least one target storage model for storing business data of N business types is determined from the M storage models, including: determining the storage cost-effectiveness of each storage model based on the execution efficiency and storage space occupied by each storage model; and determining at least one target storage model for storing business data of N business types from the M storage models based on the storage cost-effectiveness of each of the M storage models.

[0098] Optionally, the processing unit 502 is specifically used to: sort the obtained multiple storage cost-effectiveness ratios to obtain a target sorting result; select a target storage model from M storage models according to the target sorting result and save it to a preset storage area until the remaining storage space in the preset storage area is less than the storage space occupied by the next added target storage model; and select at least one target storage model that has been added to the preset storage area to store business data of N business types.

[0099] Optionally, the acquisition unit 501 is specifically used to: acquire multiple historical query statements; convert each historical query statement into a corresponding candidate query statement using a target template; and select multiple target query statements from the multiple candidate query statements based on the access frequency corresponding to each of the multiple candidate query statements.

[0100] Optionally, the acquisition unit 501 is specifically used to: cluster the multiple candidate query statements based on their respective access frequencies to obtain multiple query statement clusters; and select multiple target query statements from the multiple query statement clusters.

[0101] Optionally, the acquisition unit 501 is specifically used to: input each candidate query statement into the classification model to determine the access frequency type of each candidate query statement, wherein the training samples of the classification model include: a set of multiple query statements obtained by clustering based on historical access frequencies; and select multiple target query statements from the multiple candidate query statements based on the access frequency type of the multiple candidate query statements.

[0102] Optionally, the processing unit 502 is specifically used to: determine the target business type of each target query statement from multiple business types based on the parse tree characteristics of each target query statement; and determine the proportion of query statements corresponding to each of the multiple business types based on the target business types corresponding to each of the multiple target query statements.

[0103] Optionally, the processing unit 502 is specifically used to: perform secondary clustering on multiple target query statements based on the parse tree features of multiple target query statements, and determine the target business type of each target query statement from multiple business types based on the clustering results.

[0104] Based on the same technical concept, embodiments of this application also provide an electronic device, such as... Figure 6 As shown, the electronic device 600 includes at least one processor 601 and a memory 602 connected to the at least one processor. In this embodiment, the specific connection medium between the processor 601 and the memory 602 is not limited. Figure 6 Taking the connection between the processor 601 and the memory 602 via a bus as an example, the bus can be divided into address bus, data bus, control bus, etc.

[0105] In this embodiment of the application, the memory 602 stores instructions that can be executed by at least one processor 601. By executing the instructions stored in the memory 602, at least one processor 601 can perform the steps included in the aforementioned storage method.

[0106] The processor 601 is the control center of the computing device. It can connect to various parts of the computing device using various interfaces and lines, and performs data processing by running or executing instructions stored in the memory 602 and calling data stored in the memory 602. Optionally, the processor 601 may include one or more processing units. The processor 601 may integrate an application processor and a modem processor. The application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles issuing instructions. It is understood that the modem processor may not be integrated into the processor 601. In some embodiments, the processor 601 and the memory 602 may be implemented on the same chip; in some embodiments, they may also be implemented on separate chips.

[0107] Processor 601 can be a general-purpose processor, such as a central processing unit (CPU), digital signal processor, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the method embodiments combined with storage can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0108] Memory 602, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory 602 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic storage, magnetic disk, optical disk, etc. Memory 602 can be any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto. In the embodiments of this application, memory 602 may also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.

[0109] Based on the same technical concept, embodiments of this application also provide a computer-readable storage medium storing a computer program executable by a computing device, which, when run on the computing device, causes the computing device to perform the steps of the stored method described above.

[0110] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0111] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0112] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0113] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0114] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0115] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A storage method, characterized in that, include: Retrieve multiple target query statements; Based on the matching relationship between each target query statement and N business types, determine the proportion of query statements corresponding to each of the N business types; Based on the proportion of query statements for the N business types and the execution cost of each of the N business types when stored under the M storage models, the execution efficiency of each of the M storage models is determined, where N and M are positive integers; wherein, the execution efficiency of each of the M storage models is determined by the product of the proportion of query statements for the N business types and the execution cost of each of the N business types when stored under the M storage models. Based on the execution efficiency of each of the M storage models and the storage space occupied by each of the M storage models, at least one target storage model is determined from the M storage models to store the business data of the N business types.

2. The method as described in claim 1, characterized in that, The step of determining at least one target storage model from the M storage models for storing business data of the N business types, based on the execution efficiency and storage space occupied by each of the M storage models, includes: Based on the execution efficiency and storage space occupied by each storage model, the storage cost-effectiveness of each storage model is determined. Based on the storage cost-effectiveness of each of the M storage models, at least one target storage model is determined from the M storage models to store the business data of the N business types.

3. The method as described in claim 2, characterized in that, The step of determining at least one target storage model from the M storage models for storing business data of the N business types, based on the storage cost-effectiveness of each of the M storage models, includes: The obtained storage cost-effectiveness ratios are sorted to obtain the target sorting result; According to the target sorting result, target storage models are selected sequentially from the M storage models and saved to the preset storage area until the remaining storage space in the preset storage area is less than the storage space occupied by the next added target storage model; Select at least one target storage model that has been added to the preset storage area to store the business data of the N business types.

4. The method as described in claim 1, characterized in that, The process of obtaining multiple target query statements includes: Retrieve multiple historical query statements; Each historical query statement is converted into a corresponding candidate query statement using a target template; Based on the access frequency of each of the multiple candidate query statements, multiple target query statements are selected from the multiple candidate query statements.

5. The method as described in claim 4, characterized in that, The step of filtering out multiple target query statements based on the access frequency of each of the multiple candidate query statements includes: Based on the access frequency of each of the multiple candidate query statements, the multiple candidate query statements are clustered to obtain multiple query statement clusters. Multiple target query statements are selected from the multiple query statement families.

6. The method as described in claim 4, characterized in that, The step of filtering out multiple target query statements based on the access frequency of each of the multiple candidate query statements includes: Each candidate query statement is input into a classification model to determine the access frequency type of each candidate query statement. The training samples of the classification model include a set of multiple query statements obtained by clustering based on historical access frequencies. Based on the access frequency type of the multiple candidate query statements, multiple target query statements are selected from the multiple candidate query statements.

7. The method as described in claim 1, characterized in that, The step of determining the proportion of query statements corresponding to each of the multiple business types based on the matching relationship between each target query statement and multiple business types includes: Based on the parse tree characteristics of each target query statement, the target business type of each target query statement is determined from multiple business types; Based on the target business type corresponding to each of the multiple target query statements, the proportion of query statements corresponding to each of the multiple business types is determined.

8. The method as described in claim 7, characterized in that, The step of determining the target business type of each target query statement from multiple business types based on the parse tree characteristics of each target query statement includes: Based on the parse tree features of the multiple target query statements, the multiple target query statements are clustered in a secondary manner, and the target business type of each target query statement is determined from multiple business types based on the clustering results.

9. A storage device, characterized in that, include: The retrieval unit is used to retrieve multiple target query statements; The processing unit is used to determine the proportion of query statements corresponding to each of the N business types based on the matching relationship between each target query statement and N business types; Based on the proportion of query statements for the N business types and the execution cost of each of the N business types when stored under the M storage models, the execution efficiency of each of the M storage models is determined, where N and M are positive integers; the execution efficiency of each of the M storage models is determined by multiplying the proportion of query statements for the N business types and the execution cost of each of the N business types when stored under the M storage models; based on the execution efficiency of each of the M storage models and the storage space occupied by each of the M storage models, at least one target storage model for storing the business data of the N business types is determined from the M storage models.

10. An electronic device, characterized in that, The method includes at least one processor and at least one memory, wherein the memory stores a computer program that, when executed by the processor, causes the processor to perform the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The storage medium stores a program that, when run on a computer, causes the computer to perform any one of claims 1 to 8.

Images