Business data storage method, equipment and medium

By labeling business data with temperature and storing it in partitions, the problems of low data storage resource consumption and low query efficiency in enterprises are solved, thereby reducing costs and maximizing data value, and providing a unified data management interface and query optimization.

CN118964425APending Publication Date: 2024-11-15INSPUR GENERSOFT CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411042186.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-31
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

Enterprises face the problem of increased storage costs due to the growth of new business data, while historical data is becoming less valuable, query efficiency is low, and the value of data cannot be effectively explored.

Method used

By labeling business data with temperature, partitioning and storing it based on temperature values, and using a temperature partitioning rule table to store data in hot, warm, and cold data areas, a unified data management interface is provided, and query efficiency is optimized through a preheating task.

Benefits of technology

Reduce storage costs, improve query efficiency, maximize the value of data, provide a unified data management interface, and improve development and query efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118964425A_ABST
    Figure CN118964425A_ABST
Patent Text Reader

Abstract

The invention discloses a service data storage method and device and a medium, and the method comprises the steps: analyzing a query request, and obtaining to-be-queried service data; retrieving a temperature value of the to-be-queried business data in a pre-constructed temperature labeling relation table; the temperature labeling relation table comprises a temperature value corresponding to each piece of business data; retrieving a storage partition of the temperature value in a pre-constructed temperature partition rule table; the temperature partition rule table comprises storage partitions corresponding to different temperature intervals respectively; and querying the storage partition to obtain the target business data. According to the technical scheme, the temperature value of the business data in the database can be marked in real time, and then the business data are stored in a partitioned manner based on the temperature value of the business data, so that query is facilitated and query efficiency is improved on the basis of reducing the storage cost. In addition, temperature labeling can be carried out on the business data in real time, and the value of the business data is explored to the maximum extent.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device and medium for storing business data. Background Art

[0002] As the scale of enterprises rapidly expands and the degree of enterprise informatization continues to improve, the following problems will be faced: First, the number of new business data continues to increase, but the historical data has always been in the database, occupying a large amount of storage resources, increasing the enterprise's storage resources and operation and maintenance costs. Secondly, as the date progresses, the value of data is also decreasing. If all data is stored in one database, the enterprise will not be able to effectively explore the value of the data, which is not conducive to the long-term development of the enterprise. Finally, the most common way for enterprises to cut historical data is to select a time node, store the historical data separately on the hard disk or other file system, and re-initialize the new data and record it in the database. However, when it is necessary to query historical data, it is often complicated and cumbersome, which seriously affects work efficiency.

[0003] In summary, a simpler and more efficient technical solution for business data storage is needed. Summary of the invention

[0004] The embodiments of the present application provide a business data storage method, device and medium for solving the problem of needing a simpler and more efficient technical solution for business data storage.

[0005] The present application embodiment adopts the following technical solutions:

[0006] On the one hand, an embodiment of the present application provides a business data storage method, the method comprising: parsing a query request to obtain business data to be queried; retrieving the temperature value of the business data to be queried in a pre-constructed temperature annotation relationship table; the temperature annotation relationship table includes the temperature values ​​corresponding to each business data; retrieving the storage partition of the temperature value in a pre-constructed temperature partition rule table; the temperature partition rule table includes storage partitions corresponding to different temperature intervals; and querying the storage partition to obtain target business data.

[0007] In one example, before parsing the query request to obtain the business data to be queried, the method also includes: generating a storage rule base; the storage rule base is used to represent the business status temperature value of each business status, the timeliness temperature value of different time intervals, the browsing volume temperature value of different browsing volume intervals, the temperature coefficient value of different business statuses, and the storage partitions of different temperature value intervals; for each business data in the database, matching is performed in the storage rule base to determine the business status temperature value, timeliness temperature value, browsing volume temperature value and temperature coefficient value of each business data; according to their respective temperature coefficient values, the business status temperature value, the timeliness temperature value and the browsing volume temperature value are weighted and summed to obtain the temperature value of each business data; according to the temperature value of each business data, a temperature annotation relationship table is generated; based on the temperature value of each business data, the storage partition of the temperature value of each business data is determined, and each data is stored in the corresponding storage partition.

[0008] In one example, the generation of a storage rule base specifically includes: generating a business status temperature value rule table; the fields of the business status temperature value rule table include the business status name and the business status temperature value; generating a data timeliness temperature value rule table; the fields of the data timeliness temperature value rule table include the start time, the end time and the timeliness temperature value; generating a data browsing volume temperature value rule table; the fields of the data browsing volume temperature value rule table include the minimum number of browsing times, the maximum number of browsing times and the browsing volume temperature value; generating a dimension coefficient rule table; the fields of the dimension coefficient rule table include the dimension name and the temperature coefficient value; the dimension name includes the business status, timeliness and browsing volume; generating a temperature partition rule table; the fields of the temperature partition rule table include the minimum temperature value, the maximum temperature value, the storage partition and the storage capacity.

[0009] In one example, the method further includes: marking storage partitions as hot data partitions, warm data partitions, and cold data partitions; the temperature value of the hot data partition is higher than the temperature value of the warm data partition, and the temperature value of the warm data partition is higher than the temperature value of the cold data partition.

[0010] In one example, after storing each piece of data in the corresponding storage partition, the method further includes: determining the table structure of the preheating task data table; the fields of the table structure include the preheating task unique identifier, the preheating task execution status, the SQL statement that needs to be preheated, the preheating task start execution time, the preheating task end execution time, the preheating task execution frequency and the cache data retention time; the SQL statement is used to cache the business data of the cold data partition to the warm data partition; obtain the relevant information of a single preheating task, and fill the relevant information of the single preheating task into the preheating task table; according to the preheating task table, obtain the business data that needs to be cached to the warm data partition from the cold data partition.

[0011] In one example, according to the preheating task table, the business data that needs to be cached to the warm data partition is obtained from the cold data partition, specifically including: for a single preheating task, when the preheating task execution start time is reached, a SQL statement query is performed according to the preheating task execution frequency to query the cold data partition for the business data to be cached, and the business data to be cached is cached to the warm data partition, and a cache identifier for the cached business data is generated in the warm data partition.

[0012] In one example, the querying of the storage partition to obtain the target business data specifically includes: determining that the storage partition of the business data to be queried has a cold data partition; judging whether the business data stored in the cold data partition has a cache identifier in the warm data partition; if so, querying the business data cached in the warm data partition to obtain the target business data.

[0013] In one example, the query in the business data cached in the warm data partition to obtain the target business data specifically includes: querying the business data cached in the warm data partition to obtain first business data; determining that the storage partition of the business data to be queried has a hot data partition and a warm data partition; querying in the hot data partition to obtain second business data, and querying in the warm data partition to obtain third business data; and obtaining the target business data for the first business data, the second business data, and the third business data.

[0014] On the other hand, an embodiment of the present application provides a business data storage device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute a business data storage method described in any one of the above.

[0015] On the other hand, an embodiment of the present application provides a non-volatile computer storage medium for storing business data, which stores computer-executable instructions, and the computer-executable instructions can execute any of the business data storage methods described above.

[0016] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects:

[0017] It can annotate the temperature value of the business data in the database in real time, and then partition and store the business data based on the temperature value of the business data, so as to facilitate query and improve query efficiency while reducing storage costs. In addition, a unified data management interface is provided to the outside world, and the data interface is transparent to the outside world. There is no need to pay attention to the partitioning and storage methods corresponding to business data with different temperatures, which can also greatly improve development efficiency and query efficiency. In addition, it can annotate the temperature of business data in real time to maximize the value of business data. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solution of the present application, some embodiments of the present application will be described in detail below in conjunction with the accompanying drawings, in which:

[0019] Figure 1 A flowchart of a business data storage method provided in an embodiment of the present application;

[0020] Figure 2 A data flow diagram of a business data storage method provided in an embodiment of the present application;

[0021] Figure 3 A schematic diagram of the structure of a business data storage device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0023] Some embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0024] Figure 1 A flowchart of a business data storage method provided in an embodiment of the present application. The method can be applied to different business fields, such as Internet financial business field, e-commerce business field, instant messaging business field, game business field, official business field, etc. The process can be executed by a computing device in the corresponding field, and some input parameters or intermediate results in the process allow manual intervention and adjustment to help improve accuracy.

[0025] The analysis method involved in the embodiments of the present application can be implemented by a terminal device or a server, and the present application does not impose any special restrictions on this. For the convenience of understanding and description, the following embodiments are described in detail by taking a server as an example.

[0026] It should be noted that the server may be a single device or a system consisting of multiple devices, that is, a distributed server, and this application does not make any specific limitation on this.

[0027] Figure 1 The process in includes the following steps:

[0028] S101: Parse the query request to obtain the business data to be queried.

[0029] It should be noted that a SQL statement corresponding to the query request is parsed to obtain the business data to be queried. The business data to be queried may be one piece of business data or multiple pieces of business data.

[0030] In some embodiments of the present application, the temperature value of each business data is comprehensively evaluated based on the three dimensions of data business status, data timeliness and data browsing volume. Based on this, it is necessary to pre-build a storage rule base, which includes a business status temperature value rule table, a data timeliness temperature value rule table, a data browsing volume temperature value rule table, a dimension coefficient rule table, and a temperature partition rule table.

[0031] It should be noted that the business status temperature value rule table is used to represent the business status temperature value of each business status, the data timeliness temperature value rule table is used to represent the timeliness temperature values ​​of different time intervals, the data browsing volume temperature value rule table is used to represent the browsing volume temperature values ​​of different browsing volume intervals, the dimension coefficient rule table is used to represent the temperature coefficient values ​​of different business statuses, and the temperature partition rule table is used to represent storage partitions of different temperature value intervals.

[0032] Among them, the fields of the business state temperature value rule table include business state names and business state temperature values. For example, as shown in Table 1, the fields of Table 1 include business state names, enumeration values, and business state temperature values.

[0033] Table 1

[0034]

[0035]

[0036] It should be noted that for business data without business status, the system default temperature value is 0.

[0037] In addition, the fields of the data timeliness temperature value rule table include a start time, an end time, and a timeliness temperature value, for example, as shown in Table 2.

[0038] Table 2

[0039] Start time End Time Aging temperature value Null Null 0 2010 / 1 / 1 00:00 2020 / 12 / 31 24:00 10 2021 / 1 / 1 00:00 2022 / 12 / 31 24:00 35 2023 / 1 / 1 00:00 2023 / 12 / 31 24:00 50 2024 / 1 / 1 00:00 2024 / 12 / 31 24:00 70

[0040] It should be noted that for data without a time field description, the system defaults to a temperature value of 0. The more recently generated business data, that is, the later the start time, the higher the timeliness, and the higher the corresponding timeliness temperature value. For example, in an order management system, order data may have start time and end time fields. The start time can indicate the creation time of the order, and the end time can indicate the completion time or expiration time of the order.

[0041] In addition, the fields of the data browsing temperature value rule table include the minimum browsing times, the maximum browsing times and the browsing temperature value. For example, as shown in Table 3.

[0042] Table 3

[0043] Minimum number of views Highest Views Views Temperature Value NULL NULL 0 0 100 10 101 1000 20 1001 10000 30 10001 1000000000 70

[0044] It should be noted that for data without pageview description, the system defaults to a pageview temperature value of 0. Among them, the higher the average pageview volume of the business data in the pageview interval, the higher the pageview temperature value.

[0045] It should be noted that the page views of the business data may be updated regularly, for example, the page views of the business data may be recalculated every month.

[0046] In addition, the fields of the dimension coefficient rule table include dimension names and temperature coefficient values. The dimension names include business status, timeliness, and pageviews. For example, as shown in Table 4.

[0047] Table 4

[0048] Dimension name Temperature coefficient value Business Status 50% Timeliness 30% Views 20%

[0049] It should be noted that different temperature values ​​and temperature coefficient values ​​can be preset for different scenarios and different types of business data. For example, for some data without business status, the business status temperature coefficient value can be set to 0, so that the data temperature labeling is determined only according to the two dimensions of timeliness and page views.

[0050] In addition, the fields of the temperature partition rule table include the minimum temperature value, the maximum temperature value, the storage partition and the storage capacity, as shown in Table 5, for example.

[0051] Table 5

[0052] Minimum temperature value Maximum temperature value Storage Partitions Storage capacity 70 100 Memory 512M 40 69 Server Disk 20T 0 39 S3, HDFS 80T

[0053] It should be noted that when data is stored in memory, the maximum occupied memory capacity can be configured; the memory management algorithm can adopt an improved LRU algorithm.

[0054] Based on this, it is necessary to calculate the temperature value for each business data in the database. The process is as follows:

[0055] First, for each piece of business data in the database, a match is performed in the storage rule base to determine the business status temperature value, timeliness temperature value, pageview temperature value and temperature coefficient value of each piece of business data.

[0056] That is to say, according to the business status name of each business data, the business status temperature value rule table is matched to obtain the business status temperature value. In addition, according to the start time and end time of each business data, the data timeliness temperature value rule table is matched to determine the time interval in which each is located, thereby determining the timeliness temperature value of the time interval. In addition, according to the data browsing volume of each business data, the data browsing volume temperature value rule table is matched to determine the browsing volume interval in which each is located, thereby determining the browsing volume temperature value of the browsing volume interval.

[0057] Then, according to their respective temperature coefficient values, the business status temperature value, timeliness temperature value and pageview temperature value are weighted and summed to obtain the temperature value of each business data. For example, the data temperature calculation formula is: data temperature = business status temperature value * business status temperature coefficient value + data timeliness temperature value * timeliness temperature coefficient value + data pageview temperature value * pageview temperature coefficient value.

[0058] Then, according to the temperature value of each business data, a temperature labeling relation table is generated. The fields of the temperature labeling relation table include the unique identifier of the business data and the temperature value, so as to realize temperature labeling of the business data.

[0059] Finally, based on the storage rule base, the storage partition of the temperature value of each business data is determined, and each data is stored in the corresponding storage partition.

[0060] The temperature value of each piece of business data is matched in the temperature partition rule table to determine the temperature interval in which the temperature value is located, thereby determining the storage partition of the temperature interval.

[0061] It should be noted that the storage partitions are marked as hot data partitions, warm data partitions, and cold data partitions. The temperature value of the hot data partition is higher than the temperature value of the warm data partition, and the temperature value of the warm data partition is higher than the temperature value of the cold data partition. That is, it is equivalent to marking the corresponding business data as hot data, warm data, and cold data.

[0062] For example, the hot data partition is stored in memory, the warm data partition is stored in server disk, and the cold data partition is stored in S3 or HDFS.

[0063] It should be noted that when the business status, time interval, page views and other dimensions of a single piece of business data in the database change, it is necessary to recalculate the temperature value of the single piece of business data, that is, re-label the temperature value of the business data, so as to update the storage partition of the single piece of business data.

[0064] S102: Retrieve the temperature value of the business data to be queried from a pre-constructed temperature annotation relationship table; the temperature annotation relationship table includes the temperature value corresponding to each piece of business data.

[0065] S103: Retrieving the storage partition of the temperature value in a pre-constructed temperature partition rule table; the temperature partition rule table includes storage partitions corresponding to different temperature intervals.

[0066] S104: Query the storage partition to obtain target business data.

[0067] In some embodiments of the present application, a preheating task is set to load the business data of the cold data partition to the warm data partition in advance before the business data is actually queried, thereby reducing the delay during query, improving the performance of the system, and optimizing the efficiency of cold data query. It should be noted that when the data query request hits the cold data, the system will automatically perform preheating work and cache the cold data from S3 and HDFS to the server disk. The preheating data can limit the storage capacity and clean up the preheating data through the LRU algorithm.

[0068] Based on this, it is necessary to pre-build a preheating task data table. The fields of the table structure include the preheating task unique identifier, the preheating task execution status, the SQL statement that needs to be preheated, the preheating task start execution time, the preheating task end execution time, the preheating task execution frequency and the cache data retention time; the SQL statement is used to cache the business data of the cold data partition to the warm data partition.

[0069] More intuitively, for example, the table structure of the warm-up task data table is shown in Table 6.

[0070] Table 6

[0071] Fields Reference Field Type Business Rules ID LARGEINT Unique ID of the warm-up task State INT Warm-up task execution status, 0 not in progress; 1 completed; 2 abnormal QueryTxt Varchar(4000) SQL statements that need to be preheated for execution Start Date Warm-up task start time End Date Warm-up task end execution time Frequency Varchar(64) Warm-up task execution frequency Retention Int Cache data retention time (in minutes)

[0072] Based on this, the warm-up tasks are filled into the warm-up task table. The process is as follows:

[0073] First, obtain the relevant information of a single warm-up task and fill the relevant information of the single warm-up task into the warm-up task table. Then, according to the warm-up task table, obtain the business data that needs to be cached to the warm data partition from the cold data partition.

[0074] It should be noted that the relevant information refers to the field content required by the table structure field.

[0075] Among them, when obtaining business data that needs to be cached to the warm data partition from the cold data partition, for a single preheating task, when the preheating task execution start time is reached, a SQL statement query is performed according to the preheating task execution frequency to query the cold data partition for the business data to be cached, and the business data to be cached is cached to the warm data partition, and a cache identifier for the cached business data is generated in the warm data partition.

[0076] It should be noted that the preheated data is managed using the LRU algorithm. When the cache data retention time is reached, the system automatically destroys the cache in the cold data partition to release storage space.

[0077] Based on this, the process of querying the storage partition to obtain the target business data is as follows:

[0078] Make sure that the storage partition of the business data to be queried has a cold data partition.

[0079] Determine whether the business data stored in the cold data partition has a cache identifier in the warm data partition.

[0080] If so, query the business data in the warm data partition cache to obtain the first business data.

[0081] In addition, when it is determined that the storage partition of the business data to be queried has a hot data partition and a warm data partition, a query is performed in the hot data partition to obtain the second business data, and a query is performed in the warm data partition to obtain the third business data.

[0082] Finally, target business data is obtained from the first business data, the second business data and the third business data.

[0083] It should be noted that when the business data stored in the cold data partition does not have a cache identifier in the warm data partition, the cold data partition is directly queried to obtain the first business data.

[0084] At present, the information system of enterprises has the following problems: 1. With the expansion of enterprise scale and the continuous improvement of enterprise informatization, business data is growing rapidly. Massive historical data is stored in the business database, which occupies a large amount of storage resources and increases enterprise costs. 2. All business data is stored in one database, resulting in a decrease in daily reading and writing performance, affecting work efficiency. 3. All business data is stored in one database, which also takes a lot of time for data backup and fault recovery. 4. All business data is stored in one database, and it is impossible to distinguish between high-value data and low-value data, which is not conducive to the maximum exploration of data value by enterprises.

[0085] Based on this, the technical effects of this application are as follows: First, it can perform temperature marking on business data in real time, divide business data into hot data, warm data and cold data, and store them in categories. Hot data is stored in memory, warm data is stored on server disks, and cold data is stored in S3 or other file systems such as HDFS, which can effectively reduce data storage costs.

[0086] The second point: It can provide a unified data management interface to the outside world. The data interface is transparent to the outside world, and there is no need to pay attention to the partitioning and storage methods corresponding to data of different temperatures, which can greatly improve development efficiency and query efficiency.

[0087] The third point: It can label business data in real time, maximize the value of business data, and help enterprises make decisions using data as a handle.

[0088] Fourth point: Temperature labeling is performed based on dimensions such as data business status, so that data from different periods can be labeled with different temperatures. This does not require the use of complex prediction models and artificial intelligence models, making it convenient for users to configure and implement them. It is not a simple one-size-fits-all model that is segmented based on a certain historical moment node. It can maintain the correlation between data and facilitate query.

[0089] Fifth point: It can preheat data for specific business scenarios, cache cold data in advance, and improve query efficiency.

[0090] It should be noted that although the embodiments of the present application are based on Figure 1 Steps S101 to S104 are described in sequence, but this does not mean that steps S101 to S104 must be performed in a strict order. Figure 1 The order shown in the figure is to introduce and explain step S101 to step S104 in sequence, in order to facilitate those skilled in the art to understand the technical solution of the embodiment of the present application. In other words, in the embodiment of the present application, the order between step S101 to step S104 can be appropriately adjusted according to actual needs.

[0091] pass Figure 1 The method can mark the temperature value of the business data in the database in real time, and then partition and store the business data based on the temperature value of the business data, so as to facilitate query and improve query efficiency while reducing storage costs. In addition, a unified data management interface is provided to the outside world, and the data interface is transparent to the outside world. There is no need to pay attention to the partition and storage method corresponding to business data with different temperatures, which can also greatly improve development efficiency and query efficiency. In addition, the temperature of business data can be marked in real time to maximize the value of business data.

[0092] More intuitively, Figure 2A data flow diagram of a business data storage method provided in an embodiment of the present application.

[0093] exist Figure 2 In the process, the data query module obtains the query data request, then parses the query request, queries the data in the cold, warm, hot and preheated data, thereby obtaining the organization query result, and finally returns the result.

[0094] It should be noted that the data preheating module preheats the cold data to cache and obtain preheated data.

[0095] Based on the same idea, some embodiments of the present application also provide devices and non-volatile computer storage media corresponding to the above methods.

[0096] Figure 3 A schematic diagram of the structure of a business data storage device provided in an embodiment of the present application includes:

[0097] at least one processor; and,

[0098] a memory communicatively connected to the at least one processor; wherein,

[0099] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any one of the business data storage methods described above.

[0100] Some embodiments of the present application provide a non-volatile computer storage medium for storing business data, which stores computer-executable instructions, and the computer-executable instructions can execute any of the business data storage methods described above.

[0101] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0102] The devices and media provided in the embodiments of the present application correspond one-to-one to the methods. Therefore, the devices and media also have similar beneficial technical effects as the corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media will not be repeated here.

[0103] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0104] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0105] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0106] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0107] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0108] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0109] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0110] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0111] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the technical principle of the present application should all fall within the protection scope of the present application.

Claims

1. A business data storage method, characterized in that: The method comprises: Parse the query request to obtain the business data to be queried; Retrieving the temperature value of the business data to be queried in a pre-constructed temperature annotation relationship table; the temperature annotation relationship table includes the temperature value corresponding to each business data; Retrieving the storage partition of the temperature value in a pre-constructed temperature partition rule table; the temperature partition rule table includes storage partitions corresponding to different temperature intervals; A query is made to the storage partition to obtain target business data.

2. The method according to claim 1, characterized in that Before parsing the query request to obtain the service data to be queried, the method further includes: Generate a storage rule base; the storage rule base is used to represent the business state temperature value of each business state, the timeliness temperature value of different time intervals, the browsing volume temperature value of different browsing volume intervals, the temperature coefficient value of different business states and the storage partitions of different temperature value intervals; For each piece of business data in the database, matching is performed in the storage rule base to determine the business status temperature value, timeliness temperature value, pageview temperature value and temperature coefficient value of each piece of business data; According to their respective temperature coefficient values, the business status temperature value, timeliness temperature value and pageview temperature value are weighted and summed to obtain the temperature value of each business data; Generate a temperature annotation relationship table based on the temperature value of each piece of business data; Based on the storage rule base, a storage partition for the temperature value of each piece of business data is determined, and each piece of data is stored in a corresponding storage partition.

3. The method according to claim 2, characterized in that The generating and storing rule base specifically includes: Generate a business status temperature value rule table; the fields of the business status temperature value rule table include a business status name and a business status temperature value; Generate a data timeliness temperature value rule table; the fields of the data timeliness temperature value rule table include a start time, an end time and a timeliness temperature value; Generate a data browsing volume temperature value rule table; the fields of the data browsing volume temperature value rule table include a minimum browsing number, a maximum browsing number and a browsing volume temperature value; Generate a dimension coefficient rule table; the fields of the dimension coefficient rule table include dimension name and temperature coefficient value; the dimension name includes business status, timeliness and pageviews; Generate a temperature partition rule table; the fields of the temperature partition rule table include a minimum temperature value, a maximum temperature value, a storage partition, and a storage capacity.

4. The method according to claim 2, characterized in that: The method further comprises: The storage partitions are marked as hot data partitions, warm data partitions and cold data partitions; the temperature value of the hot data partition is higher than the temperature value of the warm data partition, and the temperature value of the warm data partition is higher than the temperature value of the cold data partition.

5. The method according to claim 4, characterized in that After storing each piece of data in a corresponding storage partition, the method further includes: Determine the table structure of the preheating task data table; the fields of the table structure include the preheating task unique identifier, the preheating task execution status, the SQL statement that needs to be preheated, the preheating task start execution time, the preheating task end execution time, the preheating task execution frequency and the cache data retention time; the SQL statement is used to cache the business data of the cold data partition to the warm data partition; Obtain relevant information of a single preheating task, and fill the relevant information of the single preheating task into the preheating task table; According to the preheating task table, the business data that needs to be cached to the warm data partition is obtained from the cold data partition.

6. The method according to claim 5, characterized in that The acquiring, from the cold data partition, the business data that needs to be cached to the warm data partition according to the preheating task table specifically includes: For a single preheating task, when the preheating task execution start time is reached, a SQL statement query is performed according to the preheating task execution frequency to query the cold data partition for the business data to be cached, and the business data to be cached is cached to the warm data partition, and a cache identifier for the cached business data is generated in the warm data partition.

7. The method according to claim 6, characterized in that The querying the storage partition to obtain the target business data specifically includes: Determine that the storage partition of the business data to be queried has a cold data partition; Determine whether the business data stored in the cold data partition has a cache identifier in the warm data partition; If so, query the business data in the warm data partition cache to obtain the target business data.

8. The method according to claim 7, characterized in that The querying of the business data in the warm data partition cache to obtain the target business data specifically includes: Query the business data in the warm data partition cache to obtain first business data; Determine that the storage partition of the business data to be queried has a hot data partition and a warm data partition; Performing a query in the hot data partition to obtain the second business data, and performing a query in the warm data partition to obtain the third business data; For the first service data, the second service data and the third service data, target service data is obtained.

9. A business data storage device, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute a business data storage method as described in any one of claims 1 to 8.

10. A business data storage non-volatile computer storage medium storing computer executable instructions, characterized in that: The computer executable instructions can execute a business data storage method as described in any one of claims 1 to 8.