Data processing method, medium, and program

Abstract

The invention relates to the technical field of data processing, in particular to a data processing method, a medium and a program.The method comprises the steps that a to-be-stored data set and a data identifier corresponding to each piece of to-be-stored data are obtained; according to the data identifier, utilizing a preset algorithm to calculate a partition value to which the to-be-stored data belongs in the storage space, and storing the to-be-stored data to a region corresponding to the partition value to which the to-be-stored data belongs; the storage duration of the to-be-stored data is recognized, and if the storage duration meets the first preset duration, the to-be-stored data of the preset duration before the current moment is archived to generate a preset file to the storage space. Therefore, the problems that in the related technology, a large amount of data occupies more resources, a user is not supported to check historical archived data online, a special interface needs to be inquired and the like, and the user experience feeling is poor are solved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data processing method, medium and program. Background Art

[0002] Any company will face a large amount of non-hot data. This data is usually not accessed for a long time or not accessed at all after being used by users. For example, data, in-site messages, notifications, etc. are usually only accessed in the recent period when the user receives the data. In other time periods, the data is accessed infrequently or even not accessed at all. This data takes up a lot of storage space, and the company will also spend a high cost on this storage.

[0003] In related technologies, data is mostly manually archived by operations personnel at specific time points, such as once a year, for specific business data. The methods adopted are: (1) importing data from the existing storage system to the same but low-performance and inexpensive storage system; if users need to view previous data, they need to redevelop a new page or go to a specific page to view it; (2) directly storing data in the file system, which does not support online viewing of historical archived data and requires users to apply and manually process it. Summary of the invention

[0004] The present application provides a data processing method, medium and program to solve the problems in related technologies such as large amounts of data occupying more resources, not supporting users to view historical archived data online, requiring a special query interface, etc., resulting in a poor user experience.

[0005] A first aspect of the present application provides a data processing method, which is applied to a server, wherein the method includes the following steps: obtaining a data set to be stored and a data identifier corresponding to each data to be stored; calculating the partition value of the data to be stored in the storage space using a preset algorithm according to the data identifier, and storing the data to be stored in the area corresponding to the partition value; identifying the storage duration of the data to be stored, and if the storage duration meets a first preset duration, archiving the data to be stored for the preset duration before the current moment to generate a preset file to the storage space.

[0006] Optionally, archiving the data to be stored for a preset length of time before the current moment includes: creating a message archiving main task, using the message archiving main task to generate statistical results for the total amount of data to be stored in the month to be archived; dividing the message archiving sub-tasks according to the statistical results, and using each message archiving sub-task to process a preset amount of data to be stored.

[0007] Optionally, using each message archiving subtask to process a preset amount of data to be stored includes: identifying the archiving month and archiving area of ​​the message archiving subtask; querying the data to be stored and writing it into a temporary message file based on the archiving month and archiving area; and compressing the temporary message file and uploading it to the storage space.

[0008] Optionally, after compressing the temporary message file and uploading it to the storage space, it includes: identifying the save file path of the temporary message file after it is compressed and uploaded to the storage space; writing the save file path into the archiving subtask table for storage; wherein the temporary message file includes data identification, data title, data content, partition value, data creation time and data modification time.

[0009] Optionally, after generating a preset file to the storage space for archiving the data to be stored for a preset length of time before the current moment, it includes: performing interval sampling detection on the archived data to be stored; and triggering a corresponding target operation according to the sampling detection result.

[0010] The second aspect of the present application provides a data processing method, which is applied to a client, wherein the method includes the following steps: identifying historical data that a user needs to view at the current moment; and triggering thawing logic to thaw the corresponding historical data according to the current moment.

[0011] Optionally, the unfreezing logic triggered according to the current moment to unfreeze the corresponding historical data includes: determining the month of the historical data that needs to be unfrozen according to the current moment; if the month of the historical data meets a preset condition, calculating the area of ​​the storage space where the historical data is located according to the data identifier, querying the archive file address of the historical data according to the data identifier to obtain the compressed historical data, and storing the compressed historical data in a temporary unfreezing data table after unfreezing.

[0012] Optionally, it also includes: obtaining the actual intention of the user; if the actual intention is a query intention, identifying the user's query requirements, and filtering out historical data of a corresponding time range according to the query requirements.

[0013] The third aspect of the present application provides a computer-readable storage medium on which a computer program is stored. The program is executed by a processor to perform the data processing method described in the above embodiment.

[0014] The fourth aspect of the present application provides a computer program product, including a computer program or instructions, characterized in that when the computer program or instructions are executed, the data processing method described in the above embodiment is implemented.

[0015] Therefore, this application has at least the following beneficial effects:

[0016] (1) The embodiments of the present application can specify corresponding groups for low-frequency data with huge data volume, use distributed and multi-threaded technology to realize rapid data processing, and achieve regular and rapid data archiving.

[0017] (2) After completing data archiving, the embodiment of the present application can perform random checks on the archived files and data to ensure the correctness of the data archiving.

[0018] (3) The embodiments of the present application can use an efficient file compression mechanism to compress archived files, thereby saving file storage space and costs to the greatest extent.

[0019] (4) The embodiments of the present application can efficiently implement a data thawing and recovery mechanism, enable users to operate without being aware of the current situation, improve user experience, and restore data that users need to view and has been archived, thereby improving processing efficiency.

[0020] (5) The embodiments of the present application can be customized by configuring the archived data according to the user's different business needs.

[0021] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0023] Figure 1 A flowchart of a data processing method provided according to an embodiment of the present application;

[0024] Figure 2 A data storage flow chart provided according to an embodiment of the present application;

[0025] Figure 3 A flowchart of data archiving provided according to an embodiment of the present application;

[0026] Figure 4 A flowchart of file compression provided according to an embodiment of the present application;

[0027] Figure 5 A flowchart of an archive check provided according to an embodiment of the present application;

[0028] Figure 6 A flowchart of a data processing method provided according to another embodiment of the present application;

[0029] Figure 7 A schematic diagram of a data thawing time spectrum provided according to an embodiment of the present application;

[0030] Figure 8 A data thawing triggering flow chart provided according to an embodiment of the present application;

[0031] Fig. 9 A flowchart of data thawing provided according to an embodiment of the present application;

[0032] Fig.10 The present invention provides a flowchart of data retrieval and thawing according to an embodiment of the present application. DETAILED DESCRIPTION

[0033] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0034] The data processing method, storage medium and program of the embodiments of the present application are described below with reference to the accompanying drawings.

[0035] Specifically, Figure 1 A flowchart of a data processing method provided by an embodiment of the present application.

[0036] like Figure 1 As shown, the data processing method is applied to a server, wherein the method comprises the following steps:

[0037] In step S101, a data set to be stored and a data identifier corresponding to each data to be stored are obtained.

[0038] It is understandable that the embodiment of the present application can obtain the data set to be stored and the data identifier corresponding to each data to be stored, so as to subsequently calculate the partition value of the data to be stored in the storage space according to the data identifier using a preset algorithm to store the data.

[0039] In step S102, a partition value to which the data to be stored belongs in the storage space is calculated using a preset algorithm according to the data identifier, and the data to be stored is stored in an area corresponding to the partition value.

[0040] It can be understood that the embodiment of the present application can calculate the partition value of the storage space to which the data to be stored belongs using a preset algorithm based on the data identifier, and store the data to be stored in the area corresponding to the partition value, so as to facilitate the subsequent storage of the data to be stored in the corresponding area.

[0041] It should be noted that the data is stored in a tablestore grid database and is divided into intervals according to 2 to the 16th power, that is, 65536. Taking 3 million customers as an example, using the hash algorithm, there are an average of 76 accounts in each interval.

[0042] Assuming that 50 pieces of data are generated every day, 50 (pieces) x 30 (days) x 3 million (million customers) = 4.5 billion, that is, 4.5 billion pieces of data can be generated every month; the average number of pieces of data per interval per month is roughly: 50 (pieces) x 30 (days) x 76 = 114,000; each piece of data is calculated at 1kb, so the data size in each interval is 112M, and 7168G of data will be generated every month.

[0043] Specifically, if Figure 2 As shown, before the app data is stored in the database, the APP will generate a message push, and then the CityHash64 algorithm is used to quickly calculate the partition value to which the data belongs according to the customer ID to which the data belongs, and store it.

[0044] For example, taking account 50005310002 as an example, the partition calculated by the CityHash64 algorithm for this account is 59557. By analogy, the interval where the account is located is distributed in [0,65535]. In this way, data grouping is completed, which is convenient for subsequent grouping for data archiving and thawing.

[0045] In step S103, the storage duration of the data to be stored is identified. If the storage duration satisfies a first preset duration, the data to be stored for the preset duration before the current moment is archived to generate a preset file in the storage space.

[0046] Among them, the first preset period can be 6 months, which can be set according to actual needs without specific limitation.

[0047] It can be understood that the embodiment of the present application identifies the storage duration of the data to be stored. If the storage duration meets the first preset duration, the data to be stored for the preset duration before the current moment is archived to generate a preset file to the storage space, so as to achieve regular and rapid data archiving and improve data processing efficiency.

[0048] For example, on the 18th of every month, data from six months ago will be archived, and only one month will be archived each time. For example, if it is November 2023, the data from May 2020 will be archived. In October 2023, the data from April 2023 will be archived, and so on.

[0049] In an embodiment of the present application, archiving data to be stored for a preset time before the current moment includes: creating a message archiving main task, using the message archiving main task to count the total amount of data to be stored in the month to be archived to generate statistical results; dividing a plurality of message archiving subtasks according to the statistical results, and using each message archiving subtask to process a preset amount of data to be stored.

[0050] It can be understood that the embodiment of the present application can create a message archiving main task, and use the message archiving main task to generate statistical results by counting the total amount of data to be stored in the month to be archived; divide multiple message archiving sub-tasks according to the statistical results, and use each message archiving sub-task to process a preset amount of data to be stored, thereby using distributed and multi-threaded technology to realize rapid data processing, achieve regular and rapid data archiving, and improve data processing efficiency.

[0051] Specifically, if Figure 3 As shown, when data archiving starts, a data archiving main task is created to count the total amount of data in the archived month, and a data archiving subtask is created. If 16 partitions are used as one subtask, 65536 / 16=4096 subtasks are required.

[0052] The number of data archived by each subtask is approximately: 114,000 x 16 = 1.824 million data items; when each subtask is created, the actual number of data that needs to be archived and the interval (partition) that needs to be archived are counted, for example: subtask 1, the data interval that needs to be archived (0, 1, 2, ....., 15), subtask 2, the interval that needs to be archived (16, 17, 18, ..., 31), and so on; among them, the archiving intervals of subtasks may not be adjacent.

[0053] Start the archiving task, and distribute 4096 subtasks to different service instances through ons. For example, 8 services will execute 512 data archiving subtasks for each service. The number of archiving subtasks executed by each service is actually related to the server configuration, performance, and number of instances. The actual processing capacity is the standard. In theory, the more server instances there are, the faster the data is archived. However, service instances cannot be expanded infinitely. Too many will waste resources, and too few will affect the archiving speed. Function computing and dynamic expansion technology can be used to solve the problem of dynamic server expansion.

[0054] It should be noted that Function Compute: performs data query (data query from tablestore) based on the archiving month and partition to be archived on the archiving subtask, generates temporary data files, and writes data. The data files are compressed and uploaded to OSS, with one file per partition per month, for example: http: / / xxxxxx / 2023 / 05 / 59557.csv, with the file directory and year / month / partition.csv as the path. In each file, each line of data is a complete data json, for example: {"accountId":"xxx","title":"xxx","content":"xxx","partition":"","gmtCreated":"xxx","gmtModified":"xxx"}.

[0055] In an embodiment of the present application, each message archiving subtask is used to process a preset amount of data to be stored, including: identifying the archiving month and archiving area of ​​the message archiving subtask; querying the data to be stored according to the archiving month and archiving area and writing them into a temporary message file; and compressing the temporary message file and uploading it to the storage space.

[0056] It can be understood that the embodiments of the present application can identify the archiving month and archiving area of ​​the message archiving subtask; query the data to be stored according to the archiving month and archiving area and write them into a temporary message file; compress the temporary message file and upload it to the storage space, thereby saving a lot of storage costs.

[0057] In an embodiment of the present application, after the temporary message file is compressed and uploaded to the storage space, it includes: identifying the save file path of the temporary message file after it is compressed and uploaded to the storage space; writing the save file path into the archiving subtask table for storage; wherein the temporary message file includes data identification, data title, data content, partition value, data creation time and data modification time.

[0058] It can be understood that the embodiment of the present application can identify the saved file path of the temporary message file after it is compressed and uploaded to the storage space; and write the saved file path into the archiving subtask table for storage to facilitate searching.

[0059] Specifically, if Figure 4As shown in the figure, APP data is read from the tablestore database and written into temporary files through the data archiving task. In order to solve the storage space problem, the files need to be compressed. This system uses zip file compression technology to compress the files and upload them to the OSS system. At the same time, the file address is written back to the archiving subtask table; the file compression rate is about 10-13%. That is, a 100M file is about 10M after compression. For example, the data for one month mentioned above is about 7168G, and after file compression, it is about 716G, saving a lot of storage costs.

[0060] In an embodiment of the present application, after archiving the data to be stored for a preset length of time before the current moment to generate a preset file to the storage space, it includes: performing interval sampling detection on the archived data to be stored; and triggering a corresponding target operation according to the sampling detection result.

[0061] It is understandable that the embodiments of the present application can perform interval sampling detection on the data to be stored after archiving; trigger corresponding target operations based on the sampling detection results, thereby ensuring that the data is archived correctly and improving the accuracy of subsequent data calls.

[0062] Specifically, if Figure 5 As shown in the figure, after data archiving is completed, data archiving detection will be triggered. This system adopts the interval sampling detection mechanism. For the archived data, it will be checked three times a day on the 21st, 22nd, 25th, 28th, and 30th days of each month. When a problem is found in a data archiving task, the data will be sent to the designated person in charge by WeChat, email, SMS, etc. After receiving the data, the person in charge needs to troubleshoot the problem and trigger the data to be re-archived. You can archive the app data twice or even multiple times in full or by specifying a subtask. And conduct subsequent data archiving checks to ensure that the data is archived correctly. Each time, 10% (configurable) proportion is randomly sampled, that is, 65536x 10% = 656 partitions will be selected each time for data inspection. Read the file content and compare it with the data in the tablestore database one by one. When a problem is found in the archived data, a notification is sent.

[0063] According to the data processing method proposed in the embodiment of the present application, a data set to be stored and a customer identifier corresponding to each data to be stored are obtained; a partition value to which the data to be stored belongs in the storage space is calculated using a preset algorithm based on the customer identifier, and the data to be stored is stored in an area corresponding to the partition value; the storage duration of the data to be stored is identified, and if the storage duration meets a first preset duration, the data to be stored for the preset duration before the current moment is archived to generate a preset file to the storage space, so as to achieve regular and rapid data archiving and improve data processing efficiency.

[0064] Specifically, Figure 6 A flowchart of a data processing method provided in another embodiment of the present application.

[0065] like Figure 6 As shown, the data processing method is applied to a client, wherein the method comprises the following steps:

[0066] In step S201, the historical data that the user needs to view at the current moment is identified.

[0067] It is understandable that the embodiment of the present application can identify the historical data that the user needs to view at the current moment, so as to subsequently trigger the thawing logic to thaw the corresponding historical data according to the current moment.

[0068] In step S202, the thawing logic is triggered according to the current moment to thaw the corresponding historical data.

[0069] It can be understood that the embodiment of the present application can trigger the thawing logic to thaw the corresponding historical data according to the current moment, thereby realizing an efficient data thawing and recovery mechanism, allowing users to be unaware of the existing services and improving the user experience.

[0070] In an embodiment of the present application, the corresponding historical data is thawed according to the thawing logic triggered at the current moment, including: determining the month of the historical data that needs to be thawed according to the current moment; if the month of the historical data meets the preset conditions, calculating the area of ​​the storage space where the historical data is located according to the data identifier, querying the archive file address of the historical data according to the data identifier to obtain the compressed historical data, and thawing the compressed historical data and storing it in the thawing temporary data table.

[0071] Among them, the preset condition can be that the month of the historical data is less than 6 months away from the current month, which can be set according to actual needs without specific limitation.

[0072] It can be understood that the embodiment of the present application can determine the month of historical data that needs to be unfrozen based on the current moment; if the month of the historical data meets the preset conditions, the area where the historical data is located in the storage space is calculated based on the data identifier, and the archive file address of the historical data is queried based on the data identifier to obtain the compressed historical data, and the compressed historical data is unfrozen and stored in a temporary unfrozen data table, thereby realizing an efficient data unfreezing and recovery mechanism, which can achieve users' unawareness of the existing business and improve user experience.

[0073] Specifically, if Figure 8-Figure 9As shown in the figure, each data archiving subtask will archive 16 partitions, and each partition will generate a file, that is, a total of 65536 files will be generated for monthly data archiving, and the file name is a .csv file named after the partition. For example, the customer with customer id = 50005310002 is in partition = 59557, and the customer id is calculated by hashing 65536. The generated archive file is: http: / / xxxxxx / 2023 / 05 / 59557.csv. After the customer logs in to the app and enters the data center, when viewing historical data, he will continue to slide to view previous historical data. Taking the current time 2023-11-7 as an example, when the user slides the data to view the data of 2023-10, the data thawing logic is triggered to thaw the archived data of May 2023. Similarly, when the user views the data of September 2023, the data of April 2023 will be thawed, and so on. According to customer id = 50005310002, calculate the interval where its data is located (partition = 59557), concatenate and calculate the archive file address, read the file to find the data of the account, and write it back to the thawing data information table (this table is not the same as the normal data table). This table is a temporary data table, and the data in it will be stored for 1 month. After one month, the data will be automatically deleted. If the user needs to view the previous historical data, the data thawing logic needs to be restarted; the data thawing time map is as follows Figure 7 As shown, the time node of the thawing logic trigger can be clearly seen.

[0074] In an embodiment of the present application, it also includes: obtaining the user's actual intention; if the actual intention is a query intention, identifying the user's query requirements, and filtering out historical data of the corresponding time range according to the query requirements.

[0075] It is understandable that the embodiments of the present application can obtain the actual intention of the user; if the actual intention is a query intention, the user's query requirements are identified, and historical data of the corresponding time range is filtered out according to the query requirements, thereby improving the user's usage experience.

[0076] Specifically, when users query data through the search function, it is necessary to consider how to retrieve the books that users need from the files; this aspect can be handled from the business perspective, that is, when users search, let the users check and select the time range for querying data.

[0077] There are two situations to consider: First, if the user does not check the query more option, the search range is limited to the last six months, and data can be directly retrieved from the database. Second, if the user checks the query more option, a pop-up window will appear asking the user to select a time range for the query. The system can determine whether to unfreeze the data before querying or to query directly based on the time range selected by the user.

[0078] For example, taking the current time as 2023-11-10, when the user searches for the historical time with the content "product removed from the shelves", the user selects the query time range as: start time (2023-03-01), end time (2023-11-10), the system will determine that the data before 2023-06 has been archived, and the search is divided into two parts. One is to directly query the data with the content "product removed from the shelves" from the tablestore, and to parse, unfreeze and restore the archived data of the user from March to May in 2023, and query the required data from it, and return it to the user together with the previously queried data; the specific process is as follows Fig.10 shown.

[0079] According to the data processing method proposed in the embodiment of the present application, the historical data that the user needs to view at the current moment is identified, and the thawing logic is triggered according to the current moment to thaw the corresponding historical data, thereby realizing an efficient data thawing and recovery mechanism, which can achieve the user's unawareness of the existing business and improve the user experience.

[0080] An embodiment of the present application also provides a computer-readable storage medium on which a computer program or instruction is stored. When the computer program or instruction is executed by a processor, the above data processing method is implemented.

[0081] An embodiment of the present application also provides a computer program product, including a computer program or instructions, characterized in that when the computer program or instructions are executed, the above data processing method is implemented.

[0082] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0083] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0084] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0085] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one or a combination of multiple of the following technologies known in the art: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0086] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

Claims

1. A data processing method, characterized in that: The method is applied to a server, wherein the method comprises the following steps: Obtain the data set to be stored and the data identifier corresponding to each data to be stored; Calculate the partition value of the storage space to which the data to be stored belongs using a preset algorithm according to the data identifier, and store the data to be stored in the area corresponding to the partition value; The storage duration of the data to be stored is identified. If the storage duration satisfies a first preset duration, the data to be stored for the preset duration before the current moment is archived to generate a preset file in the storage space.

2. The data processing method according to claim 1, characterized in that: The archiving of the data to be stored for a preset time before the current time includes: Create a message archiving main task, and use the message archiving main task to generate statistical results by counting the total amount of data to be stored in the month to be archived; A plurality of message archiving subtasks are divided according to the statistical result, and each message archiving subtask is used to process a preset amount of data to be stored.

3. The data processing method according to claim 2, characterized in that: The method of processing a preset amount of data to be stored by using each message archiving subtask includes: Identify the archiving month and archiving area for the message archiving subtask; According to the archiving month and archiving area, query the data to be stored and write it into a temporary message file; The temporary message file is compressed and uploaded to the storage space.

4. The data processing method according to claim 3, characterized in that: After compressing the temporary message file and uploading it to the storage space, the method includes: Identify the file path where temporary message files are saved after being compressed and uploaded to the storage space; The saved file path is written into the archiving subtask table for storage; wherein the temporary message file includes data identification, data title, data content, belonging partition value, data creation time and data modification time.

5. The data processing method according to claim 1, characterized in that: After archiving the data to be stored for a preset time before the current time to generate a preset file to the storage space, it includes: Performing interval sampling detection on the archived data to be stored; Trigger the corresponding target operation based on the sampling detection results.

6. A data processing method, characterized in that: The method is applied to a client, wherein the method comprises the following steps: Identify the historical data that the user needs to view at the current moment; The corresponding historical data is unfrozen according to the unfreezing logic triggered at the current moment.

7. The data processing method according to claim 6, characterized in that: The triggering of the thawing logic according to the current moment to thaw the corresponding historical data includes: Determine the month of historical data that needs to be unfrozen according to the current moment; If the month of the historical data meets the preset conditions, the area of ​​the storage space where the historical data is located is calculated according to the data identifier, the archive file address of the historical data is queried according to the data identifier to obtain the compressed historical data, and the compressed historical data is thawed and stored in the thawed temporary data table.

8. The data processing method according to claim 6, characterized in that: Also includes: Get the user's actual intention; If the actual intention is a query intention, the user's query requirements are identified, and historical data of the corresponding time range is filtered out according to the query requirements.

9. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed by a processor, it is used to implement the data processing method according to any one of claims 1 to 8.

10. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed, the data processing method according to any one of claims 1 to 8 is implemented.

Images