Data processing method, device, equipment and storage medium

Abstract

The present application provides a data processing method, apparatus, device, and storage medium, the method comprising: determining target data to be removed in a first index tree, deleting an association pointer between a first target index node corresponding to the target index and the target data in the first index tree, and establishing an association pointer between the target data and a second target index node corresponding to the target index in a second index tree, so as to achieve migration of the target data from the first index tree to the second index tree. In this technical solution, there is no need to copy, migrate, or delete the target data, and the storage location of the target data at the storage level does not change. Data migration can be achieved by simply changing the association pointer between the data and the index node, which simplifies the processing process, improves processing efficiency, and thus increases the running speed of the software system.

Description

Technical Field

[0001] The present application relates to the field of science and technology finance technology, and in particular to a data processing method, device, equipment and storage medium. Background Art

[0002] With the development of computer technology, more and more technologies are being applied in the financial sector. Traditional finance is gradually shifting towards Fintech (Fintech), and Internet technology (IT) is no exception. IT software systems enable offline financial transactions, effectively improving the efficiency and convenience of these transactions. However, as software systems operate over time, the amount of data in the current tables increases, causing the software system to run more slowly, resulting in performance issues and unavailable functions.

[0003] To solve the above problem, the current method mainly involves selecting historical data that is not used when processing financial business in the current table according to the configuration information box pre-configured by relevant staff, copying and migrating the data to the historical table, and deleting the data in the current table at the same time, so as to achieve the purpose of reducing the data volume in the current table.

[0004] However, the existing technology has the problems of low data processing efficiency and slow software system operation. Summary of the Invention

[0005] The present application provides a data processing method, apparatus, device and storage medium to solve the problems of low data processing efficiency and slow software system operation in the prior art.

[0006] In a first aspect, an embodiment of the present application provides a data processing method, comprising:

[0007] Determining target data to be removed from a first index tree, wherein the first index tree includes at least one index node and at least one data node, each data node includes at least one piece of data, the data of each data node is associated with the corresponding index node via an association pointer, and the target data includes a target index;

[0008] Deleting, in the first index tree, an association pointer between a first target index node corresponding to the target index and the target data;

[0009] An association pointer is established between the target data and a second target index node corresponding to the target index in the second index tree to achieve migration of the target data from the first index tree to the second index tree, where the second index tree is used to store historical data.

[0010] In a possible design of the first aspect, determining the target data to be removed from the first index tree includes:

[0011] The data whose target index value in the first index tree meets the preset conditions is determined as the target data.

[0012] Optionally, before determining the target data to be removed from the first index tree, the method further includes:

[0013] Establishing the first index tree according to a plurality of data, wherein the plurality of data includes at least one index;

[0014] The second index tree is established according to the index nodes of the first index tree.

[0015] Optionally, establishing the first index tree according to the plurality of data includes:

[0016] Establishing an initial index tree according to at least one index of the plurality of data;

[0017] Establish at least one data node according to the plurality of data;

[0018] According to the corresponding relationship, the data in each data node is associated with the corresponding index node in the initial index tree through an association pointer to generate the first index tree. The corresponding relationship is the corresponding relationship between data, index and index value.

[0019] Optionally, establishing at least one data node according to the plurality of pieces of data includes:

[0020] According to the multiple pieces of data and the corresponding relationship, a first data node corresponding to each index node is established respectively.

[0021] Optionally, establishing at least one data node according to the plurality of pieces of data includes:

[0022] At least one second data node is established in a preset data module according to the plurality of pieces of data.

[0023] Optionally, associating the data in each data node with the corresponding index node in the initial index tree through an association pointer according to the corresponding relationship to generate the first index tree includes:

[0024] According to the corresponding relationship, the data in each first data node is associated with the corresponding index node in the initial index tree through a first association pointer;

[0025] Establishing a third data node in a preset data module, wherein the third data node is used to store duplicate data between the first data nodes;

[0026] Establishing a second association pointer between the data in each third data node and the data in the corresponding first data node;

[0027] Establishing a third association pointer between each index node and the data in the corresponding third data node according to the first association pointer and the second association pointer;

[0028] The first association pointer, the second association pointer, and duplicate data in each first data node are deleted to generate the first index tree.

[0029] In another possible design of the first aspect, when the index is a joint index, the index node corresponding to the index includes multiple levels of child nodes, wherein the last level of child nodes is associated with the data in the corresponding data node through an association pointer.

[0030] Optionally, the method further includes:

[0031] When the index node corresponding to the preset condition does not exist in the first index tree, updating the first index tree according to the preset condition to obtain an updated first index tree;

[0032] The second index tree is updated according to the index nodes of the updated first index tree to obtain an updated second index tree.

[0033] In a second aspect, an embodiment of the present application provides a data processing device, including:

[0034] a determination module, configured to determine target data to be removed in a first index tree, wherein the first index tree includes at least one index node and at least one data node, each data node includes at least one piece of data, the data of each data node is associated with the corresponding index node via an association pointer, and the target data includes a target index;

[0035] a deletion module, configured to delete, in the first index tree, an association pointer between a first target index node corresponding to the target index and the target data;

[0036] An establishment module is used to establish an association pointer between the target data and a second target index node corresponding to the target index in the second index tree, so as to migrate the target data from the first index tree to the second index tree, where the second index tree is used to store historical data.

[0037] In a possible design of the second aspect, the determining module is specifically configured to:

[0038] The data whose target index value in the first index tree meets the preset conditions is determined as the target data.

[0039] Optionally, before determining the target data to be removed from the first index tree, the establishing module is further configured to:

[0040] Establishing the first index tree according to a plurality of data, wherein the plurality of data includes at least one index;

[0041] The second index tree is established according to the index nodes of the first index tree.

[0042] Optionally, the establishment module is specifically used to:

[0043] Establishing an initial index tree according to at least one index of the plurality of data;

[0044] Establish at least one data node according to the plurality of data;

[0045] According to the corresponding relationship, the data in each data node is associated with the corresponding index node in the initial index tree through an association pointer to generate the first index tree. The corresponding relationship is the corresponding relationship between data, index and index value.

[0046] Optionally, the establishment module is specifically used to:

[0047] According to the multiple pieces of data and the corresponding relationship, a first data node corresponding to each index node is established respectively.

[0048] Optionally, the establishment module is specifically used to:

[0049] At least one second data node is established in a preset data module according to the plurality of pieces of data.

[0050] Optionally, the establishment module is specifically used to:

[0051] According to the corresponding relationship, the data in each first data node is associated with the corresponding index node in the initial index tree through a first association pointer;

[0052] Establishing a third data node in a preset data module, wherein the third data node is used to store duplicate data between the first data nodes;

[0053] Establishing a second association pointer between the data in each third data node and the data in the corresponding first data node;

[0054] Establishing a third association pointer between each index node and the data in the corresponding third data node according to the first association pointer and the second association pointer;

[0055] The first association pointer, the second association pointer, and duplicate data in each first data node are deleted to generate the first index tree.

[0056] In another possible design of the second aspect, when the index is a joint index, the index node corresponding to the index includes multiple levels of child nodes, wherein the last level of child nodes is associated with the data in the corresponding data node through an association pointer.

[0057] Optionally, the device further includes:

[0058] an updating module, configured to update the first index tree according to the preset condition when an index node corresponding to the preset condition does not exist in the first index tree, and obtain an updated first index tree;

[0059] The updating module is further configured to update the second index tree according to the index nodes of the updated first index tree, and obtain the updated second index tree.

[0060] In a third aspect, an embodiment of the present application provides an electronic device comprising: a processor, a memory, and computer program instructions stored on the memory and executable on the processor, wherein the processor executes the computer program instructions to implement the first aspect and the methods provided by various possible designs.

[0061] In a fourth aspect, an embodiment of the present application may provide a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the first aspect and the methods provided by various possible designs.

[0062] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, is used to implement the first aspect and the methods provided by various possible designs.

[0063] The data processing method, apparatus, device, and storage medium provided by the embodiments of the present application determine the target data to be removed in a first index tree, delete the association pointer between the first target index node corresponding to the target index and the target data in the first index tree, and establish an association pointer between the target data and the second target index node corresponding to the target index in the second index tree, so as to achieve the migration of the target data from the first index tree to the second index tree. In this technical solution, there is no need to copy, migrate, or delete the target data, and the storage location of the target data at the storage level does not change. Data migration can be achieved by simply changing the association pointer between the data and the index node, which simplifies the processing process, improves processing efficiency, and thus increases the operating speed of the software system. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0065] Figure 1 A flowchart of a first embodiment of a database table processing method provided by the prior art;

[0066] Figure 2 A flowchart of a first embodiment of the data processing method provided in an embodiment of the present application;

[0067] Figure 3 A schematic diagram of the structure of a first index tree embodiment 1 provided in an embodiment of the present application;

[0068] Figure 4 A flowchart of a second embodiment of the data processing method provided in the present application;

[0069] Figure 5 A schematic diagram of the structure of the first index tree after archiving provided in an embodiment of the present application;

[0070] Figure 6 A schematic diagram of the structure of the second index tree after archiving provided in an embodiment of the present application;

[0071] Figure 7 A flowchart of a third embodiment of the data processing method provided in the present application;

[0072] Figure 8 Another structural diagram of the first index tree after archiving provided in an embodiment of the present application;

[0073] Figure 9 Another structural diagram of the second index tree after archiving provided in an embodiment of the present application;

[0074] Figure 10 A schematic diagram of the structure of the second embodiment of the first index tree provided in the embodiment of the present application;

[0075] Figure 11 A schematic diagram of the structure of the second index tree embodiment 1 provided in the embodiment of the present application;

[0076] Figure 12 A schematic diagram of the structure of a third embodiment of the first index tree provided in an embodiment of the present application;

[0077] Figure 13 A flowchart of a fourth embodiment of the data processing method provided in an embodiment of the present application;

[0078] Figure 14 A flowchart of a fifth embodiment of the data processing method provided in an embodiment of the present application;

[0079] Figure 15 A schematic diagram of the structure of a fourth embodiment of the first index tree provided in an embodiment of the present application;

[0080] Figure 16 A schematic diagram of the structure of a fifth embodiment of the first index tree provided in an embodiment of the present application;

[0081] Figure 17 A schematic diagram of the structure of a sixth embodiment of the first index tree provided in an embodiment of the present application;

[0082] Figure 18 A structural diagram of a seventh embodiment of the first index tree provided in an embodiment of the present application;

[0083] Figure 19 A schematic diagram of the structure of an eighth embodiment of the first index tree provided in an embodiment of the present application;

[0084] Figure 20 A schematic diagram of the structure of a second index tree embodiment 2 provided in an embodiment of the present application;

[0085] Figure 21 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application;

[0086] Figure 22 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0087] The above drawings illustrate specific embodiments of the present application, which will be described in more detail below. These drawings and the textual description are not intended to limit the scope of the present application in any way, but rather to illustrate the concepts of the present application to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0088] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0089] Before introducing the embodiments of the present application, the application background of the embodiments of the present application is first explained:

[0090] As the software system operates for a longer time, the data in the software system database will become more and more, and the database tables will become larger and larger, resulting in slower database table reading and writing speeds and slower software system operation, performance issues in the software system, and unavailability of related functions.

[0091] To solve the above problem, the Structured Query Language (SQL) of a relational database is usually used to archive and migrate the unaccessed or infrequently used historical data from the current table (ie the above database table) to a history table.

[0092] Next, the implementation principle of the existing technology is described in detail.

[0093] Figure 1 The flowchart of the first embodiment of the database table processing method provided by the prior art is as follows. Figure 1 As shown, the database table processing method includes the following steps:

[0094] Step 1: Get the configuration information for database table archiving.

[0095] The configuration information may include the following:

[0096] Task Type: used to indicate whether to perform data archiving or data cleansing operations on the data.

[0097] Current table name: The name of the table to be processed. The data in the current table will need to be operated on later.

[0098] Historical table name: The name of the target table where the data needs to be archived.

[0099] Parameter condition: The where condition of the SQL statement is used to define the data range that needs to be operated in the current table, generally defining the time range and data status, etc.

[0100] Table primary key: The primary key fields of the current table and the history table, separated by commas.

[0101] Maximum transaction: Since the amount of data framed may be very large, it can be optimized into batch operations. The maximum transaction is the maximum amount of data in each batch operation.

[0102] For example, the configuration information can be represented by Table 1.

[0103] Table 1

[0104]

[0105]

[0106] As shown in Table 1, Table 1 includes four rows of configuration information. Taking the configuration information in the first row as an example, the configuration information means: move the data in the current table 1 whose exec_date field is 30 days ago to the history table 1, moving one thousand data at a time until the data that meets the conditions (exec_date field is 30 days ago) is moved; taking the configuration information in the third row as an example, the configuration information means: delete the data in the current table 3 whose exec_date field is 60 days ago.

[0107] The electronic device may display a graphical configuration interface on its display screen, allowing the user to configure the configuration information through the graphical configuration interface. Accordingly, the electronic device may obtain the input configuration information in response to the operation of the graphical configuration interface by the relevant staff.

[0108] Optionally, the relevant staff may also configure the configuration file and input the configured configuration file into the electronic device. Correspondingly, the electronic device may receive the configuration file input by the relevant staff and obtain the configuration information carried in the configuration file.

[0109] Optionally, relevant staff may also use a database table to input configuration information, and correspondingly, the electronic device receives the configuration information input by the relevant staff.

[0110] It should be understood that the manner in which relevant staff configure configuration information and the manner in which electronic devices obtain configuration information are not limited here, and can be determined according to actual circumstances, and will not be elaborated here.

[0111] Step 2: Generate SQL to parse and verify the parameter conditions.

[0112] This step can be achieved by the following steps:

[0113] Step 1: Generate SQL statements to parse parameter conditions.

[0114] Step 2: Generate SQL statements to take SQL injection prevention measures for the parsed parameter conditions.

[0115] Because parameter conditions are manually configured by relevant personnel and may contain some non-standard data, you can filter out SQL keywords that cause changes to the database table structure and data, such as CREATE, DROP, TRUNCATE, UPDATE, and DELETE. This will prevent SQL parameter conditions containing these SQL keywords from continuing to execute subsequent processes. Optionally, you can only retain SQL parameter conditions containing AND and OR keywords.

[0116] Step 3: Generate SQL statements to verify the validity of parameter conditions after injection prevention measures.

[0117] Optionally, you can combine the SQL parameter conditions in the configuration information into a data statistics statement for archiving or clearing the current table. Execute this data statistics statement to query the current table to obtain the required data volume. If the required data volume is the same as the correct data volume, the SQL parameter conditions are valid.

[0118] For example, taking the first row of configuration information in Table 1 as an example, the data statistics statement can be expressed by the following code:

[0119] Select count(0) from current table 1 where exec_date <DATE_FORMAT(date_add(CURDATE(),interval-30day),'%Y%m%d')

[0120] The amount of data that needs to be operated on by the data statistics statement is the amount of data before the exec_date date field is 30 days ago.

[0121] Step 3: Generate data frame definition SQL.

[0122] The data frame definition SQL can be generated based on the configuration parameters verified in step 2.

[0123] For example, taking the first row of configuration information in Table 1 as an example, the data frame definition SQL can be expressed by the following code:

[0124] Select * from current table 1 where exec_date <DATE_FORMAT(date_add(CURDATE(),interval-30day),'%Y%m%d')

[0125] The data frame SQL is used to select data with the exec_date field set to 30 days ago.

[0126] Furthermore, the above data frame definition SQL can be further optimized based on the maximum transaction. The optimized data frame definition SQL can be expressed by the following code:

[0127] Select * from current table 1 where exec_date <DATE_FORMAT(date_add(CURDATE(),interval-30day),'%Y%m%d')LIMIT 1000

[0128] The optimized data framing SQL is used to select data with an exec_date field that is 30 days old, and the amount of data that can be selected each time is limited to 1000.

[0129] The subsequent steps will be to cyclically execute the above optimized data framing SQL according to the total amount of data until all the data to be processed is framed.

[0130] The number of times the data-framing SQL statement is executed in a loop can be expressed by the following formula: loop = (n / maximum transaction) + 1. Where loop is the number of times the data-framing SQL statement is executed in a loop, and n is the total amount of data.

[0131] Step 4: Generate SQL for the archive history table.

[0132] Optionally, during the data archiving process, it is necessary to use a data splicing insert (insert) SQL statement to splice the selected data into an insert data format so that the selected data can be inserted into the history table.

[0133] For example, taking the first row of configuration information in Table 1 as an example, the above data splicing insert SQL statement can be implemented by the following code:

[0134] INSERT INTO history table 1 SELECT * FROM current table 1

[0135] Among them, this code can avoid special characters, transcoding and splicing syntax problems, and can filter data by the specified primary key. The value of the primary key comes from the data frame defined in the SQL frame, thereby ensuring the correctness of the data.

[0136] Optionally, taking the first row of configuration information in Table 1 as an example, the SQL for the archive history table can be generated by the following code:

[0137] INSERT INTO history table 1 SELECT * FROM current table 1 WHERE execid in ('execid1','execid2','execid3',…,'execid1000')

[0138] Step 5: Generate data cleanup SQL.

[0139] The difference between data cleaning and data archiving operations lies in whether the selected data needs to be written to the history table. In data archiving, after writing the selected data to the history table, the data in the current table needs to be deleted, while in data cleaning, the data is directly deleted after the data is selected.

[0140] Optionally, in the data archiving operation, since the deleted data is the archived data and the conditions for selecting the data are the same, a data cleanup SQL can be generated based on the archive history table SQL.

[0141] For example, taking the first row of configuration information in Table 1 as an example, the data cleanup SQL can be implemented by the following code:

[0142] DELETE FROM current table 1WHERE execid in('execid1','execid2','execid3',…,'execid1000')

[0143] Step 6: Execute the corresponding SQL according to the configuration information.

[0144] In this step, data archiving operations and data cleaning operations are described in detail.

[0145] For data archiving operations, determine whether the corresponding history table exists based on the configuration information. If not, execute the SQL statement to create the history table and execute the SQL statements in steps 2, 3, 4, and 5 in sequence. If it exists, directly execute the SQL statements in steps 2, 3, 4, and 5 in sequence.

[0146] For data cleaning operations, execute the SQL statements in steps 2, 3, and 5 in sequence.

[0147] However, when the above method uses SQL statements to migrate data from the current table to the historical table, it is necessary to copy a large amount of data from the current table to the historical table and delete the data in the current table. The processing time is long, and the processing efficiency and the reading and writing speed of the database table are slow, which affects the running speed of the software system and causes performance problems in the software system.

[0148] Based on the above problems, an embodiment of the present application provides a data processing method, which can pre-establish a first index tree and a second index tree, wherein the first index tree is used to store data in the current table, and the second index tree is used to store historical data in the historical table. In this way, when it is necessary to migrate the target data in the current table to the historical table, it is only necessary to delete the association pointer between the target data and the corresponding first target index node in the first index tree, and at the same time establish an association pointer between the target data and the second target index node in the second index tree, so that the target data can be migrated from the first index tree to the second index tree. The first target index node is the index node corresponding to the target index of the target data in the first index tree, and the second target index node is the index node corresponding to the target index of the target data in the second index tree. Compared with the prior art, there is no need to copy, migrate, or delete the target data, and the location of the actual physical data at the storage level is neither moved nor copied. It is only necessary to change the association pointer between the target data and the index node to achieve the migration of the target data, which simplifies the processing process, improves processing efficiency, and thus improves the running speed of the software system.

[0149] The technical solution of the present application is described in detail below through specific embodiments.

[0150] It should be noted that the following specific embodiments may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0151] Figure 2 This is a flow chart of the first embodiment of the data processing method provided in the present application. Figure 2 As shown, the data processing method may include the following steps:

[0152] S201: Determine target data to be removed in a first index tree.

[0153] In an embodiment of the present application, the data in the current table is stored through the first index tree, and the software system can search for data required for processing the business through the first index tree.

[0154] The first index tree includes at least one index node and at least one data node, each data node includes at least one piece of data, the data of each data node is associated with the corresponding index node through an association pointer, and the target data includes a target index.

[0155] Optionally, the target data may also include target index values, parameter fields, other indexes, and index values ​​corresponding to other indexes, which can be determined based on actual conditions. The embodiments of the present application do not impose specific restrictions on this.

[0156] Optionally, the first index tree is established in advance based on multiple pieces of data.

[0157] Optionally, the first index tree may further include a root node, and the root node may be determined according to an identifier of the first index tree.

[0158] Optionally, the association pointer between the data in the data node and the index node may further include an index value.

[0159] Optionally, when the index is a joint index, the index node corresponding to the index includes multiple levels of child nodes, where the last level of child nodes is associated with the data in the corresponding data node through an association pointer, so that the software system can quickly locate the corresponding data in the first index tree through the joint index when processing business.

[0160] It should be understood that the embodiments of the present application do not limit the number of index nodes, the number of data nodes, and the amount of data in each data node, which can be determined based on actual conditions.

[0161] S202: Delete the associated pointer between the first target index node corresponding to the target index and the target data in the first index tree.

[0162] It should be understood that the associated pointer can be deleted using existing methods, and the specific method of deleting the associated pointer is not limited here.

[0163] S203: Establish an association pointer between the target data and a second target index node corresponding to the target index in the second index tree to migrate the target data from the first index tree to the second index tree, where the second index tree is used to store historical data.

[0164] The second index tree is pre-established based on the first index tree, and the index nodes of the second index tree are the same as those of the first index tree.

[0165] It should be understood that the association pointer can be established through existing methods, and the specific method of establishing the association pointer is not limited here.

[0166] The data processing method provided in an embodiment of the present application determines the target data to be removed in a first index tree, deletes the association pointer between the first target index node corresponding to the target index and the target data in the first index tree, and establishes an association pointer between the target data and the second target index node corresponding to the target index in the second index tree, thereby migrating the target data from the first index tree to the second index tree. In this technical solution, there is no need to copy, migrate, or delete the target data, and the storage location of the target data at the storage level does not change. Data migration can be achieved by simply changing the association pointer between the data and the index node, which simplifies the processing process, improves processing efficiency, and thus increases the operating speed of the software system.

[0167] In a possible implementation, the above S201 may be implemented by the following steps:

[0168] The data whose target index value in the first index tree meets the preset conditions is determined as the target data.

[0169] Optionally, the above preset conditions are pre-configured by relevant staff and are used to search for target data stored in the first index tree.

[0170] Exemplarily, the preset condition may include a target index. For example, the preset condition may include a date index. The preset condition may be that the difference between the date index value and the current date is greater than or equal to a preset number of days.

[0171] In the above embodiment, the target data can be determined by pre-configured preset conditions so that the first index tree can be dynamically monitored. Once the target data that meets the preset conditions exists, the migration process is performed, which effectively ensures the processing efficiency and reduces the amount of redundant data in the first index tree.

[0172] Next, the above embodiment is illustrated by two specific examples.

[0173] Example 1:

[0174] When we need to archive some data from the first index tree to the second index tree, a common operation is to archive data based on date. For example, every day, data from the first index tree that is older than a preset number of days can be archived to the second index tree. In actual archiving applications, the preset conditions can include a date index and, optionally, at least one processing status index.

[0175] Illustratively, the data stored in the first index tree can be represented by Table 2.

[0176] Table 2

[0177] Data Identification Index A field Index field B Parameter C field Parameter D field Parameter E field Parameter F field Data 1 A1 B1 C D E F Data 2 A1 B2 C D E F Data 3 A2 B1 C D E F Data 4 A2 B2 C D E F

[0178] As shown in Table 2, the multiple data are data 1, data 2, data 3 and data 4, and the indexes of the multiple data include index A, index B and joint index AB. For index A, the index values ​​corresponding to data 1, data 2, data 3 and data 4 are A1, A1, A2 and A2 respectively; for index B, the index values ​​corresponding to data 1, data 2, data 3 and data 4 are B1, B2, B1 and B2 respectively; for index AB, the index values ​​corresponding to data 1, data 2, data 3 and data 4 are A1B1, A1B2, A2B1 and A2B2 respectively. Among them, C, D, E and F are parameter fields in each data. It should be understood that the data can also include other content and other representation forms, which can be determined according to actual conditions, and the embodiments of the present application do not impose specific restrictions on this.

[0179] Based on Table 2, assume that the pre-defined conditions for data archiving include three indexes: Index A, Index B, and Combined Index AB. Index A is a date index, and Index B is a processing status index (an index value of 0 indicates that the data has not been processed, and an index value of 1 indicates that the data has been processed). The index values ​​of Index A and Index B are represented using numbers, while other parameter fields are still represented using alphabetic variables. Based on the above processing, the data in the first index tree can be represented by Table 3.

[0180] Table 3

[0181] Data Identification Index A field Index field B Parameter C field Parameter D field Parameter E field Parameter F field Data 1 20220218 0 C D E F Data 2 20220218 1 C D E F Data 3 20220219 1 C D E F Data 4 20220220 0 C D E F

[0182] Optional, based on Table 3, Figure 3 This is a structural diagram of the first index tree embodiment provided in the present application. Figure 3 As shown, business X1 is the root node of the first index tree, index A, index B, and joint index AB are the three index nodes of the first index tree, and the nodes of each data integration are data nodes. Among them, index A is the date index mentioned above, and index B is the processing status index.

[0183] It should be understood that, taking the index A of the first index tree as an example, index A is associated with data 1, data 2, data 3 and data 4 through associated pointers. To facilitate data retrieval, data 1 and data 2 with the same index value are integrated into one data node, and data 3 and data 4 are integrated into two other data nodes. Figure 3 The association pointers between each data in a data node and the corresponding index node are merged into one, that is, the solid arrows from index A to the data nodes where data 1 and data 2 are located are the association pointers between index A and data 1 and between index A and data 2.

[0184] That is, in the following figures, when a data node contains multiple pieces of data, when the arrow between the index node and the data node points to the upper or lower side of the data node (such as Figure 3 In the example, if there is an arrow from index A pointing to the data node where data 1 and data 2 are located), it means that the index node is associated with each data in the data node through an associated pointer; if the arrow between the index node and the data node points to the left or right of the data node (such as Figure 10 ), it indicates that the index node and the data pointed to by the arrow in the data node are associated through an association pointer.

[0185] Assuming that the preset condition is that the difference between the date index value and the current date is greater than or equal to 3 days, and the processing status index value is 1, then when the current date is 20220221 (February 21, 2022), the target data is data 2, that is, data 2 in the first index tree needs to be archived to the second index tree; when the current date is 20220222 (February 22, 2022), the target data is data 2 and data 3, that is, data 2 and data 3 in the first index tree need to be archived to the second index tree.

[0186] For example, the current date is 20220221. Figure 4 This is a flow chart of the second embodiment of the data processing method provided in the present application. Figure 4 As shown, first determine the target data according to the preset conditions ( Figure 4 The gray part in the image is data 2), and then the archiving process is performed to delete the association pointer between data 2 and the corresponding index node in the first index tree, and to establish the association pointer between data 2 and the corresponding index node in the second index tree, thereby realizing data archiving of data 2.

[0187] Optional, in Figure 4 On the basis of Figure 5 A schematic diagram of the structure of the first index tree after archiving provided in an embodiment of the present application is provided. Figure 6 A structural diagram of the second index tree after archiving provided in the embodiment of the present application. Figure 5 and Figure 6 As shown, the first index tree after archiving processing includes data 1, data 3 and data 4, and the second index tree after archiving processing includes data 2.

[0188] For example, the current date is 20220222. Figure 7 This is a flow chart of the third embodiment of the data processing method provided in the embodiment of the present application. Figure 7 As shown, first determine the target data according to the preset conditions ( Figure 7The gray part in the image is data 2 and data 3), and then the archiving process is performed to delete the association pointers between the target data (data 2 and data 3) and the corresponding index nodes in the first index tree, and establish the association pointers between the target data and the corresponding index nodes in the second index tree, thereby realizing data archiving of the target data.

[0189] Optional, in Figure 7 On the basis of Figure 8 Another structural diagram of the first index tree after archiving provided in an embodiment of the present application is shown. Figure 9 Another structural diagram of the second index tree after archiving provided in the embodiment of the present application. Figure 8 and Figure 9 As shown, the first index tree after archiving processing includes data 1 and data 4, and the second index tree after archiving processing includes data 2 and data 3.

[0190] Example 2:

[0191] In this example, the data stored in the first index tree is taken as the data in Table 2. Figure 10 This is a structural diagram of the second embodiment of the first index tree provided in the embodiment of the present application. Figure 11 This is a structural diagram of the second index tree embodiment 1 provided in the embodiment of the present application. Assuming that the target data is data 1, it is necessary to delete the associated pointers between data 1 and index A, index B, and joint index AB respectively. Figure 12 This is a structural diagram of the third embodiment of the first index tree provided in the embodiment of the present application. Figure 12 As shown in the figure, the arrow covered by the cross is the associated pointer that needs to be deleted.

[0192] Furthermore, association pointers are established between data 1 and index A, index B, and combined index AB of the second index tree, respectively. Figure 13 This is a flow chart of the fourth embodiment of the data processing method provided in the present application. Figure 13 As shown, the first index tree after archiving processing includes data 2, data 3 and data 4, and the second index tree after archiving processing includes data 1.

[0193] In some possible implementations, before S201, the data processing method may further include the following steps:

[0194] Step 1: Create a first index tree based on multiple pieces of data.

[0195] The plurality of data includes at least one index.

[0196] Optionally, the multiple pieces of data may be represented in the form of Table 2.

[0197] Optionally, the multiple pieces of data may be data generated when the software system processes business.

[0198] In one possible implementation, a root node can be established based on the identifier of the first index tree, at least one index node can be established based on the index of the data, and at least one data node can be established based on the plurality of data items, where each data node includes at least one piece of data. Furthermore, each index node is associated with the root node, and the data in each data node is associated with the corresponding index node using an association pointer based on the data's index value, thereby generating the first index tree. Alternatively, the index node and the root node can also be associated using an association pointer.

[0199] In another possible implementation, a root node can be established based on the identifier of the first index tree, at least one index node can be established based on the index of the data, and each index node can be associated with the root node to generate an initial index tree. Furthermore, each index node of the initial index tree can be used as a root node, and at least one data node corresponding to each index node can be established based on the index value of the plurality of data items. The data in each data node can be associated with the corresponding index node via an association pointer, thereby generating at least one index data tree. Finally, the identical index nodes in the initial index tree and each index data tree can be merged to generate a first index tree.

[0200] Optionally, based on Table 2, Figure 14 This is a flow chart of the fifth embodiment of the data processing method provided in the present application. Figure 14 As shown, an initial index tree and three index data trees are generated respectively, the index A of the initial index tree is merged with the index A of each index data tree, the index B of the initial index tree is merged with the index B of each index data tree, and the joint index AB of the initial index tree is merged with the joint index AB of each index data tree, thereby generating a first index tree.

[0201] In this implementation, because the software system generates a large amount of data when processing business, the processing time required to associate each piece of data with an index node is long. Therefore, the initial index tree and the index data tree can be generated and processed simultaneously, which can effectively improve processing efficiency. At the same time, in scenarios with massive data and distributed data storage, the initial index tree and each index data tree may be stored in different storage devices. Therefore, it is necessary to merge the initial index tree and each index data tree to ensure the integrity of the first index tree, so that the software system can quickly find data through the first index tree.

[0202] The first index tree includes at least one index node and at least one data node. Each data node includes at least one piece of data. The data of each data node is associated with the corresponding index node through an association pointer.

[0203] Step 2: Create a second index tree based on the index nodes of the first index tree.

[0204] The second index tree is used to store historical data that is not involved in business processing. This allows the target data to be migrated from the first index tree to the second index tree during data processing by deleting the associated pointer between the target data and the first target index node of the first index tree and simultaneously establishing an associated pointer between the target data and the second target index node of the second index tree, thereby reducing the data volume of the first index tree. It should be understood that the format of the historical data is consistent with that of the aforementioned multiple pieces of data; that is, the index of the historical data is the same as that of the aforementioned multiple pieces of data.

[0205] Optionally, a root node may be created according to the identifier of the first index tree, and then the same index node as the first index tree may be created. Finally, the root node and the index node may be associated to generate the second index tree.

[0206] Optionally, a root node may be created based on the table name of the history table corresponding to the current table.

[0207] The data processing method provided in the embodiments of the present application establishes a first index tree based on multiple pieces of data and a second index tree based on the index nodes of the first index tree. This technical solution, by establishing a first index tree to replace the current table and a second index tree to replace the historical table, lays the foundation for subsequent data migration and improves data processing efficiency.

[0208] In a possible implementation, establishing the first index tree based on the plurality of data may be achieved by the following steps:

[0209] Step 1: Create an initial index tree based on at least one index of multiple data pieces.

[0210] Optionally, a root node may be established according to the identifier of the first index tree, at least one index node may be established according to the index of the data, and each index node may be associated with the root node, thereby generating an initial index tree.

[0211] Step 2: Create at least one data node based on multiple pieces of data.

[0212] Optionally, a data node corresponding to each index node may be established respectively, or at least one data node may be directly established based on the plurality of data. This may be determined based on actual conditions and will not be described in detail here.

[0213] Step 3: According to the corresponding relationship, the data in each data node is associated with the corresponding index node in the initial index tree through the association pointer to generate a first index tree.

[0214] The corresponding relationship is the corresponding relationship between data, index and index value. The corresponding relationship can be represented by Table 2 and will not be repeated here.

[0215] Optionally, the data in the data node and the index node may have a one-to-one correspondence or a one-to-many correspondence, which may be determined based on actual conditions and will not be described in detail here.

[0216] In the above embodiment, an initial index tree can be first established based on the indexes of multiple data pieces, and then at least one data node can be established based on the multiple data pieces. Finally, through the correspondence between each data piece and the index value, the data in each data node can be associated with the corresponding index node in the initial index tree through an association pointer, thereby generating a first index tree containing all data information, ensuring the integrity of the first index number.

[0217] In a possible implementation, the above-mentioned establishment of at least one data node based on the multiple pieces of data can be achieved through the following steps: based on the multiple pieces of data and the corresponding relationships, respectively establish a first data node corresponding to each index node.

[0218] Optionally, for any index node, the multiple pieces of data may be divided according to index values, and data with the same index value may be integrated into a first data node, thereby establishing a first data node corresponding to each index node.

[0219] For example, based on Table 2, Figure 15 This is a structural diagram of the fourth embodiment of the first index tree provided in the embodiment of the present application. Figure 15 As shown, for index A, data 1 and data 2 are its first data node, and data 3 and data 4 are its other data node; for index B, data 1 and data 3 are its first data node, and data 2 and data 4 are its other data node; for joint index AB, data 1, data 2, data 3, and data 4 are all its first data nodes. The data in the first data node of the first index tree corresponds one-to-one to the index node, that is, each index node is associated with an independent piece of data.

[0220] Optionally, for any index node, each piece of data in the plurality of data may be used as an independent first data node, thereby establishing a first data node corresponding to each index node.

[0221] Optionally, for any index node, the above-mentioned multiple data may be integrated into a first data node, thereby establishing a first data node corresponding to each index node.

[0222] In the above embodiment, a corresponding first data node is established for each index node, so that each index node is associated with an independent copy of data. In this way, if a problem occurs with the data in any first data node, it will only be affected when retrieving data through the index node corresponding to the first data node, without affecting the use of other index nodes, thereby improving the stability of the first index tree.

[0223] In a possible implementation, the above-mentioned establishment of at least one data node based on the multiple pieces of data may be implemented by the following steps: establishing at least one second data node in a preset data module based on the multiple pieces of data.

[0224] In the preset data module, all the multiple data can be integrated into one second data node, or part of the multiple data can be integrated into one second data node, and the remaining data can be integrated into another (or multiple) second data nodes, thereby establishing at least one second data node in the preset data module.

[0225] The preset data module may be pre-set by relevant staff, and the data in the data module is stored in a data block on a disk.

[0226] In this embodiment, the data in the second data node and the index node have a one-to-many relationship, that is, only one copy of the data is stored in the preset module, which effectively saves disk space.

[0227] In one possible implementation, Figure 15 Based on the above correspondence, the data in each data node is associated with the corresponding index node in the initial index tree through the association pointer to generate the first index tree, which can be achieved by the following steps:

[0228] Step 1: According to the corresponding relationship, the data in each first data node is associated with the corresponding index node in the initial index tree through the first association pointer.

[0229] The specific implementation method and principle of this step have been explained in detail in the above embodiments and will not be repeated here.

[0230] Step 2: Create a third data node in the preset data module.

[0231] The third data node is used to store duplicate data in each first data node.

[0232] Optionally, whether there is duplicate data in each first data node can be determined through the Message-Digest Algorithm (MD5) or the Secure Hash Algorithm 1 (SHA1), or each field value in each row of data in different first data nodes can be directly compared to see whether they are the same. The embodiment of the present application does not limit the method for determining whether there is duplicate data in each first data node.

[0233] Optional, in Figure 15 On the basis of Figure 16 This is a structural diagram of the fifth embodiment of the first index tree provided in the embodiment of the present application. Figure 16 As shown, the duplicate data in the first data node are data 1, data 2, data 3 and data 4 respectively, so the third data node integrates data 1, data 2, data 3 and data 4.

[0234] Step 3: Establish a second association pointer between the data in each third data node and the data in the corresponding first data node.

[0235] Optionally, when duplicate data exists between the first data node and other first data nodes, the duplicate data is associated with the data in the third data node through the second association pointer.

[0236] Optional, in Figure 16 On the basis of Figure 17 This is a structural diagram of the sixth embodiment of the first index tree provided in the embodiment of the present application. Figure 17 As shown, the first association pointer is a solid arrow, and the second association pointer is a dotted arrow.

[0237] Step 4: Establish a third association pointer between each index node and the data in the corresponding third data node based on the first association pointer and the second association pointer.

[0238] Optionally, for any index node, data in a third data node corresponding to the index node is determined according to the first association pointer and the second association pointer, and association is performed through the third association pointer.

[0239] Step 5: Delete the first association pointer, the second association pointer, and duplicate data in each first data node to generate a first index tree.

[0240] Optional, in Figure 17 On the basis of Figure 18 This is a structural diagram of the seventh embodiment of the first index tree provided in the embodiment of the present application. Figure 18As shown, the index node in the first index tree is directly associated with the corresponding data in the third data node through the third association pointer, and the third association pointer is a bold solid arrow.

[0241] In the above embodiment, at the initial stage of software system operation, due to the small amount of data, it is possible to Figure 15 The first index tree structure shown for storing data can effectively improve the stability of the first index tree and ensure the smooth operation of the software system. However, as the software system's runtime increases, the amount of data will also increase dramatically. By integrating duplicate data from the first data node into a third data node and directly associating the index node with the corresponding data in the third data node through a third association pointer, disk storage space can be effectively conserved, avoiding software system lags caused by insufficient storage space.

[0242] In a possible implementation, the data processing method may further include the following steps:

[0243] Step 1: When there is no index node corresponding to the preset condition in the first index tree, the first index tree is updated according to the preset condition to obtain an updated first index tree.

[0244] Optionally, when there is no index node corresponding to the preset condition in the first index tree, a new index node can be generated according to the preset condition, and the index node can be associated with the root node of the first index tree. At the same time, if there is a data node corresponding to the index node, the index node is associated with the data in the corresponding data node through an association pointer. If there is no corresponding data node, a corresponding data node is generated based on the above-mentioned multiple data and corresponding relationships, and the index node is associated with the data in the corresponding data node through an association pointer, thereby generating an updated first index tree.

[0245] For example, based on Table 2, multiple pieces of data after index update can be represented by Table 4.

[0246] Table 4

[0247] Data Identification Index A field Index field B Index C field Parameter D field Parameter E field Parameter F field Data 1 A1 B1 C1 D E F Data 2 A1 B2 C2 D E F Data 3 A2 B1 C2 D E F Data 4 A2 B2 C1 D E F

[0248] As shown in Table 4, the newly added indexes include index C, joint index AC, joint index BC, and joint index ABC.

[0249] Based on Table 4 and Figure 10 The content shown, Figure 19 This is a structural diagram of the first index tree embodiment 8 provided in the embodiment of the present application. Figure 19 As shown, according to the newly added joint index ABC, Figure 10The first index tree shown is updated to generate an updated first index tree. It should be understood that the updated first index tree can be updated according to at least one of the index C, the joint index AC, the joint index BC and the joint index ABC. Figure 10 The first index tree shown is updated, and here only the updating of the first index data according to the combined index ABC is used as an example for explanation.

[0250] Optionally, when the software system generates new data when processing business, the data nodes of the first index tree may be updated according to the newly generated data, thereby obtaining an updated first index tree.

[0251] Step 2: Update the second index tree according to the updated index nodes of the first index tree to obtain the updated second index tree.

[0252] Optionally, new index nodes may be added based on the updated index nodes of the first index tree to generate an updated second index tree, so that the index nodes of the updated second index tree are the same as the index nodes of the updated first index tree.

[0253] For example, in Figure 11 and Figure 19 On the basis of Figure 20 This is a structural diagram of the second index tree embodiment provided in the present application. Figure 20 As shown, due to Figure 20 The updated first index tree adds the joint index ABC, so it is necessary to add the joint index ABC to the second index tree to generate an updated second index node.

[0254] The following are device embodiments of the present application, which can be used to implement the method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.

[0255] Figure 21 This is a schematic diagram of the structure of the data processing device provided in the embodiment of the present application. Figure 21 As shown, the data processing device includes:

[0256] Determining module 211, configured to determine target data to be removed from a first index tree, wherein the first index tree includes at least one index node and at least one data node, each data node includes at least one piece of data, the data of each data node is associated with the corresponding index node via an association pointer, and the target data includes a target index;

[0257] The deletion module 212 is configured to delete the associated pointer between the first target index node corresponding to the target index and the target data in the first index tree;

[0258] The establishing module 213 is used to establish an association pointer between the target data and the second target index node corresponding to the target index in the second index tree, so as to achieve migration of the target data from the first index tree to the second index tree, where the second index tree is used to store historical data.

[0259] In a possible design of an embodiment of the present application, the determination module 211 is specifically configured to:

[0260] The data whose target index value in the first index tree meets the preset conditions is determined as the target data.

[0261] Optionally, before determining the target data to be removed from the first index tree, the module 213 is established to further:

[0262] Establishing a first index tree based on the plurality of data, wherein the plurality of data includes at least one index;

[0263] A second index tree is created based on the index nodes of the first index tree.

[0264] Optionally, a module 213 is established, specifically configured to:

[0265] Establishing an initial index tree according to at least one index of the plurality of data;

[0266] Establish at least one data node based on multiple pieces of data;

[0267] According to the corresponding relationship, the data in each data node is associated with the corresponding index node in the initial index tree through the association pointer to generate a first index tree. The corresponding relationship is the corresponding relationship between data, index and index value.

[0268] Optionally, a module 213 is established, specifically configured to:

[0269] According to the plurality of data and the corresponding relationship, a first data node corresponding to each index node is established respectively.

[0270] Optionally, a module 213 is established, specifically configured to:

[0271] At least one second data node is established in a preset data module according to the plurality of data.

[0272] Optionally, a module 213 is established, specifically configured to:

[0273] According to the corresponding relationship, the data in each first data node is associated with the corresponding index node in the initial index tree through the first association pointer;

[0274] Establishing a third data node in the preset data module, the third data node is used to store duplicate data between the first data nodes;

[0275] Establishing a second association pointer between the data in each third data node and the data in the corresponding first data node;

[0276] Establishing a third association pointer between each index node and the data in the corresponding third data node according to the first association pointer and the second association pointer;

[0277] The first association pointer, the second association pointer, and duplicate data in each first data node are deleted to generate a first index tree.

[0278] In another possible design of an embodiment of the present application, when the index is a joint index, the index node corresponding to the index includes multiple levels of child nodes, where the last level of child nodes is associated with the data in the corresponding data node through an association pointer.

[0279] Optionally, the device further includes:

[0280] An updating module, configured to update the first index tree according to the preset condition when an index node corresponding to the preset condition does not exist in the first index tree, and obtain an updated first index tree;

[0281] The updating module is further configured to update the second index tree according to the updated index nodes of the first index tree, and obtain the updated second index tree.

[0282] The data processing device provided in the embodiments of the present application can be used to execute the data processing method in any of the above embodiments. Its implementation principles and technical effects are similar and will not be repeated here.

[0283] It should be noted that it should be understood that the division of the various modules of the above device is merely a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. Moreover, these modules can all be implemented in the form of software called by processing elements; they can also all be implemented in the form of hardware; some modules can also be implemented in the form of software called by processing elements, and some modules can be implemented in the form of hardware. In addition, these modules can be fully or partially integrated together or implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each of the above modules can be completed by the hardware integrated logic circuit in the processor element or by instructions in the form of software.

[0284] Figure 22 This is a schematic diagram of the structure of the electronic device provided in the embodiment of the present application. Figure 22As shown, the electronic device may include: a processor 221, a memory 222, and computer program instructions stored in the memory 222 and executable on the processor 221. When the processor 221 executes the computer program instructions, the data processing method provided in any of the aforementioned embodiments is implemented.

[0285] Optionally, the above-mentioned components of the electronic device may be connected via a system bus.

[0286] The memory 222 may be a separate storage unit or a storage unit integrated in the processor. The number of processors may be one or more.

[0287] Optionally, the electronic device may further include an interface for interacting with other devices.

[0288] The transceiver is used to communicate with other computers and constitutes a communication interface.

[0289] It should be understood that the processor 221 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this application can be directly implemented as being executed by a hardware processor, or can be executed by a combination of hardware and software modules in the processor.

[0290] The system bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. System buses can be divided into address buses, data buses, and control buses. For ease of illustration, the diagram uses only a single thick line, but this does not imply a single bus or type of bus. Memory may include random access memory (RAM) and non-volatile memory (NVM), such as at least one disk drive.

[0291] All or part of the steps of the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a readable memory. When the program is executed, it performs the steps of the above-mentioned method embodiments; and the aforementioned memory (storage medium) includes: read-only memory (ROM), RAM, flash memory, hard disk, solid-state drive, magnetic tape, floppy disk, optical disc, and any combination thereof.

[0292] The electronic device provided in the embodiment of the present application can be used to execute the data processing method provided in any of the above method embodiments. Its implementation principles and technical effects are similar and will not be repeated here.

[0293] An embodiment of the present application provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed on a computer, the computer executes the above-mentioned data processing method.

[0294] The computer-readable storage medium mentioned above may be implemented by any type of volatile or non-volatile memory device, or a combination thereof, such as static random access memory, electrically erasable programmable read-only memory, erasable programmable read-only memory, programmable read-only memory, read-only memory, magnetic storage, flash memory, magnetic disk, or optical disk. The computer-readable storage medium may be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0295] Optionally, a readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be an integral part of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0296] An embodiment of the present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium, and the at least one processor can implement the above-mentioned data processing method when executing the computer program.

[0297] It should be understood that the present application is not limited to the exact structure described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A data processing method, characterized in that: include: Determining target data to be removed from a first index tree, wherein the first index tree includes at least one index node and at least one data node, each data node includes at least one piece of data, the data of each data node is associated with the corresponding index node via an association pointer, and the target data includes a target index; Deleting, in the first index tree, an association pointer between a first target index node corresponding to the target index and the target data; Establishing an association pointer between the target data and a second target index node corresponding to the target index in a second index tree, so as to migrate the target data from the first index tree to the second index tree, where the second index tree is used to store historical data; The first index tree is generated in the following manner: According to the corresponding relationship, the data in each first data node is associated with the corresponding index node in the initial index tree through a first association pointer; the corresponding relationship is a correspondence between data, index, and index value; each first data node corresponds to each index node and is established based on multiple data and the corresponding relationship; the multiple data include at least one index, and the initial index tree is established based on the at least one index of the multiple data; Establishing a third data node in a preset data module, wherein the third data node is used to store duplicate data between the first data nodes; Establishing a second association pointer between the data in each third data node and the data in the corresponding first data node; Establishing a third association pointer between each index node and the data in the corresponding third data node according to the first association pointer and the second association pointer; The first association pointer, the second association pointer, and duplicate data in each first data node are deleted to generate the first index tree.

2. The method according to claim 1, characterized in that The determining target data to be removed from the first index tree includes: The data whose target index value in the first index tree meets the preset conditions is determined as the target data.

3. The method according to claim 2, characterized in that Before determining the target data to be removed from the first index tree, the method further includes: Establishing the first index tree according to the plurality of data; The second index tree is established according to the index nodes of the first index tree.

4. The method according to claim 3, characterized in that The step of establishing the first index tree according to the plurality of data includes: Establishing the initial index tree according to at least one index of the plurality of pieces of data; Establish at least one data node according to the plurality of data; According to the corresponding relationship, the data in each data node is associated with the corresponding index node in the initial index tree through an association pointer to generate the first index tree.

5. The method according to claim 4, characterized in that The step of establishing at least one data node according to the plurality of pieces of data includes: According to the multiple pieces of data and the corresponding relationship, a first data node corresponding to each index node is established respectively.

6. The method according to claim 4, characterized in that The step of establishing at least one data node according to the plurality of pieces of data includes: At least one second data node is established in a preset data module according to the plurality of pieces of data.

7. The method according to any one of claims 1 to 6, characterized in that When the index is a joint index, the index node corresponding to the index includes multiple levels of child nodes, wherein the last level of child nodes is associated with the data in the corresponding data node through an association pointer.

8. The method according to any one of claims 2 to 6, characterized in that: The method further comprises: When the index node corresponding to the preset condition does not exist in the first index tree, updating the first index tree according to the preset condition to obtain an updated first index tree; The second index tree is updated according to the index nodes of the updated first index tree to obtain an updated second index tree.

9. A data processing device, characterized in that: include: a determination module, configured to determine target data to be removed in a first index tree, wherein the first index tree includes at least one index node and at least one data node, each data node includes at least one piece of data, the data of each data node is associated with the corresponding index node via an association pointer, and the target data includes a target index; a deletion module, configured to delete, in the first index tree, an association pointer between a first target index node corresponding to the target index and the target data; an establishing module, configured to establish an association pointer between the target data and a second target index node corresponding to the target index in a second index tree, so as to achieve migration of the target data from the first index tree to the second index tree, wherein the second index tree is used to store historical data; The establishing module is further configured to generate the first index tree; when the establishing module is configured to generate the first index tree, it is specifically configured to: According to the corresponding relationship, the data in each first data node is associated with the corresponding index node in the initial index tree through a first association pointer; the corresponding relationship is a correspondence between data, index, and index value; each first data node corresponds to each index node and is established based on multiple data and the corresponding relationship; the multiple data include at least one index, and the initial index tree is established based on the at least one index of the multiple data; Establishing a third data node in a preset data module, wherein the third data node is used to store duplicate data between the first data nodes; Establishing a second association pointer between the data in each third data node and the data in the corresponding first data node; Establishing a third association pointer between each index node and the data in the corresponding third data node according to the first association pointer and the second association pointer; The first association pointer, the second association pointer, and duplicate data in each first data node are deleted to generate the first index tree.

10. An electronic device comprising: A processor, a memory, and computer program instructions stored in the memory and executable on the processor, wherein the processor is used to implement the data processing method according to any one of claims 1 to 8 when executing the computer program instructions.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the data processing method according to any one of claims 1 to 8.

Images