Data archiving method, apparatus, electronic device, medium, and program product

Abstract

Embodiments of the present disclosure disclose data archiving method, device, electronic equipment, medium and program product. A specific embodiment of the method comprises: performing a main table archiving task corresponding to main table archiving task information to generate main table data to be archived, wherein the main table corresponding to the main table archiving task information has a field association relationship with a set of sub-tables; generating at least one sub-table archiving task information corresponding to at least one target sub-table in the set of sub-tables; for each archiving task information in the set of archiving task information, the following consumption steps are performed: generating an archiving message group for the archiving task information; performing message consumption on the archiving message group to obtain a consumption result group; and generating archiving information for the main table and the at least one target sub-table according to the obtained set of consumption result groups. This embodiment is related to data archiving, and utilizes the generation and consumption of messages to perform data archiving on the data in the main table and the plurality of sub-tables, thereby improving the completeness and archiving efficiency of data archiving.

Description

Technical Field

[0001] Embodiments of this disclosure relate to the field of computer technology, and more specifically to data archiving methods, apparatus, electronic devices, media, and program products. Background Technology

[0002] Data archiving involves archiving historical data from a database to reduce the amount of data and improve database performance. Currently, the common method for data archiving is to use data archiving toolkits to archive data from a single table.

[0003] However, the inventors discovered that when using the above method to archive data, the following technical problems often arise:

[0004] Archiving data from a single table may result in incomplete archived data, leading to low archiving efficiency and high archiving resource consumption.

[0005] The information disclosed in this background section is only intended to enhance the understanding of the background of the inventive concept, and therefore may contain information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] The summary portion of this disclosure is intended to provide a brief overview of the concepts, which will be described in detail in the detailed description portion. This summary portion is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0007] Some embodiments of this disclosure provide data archiving methods, apparatuses, electronic devices, media, and program products to address the technical problems mentioned in the background section above.

[0008] In a first aspect, some embodiments of this disclosure provide a data archiving method, including: executing a main table archiving task corresponding to main table archiving task information to generate main table data to be archived, wherein the main table corresponding to the main table archiving task information has a field association relationship with a set of sub-tables; generating at least one sub-table archiving task information corresponding to at least one target sub-table in the sub-table set, wherein the at least one target sub-table is at least one sub-table obtained based on the attribute fields corresponding to the main table data to be archived; for each archiving task information in the archiving task information set, performing the following consumption steps: generating an archiving message group for the archiving task information, wherein the archiving task information set includes: the main table archiving task information and the at least one sub-table archiving task information; consuming messages from the archiving message group to obtain a consumption result group; and generating archiving information for the main table and the at least one target sub-table based on the obtained consumption result group set, wherein the archiving information is information characterizing whether the main table data to be archived and the corresponding at least one sub-table data to be archived are archived to a target database.

[0009] Optionally, generating the archive message group for the archive task information includes: in response to determining that the archive task information is sub-table archive task information, determining the archive server corresponding to the archive task information; controlling the archive server to execute the sub-table archive task corresponding to the archive task information to obtain sub-table data to be archived corresponding to the sub-table archive task information; splitting the sub-table data to be archived to obtain a split dataset to be archived; and generating an archive message for each split dataset to be archived in the split dataset to be archived.

[0010] Optionally, determining the archive server corresponding to the archive task information includes: determining the sub-table archive identifier corresponding to the archive task information; and determining the archive server corresponding to the sub-table archive identifier in response to determining that the ratio of the number of sub-table archive task information to the number of archive servers is a first integer, wherein the number of sub-table archive task information is the number corresponding to at least one sub-table archive task information, and the number of archive servers is the number corresponding to the archive server set.

[0011] Optionally, after determining the archive server corresponding to the sub-table archive identifier in response to determining that the ratio of the number of sub-table archive task information to the number of archive servers is a first integer, the method further includes: in response to determining that the ratio of the number of sub-table archive task information to the number of archive servers is not an integer, rounding the ratio to obtain a second integer; multiplying the second integer by the number of archive servers to determine the number of archive multiple task information; determining the difference between the number of sub-table archive task information and the number of archive multiple task information as the number of remaining task information; determining the sub-table archive task information set corresponding to the number of remaining task information as the remaining sub-table archive task information set; and in response to determining that the sub-table archive identifier is located in the archive identifier set corresponding to the remaining sub-table archive task information set, determining the archive server corresponding to the sub-table archive identifier based on the archive identifier set and the archive server identifier set corresponding to the archive server set.

[0012] Optionally, the method further includes: in response to determining that the archived information is information representing the archived data of the main table and the corresponding at least one sub-table to be archived to the target database, for each consumption result in the consumption result set, performing the following message consumption steps: performing data verification on the archived data corresponding to the consumption result to obtain a first verification result; obtaining the archived data for the consumption result; performing data verification on the archived data to obtain a second verification result; in response to determining that the first verification result and the second verification result are the same, generating a deletion message for the archived data; consuming the deletion message to obtain a message consumption result; and generating deletion information for the main table and the at least one target sub-table based on the obtained message consumption result set, wherein the deletion information is information representing whether the archived data of the main table and the archived data of the sub-table have been deleted from the source database.

[0013] Optionally, the above-mentioned data verification of the data to be archived corresponding to the consumption result to obtain a first verification result includes: concatenating the data to be archived according to a preset polynomial to obtain concatenated data to be archived; and verifying the concatenated data to be archived to obtain a first verification result.

[0014] Optionally, generating archiving information for the main table and at least one target sub-table based on the obtained consumption result set includes: in response to determining that there are consumption results in the consumption result set that indicate archiving failure, determining at least one consumption result that indicates archiving failure; for each of the at least one consumption result, performing the following re-consumption steps based on the archiving message corresponding to the consumption result: determining the number of times the archiving message has been executed; re-consuming the archiving message corresponding to the consumption result to obtain a re-consumption result; in response to determining that the re-consumption result indicates successful archiving, ending the re-consumption steps; in response to determining that the re-consumption result indicates archiving failure and that the number of times the execution has been executed is greater than or equal to a preset number of executions, ending the re-consumption steps and sending the archiving message to the monitoring terminal; in response to determining that the re-consumption result indicates archiving failure and that the number of times the execution has been executed is less than the preset number of executions, re-executing the re-consumption steps.

[0015] Optionally, the above-mentioned execution of the main table archiving task information corresponds to the main table archiving task to generate the main table data to be archived, including: generating the main table archiving statement corresponding to the above-mentioned main table archiving task information; determining the archiving server corresponding to the above-mentioned main table archiving task information; and controlling the above-mentioned archiving server to parse the above-mentioned main table archiving statement to obtain the main table data to be archived.

[0016] Secondly, some embodiments of this disclosure provide a data archiving apparatus, comprising: a first execution unit configured to execute a main table archiving task corresponding to main table archiving task information to generate main table data to be archived, wherein the main table corresponding to the main table archiving task information has a field association relationship with a set of sub-tables; a first generation unit configured to generate at least one sub-table archiving task information corresponding to at least one target sub-table in the sub-table set, wherein the at least one target sub-table is at least one sub-table obtained based on the attribute fields corresponding to the main table data to be archived; a second execution unit configured to perform the following consumption steps for each archiving task information in the archiving task information set: generating an archiving message group for the archiving task information, wherein the archiving task information set includes: the main table archiving task information and the at least one sub-table archiving task information; consuming the archiving message group to obtain a consumption result group; and a second generation unit configured to generate archiving information for the main table and the at least one target sub-table based on the obtained consumption result group set, wherein the archiving information is information characterizing whether the main table data to be archived and the corresponding at least one sub-table data to be archived are archived to a target database.

[0017] Optionally, the execution unit may be further configured to: in response to determining that the above-mentioned archiving task information is sub-table archiving task information, determine the archiving server corresponding to the above-mentioned archiving task information; control the above-mentioned archiving server to execute the sub-table archiving task corresponding to the above-mentioned archiving task information, and obtain the sub-table data to be archived corresponding to the above-mentioned sub-table archiving task information; split the above-mentioned sub-table data to be archived to obtain the split data set to be archived; and generate an archiving message for each split data set to be archived in the split data set to be archived.

[0018] Optionally, the execution unit is further configured to: determine the sub-table archive identifier corresponding to the above-mentioned archive task information; and in response to determining that the ratio of the number of sub-table archive task information to the number of archive servers is a first integer, determine the archive server corresponding to the above-mentioned sub-table archive identifier, wherein the number of the above-mentioned sub-table archive task information is the number corresponding to at least one of the above-mentioned sub-table archive task information, and the number of the above-mentioned archive servers is the number corresponding to the set of archive servers.

[0019] Optionally, the execution unit may be further configured to: in response to determining that the ratio of the number of sub-table archive task information to the number of archive servers is not an integer, round the ratio to obtain a second integer; multiply the second integer by the number of archive servers to determine the number of archive multiple task information; determine the difference between the number of sub-table archive task information and the number of archive multiple task information to determine the number of remaining task information; determine the sub-table archive task information set corresponding to the number of remaining task information as the remaining sub-table archive task information set; and in response to determining that the sub-table archive identifier is located in the archive identifier set corresponding to the remaining sub-table archive task information set, determine the archive server corresponding to the sub-table archive identifier based on the archive identifier set and the archive server identifier set corresponding to the archive server set.

[0020] Optionally, the data archiving apparatus further includes a third execution unit and a third generation unit. The third execution unit is configured to: in response to determining that the archiving information represents the archiving of the main table data to be archived and the corresponding at least one sub-table data to be archived to the target database, for each consumption result in the consumption result set, perform the following message consumption steps: perform data verification on the data to be archived corresponding to the consumption result to obtain a first verification result; obtain the archived data for the consumption result; perform data verification on the archived data to obtain a second verification result; in response to determining that the first verification result and the second verification result are the same, generate a deletion message for the data to be archived; consume the deletion message to obtain a message consumption result. The third generation unit is configured to: generate deletion information for the main table and the at least one target sub-table based on the obtained message consumption result set, wherein the deletion information represents whether the main table data to be archived and the sub-table data to be archived have been deleted from the source database.

[0021] Optionally, the third execution unit is further configured to: concatenate the above-mentioned data to be archived according to a preset polynomial to obtain concatenated data to be archived; and verify the above-mentioned concatenated data to be archived to obtain a first verification result.

[0022] Optionally, the second generation unit is further configured to: in response to determining that there are consumption results in the above-mentioned consumption result set that indicate archiving failure, determine at least one consumption result that indicates archiving failure; for each of the above-mentioned at least one consumption result, according to the archived message corresponding to the above-mentioned consumption result, perform the following re-consumption steps: determine the number of times the archived message has been executed; re-consume the archived message corresponding to the above-mentioned consumption result to obtain a re-consumption result; in response to determining that the above-mentioned re-consumption result indicates successful archiving, end the above-mentioned re-consumption steps; in response to determining that the above-mentioned re-consumption result indicates archiving failure and the number of times the execution has been performed is greater than or equal to the preset number of executions, end the above-mentioned re-consumption steps and send the above-mentioned archived message to the monitoring terminal; in response to determining that the above-mentioned re-consumption result indicates archiving failure and the number of times the execution has been performed is less than the preset number of executions, perform the above-mentioned re-consumption steps again.

[0023] Optionally, the first execution unit is further configured to: generate a master table archiving statement corresponding to the master table archiving task information; determine an archiving server corresponding to the master table archiving task information; and control the archiving server to parse the master table archiving statement to obtain the master table data to be archived.

[0024] Thirdly, some embodiments of this disclosure provide an electronic device, including: one or more processors; and a storage device having one or more programs stored thereon, such that when the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any implementation of the first aspect.

[0025] Fourthly, some embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method as described in any implementation of the first aspect.

[0026] Fifthly, some embodiments of this disclosure provide a computer program product, including a computer program that, when executed by a processor, implements the method described in any of the implementations of the first aspect above.

[0027] The above embodiments of this disclosure have the following beneficial effects: By using the data archiving methods of some embodiments of this disclosure, data archiving is performed on the main table and multiple sub-tables through message generation and consumption, thereby improving the integrity and efficiency of archiving. Specifically, the reasons for low database performance, high resource consumption, low archiving efficiency, and long archiving time are: archiving data in a single table may result in incomplete archived data, leading to low archiving efficiency and consequently high archiving resource consumption. Based on this, the data archiving method of some embodiments of this disclosure first executes the main table archiving task corresponding to the main table archiving task information to generate main table data to be archived, wherein the main table corresponding to the main table archiving task information has a field association relationship with the sub-table set. Here, the obtained main table data to be archived and the field association relationship can be used to subsequently determine at least one target sub-table. Second, at least one sub-table archiving task information corresponding to at least one target sub-table in the sub-table set is generated, wherein the at least one target sub-table is at least one sub-table obtained based on the attribute fields corresponding to the main table data to be archived. Here, the at least one target sub-table obtained can determine the data archiving scope of the archived data in the sub-table. Then, for each archived task information in the archived task information set, the following consumption steps are performed: First, an archived message group is generated for the above-mentioned archived task information, wherein the above-mentioned archived task information set includes: the above-mentioned main table archived task information and the above-mentioned at least one sub-table archived task information. Here, through the mapping relationship between the archived task information and at least one archived message, subsequent consumption of at least one archived message can effectively improve archiving efficiency and reduce the database archiving load. Second, the above-mentioned archived message group is consumed to obtain a consumption result group. Here, the data archiving status can be understood through the consumption result group. By re-archiving the consumption results that indicate archiving failure, the integrity of the archived data is ensured. Finally, based on the obtained consumption result group set, archived information is generated for the above-mentioned main table and the above-mentioned at least one target sub-table, wherein the above-mentioned archived information indicates whether the data to be archived in the above-mentioned main table and the corresponding data to be archived in the at least one sub-table have been archived to the target database. Here, data archiving of the data in the main table and the corresponding at least one target sub-table can improve the integrity of the archived data. Therefore, by utilizing message generation and consumption, data can be archived in the main table and multiple sub-tables, thereby improving the integrity and efficiency of data archiving. Attached Figure Description

[0028] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.

[0029] Figure 1 This is a schematic diagram illustrating an application scenario of a data archiving method according to some embodiments of the present disclosure;

[0030] Figure 2 This is a flowchart of some embodiments of the data archiving method according to this disclosure;

[0031] Figure 3 These are flowcharts of other embodiments of the data archiving method according to this disclosure;

[0032] Figure 4 This is a flowchart of yet another embodiment of the data archiving method according to this disclosure;

[0033] Figure 5 These are schematic diagrams illustrating the structure of some embodiments of the data archiving apparatus according to this disclosure;

[0034] Figure 6 This is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. Detailed Implementation

[0035] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0036] It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0037] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0038] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0039] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0040] Before performing any of the operations involving the collection, storage, or use of user personal information (such as user browsing history information and user purchase history information) disclosed in this disclosure, the relevant organizations or individuals shall fulfill their obligations, including conducting personal information security impact assessments, informing personal information subjects, and obtaining prior authorization and consent from personal information subjects.

[0041] This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0042] Figure 1 This is a schematic diagram illustrating an application scenario of a data archiving method according to some embodiments of the present disclosure.

[0043] exist Figure 1In this application scenario, electronic device 101 can first execute the main table archiving task corresponding to the main table archiving task information 1021 to generate the main table data to be archived. The main table 103 corresponding to the aforementioned main table archiving task information has field relationships with the sub-table set 104. In this application scenario, the aforementioned sub-table set can include: a first sub-table 1041 and a second sub-table 1042. Fields in the main table 103 can include: ID, activity ID, user ID, product name, and creation time. Fields in the first sub-table 1041 can include: ID, activity ID, user ID, and creation time. Fields in the second sub-table 1042 can include: ID, activity ID, product name, and creation time. Next, at least one sub-table archiving task information 1022 corresponding to at least one target sub-table 105 in the sub-table set 104 is generated, wherein at least one target sub-table 105 is at least one sub-table obtained based on the attribute fields corresponding to the main table data to be archived. In this scenario, the at least one target sub-table 105 may include: a first target sub-table 1051 and a second target sub-table 1052. The second target sub-table may be an empty table. Fields in the first target sub-table 1051 may include: ID, activity ID, user ID, and creation time. The corresponding attribute field may have an activity ID of 1001. The at least one sub-table archive task information 1022 may include: a first sub-table archive task information 10221 and a second sub-table archive task information 10222. Then, for each archive task information in the archive task information set 102, the following consumption steps are performed: An archive message group 106 is generated for the archive task information 102. In this application scenario, the archive message group 106 may include: a first archive message, a second archive message, a third archive message, and a fourth archive message. Messages are consumed from the archive message group 106 to obtain a consumption result group 107. In this application scenario, the consumption result group 107 may include: a first consumption result, a second consumption result, a third consumption result, and a fourth consumption result. Finally, based on the obtained set of consumption results, an archive message is generated for the main table 103 and at least one target sub-table 105. This archive message indicates whether the data to be archived in the main table and the corresponding data to be archived in the at least one sub-table are archived to the target database.

[0044] It should be noted that the aforementioned electronic device 101 can be either hardware or software. When the electronic device is hardware, it can be implemented as a distributed cluster consisting of multiple servers or terminal devices, or as a single server or a single terminal device. When the electronic device is software, it can be installed in the hardware devices listed above. It can be implemented as, for example, multiple software programs or software modules used to provide distributed services, or as a single software program or software module. No specific limitations are made here.

[0045] It should be understood that Figure 1The number of electronic devices shown is merely illustrative. Any number of electronic devices can be used depending on the implementation requirements.

[0046] Continue to refer to Figure 2 The diagram illustrates a flow 200 of some embodiments of a data archiving method according to the present disclosure. This data archiving method includes the following steps:

[0047] Step 201: Execute the main table archiving task corresponding to the main table archiving task information to generate the main table data to be archived.

[0048] In some embodiments, the entity executing the above data archiving method (e.g.) Figure 1 The electronic device 101 shown can execute the main table archiving task corresponding to the main table archiving task information to generate main table data to be archived. The main table corresponding to the main table archiving task information has field relationships with the sub-table set. The main table archiving task information can be the task information for the main table archiving task. The main table archiving task can be a task that archives data in the main table and consists of multiple steps. The main table archiving task can include the following steps: First, using SQL (Structured Query Language) statements, obtain the main table data to be archived corresponding to the main table archiving task information. Second, archive the main table data to be archived to the target database. The main table data to be archived can be data from the main table that needs to be archived. For example, the main table data to be archived can be data from the active table. The sub-table set can be a set of sub-tables that have field relationships with the main table. The sub-table relationships can include multiple relationships. The relationships can be table relationships between the main table and sub-tables in the sub-table set. The table relationships can be relationships where the primary key value of the main table is the same as or corresponds to the foreign key value of the sub-table.

[0049] In some optional implementations of certain embodiments, the aforementioned execution entity may execute the main table archiving task corresponding to the main table archiving task information to generate the main table data to be archived, which may include the following steps:

[0050] The first step is for the execution entity to generate a master table archiving statement corresponding to the master table archiving task information. This master table archiving statement can be a statement that extracts data to be archived from the master table. For example, the master table archiving statement can be an SQL statement.

[0051] In practice, the aforementioned executing entity can first determine the content of the main table fields related to the archiving task information. The main table field content can be the content of the data to be archived in the main table. This main table field content can include, but is not limited to, at least one of the following: source database information, main table name, archiving time period, and field relationships. The source database information can be the information of the source database from which the data to be archived is extracted. For example, the source database can be a relational database or a non-relational database. Then, based on the aforementioned main table field content, the main table archiving statement is generated.

[0052] The second step involves the execution entity identifying the archive server corresponding to the main table archive task information. This archive server can be the server that executes the main table archive task information.

[0053] In practice, the aforementioned executing entity can use a random allocation algorithm to determine the archive server corresponding to the archive task information in the aforementioned master table.

[0054] Third, the aforementioned executing entity can control the aforementioned archiving server to parse the aforementioned main table archiving statements and obtain the main table data to be archived.

[0055] Step 202: Generate at least one sub-table archiving task information corresponding to at least one target sub-table in the sub-table set.

[0056] In some embodiments, the execution entity can generate at least one sub-table archiving task information corresponding to at least one target sub-table in the sub-table set. The at least one target sub-table is at least one sub-table obtained based on the attribute fields corresponding to the data to be archived in the main table. The target sub-table in the at least one target sub-table can be the sub-table corresponding to the sub-table data to be archived. The corresponding attribute fields can be fields representing the attributes of the data to be archived in the main table. For example, the corresponding attribute fields can include, but are not limited to, at least one of the following: activity number, activity name, and activity time. The at least one sub-table archiving task information can represent the information of the sub-table archiving task. The sub-table archiving task can be a task that archives data in a sub-table and includes multiple steps. The sub-table archiving task can include the following steps: using SQL (Structured Query Language) statements to obtain the sub-table data to be archived corresponding to the sub-table archiving task information; and secondly, archiving the sub-table data to be archived to the target database.

[0057] As an example, the execution entity can first determine the related fields in the main table. Secondly, it can determine the set of related tables including the related fields, as a set of sub-tables. Thirdly, it can determine the attribute fields of the related fields. Then, it can filter out the sub-table data corresponding to the attribute fields from each sub-table in the set of sub-tables, as target sub-table data, obtaining at least one target sub-table. Finally, it can archive the at least one target sub-table data as at least one target sub-table, obtaining at least one target sub-table.

[0058] Step 203: For each archive task in the archive task information set, perform the following consumption steps:

[0059] Step 2031: Generate an archive message group for archive task information.

[0060] In some embodiments, the executing entity can generate an archive message group for the aforementioned archive task information. The archive task information may include: main table archive task information and at least one sub-table archive task information. The archive messages in the archive message group may be messages that archive data to be archived to the target database. The data to be archived may be data to be archived from the main table or data to be archived from a sub-table. In practice, the executing entity can encapsulate the data to be archived corresponding to the archive task information to obtain the archive message group.

[0061] Optionally, the aforementioned executing entity may generate an archive message group for the aforementioned archive task information, which may include the following steps:

[0062] The first step is to determine the archive server corresponding to the above archive task information in response to the main table archive task message.

[0063] In practice, the aforementioned executing entity can use a random allocation algorithm to determine the archive server corresponding to the aforementioned archive task information.

[0064] The second step involves controlling the aforementioned archive server to execute the main table archive task corresponding to the main table archive task information, thereby obtaining the main table data to be archived corresponding to the main table archive task information. It should be noted that for the specific implementation and its resulting technical effects, please refer to [link to relevant documentation / reference]. Figure 2 Step 201 in the corresponding embodiment will not be repeated here.

[0065] The third step is to split the data to be archived in the main table to obtain the split main table dataset to be archived.

[0066] The dataset to be archived in the main table after the above split can be a dataset containing a preset data volume. The preset data volume can be any value in [50, 100].

[0067] As an example, the aforementioned execution entity can split the data to be archived in the main table according to a preset data volume to obtain the split main table dataset to be archived.

[0068] The fourth step is to generate an archive message for each piece of data in the split master table to be archived in the above-mentioned split master table to be archived data.

[0069] In practice, the aforementioned executing entity can encapsulate the data to be archived in the split main table to obtain an archive message for the data to be archived in the split main table.

[0070] Step 2032: Consume messages from the archived message group to obtain the consumption result group.

[0071] In some embodiments, the execution entity can consume the archived message group to obtain a consumption result group. The consumption result in the consumption result group indicates whether the consumption of the archived messages was successful.

[0072] In practice, the aforementioned executing entity can first receive the archive message group. Then, it can control the corresponding set of archive servers to execute the archive tasks in the archive message group and obtain the consumption result group.

[0073] Step 204: Based on the obtained consumption result set, generate archive information for the main table and at least one target sub-table.

[0074] In some embodiments, the executing entity can generate archiving information for the main table and the at least one target sub-table based on the obtained consumption result set. The archiving information indicates whether the archived data of the main table and the corresponding archived data of the at least one sub-table have been archived to the target database. The target database can be a database that stores archived data. For example, the target database can be a relational database or a non-relational database.

[0075] As an example, the aforementioned execution entity may first determine whether each consumption result in the obtained consumption result set indicates successful archiving. Then, in response to determining that each consumption result in the aforementioned consumption result set indicates successful archiving, a message is obtained regarding the archiving of the aforementioned main table and at least one of the aforementioned target sub-tables to the aforementioned target database. Finally, in response to determining that there are consumption results in the aforementioned consumption result set indicating archiving failure, a message is obtained regarding the data to be archived corresponding to the consumption results indicating archiving failure not being archived to the aforementioned target database.

[0076] In some optional implementations of certain embodiments, the execution entity may generate archive information for the main table and at least one target sub-table based on the obtained consumption result set, which may include the following steps:

[0077] The first step is to determine, in response to the determination that there is a consumption result in the above-mentioned consumption result set that represents archiving failure, at least one consumption result that represents archiving failure.

[0078] The second step involves performing the following re-consumption steps for each of the at least one consumption result mentioned above, based on the archived message corresponding to that consumption result:

[0079] The first sub-step involves determining the number of times the aforementioned archived message has been executed. This number of executions can represent the number of times an archived message that failed to be archived has been consumed. The initial value of this number of executions can be 1.

[0080] The second sub-step involves re-consuming the archived messages corresponding to the aforementioned consumption results to obtain a re-consumption result. This re-consumption result represents the consumption result obtained by re-consuming archived messages that failed to be archived.

[0081] The third sub-step, in response to determining that the above re-consumption result represents successful archiving, ends the above re-consumption step.

[0082] The fourth sub-step involves, in response to determining that the re-consumption result indicates archiving failure and that the number of executions is greater than or equal to a preset number of executions, ending the re-consumption step and sending the archived message to the monitoring terminal. The preset number of executions can be 3. The monitoring terminal can be a terminal that receives the consumption message indicating archiving failure.

[0083] Third, in response to the determination that the above-mentioned re-consumption result indicates archiving failure and that the number of executions is less than the above-mentioned preset number of executions, the above-mentioned re-consumption step is executed again.

[0084] The above embodiments of this disclosure have the following beneficial effects: By using the data archiving methods of some embodiments of this disclosure, data archiving is performed on the main table and multiple sub-tables through message generation and consumption, thereby improving the integrity and efficiency of archiving. Specifically, the reasons for low database performance, high resource consumption, low archiving efficiency, and long archiving time are: archiving data in a single table may result in incomplete archived data, leading to low archiving efficiency and consequently high archiving resource consumption. Based on this, the data archiving method of some embodiments of this disclosure first executes the main table archiving task corresponding to the main table archiving task information to generate main table data to be archived, wherein the main table corresponding to the main table archiving task information has a field association relationship with the sub-table set. Here, the obtained main table data to be archived and the field association relationship can be used to subsequently determine at least one target sub-table. Second, at least one sub-table archiving task information corresponding to at least one target sub-table in the sub-table set is generated, wherein the at least one target sub-table is at least one sub-table obtained based on the attribute fields corresponding to the main table data to be archived. Here, the at least one target sub-table obtained can determine the data archiving scope of the archived data in the sub-table. Then, for each archived task information in the archived task information set, the following consumption steps are performed: First, an archived message group is generated for the above-mentioned archived task information, wherein the above-mentioned archived task information set includes: the above-mentioned main table archived task information and the above-mentioned at least one sub-table archived task information. Here, through the mapping relationship between the archived task information and at least one archived message, subsequent consumption of at least one archived message can effectively improve archiving efficiency and reduce the database archiving load. Second, the above-mentioned archived message group is consumed to obtain a consumption result group. Here, the data archiving status can be understood through the consumption result group. By re-archiving the consumption results that indicate archiving failure, the integrity of the archived data is ensured. Finally, based on the obtained consumption result group set, archived information is generated for the above-mentioned main table and the above-mentioned at least one target sub-table, wherein the above-mentioned archived information indicates whether the data to be archived in the above-mentioned main table and the corresponding data to be archived in the at least one sub-table have been archived to the target database. Here, data archiving of the data in the main table and the corresponding at least one target sub-table can improve the integrity of the archived data. Therefore, by utilizing message generation and consumption, data can be archived in the main table and multiple sub-tables, thereby improving the integrity and efficiency of data archiving.

[0085] Further reference Figure 3 The diagram illustrates a flow 300 of another embodiment of the data archiving method according to this disclosure. This data archiving method includes the following steps:

[0086] Step 301: Execute the main table archiving task corresponding to the main table archiving task information to generate the main table data to be archived.

[0087] Step 302: Generate at least one sub-table archiving task information corresponding to at least one target sub-table in the sub-table set.

[0088] Step 303: For each archive task in the archive task information set, perform the following consumption steps:

[0089] Step 3031: Generate an archive message group for archive task information.

[0090] Step 3032: Consume the archived message group to obtain the consumption result group.

[0091] Step 304: Based on the obtained consumption result set, generate archive information for the main table and at least one target sub-table.

[0092] In some embodiments, the specific implementation of steps 301-304 and the resulting technical effects can be found in [reference needed]. Figure 2 Steps 201-204 in the corresponding embodiments will not be repeated here.

[0093] Step 305: In response to determining that the archiving information is information representing the archiving of data to be archived in the main table and corresponding data to be archived in at least one sub-table to the target database, for each consumption result in the consumption result set, perform the following message consumption steps:

[0094] Step 3051: Perform data verification on the data to be archived corresponding to the consumption result to obtain the first verification result.

[0095] In some embodiments, the executing entity may perform data verification on the data to be archived corresponding to the consumption result to obtain a first verification result. This first verification result may be a result indicating whether the data to be archived is completely archived.

[0096] In practice, the aforementioned executing entity can use a message digest algorithm to perform data verification on the data to be archived corresponding to the aforementioned consumption results, and obtain the first verification result.

[0097] In some optional implementations of certain embodiments, the execution entity may perform data verification on the data to be archived corresponding to the consumption result to obtain a first verification result, including:

[0098] The first step involves the aforementioned execution entity concatenating the data to be archived according to a preset polynomial to obtain concatenated data to be archived. The preset polynomial can be a polynomial used to verify the data to be archived. The preset polynomial can be a pre-determined polynomial represented in binary form with coefficients of 1 or 0. The concatenated data to be archived can be the data obtained by concatenating the data to be archived with the preset polynomial. Alternatively, the concatenated data to be archived can be a binary representation obtained by concatenating the binary representation of the data to be archived and the binary representation of the preset polynomial, followed by parsing the binary representation to obtain the data to be archived.

[0099] As an example, the aforementioned execution entity can first represent the data to be archived in binary form, obtaining a binary representation of the data to be archived. Then, it can represent the aforementioned preset polynomial in binary form, obtaining a binary representation of the polynomial. Finally, it can concatenate the binary representation of the data to be archived and the binary representation of the polynomial, obtaining a concatenated binary representation. Finally, it can parse the concatenated binary representation to obtain the concatenated data to be archived.

[0100] The second step is for the aforementioned executing entity to verify the spliced ​​data to be archived and obtain the first verification result.

[0101] In practice, the aforementioned executing entity can first represent the concatenated data to be archived in binary form, obtaining the binary representation of the concatenated data to be archived. Then, it can perform a modulo-2 operation on the aforementioned preset polynomial to obtain the first verification result.

[0102] Step 3052: Obtain the archived data for the consumption results.

[0103] In some embodiments, the executing entity may obtain archived data related to the consumption results. This archived data may be data already archived in the target database. The archived data may be data from a main table in the target database, or data from a sub-table in the target database.

[0104] Step 3053: Perform data verification on the archived data to obtain the second verification result.

[0105] In some embodiments, the aforementioned executing entity may perform data verification on the archived data to obtain a second verification result. The second verification result indicates whether the archived data is completely archived. If the second verification result indicates that the archived data is incomplete, it means that an error occurred in the data to be archived during the archiving process. If the second verification result indicates that the archived data is complete, it means that no errors occurred in the data to be archived during the archiving process. The specific implementation of step 3053 and its resulting technical effects can be referred to step 3051, and will not be repeated here.

[0106] Step 3054: In response to determining that the first verification result and the second verification result are the same, a deletion message is generated for the data to be archived.

[0107] In some embodiments, the executing entity may generate a deletion message for the data to be archived in response to determining that the first verification result and the second verification result are the same. The deletion message may be a message indicating that the data to be archived corresponding to the consumption result is deleted from the source database.

[0108] Step 3055: Consume the deleted messages to obtain the message consumption results.

[0109] In some embodiments, the execution entity can consume the deletion message to obtain a message consumption result. This message consumption result can be a result indicating whether the deletion from the source database was successful.

[0110] Step 306: Based on the obtained message consumption result set, generate deletion information for the main table and at least one target sub-table.

[0111] In some embodiments, the execution entity can generate deletion information for the main table and at least one target sub-table based on the obtained message consumption result set. The deletion information may be information indicating whether to delete the archived data in the main table and the archived data in the sub-table from the source database.

[0112] As an example, the aforementioned execution entity can first determine whether each message consumption result in the obtained message consumption result group indicates successful deletion. Then, in response to determining that each message consumption result in the aforementioned message consumption result group indicates successful deletion, a message indicating successful deletion from the aforementioned source database is obtained for the aforementioned main table and at least one of the aforementioned target sub-tables. Finally, in response to determining that there are message consumption results in the aforementioned message consumption result group indicating failed deletion, a message indicating that the archived data corresponding to the message consumption results indicating failed deletion was not successfully deleted from the source database is obtained.

[0113] from Figure 3 It can be seen from this that, with Figure 2 Compared to the description of some corresponding embodiments, Figure 3 The data archiving method flow 300 in some corresponding embodiments further emphasizes the extension of the step of deleting the data to be archived from the aforementioned source database. Therefore, the schemes described in these embodiments perform data verification on both the data to be archived and the archived data, which can improve the accuracy of the archived data and the efficiency of the data verification.

[0114] Further reference Figure 4 The diagram illustrates a flow 400 of another embodiment of a data archiving method according to the present disclosure. This data archiving method includes the following steps:

[0115] Step 401: Execute the main table archiving task corresponding to the main table archiving task information to generate the main table data to be archived.

[0116] Step 402: Generate at least one sub-table archiving task information corresponding to at least one target sub-table in the sub-table set.

[0117] In some embodiments, the specific implementation of steps 401-402 and the resulting technical effects can be found in [reference needed]. Figure 2 Steps 201-202 in the corresponding embodiments will not be repeated here.

[0118] Step 403: For each archive task in the archive task information set, perform the following consumption steps:

[0119] Step 4031: In response to determining that the archiving task information is a sub-table archiving task information, determine the archiving server corresponding to the archiving task information.

[0120] In some embodiments, the execution entity may determine the archive server corresponding to the archive task information in response to determining that the archive task information is sub-table archive task information.

[0121] In some optional implementations of certain embodiments, the execution entity may determine the archiving server corresponding to the archiving task information, including the following steps:

[0122] The first step is for the executing entity to determine the sub-table archive identifier corresponding to the aforementioned archived task information. This sub-table archive identifier represents the identity information of the aforementioned sub-table archived task information.

[0123] Optionally, the sub-table archiving identifier can represent the order information of the sub-table archiving task information in the at least one sub-table archiving task information.

[0124] In the second step, the aforementioned executing entity may, in response to determining that the ratio of the number of sub-table archiving task information to the number of archiving servers is a first integer, determine the archiving server corresponding to the aforementioned sub-table archiving identifier. Here, the number of sub-table archiving task information can be the number of sub-table archiving task information items among the aforementioned at least one sub-table archiving task information. The number of archiving servers can be the number of archiving servers in an archive server set. The archive server set can be a set of servers that archive the aforementioned main table data to be archived and the corresponding at least one sub-table data to be archived to the aforementioned target database. The aforementioned first integer can be any integer.

[0125] In practice, the aforementioned executing entity can first determine the ratio of the number of sub-table archiving task information to the number of archiving servers. Then, based on at least one sub-table archiving task information corresponding to at least one sub-table archiving identifier and in ascending order, the sub-table archiving task information of the ratio is evenly distributed to each archiving server in the aforementioned archiving server set. Finally, the archiving server corresponding to the sub-table archiving task information is determined through the sub-table archiving identifier.

[0126] Optionally, after determining the archive server corresponding to the sub-table archive identifier in response to determining that the ratio of the number of sub-table archive task information to the number of archive servers is a first integer, the execution entity may further include the following steps:

[0127] First, the executing entity may, in response to determining that the ratio of the number of archived task information in the sub-table to the number of archived servers is not an integer, round the ratio to obtain a second integer.

[0128] In practice, the aforementioned executing entity can round down the ratio to obtain a second integer. This second integer can be the largest integer smaller than the ratio.

[0129] The second step is for the executing entity to determine the number of archiving multiplier task information by multiplying the second integer by the number of archiving servers.

[0130] Third, the aforementioned executing entity can determine the remaining number of task information by the difference between the number of archived task information in the aforementioned sub-table and the number of archived multiple task information.

[0131] Fourth, the aforementioned executing entity can determine the set of sub-table archived task information corresponding to the number of remaining task information, and use it as the set of remaining sub-table archived task information.

[0132] Fifth, in response to determining that the sub-table archiving identifier is located within the archiving identifier set corresponding to the remaining sub-table archiving task information set, the executing entity determines the archiving server corresponding to the sub-table archiving identifier based on the archiving identifier set and the archiving server identifier set corresponding to the archiving server set. The archiving server identifiers in the archiving server identifier set can represent the identity information of the archiving server. The corresponding archiving server identifier set can be an identifier set with a one-to-one correspondence in quantity with the aforementioned archiving identifier set. For example, the aforementioned archiving identifier set may include: a first archiving identifier and a second archiving identifier. Then, the corresponding archiving server identifier set may include: a first archiving server identifier and a second archiving server identifier. The server executing the sub-table archiving task information corresponding to the first archiving identifier is the archiving server corresponding to the first archiving server identifier. The server executing the sub-table archiving task information corresponding to the second archiving identifier is the archiving server corresponding to the second archiving server identifier.

[0133] As an example, the aforementioned executing entity can first determine the server archive identifier set corresponding to the aforementioned archive identifier set. Then, according to the archive server identifier set, the aforementioned sub-table archive identifier sets are sequentially assigned to the archive servers corresponding to the aforementioned archive server identifier sets, thereby determining the archive servers corresponding to the aforementioned sub-table archive identifiers.

[0134] Optionally, the aforementioned archive server identifier may be an identifier of the location information of the corresponding archive server within the aforementioned archive servers.

[0135] Step 4032: Control the archiving server to execute the archiving task corresponding to the sub-table of the archiving task information, and obtain the sub-table data to be archived corresponding to the sub-table archiving task information.

[0136] In some embodiments, the execution entity can control the archiving server to execute the sub-table archiving task corresponding to the sub-table archiving task information, thereby obtaining the sub-table data to be archived corresponding to the sub-table archiving task information. For the specific implementation of step 4032 and its resulting technical effects, please refer to... Figure 2 Step 201 in the corresponding embodiment will not be repeated here.

[0137] Step 4033: Split the data to be archived in the sub-table to obtain the split dataset to be archived.

[0138] In some embodiments, the execution entity may split the data to be archived in the sub-table to obtain a split dataset to be archived. Each split data point in the split dataset may contain a preset data volume. The preset data volume may be any value within the range [50, 100]. The execution entity may split the data to be archived based on the preset data volume to obtain a split dataset to be archived.

[0139] Step 4034: For each split data to be archived in the split data set to be archived, generate an archive message for the split data to be archived.

[0140] In some embodiments, the execution entity may generate an archive message for each of the split data to be archived in the split data set to be archived.

[0141] As an example, the aforementioned execution entity can perform the following generation steps for each of the split data to be archived in the split data set: encapsulate the split data to be archived to obtain an archive message.

[0142] Step 4035: Consume messages from the archived message group to obtain the consumption result group.

[0143] In some embodiments, the specific implementation of step 4035 and its resulting technical effects can be found in [reference needed]. Figure 2 Step 2032 in the corresponding embodiment will not be repeated here.

[0144] Step 404: Based on the obtained consumption result set, generate archive information for the main table and at least one target sub-table.

[0145] In some embodiments, the specific implementation of step 404 and its resulting technical effects can be found in [reference needed]. Figure 2 Step 204 in the corresponding embodiment will not be repeated here.

[0146] from Figure 4 It can be seen from this that, with Figure 2 Compared to the description of some corresponding embodiments, Figure 4 The data archiving method flow 400 in some corresponding embodiments further emphasizes the specific steps of generating archive message groups for archiving task information. Therefore, the solutions described in these embodiments, by splitting the data to be archived, ensure the amount of data in the archive messages, thereby reducing the load on the database, improving the archiving efficiency of the data to be archived, controlling the scope of archiving failures, and reducing the amount of archived data to be re-archived and the archiving time.

[0147] Further reference Figure 5As an implementation of the methods shown in the above figures, this disclosure provides some embodiments of a data archiving apparatus, which are similar to... Figure 2 Corresponding to the method embodiments shown, this data archiving device can be specifically applied to various electronic devices.

[0148] like Figure 5 As shown, a data archiving device 500 includes: a first execution unit 501, a first generation unit 502, a second execution unit 503, and a second generation unit 504. The first execution unit 501 is configured to: execute the main table archiving task information corresponding to the main table archiving task information to generate main table data to be archived, wherein the main table corresponding to the main table archiving task information has field association relationships with the sub-table set. The first generation unit 502 is configured to: generate at least one sub-table archiving task information corresponding to at least one target sub-table in the sub-table set, wherein the at least one target sub-table is at least one sub-table obtained based on the attribute fields corresponding to the main table data to be archived. The second execution unit 503 is configured to: for each archiving task information in the archiving task information set, perform the following consumption steps: generate an archiving message group for the archiving task information, wherein the archiving task information set includes: the main table archiving task information and the at least one sub-table archiving task information; consume the archiving message group to obtain a consumption result group. The second generation unit 504 is configured to generate archiving information for the main table and the at least one target sub-table based on the obtained consumption result set, wherein the archiving information is information indicating whether the data to be archived in the main table and the corresponding data to be archived in the at least one sub-table are archived to the target database.

[0149] In some optional implementations of certain embodiments, the apparatus 500 further includes a third execution unit and a third generation unit (not shown in the figure). The third execution unit can be configured to: in response to determining that the archiving information represents information indicating that the main table data to be archived and the corresponding at least one sub-table data to be archived have been archived to the target database, for each consumption result in the consumption result set, perform the following message consumption steps: perform data verification on the data to be archived corresponding to the consumption result to obtain a first verification result; obtain the archived data for the consumption result; perform data verification on the archived data to obtain a second verification result; in response to determining that the first verification result and the second verification result are the same, generate a deletion message for the data to be archived; consume the deletion message to obtain a message consumption result. The third generation unit can be configured to: generate deletion information for the main table and the at least one target sub-table based on the obtained message consumption result set, wherein the deletion information represents information indicating whether the main table data to be archived and the sub-table data to be archived have been deleted from the source database.

[0150] In some optional implementations of certain embodiments, the execution unit may be further configured to: in response to determining that the above-mentioned archiving task information is sub-table archiving task information, determine the archiving server corresponding to the above-mentioned archiving task information; control the above-mentioned archiving server to execute the sub-table archiving task corresponding to the above-mentioned archiving task information to obtain sub-table data to be archived corresponding to the above-mentioned sub-table archiving task information; split the above-mentioned sub-table data to be archived to obtain a split dataset to be archived; and generate an archiving message for each split dataset to be archived in the split dataset to be archived.

[0151] In some optional implementations of some embodiments, the execution unit is further configured to: determine the sub-table archive identifier corresponding to the above-mentioned archive task information; and, in response to determining that the ratio of the number of sub-table archive task information to the number of archive servers is a first integer, determine the archive server corresponding to the above-mentioned sub-table archive identifier, wherein the number of the above-mentioned sub-table archive task information is the number corresponding to the at least one sub-table archive task information, and the number of archive servers is the number corresponding to the archive server set.

[0152] In some optional implementations of certain embodiments, the execution unit may be further configured to: in response to determining that the ratio of the number of sub-table archived task information to the number of archived servers is not an integer, round the ratio to obtain a second integer; multiply the second integer by the number of archived servers to determine the number of archived multiple task information; determine the difference between the number of sub-table archived task information and the number of archived multiple task information to determine the number of remaining task information; determine the sub-table archived task information set corresponding to the number of remaining task information as the remaining sub-table archived task information set; in response to determining that the sub-table archived identifier is located in the archived identifier set corresponding to the remaining sub-table archived task information set, determine the archived server corresponding to the sub-table archived identifier based on the archived identifier set and the archived server identifier set corresponding to the archived server set.

[0153] In some optional implementations of some embodiments, the third execution unit is further configured to: concatenate the data to be archived according to a preset polynomial to obtain concatenated data to be archived; and verify the concatenated data to be archived to obtain a first verification result.

[0154] In some optional implementations of certain embodiments, the second generation unit is further configured to: in response to determining that there are consumption results in the aforementioned consumption result set that indicate archiving failure, determine at least one consumption result indicating archiving failure; for each of the aforementioned at least one consumption result, perform the following re-consumption steps based on the archived message corresponding to the aforementioned consumption result: determine the number of times the archived message has been executed; re-consume the archived message corresponding to the aforementioned consumption result to obtain a re-consumption result; in response to determining that the re-consumption result indicates successful archiving, end the aforementioned re-consumption steps; in response to determining that the re-consumption result indicates archiving failure and that the number of times the execution has been executed is greater than or equal to a preset number of executions, end the aforementioned re-consumption steps and send the archived message to the monitoring terminal; in response to determining that the re-consumption result indicates archiving failure and that the number of times the execution has been executed is less than the preset number of executions, perform the aforementioned re-consumption steps again.

[0155] In some optional implementations of some embodiments, the first execution unit is further configured to: generate a main table archiving statement corresponding to the main table archiving task information; determine an archiving server corresponding to the main table archiving task information; and control the archiving server to parse the main table archiving statement to obtain the main table data to be archived.

[0156] It is understandable that the units recorded in the data archiving device 500 are related to the reference. Figure 2 The steps in the described method correspond accordingly. Therefore, the operations, features, and beneficial effects described above for the method also apply to the data archiving device 500 and the units contained therein, and will not be repeated here.

[0157] The following is for reference. Figure 6 It illustrates electronic devices suitable for implementing some embodiments of this disclosure (e.g., Figure 1 A schematic diagram of the structure of electronic device 101)600 in the middle. Figure 6 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this disclosure.

[0158] like Figure 6 As shown, the electronic device 600 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory 602 or a program loaded from a storage device 608 into a random access memory 603. The random access memory 603 also stores various programs and data required for the operation of the electronic device 600. The processing unit 601, the read-only memory 602, and the random access memory 603 are interconnected via a bus 604. An input / output interface 605 is also connected to the bus 604.

[0159] Typically, the following devices can be connected to the input / output interface 605: input devices 606 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 607 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 608 including, for example, magnetic tape, hard disk, etc.; and communication devices 609. Communication device 609 allows electronic device 600 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 6 An electronic device 600 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 6 Each box shown can represent a device or multiple devices as needed.

[0160] In particular, according to some embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 609, or installed from a storage device 608, or installed from a read-only memory 602. When the computer program is executed by the processing device 601, it performs the functions defined above in the methods of some embodiments of this disclosure.

[0161] It should be noted that, in some embodiments of this disclosure, the computer-readable medium described above may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In some embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0162] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0163] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs. When the aforementioned one or more programs are executed by the electronic device, the electronic device causes the following: to execute a main table archiving task corresponding to the main table archiving task information to generate main table data to be archived, wherein the main table corresponding to the main table archiving task information has field association relationships with the sub-table set; to generate at least one sub-table archiving task information corresponding to at least one target sub-table in the sub-table set, wherein the at least one target sub-table is at least one sub-table obtained based on the attribute fields corresponding to the main table data to be archived; and for each archiving task information in the archiving task information set, to perform the following consumption steps: to generate an archiving message group for the archiving task information, wherein the archiving task information set includes: the main table archiving task information and the at least one sub-table archiving task information; to consume the archiving message group to obtain a consumption result group; and to generate archiving information for the main table and the at least one target sub-table based on the obtained consumption result group set, wherein the archiving information is information indicating whether the main table data to be archived and the corresponding at least one sub-table data to be archived are archived to the target database.

[0164] Computer program code for performing operations of some embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0165] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0166] The units described in some embodiments of this disclosure can be implemented in software or hardware. The described units can also be housed in a processor; for example, a processor may be described as including a first execution unit, a first generation unit, a second execution unit, and a second generation unit. The names of these units do not necessarily limit the unit itself; for example, the first execution unit may also be described as "a unit that executes a main table archiving task corresponding to the main table archiving task information to generate main table data to be archived, wherein the main table corresponding to the main table archiving task information has field association relationships with the sub-table set."

[0167] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0168] Some embodiments of this disclosure also provide a computer program product, including a computer program that, when executed by a processor, implements any of the data archiving methods described above.

[0169] The above description is merely a selection of preferred embodiments of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A data archiving method, comprising: The main table archiving task information is executed to generate the main table data to be archived. The main table and the sub-table set corresponding to the main table archiving task information have field association relationships. Generate archiving task information for at least one target sub-table corresponding to at least one target sub-table in the sub-table set, wherein the at least one target sub-table is at least one sub-table obtained based on the attribute fields corresponding to the data to be archived in the main table; For each archive task in the archive task information set, perform the following consumption steps: Generate an archive message group for the archive task information, wherein the archive task information set includes: the main table archive task information and the archive task information of at least one sub-table; The process of consuming the archived message group to obtain a consumption result group includes: receiving the archived message group; controlling the corresponding archived server set to execute the archived tasks in the archived message group to obtain the consumption result group, wherein the consumption result in the consumption result group indicates whether the consumption of the archived message was successful; Based on the obtained consumption result set, archive information is generated for the main table and the at least one target sub-table, wherein the archive information is information that characterizes whether the data to be archived in the main table and the corresponding data to be archived in the at least one sub-table are archived to the target database.

2. The method according to claim 1, wherein, The generation of the archive message group for the archive task information includes: In response to determining that the archiving task information is sub-table archiving task information, the archiving server corresponding to the archiving task information is determined; Control the archiving server to execute the archiving task corresponding to the archiving task information of the sub-table, and obtain the sub-table data to be archived corresponding to the archiving task information of the sub-table; The data to be archived in the sub-table is split to obtain the split dataset to be archived; For each piece of data to be archived in the split and archived dataset, generate an archive message for that piece of data.

3. The method according to claim 2, wherein, The step of determining the archive server corresponding to the archive task information includes: Determine the archive identifier of the sub-table corresponding to the archive task information; In response to determining that the ratio of the number of sub-table archive task information to the number of archive servers is a first integer, the archive server corresponding to the sub-table archive identifier is determined, wherein the number of sub-table archive task information is the number corresponding to the at least one sub-table archive task information, and the number of archive servers is the number corresponding to the archive server set.

4. The method according to claim 3, wherein, After determining the archive server corresponding to the sub-table archive identifier in response to determining that the ratio of the number of sub-table archive task information to the number of archive servers is a first integer, the method further includes: In response to determining that the ratio of the number of archived task information in the sub-table to the number of archived servers is not an integer, the ratio is rounded down to obtain a second integer. The product of the second integer and the number of archive servers is determined as the number of archive multiplier task information. The difference between the number of archived task information in the sub-table and the number of archived multiple task information is determined as the number of remaining task information. Determine the set of sub-table archived task information corresponding to the number of remaining task information, and use it as the set of remaining sub-table archived task information. In response to determining that the sub-table archive identifier is located within the archive identifier set corresponding to the remaining sub-table archive task information set, the archive server corresponding to the sub-table archive identifier is determined based on the archive identifier set and the archive server identifier set corresponding to the archive server set.

5. The method according to claim 1, wherein, The method further includes: In response to determining that the archiving information is information characterizing the data to be archived in the main table and the corresponding data to be archived in at least one sub-table, the following message consumption steps are performed for each consumption result in the consumption result set: The data to be archived corresponding to the consumption result is validated to obtain the first validation result; Retrieve the archived data for the consumption results; The archived data is validated to obtain a second validation result; In response to determining that the first verification result and the second verification result are the same, a deletion message is generated for the data to be archived. The deleted message is consumed to obtain the message consumption result; Based on the obtained message consumption result set, deletion information is generated for the main table and the at least one target sub-table, wherein the deletion information is information indicating whether the data to be archived in the main table and the data to be archived in the sub-table have been deleted from the source database.

6. The method according to claim 5, wherein, The step of performing data verification on the data to be archived corresponding to the consumption result to obtain a first verification result includes: The data to be archived is concatenated according to a preset polynomial to obtain concatenated data to be archived. The spliced ​​data to be archived is verified to obtain the first verification result.

7. The method according to claim 1, wherein, The step of generating archive information for the main table and the at least one target sub-table based on the obtained consumption result set includes: In response to determining that there is a consumption result in the set of consumption results that represents archiving failure, at least one consumption result representing archiving failure is determined; For each of the at least one consumption result, the following re-consumption steps are performed based on the archived message corresponding to the consumption result: Determine the number of times the archived message has been executed; The archived message corresponding to the consumption result is re-consumed to obtain a re-consumption result; Upon determining that the re-consumption result indicates successful archiving, the re-consumption step ends; In response to determining that the re-consumption result indicates archiving failure and that the number of executions is greater than or equal to the preset number of executions, the re-consumption step is terminated, and the archived message is sent to the monitoring terminal; In response to determining that the re-consumption result indicates archiving failure and that the number of executions is less than the preset number of executions, the re-consumption step is executed again.

8. The method according to claim 1, wherein, The execution of the main table archiving task information corresponds to the main table archiving task, which generates the main table data to be archived, including: Generate the main table archiving statement corresponding to the main table archiving task information; Determine the archive server corresponding to the archive task information in the main table; Control the archive server to parse the archive statements in the main table and obtain the data to be archived in the main table.

9. A data archiving apparatus, comprising: The first execution unit is configured to execute the main table archiving task corresponding to the main table archiving task information to generate the main table to be archived data, wherein the main table corresponding to the main table archiving task information has a field association relationship with the sub-table set; The first generation unit is configured to generate at least one sub-table archiving task information corresponding to at least one target sub-table in the sub-table set, wherein the at least one target sub-table is at least one sub-table obtained according to the attribute fields corresponding to the data to be archived in the main table; The second execution unit is configured to perform the following consumption steps for each archive task information in the archive task information set: generating an archive message group for the archive task information, wherein the archive task information set includes: the main table archive task information and the at least one sub-table archive task information; consuming the archive message group to obtain a consumption result group, including: receiving the archive message group; controlling the corresponding archive server set to execute the archive tasks in the archive message group to obtain a consumption result group, wherein the consumption result in the consumption result group indicates whether the consumption of the archive message was successful; The second generation unit is configured to generate archiving information for the main table and the at least one target sub-table based on the obtained consumption result set, wherein the archiving information is information characterizing whether the data to be archived in the main table and the corresponding data to be archived in the at least one sub-table are archived to the target database.

10. An electronic device, comprising: One or more processors; Storage device, on which one or more programs are stored, When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-8.

11. A computer-readable medium having a computer program stored thereon, wherein, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-8.

12. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-8.

Images