Snapshot processing method, device and electronic device

Abstract

The present invention relates to the field of snapshot technology, and specifically to a snapshot processing method, device, and electronic device, which are applied to a distributed database search and analysis engine. The method includes obtaining a snapshot creation request, the snapshot creation request including a target snapshot name and a target logical table; determining the physical table corresponding to each target logical table based on the status of a target field in the target logical table, the target field being used to represent the database to which the physical table belongs; creating a corresponding snapshot based on the physical table and determining the creation status of each snapshot; and determining snapshot record information corresponding to the target snapshot name based on the snapshot creation request, the physical table, and the creation status, so as to process each snapshot using the snapshot record information. Through this snapshot record information, the complexity of different databases is shielded from the outside. Users only need to access the snapshot record information to know the snapshot creation status corresponding to each physical table, without having to enter the internal query of these two components, thereby improving the efficiency of snapshot processing.

Description

Technical Field

[0001] The present invention relates to the field of snapshot technology, and in particular to a snapshot processing method, device and electronic equipment. Background Art

[0002] In daily development, a popular solution for storing and querying large amounts of data is the combination of HBase and Elasticsearch (ES). HBase stores the full data, while ES indexes and queries the data, leveraging the strengths of each to achieve a two-way improvement in business system storage and query performance. For the above scenario, redundant data backups are necessary to ensure data security, and snapshots are the best option. This is because the backend operation logic of HBase and ES snapshots is completely inconsistent. For example, for snapshots of logical tables A and B, Table A is not partitioned, with only one physical table A; while Table B is partitioned by year, there are two physical tables, B_2020 and B_2021. ES can create these tables all at once, but HBase can only create them one by one, and they may fail at any time due to network or other issues. Therefore, how to better unify and manage them is a problem that must be solved.

[0003] The existing approach is for users to select a specific table and directly call the HBase and ES APIs to create a snapshot interface. However, since there is no connection between HBase and ES snapshots and no unified management, users must perform separate operations, involving the operation of two system components, which is cumbersome. For example, snapshot operations are time-consuming due to large data volumes. Snapshots are typically asynchronous operations. Therefore, if an index snapshot fails, users have no way of knowing the specific status and can only check HBase and ES for their respective snapshot status and error information, resulting in inefficient snapshot processing. Furthermore, there is no effective way for users to manage the corresponding relationships between these snapshots, making them difficult to maintain. This is particularly evident when there are many tables, making it more prone to errors. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a snapshot processing method, device, and electronic device to solve the problem of low snapshot processing efficiency.

[0005] According to a first aspect, an embodiment of the present invention provides a snapshot processing method, which is applied to a distributed database search and analysis engine. The method includes:

[0006] Obtaining a snapshot creation request, wherein the snapshot creation request includes a target snapshot name and a target logical table;

[0007] Determining the physical table corresponding to each target logical table based on the state of the target field in the target logical table, wherein the target field is used to indicate the database to which the physical table belongs;

[0008] Creating corresponding snapshots based on the physical table, and determining the creation status of each snapshot;

[0009] Snapshot record information corresponding to the target snapshot name is determined according to the snapshot creation request, the physical table, and the creation status, so as to process each snapshot using the snapshot record information.

[0010] The snapshot processing method provided by the embodiment of the present invention automatically queries the physical table corresponding to the target logical table by providing the target snapshot name and the target logical table when creating a snapshot, thereby creating a corresponding snapshot, wherein the physical table may belong to the first database or the second database, and the user does not need to be aware of in which database the physical snapshot is created; the snapshot record information is then used to record the relevant information of the snapshot, and subsequently only the snapshot record information needs to be searched for to store the snapshot information of each database, shielding the complexity of different databases from the outside. The user only needs to access the snapshot record information to know the snapshot creation status corresponding to each physical table, without entering the internal query of the two components, thereby improving the efficiency of snapshot processing to a certain extent.

[0011] In combination with the first aspect, in a first implementation of the first aspect, determining the physical table corresponding to each target logical table based on the state of the target field in the target logical table includes:

[0012] Determine whether a target field exists in the target logical table;

[0013] When the target field exists, it is determined that the target logical table is a composite table and the physical table includes a physical table of the first database and a physical table of the second database.

[0014] The snapshot processing method provided by the embodiment of the present invention has a target field that can indicate whether the target logical table is used to store attachment information, that is, whether a large amount of data needs to be stored. When a large amount of data needs to be stored, it is necessary to use the physical table corresponding to the second database to create a snapshot, thereby automatically determining the corresponding database. From the user's perspective, they only know that a table has been created, and they have no idea that the underlying physical tables are corresponding to two databases. This facilitates user operation and improves snapshot processing efficiency.

[0015] In combination with the first implementation of the first aspect, in the second implementation of the first aspect, determining the physical table corresponding to each target logical table based on the state of the target field in the target logical table further includes:

[0016] When the target field does not exist, it is determined that the target logical table does not have the composite table and the physical table includes a physical table of the first database.

[0017] The snapshot processing method provided by the embodiment of the present invention indicates that when the target field does not exist, the snapshot processing of a small amount of data is performed at this time, which is a case where the second database is not used. Only the first database is used to create the snapshot, thereby expanding the usage scenarios and avoiding unnecessary waste of resources.

[0018] In combination with the first aspect, in a third implementation of the first aspect, determining snapshot record information corresponding to the target snapshot name based on the snapshot creation request, the physical table, and the creation status, and processing each snapshot using the snapshot record information includes:

[0019] Periodically query the snapshot record information to determine whether there is snapshot information with an abnormal status;

[0020] When snapshot information with abnormal status exists, the creation status is queried in the corresponding database cluster based on the snapshot name corresponding to the snapshot information with abnormal status, so as to update the snapshot record information.

[0021] The snapshot processing method provided by the present invention uses scheduled tasks to compensate for abnormal snapshot states, ensuring that state is not lost. For example, if, after completing an HBase or ES snapshot, the system experiences a power outage while updating snapshot records, this method can be used to rewrite the snapshot state.

[0022] In combination with the third implementation manner of the first aspect, in the fourth implementation manner of the first aspect, the snapshot record information further includes an identification of whether each logical table exists as a composite table, and querying the creation status in the corresponding database cluster based on the snapshot name corresponding to the snapshot information with the abnormal status to update the snapshot record information includes:

[0023] Based on the snapshot record information, determining whether a composite table exists in the logical table corresponding to the snapshot information with the abnormal state;

[0024] When the composite table exists, determining a target database corresponding to the snapshot information with abnormal status;

[0025] Based on the snapshot name corresponding to the snapshot information with abnormal status and the target database, the creation status is queried in the target database cluster to update the snapshot record information.

[0026] The snapshot processing method provided by the embodiment of the present invention introduces information about whether a composite table exists into snapshot record information, thereby ensuring that the solution is still applicable when the first database is used alone, and diversifying usage scenarios.

[0027] In combination with the fourth implementation of the first aspect, in the fifth implementation of the first aspect, querying the creation status in the corresponding database cluster based on the snapshot name corresponding to the snapshot information with the abnormal status to update the snapshot record information further includes:

[0028] When the composite table does not exist, determining that the target database corresponding to the snapshot information with the abnormal state is the first database;

[0029] Based on the snapshot name corresponding to the snapshot information with abnormal status, the creation status is queried in the first database cluster to update the snapshot record information.

[0030] In the snapshot processing method provided by the embodiment of the present invention, since there is no composite table, the snapshot in the abnormal state cannot exist in the second database. Therefore, only the creation state needs to be queried in the first database, thereby improving query efficiency.

[0031] In combination with the fourth implementation manner of the first aspect, in the sixth implementation manner of the first aspect, when the target database is the second database, querying the creation status in the target database cluster based on the snapshot name corresponding to the snapshot information with the abnormal status and the target database to update the snapshot record information includes:

[0032] Determining, based on the snapshot information of the abnormal state, a name of a physical table corresponding to the snapshot of the abnormal state in the second database;

[0033] Concatenate the snapshot name corresponding to the snapshot information with abnormal status with the physical table name to determine the snapshot name of the snapshot information with abnormal status in the second database;

[0034] The creation status is queried in the second database cluster based on the snapshot name in the second database to update the snapshot record information.

[0035] The snapshot processing method provided in the embodiment of the present invention can obtain the corresponding snapshot names in the first database and the second database through the splicing rules as long as the corresponding snapshot name is obtained, thereby realizing operations and information acquisition for the two snapshots, facilitating management.

[0036] According to a second aspect, an embodiment of the present invention further provides a snapshot processing device, which is applied to a distributed database search and analysis engine, and the device includes:

[0037] An acquisition module, configured to acquire a snapshot creation request, wherein the snapshot creation request includes a target snapshot name and a target logical table;

[0038] A first determining module is configured to determine a physical table corresponding to each target logical table based on a state of a target field in the target logical table, wherein the target field is used to indicate a database to which the physical table belongs;

[0039] a second determining module, configured to create corresponding snapshots based on the physical table and determine a creation status of each snapshot;

[0040] The third determining module is configured to determine snapshot record information corresponding to the target snapshot name according to the snapshot creation request, the physical table, and the creation status, so as to process each snapshot using the snapshot record information.

[0041] In conjunction with the second aspect, in a first implementation of the second aspect, the first determining module includes:

[0042] A judging unit, configured to judge whether a target field exists in the target logical table;

[0043] The first determining unit is configured to, when the target field exists, determine that the target logical table includes a composite table and the physical table includes a physical table of the first database and a physical table of the second database.

[0044] In combination with the first implementation of the first aspect, in the second implementation of the first aspect, the first determining module further includes:

[0045] The second determining unit is configured to, when the target field does not exist, determine that the target logical table does not contain the composite table and the physical table includes a physical table of the first database.

[0046] In combination with the second aspect, in a third implementation of the second aspect, the third determining module includes:

[0047] a first query unit, configured to periodically query the snapshot record information to determine whether there is snapshot information with an abnormal status;

[0048] The second query unit is configured to query the creation status in the corresponding database cluster based on the snapshot name corresponding to the snapshot information with abnormal status when the snapshot information with abnormal status exists, so as to update the snapshot record information.

[0049] In combination with the third implementation manner of the second aspect, in the fourth implementation manner of the second aspect, the snapshot record information further includes an identification of whether each logical table has a composite table, and the second query unit includes:

[0050] A first determining subunit is configured to determine, based on the snapshot record information, whether a composite table exists in the logical table corresponding to the snapshot information with the abnormal state;

[0051] A second determining subunit is configured to determine, when the composite table exists, a target database corresponding to the snapshot information with abnormal status;

[0052] The first query sub-unit is configured to query a creation status in a target database cluster based on a snapshot name corresponding to the snapshot information with abnormal status and the target database, so as to update the snapshot record information.

[0053] In combination with the fourth implementation of the second aspect, in the fifth implementation of the second aspect, the second query unit further includes:

[0054] a third determining subunit, configured to, when the composite table does not exist, determine that the target database corresponding to the snapshot information with the abnormal state is the first database;

[0055] The second query sub-unit is configured to query the creation status in the first database cluster based on the snapshot name corresponding to the snapshot information with abnormal status, so as to update the snapshot record information.

[0056] In combination with the fourth implementation of the second aspect, in the sixth implementation of the second aspect, when the target database is the second database, the first query subunit includes:

[0057] an abnormal state determining subunit, configured to determine, based on the snapshot information of the abnormal state, a name of a physical table corresponding to the snapshot of the abnormal state in the second database;

[0058] a concatenation subunit, configured to concatenate the snapshot name corresponding to the snapshot information with the abnormal state with the physical table name, and determine the snapshot name of the snapshot information with the abnormal state in the second database;

[0059] An updating subunit is configured to query a creation status in the second database cluster based on the snapshot name in the second database, so as to update the snapshot record information.

[0060] According to a third aspect, an embodiment of the present invention provides an electronic device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the snapshot processing method described in the first aspect or any one of the embodiments of the first aspect by executing the computer instructions.

[0061] According to a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the snapshot processing method described in the first aspect or any one of the embodiments of the first aspect.

[0062] It should be noted that the corresponding beneficial effects of the snapshot processing device, electronic device, and computer-readable storage medium provided by the embodiments of the present invention can be found in the description of the corresponding contents in the snapshot processing method, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0064] Figure 1 is a flowchart of a snapshot processing method according to an embodiment of the present invention;

[0065] Figure 2 is a schematic diagram of snapshot creation according to an embodiment of the present invention;

[0066] Figure 3 is a flowchart of a snapshot processing method according to an embodiment of the present invention;

[0067] Figure 4 is a schematic diagram of processing abnormal status according to an embodiment of the present invention;

[0068] Figure 5 is a schematic diagram of a composite snapshot according to an embodiment of the present invention;

[0069] Figure 6 is a flowchart of a snapshot processing method according to an embodiment of the present invention;

[0070] Figure 7 is a structural block diagram of a snapshot processing device according to an embodiment of the present invention;

[0071] Figure 8 It is a schematic diagram of the hardware structure of the electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0072] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of the present invention.

[0073] In this embodiment of the present invention, a detailed description is provided using an ES database as the first database and an HBase database as the second database. This approach is of course applicable to other databases, and there are no restrictions on the database types. The only requirement is that the first database is used for data backup, while the second database is primarily used for attachment storage and data backup. Furthermore, the snapshot creation logic in the first and second databases differs.

[0074] The snapshot processing method provided by the embodiment of the present invention is applied to a distributed database search and analysis engine. It uses a target snapshot name and a target logical table. The electronic device uses the target field in the target logical table to automatically determine the physical table involved in the current snapshot creation, and then creates the snapshot. After the snapshot is created, the information corresponding to the snapshot is recorded through the snapshot record information, including but not limited to the snapshot name, the logical table involved in the snapshot, the physical table, and the creation status, etc. Among them, after each snapshot is created, corresponding snapshot record information is formed. These snapshot record information are recorded in the form of a data table to form a message table. When the corresponding snapshot is subsequently processed, the relevant information of each snapshot can be known by searching the message table.

[0075] The process of generating snapshot record information will be described in detail below.

[0076] Taking the example of an ES database as the first database and an HBase database as the second database, the snapshot processing method provided by the embodiments of the present invention ensures convenient and unified management of HBase+ES composite snapshots and individual ES snapshots through snapshot record information. When creating a snapshot, users only need to provide a snapshot name and a logical table. There is no need for users to connect to HBase and ES separately, nor do they need to record and maintain the corresponding relationship between their snapshots. This allows for convenient management of snapshot creation for HBase and ES clusters. Users can also directly query the specific creation status of HBase and ES snapshots based on a snapshot name, obtaining detailed information about each snapshot, including the physical tables included in the HBase and ES snapshots and the error messages in the event of an error.

[0077] According to an embodiment of the present invention, an embodiment of a snapshot processing method is provided. It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0078] In this embodiment, a snapshot processing method is provided, which can be used in electronic devices, such as distributed database search and analysis engines, specifically computers, tablet computers, servers, etc. Figure 1 is a flowchart of a snapshot processing method according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0079] S11, obtaining a snapshot creation request.

[0080] The snapshot creation request includes a target snapshot name and a target logical table.

[0081] When creating a snapshot, the user sends a snapshot creation request to the electronic device, that is, the user can enter the target snapshot name and target logic table on the human-computer interaction interface provided by the electronic device. Accordingly, the electronic device can obtain the snapshot creation request.

[0082] The target logical tables included in the snapshot creation request may be one, two, or more, and there is no limitation on the specific number.

[0083] S12: Determine the physical table corresponding to each target logical table based on the status of the target field in the target logical table.

[0084] The target field is used to indicate the database to which the physical table belongs. When the physical table corresponding to the target logical table belongs to at least two databases, a target field will exist in the target logical table, and the electronic device can use this target field to determine the physical table corresponding to the target logical table. For example, the target logical table obtained by the electronic device includes Table A and Table B, where Table A is not partitioned and has only one physical table A; Table B is a yearly partitioned table and has two physical tables, B_2020 and B_2021.

[0085] Specifically, if a target logical table contains an attachment field, which is used to store attachment information, an HBase snapshot must be created for the table. This is because when a user first creates a table, as long as the attachment field exists, the electronic device automatically creates an HBase table for it to store attachments (commonly used images and Word documents, etc.). From the user's perspective, they only know that a single table has been created, unaware that two underlying tables exist: one in ES and one in HBase.

[0086] After acquiring the target logic table, the electronic device analyzes the target fields in each target logic table in turn, for example, to determine the processing data shown in Table 1.

[0087] Table 1 Processing data

[0088]

[0089] As shown in Table 1, the corresponding physical tables in the first database ES include: A, B_2020, and B_2021; the physical tables in the second database HBase include: A, B_2020, and B_2021.

[0090] S13: Create corresponding snapshots based on the physical table and determine the creation status of each snapshot.

[0091] After the electronic device determines the physical table, it can perform snapshot processing on the corresponding database. Figure 2 As shown, if the target field exists in the submitted target logical table, it means that HBase operation and ES operation are required. Then, the electronic device submits HBase snapshot creation and ES snapshot creation requests respectively, and then creates snapshots in the corresponding databases respectively, and records the creation status.

[0092] Specifically, if the asynchronous creation of the HBase snapshot succeeds, the HBase creation status is recorded as true; otherwise, the HBase creation status is recorded as false. If the asynchronous creation of the ES snapshot succeeds, the ES creation status is recorded as true; otherwise, the ES creation status is recorded as false.

[0093] S14: Determine snapshot record information corresponding to the target snapshot name according to the snapshot creation request, the physical table, and the creation status, so as to process each snapshot using the snapshot record information.

[0094] When the electronic device determines the creation status of each snapshot, it records the snapshot information corresponding to each target snapshot name through a snapshot record information. For example, the target logical table, physical table, and the creation status of the snapshot, etc. The snapshot record information can be recorded in the form of a table or other methods. There is no limitation on the recording method. Taking the table record as an example, as shown in Table 2, Table 2 Snapshot Record Information

[0095]

[0096] As shown in Table 2, the snapshot record information records the snapshot information corresponding to the snapshot name, and the HBase snapshot information and the ES snapshot information are combined in an information table, thereby realizing the unified management of HBase and ES. Specifically, after the electronic device determines the corresponding physical table in S12, it can create snapshot record information corresponding to the target snapshot name in the table, and then submit the snapshot request to the ES cluster and the HBase cluster respectively. If the submission is successful, the submission success will be returned. Subsequently, after the HBase snapshot and ES snapshot creation processing are completed, the status or error information is written to the information table through the callback. If all are written successfully, the corresponding snapshot record information is shown in Table 2. Among them, regarding the start time and the request start time in Table 2, the end time is based on the last end time of HBase and ES. Regardless of who completes the execution, the end time is updated in the final callback.

[0097] The snapshot processing method provided in this embodiment automatically queries the physical table corresponding to the target logical table by providing the target snapshot name and the target logical table when creating a snapshot, thereby creating a corresponding snapshot. The physical table may belong to the first database or the second database, and the user does not need to know in which database the physical snapshot is created. The snapshot record information is then used to record the relevant information of the snapshot. Subsequently, only the snapshot record information needs to be searched for to store the snapshot information of each database, thereby shielding the complexity of different databases from the outside. The user only needs to access the snapshot record information to know the snapshot creation status corresponding to each physical table, without having to enter the internal query of the two components, thereby improving the efficiency of snapshot processing to a certain extent.

[0098] In this embodiment, a snapshot processing method is provided, which can be used in electronic devices, such as distributed database search and analysis engines, specifically computers, tablet computers, servers, etc. Figure 3 is a flowchart of a snapshot processing method according to an embodiment of the present invention. Figure 3 As shown, the process includes the following steps:

[0099] S21: Get a snapshot creation request.

[0100] The snapshot creation request includes a target snapshot name and a target logical table.

[0101] For details, please see Figure 1 S11 of the illustrated embodiment will not be described in detail here.

[0102] S22: Determine the physical table corresponding to each target logical table based on the status of the target field in the target logical table.

[0103] The target field is used to indicate the database to which the physical table belongs.

[0104] For details, please see Figure 1 S12 of the illustrated embodiment will not be described in detail here.

[0105] S23: Create corresponding snapshots based on the physical table and determine the creation status of each snapshot.

[0106] For details, please see Figure 1 S13 of the illustrated embodiment will not be described in detail here.

[0107] S24 , determining snapshot record information corresponding to the target snapshot name according to the snapshot creation request, the physical table, and the creation status, so as to process each snapshot using the snapshot record information.

[0108] In this embodiment, a compensation mechanism is mainly used to ensure that in abnormal situations, for example, if the user does not write the HBase and ES snapshot creation status to the message table after creating the HBase and ES snapshot, the HBase and ES creation status and error information can still be obtained and updated to the information table in time, making it easier for users to obtain status and error information.

[0109] For example, if the system suddenly loses power after user creation, the status cannot be successfully written to the message table. Based on this, an exception compensation mechanism is provided. A scheduled scheduling scheme is designed to query snapshot records at regular intervals. If snapshot information with abnormal status is found, the snapshot name is obtained and the creation status is queried in HBase and ES clusters in turn. The creation information is updated in a timely manner to ensure that the creation status is displayed even in abnormal situations, completely eliminating the problem of status loss.

[0110] Specifically, the above S24 includes:

[0111] S241 , periodically querying snapshot record information to determine whether there is snapshot information with an abnormal status.

[0112] The query period for snapshot record information can be every 5 minutes or every 10 minutes. There is no specific limit here and this period can be set according to actual needs. The electronic device queries each record of the snapshot record information to check whether there is snapshot information with an abnormal status. An abnormal status refers to a creation status in the snapshot record information being empty or having another value. In other words, whether a snapshot information has an abnormal status is determined by the specific value of the creation status.

[0113] S242: When snapshot information with abnormal status exists, query the creation status in the corresponding database cluster based on the snapshot name corresponding to the snapshot information with abnormal status to update the snapshot record information.

[0114] When an electronic device determines that snapshot information with an abnormal state exists, it can obtain information such as the snapshot name and the physical table corresponding to the snapshot from the snapshot record information. Using this information, the electronic device can determine the database where the abnormal snapshot resides, query the snapshot's creation status from the corresponding database cluster, and update the corresponding snapshot record information with the retrieved creation status.

[0115] like Figure 4 As shown, the above S24 may include:

[0116] S1, scheduled task, triggered every five minutes;

[0117] S2, obtains the list of HBase and ES status empty information in the information table in real time;

[0118] S3, query the snapshot creation status in HBase based on the list;

[0119] S4, query the snapshot creation status on ES based on the list.

[0120] Specifically, when the electronic device determines that there is a snapshot with an abnormal state, it obtains in real time the information list of HBase and ES states that are empty in the information table, and then queries the creation status from the corresponding database respectively.

[0121] The snapshot processing method provided in this embodiment uses a scheduled task to trigger snapshot information with abnormal states, ensuring that the state is not lost. For example, after completing an HBase or ES snapshot, if the system suddenly loses power while updating the snapshot record information, this method can be used to complete the snapshot state writing.

[0122] In some optional implementations of this embodiment, the snapshot record information also includes an identifier of whether each logical table is a composite table. The composite table, that is, for the same table A, is stored both in HBase and in ES, as shown in Table 2. When forming the snapshot record information, if the target field exists in the target logical table, it means that the target logical table has a composite table. Accordingly, this information can be stored in the snapshot record information. Based on this, the above S242 may include:

[0123] (1) Based on the snapshot record information, determine whether there is a composite table in the logical table corresponding to the snapshot information with abnormal status.

[0124] When the composite table exists, execute step (2); otherwise, execute step (4).

[0125] Specifically, the electronic device first uses snapshot record information to determine whether the snapshot corresponding to the abnormal snapshot information is a composite table. If a composite table is present, the composite table status is true. If a composite table exists, the abnormal snapshot may correspond to both the first and second databases, that is, to Elasticsearch and HBase. If no composite table exists, the abnormal snapshot may only correspond to the first database, Elasticsearch.

[0126] (2) Determine the target database corresponding to the snapshot information with abnormal status.

[0127] When the electronic device determines the presence of a composite table, it can again use the snapshot record information to determine the target database corresponding to the abnormal snapshot information. If the creation status corresponding to ES is empty, the target database includes ES; if the creation status corresponding to HBase is empty, the target database includes HBase; if the creation status corresponding to both is empty, the target database includes both ES and HBase.

[0128] When the target database is the first database, step (4) is executed. When the target database is the second database, the above step (2) may include:

[0129] 2.1) Based on the snapshot information of the abnormal state, determine the physical table name corresponding to the snapshot of the abnormal state in the second database.

[0130] 2.2) Concatenate the snapshot name corresponding to the snapshot information with abnormal status with the physical table name to determine the snapshot name of the snapshot information with abnormal status in the second database.

[0131] 2.3) Based on the snapshot name in the second database, query the creation status in the second database cluster to update the snapshot record information.

[0132] Corresponding to the snapshot record information shown in Table 2, there is only one snapshot name represented, that is, for the user, only one snapshot name needs to be provided to perform snapshot processing in the two databases. Specifically, a mapping relationship between the two databases can be set in the electronic device. For example, the snapshot name in the first database is directly the snapshot name entered by the user, and the snapshot name in the second database is formed by combining the snapshot name entered by the user with the mapping relationship. However, the user is not aware of this process, and it is automatically processed by the electronic device.

[0133] For example, electronic devices establish a mapping relationship between HBase and ES snapshots to ensure that one snapshot name can obtain all snapshot information of HBase and ES, so that the system can shield the background complexity and display one snapshot information externally. The mapping rules are as follows:

[0134] User input snapshot name = ES snapshot name;

[0135] ES snapshot name + physical table name = HBase snapshot name.

[0136] Specifically, if Figure 5 As shown in the figure, the corresponding mapping rule process is as follows: the user specifies a snapshot name, snapshot_a. This snapshot contains two logical tables, A and B. Table A is unpartitioned and named A; table B is partitioned and named B_2020 and B_2021. The ES cluster directly uses this snapshot name. Due to the limitations of the HBase cluster, a snapshot can only contain one table, so the corresponding HBase snapshot names are snapshot_a_A, snapshot_a_B_2020, and snapshot_a_B_2021. Externally, the user only needs to specify an ES snapshot name; the rest is similar to the HBase snapshot name. The electronic device automatically completes the mapping. When you need to operate the snapshot later, you can directly obtain all related HBase snapshots based on the ES snapshot name and the HBase physical table.

[0137] (3) Based on the snapshot name corresponding to the snapshot information with abnormal status and the target database, query the creation status in the target database cluster to update the snapshot record information.

[0138] After obtaining the snapshot name corresponding to the snapshot information with abnormal status, the electronic device uses the mapping rule to determine the snapshot name corresponding to each database, and then uses the snapshot name to query the creation status in the corresponding target database cluster to update the snapshot record information.

[0139] Mapping rules are designed within the electronic device. Once a snapshot with an abnormal status is obtained, the ES snapshot name and the HBase snapshot name can be obtained through the concatenation rule. This allows operations and information retrieval for both snapshots, facilitating management. For example, deleting HBase and ES snapshots and obtaining their snapshot statuses requires only specifying a single snapshot name, making management easier.

[0140] (4) Determine that the target database corresponding to the snapshot information with abnormal status is the first database.

[0141] (5) Based on the snapshot name corresponding to the snapshot information with abnormal status, query the creation status in the first database cluster to update the snapshot record information.

[0142] When it is determined that the target database is the first database, the electronic device directly uses the snapshot name of the snapshot information with abnormal status to query the corresponding creation status in the first database cluster, and writes the queried creation status into the snapshot record information to update the snapshot record information.

[0143] By including information about the existence of composite tables in the snapshot record, this solution remains applicable even when the first database is used alone, facilitating diverse usage scenarios. If composite tables do not exist, the snapshot with the abnormal state cannot exist in the second database. Therefore, only the creation state needs to be queried in the first database, improving query efficiency.

[0144] In this embodiment, a snapshot processing method is provided, which can be used in electronic devices, such as distributed database search and analysis engines, specifically computers, tablet computers, servers, etc. Figure 6 is a flowchart of a snapshot processing method according to an embodiment of the present invention. Figure 6 As shown, the process includes the following steps:

[0145] S31: Get a snapshot creation request.

[0146] The snapshot creation request includes a target snapshot name and a target logical table.

[0147] For details, please see Figure 1 S11 of the illustrated embodiment will not be described in detail here.

[0148] S32: Determine the physical table corresponding to each target logical table based on the state of the target field in the target logical table, wherein the target field is used to indicate the database to which the physical table belongs.

[0149] Specifically, the above S32 includes:

[0150] S321, determine whether the target field exists in the target logical table.

[0151] As described above, when the target logical table has a composite table, it will have a corresponding target field, and the database to which the physical table belongs can be determined through the target field. If the target field exists, execute S322; otherwise, execute S323.

[0152] S322 , determining that the target logical table includes a composite table and the physical table includes a physical table of the first database and a physical table of the second database.

[0153] S323: Determine that the target logical table does not have a composite table and the physical table includes a physical table of the first database.

[0154] For example, a user selects target logical table A and creates snapshot A. The request is sent to the electronic device, which first determines whether table A has an attachment field. If table A is partitioned by year and has an attachment field, and two physical tables exist, A_2020 and A_2021, then HBase must also have two corresponding physical tables. In this case, operations on ES and HBase are required.

[0155] If table A does not have an attachment field, there is no need to perform operations in HBase; only ES operations are required. The core function of HBase is to store attachments and to provide data redundancy.

[0156] S33: Create corresponding snapshots based on the physical table and determine the creation status of each snapshot.

[0157] For details, please see Figure 1 S13 of the illustrated embodiment will not be described in detail here.

[0158] S34 , determining snapshot record information corresponding to the target snapshot name according to the snapshot creation request, the physical table, and the creation status, so as to process each snapshot using the snapshot record information.

[0159] For details, please see Figure 3 S24 of the illustrated embodiment will not be described in detail here.

[0160] The snapshot processing method provided in this embodiment uses a target field to indicate whether the target logical table is used to store attachment information, that is, whether a large amount of data needs to be stored. When large amounts of data are needed, a snapshot is created using the physical table corresponding to the second database, thereby automatically determining the corresponding database. From the user's perspective, they only know that a table has been created, unaware that the underlying physical tables correspond to two databases. This facilitates user operation and improves snapshot processing efficiency. The absence of a target field indicates that a small amount of data is being snapshot processed, meaning that the second database is not used, and only the first database is used for snapshot creation. This expands usage scenarios and avoids unnecessary resource waste.

[0161] This embodiment also provides a snapshot processing device for implementing the above-mentioned embodiments and preferred implementations. Details already described will not be repeated here. As used below, the term "module" may refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0162] This embodiment provides a snapshot processing device, which is applied to a distributed database search and analysis engine, such as Figure 7 As shown, the device includes:

[0163] An acquisition module 41 is configured to acquire a snapshot creation request, wherein the snapshot creation request includes a target snapshot name and a target logical table;

[0164] A first determining module 42 is configured to determine the physical table corresponding to each target logical table based on the state of a target field in the target logical table, wherein the target field is used to indicate the database to which the physical table belongs;

[0165] A second determining module 43 is configured to create corresponding snapshots based on the physical table and determine a creation status of each snapshot;

[0166] The third determining module 44 is configured to determine snapshot record information corresponding to the target snapshot name according to the snapshot creation request, the physical table, and the creation status, so as to process each snapshot using the snapshot record information.

[0167] Optionally, the first determining module includes:

[0168] A judging unit, configured to judge whether a target field exists in the target logical table;

[0169] The first determining unit is configured to, when the target field exists, determine that the target logical table includes a composite table and the physical table includes a physical table of the first database and a physical table of the second database.

[0170] In combination with the first implementation of the first aspect, in the second implementation of the first aspect, the first determining module further includes:

[0171] The second determining unit is configured to, when the target field does not exist, determine that the target logical table does not contain the composite table and the physical table includes a physical table of the first database.

[0172] Optionally, the third determining module includes:

[0173] a first query unit, configured to periodically query the snapshot record information to determine whether there is snapshot information with an abnormal status;

[0174] The second query unit is configured to query the creation status in the corresponding database cluster based on the snapshot name corresponding to the snapshot information with abnormal status when the snapshot information with abnormal status exists, so as to update the snapshot record information.

[0175] Optionally, the snapshot record information further includes an identifier of whether a composite table exists in each logical table, and the second query unit includes:

[0176] A first determining subunit is configured to determine, based on the snapshot record information, whether a composite table exists in the logical table corresponding to the snapshot information with the abnormal state;

[0177] A second determining subunit is configured to determine, when the composite table exists, a target database corresponding to the snapshot information with abnormal status;

[0178] The first query sub-unit is configured to query a creation status in a target database cluster based on a snapshot name corresponding to the snapshot information with abnormal status and the target database, so as to update the snapshot record information.

[0179] Optionally, the second query unit further includes:

[0180] a third determining subunit, configured to, when the composite table does not exist, determine that the target database corresponding to the snapshot information with the abnormal state is the first database;

[0181] The second query sub-unit is configured to query the creation status in the first database cluster based on the snapshot name corresponding to the snapshot information with abnormal status, so as to update the snapshot record information.

[0182] Optionally, when the target database is a second database, the first query subunit includes:

[0183] an abnormal state determining subunit, configured to determine, based on the snapshot information of the abnormal state, a name of a physical table corresponding to the snapshot of the abnormal state in the second database;

[0184] a concatenation subunit, configured to concatenate the snapshot name corresponding to the snapshot information with the abnormal state with the physical table name, and determine the snapshot name of the snapshot information with the abnormal state in the second database;

[0185] An updating subunit is configured to query a creation status in the second database cluster based on the snapshot name in the second database, so as to update the snapshot record information.

[0186] The snapshot processing device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0187] The further functional description of each of the above modules is the same as that of the above corresponding embodiments and will not be repeated here.

[0188] An embodiment of the present invention further provides an electronic device having the above Figure 7 The snapshot processing device shown.

[0189] See also Figure 8, Figure 8 is a structural diagram of an electronic device provided by an optional embodiment of the present invention, such as Figure 8 As shown, the electronic device may include: at least one processor 51, such as a CPU (Central Processing Unit), at least one communication interface 53, a memory 54, and at least one communication bus 52. The communication bus 52 is used to realize the connection and communication between these components. The communication interface 53 may include a display screen (Display), a keyboard (Keyboard), and the optional communication interface 53 may also include a standard wired interface and a wireless interface. The memory 54 may be a high-speed RAM memory (Random Access Memory, volatile random access memory) or a non-volatile memory (non-volatile memory), such as at least one disk memory. The memory 54 may optionally be at least one storage device located away from the aforementioned processor 51. The processor 51 may be combined with Figure 7 In the described apparatus, the memory 54 stores an application program, and the processor 51 calls the program code stored in the memory 54 to execute any of the above method steps.

[0190] The communication bus 52 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The communication bus 52 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0191] Among them, the memory 54 may include volatile memory (English: volatile memory), such as random-access memory (English: random-access memory, abbreviated: RAM); the memory may also include non-volatile memory (English: non-volatile memory), such as flash memory (English: flash memory), hard disk drive (English: hard disk drive, abbreviated: HDD) or solid-state drive (English: solid-state drive, abbreviated: SSD); the memory 54 may also include a combination of the above types of memory.

[0192] The processor 51 may be a central processing unit (CPU), a network processor (NP), or a combination of a CPU and a NP.

[0193] The processor 51 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0194] Optionally, the memory 54 is further configured to store program instructions. The processor 51 may call the program instructions to implement the snapshot processing method as shown in any embodiment of the present application.

[0195] An embodiment of the present invention further provides a non-transitory computer storage medium storing computer-executable instructions that can execute the snapshot processing method of any of the above method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD), or a solid-state drive (SSD); the storage medium can also include a combination of the above types of memory.

[0196] Although the embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention. Such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A snapshot processing method, characterized in that: Applied to a distributed database search and analysis engine, the method includes: Obtaining a snapshot creation request, wherein the snapshot creation request includes a target snapshot name and a target logical table; Determining the physical table corresponding to each target logical table based on the state of the target field in the target logical table, wherein the target field is used to indicate the database to which the physical table belongs; Creating corresponding snapshots based on the physical table, and determining the creation status of each snapshot; determining snapshot record information corresponding to the target snapshot name according to the snapshot creation request, the physical table, and the creation status, so as to process each snapshot using the snapshot record information; The step of determining the physical table corresponding to each target logical table based on the state of the target field in the target logical table includes: Determine whether a target field exists in the target logical table; When the target field exists, determining that the target logical table is a composite table and the physical table includes a physical table of the first database and a physical table of the second database; When the target field does not exist, it is determined that the target logical table does not have the composite table and the physical table includes a physical table of the first database.

2. The method according to claim 1, characterized in that The determining, according to the snapshot creation request, the physical table, and the creation status, snapshot record information corresponding to the target snapshot name, and processing each snapshot using the snapshot record information, includes: Periodically query the snapshot record information to determine whether there is snapshot information with an abnormal status; When snapshot information with abnormal status exists, the creation status is queried in the corresponding database cluster based on the snapshot name corresponding to the snapshot information with abnormal status, so as to update the snapshot record information.

3. The method according to claim 2, characterized in that The snapshot record information also includes an identifier of whether a composite table exists in each logical table. The snapshot name corresponding to the snapshot information with abnormal status is queried in the corresponding database cluster for the creation status to update the snapshot record information, including: Based on the snapshot record information, determining whether a composite table exists in the logical table corresponding to the snapshot information with the abnormal state; When the composite table exists, determining a target database corresponding to the snapshot information with abnormal status; Based on the snapshot name corresponding to the snapshot information with abnormal status and the target database, the creation status is queried in the target database cluster to update the snapshot record information.

4. The method according to claim 3, characterized in that The method further includes querying the creation status in the corresponding database cluster based on the snapshot name corresponding to the snapshot information of the abnormal state to update the snapshot record information: When the composite table does not exist, determining that the target database corresponding to the snapshot information with the abnormal state is the first database; Based on the snapshot name corresponding to the snapshot information with abnormal status, the creation status is queried in the first database cluster to update the snapshot record information.

5. The method according to claim 3, characterized in that When the target database is the second database, querying the creation status in the target database cluster based on the snapshot name corresponding to the snapshot information with abnormal status and the target database to update the snapshot record information includes: Determining, based on the snapshot information of the abnormal state, a name of a physical table corresponding to the snapshot of the abnormal state in the second database; Concatenate the snapshot name corresponding to the snapshot information with abnormal status with the physical table name to determine the snapshot name of the snapshot information with abnormal status in the second database; The creation status is queried in the second database cluster based on the snapshot name in the second database to update the snapshot record information.

6. A snapshot processing device, characterized in that: Applied to a distributed database search and analysis engine, the device comprises: An acquisition module, configured to acquire a snapshot creation request, wherein the snapshot creation request includes a target snapshot name and a target logical table; A first determining module is configured to determine a physical table corresponding to each target logical table based on a state of a target field in the target logical table, wherein the target field is used to indicate a database to which the physical table belongs; a second determining module, configured to create corresponding snapshots based on the physical table and determine a creation status of each snapshot; a third determining module, configured to determine snapshot record information corresponding to the target snapshot name according to the snapshot creation request, the physical table, and the creation status, so as to process each snapshot using the snapshot record information; The step of determining the physical table corresponding to each target logical table based on the state of the target field in the target logical table includes: Determine whether a target field exists in the target logical table; When the target field exists, determining that the target logical table is a composite table and the physical table includes a physical table of the first database and a physical table of the second database; When the target field does not exist, it is determined that the target logical table does not have the composite table and the physical table includes a physical table of the first database.

7. An electronic device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the snapshot processing method according to any one of claims 1 to 5 by executing the computer instructions.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the snapshot processing method according to any one of claims 1 to 5.

Images