Method and device for processing cold data

Abstract

Provided in the present application are a method and device for processing cold data. The method comprises: a database management apparatus receiving a mounting operation triggered by a user on a mounting configuration interface of a cloud computing platform, wherein the mounting configuration interface is used by the user to select a database instance that is about to access a cold table in a cloud storage, and the mounting operation comprises an identifier of a target database instance selected by the user; the database management apparatus sending a mounting instruction to the target database instance according to the mounting operation, wherein the mounting instruction carries an identifier of the cold table; the target database instance acquiring metadata information of the cold table from the cloud storage according to the mounting instruction, and storing the metadata information of the cold table; and then using the metadata information of the cold table to access cold data in the cold table. Thus, cross-instance access to cold data is realized, and the flexibility of access to the cold data can be improved.

Description

Cold data processing method and device

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of the People's Republic of China on July 19, 2023, with application number 202310889687.8, and invention name “A cold table data processing method, device and computing equipment cluster”, the entire contents of which are incorporated by reference into this application; this application claims priority to the Chinese patent application filed with the State Intellectual Property Office of the People's Republic of China on January 29, 2024, with application number 202410121668.5, and invention name “A cold data processing method and device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present application relates to the field of database technology, and in particular to a cold data processing method and device. Background Art

[0004] During database usage, large amounts of data are often written and updated. At the same time, databases also store infrequently accessed data, known as cold data. When cold data is stored in a database for an extended period, it can lead to a significant waste of storage resources and excessive costs.

[0005] To solve the problem of storage resource waste, the current common method is to migrate cold data from the database to third-party storage. However, the migrated cold data is bound to the database instance and can only be accessed by the original database instance, which has poor flexibility.

[0006] Summary of the Invention

[0007] The present application provides a cold data processing method and device to improve the flexibility of cold data access.

[0008] In a first aspect, the present application provides a cold data processing method, which is applied to a system provided by a cloud computing platform and includes a database system, a database management device, and cloud storage, wherein the database system includes a first database instance and a second database instance, the database management device is used to manage the database system, the first database instance and the second database instance are different instances belonging to the same user under the cloud computing platform, the first database instance is used to store a cold table of the database in the cloud storage, and the cold table is used to store cold data of the database. Specifically, the method includes: the database management device receives a mount operation triggered by a user on a mount configuration interface of the cloud computing platform, the mount configuration interface allows the user to select a database instance of the cold table in the cloud storage to be accessed, the mount operation includes an identifier of the second database instance selected by the user; then, the database management device sends a mount instruction to the second database instance according to the mount operation, the mount instruction carrying the identifier of the cold table; then, the second database instance obtains metadata information of the cold table from the cloud storage according to the mount instruction and stores the metadata information of the cold table; finally, the second database instance uses the metadata information of the cold table to access the cold data of the cold table.

[0009] Through the above technical solution, the cold data of a cold table of a database can be mounted (or, can also be called: associated) with another database, thereby achieving cross-instance access to the cold data of the cold table, which can improve the flexibility of cold data access.

[0010] In a possible implementation, the first database instance and the second database instance are deployed on the same computing node, or the first database instance and the second database instance are deployed on different computing nodes.

[0011] In a possible implementation, the method further includes: the second database instance sending a first update instruction to the cloud storage, where the first update instruction is used to instruct the cloud storage to add a mounting relationship between the cold table of the database and the second database instance.

[0012] Through the above technical solution, after the cold table of the database is mounted with other databases, the cloud storage can record and store the mounting relationship between the cold table and the database instance, so as to manage the mounting relationship between the database instance and the cold table.

[0013] In a possible implementation, the method further includes: a database management device receiving an uninstallation operation triggered by a user on an uninstallation configuration interface of the cloud computing platform, wherein the uninstallation configuration interface allows the user to select a database instance to be uninstalled, and the uninstallation operation includes an identifier of the second database instance selected by the user; then, the database management device sends an uninstallation instruction to the second database instance according to the uninstallation operation, wherein the uninstallation instruction carries the identifier of the cold table; finally, the second database instance deletes the metadata information of the cold table from the local storage according to the uninstallation instruction.

[0014] Through the above technical solution, the metadata information of the cold table stored locally in the second database instance can be deleted to achieve unloading of the cold table and the mounting of the second database instance. Subsequently, the second database instance will not be able to access the cold data of the cold table.

[0015] In one possible implementation, the method further includes: the second database instance deleting cache information of the cold table from a local cache, so that the local cache of the second database instance has no query records of the cold table and subsequently cannot access cold data in the cold table.

[0016] In a possible implementation, the method further includes: the second database instance sending a second update instruction to the cloud storage, where the second update instruction is used to instruct the cloud storage to delete the mounting relationship between the cold table of the database and the second database instance.

[0017] Through the above technical solution, after the cold table of the downloaded database is mounted with other databases, the cloud storage can record and store the unloading relationship between the cold table and the database instance, so as to manage the association relationship between the database instance and the cold table.

[0018] In a possible implementation, the method further includes: a database management device receiving a deletion operation triggered by a user on a deletion configuration interface of the cloud computing platform, the deletion configuration interface allowing the user to select cold data of a cold table to be deleted, the deletion operation including an identifier of the cold table selected by the user; the second database instance sending a deletion instruction to the cloud storage, the deletion instruction carrying the identifier of the cold table; and the cloud storage deleting the cold data of the cold table according to the deletion instruction.

[0019] Through the above technical solution, cold data in cold tables can be deleted according to user-triggered operation instructions to improve memory utilization.

[0020] In a second aspect, the present application provides a cold meter data processing device, which includes a module for implementing the cold meter data processing method provided by the above-mentioned first aspect or any possible implementation manner of the first aspect.

[0021] In a third aspect, the present application provides a computing device cluster, comprising at least one computing device, each computing device comprising a processor and a memory; the processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster performs the cold table data processing method provided in the first aspect or any possible implementation of the first aspect.

[0022] In a fourth aspect, the present application provides a computer program product comprising instructions, which, when executed by a computing device cluster, enables the computing device cluster to execute the cold table data processing method provided in the first aspect or any possible implementation of the first aspect.

[0023] In a fifth aspect, the present application provides a computer-readable storage medium comprising computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster executes the cold table data processing method provided in the first aspect or any possible implementation of the first aspect.

[0024] For each of the above-mentioned aspects from the second to the fifth aspects and the technical effects that may be achieved by each of the aspects, please refer to the above-mentioned description of the technical effects that can be achieved by various possible solutions in the first aspect, and no further details will be given here. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] FIG1 is a schematic diagram of a system architecture provided by an embodiment of the present application;

[0026] FIG2 is a flow chart of a method for accessing cold data provided by an embodiment of the present application;

[0027] FIG3 is a schematic diagram of a configuration interface of a cloud computing platform provided in an embodiment of the present application;

[0028] FIG4 is a schematic diagram of a method for accessing cold data provided by an embodiment of the present application;

[0029] FIG5 is a flow chart of a method for stopping access to cold data provided by an embodiment of the present application;

[0030] FIG6 is a schematic diagram of a method for stopping access to cold data provided by an embodiment of the present application;

[0031] FIG7 is a schematic diagram of a reference relationship in external storage provided by an embodiment of the present application;

[0032] FIG8 is a schematic diagram of a cold data processing device provided in an embodiment of the present application;

[0033] FIG9 is a schematic diagram of the structure of a computing device provided in an embodiment of the present application;

[0034] FIG10 is a schematic diagram of the structure of a computing device cluster provided in an embodiment of the present application;

[0035] FIG11 is a schematic diagram of the structure of another computing device cluster provided in an embodiment of the present application. DETAILED DESCRIPTION

[0036] In the description of the embodiments of this application, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to." The term "based on" should be understood as "based at least in part on." The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment." The terms "first," "second," etc. can refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0037] In this application, "at least one" means one or more, and "more" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: including the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0038] Below, some terms used in the embodiments of the present application are first explained to facilitate understanding by those skilled in the art.

[0039] 1. Cold Data

[0040] Cold data refers to data that has been accessed or used less than a set threshold but needs to be preserved for a long time. For example, cold data in a banking system could be historical transaction records from five years ago.

[0041] 2. Cold meter

[0042] A database may include multiple data tables. When the data in these data tables are cold data, these data tables may be referred to as cold tables.

[0043] 3. Database instance

[0044] In a distributed database, a large number of database instances are typically created. For example, a database instance is created for each tenant. A database instance is considered a database server, where one or more databases can be created. Each database can have one or more data tables, which can store data.

[0045] FIG1 is a schematic diagram of a system architecture provided in an embodiment of the present application. Referring to FIG1 , the system architecture may include a database system 10, a database management device 20, and cloud storage 30. The database system may include multiple database instances. For example, two database instances may include a first database instance 100 and a second database instance 200. The database management device 20 is used to manage the database system 10. The first database instance 100 and the second database instance 200 may be different instances belonging to the same user on a cloud computing platform. Cloud storage 30 is used to store cold database data.

[0046] In some embodiments, the first database instance 100 may include a processor 110 and a first memory 120. The first memory 120 may include a volatile memory, such as a random access memory (RAM). The first memory 120 may also include a non-volatile memory (NVM), such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid-state drive (SSD). The first memory 120 may store metadata information of cold tables of the first database.

[0047] The structure of the second database instance 200 is similar to that of the first database instance 100. The second database instance 200 includes a processor 210 and a second memory 220. The description of the processor 210 and the second memory 220 is similar to that of the processor 110 and the first memory 120. Please refer to the above content and will not be repeated here.

[0048] Optionally, the first database instance 100 and the second database instance 200 may further include a memory, a network card, etc. For example, the memory may be located in the first memory 120 or the second memory 220, which is not limited in this application.

[0049] Cloud storage (also known as external storage or third-party storage) 30 can store the cold table of the first database and / or the cold table of the second database. The cold table is used to store cold data in the database. It should be understood that in the embodiments of the present application, cloud storage 30 can also be a physical storage device, which is not limited in this application.

[0050] Taking the first database instance 100 as an example, in an embodiment of the present application, the cold table of the first database can be stored in the cloud storage 30. The metadata information of the cold table can be stored in the first memory 120 of the first database instance 100. When the second database instance 200 needs to access the cold table of the first database, it can obtain the metadata information of the cold table from the cloud storage 30. Then, the second database instance 200 can access the cold data of the cold table based on the metadata information of the cold table.

[0051] When the second database instance 200 does not need to access the cold table of the first database, the metadata information of the cold table can be deleted in the local storage, so that the second database instance 200 cannot access the cold data of the cold table based on the metadata information of the cold table.

[0052] The following describes a method for accessing cold data provided by an embodiment of the present application in conjunction with a flowchart. Referring to FIG2 , the method may include the following steps:

[0053] It should be understood that the embodiment shown in FIG2 is described using the example of database instance 200 needing to access cold data in a database included in database instance 100. For the specific implementation of database instance 100 needing to access cold data in a database included in database instance 200 or in other database instances, please refer to the description of the embodiment shown in FIG2 . Database instance 200 refers to the second database instance 200 in the system architecture shown in FIG1 , and database instance 100 refers to the first database instance 100 in the system architecture shown in FIG1 .

[0054] Step 201: The database management device 20 receives a mount operation triggered by a user on a mount configuration interface of a cloud computing platform.

[0055] The mount configuration interface allows the user to select the database instance of the cold table in cloud storage 30 to be accessed. The mount operation includes the identifier of the second database instance selected by the user. For example, FIG3 shows a configuration interface diagram of a cloud computing platform provided in an embodiment of the present application. The operation bar 301 may include a "Bind" button, an "Unbind" button, a "Remove" button, and a "Delete" button.

[0056] Among them, binding refers to binding the cold table data to the database instance, unbinding refers to disassociating the cold table data from the database instance, migrating refers to migrating the cold table data from the external storage back to the local database instance, and deleting refers to deleting the cold table data. It should be understood that the schematic diagram shown in Figure 3 is an interface displayed based on the cold table data of a database instance purchased by a user. In addition, the button names shown in Figure 3 are only for reference. In actual applications, they may also be named other ways, such as "bind", "mount", or "associate", etc. This application does not limit this.

[0057] The interface shown in FIG3 can be used by a user to select a database instance for accessing a cold table in cloud storage 30. For example, when a user clicks the "Bind" button, database management device 20 may receive a mount operation triggered by the user, and then the mount configuration interface may display other database instances purchased by the user. The user may select a target database instance from among the other database instances. For example, if the cold table is a cold table of a database included in database instance 100, the user may select the identification information of database instance 200, thereby completing the process of selecting a database instance for accessing a cold table in cloud storage 30. It should be understood that in the embodiment of the present application, database instance 200 and database instance 100 are different instances of the same user under the cloud computing platform.

[0058] Step 202: The database management device 20 sends a mount instruction to the database instance 200 according to the mount operation.

[0059] The mount instruction may carry identification information of the cold table. For ease of description, the present embodiment takes the first cold table (eg, the identification information of the cold table carried in the mount instruction is table1) as an example for description.

[0060] In some embodiments, the database management device 20 may send a mount instruction to the database instance 200 according to the mount operation. For example, using the structured query language (SQL) as an example, the mount instruction may be: MOUNT external_storage.table_name TO db1.table_name.

[0061] It should be understood that the above-mentioned mount instruction is only an illustration. In actual applications, it can also be other language instructions. As long as the function of accessing the cold table can be implemented, it can be included in the protection scope of this application.

[0062] In addition, the embodiment shown in FIG2 is only an example of accessing a cold table. In the present application, the database management device 20 may also receive multiple mount instructions at the same time, and the multiple access instructions are used to access different cold tables.

[0063] In the embodiment of the present application, the database management device 20 may verify the user's permission information before receiving a cold table mount instruction. If the user's permission information is authenticated, the user may issue the cold table mount instruction. Alternatively, it can be understood that after the user successfully logs in using their account and password, the user may issue the cold table mount instruction in the operation interface.

[0064] Step 203: the database instance 200 obtains metadata information of the first cold table from the cloud storage 30 according to the mount instruction.

[0065] In this embodiment of the present application, a cold table of the database can be stored in cloud storage 30, and the cold table can be used to store cold data in the database. Therefore, database instance 200 can find the cold table corresponding to the identification information in cloud storage 30 based on the identification information of the cold table in the mount instruction, and then obtain metadata information of the first cold table.

[0066] Step 204: The database instance 200 stores metadata information of the first cold table.

[0067] In some embodiments, after the database instance 200 obtains the metadata information of the first cold table, the metadata information of the first cold table may be persistently stored locally, which facilitates subsequent access to the first cold table.

[0068] Step 205: The database instance 200 uses the metadata information of the first cold table to access the cold data of the first cold table.

[0069] After obtaining the metadata information of the first cold table, the database instance 200 can access the cold data of the first cold table of the database included in the database instance 100 based on the metadata information of the first cold table, thereby realizing the function of accessing cold data across instances and improving the access flexibility and sharing of cold data.

[0070] As a possible implementation method, before the database instance 200 accesses the cold data of the first cold table based on the metadata information of the first cold table, that is, before step 205 is executed, the database instance 200 can also perform storage engine adaptation processing with the cloud storage 30, such as modifying the storage engine type so that it can match the cloud storage 30, so that the database instance can read the data in the cloud storage 30.

[0071] In the embodiment of the present application, if database instance 200 can access the cold data of the first cold table of the database included in database instance 100, the associated information (or, alternatively, reference information or mounting relationship, etc.) on cloud storage 30 can be updated. That is, reference information for database instance 200 to access the cold data of the first cold table is added to cloud storage 30. The reference relationship on cloud storage 30 will be described in detail below and will not be explained in detail here.

[0072] Through the above embodiment, the cold data of the cold table of the database included in a database instance, such as database instance A, can be mounted to another database instance, such as database instance B. When database instance B needs to access the cold data of the database included in database instance A, it can read the metadata information of the cold table locally from database instance B, and then access the corresponding cold data based on the metadata information.

[0073] The following describes the above process using two database instances as examples. For example, FIG4 is a schematic diagram of a cold data access method provided by an embodiment of the present application. Referring to FIG4 , the method may include:

[0074] It should be understood that before executing the method of the embodiment shown in FIG4 , the cold table (tn data) of the database included in database instance 1 (Instance 1) has been stored in external storage (External Storage), and the metadata information (tn_ref) of the cold table is stored locally in Instance 1, such as in an SSD. The metadata information of the cold table of the database included in database instance 1 is not yet stored locally in database instance 2 (Instance 2), such as in an SSD.

[0075] Step 401: The SQL layer of database instance 2 (Instance 2) receives a mount instruction for the first cold table data.

[0076] As a possible implementation method, when Instance2 needs to access the cold data of the database included in Instance1, the SQL layer of Instance2 can receive the mount instruction (e.g., Attach command) of the first cold table data issued by the user. Exemplarily, the mount instruction of the cold table data can be: MOUNT external_storage.table_name TO db1.table_name. It can also be understood as: mounting the first cold table data in Instance1 to Instance2. It should be understood that the mount instruction of the cold table data can be the mount instruction sent by the database management device 20 to the database instance 200 after the user triggers the "Bind" button in the embodiment shown in Figure 3.

[0077] Step 402: Database instance 2 (Instance 2) obtains metadata information of the first cold table from external storage.

[0078] Since the cold table of the database included in Instance 1 is stored in external storage (External Storage), Instance 2 can obtain metadata information of the first cold table from External Storage.

[0079] Step 403: Database instance 2 (Instance 2) updates metadata information locally.

[0080] After database instance 2 (Instance 2) obtains metadata information for the first cold table data, such as tn_ref, it can locally update the metadata information. It should be understood that the metadata information tn_ref in the embodiment shown in FIG3 can be a general reference or a specific reference. For example, tn_ref can refer to all metadata information for cold table data or to metadata information for a specific cold table data. Different references to different information have different corresponding contents.

[0081] In this embodiment of the present application, metadata information can be stored on the disk of the database instance. When the data table corresponding to the metadata information is queried, the metadata information can be cached in memory. That is, the metadata information is stored both on the disk and in memory of the database instance. Specifically, in step 403, updating the metadata information locally for database instance 2 (Instance 2) refers to updating the metadata information, such as the metadata structure information, in memory.

[0082] Step 404: Database instance 2 (Instance 2) saves the metadata information of the first cold table locally.

[0083] Database instance 2 (Instance 2) may permanently store the metadata information of the first cold table in the local storage of database instance 2 (Instance 2), such as an SSD.

[0084] Step 405: Update the reference information (Ref meta) of the cold table data on the external storage (External Storage).

[0085] In some embodiments, external storage can store reference information (ref meta) for cold table data. This reference information indicates the specific information that the cold table data is referenced by the database instance. After the metadata of the first cold table data is attached to database instance 2 (Instance 2), new reference information can be added to the external storage, such as adding Instance 2 to reference the first cold table data.

[0086] As a possible implementation method, the reference information of the cold table data on the external storage (External Storage) can be updated by the external storage (External Storage) itself. For example, the database instance 2 (Instance2) can send an update instruction to the external storage (External Storage). When the external storage (External Storage) receives the update instruction, it can update the reference information of the cold table data. Continuing with Figure 3 as an example, for example, when the user clicks the "Bind" button, the database management device 20 can receive the mount operation triggered by the user. When the database instance 200 obtains the metadata information of the cold table from the external storage (External Storage), it can also send an update instruction to the external storage (External Storage). After the external storage (External Storage) receives the update instruction, it can add the association information between the first cold table and the database instance 200.

[0087] Furthermore, after mounting the cold table of the database included in database instance A to database instance B, if the cold data in the cold table is no longer needed for querying, the association between the cold data in the cold table and database instance B can be disassociated, so that database instance B stops accessing the cold data in the cold table of the database included in database instance A. For example, FIG5 is a flowchart of a method for stopping access to cold data provided in an embodiment of the present application. Referring to FIG5 , the method may include the following steps:

[0088] Step 501: The database management device 20 receives an uninstallation operation triggered by a user on an uninstallation configuration interface of a cloud computing platform.

[0089] The uninstall configuration interface is used for the user to select a database instance to be uninstalled, and the uninstall operation includes an identifier of the second database instance selected by the user.

[0090] It should be understood that in the embodiments of the present application, the mount configuration interface and the unmount configuration interface can be the same interface. In actual applications, they can also be different interfaces, and this application does not limit this. Continuing with reference to FIG3 , illustratively, when a user clicks the "Unbind" button, the database management device 20 can receive the user-triggered unmount operation, and then the unmount configuration interface can display all database instances mounted to the cold table. The user can select a target database instance from the mounted database instances, such as selecting the identification information of database instance 200, and thus choose to unmount database instance 200 from the first cold table.

[0091] Step 502: The database management device 20 sends an uninstall instruction to the database instance 200 according to the uninstall operation.

[0092] The mount instruction may carry identification information of the cold table. For ease of description, the present embodiment takes the first cold table (eg, the identification information of the cold table carried in the unmount instruction is table1) as an example for description.

[0093] In some embodiments, the database management device 20 may send an uninstall instruction to the database instance 200 according to the uninstall operation. For example, using the structured query language (SQL) as an example, the uninstall instruction may be: UNMOUNT db1 / table_sample.

[0094] It should be understood that the above-mentioned uninstallation instruction is only an illustration. In actual applications, it can also be an instruction in other languages. As long as it can achieve the function of stopping access to the cold data of the first cold table, it can be included in the protection scope of this application.

[0095] Step 503: the database instance 200 deletes the metadata information of the first cold table from the local storage according to the uninstall instruction.

[0096] When the database instance 200 receives the uninstall instruction, it may delete the metadata information of the first cold table that is persistently stored locally.

[0097] In addition, database instance 200 may delete cache information of the first cold table data in the local cache. In the embodiment of the present application, when database instance 200 receives the uninstall instruction, regardless of whether cache information of the first cold table is stored in the local cache of the database instance, the cache information of the cold table data in the cache is deleted.

[0098] In an embodiment of the present application, if the database instance 200 cannot access the cold data of the first cold table of the database included in the database instance 100, the database instance 200 will subsequently be unable to access the cold data of the first cold table through the metadata information of the first cold table. At this time, the reference information on the cloud storage 30 can be updated, that is, the association information between the database instance 200 and the first cold table can be deleted on the cloud storage 30.

[0099] Through the above embodiment, the cold table data mounted to the database instance B can be uninstalled. For example, when the database instance B does not need to access the cold data of the database included in the database instance A, the metadata information of the cold table data can be deleted from the local of the database instance B, and then the database instance B will not be able to access the cold data of the cold table of the database included in the database instance A.

[0100] The following continues to introduce the above process using the two database instances in the example shown in Figure 4 as an example. For example, as shown in Figure 6, a schematic diagram of a method for stopping access to cold table data provided by an embodiment of the present application is shown. Referring to Figure 6, the method may include:

[0101] It should be understood that before executing the method of the embodiment shown in FIG6 , the cold table (tn data) of the database included in database instance 1 (Instance 1) has already been stored in external storage (External Storage), and the metadata information (tn_ref) of the cold table is stored locally in Instance 1, such as in an SSD. The metadata information of the cold table of the database included in database instance 1 has already been stored locally in database instance 2 (Instance 2), such as in an SSD.

[0102] Step 601: The SQL layer of database instance 2 (Instance 2) receives an instruction to unload data from a first cold table.

[0103] As a possible implementation, when Instance2 no longer needs to access the cold table data of Instance1, the SQL layer of Instance2 can receive a user-issued uninstall instruction (e.g., a Detach command). For example, this uninstall instruction can be: UNMOUNT db1 / table_sample. This can also be understood as separating the first cold table data in Instance1 from Instance2. It should be understood that this uninstall instruction for the cold table data can be the uninstall instruction that the database management device 20 sends to the database instance 200 after the user triggers the "Unbind" button in the embodiment shown in FIG3 .

[0104] Step 602: Database instance 2 (Instance 2) clears cache information of the first cold table data included in the local storage.

[0105] Since the metadata information of the first cold table data is stored locally in database instance 2 (Instance 2), when Instance 2 needs to access the first cold table data, it will use the metadata information of the first cold table data to access the first cold table data. Therefore, cache information of the first cold table data may be stored locally. At this time, the cache information of the first cold table data included in the local storage can be cleared (deleted).

[0106] Step 603: Database instance 2 (Instance 2) deletes metadata information of the first cold table stored locally.

[0107] In an embodiment of the present application, after the first cold table data is mounted to database instance 2 (Instance 2), the metadata information of the first cold table can be stored locally in database instance 2 (Instance 2). When the SQL layer of database instance 2 (Instance 2) issues a stop download instruction, the locally stored metadata information of the first cold table can be deleted. In this way, subsequent database instance 2 (Instance 2) can no longer access the cold data of the first cold table through the metadata information of the first cold table.

[0108] Step 604: Update the reference information (Ref meta) of the cold table data on the external storage (External Storage).

[0109] In some embodiments, after the metadata information of the first cold table data in the local storage of database instance 2 (Instance2) is separated, the reference information may be deleted on the External Storage, for example, deleting the reference to the first cold table data by Instance2.

[0110] For example, continuing with Figure 3 as an example, when the user clicks the "Unbind" button, the database management device 20 can receive the uninstallation operation triggered by the user. After the database instance 200 receives the uninstallation instruction sent by the database management device 20, it can also send an update instruction to the external storage (External Storage). After the external storage (External Storage) receives the update instruction, it can delete the association information between the first cold table and the database instance 200.

[0111] Through the above embodiments, cold table data can be attached to other database instances to achieve cross-instance access to cold table data. Furthermore, when access to cold table data is no longer needed, the cold table data can be separated from the attached database instance. Since the cold table data includes metadata information of the cold table data, and the database instance only stores metadata information, other data is still stored in external storage. In this way, the cold table data can exist as independent data and can be accessed by other authorized and format-compatible database instances. This solves the problem of binding database instances and cold table data in the prior art, making cold table data more flexible.

[0112] Furthermore, in the embodiment of the present application, the cloud storage 30 can store reference information (Ref meta) of the cold table data. The reference information (Ref meta) is used to record the specific information of the cold table data in each database instance. It should be understood that the reference relationship can also be understood as association information. As a possible implementation method, the reference information can be in the form shown in the following schematic:

[0113] (1) The reference information of each cold table is recorded by an object (or, it can also be understood as: file). That is, the cold table and the object that records the reference information are in a one-to-one relationship, and each object records the reference information of a cold table.

[0114] (2) The reference information of each cold table is recorded by three objects. That is, there is a one-to-many relationship between the cold table and the object that records the reference information. For example, each cold table can correspond to three types of objects: bind (REF), unbind (DEREF), and delete (DROP). These three types of objects respectively describe the situation in which each table is referenced by the external database instance.

[0115] (3) All cold tables share one object and record corresponding reference information separately.

[0116] It should be understood that the form of the reference information includes but is not limited to the above forms, and this application does not make any specific limitations on this.

[0117] The second form is introduced below as an example. For example, as shown in FIG7 , a schematic diagram of a reference relationship in an external storage provided in an embodiment of the present application is provided. The external storage 300 may include multiple cold table data, such as Table 1 (table1), Table 2 (table1) ... Table X (table1x), and a reference relationship record may be added for each cold table.

[0118] In the second form, REF indicates marking a new reference relationship. A reference tag is added when migrating or appending cold table data. The tag can be named as ref_timestamp. DEREF indicates marking a deleted reference relationship. A reference tag is added when separating or remigrating cold table data. The tag can be named as deref_timestamp. DROP indicates marking a deleted reference relationship. When the number of references marked with the DROP tag drops to 0, the cold table data needs to be deleted, triggering the cold table data cleanup mechanism. The tag can be named as drop_timestamp.

[0119] Taking table1 as an example, the reference relationship tag information for table1 can be: 1. Alter table migrates cold table data from local storage to external storage, automatically adding reference records; 2. Attaching a cold table to a new instance automatically adds reference records if the table name is referenced by another instance; 3. The instance drops the cold table in external storage, adding and deleting reference records. If the add and delete reference tags are not equal at this time, the actual DROP operation is similar to detaching the cold table, and the actual data on the external storage will not be deleted; 4. The cold table data is migrated back to hot table data, adding and deleting reference records. After the migration, the cold table data on the third-party storage will not be deleted. 5 and 6 represent the detach cold table operation, adding and deleting reference records. This operation does not delete the cold table data on the third-party storage, but only cleans up the metadata of the cold table data on the local database instance. After the detach, the database instance cannot access the cold table data.

[0120] If this is a record drop operation, the number of ref references in the REF meta data is counted after the operation completes, and a check is performed to see if the number of ref references equals the number of drop + deref operations. If so, the cold table data on the third-party storage is cleaned up, including the reference relationship records. Otherwise, the cold table data on the third-party storage is not cleaned up.

[0121] In the embodiment of the present application, the cleaning of cold table data can be triggered by the user, or the user can choose to have the third-party storage system clean up the data. For example, a control switch can be added to the user interface, allowing the user to choose whether the third-party storage system automatically cleans the data or the user can clean the data themselves according to the actual situation.

[0122] The present application further provides a cold data processing device 800 , as shown in FIG8 , comprising: a transceiver module 801 , an acquisition module 802 , a storage module 803 and a processing module 804 .

[0123] Exemplarily, transceiver module 801 is used to trigger a mount operation on the cloud computing platform's mount configuration interface. Acquisition module 802 is used to retrieve metadata information for the cold table from cloud storage in accordance with the mount instruction. Storage module 803 is used to store the metadata information for the cold table. Processing module 804 is used to use the metadata information for the cold table to access the cold data in the cold table.

[0124] In the embodiment of the present application, the transceiver module 801, the acquisition module 802, the storage module 803, and the processing module 804 in the cold data processing device 800 can all be implemented via software or hardware. The following describes the implementation of the processing module 804, taking the processing module 804 as an example. Similarly, the implementation of the transceiver module 801, the acquisition module 802, and the storage module 803 can refer to the implementation of the processing module 804.

[0125] The module may include code running on a computing instance. The processing module 804 may include code running on a computing instance. The computing instance may be at least one of a physical host (computing device), a virtual machine, a container, and other computing devices. Furthermore, the computing instance may be one or more. For example, the processing module 804 may include code running on multiple hosts, or virtual machines, or containers. It should be noted that the multiple hosts, or virtual machines, or containers used to run the code may be distributed in the same region (region) or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple geographically close data centers. Typically, a region may include multiple AZs.

[0126] Similarly, multiple hosts, virtual machines, or containers running the code can be distributed within the same virtual private cloud (VPC) or across multiple VPCs. Typically, a region can include multiple AZs. Typically, a VPC is located within a region. Cross-region communication between two VPCs within the same region, or between VPCs in different regions, requires a communication gateway within each VPC to interconnect the VPCs.

[0127] As an example of a hardware functional unit, processing module 804 may include at least one computing device, such as a server. Alternatively, processing module 804 may be implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0128] The multiple computing devices included in processing module 804 can be distributed in the same AZ or in different AZs. The multiple computing devices included in processing module 804 can be distributed in the same region or in different regions. Similarly, the multiple computing devices included in processing module 804 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of servers, ASICs, PLDs, CPLDs, FPGAs, GALs, and other computing devices.

[0129] It should be noted that, in other embodiments, the processing module 804 can be used to execute any step in the cold data processing method, and the transceiver module 801 can be used to execute any step in the cold data processing method. The steps that the transceiver module 801, the acquisition module 802, the storage module 803, and the processing module 804 are responsible for implementing can be specified as needed. The full functionality of the cold data processing device is achieved by having the transceiver module 801, the acquisition module 802, the storage module 803, and the processing module 804 each implement different steps in the cold data processing method.

[0130] This application also provides a computing device 900. As shown in Figure 9, computing device 900 includes a bus 902, a processor 904, a memory 906, and a communication interface 908. Processor 904, memory 906, and communication interface 908 communicate with each other via bus 902. Computing device 900 can be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in computing device 900.

[0131] Bus 902 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, among others. Buses may be classified as address buses, data buses, control buses, and the like. For ease of illustration, FIG9 illustrates a single bus line, but this does not imply a single bus or type of bus. Bus 902 may include a path for transmitting information between various components of computing device 900 (e.g., memory 906, processor 904, and communication interface 908).

[0132] The processor 904 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0133] The memory 906 may include volatile memory, such as random access memory (RAM). The memory 906 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0134] The memory 906 stores executable program code, and the processor 904 executes the executable program code to respectively implement the functions of the multiple modules in the aforementioned cold meter data processing device, thereby implementing the cold meter data processing method. In other words, the memory 906 stores instructions for executing the cold meter data processing method.

[0135] The communication interface 908 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 900 and other devices or a communication network.

[0136] This application also provides a computing device cluster. Figure 10 is a schematic diagram of the computing device cluster. The computing device cluster can implement the cold table data processing method in the above embodiment. As shown in Figure 10, the computing device cluster includes at least one computing device 900 as shown in Figure 9. The computing device 900 can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0137] Instructions for executing the cold table data processing method may be stored in the memory 906 of one or more computing devices 900 in the computing device cluster. When at least one computing device in the computing device cluster executes the instructions, the computing device cluster may implement the cold data processing method described in the embodiment.

[0138] In some possible implementations, the memory 906 of one or more computing devices 900 in the computing device cluster may also store partial instructions for executing the cold table data processing method. In other words, the combination of one or more computing devices 900 can jointly execute the instructions for executing the cold data processing method.

[0139] It should be noted that the memory 906 in different computing devices 900 in the computing device cluster can store different instructions, each for executing a portion of the functions of the cold table data processing device. In other words, the instructions stored in the memory 906 in different computing devices 900 can implement the functions of one or more of the transceiver module 801, acquisition module 802, storage module 803, and processing module 804 in the cold data processing device.

[0140] In some possible implementations, one or more computing devices in a computing device cluster may be connected via a network. The network may be a wide area network (WAN), a local area network (LAN), or the like. FIG11 illustrates one possible implementation. As shown in FIG11 , two computing devices 900A and 900B are connected via a network. Specifically, the connection to the network is achieved via a communication interface within each computing device. In this type of possible implementation, the memory 906 within computing device 900A stores instructions for executing the functions of the processing module 804 within the cold data processing apparatus. Simultaneously, the memory 906 within computing device 900B stores instructions for executing the functions of the transceiver module 801, the acquisition module 802, and the storage module 803 within the cold data processing apparatus.

[0141] It should be understood that the functionality of the computing device 900A shown in FIG11 may also be implemented by multiple computing devices 900. Similarly, the functionality of the computing device 900B may also be implemented by multiple computing devices 900.

[0142] The present application also provides a computer program product comprising instructions. The computer program product may be software or a program product comprising instructions that can be executed on a computing device or stored in any available medium. When the computer program product is executed on at least one computing device, the computer program product causes the at least one computing device to perform a cold data processing method.

[0143] The present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device, or a data storage device such as a data center that contains one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to execute the cold table data processing method, or instruct the computing device to execute the cold data processing method.

[0144] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the protection scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A cold data processing method, characterized in that: A cloud computing platform provides a database system, a database management device and cloud storage, wherein the database system includes a first database instance and a second database instance, the database management device is used to manage the database system, the first database instance and the second database instance are different instances belonging to the same user under the cloud computing platform, the first database instance is used to store a cold table of the database in the cloud storage, and the cold table is used to store cold data of the database; the method comprises: The database management device receives a mount operation triggered by a user on a mount configuration interface of the cloud computing platform, wherein the mount configuration interface allows the user to select a database instance of the cold table in the cloud storage to be accessed, and the mount operation includes an identifier of the second database instance selected by the user; The database management device sends a mount instruction to the second database instance according to the mount operation, wherein the mount instruction carries an identifier of the cold table; The second database instance obtains metadata information of the cold table from the cloud storage according to the mounting instruction; The second database instance stores metadata information of the cold table; The second database instance uses the metadata information of the cold table to access the cold data of the cold table.

2. The method according to claim 1, characterized in that The first database instance and the second database instance are deployed on the same computing node or on different computing nodes.

3. The method according to claim 1 or 2, characterized in that The method further comprises: The second database instance sends a first update instruction to the cloud storage, where the first update instruction is used to instruct the cloud storage to add a mounting relationship between the cold table of the database and the second database instance.

4. The method according to any one of claims 1 to 3, characterized in that The method further comprises: The database management device receives an uninstallation operation triggered by a user on an uninstallation configuration interface of the cloud computing platform, wherein the uninstallation configuration interface allows the user to select a database instance to be uninstalled, and the uninstallation operation includes an identifier of the second database instance selected by the user; The database management device sends an uninstall instruction to the second database instance according to the uninstall operation, wherein the uninstall instruction carries an identifier of the cold table; The second database instance deletes the metadata information of the cold table from the local storage according to the uninstall instruction.

5. The method according to claim 4, characterized in that The method further comprises: The second database instance deletes cache information of the cold table from the local cache.

6. The method according to claim 4 or 5, characterized in that The method further comprises: The second database instance sends a second update instruction to the cloud storage, where the second update instruction is used to instruct the cloud storage to delete the mounting relationship between the cold table of the database and the second database instance.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: The database management device receives a deletion operation triggered by a user on a deletion configuration interface of the cloud computing platform, wherein the deletion configuration interface allows the user to select cold data of a cold table to be deleted, and the deletion operation includes an identifier of the cold table selected by the user; The second database instance sends a deletion instruction to the cloud storage, where the deletion instruction carries an identifier of the cold table; The cloud storage deletes the cold data of the cold table according to the deletion instruction.

8. A cold data processing device, characterized in that: The device comprises: A transceiver module, configured to receive a mount operation triggered by a user on a mount configuration interface of a cloud computing platform, wherein the mount configuration interface allows the user to select a database instance of a cold table in a cloud storage to be accessed, and the mount operation includes an identifier of a second database instance selected by the user; The transceiver module is further configured to send a mount instruction to the second database instance according to the mount operation, wherein the mount instruction carries an identifier of the cold table; An acquisition module, used for acquiring metadata information of the cold table from the cloud storage according to the mounting instruction; A storage module, used for storing metadata information of the cold table; The processing module is used to access the cold data of the cold table by using the metadata information of the cold table.

9. The device according to claim 8, characterized in that The transceiver module is further used to send a first update instruction to the cloud storage, where the first update instruction is used to instruct the cloud storage to add a mounting relationship between the cold table of the database and the second database instance.

10. The device according to claim 8 or 9, characterized in that The transceiver module is further configured to receive an uninstallation operation triggered by a user on an uninstallation configuration interface of the cloud computing platform, wherein the uninstallation configuration interface allows the user to select a database instance to be uninstalled, and the uninstallation operation includes an identifier of the second database instance selected by the user; and according to the uninstallation operation, send an uninstallation instruction to the second database instance, wherein the uninstallation instruction carries the identifier of the cold table; The processing module is further configured to delete the metadata information of the cold table from the local storage according to the uninstall instruction.

11. The device according to claim 10, characterized in that The processing module is further configured to delete cache information of the cold table from the local cache.

12. The device according to claim 10 or 11, characterized in that The transceiver module is further used to send a second update instruction to the cloud storage, where the second update instruction is used to instruct the cloud storage to delete the mounting relationship between the cold table of the database and the second database instance.

13. The device according to any one of claims 8 to 12, characterized in that: The transceiver module is further used to receive a deletion operation triggered by a user on a deletion configuration interface of the cloud computing platform, wherein the deletion configuration interface allows the user to select cold data of a cold table to be deleted, and the deletion operation includes an identifier of the cold table selected by the user; and send a deletion instruction to the cloud storage, wherein the deletion instruction carries the identifier of the cold table; The processing module is further used to delete the cold data in the cold table according to the deletion instruction.

14. A computing device cluster, characterized in that: comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method according to any one of claims 1 to 7.

15. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device cluster, the computing device cluster executes the method according to any one of claims 1 to 7.

16. A computer-readable storage medium, characterized in that: The method comprises computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster performs the method according to any one of claims 1 to 7.

Images