Cold data processing method and equipment

Abstract

The invention provides a cold data processing method and equipment. The method comprises the steps that a database management device receives a mounting operation triggered by a user on a mounting configuration interface of a cloud computing platform, the mounting configuration interface is used for the user to select a database instance of a cold table in a cloud storage to be accessed, and the mounting operation comprises an identifier of a target database instance selected by the user; the database management device sends a mounting instruction to the target database instance according to the mounting operation, wherein the mounting instruction carries an identifier of the cold table; the target database instance obtains metadata information of the cold table from cloud storage according to the mounting instruction and stores the metadata information of the cold table; and accessing the cold data of the cold table by using the metadata information of the cold table. Therefore, cross-instance access to the cold data is realized, and the access flexibility of the cold data can be improved.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to the Chinese patent application filed with the Intellectual Property Office of the People's Republic of China on July 19, 2023, with application number 202310889687.8 and invention name "A cold table data processing method, device and computing device cluster", all contents of which are incorporated by reference in this application. Technical Field

[0003] The present application relates to the field of database technology, and in particular to a cold data processing method and device. Background Art

[0004] In the process of using the database, a large amount of data is often written and updated. At the same time, the database also stores some data that is not frequently accessed, which is called cold data. When cold data is stored in the database for a long time, it will cause a serious waste of storage resources and high costs.

[0005] In order to solve the problem of storage resource waste, the current common method is to migrate cold data from the database to a third-party storage. However, the migrated cold data is bound to the database instance and can only be accessed by the original database instance, which has poor flexibility. Summary of the invention

[0006] The present application provides a cold data processing method and device to improve the flexibility of cold data access.

[0007] In a first aspect, the present application provides a cold data processing method, which is applied to a system including a database system, a database management device and a cloud storage provided by a cloud computing platform, wherein the database system includes a first database instance and a second database instance, the database management device is used to manage the database system, the first database instance and the second database instance are different instances belonging to the same user under the cloud computing platform, the first database instance is used to store a cold table of the database in the cloud storage, and the cold table is used to store cold data of the database. Specifically, the method includes: the database management device receives a mount operation triggered by a user on a mount configuration interface of the cloud computing platform, the mount configuration interface allows the user to select a database instance of the cold table to be accessed in the cloud storage, the mount operation includes an identifier of the second database instance selected by the user; then, the database management device sends a mount instruction to the second database instance according to the mount operation, the mount instruction carries the identifier of the cold table; then, the second database instance obtains metadata information of the cold table from the cloud storage according to the mount instruction, and stores the metadata information of the cold table; finally, the second database instance uses the metadata information of the cold table to access the cold data of the cold table.

[0008] Through the above technical solution, the cold data of a cold table of a database can be mounted (or, can also be called: associated) with another database, so as to achieve cross-instance access to the cold data of the cold table, which can improve the flexibility of cold data access.

[0009] In a possible implementation, the first database instance and the second database instance are deployed on the same computing node, or the first database instance and the second database instance are deployed on different computing nodes.

[0010] In a possible implementation, the method further includes: the second database instance sending a first update instruction to the cloud storage, where the first update instruction is used to instruct the cloud storage to add a mounting relationship between the cold table of the database and the second database instance.

[0011] Through the above technical solution, after the cold table of the database is mounted with other databases, the cloud storage can record and store the mounting relationship between the cold table and the database instance, so as to manage the mounting relationship between the database instance and the cold table.

[0012] In a possible implementation, the method further includes: a database management device receives an uninstallation operation triggered by a user on an uninstallation configuration interface of the cloud computing platform, the uninstallation configuration interface allowing the user to select a database instance to be uninstalled, the uninstallation operation including an identifier of the second database instance selected by the user; then, the database management device sends an uninstallation instruction to the second database instance according to the uninstallation operation, the uninstallation instruction carrying the identifier of the cold table; finally, the second database instance deletes the metadata information of the cold table from the local storage according to the uninstallation instruction.

[0013] Through the above technical solution, the metadata information of the cold table stored locally in the second database instance can be deleted to achieve unloading of the cold table and mounting of the second database instance. Subsequently, the second database instance will not be able to access the cold data of the cold table.

[0014] In a possible implementation, the method further includes: the second database instance deletes cache information of the cold table from the local cache, so that there is no query record of the cold table in the local cache of the second database instance, and the cold data of the cold table cannot be accessed subsequently.

[0015] In a possible implementation, the method further includes: the second database instance sending a second update instruction to the cloud storage, where the second update instruction is used to instruct the cloud storage to delete the mounting relationship between the cold table of the database and the second database instance.

[0016] Through the above technical solution, after the cold table of the download database is mounted with other databases, the cloud storage can record and store the uninstallation relationship between the cold table and the database instance, so as to manage the association relationship between the database instance and the cold table.

[0017] In a possible implementation, the method further includes: a database management device receives a deletion operation triggered by a user on a deletion configuration interface of the cloud computing platform, the deletion configuration interface allowing the user to select cold data of a cold table to be deleted, the deletion operation including an identifier of the cold table selected by the user; the second database instance sends a deletion instruction to the cloud storage, the deletion instruction carries the identifier of the cold table; the cloud storage deletes the cold data of the cold table according to the deletion instruction.

[0018] Through the above technical solution, cold data in the cold table can be deleted according to the operation instruction triggered by the user to improve memory utilization.

[0019] In a second aspect, the present application provides a cold meter data processing device, which includes a module for implementing the cold meter data processing method provided by the above-mentioned first aspect or any possible implementation manner of the first aspect.

[0020] In a third aspect, the present application provides a computing device cluster, comprising at least one computing device, each computing device comprising a processor and a memory; the processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the cold table data processing method provided in the first aspect or any possible implementation of the first aspect.

[0021] In a fourth aspect, the present application provides a computer program product comprising instructions, which, when executed by a computing device cluster, enables the computing device cluster to execute a cold table data processing method as provided in the first aspect or any possible implementation of the first aspect.

[0022] In a fifth aspect, the present application provides a computer-readable storage medium comprising computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster executes the cold table data processing method provided in the above-mentioned first aspect or any possible implementation of the first aspect.

[0023] For each aspect from the second to the fifth aspect and the technical effects that may be achieved by each aspect, please refer to the above description of the technical effects that can be achieved by various possible solutions in the first aspect, and no further details will be given here. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A schematic diagram of a system architecture provided for an embodiment of the present application;

[0025] Figure 2 A flow chart of a method for accessing cold data provided in an embodiment of the present application;

[0026] Figure 3 A schematic diagram of a configuration interface of a cloud computing platform provided in an embodiment of the present application;

[0027] Figure 4 A schematic diagram of a method for accessing cold data provided in an embodiment of the present application;

[0028] Figure 5 A flow chart of a method for stopping access to cold data provided by an embodiment of the present application;

[0029] Figure 6 A schematic diagram of a method for stopping access to cold data provided by an embodiment of the present application;

[0030] Figure 7 A schematic diagram of a reference relationship in an external storage provided in an embodiment of the present application;

[0031] Figure 8A schematic diagram of a cold data processing device provided in an embodiment of the present application;

[0032] Fig. 9 A schematic diagram of the structure of a computing device provided in an embodiment of the present application;

[0033] Fig.10 A schematic diagram of the structure of a computing device cluster provided in an embodiment of the present application;

[0034] Fig.11 A schematic diagram of the structure of another computing device cluster provided in an embodiment of the present application. DETAILED DESCRIPTION

[0035] In the description of the embodiments of the present application, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc. may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0036] In the present application, "at least one" means one or more, and "plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: including the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0037] Below, some terms in the embodiments of the present application are first explained to facilitate understanding by those skilled in the art.

[0038] 1. Cold Data

[0039] Refers to data that has been accessed less than a set threshold or used less than a set threshold, but needs to be stored for a long time. For example, cold data in a banking system can be historical transaction records of the banking system five years ago.

[0040] 2. Cold table

[0041] A database may include multiple data tables. When the data in these data tables are cold data, these data tables may be called cold tables.

[0042] 3. Database instance

[0043] In a distributed database, a large number of database instances are usually created. For example, a database instance is created for each tenant. The database instance here can be understood as a database server, in which one or more databases can be created, and one or more data tables can be created in each database, and data can be stored in the data tables.

[0044] like Figure 1 FIG. 1 is a schematic diagram of a system architecture provided in an embodiment of the present application. Figure 1 As shown, the system architecture may include a database system 10, a database management device 20, and a cloud storage 30. The database system may include multiple database instances, for example, two database instances, such as a first database instance 100 and a second database instance 200. The database management device 20 is used to manage the database system 10, and the first database instance 100 and the second database instance 200 may be different instances belonging to the same user under the cloud computing platform. The cloud storage 30 is used to store cold data of the database.

[0045] In some embodiments, the first database instance 100 may include a processor 110 and a first memory 120, and the first memory 120 may include a volatile memory (volatile memory), such as a random access memory (RAM). The first memory 120 may also include a non-volatile memory (non-volatile memory, NVM), such as a read-only memory (read-only memory, ROM), a flash memory, a hard disk drive (hard disk drive, HDD) or a solid-state drive (solid-state drive, SSD). The first memory 120 may store metadata information of a cold table of the first database.

[0046] The structure of the second database instance 200 is similar to that of the first database instance 100, and the second database instance 200 includes a processor 210 and a second memory 220. The description of the processor 210 and the second memory 220 is similar to that of the processor 110 and the first memory 120, and can be found in the above content, which will not be repeated here.

[0047] Optionally, the first database instance 100 and the second database instance 200 may further include a memory, a network card, etc. For example, the memory may be located in the first memory 120 or the second memory 220, which is not limited in the present application.

[0048] Cloud storage (also referred to as external storage or third-party storage) 30 can store the cold table of the first database and / or the cold table of the second database. The cold table is used to store cold data of the database. It should be understood that in the embodiment of the present application, the cloud storage 30 can also be a physical storage device, which is not limited in the present application.

[0049] Taking the first database instance 100 as an example, in the embodiment of the present application, the cold table of the first database can be stored in the cloud storage 30. The metadata information of the cold table can be stored in the first memory 120 of the first database instance 100. When the second database instance 200 needs to access the cold table of the first database, the metadata information of the cold table can be obtained from the cloud storage 30, and then the second database instance 200 can access the cold data of the cold table based on the metadata information of the cold table.

[0050] When the second database instance 200 does not need to access the cold table of the first database, the metadata information of the cold table can be deleted in the local storage, so that the second database instance 200 cannot access the cold data of the cold table based on the metadata information of the cold table.

[0051] The following is an introduction to a method for accessing cold data provided by an embodiment of the present application in conjunction with a flow chart. Figure 2 As shown, the method may include the following steps:

[0052] It should be understood that Figure 2 The embodiment shown is described by taking the case where the database instance 200 needs to access the cold data of the database included in the database instance 100 as an example. For the specific implementation of the case where the database instance 100 needs to access the cold data of the database included in the database instance 200 or other database instances, please refer to Figure 2 The description in the illustrated embodiment. Database instance 200 refers to Figure 1 The second database instance 200 in the system architecture shown, the database instance 100 refers to Figure 1 A first database instance 100 in the system architecture is shown.

[0053] Step 201: The database management device 20 receives a mount operation triggered by a user on a mount configuration interface of a cloud computing platform.

[0054] The mount configuration interface allows the user to select a database instance of a cold table in the cloud storage 30 to be accessed, and the mount operation includes an identifier of the second database instance selected by the user. Figure 3 , which is a schematic diagram of a configuration interface of a cloud computing platform provided by an embodiment of the present application. The operation bar 301 may include a "bind" button, an "unbind" button, a "re-migrate" button, and a "delete" button.

[0055] Among them, binding refers to binding the cold table data to the database instance, unbinding refers to removing the relationship between the cold table data and the database instance, relocation refers to migrating the cold table data from the external storage back to the local database instance, and deletion refers to deleting the cold table data. It should be understood that Figure 3 The diagram shown is an interface that displays the cold table data of a database instance purchased by a user. Figure 3 The button names shown are only for reference. In actual applications, they may also be other names, such as "bind", "mount" or "associate", etc. This application does not limit this.

[0056] Figure 3 The interface shown can be used for the user to select the database instance of the cold table in the cloud storage 30 to be accessed. Exemplarily, when the user clicks the "bind" button, the database management device 20 can receive the mount operation triggered by the user, and then the mount configuration interface can display other database instances purchased by the user. The user can select the target database instance from other database instances. For example, if the cold table is a cold table of the database included in the database instance 100, the identification information of the database instance 200 can be selected at this time, thereby completing the process of selecting the database instance of the cold table in the cloud storage 30 to be accessed. It should be understood that in the embodiment of the present application, the database instance 200 and the database instance 100 are different instances of the same user under the cloud computing platform.

[0057] Step 202: the database management device 20 sends a mount instruction to the database instance 200 according to the mount operation.

[0058] The mount instruction may carry identification information of the cold table. For the convenience of description, the first cold table (for example, the identification information of the cold table carried in the mount instruction is table1) is used as an example for description in the embodiment of the present application.

[0059] In some embodiments, the database management device 20 may send a mount instruction to the database instance 200 according to the mount operation. For example, taking the structured query language (SQL) as an example, the mount instruction may be: MOUNT external_storage.table_name TO db1.table_name.

[0060] It should be understood that the above-mentioned mounting instruction is only an illustration. In actual application, it can also be other language instructions. As long as the function of accessing the cold table can be realized, it can be included in the protection scope of this application.

[0061] in addition, Figure 2The illustrated embodiment only takes accessing a cold table as an example. In the present application, the database management device 20 may also receive multiple mount instructions at the same time, and the multiple access instructions are used to access different cold tables.

[0062] In the embodiment of the present application, the database management device 20 can verify the user's authority information before receiving the cold table mount instruction. When the user's authority information is authenticated, the user can issue the cold table mount instruction. It can also be understood that when the user successfully logs in with the account and password, the user can issue the cold table mount instruction in the operation interface.

[0063] Step 203: the database instance 200 obtains metadata information of the first cold table from the cloud storage 30 according to the mounting instruction.

[0064] In the embodiment of the present application, the cold table of the database can be stored in the cloud storage 30, and the cold table can be used to store the cold data of the database. Therefore, the database instance 200 can find the cold table corresponding to the identification information from the cloud storage 30 according to the identification information of the cold table in the mount instruction, and then obtain the metadata information of the first cold table.

[0065] Step 204: the database instance 200 stores metadata information of the first cold table.

[0066] In some embodiments, after the database instance 200 obtains the metadata information of the first cold table, the metadata information of the first cold table may be persistently stored locally, which can facilitate subsequent access to the first cold table.

[0067] Step 205: the database instance 200 uses the metadata information of the first cold table to access the cold data of the first cold table.

[0068] After obtaining the metadata information of the first cold table, the database instance 200 can access the cold data of the first cold table of the database included in the database instance 100 based on the metadata information of the first cold table, thereby realizing the function of accessing cold data across instances and improving the access flexibility and sharing of cold data.

[0069] As a possible implementation method, before the database instance 200 accesses the cold data of the first cold table according to the metadata information of the first cold table, that is, before step 205 is executed, the database instance 200 can also perform storage engine adaptation processing with the cloud storage 30, such as modifying the storage engine type so that it can match the cloud storage 30, so that the database instance can read the data in the cloud storage 30.

[0070] In the embodiment of the present application, if the database instance 200 can access the cold data of the first cold table of the database included in the database instance 100, the associated information (or, also referred to as: reference information or mounting relationship, etc.) on the cloud storage 30 can be updated, that is, the reference information of the database instance 200 accessing the cold data of the first cold table is added on the cloud storage 300. The reference relationship on the cloud storage 300 will be described in detail below, and will not be described in detail here.

[0071] Through the above embodiments, the cold data of the cold table of the database included in a database instance, such as database instance A, can be mounted to another database instance, such as database instance B. When database instance B needs to access the cold data of the database included in database instance A, the metadata information of the cold table can be read locally from database instance B, and then the corresponding cold data can be accessed based on the metadata information.

[0072] The following is an introduction to the above process using two database instances as examples. Figure 4 FIG. 1 is a schematic diagram of a method for accessing cold data provided by an embodiment of the present application. Figure 4 As shown, the method may include:

[0073] It should be understood that Figure 4 Before the method of the illustrated embodiment is executed, the cold table (tn data) of the database included in the database instance 1 (Instance1) has been stored in the external storage (External Storage), and the metadata information (tn_ref) of the cold table is stored locally in Instance1, such as in an SSD. The metadata information of the cold table of the database included in the database instance 1 is not yet stored locally in the database instance 2 (Instance2), such as in an SSD.

[0074] Step 401: The SQL layer of database instance 2 (Instance 2) receives a mount instruction for the first cold table data.

[0075] As a possible implementation method, when Instance2 needs to access the cold data of the database included in Instance1, the SQL layer of Instance2 can receive the mount command (for example, Attachcommand) of the first cold table data issued by the user. Exemplarily, the mount command of the cold table data can be: MOUNT external_storage.table_name TO db1.table_name. It can also be understood as: mount the first cold table data in Instance1 to Instance2. It should be understood that the mount command of the cold table data can be Figure 3In the illustrated embodiment, after the user triggers the “bind” button, the database management device 20 sends a mount instruction to the database instance 200 .

[0076] Step 402: Database instance 2 (Instance 2) obtains metadata information of the first cold table from external storage.

[0077] Since the cold table of the database included in Instance 1 is stored in external storage (External Storage), Instance 2 can obtain metadata information of the first cold table from External Storage.

[0078] Step 403: Database instance 2 (Instance 2) updates metadata information locally.

[0079] When database instance 2 (Instance2) obtains the metadata information of the first cold table data, such as tn_ref, the metadata information can be updated locally. It should be understood that Figure 3 The metadata information tn_ref in the illustrated embodiment may be a general reference or a specific reference. For example, tn_ref may refer to all metadata information of cold table data or metadata information of a certain cold table data. When the referenced information is different, the corresponding content is also different.

[0080] In the embodiment of the present application, the metadata information can be stored in the disk of the database instance, and when the data table corresponding to the metadata information is queried, the metadata information can be cached in the memory. That is, the disk and memory of the database instance both include metadata information. Specifically, in step 403, the local update of metadata information by database instance 2 (Instance2) refers to updating metadata information in the memory, such as metadata structure information.

[0081] Step 404: Database instance 2 (Instance 2) saves the metadata information of the first cold table locally.

[0082] Database instance 2 (Instance 2) may permanently store the metadata information of the first cold table in the local storage of database instance 2 (Instance 2), such as an SSD.

[0083] Step 405: Update the reference information (Ref meta) of the cold table data on the external storage (External Storage).

[0084] In some embodiments, reference information (Ref meta) of cold table data may be stored on external storage (External Storage), and the reference information of the cold table data is used to indicate specific information that the cold table data is referenced by the database instance. After the metadata information of the first cold table data is attached to database instance 2 (Instance2), reference information may be added to ExternalStorage, such as adding Instance2 to reference the first cold table data.

[0085] As a possible implementation, the reference information of the cold table data on the external storage can be updated by the external storage itself. For example, the database instance 2 can send an update instruction to the external storage, and when the external storage receives the update instruction, it can update the reference information of the cold table data. Figure 3 For example, when the user clicks the "Bind" button, the database management device 20 can receive the mount operation triggered by the user. When the database instance 200 obtains the metadata information of the cold table from the external storage (External Storage), it can also send an update instruction to the external storage (External Storage). After the external storage (External Storage) receives the update instruction, it can add the association information between the first cold table and the database instance 200.

[0086] Furthermore, after the cold table of the database included in database instance A is mounted to database instance B, if the cold data of the cold table does not need to be queried, the association between the cold data of the cold table and database instance B can be released, so that database instance B stops accessing the cold data of the cold table of the database included in database instance A. For example, Figure 5 As shown, it is a flow chart of a method for stopping access to cold data provided by an embodiment of the present application, see Figure 5 As shown, the method may include the following steps:

[0087] Step 501: the database management device 20 receives an uninstallation operation triggered by a user on an uninstallation configuration interface of a cloud computing platform.

[0088] The uninstall configuration interface is used for the user to select a database instance to be uninstalled, and the uninstall operation includes an identifier of a second database instance selected by the user.

[0089] It should be understood that in the embodiment of the present application, the mounting configuration interface and the uninstallation configuration interface can be the same interface, and in actual application, they can also be different interfaces, which is not limited in the present application. Figure 3 For example, when the user clicks the "Unbind" button, the database management device 20 may receive the uninstallation operation triggered by the user, and then all database instances mounted with the cold table may be displayed on the uninstallation configuration interface. The user may select the target database instance from the mounted database instances, such as selecting the identification information of the database instance 200, thereby choosing to uninstall the mounting of the database instance 200 with the first cold table.

[0090] Step 502: the database management device 20 sends an uninstall instruction to the database instance 200 according to the uninstall operation.

[0091] The mount instruction may carry identification information of the cold table. For the convenience of description, the first cold table (for example, the identification information of the cold table carried in the unmount instruction is table1) is used as an example for description in the embodiment of the present application.

[0092] In some embodiments, the database management device 20 may send an uninstall instruction to the database instance 200 according to the uninstall operation. For example, taking the structured query language (SQL) as an example, the uninstall instruction may be: UNMOUNT db1 / table_sample.

[0093] It should be understood that the above unloading instruction is only an illustration. In practical applications, it may also be an instruction in other languages. As long as the function of stopping access to the cold data in the first cold table can be achieved, it can be included in the protection scope of this application.

[0094] Step 503: the database instance 200 deletes the metadata information of the first cold table from the local storage according to the uninstall instruction.

[0095] When the database instance 200 receives the uninstall instruction, the metadata information of the first cold table that is persistently stored locally may be deleted.

[0096] In addition, the database instance 200 can delete the cache information of the first cold table data in the local cache. In the embodiment of the present application, when the database instance 200 receives the uninstall instruction, regardless of whether the cache information of the first cold table is stored in the local cache of the database instance, the cache information of the cold table data in the cache is deleted.

[0097] In an embodiment of the present application, if the database instance 200 cannot access the cold data of the first cold table of the database included in the database instance 100, the database instance 200 will subsequently be unable to access the cold data of the first cold table through the metadata information of the first cold table. At this time, the reference information on the cloud storage 30 can be updated, that is, the association information between the database instance 200 and the first cold table is deleted on the cloud storage 30.

[0098] Through the above embodiments, it is possible to uninstall the cold table data mounted to database instance B. For example, when database instance B does not need to access the cold data of the database included in database instance A, the metadata information of the cold table data can be deleted from the local of database instance B, and then database instance B will not be able to access the cold data of the cold table of the database included in database instance A.

[0099] Continue below Figure 4 The above process is described by taking two database instances in the example shown as examples. Figure 6 FIG. 1 is a schematic diagram of a method for stopping access to cold table data provided by an embodiment of the present application. Figure 6 As shown, the method may include:

[0100] It should be understood that Figure 6 Before the method of the illustrated embodiment is executed, the cold table (tn data) of the database included in the database instance 1 (Instance1) has been stored in the external storage (External Storage), and the metadata information (tn_ref) of the cold table is stored locally in Instance1, such as in an SSD. The metadata information of the cold table of the database included in the database instance 1 has been stored locally in the database instance 2 (Instance2), such as in an SSD.

[0101] Step 601: The SQL layer of database instance 2 (Instance 2) receives an instruction to unload data from a first cold table.

[0102] As a possible implementation, when Instance2 does not need to access the cold table data of Instance1, the SQL layer of Instance2 can receive an uninstallation instruction (e.g., Detach command) issued by the user. Exemplarily, the uninstallation instruction can be: UNMOUNT db1 / table_sample. It can also be understood as: separating the first cold table data in Instance1 from Instance2. It should be understood that the uninstallation instruction of the cold table data can be Figure 3 In the illustrated embodiment, after the user triggers the “unbind” button, the database management device 20 sends an uninstallation instruction to the database instance 200 .

[0103] Step 602: Database instance 2 (Instance 2) clears cache information of the first cold table data included in the local storage.

[0104] Since the metadata information of the first cold table data is stored locally in database instance 2 (Instance2), when Instance2 needs to access the first cold table data, it will use the metadata information of the first cold table data to access the first cold table data. Therefore, cache information of the first cold table data may be stored locally. At this time, the cache information of the first cold table data included in the local storage can be cleaned up (deleted).

[0105] Step 603: Database instance 2 (Instance 2) deletes metadata information of the first cold table stored locally.

[0106] In the embodiment of the present application, after the first cold table data is mounted to the database instance 2 (Instance2), the metadata information of the first cold table can be stored locally in the database instance 2 (Instance2). When the SQL layer of the database instance 2 (Instance2) issues a stop download instruction, the metadata information of the first cold table stored locally can be deleted, so that the subsequent database instance 2 (Instance2) can not access the cold data of the first cold table through the metadata information of the first cold table.

[0107] Step 604: Update the reference information (Ref meta) of the cold table data on the external storage (External Storage).

[0108] In some embodiments, after the metadata information of the first cold table data in the local storage of database instance 2 (Instance2) is separated, the reference information may be deleted on External Storage, such as deleting the reference of Instance2 to the first cold table data.

[0109] For example, continue with Figure 3 For example, when the user clicks the "Unbind" button, the database management device 20 may receive the uninstallation operation triggered by the user. After the database instance 200 receives the uninstallation instruction sent by the database management device 20, it may also send an update instruction to the external storage (External Storage). After the external storage (External Storage) receives the update instruction, it may delete the association information between the first cold table and the database instance 200.

[0110] Through the above embodiments, cold table data can be attached to other database instances to achieve cross-instance access to cold table data. In addition, when access to cold table data is not needed, cold table data can be separated on the attached database instance. Since the cold table data includes metadata information of the cold table data, and the database instance only stores metadata information, other data is still stored in the external storage, the cold table data can exist as independent data and can be accessed by other authorized and format-compatible database instances. The problem of binding database instances with cold table data in the prior art is solved, making the cold table data more flexible.

[0111] Further, in the embodiment of the present application, the reference information (Ref meta) of the cold table data can be stored on the cloud storage 30, and the reference information (Ref meta) is used to record the specific information of the cold table data in each database instance being referenced. It should be understood that the reference relationship can also be understood as association information. As a possible implementation method, the reference information can be in the form shown in the following schematic diagram:

[0112] (1) The reference information of each cold table is recorded by an object (or, can also be understood as: file). That is, there is a one-to-one relationship between the cold table and the object recording the reference information, and each object records the reference information of a cold table.

[0113] (2) The reference information of each cold table is recorded by three objects. That is, there is a one-to-many relationship between the cold table and the object recording the reference information. For example, each cold table can correspond to three types of objects: bind (REF), unbind (DEREF), and delete (DROP). These three types of objects are used to describe the situation in which each table is referenced by an external database instance.

[0114] (3) All cold tables share one object and record corresponding reference information respectively.

[0115] It should be understood that the form of the reference information includes but is not limited to the above forms, and this application does not make any specific limitations on this.

[0116] The second form is used as an example for description. Figure 7 As shown, it is a schematic diagram of a reference relationship in an external storage provided by an embodiment of the present application. The external storage 300 may include multiple cold table data, such as table 1 (table1), table 2 (table1) ... table X ((table1x), and a reference relationship record may be added to each cold table.

[0117] Among them, in the example of the second form, REF indicates marking a new reference relationship. A reference tag is added when migrating cold table data or appending cold table data. The naming method may be ref_timestamp; DEREF indicates marking a deletion of a reference relationship. A reference tag is added when separating cold table data or relocating cold table data. The naming method may be deref_timestamp; DROP indicates marking a deletion of a reference relationship. When the reference of the DROP-related tag is reduced to 0, the cold table data needs to be deleted, which may trigger a cold table data cleanup mechanism. The naming method may be drop_timestamp.

[0118] Taking table1 as an example, the tag information of the reference relationship of table1 can be: 1. Alter table migrates cold table data from local to external storage, and automatically adds reference records; 2. Attach a cold table to a new instance, and the table name is referenced by other instances, and automatically adds reference records; 3. The instance DROPs the cold table in external storage, adds and deletes reference records. If the add and delete reference tags are not equal at this time, the actual DROP operation is similar to the cold table separation operation, and the real data on the external storage will not be deleted; 4. The cold table data is migrated back to the hot table data, and the reference records are added and deleted. After the migration, the cold table data on the third-party storage will not be deleted. 5 and 6 represent the cold table separation operation, adding and deleting reference records. This operation will not delete the cold table data on the third-party storage, but will only clean up the metadata information of the cold table data local to the database instance. After separation, the database instance cannot access the cold table data.

[0119] If this is a record DROP type operation, the number of ref references in REF meta will be counted after the operation is completed, and it will be determined whether the number of ref references is equal to the number of drop+deref. If they are equal, the cold table data on the third-party storage will be cleaned up, including the reference relationship records. Otherwise, the cold table data on the third-party storage will not be cleaned up.

[0120] In the embodiment of the present application, the cleaning of cold table data can be triggered by the user, or the user can choose to let the third-party storage to clean up. For example, a control switch can be added to the user interface, and the user can choose on the interface whether the third-party storage automatically cleans up or the user cleans up according to the actual situation.

[0121] The present application also provides a cold data processing device 800, such as Figure 8 As shown, it includes: a transceiver module 801, an acquisition module 802, a storage module 803 and a processing module 804.

[0122] Exemplarily, the transceiver module 801 is used for the mount operation triggered by the user in the mount configuration interface of the cloud computing platform. The acquisition module 802 is used to obtain the metadata information of the cold table from the cloud storage according to the mount instruction. The storage module 803 is used to store the metadata information of the cold table; the processing module 804 is used to use the metadata information of the cold table to access the cold data of the cold table.

[0123] In the embodiment of the present application, the transceiver module 801, the acquisition module 802, the storage module 803, and the processing module 804 in the cold data processing device 800 can be implemented by software or by hardware. The following takes the processing module 804 as an example to introduce the implementation of the processing module 804. Similarly, the implementation of the transceiver module 801, the acquisition module 802, and the storage module 803 can refer to the implementation of the processing module 804.

[0124] The module may include code running on a computing instance. The processing module 804 may include code running on a computing instance. Among them, the computing instance may be at least one of computing devices such as a physical host (computing device), a virtual machine, and a container. Further, the above-mentioned computing instance may be one or more. For example, the processing module 804 may include code running on multiple hosts, or virtual machines, or containers. It should be noted that the multiple hosts, or virtual machines, or containers used to run the code may be distributed in the same region (region) or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple data centers with close geographical locations. Among them, usually a region may include multiple AZs.

[0125] Similarly, multiple hosts or virtual machines or containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Usually, a region can include multiple AZs. Usually, a VPC is set in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set in each VPC to realize the interconnection between VPCs through the communication gateway.

[0126] As an example of a hardware functional unit, the processing module 804 may include at least one computing device, such as a server, etc. Alternatively, the processing module 804 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0127] The multiple computing devices included in the processing module 804 can be distributed in the same AZ or in different AZs. The multiple computing devices included in the processing module 802 can be distributed in the same region or in different regions. Similarly, the multiple computing devices included in the processing module 804 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0128] It should be noted that, in other embodiments, the processing module 804 can be used to execute any step in the cold data processing method, and the transceiver module 801 can be used to execute any step in the cold data processing method. The steps that the transceiver module 801, the acquisition module 802, the storage module 803 and the processing module 804 are responsible for implementing can be specified as needed, and the transceiver module 801, the acquisition module 802, the storage module 803 and the processing module 804 respectively implement different steps in the cold data processing method to realize all functions of the cold data processing device.

[0129] The present application also provides a computing device 900. Fig. 9 As shown, the computing device 900 includes: a bus 902, a processor 904, a memory 906, and a communication interface 908. The processor 904, the memory 906, and the communication interface 908 communicate through the bus 902. The computing device 900 can be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in the computing device 900.

[0130] The bus 902 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig. 9 The bus 902 may include a path for transmitting information between various components of the computing device 900 (eg, the memory 906, the processor 904, and the communication interface 908).

[0131] The processor 904 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0132] The memory 906 may include a volatile memory, such as a random access memory (RAM). The memory 906 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0133] The memory 906 stores executable program codes, and the processor 804 executes the executable program codes to respectively implement the functions of the multiple modules in the cold meter data processing device, thereby implementing the cold meter data processing method. That is, the memory 906 stores instructions for executing the cold meter data processing method.

[0134] The communication interface 908 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 900 and other devices or communication networks.

[0135] The present application also provides a computing device cluster, Fig.10 is a schematic diagram of the computing device cluster, which can implement the cold table data processing method in the above embodiment. Fig.10 As shown, the computing device cluster includes at least one Fig. 9The computing device 900 shown may be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device may also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.

[0136] Instructions for executing the cold table data processing method may be stored in the memory 906 of one or more computing devices 900 in the computing device cluster. When at least one computing device in the computing device cluster executes the instruction, the computing device cluster may implement the cold data processing method as described in the embodiment.

[0137] In some possible implementations, the memory 906 of one or more computing devices 900 in the computing device cluster may also store partial instructions for executing the cold table data processing method. In other words, the combination of one or more computing devices 900 may jointly execute instructions for executing the cold data processing method.

[0138] It should be noted that the memory 906 in different computing devices 900 in the computing device cluster can store different instructions, which are respectively used to execute part of the functions of the cold table data processing device. That is, the instructions stored in the memory 906 in different computing devices 900 can realize the functions of one or more modules among the transceiver module 801, the acquisition module 802, the storage module 803 and the processing module 804 in the cold data processing device.

[0139] In some possible implementations, one or more computing devices in the computing device cluster may be connected via a network, which may be a wide area network or a local area network. Fig.11 A possible implementation is shown. Fig.11 As shown, two computing devices 900A and 900B are connected via a network. Specifically, the network is connected via a communication interface in each computing device. In this type of possible implementation, the memory 906 in the computing device 900A stores instructions for executing the functions of the processing module 804 in the cold data processing device. At the same time, the memory 906 in the computing device 900B stores instructions for executing the functions of the transceiver module 801, the acquisition module 802, and the storage module 803 in the cold data processing device.

[0140] It should be understood that Fig.11 The functions of the computing device 900A shown in FIG. 9A may also be completed by multiple computing devices 900. Similarly, the functions of the computing device 900B may also be completed by multiple computing devices 900.

[0141] The present application also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes the cold data processing method.

[0142] The present application also provides a computer-readable storage medium. The computer-readable storage medium may be any available medium that can be stored by a computing device or a data storage device such as a data center that includes one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct a computing device to execute a cold table data processing method, or instruct a computing device to execute a cold data processing method.

[0143] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims

1. A cold data processing method, characterized in that: A cloud computing platform provides a database system, a database management device and cloud storage, wherein the database system includes a first database instance and a second database instance, the database management device is used to manage the database system, the first database instance and the second database instance are different instances belonging to the same user under the cloud computing platform, the first database instance is used to store a cold table of the database in the cloud storage, and the cold table is used to store cold data of the database; the method comprises: The database management device receives a mount operation triggered by a user on a mount configuration interface of the cloud computing platform, wherein the mount configuration interface allows the user to select a database instance of the cold table in the cloud storage to be accessed, and the mount operation includes an identifier of the second database instance selected by the user; The database management device sends a mount instruction to the second database instance according to the mount operation, wherein the mount instruction carries an identifier of the cold table; The second database instance obtains metadata information of the cold table from the cloud storage according to the mounting instruction; The second database instance stores metadata information of the cold table; The second database instance uses the metadata information of the cold table to access the cold data of the cold table.

2. The method according to claim 1, characterized in that The first database instance and the second database instance are deployed on the same computing node or on different computing nodes.

3. The method according to claim 1 or 2, characterized in that The method further comprises: The second database instance sends a first update instruction to the cloud storage, where the first update instruction is used to instruct the cloud storage to add a mounting relationship between the cold table of the database and the second database instance.

4. The method according to any one of claims 1 to 3, characterized in that The method further comprises: The database management device receives an uninstallation operation triggered by a user on an uninstallation configuration interface of the cloud computing platform, wherein the uninstallation configuration interface allows the user to select a database instance to be uninstalled, and the uninstallation operation includes an identifier of the second database instance selected by the user; The database management device sends an uninstall instruction to the second database instance according to the uninstall operation, wherein the uninstall instruction carries an identifier of the cold table; The second database instance deletes the metadata information of the cold table from the local storage according to the uninstall instruction.

5. The method according to claim 4, characterized in that The method further comprises: The second database instance deletes cache information of the cold table from the local cache.

6. The method according to claim 4 or 5, characterized in that The method further comprises: The second database instance sends a second update instruction to the cloud storage, where the second update instruction is used to instruct the cloud storage to delete the mounting relationship between the cold table of the database and the second database instance.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: The database management device receives a deletion operation triggered by a user on a deletion configuration interface of the cloud computing platform, wherein the deletion configuration interface allows the user to select cold data of a cold table to be deleted, and the deletion operation includes an identifier of the cold table selected by the user; The second database instance sends a deletion instruction to the cloud storage, where the deletion instruction carries an identifier of the cold table; The cloud storage deletes the cold data of the cold table according to the deletion instruction.

8. A cold data processing device, characterized in that: The device comprises: A transceiver module, configured to receive a mount operation triggered by a user on a mount configuration interface of a cloud computing platform, wherein the mount configuration interface allows the user to select a database instance of a cold table in a cloud storage to be accessed, and the mount operation includes an identifier of a second database instance selected by the user; The transceiver module is further configured to send a mount instruction to the second database instance according to the mount operation, wherein the mount instruction carries an identifier of the cold table; An acquisition module, used for acquiring metadata information of the cold table from the cloud storage according to the mounting instruction; A storage module, used for storing metadata information of the cold table; The processing module is used to access the cold data of the cold table by using the metadata information of the cold table.

9. The device according to claim 8, characterized in that The transceiver module is further used to send a first update instruction to the cloud storage, where the first update instruction is used to instruct the cloud storage to add a mounting relationship between the cold table of the database and the second database instance.

10. The device according to claim 8 or 9, characterized in that The transceiver module is further configured to receive an uninstallation operation triggered by a user on an uninstallation configuration interface of the cloud computing platform, wherein the uninstallation configuration interface allows the user to select a database instance to be uninstalled, and the uninstallation operation includes an identifier of the second database instance selected by the user; and according to the uninstallation operation, send an uninstallation instruction to the second database instance, wherein the uninstallation instruction carries the identifier of the cold table; The processing module is further configured to delete the metadata information of the cold table from the local storage according to the uninstall instruction.

11. The device according to claim 10, characterized in that The processing module is further used to delete the cache information of the cold table from the local cache.

12. The device according to claim 10 or 11, characterized in that The transceiver module is further used to send a second update instruction to the cloud storage, where the second update instruction is used to instruct the cloud storage to delete the mounting relationship between the cold table of the database and the second database instance.

13. The device according to any one of claims 8 to 12, characterized in that: The transceiver module is further used to receive a deletion operation triggered by a user on a deletion configuration interface of the cloud computing platform, wherein the deletion configuration interface allows the user to select cold data of a cold table to be deleted, and the deletion operation includes an identifier of the cold table selected by the user; and send a deletion instruction to the cloud storage, wherein the deletion instruction carries the identifier of the cold table; The processing module is further used to delete the cold data in the cold table according to the deletion instruction.

14. A computing device cluster, characterized in that: comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method according to any one of claims 1 to 7.

15. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device cluster, the computing device cluster executes the method according to any one of claims 1 to 7.

16. A computer-readable storage medium, characterized in that: The method comprises computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster performs the method according to any one of claims 1 to 7.

Images