Method, device and product for releasing disk space occupied by invalid data in database

Abstract

The invention provides a method, a device and a product for releasing disk space occupied by invalid data in a database. The method comprises the following steps of: continuously arranging undeleted data records of the same physical table of a data table, placing the undeleted data records into a disk again, and writing the undeleted data records into a plurality of storage units of the physical table; re-establishing a hash index; switching the data table into a new effective version, and deleting the data record marked as deleted; the invention provides the method, the device and the product for releasing the disk space occupied by the invalid data in the database, the disk space occupied by the invalid data in the database can be quickly released, and the method, the device and the product have the characteristics of small calculation amount, high reliability and good effect.

Description

Technical Field

[0001] The invention belongs to the field of databases, and in particular relates to a method, a device and a product for releasing disk space occupied by invalid data in a database. Background Art

[0002] Some database clusters do not release disk space after deleting data, but instead mark the data as deleted. The occupied disk space will not be reused by newly added data. In this case, long-term and large-scale updates of business data will gradually increase invalid data and reduce disk utilization. In particular, when the partition table includes multiple physical tables, it occupies more disk space, which greatly affects the use of the database. Summary of the invention

[0003] In view of this, the present invention aims to overcome the defects in the prior art and proposes a method, device and product for releasing disk space occupied by invalid data in a database.

[0004] To achieve the above object, the technical solution of the present invention is achieved as follows: In a first aspect, the present invention discloses a method for releasing disk space occupied by invalid data in a database, comprising: According to the deletion mark of the data record in the current valid version data table, the undeleted data records of the same physical table of the data table are continuously arranged and re-written to the disk into several storage units of the corresponding physical table; Based on a number of storage units, a hash index is re-established according to the key values ​​of the stored data records; Switch the data table to the new valid version and delete the data records marked for deletion.

[0005] In one embodiment of the present invention, the deletion mark of the data record in the current valid version data table includes: the deletion mark is stored in the record header information of each data record.

[0006] In one embodiment of the present invention, the undeleted data records of the same physical table of the data table are continuously arranged and rewritten to the disk into several storage units of the corresponding physical table, including: the several storage units are storage units with consecutive positions.

[0007] In one embodiment of the present invention, the continuous arrangement is to arrange the data records one by one in the order of their original storage units without deletion.

[0008] In one embodiment of the present invention, the record header information includes a field for marking whether the data record has been deleted.

[0009] In one embodiment of the present invention, the field value of the field uses a binary character to indicate whether the data record has been deleted.

[0010] In a second aspect, the present invention discloses a device for releasing disk space occupied by invalid data in a database, the device comprising: The recycling module is used to continuously arrange the undeleted data records of the same physical table of the data table and re-write them into several storage units of the corresponding physical table according to the deletion marks of the data records in the current valid version data table; Establishing a hash index module, used to re-establish a hash index based on a number of storage units and according to the key values ​​of the stored data records; The switch effective module is used to switch the data table to a new effective version and delete the data records that have been marked for deletion.

[0011] In a third aspect, the present invention discloses an electronic device, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the above method.

[0012] In a fourth aspect, the present invention discloses a computer-readable storage medium having a computer program stored thereon, which implements the above method when executed by a processor.

[0013] In a fifth aspect, the present invention discloses a computer program product, including a computer program, which implements the above method when executed by a processor.

[0014] Compared with the prior art, the present invention has the following advantages: The present invention discloses a method, device and product for releasing disk space occupied by invalid data in a database, comprising: continuously arranging the undeleted data records of the same physical table of a data table and re-writing them into a plurality of storage units of the corresponding physical table; re-establishing a hash index based on the key values ​​of the stored data records based on the plurality of storage units; switching the data table to a new valid version and deleting the data records marked as deleted. The present invention discloses a method, device and product for releasing disk space occupied by invalid data in a database, which can quickly release the disk space occupied by invalid data in a database and has the characteristics of small calculation amount, high reliability and good effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0016] In the attached picture: Figure 1A schematic diagram of a method for releasing disk space occupied by invalid data in a database according to an embodiment of the present invention; Figure 2 A schematic diagram of disk space recovery of a method for releasing disk space occupied by invalid data in a database according to an embodiment of the present invention; Figure 3 A schematic diagram of obtaining a deletion mark in a method for releasing disk space occupied by invalid data in a database according to an embodiment of the present invention; Figure 4 A schematic diagram of a device for releasing disk space occupied by invalid data in a database according to an embodiment of the present invention; Figure 5 The present invention is a schematic diagram of an electronic device for releasing disk space occupied by invalid data in a database according to an embodiment of the present invention. DETAILED DESCRIPTION

[0017] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0018] In the description of the present invention, it should be further explained that the terms "first", "second", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first", "second", etc. may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "plurality" means two or more.

[0019] Some database clusters do not release disk space after deleting data, but instead mark the data as deleted. The occupied disk space will not be reused by newly added data. In this case, long-term and large-scale updates of business data will gradually increase invalid data and reduce disk utilization. In particular, partition tables include multiple physical tables, which occupy more disk space and greatly affect the use of the database.

[0020] The present invention discloses a method, device and product for releasing disk space occupied by invalid data in a database, which comprises the following steps: continuously arranging the undeleted data records of the same physical table of a data table and re-writing them into a plurality of storage units of the corresponding physical table; re-establishing a hash index; switching the data table to a new valid version and deleting the data records marked as deleted; the present invention can quickly release the disk space occupied by invalid data in a database and has the characteristics of small calculation amount, high reliability and good effect.

[0021] The present invention will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0022] In one embodiment of the present invention, Figure 1A method for releasing disk space occupied by invalid data in a database is shown, the method comprising: Step S101, according to the deletion mark of the data record in the current valid version data table, the undeleted data records of the same physical table of the data table are continuously arranged and re-written to the disk into several storage units of the corresponding physical table; In this embodiment, when the data table is a partition table, it will include multiple physical tables, and each physical table is traversed to reclaim disk space.

[0023] In this embodiment, the storage unit is used to store a maximum of several data records.

[0024] Step S102, based on a plurality of storage units, re-establishing a hash index according to the key values ​​of the stored data records; The underlying layer of hash index is hash table. When executing query, the hash function is called to obtain the corresponding key value, and then the table query is performed to obtain the actual data. The key value finds the bucket through hash mapping. Bucket is a storage unit that can store one or more data records. The structure of a bucket contains a memory pointer array. Each row of data in the bucket points to the next row, forming a linked list structure.

[0025] After the data is restored to disk, some fields are changed, so the hash index is rebuilt based on the key value.

[0026] Step S103, switching the data table to a new valid version, and deleting the data records marked for deletion.

[0027] In this embodiment, exemplarily, the data table is stored under table.info.A and table.info.B, both of which are read-write versions of the data table. After the write operation is completed on the current valid version of the data table, the data table changes, and the current valid version of the data table is switched to make the operation effective. For example, the current valid version before the disk space release operation is table.info.B, and the current valid version before and after the disk space release operation is table.info.A.

[0028] In this embodiment, data records that have been marked for deletion are deleted to release the disk space occupied by old data.

[0029] In another embodiment of the present invention, Figure 3 As shown, according to the deletion mark of the data record in the current valid version data table, the deletion mark is stored in the record header information of each data record. The record header information includes a field for marking whether the data record has been deleted. The field value of the field uses a binary character to indicate whether the data record has been deleted.

[0030] In this embodiment, if Figure 3 As shown, the delete_mask field is used to mark whether the data record has been deleted. When the delete_mask field value is 1, it means that the data record has been deleted. When the delete_mask field value is 0, it means that the data record has not been deleted.

[0031] In another embodiment of the present invention, Figure 2 As shown, the undeleted data records of the same physical table of the data table are continuously arranged and rewritten to the disk into several storage units of the corresponding physical table, including: the several storage units are storage units with consecutive positions. The continuous arrangement is to arrange the undeleted data records one by one in the position order of the original storage units.

[0032] like Figure 2 As shown, the left column represents the situation where data records are stored in the form of storage units before the disk space is released, and the right column represents the situation where data records are stored in the form of storage units after the disk space is released.

[0033] Among them, before the disk space release operation is performed, the storage units of the data records in the current valid version data table include storage units in which data records have not been deleted, storage units in which data records have been partially deleted, and storage units in which data records have been completely deleted; for example, storage unit 1 in the left column is a storage unit in which data records have not been deleted, and after the disk space is released, it is still in the position of storage unit 1, and storage unit 2 in the left column is a storage unit in which data records have been partially deleted. After the disk space is released, all the non-deleted data records therein are stored in the position of storage unit 2, but because the non-deleted data records contained are less than one storage unit, a part of the non-deleted data records from storage unit 3 in the left column are added to the position of storage unit 2 to fill up storage unit 2, and the remaining non-deleted data records are stored in the position of storage unit 3, and so on. The non-deleted data records are rearranged and written to several storage units of the physical table to which they belong, and the storage units are filled one by one in order of position. When storage unit 7 in the left column is a storage unit in which data records have been completely deleted, all the data records in storage unit 7 in the left column are discarded and skipped, and the above-mentioned storage method is continued to release the disk space.

[0034] In this embodiment, the original 11 storage units are changed into 8 storage units after releasing the disk space. The disk space release effect is obvious, the calculation amount is small, and the operation is convenient.

[0035] like Figure 4 As shown, the present invention also discloses a device for releasing disk space occupied by invalid data in a database, comprising: The recovery module 401 is used to continuously arrange the undeleted data records of the same physical table of the data table and rewrite them into several storage units of the corresponding physical table according to the deletion marks of the data records in the current valid version data table; A hash index building module 402 is used to re-build a hash index based on a plurality of storage units and according to key values ​​of stored data records; The switching validation module 403 is used to switch the data table to a new valid version and delete the data records that have been marked as deleted.

[0036] The present invention also discloses an electronic device, such as Figure 5 As shown, an embodiment is disclosed, which is a block diagram of an electronic device suitable for releasing disk space occupied by invalid data in a database.

[0037] The electronic device 50 of this embodiment includes a processor 501, which can perform various appropriate actions and processes according to the program stored in the ROM 502 or the program loaded from the storage part 508 to the RAM 503. The processor 501 may include, for example, a general-purpose microprocessor, an instruction set processor and / or a related chipset and / or a dedicated microprocessor, etc. The processor 501 may also include an onboard memory for caching purposes. The processor 501 may include a single processing unit or multiple processing units for performing different actions of the method flow according to the embodiment of the present invention.

[0038] In RAM503, various programs and data required for the operation of electronic device 50 are stored. Processor 501, ROM502 and RAM503 are connected to each other via bus 504, and processor 501 performs various operations of the method flow according to the embodiment of the present invention by executing the program in ROM502 and / or RAM503. It should be noted that the program can also be stored in one or more memories other than ROM502 and RAM503, and processor 501 can also perform various operations of the method flow according to the embodiment of the present invention by executing the program stored in one or more memories.

[0039] According to an embodiment of the present invention, the electronic device 50 may further include an I / O interface 505, which is also connected to the bus 504. The electronic device 50 may further include one or more of the following components connected to the I / O interface 505: an input portion 506 including a keyboard, a mouse, etc.; an output portion 507 including a cathode ray tube, a liquid crystal display, and a speaker; a storage portion 508 including a hard disk, etc.; and a communication portion 509 including a network interface card such as a LAN card, a modem, etc. The communication portion 509 performs communication processing via a network such as the Internet. A drive 5010 is also connected to the I / O interface 505 as needed. A removable medium 5011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 5010 as needed, so that a computer program read therefrom is installed into the storage portion 508 as needed.

[0040] The present invention also provides a computer-readable storage medium.

[0041] The computer-readable storage medium may be included in the electronic device / device system described in the above embodiment; or it may exist independently without being assembled into the electronic device / device. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0042] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory RAM, a read-only memory ROM, an erasable programmable read-only memory EPROM or a flash memory, a portable compact disk read-only memory CD-ROM, an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, apparatus, or device.

[0043] Embodiments of the present invention also include a computer program product.

[0044] The computer program product includes a computer program, which contains program codes for executing the method provided by the embodiment of the present invention. When the computer program product runs on an electronic device, the program codes are used to enable the electronic device to implement the method provided by the embodiment of the present invention.

[0045] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of a signal on a network medium. The program code included in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0046] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written by any combination of one or more programming languages, and specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages. Programming languages ​​include but are not limited to programming languages ​​such as Java, C++, python, C language or similar. The program code can be executed completely on the user computing device, partially on the user device, partially on the remote computing device, or completely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network or a wide area network, or can be connected to an external computing device.

[0047] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. It can be understood by those skilled in the art that the features recorded in the various embodiments and / or claims of the present invention can be combined and / or combined in various ways, even if such a combination or combination is not explicitly recorded in the present invention. In particular, without departing from the spirit and teaching of the present invention, the features described in the various embodiments and / or claims of the present invention may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present invention.

[0048] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although the embodiments are described above, this does not mean that the measures in the various embodiments cannot be used in combination. The scope of the present invention is limited by the attached claims and their equivalents. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A method for releasing disk space occupied by invalid data in a database, characterized in that: The method comprises: According to the deletion marks of the data records in the current valid version data table, the undeleted data records in the same physical table of the data table are continuously arranged and re-written to the disk into several storage units belonging to the physical table; Based on the plurality of storage units, and according to the key values ​​of the stored data records, re-establishing a hash index; The data table is switched to a new valid version, and the data records marked for deletion are deleted.

2. A method for releasing disk space occupied by invalid data in a database according to claim 1, characterized in that: The deletion mark of the data record in the current valid version data table includes: the deletion mark is stored in the record header information of each data record.

3. A method for releasing disk space occupied by invalid data in a database according to claim 1, characterized in that: The step of continuously arranging and re-writing the undeleted data records of the same physical table of the data table into a plurality of storage units belonging to the physical table includes: the plurality of storage units are storage units with consecutive positions.

4. A method for releasing disk space occupied by invalid data in a database according to claim 3, characterized in that: The continuous arrangement is arranged one by one in the order of the positions of the original storage units of the data records that have not been deleted.

5. A method for releasing disk space occupied by invalid data in a database according to claim 2, characterized in that: The record header information includes a field for marking whether the data record has been deleted.

6. A method for releasing disk space occupied by invalid data in a database according to claim 5, characterized in that: The field value of the field uses a binary character to indicate whether the data record has been deleted.

7. A device for releasing disk space occupied by invalid data in a database, characterized by: The device comprises: A recycling module is used to continuously arrange and rewrite the undeleted data records in the same physical table of the data table into a plurality of storage units belonging to the physical table according to the deletion marks of the data records in the current valid version data table; Establishing a hash index module, used to re-establish a hash index based on the plurality of storage units and according to the key values ​​of the stored data records; The switching validation module is used to switch the data table to a new valid version and delete the data records marked for deletion.

8. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors are caused to perform the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images