File management method and device, computer equipment and computer readable storage medium

Abstract

The embodiment of the invention discloses a file management method and device, computer equipment and a computer readable storage medium. The method comprises the steps of querying identification information of a target file stored in a preset first storage bucket in a preset distributed cache database by adopting a bloom filter; wherein the storage bucket is a container for storing files; if the identification information of the target file does not exist in the distributed cache database, deleting the target file stored in the first storage bucket; if the identification information of the target file exists in the distributed cache database, migrating the target file from the first storage bucket to a second storage bucket matched with the service type of the target file, so that useless files in the storage buckets can be quickly deleted; and the files in the storage bucket can be quickly migrated, so that the file management efficiency and convenience are improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a file management method, device, computer equipment and computer-readable storage medium. Background Art

[0002] Buckets are the carriers of objects, which can be understood as containers for storing objects, and there is no upper limit on the capacity of the container. Objects are stored in buckets in a flat structure, without the concept of folders and directories. Users can choose to store objects in one or more buckets.

[0003] During the development of business systems, business files on multiple business lines usually share one storage bucket due to the small data volume in the initial stage of the business. However, with the rapid growth of market business, if business files on multiple business lines share one storage bucket, it is not only impossible to separately count the storage costs of each business line, but there will also be a large number of useless files in the storage bucket, making subsequent operations more difficult and not conducive to the expansion of subsequent business. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention provides a file management method, apparatus, computer device and computer-readable storage medium, which can not only quickly delete useless files in a storage bucket, but also quickly migrate files in the storage bucket, thereby improving the efficiency and convenience of file management.

[0005] To solve the above problems, in a first aspect, an embodiment of the present invention provides a file management method, which includes:

[0006] In a preset distributed cache database, a Bloom filter is used to query identification information of a target file stored in a preset first storage bucket; wherein the storage bucket is a container for storing files;

[0007] If the identification information of the target file does not exist in the distributed cache database, deleting the target file stored in the first storage bucket;

[0008] If the identification information of the target file exists in the distributed cache database, the target file is migrated from the first storage bucket to a second storage bucket that matches the business type of the target file.

[0009] Furthermore, in the file management method, before using the Bloom filter to query the identification information of the target file stored in the preset first storage bucket in the distributed cache database, it also includes:

[0010] Acquire the business data of the target file from the main database;

[0011] The business data is cached in the distributed cache database, and an expiration time of the business data is set.

[0012] Furthermore, in the file management method, caching the business data in the distributed cache database includes:

[0013] Performing hash processing on the file name of the target file to obtain identification information of the target file;

[0014] The identification information and the business data are cached in the distributed cache database in the form of a key-value pair, and the value of the bit where the identification information is located is set to a preset value.

[0015] Furthermore, in the file management method, before using the Bloom filter to query the identification information of the target file stored in the preset first storage bucket in the preset distributed cache database, it also includes:

[0016] Determine the version number of the distributed cache database;

[0017] The Bloom filter is constructed in the distributed cache database according to the version number.

[0018] Furthermore, in the file management method, the step of using a Bloom filter in a preset distributed cache database to query identification information of a target file stored in a preset first storage bucket includes:

[0019] The value of the bit where the identification information is located is obtained to determine whether the identification information of the target file exists in the distributed cache database.

[0020] Furthermore, in the file management method, the method further includes:

[0021] Receiving a read request for the target file;

[0022] querying the target file in the second storage bucket according to the read request;

[0023] If the target file does not exist in the second storage bucket, the target file is searched in the first storage bucket.

[0024] Furthermore, in the file management method, the method further includes:

[0025] Receive storage request for newly added files;

[0026] Obtaining the business type of the newly added file;

[0027] The newly added file has the same business type as the target file, and the newly added file is stored in the second storage bucket;

[0028] If the business type of the newly added file is different from that of the target file, the newly added file is stored in a third storage bucket that matches the business type of the newly added file.

[0029] In a second aspect, an embodiment of the present invention further provides a file management device, which includes:

[0030] A first query unit, configured to query identification information of a target file stored in a preset first storage bucket using a Bloom filter in a preset distributed cache database; wherein the storage bucket is a container for storing files;

[0031] a deleting unit, configured to delete the target file stored in the first storage bucket if the identification information of the target file does not exist in the distributed cache database;

[0032] A migration unit is configured to migrate the target file from the first storage bucket to a second storage bucket matching the business type of the target file if the identification information of the target file exists in the distributed cache database.

[0033] In a third aspect, an embodiment of the present invention further provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the file management method as described in the first aspect above when executing the computer program.

[0034] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor executes the file management method described in the first aspect above.

[0035] The file management method, apparatus, computer device and computer-readable storage medium provided by the embodiments of the present invention pre-set a Bloom filter in a distributed cache database. When business files are stored in a storage bucket, the identification information of the business files will also be cached in the distributed cache database. When the storage bucket and the business system need to be optimized and expanded, it is only necessary to query the identification information of the corresponding business file through the Bloom filter to determine whether the identification information exists in the distributed cache database, and then it can be determined whether the business file is a useless file. If it is a useless file, it can be deleted in the storage bucket in time, thereby reducing the storage burden of the storage bucket; if it is not a useless file, the business file can be promptly migrated to a storage bucket that matches the business type of the business file, thereby enhancing the scalability of the business system. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0037] Figure 1 A flowchart of a file management method provided by an embodiment of the present invention;

[0038] Figure 2 A schematic diagram of a process flow of a file management method provided by an embodiment of the present invention;

[0039] Figure 3 Another schematic diagram of a process flow of a file management method provided by an embodiment of the present invention;

[0040] Figure 4 Another schematic diagram of a process flow of a file management method provided by an embodiment of the present invention;

[0041] Figure 5 Another schematic diagram of a process flow of a file management method provided by an embodiment of the present invention;

[0042] Figure 6 Another schematic diagram of a process flow of a file management method provided by an embodiment of the present invention;

[0043] Figure 7 A schematic block diagram of a file management device provided by an embodiment of the present invention;

[0044] Figure 8 A schematic block diagram of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0047] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0048] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0049] The file management method of the embodiment of the present invention is applied to a terminal device, and the method is executed by an application installed in the terminal device. The terminal device can be an electronic device such as a desktop computer, a laptop computer, a tablet computer or a mobile phone.

[0050] It should be noted that the application scenario of the above embodiment is only an example. The services and scenarios described in the embodiment of the present invention are intended to more clearly illustrate the technical solution of the embodiment of the present application, and do not constitute a limitation on the technical solution provided by the embodiment of the present application. It is known to those skilled in the art that with the evolution of the system and the emergence of new business scenarios, the technical solution provided by the embodiment of the present application is also applicable to similar technical problems. The following is a detailed description.

[0051] It should be noted that the description order of the following embodiments is not intended to limit the preferred order of the embodiments. The file management method is described in detail below.

[0052] See also Figure 1 , Figure 1 A flowchart of a file management method provided by an embodiment of the present invention.

[0053] It should be noted that if there are substantially the same results, the method of the present invention is not limited to Figure 2 The process sequence shown is limited to the following steps:

[0054] like Figure 1 As shown, the method includes the following steps S110 to S130.

[0055] S110. Using a Bloom filter in a preset distributed cache database to query identification information of a target file stored in a preset first storage bucket; wherein the storage bucket is a container for storing files.

[0056] It should be noted that the distributed cache database mentioned in this application adopts the Redis database, and it can also adopt other types of distributed key-value pair databases, which can be selected according to the actual application, and this application does not make specific limitations.

[0057] Among them, the Redis database is an open-source log-type, Key-Value database written in ANSI C language, supporting the network, and can be based on memory and can be persistent. That is, it is a non-relational database that can provide APIs in multiple languages ​​and can be used as a database, cache, message middleware, etc. The Redis database not only supports data persistence, but also can save the data in memory to the disk, and can be loaded again for use when restarting, and it also provides storage for data structures such as List, Set, Zset, and Hash.

[0058] Bloom Filter is a data structure used to quickly check whether an element belongs to a set. The main principle of Bloom filter is to use a set of hash functions to map elements to index positions in a set of bit arrays. When checking whether an element is in a set, hash the element and then check whether the value of the bit array corresponding to the hash value is 1. If the values ​​of the bit arrays corresponding to the hash values ​​are all 1, then the element may be in the set, otherwise the element is definitely not in the set.

[0059] Specifically, the first storage bucket is a container for storing files of multiple business lines in the initial stage of the business system, and it has various configuration attributes, including region, access rights, storage type, etc. When the files in the first storage bucket need to be migrated, the Bloom filter can be used to query the Redis database whether there is identification information of the corresponding file in the first storage bucket, that is, to query whether the value of the bit where the hash value of the identification information of the file is located is a preset value, and then determine whether the file is a useless file or a file to be migrated, so that the useless files in the first storage bucket can be quickly deleted, and the files in the first storage bucket can be quickly migrated to the corresponding storage bucket, which improves the efficiency and convenience of file management, and thus improves the scalability of the business system.

[0060] In a specific embodiment, a Bloom filter is used in a preset distributed cache database to query the identification information of a target file stored in a preset first storage bucket, specifically including: obtaining the value of the bit where the identification information of the target file is located to determine whether the identification information of the target file exists in the distributed cache database.

[0061] In this embodiment, the identification information can be the file name of the target file. Since the business data of the target file is also cached in Redis, the business data is cached in the Redis database in the form of a key-value pair, wherein the file name of the target file is processed by a hash algorithm (such as an MD5 algorithm), and the generated hash value is directly used as the Key value, thereby realizing that the business data is cached in the Redis database in the form of a key-value pair. Therefore, after obtaining the file name of the target file, the file name can be processed by a hash algorithm (such as an MD5 algorithm) to obtain a hash value of the file name, which is the Key value in the Key-Value (key-value pair), and then the hash value can be queried in the Redis database through the Bloom filter, thereby determining whether the value of the Bit position where the hash value is located in the Redis database is a preset value, that is, determining whether there is identification information of the target file in the Redis database, thereby deciding whether to migrate the target file from the first storage bucket or delete it from the first storage bucket.

[0062] Among them, the MD5 algorithm is a widely used cryptographic hash function that can generate a 128-bit (16-byte) hash value (hash value), which is used to ensure that the information transmission is complete and consistent. After any message is processed by the hash function, a unique hash value will be generated, and this hash value can be used to verify the integrity of the message. In other words, after the file name of the target file is processed by the MD5 algorithm, it can generate a unique hash value. After the hash value is cached in the Redis database, the value of the bit corresponding to the hash value in the Redis database will be changed to a preset value, and then the preset value can be used to determine whether the identification information of the target file exists in the Redis database.

[0063] In other inventive embodiments, Figure 2 As shown, before step S110, steps S210 and S220 are also included.

[0064] S210, acquiring the business data of the target file from the main database;

[0065] S220: Cache the business data in the distributed cache database, and set an expiration time for the business data.

[0066] In this embodiment, the business data is the data generated during the user's consumption of the target file. After the user completes the consumption of the target file, the corresponding business data is stored in the master data. In addition, in order to facilitate the user to query the business data more conveniently and quickly in the future, the business data can also be cached in the Redis database after being stored in the master data, and the expiration time of the business data can be set in the Redis database to reduce the memory consumption of the Redis database and maintain the efficiency of the Redis database. Among them, the expiration time is the time to delete the business data in the Redis database after the preset time.

[0067] In other inventive embodiments, Figure 3 As shown, step S220 includes steps S221 and S222.

[0068] S221, performing hash processing on the file name of the target file to obtain identification information of the target file;

[0069] S222: Cache the identification information and the business data in the form of a key-value pair in the distributed cache database, and set the value of the bit where the identification information is located to a preset value.

[0070] Specifically, the business data of the target file is cached in the Redis database in the form of Key-Value (key-value pair). Among them, the Key value is used for the file name of the target file, and the Value value is used to characterize the attribute information of the business data, such as the time and place of consumption of the target file. The file name of the target file is hashed to map the file name of the target file to a hash value of a fixed length, and the hash value is directly used as the identification information of the target file. At the same time, in order to avoid hash conflicts after the file name of the target file is hashed, this application preferentially uses the MD5 algorithm to hash the file name of the target file.

[0071] The MD5 algorithm is a one-way encryption function that accepts messages of any length as input and returns a fixed-length digest value as output to verify the original message. The MD5 algorithm is an implementation of the hash algorithm. By processing the file name of the target file with the MD5 algorithm, hash conflicts in the business data in the Redis database can be reduced, thereby reducing the misjudgment rate of the subsequent Bloom filter.

[0072] In this embodiment, the business data of the target file includes the file name of the target file. After obtaining the business data of the target file, the file name of the target file can be processed using the MD5 algorithm to generate a hash value of the file name, that is, the Key value, which can also be called the identification information of the target file, and then the attribute information of the business data is used as the Value value and cached in the Redis database. At the same time, the value of the bit where the Key value is located needs to be set from 0 to 1.

[0073] In addition, after the business data of the target file expires in the Redis database, the business data needs to be deleted in the Redis database, and the value of the bit where the key value is located is set from 1 to 0.

[0074] In other inventive embodiments, Figure 4 As shown, before step S110, steps S310 and S320 are also included.

[0075] S310, determining a version number of the distributed cache database;

[0076] S320: Construct the Bloom filter in the distributed cache database according to the version number.

[0077] Specifically, due to the different ways of setting Bloom filters in different versions of Redis databases, Redis versions before 4.0 need to use Bitmap to implement Bloom filters, while Redis databases after 4.0 can directly use the official RedisBloom to implement it. Therefore, when you need to set a Bloom filter in Redis, you need to determine the version of the Redis database in advance, and then build a Bloom filter in the Redis database that matches the Redis database.

[0078] S120: If the identification information of the target file does not exist in the distributed cache database, delete the target file stored in the first storage bucket.

[0079] Specifically, when the identification information of the target file does not exist in the Redis database, that is, the business data of the target file has expired and been deleted in the Redis database, it can be determined that the user will not consume the target file in the future, that is, the target file is a useless file. At this time, the target file stored in the first storage bucket needs to be deleted to reduce the storage cost of the file.

[0080] S130: If the identification information of the target file exists in the distributed cache database, migrate the target file from the first storage bucket to a second storage bucket that matches the business type of the target file.

[0081] Specifically, when the identification information of the target file exists in the Redis database, that is to say, the business data of the target file has not expired in the Redis database, the target file is not a useless file. Since the first storage bucket is a storage bucket shared in the initial stage of the business, in order to facilitate the subsequent cost statistical analysis of the target file, it is necessary to migrate the target file to a storage bucket that matches the business type of the target file, that is, the second storage bucket. The migration of the target file can be achieved without downtime and user perception, thereby reducing the complexity of subsequent target file reading and achieving the purpose of reducing costs and increasing efficiency.

[0082] In other inventive embodiments, Figure 5 As shown, the method further includes steps S410, S420 and S430.

[0083] S410, receiving a read request for the target file;

[0084] S420, querying the target file in the second storage bucket according to the read request;

[0085] S430: If the target file does not exist in the second storage bucket, search the first storage bucket for the target file.

[0086] In this embodiment, the read request is instruction information for the business system to request to read the target file for user consumption. After the terminal device receives the read request for the target file, since the target file may have been migrated from the first storage bucket to the second storage bucket, in order to improve the reading efficiency of the target file, this application gives priority to querying whether the target file exists in the second storage bucket. If the target file exists in the second storage bucket, the target file can be directly read from the second storage bucket for user consumption; if the target file does not exist in the second storage bucket, it can be determined that the target file is still stored in the first storage bucket and has not yet been migrated from the first storage bucket to the second storage bucket. At this time, it is necessary to query the target file from the first storage bucket for user consumption.

[0087] In other inventive embodiments, Figure 6 As shown, the method further includes steps S510 to S540.

[0088] S510, receiving a storage request for a newly added file;

[0089] S520, obtaining the service type of the newly added file;

[0090] S530: The newly added file has the same business type as the target file, and the newly added file is stored in the second storage bucket;

[0091] S540: If the business type of the newly added file is different from that of the target file, the newly added file is stored in a third storage bucket that matches the business type of the newly added file.

[0092] In this embodiment, the storage request is instruction information for the business system to request object storage of the newly added file. The newly added file may be a file of the same business type as the target file, or may be a file of two different business types as the target file. After receiving the storage request for the newly added file, the terminal device needs to pre-acquire the business type to which the newly added file belongs, and determine whether the business type of the newly added file and the business type of the target file are the same business type. If they are the same business type, the newly added file can be directly stored in the second storage bucket. If the business type of the newly added file is different from the business type of the target file, the newly added file can be stored in a pre-set third storage bucket, avoiding continuing to store the newly added file in the first storage bucket, thereby facilitating the subsequent statistics on the file storage costs of each business line, and providing better scalability for subsequent business development.

[0093] It should also be noted that if there are only two storage buckets in this application, namely the first storage bucket and the second storage bucket, then when storing new files subsequently, there is no need to obtain the business type of the new files. After receiving the storage request for the new files, the new files can be directly stored in the second storage bucket.

[0094] It can also be understood that in a scenario with a small number of business lines, the second storage bucket can reduce the storage burden of the first storage bucket, and both the first storage bucket and the second storage bucket can store various types of business files, that is, the types of business files stored in the first storage bucket and the second storage bucket can be the same or different; in a scenario with a large number of business lines, the first storage bucket can be understood as a container for storing business files in the initial stage of the business, and the second storage bucket and the third storage can be understood as containers for storing two different business types, respectively, that is, the business types of the business files stored in the second storage bucket are the same, the business types of the business files stored in the third storage bucket are the same, and the business types of the business files stored in the second storage bucket and the business files stored in the third storage bucket are different.

[0095] In the file management method provided in the embodiment of the present invention, the identification information of the target file stored in the preset first storage bucket is queried by using a Bloom filter in the preset distributed cache database; wherein the storage bucket is a container for storing files; if the identification information of the target file does not exist in the distributed cache database, the target file stored in the first storage bucket is deleted; if the identification information of the target file exists in the distributed cache database, the target file is migrated from the first storage bucket to a second storage bucket that matches the business type of the target file. The present invention can not only quickly delete useless files in the storage bucket, reducing the storage cost of the files, but also quickly migrate the files in the storage bucket, improving the migration efficiency of the files, improving the efficiency and convenience of file management, and thus improving the scalability of the business system.

[0096] The embodiment of the present invention further provides a file management device 100, which is used to execute any embodiment of the aforementioned file management method.

[0097] Specifically, see Figure 7 , Figure 7 It is a schematic block diagram of a file management device 100 provided in an embodiment of the present invention.

[0098] like Figure 7 As shown, the file management device 100 includes: a first query unit 110, a deletion unit 120 and a migration unit 130.

[0099] The first query unit 110 is used to query the identification information of the target file stored in a preset first storage bucket in a preset distributed cache database by using a Bloom filter; wherein the storage bucket is a container for storing files.

[0100] In other embodiments of the invention, the file management device 100 further includes: a first acquisition unit and a first cache unit.

[0101] The first acquisition unit is used to acquire the business data of the target file from the main database; the first cache unit is used to cache the business data in the distributed cache database and set an expiration time of the business data.

[0102] In other inventive embodiments, the first cache unit includes: a processing unit and a second cache unit.

[0103] A processing unit, used to perform hash processing on the file name of the target file to obtain the identification information of the target file; a second cache unit, used to cache the identification information and the business data in the form of key-value pairs in the distributed cache database, and set the value of the bit where the identification information is located to a preset value.

[0104] In other embodiments of the invention, the file management device 100 further includes: a determination unit and a construction unit.

[0105] A determining unit is used to determine the version number of the distributed cache database; and a constructing unit is used to construct the Bloom filter in the Redis according to the version number.

[0106] The deleting unit 120 is configured to delete the target file stored in the first storage bucket if the identification information of the target file does not exist in the distributed cache database.

[0107] The migration unit 130 is configured to migrate the target file from the first storage bucket to a second storage bucket matching the business type of the target file if the identification information of the target file exists in the distributed cache database.

[0108] In other embodiments of the invention, the file management device 100 further includes: a first receiving unit, a second query unit and a third query unit.

[0109] The first receiving unit is used to receive a read request for the target file; the second query unit is used to query the target file in the second storage bucket according to the read request; and the third query unit is used to query the target file in the first storage bucket if the target file does not exist in the second storage bucket.

[0110] In other embodiments of the invention, the file management device 100 further includes: a second receiving unit, a second acquiring unit, a first storage unit and a second storage unit.

[0111] The second receiving unit is used to receive a storage request for a newly added file; the second obtaining unit is used to obtain the business type of the newly added file; the first storage unit is used to store the newly added file in the second storage bucket if the business type of the newly added file is the same as that of the target file; the second storage unit is used to store the newly added file in a third storage bucket that matches the business type of the newly added file if the business type of the newly added file is different from that of the target file.

[0112] The file management device provided in an embodiment of the present invention is used to execute the above-mentioned use of a Bloom filter in a preset distributed cache database to query the identification information of a target file stored in a preset first storage bucket; wherein the storage bucket is a container for storing files; if the identification information of the target file does not exist in the distributed cache database, the target file stored in the first storage bucket is deleted; if the identification information of the target file exists in the distributed cache database, the target file is migrated from the first storage bucket to a second storage bucket that matches the business type of the target file.

[0113] The above-mentioned file management device can be implemented in the form of a computer program. The computer program can be Figure 8 Runs on the computer device shown.

[0114] See also Figure 8 , Figure 8 5 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device 500 may be a terminal, wherein the terminal may be a computer device with communication functions such as a smart phone, a tablet computer, a laptop computer, a desktop computer, a personal digital assistant, and a wearable device.

[0115] See also Figure 8 The computer device 500 includes a processor 502 , a memory and a network interface 505 connected via a system bus 501 , wherein the memory may include a non-volatile storage medium 503 and an internal memory 504 .

[0116] The non-volatile storage medium 503 can store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions, and when the program instructions are executed, the processor 502 can execute a file management method.

[0117] The processor 502 is used to provide computing and control capabilities to support the operation of the entire computer device 500 .

[0118] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute a file management method.

[0119] The network interface 505 is used to communicate with other devices over the network. Figure 8 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device 500 to which the solution of the present application is applied. The specific computer device 500 may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0120] The processor 502 is used to run a computer program 5032 stored in a memory to implement the following steps: using a Bloom filter to query identification information of a target file stored in a preset first storage bucket in a preset distributed cache database; wherein a storage bucket is a container for storing files; if the identification information of the target file does not exist in the distributed cache database, deleting the target file stored in the first storage bucket; if the identification information of the target file exists in the distributed cache database, migrating the target file from the first storage bucket to a second storage bucket that matches the business type of the target file.

[0121] In one embodiment, before implementing the distributed cache database using a Bloom filter to query the identification information of the target file stored in a preset first storage bucket, the processor 502 specifically implements the following steps: obtaining the business data of the target file from the main database; caching the business data in the distributed cache database, and setting the expiration time of the business data.

[0122] In one embodiment, when the processor 502 implements the caching of the business data into the distributed cache database, it specifically implements the following steps: hashing the file name of the target file to obtain identification information of the target file; caching the identification information and the business data in the form of a key-value pair in the Redis, and setting the value of the bit where the identification information is located to a preset value.

[0123] In one embodiment, before implementing the use of a Bloom filter in a distributed cache database to query identification information of a target file stored in a preset first storage bucket, the processor 502 specifically implements the following steps: determining a version number of the distributed cache database; and constructing the Bloom filter in the distributed cache database according to the version number.

[0124] In one embodiment, when the processor 502 implements the use of a Bloom filter in a distributed cache database to query the identification information of a target file stored in a preset first storage bucket, it specifically implements the following steps: obtaining the value of the bit where the identification information is located to determine whether the identification information of the target file exists in the distributed cache database.

[0125] In one embodiment, the processor 502 is used to run a computer program 5032 stored in the memory, and also implements the following steps: receiving a read request for the target file; querying the target file in the second storage bucket according to the read request; if the target file does not exist in the second storage bucket, querying the target file in the first storage bucket.

[0126] In one embodiment, the processor 502 is used to run a computer program 5032 stored in the memory, and also implements the following steps: receiving a storage request for a newly added file; obtaining the business type of the newly added file; if the business type of the newly added file is the same as that of the target file, storing the newly added file in the second storage bucket; if the business type of the newly added file is different from that of the target file, storing the newly added file in a third storage bucket that matches the business type of the newly added file.

[0127] It should be understood that in the embodiment of the present application, the processor 502 may be a central processing unit (CPU), and the processor 502 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0128] It can be understood by those skilled in the art that all or part of the processes in the method for implementing the above embodiment can be completed by instructing the relevant hardware through a computer program. The computer program includes program instructions, and the computer program can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiment of the above method.

[0129] Therefore, the present invention also provides a storage medium. The storage medium can be a computer-readable storage medium. The storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed by a processor, the processor executes the following steps: using a Bloom filter to query the identification information of a target file stored in a preset first storage bucket in a preset distributed cache database; wherein the storage bucket is a container for storing files; if the identification information of the target file does not exist in the distributed cache database, the target file stored in the first storage bucket is deleted; if the identification information of the target file exists in the distributed cache database, the target file is migrated from the first storage bucket to a second storage bucket that matches the business type of the target file.

[0130] In one embodiment, before the processor executes the program instructions to use a Bloom filter to query the identification information of the target file stored in a preset first storage bucket in the distributed cache database, it specifically implements the following steps: obtaining the business data of the target file from the main database; caching the business data in the distributed cache database, and setting the expiration time of the business data.

[0131] In one embodiment, when the processor executes the program instructions to implement caching the business data into the distributed cache database, it specifically implements the following steps: hashing the file name of the target file to obtain identification information of the target file; caching the identification information and the business data in the form of a key-value pair into the distributed cache database, and setting the value of the bit where the identification information is located to a preset value.

[0132] In one embodiment, before the processor executes the program instructions to use a Bloom filter to query the identification information of the target file stored in a preset first storage bucket in the distributed cache database, it specifically implements the following steps: determining the version number of the distributed cache database; and constructing the Bloom filter in the distributed cache database according to the version number.

[0133] In one embodiment, when the processor executes the program instructions to implement the use of a Bloom filter in a distributed cache database to query the identification information of a target file stored in a preset first storage bucket, the processor specifically implements the following steps: obtaining the value of the bit where the identification information is located to determine whether the identification information of the target file exists in the distributed cache database.

[0134] In one embodiment, when executing the program instructions, the processor also implements the following steps: receiving a read request for the target file; querying the target file in the second storage bucket according to the read request; if the target file does not exist in the second storage bucket, querying the target file in the first storage bucket.

[0135] In one embodiment, when executing the program instructions, the processor also implements the following steps: receiving a storage request for a newly added file; obtaining the business type of the newly added file; if the business type of the newly added file is the same as that of the target file, storing the newly added file in the second storage bucket; if the business type of the newly added file is different from that of the target file, storing the newly added file in a third storage bucket that matches the business type of the newly added file.

[0136] The storage medium may be a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk, etc., which are computer-readable storage media that can store program codes.

[0137] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0138] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of each unit is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0139] The steps in the method of the embodiment of the present invention can be adjusted in order, combined and deleted according to actual needs. The units in the device of the embodiment of the present invention can be combined, divided and deleted according to actual needs. In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0140] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling an electronic device (which can be a personal computer, terminal, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention.

[0141] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A file management method, characterized in that: include: In a preset distributed cache database, a Bloom filter is used to query identification information of a target file stored in a preset first storage bucket; wherein the storage bucket is a container for storing files; If the identification information of the target file does not exist in the distributed cache database, deleting the target file stored in the first storage bucket; If the identification information of the target file exists in the distributed cache database, the target file is migrated from the first storage bucket to a second storage bucket that matches the business type of the target file.

2. The file management method according to claim 1, characterized in that: Before using the Bloom filter to query the identification information of the target file stored in the preset first storage bucket in the preset distributed cache database, the method further includes: Acquire the business data of the target file from the main database; The business data is cached in the distributed cache database, and an expiration time of the business data is set.

3. The file management method according to claim 2, characterized in that: The caching the business data into the distributed cache database includes: Performing hash processing on the file name of the target file to obtain identification information of the target file; The identification information and the business data are cached in the distributed cache database in the form of a key-value pair, and the value of the bit where the identification information is located is set to a preset value.

4. The file management method according to claim 1, characterized in that: Before using the Bloom filter in the distributed cache database to query the identification information of the target file stored in the preset first storage bucket, the method further includes: Determine the version number of the distributed cache database; The Bloom filter is constructed in the distributed cache database according to the version number.

5. The file management method according to claim 1, characterized in that: The method of using a Bloom filter in a preset distributed cache database to query identification information of a target file stored in a preset first storage bucket includes: The value of the bit where the identification information is located is obtained to determine whether the identification information of the target file exists in the distributed cache database.

6. The file management method according to claim 1, characterized in that: The method further comprises: Receiving a read request for the target file; querying the target file in the second storage bucket according to the read request; If the target file does not exist in the second storage bucket, the target file is searched in the first storage bucket.

7. The file management method according to claim 1, characterized in that: The method further comprises: Receive a storage request for a newly added file; Obtaining the business type of the newly added file; If the business type of the newly added file is the same as that of the target file, storing the newly added file in the second storage bucket; If the business type of the newly added file is different from that of the target file, the newly added file is stored in a third storage bucket that matches the business type of the newly added file.

8. A file management device, characterized in that: include: A first query unit, configured to query identification information of a target file stored in a preset first storage bucket using a Bloom filter in a preset distributed cache database; wherein the storage bucket is a container for storing files; a deleting unit, configured to delete the target file stored in the first storage bucket if the identification information of the target file does not exist in the distributed cache database; A migration unit is configured to migrate the target file from the first storage bucket to a second storage bucket matching the business type of the target file if the identification information of the target file exists in the distributed cache database.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the file management method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, causes the processor to execute the file management method according to any one of claims 1 to 7.

Images