A method, device, electronic device and storage medium for counting storage usage

Abstract

The present application discloses a storage usage statistics method, device, electronic device and storage medium, which relate to the field of computer technology, and in particular to distributed storage technology. The specific implementation scheme is: the client sends a storage usage query command to each storage node in the storage cluster; receives the storage usage of each file system on each storage node returned by each storage node in the storage cluster in response to the query command; based on the storage usage of each file system on each storage node, the total storage usage of each file system is counted. The embodiment of the present application can accurately query the storage usage of each file system in real time at any time, and has the advantages of less time consumption, accurate data without errors, and no impact on normal online services.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and further to distributed storage technology, and in particular to a storage usage statistics method, device, electronic device and storage medium. Background Art

[0002] The distributed file storage system may include: client, storage cluster, metadata management cluster and NewSQL database; among them, the client is compiled into the business process and provides the business with the relevant access interface of the file system. The storage cluster consists of multiple storage nodes, which are used to store file data. The metadata management cluster consists of multiple stateless master services, each of which is completely peer-to-peer and can be horizontally expanded. The NewSQL database is used to persistently store all metadata of the distributed file system.

[0003] Before using the distributed file storage system, users must first create a file system and then upload files to the file system. In order to bill users for the storage used, it is necessary to count the storage capacity of each file system. There are currently two main solutions for counting the storage capacity of each file system:

[0004] 1) Collect logs for statistics: Every time a user uploads or deletes a file, the distributed file storage system prints a billing log, which contains the name of the file system to which the file belongs, the length of the file uploaded or deleted this time, and other information. Deploy a log collection program to regularly collect billing logs for calculation and statistics, obtain the net increase in storage capacity of each file system over a period of time, and then accumulate the net increase to obtain the storage usage of each file system. The disadvantages of this solution are: 1. It is not very timely and cannot count the storage usage of the file system in real time; 2. When the disk or machine storing the billing log is damaged, part of the billing log will be lost, resulting in inaccurate storage statistics, and such errors will accumulate. Over time, the error will become larger and larger.

[0005] 2) Dump the database table for statistics: Because the NewSQL database table records the size information of each file and the file system to which the file belongs, you can dump the data table regularly and obtain the storage usage of each file system by analyzing and counting the database table information. The disadvantages of this solution are: 1. The timeliness is worse than Solution 1. Since the database table that stores file information has a huge amount of data, it takes a very long time to dump the database table and count the results, usually a week or even longer; 2. The database table dump operation may affect normal online services. Summary of the invention

[0006] The present disclosure provides a storage usage statistics method, device, electronic device and storage medium.

[0007] In a first aspect, the present application provides a storage usage statistics method, which is applied to a client, and the method includes:

[0008] Send a query command for storage usage to each storage node in the storage cluster;

[0009] Receiving storage usage of each file system on each storage node returned by each storage node in the storage cluster in response to the query command;

[0010] Based on the storage usage of each file system on each storage node, the total storage usage of each file system is counted.

[0011] In a second aspect, the present application further provides a storage usage statistics method, which is applied to a storage node, and the method includes:

[0012] Receive a storage usage query command sent by a client;

[0013] Responding to the query command, obtaining the storage usage of each file system on the storage node;

[0014] The storage usage of each file system on the storage node is returned to the client, so that the client counts the total storage usage of each file system based on the storage usage of each file system on each storage node.

[0015] In a third aspect, the present application provides a storage usage statistics device, the device comprising: a first sending module, a first receiving module and a statistics module; wherein:

[0016] The first sending module is used to send a query command of storage usage to each storage node in the storage cluster;

[0017] The first receiving module is used to receive the storage usage of each file system on each storage node returned by each storage node in the storage cluster in response to the query command;

[0018] The statistical module is used to count the total storage usage of each file system based on the storage usage of each file system on each storage node.

[0019] In a fourth aspect, the present application further provides a storage usage statistics device, the device comprising: a second receiving module, an acquisition module, and a second sending module; wherein:

[0020] The second receiving module is used to receive a query command for storage usage sent by a client;

[0021] The acquisition module is used to acquire the storage usage of each file system on the storage node in response to the query command;

[0022] The second sending module is used to return the storage usage of each file system on the storage node to the client, so that the client can count the total storage usage of each file system based on the storage usage of each file system on each storage node.

[0023] In a fifth aspect, an embodiment of the present application provides an electronic device, including:

[0024] one or more processors;

[0025] a memory for storing one or more programs,

[0026] When the one or more programs are executed by the one or more processors, the one or more processors implement the storage usage statistics method described in any embodiment of the present application.

[0027] In a sixth aspect, an embodiment of the present application provides a storage medium on which a computer program is stored, and when the program is executed by a processor, the method for counting storage usage described in any embodiment of the present application is implemented.

[0028] In a seventh aspect, a computer program product is provided, which, when executed by a computer device, implements the method for counting storage usage described in any embodiment of the present application.

[0029] The technology according to the present application solves the technical problems of low timeliness and inability to count the storage usage of the file system in real time brought about by the solution of collecting logs for statistics in the prior art, as well as the technical problems of poor ineffectiveness, long time consumption and possible impact on normal online services brought about by the solution of dumping database tables for statistics. The technical solution provided by the present application can accurately query the storage usage of each file system in real time at any time, and has the advantages of less time consumption, accurate and error-free data and no impact on normal online services.

[0030] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0032] Figure 1This is a first flow chart of the method for counting storage usage provided by an embodiment of the present application;

[0033] Figure 2 This is a typical architecture diagram of the file storage system provided in the embodiment of the present application;

[0034] Figure 3 It is a second flow chart of the method for counting storage usage provided in an embodiment of the present application;

[0035] Figure 4 It is a third flow chart of the method for counting storage usage provided in an embodiment of the present application;

[0036] Figure 5 This is a fourth flow chart of the method for counting storage usage provided in an embodiment of the present application;

[0037] Figure 6 It is a first structural schematic diagram of a storage usage statistics device provided in an embodiment of the present application;

[0038] Figure 7 This is a second structural diagram of the storage usage statistics device provided in an embodiment of the present application.

[0039] Figure 8 It is a block diagram of an electronic device used to implement the storage usage statistics method of an embodiment of the present application. DETAILED DESCRIPTION

[0040] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0041] Embodiment 1

[0042] Figure 1 1 is a schematic diagram of the first flow chart of the storage usage statistics method provided in the embodiment of the present application. The method can be executed by a storage usage statistics device or client. The device or client can be implemented by software and / or hardware. The device or client can be integrated in any smart device with network communication function. Figure 1 As shown, the method for counting storage usage may include the following steps:

[0043] S101. Send a storage usage query command to each storage node in the storage cluster.

[0044] In this step, the client can send a query command for storage usage to each storage node in the storage cluster. Specifically, the client can send a query command for storage usage to each storage node in the storage cluster at any time; or send a query command for storage usage to each storage node in the storage cluster according to a predetermined period; this application does not limit this.

[0045] Figure 2 This is a typical architecture diagram of the file storage system provided in the embodiment of the present application. Figure 1 As shown in the figure, the distributed file system consists of the following parts: 1) Client: This is a library, compiled into the business process, providing the business with the relevant access interface of the file system, such as: creating folders or files, listing all subdirectories or subfiles under the directory, reading file data, writing data to files, etc. 2) Storage cluster: The storage cluster consists of multiple storage nodes, which are used to store file data. The data of each file is horizontally cut into multiple file blocks, and each storage node stores multiple file blocks. Each file block has a globally unique block ID to identify it; 3) Metadata management cluster: The metadata management cluster consists of multiple stateless master services, each of which is completely equal and can be expanded horizontally. The main function of the master service is to manage the directory tree, folder or file information, file block information, and file block to storage node mapping information. The master does not persist these metadata locally, and all metadata are persisted in the NewSQL database. 4) NewSQL database: persists all metadata of the distributed file system, including the directory tree, file information (size, blocks, etc.), storage nodes where each file block is distributed, etc.

[0046] S102: Receive storage usage of each file system on each storage node returned by each storage node in the storage cluster in response to a query command.

[0047] In this step, the client can receive the storage usage of each file system on each storage node returned by each storage node in the storage cluster in response to the query command. For example, a file can be divided into three file blocks, namely: block1, block2 and block3; for block1, any three storage nodes can be selected for storage; for block2, any three storage nodes can also be selected for storage; for block3, any three storage nodes can also be selected for storage. Assuming that block1 selects storage node 1, storage node 2 and storage node 3 for storage, at this time, storage node 1 can return the storage usage of each file system on storage node 1 to the client; storage node 2 can return the storage usage of each file system on storage node 2 to the client; storage node 3 can return the storage usage of each file system on storage node 3 to the client.

[0048] S103: Based on the storage usage of each file system on each storage node, the total storage usage of each file system is counted.

[0049] In this step, the client can count the total storage usage of each file system based on the storage usage of each file system on each storage node. Specifically, when all storage nodes return, the client summarizes and obtains the real-time and accurate total storage usage of the file system.

[0050] The statistical method of storage usage proposed in the embodiment of the present application is that the client first sends a query command for storage usage to each storage node in the storage cluster; then receives the storage usage of each file system on each storage node returned by each storage node in the storage cluster in response to the query command; and then based on the storage usage of each file system on each storage node, the total storage usage of each file system is counted. That is to say, the storage node in the present application can save the storage usage of each file system. Since the block ID of the file block contains the ID of the file system, when the data of each file block is added or deleted, the storage usage of the file system to which the file block belongs can be updated in real time in the memory; when it is necessary to query the current real-time storage usage of a file system, a command to query storage usage is sent to all storage nodes in the storage cluster concurrently, and the storage node that receives the command obtains the storage usage of each file system from the memory in real time and returns it. When all storage nodes return, they are summarized to obtain the real-time and accurate total storage usage of the file system. In the existing statistical methods for storage usage, a scheme of collecting logs for statistics or a scheme of dumping database tables for statistics is usually adopted. Because the present application adopts a technical means of encoding the ID of the file system to which each file block belongs into the block ID of each file block, and storing the storage usage of each file system in the memory of the storage node, it overcomes the technical problems of low timeliness and inability to count the storage usage of the file system in real time brought about by the solution of collecting logs for statistics in the prior art, as well as the technical problems of poor failure, long time consumption, and possible impact on normal online services brought about by the solution of dump database tables for statistics. The technical solution provided by the present application can accurately query the storage usage of each file system in real time at any time, and has the advantages of less time consumption, accurate and error-free data, and no impact on normal online services; moreover, the technical solution of the embodiment of the present application is simple and convenient to implement, easy to popularize, and has a wider range of applications.

[0051] Embodiment 2

[0052] Figure 3 1 is a second flow chart of the method for counting storage usage provided in the embodiment of the present application. The above technical solution is further optimized and expanded, and can be combined with the above optional implementation methods. Figure 3 As shown, the method for counting storage usage may include the following steps:

[0053] S301. Cut file data horizontally into multiple file blocks; wherein each file block is uniquely identified by a block ID.

[0054] In this step, the client can horizontally cut the file data into multiple file blocks; wherein each file block is uniquely identified by a block ID. Assume that a certain file can be horizontally cut into three file blocks, namely file block 1, file block 2 and file block 3; these three file blocks can be uniquely identified by a block ID respectively. The number of bits of the block ID in the embodiment of the present application can be preset as needed. The sizes of different file blocks can be the same or different.

[0055] S302: Encode the ID of the file system to which each file block belongs into the block ID of each file block.

[0056] In this step, the client can encode the ID of the file system to which each file block belongs into the block ID of each file block, that is, the block ID in the embodiment of the present application can include the following two parts: the ID of the file system and the globally unique random string. For example, assuming that a certain file can be horizontally cut into three file blocks, namely file block 1, file block 2 and file block 3; assuming that the file system to which file block 1 belongs is A, the block ID of file block 1 can be expressed as "A+globally unique random string 1"; assuming that the file system to which file block 2 belongs is B, the block ID of file block 2 can be expressed as "B+globally unique random string 2"; assuming that the file system to which file block 3 belongs is C, the block ID of file block 3 can be expressed as "C+globally unique random string 3".

[0057] S303: Send each file block and the block ID of each file block to the storage cluster, so that the storage cluster stores each file block and the block ID of each file block in a predetermined number of storage nodes.

[0058] In this step, the client can send each file block and the block ID of each file block to the storage cluster, so that the storage cluster stores each file block and the block ID of each file block in a predetermined number of storage nodes. For example, a file can be divided into three blocks, namely: block1, block2 and block3; for block1, any three storage nodes (storage node) can be selected for storage; these three storage nodes respectively store block1 and the corresponding block ID. For block2, any three storage nodes (storage node) can also be selected for storage; these three storage nodes respectively store block2 and the corresponding block ID. For block3, any three storage nodes (storage node) can also be selected for storage; these three storage nodes respectively store block3 and the corresponding block ID.

[0059] S304: Send a query command for storage usage to each storage node in the storage cluster.

[0060] S305: Receive the storage usage of each file system on each storage node returned by each storage node in the storage cluster in response to the query command.

[0061] S306: Based on the storage usage of each file system on each storage node, the total storage usage of each file system is counted.

[0062] In a specific embodiment of the present application, the storage node may also store information of each file block in the memory of the storage node; wherein the information of the file block includes but is not limited to: the block ID of the file block, the size of the file block, and the creation time.

[0063] The statistical method of storage usage proposed in the embodiment of the present application is that the client first sends a query command for storage usage to each storage node in the storage cluster; then receives the storage usage of each file system on each storage node returned by each storage node in the storage cluster in response to the query command; and then based on the storage usage of each file system on each storage node, the total storage usage of each file system is counted. That is to say, the storage node in the present application can save the storage usage of each file system. Since the block ID of the file block contains the ID of the file system, when the data of each file block is added or deleted, the storage usage of the file system to which the file block belongs can be updated in real time in the memory; when it is necessary to query the current real-time storage usage of a file system, a command to query storage usage is sent to all storage nodes in the storage cluster concurrently, and the storage node that receives the command obtains the storage usage of each file system from the memory in real time and returns it. When all storage nodes return, they are summarized to obtain the real-time and accurate total storage usage of the file system. In the existing statistical methods for storage usage, a scheme of collecting logs for statistics or a scheme of dumping database tables for statistics is usually adopted. Because the present application adopts a technical means of encoding the ID of the file system to which each file block belongs into the block ID of each file block, and storing the storage usage of each file system in the memory of the storage node, it overcomes the technical problems of low timeliness and inability to count the storage usage of the file system in real time brought about by the solution of collecting logs for statistics in the prior art, as well as the technical problems of poor failure, long time consumption, and possible impact on normal online services brought about by the solution of dump database tables for statistics. The technical solution provided by the present application can accurately query the storage usage of each file system in real time at any time, and has the advantages of less time consumption, accurate and error-free data, and no impact on normal online services; moreover, the technical solution of the embodiment of the present application is simple and convenient to implement, easy to popularize, and has a wider range of applications.

[0064] Embodiment 3

[0065] Figure 4 3 is a schematic diagram of a third flow chart of a storage usage statistics method provided in an embodiment of the present application. The method can be executed by a storage usage statistics device or storage node. The device or storage node can be implemented by software and / or hardware. The device or storage node can be integrated in any intelligent device with network communication function. Figure 4 As shown, the method for counting storage usage may include the following steps:

[0066] S401: Receive a storage usage query command sent by a client.

[0067] S402: In response to the query command, obtain the storage usage of each file system on the storage node.

[0068] In this step, the storage node can obtain the storage usage of each file system on the storage node in response to the query command. Specifically, the storage node can first decode the block ID of each file block in the storage node to obtain the file system to which each file block belongs; wherein the block ID may include: the ID of the file system and a globally unique random string; and then determine the storage usage of each file system on the storage node according to the file system to which each file block belongs.

[0069] S403: Return the storage usage of each file system on the storage node to the client, so that the client calculates the total storage usage of each file system based on the storage usage of each file system on each storage node.

[0070] The statistical method of storage usage proposed in the embodiment of the present application is that the client first sends a query command for storage usage to each storage node in the storage cluster; then receives the storage usage of each file system on each storage node returned by each storage node in the storage cluster in response to the query command; and then based on the storage usage of each file system on each storage node, the total storage usage of each file system is counted. That is to say, the storage node in the present application can save the storage usage of each file system. Since the block ID of the file block contains the ID of the file system, when the data of each file block is added or deleted, the storage usage of the file system to which the file block belongs can be updated in real time in the memory; when it is necessary to query the current real-time storage usage of a file system, a command to query storage usage is sent to all storage nodes in the storage cluster concurrently, and the storage node that receives the command obtains the storage usage of each file system from the memory in real time and returns it. When all storage nodes return, they are summarized to obtain the real-time and accurate total storage usage of the file system. In the existing statistical methods for storage usage, a scheme of collecting logs for statistics or a scheme of dumping database tables for statistics is usually adopted. Because the present application adopts a technical means of encoding the ID of the file system to which each file block belongs into the block ID of each file block, and storing the storage usage of each file system in the memory of the storage node, it overcomes the technical problems of low timeliness and inability to count the storage usage of the file system in real time brought about by the solution of collecting logs for statistics in the prior art, as well as the technical problems of poor failure, long time consumption, and possible impact on normal online services brought about by the solution of dump database tables for statistics. The technical solution provided by the present application can accurately query the storage usage of each file system in real time at any time, and has the advantages of less time consumption, accurate and error-free data, and no impact on normal online services; moreover, the technical solution of the embodiment of the present application is simple and convenient to implement, easy to popularize, and has a wider range of applications.

[0071] Embodiment 4

[0072] Figure 5 4 is a schematic diagram of a fourth flow chart of a method for counting storage usage provided in an embodiment of the present application. Based on the above technical solution, it is further optimized and expanded, and can be combined with the above optional implementation methods. Figure 5 As shown, the method for counting storage usage may include the following steps:

[0073] S501: Receive a storage usage query command sent by a client.

[0074] S502. In response to the query command, decode the block ID of each file block in the storage node to obtain the file system to which each file block belongs; wherein the block ID includes: the ID of the file system and a globally unique random string.

[0075] In this step, the storage node can decode the block ID of each file block in the storage node to obtain the file system to which each file block belongs. Specifically, when the user creates a file system, a globally unique file system ID is generated. This ID is a strictly increasing integer that uniquely corresponds to and identifies the file system. Each file belongs to a specific file system, and the file data is horizontally cut into multiple blocks. Each block is uniquely identified by a block id. The file system id is encoded into the block id. Each block id consists of two parts: the file system id and a globally unique random string. In this way, by decoding the block id, you can know which file system this block belongs to.

[0076] S503: Determine the storage usage of each file system on the storage node according to the file system to which each file block belongs.

[0077] S504: Return the storage usage of each file system on the storage node to the client, so that the client calculates the total storage usage of each file system based on the storage usage of each file system on each storage node.

[0078] In a specific embodiment of the present application, the block data of each file is actually stored on a storage node in the cluster. Each storage node manages multiple blocks and stores the information of each block (blockid, block size, creation time, etc.) in memory. The storage usage of each filesystem id is stored in the storage node memory. Since the block id contains the file system id, when the data of each block is added or deleted, the storage usage of the file system id to which the block belongs can be updated in real time in the memory. When it is necessary to query the current real-time storage usage of a file system, a command to query the storage usage is sent to all storage nodes in the cluster. The storage node that receives the command obtains the storage usage of the file system id from the memory in real time and returns it. When all storage nodes return, they are summarized to obtain the real-time and accurate total storage usage of the file system.

[0079] In the industry, the distributed system storage statistics generally use the method of collecting billing logs and calculating through Hadoop to obtain the total storage capacity. The disadvantages of this solution are poor timeliness and errors. The errors will accumulate and become larger and larger over time. In the present invention, the file system id is encoded into the file block ID (block id), the storage usage of each file system id is saved in real time on the storage node, and the storage usage of a certain file system is queried in real time on all storage nodes and summarized. This method has the advantages of real-time, accurate, error-free, and no impact on online services.

[0080] The statistical method of storage usage proposed in the embodiment of the present application is that the client first sends a query command for storage usage to each storage node in the storage cluster; then receives the storage usage of each file system on each storage node returned by each storage node in the storage cluster in response to the query command; and then based on the storage usage of each file system on each storage node, the total storage usage of each file system is counted. That is to say, the storage node in the present application can save the storage usage of each file system. Since the block ID of the file block contains the ID of the file system, when the data of each file block is added or deleted, the storage usage of the file system to which the file block belongs can be updated in real time in the memory; when it is necessary to query the current real-time storage usage of a file system, a command to query storage usage is sent to all storage nodes in the storage cluster concurrently, and the storage node that receives the command obtains the storage usage of each file system from the memory in real time and returns it. When all storage nodes return, they are summarized to obtain the real-time and accurate total storage usage of the file system. In the existing statistical methods for storage usage, a scheme of collecting logs for statistics or a scheme of dumping database tables for statistics is usually adopted. Because the present application adopts a technical means of encoding the ID of the file system to which each file block belongs into the block ID of each file block, and storing the storage usage of each file system in the memory of the storage node, it overcomes the technical problems of low timeliness and inability to count the storage usage of the file system in real time brought about by the solution of collecting logs for statistics in the prior art, as well as the technical problems of poor failure, long time consumption, and possible impact on normal online services brought about by the solution of dump database tables for statistics. The technical solution provided by the present application can accurately query the storage usage of each file system in real time at any time, and has the advantages of less time consumption, accurate and error-free data, and no impact on normal online services; moreover, the technical solution of the embodiment of the present application is simple and convenient to implement, easy to popularize, and has a wider range of applications.

[0081] Embodiment 5

[0082] Figure 61 is a schematic diagram of the first structure of the storage usage statistics device provided in the embodiment of the present application. Figure 6 As shown, the device 600 includes: a first sending module 601, a first receiving module 602 and a statistical module 603; wherein,

[0083] The first sending module 601 is used to send a query command of storage usage to each storage node in the storage cluster;

[0084] The first receiving module 602 is used to receive the storage usage of each file system on each storage node returned by each storage node in the storage cluster in response to the query command;

[0085] The statistical module 603 is used to count the total storage usage of each file system based on the storage usage of each file system on each storage node.

[0086] Furthermore, the device further includes: a cutting module 604 (not shown in the figure) and an encoding module 605 (not shown in the figure); wherein,

[0087] The cutting module 604 is used to horizontally cut the file data into multiple file blocks; wherein each file block is uniquely identified by a block ID;

[0088] The encoding module 605 is used to encode the ID of the file system to which each file block belongs into the block ID of each file block;

[0089] The first sending module 601 is further configured to send each file block and a block ID of each file block to the storage cluster, so that the storage cluster stores each file block and a block ID of each file block in a predetermined number of storage nodes.

[0090] Furthermore, the first sending module 601 is specifically used to send a query command of storage usage to each storage node in the storage cluster at any time; or, send a query command of storage usage to each storage node in the storage cluster according to a predetermined period.

[0091] The above storage usage statistics device can execute the method provided in any of the first and second embodiments of the present application, and has the corresponding functional modules and beneficial effects of the execution method. For technical details not fully described in this embodiment, please refer to the storage usage statistics method provided in the first and second embodiments of the present application.

[0092] Embodiment 6

[0093] Figure 7 2 is a schematic diagram of the second structure of the storage usage statistics device provided in the embodiment of the present application. Figure 7As shown, the device 700 includes: a second receiving module 701, an acquisition module 702 and a second sending module 703; wherein,

[0094] The second receiving module 701 is used to receive a query command for storage usage sent by a client;

[0095] The acquisition module 702 is used to acquire the storage usage of each file system on the storage node in response to the query command;

[0096] The second sending module 703 is used to return the storage usage of each file system on the storage node to the client, so that the client can count the total storage usage of each file system based on the storage usage of each file system on each storage node.

[0097] Furthermore, the acquisition module 702 is specifically used to decode the block ID of each file block in the storage node to obtain the file system to which each file block belongs; wherein the block ID includes: the ID of the file system and a globally unique random string; based on the file system to which each file block belongs, determine the storage usage of each file system on the storage node.

[0098] Furthermore, the device also includes: a storage module 704 (not shown in the figure), which is used to store the information of each file block in the memory of the storage node; wherein the information of the file block includes but is not limited to: the block ID of the file block, the size of the file block, and the creation time.

[0099] The above storage usage statistics device can execute the method provided in any of the third and fourth embodiments of the present application, and has the corresponding functional modules and beneficial effects of the execution method. For technical details not described in detail in this embodiment, please refer to the storage usage statistics method provided in the third and fourth embodiments of the present application.

[0100] Embodiment 7

[0101] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0102] Figure 8A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0103] like Figure 8 As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0104] A number of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0105] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as a statistical method for storage usage. For example, in some embodiments, the statistical method for storage usage may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the statistical method for storage usage described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform a statistical method for storage usage in any other appropriate manner (e.g., by means of firmware).

[0106] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0107] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0108] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0109] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0110] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0111] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0112] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution disclosed in this application can be achieved, and this document is not limited here. In the technical solution of this disclosure, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good customs.

[0113] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A storage usage statistics method, applied to a client, comprising: Sending a storage usage query command to each storage node in the storage cluster; wherein the query command is used to query the storage usage of each file system on each storage node; Receiving storage usage of each file system on each storage node returned by each storage node in the storage cluster in response to the query command; Based on the storage usage of each file system on each storage node, the total storage usage of each file system is counted; the total storage usage is used to bill the each file system; Wherein, the storage usage of each file system is stored in the memory of each storage node; When data of each file block in each storage node is added or deleted, each storage node updates the storage usage of the file system to which the file block belongs in real time in the memory; The storage node that receives the query command obtains the storage usage of each file system from the memory in real time and returns it to the client.

2. The method according to claim 1, before sending a query command of storage usage to each storage node in the storage cluster, the method further comprises: The file data is horizontally cut into multiple file blocks; each file block is uniquely identified by a block ID; Encode the ID of the file system to which each file block belongs into the block ID of each file block; Each file block and a block ID of each file block are sent to the storage cluster, so that the storage cluster stores each file block and a block ID of each file block in a predetermined number of storage nodes.

3. The method according to claim 1, sending a query command of storage usage to each storage node in the storage cluster, comprises: Send storage usage query commands to each storage node in the storage cluster at any time; Alternatively, a query command for storage usage is sent to each storage node in the storage cluster according to a predetermined period.

4. A storage usage statistics method, applied to a storage node, comprising: Receive a storage usage query command sent by a client; wherein the query command is used to query the storage usage of each file system on each storage node; Responding to the query command, obtaining the storage usage of each file system on the storage node; Returning the storage usage of each file system on the storage node to the client, so that the client calculates the total storage usage of each file system based on the storage usage of each file system on each storage node; the total storage usage is used to bill the each file system; Wherein, the method further comprises: Saving the storage usage of each file system in the memory; When the data of each file block is added or deleted, the storage usage of the file system to which the file block belongs is updated in real time in the memory; Upon receiving the query command, the storage usage of each file system is obtained from the memory in real time and returned to the client.

5. The method according to claim 4, obtaining the storage usage of each file system on the storage node comprises: Decoding the block ID of each file block in the storage node to obtain the file system to which each file block belongs; wherein the block ID includes: the ID of the file system and a globally unique random string; According to the file systems to which the various file blocks belong, the storage usage of the various file systems on the storage node is determined.

6. The method according to claim 5, further comprising: The information of each file block is stored in the memory of the storage node; wherein the information of the file block includes but is not limited to: a block ID of the file block, a size of the file block, and a creation time.

7. A device for storing usage statistics, the device comprising: A first sending module, a first receiving module and a statistical module; wherein, The first sending module is used to send a query command of storage usage to each storage node in the storage cluster; wherein the query command is used to query the storage usage of each file system on each storage node; The first receiving module is used to receive the storage usage of each file system on each storage node returned by each storage node in the storage cluster in response to the query command; The statistical module is used to count the total storage usage of each file system based on the storage usage of each file system on each storage node; the total storage usage is used to bill the various file systems; Wherein, the storage usage of each file system is stored in the memory of each storage node; When data of each file block in each storage node is added or deleted, each storage node updates the storage usage of the file system to which the file block belongs in real time in the memory; The storage node that receives the query command obtains the storage usage of each file system from the memory in real time and returns it to the client.

8. The device according to claim 7, further comprising: Cutting module and encoding module; wherein, The cutting module is used to horizontally cut the file data into multiple file blocks; wherein each file block is uniquely identified by a block ID; The encoding module is used to encode the ID of the file system to which each file block belongs into the block ID of each file block; The first sending module is further configured to send each file block and a block ID of each file block to the storage cluster, so that the storage cluster stores each file block and a block ID of each file block in a predetermined number of storage nodes.

9. According to the device of claim 7, the first sending module is specifically used to send a query command of storage usage to each storage node in the storage cluster at any time; or, send a query command of storage usage to each storage node in the storage cluster according to a predetermined period.

10. A storage usage statistics device, configured in a storage node, comprising: A second receiving module, an acquiring module and a second sending module; wherein, The second receiving module is used to receive a storage usage query command sent by a client; wherein the query command is used to query the storage usage of each file system on each storage node; The acquisition module is used to acquire the storage usage of each file system on the storage node in response to the query command; The second sending module is used to return the storage usage of each file system on the storage node to the client, so that the client counts the total storage usage of each file system based on the storage usage of each file system on each storage node; the total storage usage is used to bill the each file system; Wherein, the acquisition module is specifically used for: Saving the storage usage of each file system in the memory; When the data of each file block is added or deleted, the storage usage of the file system to which the file block belongs is updated in real time in the memory; Upon receiving the query command, the storage usage of each file system is obtained from the memory in real time and returned to the client.

11. The device according to claim 10, wherein the acquisition module is specifically used to decode the block ID of each file block in the storage node to obtain the file system to which each file block belongs; wherein, The block ID includes: the ID of the file system and a globally unique random string; according to the file system to which each file block belongs, the storage usage of each file system on the storage node is determined.

12. The device according to claim 11, further comprising: The storage module is used to store the information of each file block in the memory of the storage node; wherein the information of the file block includes but is not limited to: the block ID of the file block, the size of the file block, and the creation time.

13. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-3 or 4-6.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-3 or 4-6.

15. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1-3 or 4-6.

Images