File index information processing method, device, electronic device and storage medium

Abstract

The embodiment of the present invention provides a file index information processing method, device, electronic device and storage medium. The method includes: writing the target file data to be stored into the target storage resource under the storage management node; determining the target file index sub-table from the target file index resource associated with the target storage resource and writing the file index information of the target file data to be stored into the target file index sub-table; when reading the file data, searching for the preset file index sub-table adapted to the target file data to be queried from the preset file index resource associated with the preset storage resource under the storage management node; and performing the target file data to be queried reading operation in the preset storage resource associated with the preset file index resource to which the adapted preset file index sub-table belongs based on the file index information recorded in the searched adapted preset file index sub-table. This solution is adopted to solve the problem of slow retrieval and reading speed when storing massive image data in security business.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of storage technology, and in particular, to a file index information processing method, device, electronic device and storage medium. Background Art

[0002] With the continuous development of the economy and society, people's demand for security is getting higher and higher, which has led to the widespread application of monitoring systems. In the monitoring business, the image storage business is an important part; for example, various image businesses such as car passing pictures, illegal pictures, face pictures, structured data, etc. have generated a large number of image storage and reading needs. However, an ordinary medium-sized city generates tens of millions or even more image data every day. In the relevant solutions, according to the resource capacity and fixed format, a file name index area is created at one or more locations on the resource to record file information, which will cause the problem of slow writing and retrieval of pictures when storing massive pictures in the security business. Summary of the invention

[0003] The embodiments of the present invention provide a file index information processing method, device, electronic device and storage medium to solve the problem of slow retrieval and reading speed when storing massive image data in security business.

[0004] In a first aspect, an embodiment of the present invention provides a method for processing file index information, the method comprising:

[0005] Write the target file data to be stored into the target storage resource under the storage management node;

[0006] Determine a target file index sub-table from the target file index resources associated with the target storage resource and write file index information of the target file data to be stored into the target file index sub-table;

[0007] When reading file data, find the preset file index sub-table that is compatible with the target file data to be queried from the preset file index resources associated with the preset storage resources under the storage management node;

[0008] According to the file index information recorded in the matched preset file index sub-table, a data reading operation of the target file to be queried is performed in the preset storage resource associated with the preset file index resource to which the matched preset file index sub-table belongs.

[0009] In a second aspect, an embodiment of the present invention further provides a file index information processing device, the device comprising:

[0010] A module for writing target file data to be stored, used for writing the target file data to be stored into the target storage resource under the storage management node;

[0011] A target file index sub-table determining module, used to determine a target file index sub-table from the target file index resources associated with the target storage resource and write file index information of the target file data to be stored into the target file index sub-table;

[0012] A preset file index sub-table acquisition module is used to search for a preset file index sub-table that is compatible with the target file data to be queried from the preset file index resources associated with the preset storage resources under the storage management node when reading file data;

[0013] The target file data to be queried reading module is used to perform a target file data to be queried reading operation in a preset storage resource associated with a preset file index resource to which the adapted preset file index sub-table belongs, based on searching for file index information recorded in the adapted preset file index sub-table.

[0014] In a third aspect, an embodiment of the present invention further provides an electronic device, including:

[0015] one or more processing devices;

[0016] A storage device for storing one or more programs;

[0017] When the one or more programs are executed by the one or more processing devices, the one or more processing devices implement any of the file index information processing methods described in the embodiments of the present invention.

[0018] In a fourth aspect, an embodiment of the present invention further provides a readable medium on which a computer program is stored, and when the program is executed by a processing device, any file index information processing method described in the embodiments of the present invention is implemented.

[0019] The embodiment of the present invention provides a method, device, electronic device and storage medium for processing file index information. The method comprises: writing the target file data to be stored into the target storage resource under the storage management node; determining the target file index sub-table from the target file index resource associated with the target storage resource and writing the file index information of the target file data to be stored into the target file index sub-table; when reading file data, searching for the preset file index sub-table adapted to the target file data to be queried from the preset file index resource associated with the preset storage resource under the storage management node; and performing the operation of reading the target file data to be queried in the preset storage resource associated with the preset file index resource to which the adapted preset file index sub-table belongs according to the file index information recorded in the searched adapted preset file index sub-table.

[0020] By adopting the application scheme, the file name index information and data resources are separated, the flexibility of file index storage resource configuration is improved, and the disk pressure caused by traversing the disk when retrieving file information is avoided; the hash value is calculated by the file name and written into the corresponding sub-table, which avoids traversing all sub-tables when searching for file indexes; the method of predicting the number of files in the directory is adopted to avoid the problem of too many sub-tables or too large storage of a single table, so as to solve the problem of slow retrieval and reading speed when storing massive image data in security business.

[0021] The above invention content is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Other features, objects and advantages of the present invention will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings. The drawings are only for the purpose of illustrating preferred embodiments and are not to be considered as limiting the present invention. Also, the same reference symbols are used throughout the drawings to represent the same parts. In the drawings:

[0023] Figure 1A is a flow chart of a method for processing file index information provided by Embodiment 1 of the present invention;

[0024] Figure 1B This is a schematic diagram of a file name index area management solution provided by Embodiment 1 of the present invention;

[0025] Figure 1C This is a schematic diagram of a database organization structure provided by Embodiment 1 of the present invention;

[0026] Figure 2A is a flow chart of a method for processing file index information provided by Embodiment 2 of the present invention;

[0027] Figure 2B is a flow chart of a method for writing file data provided by Embodiment 2 of the present invention;

[0028] Figure 3A is a flowchart of a method for processing file index information provided by Embodiment 3 of the present invention;

[0029] Figure 3B This is a schematic diagram of cloud storage management provided by Embodiment 3 of the present invention;

[0030] Figure 3C is a flow chart of a file data reading method provided by Embodiment 3 of the present invention;

[0031] Figure 4is a structural block diagram of a file index information processing device provided by a fourth embodiment of the present invention;

[0032] Figure 5 It is a structural schematic diagram of an electronic device provided in Embodiment 5 of the present invention. DETAILED DESCRIPTION

[0033] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only the parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0034] Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe the operations (or steps) as sequential processes, many of the operations (or steps) therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but can also have additional steps not included in the accompanying drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0035] Embodiment 1

[0036] Figure 1A This is a flowchart of a method for processing file index information provided by Embodiment 1 of the present invention. This embodiment is applicable to the case of processing file data. The method of this embodiment can be executed by a file index information processing device, and the device can be implemented in hardware and / or software. The device can be configured in a server for processing file index information. The method specifically includes the following steps:

[0037] S110. Write the target file data to be stored into the target storage resource under the storage management node.

[0038] According to the resource capacity and fixed format, a file name index area is created at one or more locations on the resource to record file information. However, there are certain defects in creating a file name index area on the resource itself. For example, when reading and writing file information and file data, it is necessary to search at multiple locations on the disk, which increases disk latency; the retrieval efficiency of the file name index area is strongly bound to the disk performance and has poor flexibility; when searching for massive data, the data is searched by comparing one by one, resulting in low performance. Therefore, the present application provides a method for processing file index information.

[0039] The target file data to be stored may refer to the image data to be stored in the security business, such as the vehicle passing pictures, violation pictures, face pictures and structured data in the monitoring system. Various image businesses generate a large number of image storage and reading requirements.

[0040] The storage management node refers to the storage management node server (CDV), which is responsible for selecting specific resource record file data and file index data information. Figure 1B Schematic diagram of a file name index area management solution provided by the first embodiment of the present invention. Figure 1B As shown, data resources and file index resources are separated, and a one-to-one correspondence strategy is adopted between data resources and file index database data.

[0041] Write the image data into the storage management node server management software. The storage resources of the storage management node server CDV include the data resource pool RES and database resources. Each CDV manages several resources RES1-RESn and 1 database resource. The size of the database resource is 1 / 128 of the size of the CDV data resource pool. Each data resource corresponds to a sqlite database named uuid.db, where uuid is the unique number of the corresponding CDV data resource. All databases share database resources.

[0042] Figure 1C is a schematic diagram of a database organization structure provided by the first embodiment of the present invention, such as Figure 1C As shown in the figure, each database uuid.db contains two types of data tables, namely the directory relationship data table and the file index information data table. The directory relationship data table tbl_dir_tblname_relation records the directory information and the relationship between the directory and the file index information table. tbl_dir_tblname_relation_bak is the backup of tbl_dir_tblname_relation. The file index information table records detailed file index information. Each directory corresponds to several file index information data tables. The table name is tbl_dirindx-m_info, where dirindx is the index value of the corresponding directory and m is the mth sub-table. Separating the file name index information from the data resources improves the flexibility of the file index storage resource configuration and avoids disk pressure caused by traversing the disk when retrieving file information.

[0043] S120: Determine a target file index sub-table from the target file index resources associated with the target storage resource and write file index information of the target file data to be stored into the target file index sub-table.

[0044] The target file index sub-table may refer to an index sub-table established based on the file index information of the target file data to be stored. When the amount of target file data in a single directory is large, the writing and reading speeds are seriously affected. A target file index sub-table is established for the target file data in each directory. The target file index sub-table to be written can be quickly queried based on the file index information of the target file data to be stored, and the target file data to be read can be queried based on the target file index sub-table. The file index information includes file index data for recording file information such as the creation time, modification time, file index, and directory index of the file data.

[0045] S130. When reading file data, search for a preset file index sub-table that is compatible with the target file data to be queried from the preset file index resources associated with the preset storage resources under the storage management node.

[0046] Among them, the preset file index resource may refer to a directory index to be queried including but not limited to the target file data to be queried. For example, the directory relationship table of the preset file index resource is queried according to the directory index to be queried of the target file data to be queried, and it is determined that the directory relationship table of the preset file index resource includes the directory index to be queried of the target file data to be queried; if the directory index to be queried of the target file data to be queried is contained, the number of sub-tables of the file index sub-table under the directory index to be queried of the target file data to be queried is obtained; and the number of sub-tables and the file identifier corresponding to the target file data to be queried are hashed to find the preset file index sub-table adapted to the target file data to be queried.

[0047] Among them, hashing can refer to compressing a message of any length into a message of a fixed length, for example, hashing the file identifier corresponding to the number of sub-tables and the target file data to be stored, and using the processed hash value as the sub-table serial number to determine the target file index; that is, compressing the file identification information corresponding to the number of sub-tables and the target file data to be stored, transforming it into the sub-table serial number for output, avoiding traversing all sub-tables when searching for file indexes.

[0048] The number of sub-tables may refer to sub-tables of all file index sub-tables according to the to-be-queried directory index of the to-be-queried target file data, thereby avoiding traversing all sub-tables when searching for file indexes and improving file retrieval efficiency.

[0049] S140: According to the file index information recorded in the matched preset file index sub-table, a data reading operation of the target file to be queried is performed in the preset storage resource associated with the preset file index resource to which the matched preset file index sub-table belongs.

[0050] Among them, the preset file index sub-table may refer to a hash calculation of the file identifier and the number of sub-tables corresponding to the target file data to be queried, and the hash processing result is used as the table number of the file index sub-table. The preset file index sub-table is determined according to the table number. The file index sub-table records file index information. Through the file index information, the preset storage resource associated with the preset file index resource to which the adapted preset file index sub-table belongs can be used to perform a reading operation on the target file data to be queried.

[0051] For example, the number of sub-tables of the file index sub-table under the directory index to be queried is calculated, and the number of sub-tables is recorded in the tbl_dir_tblname_relation table. According to the file identifier and the number of sub-tables corresponding to the target file data to be queried, the hash value corresponding to the target file data to be queried is calculated using the consistent hashing algorithm as the sub-table sequence number for reading the file index information. When performing the operation of reading the target file data to be queried, the preset file index sub-table adapted to the target file data to be queried is searched according to the sub-table sequence number.

[0052] According to the file index information processing method provided by the embodiment of the present invention, the file name index information and data resources are separated, so as to improve the flexibility of file index storage resource configuration and avoid traversing the disk and causing disk pressure when retrieving file information; the hash value is calculated by the file name and written into the corresponding sub-table, so as to avoid traversing all sub-tables when searching for file indexes; the method of predicting the number of files under the directory is adopted, so as to avoid the problem of too many sub-tables or too large storage of a single table, so as to solve the problem of slow retrieval and reading speed when storing massive image data in security business.

[0053] Embodiment 2

[0054] Figure 2A 1 is a flowchart of a method for processing file index information provided by Embodiment 2 of the present invention. The embodiment of the present invention further optimizes the above embodiment on the basis of the above embodiment, and the embodiment of the present invention can be combined with various optional solutions in one or more of the above embodiments. Figure 2A As shown, the file index information processing method provided in the embodiment of the present invention may include the following steps:

[0055] S210: Write the target file data to be stored into the target storage resource under the storage management node.

[0056] S220: Determine a target file index sub-table from the target file index resources associated with the target storage resource.

[0057] Among them, according to the characteristics of the security image business, the number of checkpoints and collected images has obvious peaks and troughs, and there is a certain pattern. In the security business, the image directory is usually in units of hours or days. The number of images stored in a single directory is between 10 and 10 million, depending on the time and business scale. When the amount of data in a single directory is large, it seriously affects the writing and retrieval speed, so it is necessary to divide the file index table corresponding to each directory into separate tables.

[0058] Optionally, determining a directory index to be stored in the target file index resource for storing file index information and the number of sub-tables of the file index sub-table under the directory index to be stored;

[0059] Hash the file identifier corresponding to the number of sub-tables and the target file data to be stored;

[0060] The hash processing result is used as the sub-table serial number of the file index sub-table for storing the file index information of the target file data to be stored under the directory index to be stored, so as to determine the target file index sub-table.

[0061] The calculation formula for the number of sub-tables is as follows:

[0062]

[0063] The directory is in hours, the default value is 3, tblmin is 1, and tblmax is 20;

[0064] The directory is in days, the default value is 20, tblmin is 1, and tblmax is 50.

[0065] in:

[0066] m can refer to the mth directory;

[0067] lastdirtblnum can refer to the number of the last directory file index data table;

[0068] num[m-2] can refer to the number of files stored in the first two directories;

[0069] num[m-1] can refer to the number of files stored in the previous directory;

[0070] defpicnum can refer to the default amount of image file index information data recorded in each file name index table, which is 20000 by default.

[0071] Optionally, determine the number of sub-tables of the file index sub-table under the previous directory index before the directory index to be stored;

[0072] Determine the amount of file index information data stored in the last directory index and the amount of file index information data in the last two directory indexes before the directory index to be stored;

[0073] According to the number of file index sub-tables under the previous directory index and the amount of file index information data stored under the previous directory index and the previous two directory indexes, the number of file index sub-tables under the directory index to be stored is determined to create the file index sub-tables.

[0074] In an optional solution of the embodiment of the present invention, the hourly directory is used as an example for explanation:

[0075] The first directory is 05, with a default number of sub-tables of 3 and a record file index number of 5000;

[0076] The second directory is 06, with a default number of sub-tables of 3 and a record file index number of 15,000;

[0077] The third directory is 07, corresponding to the number of sub-tables max{1,15000 / max{5000,20000}*3}=3 or min{20,15000 / max{5000,20000}*3}=3, assuming that the number of record file indexes is 50,000;

[0078] The fourth directory is 08, and the corresponding number of sub-tables is max{1,50000 / max{15000,20000}*3} or min{20,50000 / max{15000,20000}*3}=9.

[0079] After calculating the number of tables in each directory, record the number of sub-tables in the tbl_dir_tblname_relation table. Based on the file identifier and the number of sub-tables, use the consistent hashing algorithm to calculate the hash value corresponding to the file identifier as the sub-table serial number for writing the file index information, and then write the file information into the corresponding file index information data table. Using the file name to calculate the hash value and write it into the corresponding sub-table avoids traversing all sub-tables when searching for file indexes, thereby improving the efficiency of file data retrieval.

[0080] S230: Searching for a preset file index sub-table that matches the target file data to be queried from preset file index resources associated with preset storage resources under the storage management node.

[0081] S240: According to the file index information recorded in the matched preset file index sub-table, a target file data reading operation is performed in the preset storage resource associated with the preset file index resource to which the matched preset file index sub-table belongs.

[0082] In an optional solution of the embodiment of the present invention, Figure 2B is a flow chart of a method for writing file data provided by Embodiment 2 of the present invention. Figure 2B As shown, the method includes:

[0083] CDM selects CDV based on the load balancing principle;

[0084] CDV selects resources based on the load balancing principle;

[0085] Determine whether it is the first picture in the directory;

[0086] If it is the first picture in the directory, calculate the number of file index sub-tables;

[0087] The number of sub-tables and directory information are entered into the tbl_dir_tblname_relation table;

[0088] The consistent hash algorithm is used to calculate the hash value based on the file name and the number of sub-tables to determine the table to be written;

[0089] File index information is stored in the database;

[0090] If it is not the first picture in the directory, the number of sub-tables in the corresponding directory is obtained, and the hash value is calculated using a consistent hash algorithm based on the file name and the number of sub-tables to determine the table to be written; the file index information is stored in the database.

[0091] According to the file index information processing method provided in the embodiment of the present invention, the directory index to be stored in the target file index resource for storing the file index information and the number of sub-tables of the file index sub-table under the directory index to be stored are determined; the file identifier corresponding to the number of sub-tables and the target file data to be stored is hashed; the hash processing result is used as the sub-table sequence number of the file index sub-table for storing the file index information of the target file data to be stored under the directory index to be stored, so as to determine the target file index sub-table. The hash value is calculated by using the file identifier and written into the corresponding sub-table, which avoids traversing all sub-tables when searching for the file index, improves the retrieval efficiency of the file data, and solves the problem of slow retrieval and reading speed when storing massive image data in the security business.

[0092] Embodiment 3

[0093] Figure 3A 1 is a flowchart of a method for processing file index information provided by Embodiment 3 of the present invention. The embodiment of the present invention further optimizes the above embodiment on the basis of the above embodiment, and the embodiment of the present invention can be combined with various optional solutions in one or more of the above embodiments. Figure 3A As shown, the file index information processing method provided in the embodiment of the present invention may include the following steps:

[0094] S310: respond to the storage request of the metadata management server and determine the target storage resource from the preset data resources associated with the storage management node server.

[0095] Figure 3B Schematic diagram of cloud storage management provided by Embodiment 3 of the present invention. Figure 3B As shown in the figure, the CDS cloud storage management system (CDS) is a storage system launched to solve the shortcomings of small storage capacity, difficult expansion and no disaster recovery of single-machine systems based on the characteristics of monitoring business. This system consists of a metadata management server (CDM) and a storage management node server (CDV). Among them, CDM is a lightweight metadata management server that does not store specific metadata, but only selects a suitable CDV for storage according to a certain algorithm strategy for the data to be written or read. CDV is responsible for selecting specific resources to record file data and file index data information. When writing data, CDM selects a resource of a suitable CDV according to the load balancing principle, first writes the data to the resource data area, and then writes the file index information to the appropriate position in the file name index area; when reading data, CDM parses the CDV that records the file according to the file path, then traverses the resource file name index area in the CDV to search, and finally reads the file data in the data area according to the information recorded in the file name index area. In summary, a good file name index area management solution is of great significance to both writing and reading files.

[0096] The metadata management server selects a suitable storage management node server according to the writing requirements of the target file data to be stored, and determines the target storage resource from the preset data resources associated with the storage management node server.

[0097] S320: Write the target file data to be stored into the target storage resource.

[0098] S330: Determine a target file index sub-table from the target file index resources associated with the target storage resource and write file index information of the target file data to be stored into the target file index sub-table.

[0099] S340: When reading file data, find a preset file index sub-table that is compatible with the target file data to be queried from the preset file index resources associated with the preset storage resources under the storage management node.

[0100] When reading the file data, the number of sub-tables of the file index sub-table under the directory index to be queried is determined, and a hash calculation is performed, and a preset file index sub-table adapted to the target file data to be queried is searched according to the sub-table sequence number corresponding to the hash value.

[0101] Optionally, if it is determined that the directory relationship data table of the preset file index resource associated with the preset storage resource under the storage management node includes the directory index to be queried of the target file data to be queried, then the number of sub-tables of the file index sub-table under the directory index to be queried of the target file data to be queried is determined;

[0102] Hash the file identifier corresponding to the sub-table quantity and the target file data to be queried;

[0103] According to the sub-table serial number corresponding to the hash processing result, the preset file index sub-table adapted to the target file data to be queried is searched under the directory index to be queried.

[0104] When reading file data, the preset file index sub-table that is compatible with the target file data to be queried is searched from the preset file index resources associated with the preset storage resources under the storage management node. The file identifier is used to calculate the hash value to find the corresponding sub-table, which avoids traversing all sub-tables when searching for file indexes; the method of predicting the number of files in the directory is used to avoid the problem of too many sub-tables or too large storage of a single table.

[0105] S350: According to the file index information recorded in the matched preset file index sub-table, a data reading operation of the target file to be queried is performed in the preset storage resource associated with the preset file index resource to which the matched preset file index sub-table belongs.

[0106] In an optional solution of the embodiment of the present invention, optionally, Figure 3C is a flowchart of a file data reading method provided by Embodiment 3 of the present invention. Figure 3C As shown, the method includes:

[0107] CDM searches for the corresponding CDV based on the directory information;

[0108] Concurrent search of all resources under CDV;

[0109] Determine whether tbl_dir_tblname_relation contains directory information;

[0110] If it contains directory information, obtain the number of sub-tables, calculate the hash value based on the file name, search for the file in the corresponding sub-table, and merge the search results;

[0111] If there is no directory information, the search results will be merged.

[0112] According to the file index information processing method provided in the embodiment of the present invention, when reading file data, a preset file index sub-table adapted to the target file data to be queried is searched from the preset file index resources associated with the preset storage resources under the storage management node; a file identifier is used to calculate a hash value to search for the corresponding sub-table, thereby avoiding traversing all sub-tables when searching for the file index; a method for predicting the number of files under a directory is used to avoid the problem of too many sub-tables or too large storage of a single table; and the problem of slow retrieval and reading speed when storing massive image data in security business is solved.

[0113] Embodiment 4

[0114] Figure 4 This is a structural block diagram of a file index information processing device provided in Embodiment 4 of the present invention. This embodiment scheme can be applied to the case where a large amount of file data is processed, especially for the case where a large amount of metadata is read in the monitoring business. The device can be implemented in software and / or hardware, and integrated on any electronic device with network communication function. Among them, the electronic device can be a cloud storage node device in a cloud storage system.

[0115] like Figure 4 As shown, the file index information processing device in the embodiment of the present application may include the following: a target file data to be stored writing module 410, a target file index sub-table determining module 420, a preset file index sub-table obtaining module 430 and a target file data to be queried reading module 440. Among them:

[0116] The target file data to be stored is written into the module 410, which is used to write the target file data to be stored into the target storage resource under the storage management node;

[0117] A target file index sub-table determining module 420 is used to determine a target file index sub-table from the target file index resources associated with the target storage resource and write file index information of the target file data to be stored into the target file index sub-table;

[0118] The preset file index sub-table acquisition module 430 is used to search for a preset file index sub-table adapted to the target file data to be queried from the preset file index resources associated with the preset storage resources under the storage management node when reading file data;

[0119] The target file data to be queried reading module 440 is used to perform a target file data to be queried reading operation in the preset storage resource associated with the preset file index resource to which the adapted preset file index sub-table belongs according to the file index information recorded in the adapted preset file index sub-table.

[0120] Based on the above embodiment, optionally, the target file data to be stored writing module 410 includes:

[0121] In response to a storage request from a metadata management server, a target storage resource is determined from preset data resources associated with a storage management node server;

[0122] The target file data to be stored is written into the target storage resource.

[0123] Based on the above embodiment, optionally, determining the target file index sub-table from the target file index resource associated with the target storage resource includes:

[0124] Determine a directory index to be stored in the target file index resource for storing file index information and the number of sub-tables of the file index sub-table under the directory index to be stored;

[0125] Hash the file identifier corresponding to the number of sub-tables and the target file data to be stored;

[0126] The hash processing result is used as the sub-table serial number of the file index sub-table for storing the file index information of the target file data to be stored under the directory index to be stored, so as to determine the target file index sub-table.

[0127] Based on the above embodiment, optionally, determining the number of sub-tables of the file index sub-table under the directory index to be stored includes:

[0128] Determine the number of sub-tables of the file index sub-table under the previous directory index before the directory index to be stored;

[0129] Determine the amount of file index information data stored in the last directory index and the amount of file index information data in the last two directory indexes before the directory index to be stored;

[0130] According to the number of file index sub-tables under the previous directory index and the amount of file index information data stored under the previous directory index and the previous two directory indexes, the number of file index sub-tables under the directory index to be stored is determined to create the file index sub-tables.

[0131] Based on the above embodiment, optionally, searching for a preset file index sub-table adapted to the target file data to be queried from the preset file index resources associated with the preset storage resources under the storage management node includes:

[0132] If it is determined that the directory relationship data table of the preset file index resource associated with the preset storage resource under the storage management node includes the directory index to be queried of the target file data to be queried, then the number of sub-tables of the file index sub-table under the directory index to be queried of the target file data to be queried is determined;

[0133] Hash the file identifier corresponding to the sub-table quantity and the target file data to be queried;

[0134] According to the sub-table serial number corresponding to the hash processing result, the preset file index sub-table adapted to the target file data to be queried is searched under the directory index to be queried.

[0135] The file index information processing device provided in the embodiment of the present invention can execute the file index information processing method provided in any embodiment of the present invention mentioned above, and has the corresponding functions and beneficial effects of executing the file index information processing method. For technical details not described in detail in the above embodiments, please refer to the file index information processing method provided in any embodiment of the present application.

[0136] Embodiment 5

[0137] Figure 5 Schematic diagram of the structure of an electronic device provided by Embodiment 5 of the present invention. Figure 5 As shown in the structure, the electronic device 500 provided in the embodiment of the present invention includes: one or more storage devices 510 and a processor 520; the storage device 510 in the electronic device can be one or more, Figure 5 A storage device 510 is taken as an example; the processor 520 is used to store one or more programs; the one or more programs are executed by the one or more storage devices 510, so that the one or more storage devices 510 implement the file index information processing method as described in any one of the embodiments of the present invention.

[0138] The electronic device may further include: an input device 530 and an output device 540 .

[0139] The storage device 510, processor 520, input device 530 and output device 540 in the electronic device can be connected via a bus or other means. Figure 5 The connection via bus 550 is taken as an example.

[0140] The processor 520 in the electronic device is a computer-readable storage medium that can be used to store one or more programs, which can be software programs, computer executable programs, and modules, such as program instructions / modules corresponding to the file index information processing method provided in the embodiment of the present invention. The storage device 510 executes various functional applications and data processing of the electronic device by running the software programs, instructions, and modules stored in the processor 520, that is, the file index information processing method in the above method embodiment is implemented.

[0141] The processor 520 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the processor 520 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the processor 520 may further include a memory remotely arranged relative to the storage device 510, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0142] The input device 530 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the electronic device. The output device 540 may include a display device such as a display screen.

[0143] Furthermore, when one or more programs included in the electronic device are executed by the one or more storage devices 510, the program performs the following operations:

[0144] Write the target file data to be stored into the target storage resource under the storage management node;

[0145] Determine a target file index sub-table from the target file index resources associated with the target storage resource and write file index information of the target file data to be stored into the target file index sub-table;

[0146] When reading file data, find the preset file index sub-table that is compatible with the target file data to be queried from the preset file index resources associated with the preset storage resources under the storage management node;

[0147] According to the file index information recorded in the matched preset file index sub-table, a data reading operation of the target file to be queried is performed in the preset storage resource associated with the preset file index resource to which the matched preset file index sub-table belongs.

[0148] Of course, those skilled in the art can understand that when one or more programs included in the above electronic device are executed by the one or more storage devices 510, the program can also perform related operations in the file index information processing method provided in any embodiment of the present invention.

[0149] Embodiment 6

[0150] Embodiment 6 of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the program is used to execute a file index information processing method, the method comprising:

[0151] Write the target file data to be stored into the target storage resource under the storage management node;

[0152] Determine a target file index sub-table from the target file index resources associated with the target storage resource and write file index information of the target file data to be stored into the target file index sub-table;

[0153] When reading file data, find the preset file index sub-table that is compatible with the target file data to be queried from the preset file index resources associated with the preset storage resources under the storage management node;

[0154] According to the file index information recorded in the matched preset file index sub-table, a data reading operation of the target file to be queried is performed in the preset storage resource associated with the preset file index resource to which the matched preset file index sub-table belongs.

[0155] Optionally, when the program is executed by a processor, it can also be used to execute the file index information processing method provided in any embodiment of the present invention.

[0156] The computer storage medium of the embodiment of the present invention may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable CD-ROM, an optical storage device, a magnetic storage device, or any suitable combination of the above. The computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device.

[0157] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to: electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0158] The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, radio frequency (RF), etc., or any suitable combination of the above.

[0159] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0160] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0161] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A method for processing file index information, characterized in that: The method comprises: Write the target file data to be stored into the target storage resource under the storage management node; Determine a target file index sub-table from the target file index resources associated with the target storage resource and write file index information of the target file data to be stored into the target file index sub-table; When reading file data, find the preset file index sub-table that is compatible with the target file data to be queried from the preset file index resources associated with the preset storage resources under the storage management node; According to the file index information recorded in the adapted preset file index sub-table, a reading operation of the target file data to be queried is performed in the preset storage resource associated with the preset file index resource to which the adapted preset file index sub-table belongs; wherein the preset file index sub-table performs a hash calculation on the file identifier and the number of sub-tables corresponding to the target file data to be queried, and uses the hash processing result as the table number of the file index sub-table, which is determined according to the table number.

2. The method according to claim 1, characterized in that Write the target file data to be stored into the target storage resource under the storage management node, including: In response to a storage request from a metadata management server, a target storage resource is determined from preset data resources associated with a storage management node server; The target file data to be stored is written into the target storage resource.

3. The method according to claim 1, characterized in that Determining a target file index sub-table from the target file index resource associated with the target storage resource includes: Determine a directory index to be stored in the target file index resource for storing file index information and the number of sub-tables of the file index sub-table under the directory index to be stored; Hash the file identifier corresponding to the number of sub-tables and the target file data to be stored; The hash processing result is used as the sub-table serial number of the file index sub-table for storing the file index information of the target file data to be stored under the directory index to be stored, so as to determine the target file index sub-table.

4. The method according to claim 3, characterized in that Determining the number of sub-tables of the file index sub-table under the directory index to be stored includes: Determine the number of sub-tables of the file index sub-table under the previous directory index before the directory index to be stored; Determine the amount of file index information data stored in the last directory index and the amount of file index information data in the last two directory indexes before the directory index to be stored; According to the number of file index sub-tables under the previous directory index and the amount of file index information data stored under the previous directory index and the previous two directory indexes, the number of file index sub-tables under the directory index to be stored is determined to create the file index sub-tables.

5. The method according to claim 1, characterized in that From the preset file index resources associated with the preset storage resources under the storage management node, find the preset file index sub-table that is adapted to the target file data to be queried, including: If it is determined that the directory relationship data table of the preset file index resource associated with the preset storage resource under the storage management node includes the directory index to be queried of the target file data to be queried, then the number of sub-tables of the file index sub-table under the directory index to be queried of the target file data to be queried is determined; Hash the file identifier corresponding to the sub-table quantity and the target file data to be queried; According to the sub-table serial number corresponding to the hash processing result, the preset file index sub-table adapted to the target file data to be queried is searched under the directory index to be queried.

6. A file index information processing device, characterized in that: The device comprises: A module for writing target file data to be stored, used for writing the target file data to be stored into the target storage resource under the storage management node; A target file index sub-table determining module, used to determine a target file index sub-table from the target file index resources associated with the target storage resource and write file index information of the target file data to be stored into the target file index sub-table; A preset file index sub-table acquisition module is used to search for a preset file index sub-table that is compatible with the target file data to be queried from the preset file index resources associated with the preset storage resources under the storage management node when reading file data; The module for reading the target file data to be queried is used to perform a reading operation on the target file data to be queried in the preset storage resource associated with the preset file index resource to which the adapted preset file index sub-table belongs, based on the file index information recorded in the searched adapted preset file index sub-table; wherein the preset file index sub-table performs a hash calculation on the file identifier and the number of sub-tables corresponding to the target file data to be queried, and uses the hash processing result as the table number of the file index sub-table, which is determined according to the table number.

7. The device according to claim 6, characterized in that The target file data to be stored is written into the module including: In response to a storage request from a metadata management server, a target storage resource is determined from preset data resources associated with a storage management node server; The target file data to be stored is written into the target storage resource.

8. The device according to claim 6, characterized in that Determining a target file index sub-table from the target file index resource associated with the target storage resource includes: Determine a directory index to be stored in the target file index resource for storing file index information and the number of sub-tables of the file index sub-table under the directory index to be stored; Hash the file identifier corresponding to the number of sub-tables and the target file data to be stored; The hash processing result is used as the sub-table serial number of the file index sub-table for storing the file index information of the target file data to be stored under the directory index to be stored, so as to determine the target file index sub-table.

9. An electronic device, characterized in that: include: one or more processing devices; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processing devices, the one or more processing devices implement the file index information processing method described in any one of claims 1-5.

10. A readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processing device, a file index information processing method described in any one of claims 1-5 is implemented.

Images