A file storage location testing method, device and readable storage medium
By comparing the storage records and object slices, the accuracy of the file storage location is determined, which solves the problem of inaccurate file storage location confirmation in the prior art and improves the accuracy of file recognition.
Patent Information
- Application Number
- CN202211450771.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-11-18
- Publication Date
- 2026-02-06
- Estimated Expiration
- 2042-11-18
AI Technical Summary
Existing technologies cannot confirm the accuracy of file storage locations, which prevents the optimization of the accuracy of file recognition methods.
By obtaining the storage records of the preset storage pool, the type of the file is identified, and the file type is determined according to the object slice. It is then determined whether the object slice and the storage record type are the same. If they are the same, it indicates that the storage is correct; if they are different, it indicates that the storage is incorrect.
The accuracy of file recognition methods has been improved, enabling accurate determination of file storage locations and optimizing file recognition methods.
Smart Images

Figure CN115756926B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the field of electronic information, and in particular to a file storage location testing method and device and readable storage medium. BACKGROUND
[0002] In recent years, with the rapid development of electronic information technology, data storage has become a popular technical field. High-cost and high-speed devices such as hierarchical storage technology, solid state drives (SSD) disks, and non-volatile memory express (NVME) disks are used to store frequently accessed important information, generally storing small files. Mechanical hard drives (HDD) with lower costs are used to store information with lower access frequency, generally storing large files. Therefore, when storing files, an algorithm is used to identify the size of the file to distinguish between large files and small files, and then save the file in the corresponding location according to the type of the file.
[0003] However, the current scheme cannot confirm whether the file storage location is correct after storing the file, that is, it cannot determine the accuracy of the file identification method, thereby preventing optimization of the file identification method.
[0004] In view of the above technology, how to improve the identification accuracy of the file identification method is a problem to be solved by those skilled in the art. SUMMARY
[0005] The purpose of the present application is to provide a file storage location testing method, device and readable storage medium to improve the identification accuracy of the file identification method.
[0006] To solve the above problems, the present application also provides a file storage location testing method, comprising:
[0007] Obtaining a storage record of an identified file corresponding to a preset storage pool;
[0008] Confirming the type of the identified file according to the storage record, wherein the type of the identified file includes large files and small files, the large files are files with a data volume greater than a threshold value, and the small files are files with a data volume less than the threshold value;
[0009] Determining the type of the identified file according to the object slice of the identified file; wherein the object slice is an object obtained by slicing the identified file when storing;
[0010] Determining whether the type determined according to the object slice is the same as the type determined according to the storage record;
[0011] If they are the same, it indicates that the identification file is stored correctly;
[0012] If they are different, it indicates that the identified file was stored incorrectly.
[0013] Preferably, determining whether the type determined based on the object slice is the same as the type determined by the storage record includes:
[0014] Determine whether the storage pool to which the object slice No. 0 in the information returned from the storage system of the identification file belongs is stored in the preset storage pool, wherein the object slice No. 0 is the first slice among the object slices that are equally divided according to the preset storage size and sorted in order when the identification file is stored;
[0015] If the identified file in the storage record is a small file, and the 0th object slice is stored in the preset storage pool, then it is determined whether the 1st object slice of the identified file exists. If the 1st object slice does not exist, then it is determined that the type of the identified file is the same as the type determined by the storage record. The 1st object slice is the second slice among the object slices that are equally divided according to the preset storage size and sorted in order when the identified file is stored.
[0016] If the identified file in the storage record is a large file, and the 0th object slice is stored in the preset storage pool, then it is determined whether the 1st object slice of the identified file exists and the storage location of the 1st object slice. If the 1st object slice exists and the storage location of the 1st object slice is the preset storage pool, then it is determined that the type of the identified file is the same as the type determined by the storage record.
[0017] Preferably, the storage record is generated by the following steps:
[0018] Mount the shared directory of the storage system and control the shell script to generate files of varying sizes to be stored;
[0019] Record the file size, file name, and corresponding storage pool of the file to be stored, and generate the storage record.
[0020] Preferably, the generation of the object slice includes the following steps:
[0021] After writing the file to be stored into the corresponding storage pool, the storage system CLI command line is invoked to slice the stored file;
[0022] Obtaining stored slice information, wherein the slice is a slice object obtained by dividing the stored file into a plurality of equal sizes according to a preset size, and when the size of the stored file cannot be evenly divided by the preset size, the size of the last slice object is smaller than the preset size, and the object slice is sequentially numbered.
[0023] Preferably, after obtaining the storage result of all the storage files in the preset storage pool, the method further comprises:
[0024] According to each storage result, the accuracy of the storage file in the preset storage pool is obtained, and the identification file with storage error is generated into an error file list.
[0025] Preferably, after obtaining the accuracy of the storage file in the preset storage pool according to each storage result, and generating the error file list of the identification file with storage error, the method further comprises:
[0026] If the accuracy is less than a preset accuracy, it is confirmed that the storage mode of the preset storage pool has a problem, and an alarm is sent so that the user can timely adjust the storage mode of the preset storage pool.
[0027] Preferably, generating the storage record comprises:
[0028] Inquiring the storage record of the storage file, and recording the file size, file name and corresponding storage pool of the file to be stored in the form of a log, and taking the recorded log as the storage record.
[0029] To solve the above problems, the application also provides a test device for file storage position, comprising:
[0030] An acquisition module is configured to acquire the storage record of the corresponding identification file of a preset storage pool;
[0031] A first determination module is configured to determine the type of the identification file according to the storage record, wherein the type of the identification file includes a large file and a small file, the large file is a file with a data amount greater than a threshold value, and the small file is a file with a data amount less than the threshold value;
[0032] A second determination module is configured to determine the type of the identification file according to the object slice of the identification file, wherein the object slice is an object obtained by dividing the identification file during storage;
[0033] A judgment module is configured to judge whether the type determined according to the object slice is the same as the type determined according to the storage record; if the same, it indicates that the identification file is stored correctly; if different, it indicates that the identification file is stored incorrectly.
[0034] To solve the above problems, the application further provides a file storage location testing device, comprising a memory for storing a computer program;
[0035] A processor is configured to execute the computer program to implement the steps of the file storage location testing method.
[0036] To solve the above problems, the application further provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the file storage location testing method.
[0037] The file storage location testing method provided by the application comprises the following steps: obtaining a storage record of a corresponding identification file of a preset storage pool; determining the type of the identification file according to the storage record, wherein the type of the identification file comprises a large file and a small file, the large file is a file with a data amount greater than a threshold value, and the small file is a file with a data amount less than the threshold value; determining the type of the identification file according to an object slice of the identification file, wherein the object slice is an object obtained by slicing the identification file during storage; and determining whether the type determined according to the object slice is the same as the type determined according to the storage record. If the types are the same, it is indicated that the identification file is stored correctly. If the types are different, it is indicated that the identification file is stored incorrectly. In the application, the file is sliced into multiple object slices during storage. The preset size of the slice can be set to the maximum value in the size range of the small file, so that the object slice of the corresponding storage file can be identified to determine whether the file is a small file and whether the storage position of the file is accurate. According to the statistical information, the file identification method can be optimized to improve the accuracy of file identification.
[0038] The file storage location testing device and the computer readable storage medium provided by the application correspond to the file storage location testing method, and have the same beneficial effects. BRIEF DESCRIPTION OF DRAWINGS
[0039] To more clearly illustrate the embodiments of the application, the drawings needed in the embodiments will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.
[0040] Figure 1 A file storage location testing method flowchart provided by an embodiment of the application;
[0041] Figure 2 A testing method flowchart provided by an embodiment of the application;
[0042] Figure 3 A file recognition judgment flowchart provided for an embodiment of the present application;
[0043] Figure 4 A file storage location test device schematic diagram provided for an embodiment of the present application;
[0044] Figure 5 A file storage location test device structural diagram provided for another embodiment of the present application. DETAILED DESCRIPTION
[0045] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present application.
[0046] The core of the present application is to provide a file storage location test method, device and readable storage medium, so as to improve the recognition accuracy of the file recognition method.
[0047] In order for those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the drawings and specific embodiments.
[0048] The current storage test tool can only verify the read-write delay and other performance values of the storage performance, and cannot realize the statistical analysis of the accuracy of the actual writing of the large file recognition to the corresponding storage pool of the large file separation storage system. In order to solve the above technical problems, the present application provides a file storage location test method, Figure 1 A file storage location test method flowchart provided for an embodiment of the present application, as shown in Figure 1 The method comprises the following steps:
[0049] S10: Obtain the storage record of the corresponding identified file of the preset storage pool.
[0050] S11: Confirm the type of the identified file according to the storage record.
[0051] The type of the identified file includes a large file and a small file, the large file is a file with a data amount greater than a threshold, and the small file is a file with a data amount less than the threshold. The type of the identified file in the storage record is the type identified by the original identification method, which may be incorrect. In order to improve the accuracy of the identification scheme, the application needs to obtain the accuracy of the identification method. It should be noted that in this embodiment, the corresponding identified file in the preset storage pool and the storage record are not limited, and the size of the large file and the small file is also not limited. According to the user's needs, the definition of the large file and the small file can be changed, for example, a file with a size less than 2M can be set as a small file, and a file with a size greater than 2M can be set as a large file. In this embodiment, the acquisition method and the generation method of the storage record are not limited.
[0052] S12: Determine the type of the identified file according to the object slice of the identified file.
[0053] The object slice is the object obtained by slicing the identified file when it is stored, and the type determined at this time is the actual type of the identified file.
[0054] S13: Determine whether the type determined according to the object slice is the same as the type determined by the storage record; if the same, go to step S14; if different, go to step S15.
[0055] S14: Determine that the identified file is stored correctly.
[0056] S15: Determine that the identified file is stored incorrectly.
[0057] The type determined by the object slice is compared with the type determined by the storage record. If they are the same, it means that the identified file is stored correctly, and the file identification is correct. This embodiment realizes the automation of the storage system large file identification accuracy calculation, and provides data support for the optimization and improvement of the large file identification algorithm in the storage system. By using the implementation logic of the large file separation storage technology, by recording the file name and size information written by the test tool, and by comparing the storage pool information of the file 0 and 1 objects obtained from the storage system, the difference between the number of large files and small files expected to be written into the storage and the number of large files and small files identified in the actual cluster is analyzed. The identification accuracy of the large file and the small file is output in detail, and the detailed list information of the un-identified file is output.
[0058] Specifically, the determining whether the type determined according to the object slice is same as the type determined according to the storage record includes: determining whether the storage pool to which the 0th object slice in the returned information of the identification file from the storage system belongs is stored in the preset storage pool, the 0th object slice being the first slice in the object slices of the identification file which are equally divided according to the preset storage size and sorted in order when the identification file is stored. If the identification file in the storage record is a small file, and the 0th object slice is stored in the preset storage pool, it is determined whether there is a 1st object slice of the identification file, if there is no 1st object slice, it is determined that the type of the identification file is same as the type determined according to the storage record, the 1st object slice being the second slice in the object slices of the identification file which are equally divided according to the preset storage size and sorted in order when the identification file is stored. If the identification file in the storage record is a large file, and the 0th object slice is stored in the preset storage pool, it is determined whether there is a 1st object slice of the identification file and the storage location of the 1st object slice, if there is a 1st object slice and the storage location of the 1st object slice is the preset storage pool, it is determined that the type of the identification file is same as the type determined according to the storage record. It can be understood that if the equal division size of the object slice is the maximum value in the preset small file size range, all files with only the 0th object slice can be regarded as small files, and all files containing the 1st object slice can be regarded as large files.
[0059] The embodiment aims to continuously write data to the storage on the client side by designing a test method, compare the storage pool information of the file in the metadata information of the written file obtained from the storage system with the expected storage pool information of the file written by the client, and finally output the size file recognition accuracy rate result of the storage system after a plurality of client batch mixed files are written to the storage system. Through the optimization of the test method, the size file recognition accuracy rate problem in the previous test scheme cannot be counted when a large batch of mixed files are written to the storage system, and then the result analysis is used to promote the optimization and improvement of the size file recognition algorithm of the storage system. The embodiment can mount the NFS shared directory of the storage system under the linux client, realize the cooperative writing of data by multiple clients through the Remote Shell (RSH) service, and count the written data information. At the same time, the storage pool information of the 0th object and the 1st object of the file slice information written into the storage system is obtained through the CLI command line of the storage system. The scheme can also analyze the files not normally recognized in the written file by comparing the expected storage pool of the file with the actual storage pool of the 0th object and the 1st object, and finally output the size file failure rate and the detailed list of failed files.
[0060] The test method for the file storage location provided in the embodiments of the present application comprises the following steps: obtaining the storage record of the corresponding identification file of the preset storage pool, then confirming the type of the identification file according to the storage record, the type of the identification file comprises a large file and a small file, the large file is a file with a data volume greater than a threshold value, and the small file is a file with a data volume less than the threshold value; determining the type of the identification file according to the object slice of the identification file; the object slice is an object after the identification file is cut when being stored; finally, judging whether the type determined according to the object slice is the same as the type determined according to the storage record; if yes, it indicates that the identification file is stored correctly; if no, it indicates that the identification file is stored incorrectly. In the embodiments of the present application, the storage file is cut into multiple object slices by using the slice function when being stored, the preset size of the slice can be set as the maximum value in the size range of the small file, so that when the identification file is identified, only the object slice of the corresponding storage file needs to be identified, and whether the file is a small file can be identified at the same time, so as to judge whether the storage location of the file is accurate, according to the statistical information, the file identification mode can be optimized, and the accuracy of file identification can be improved.
[0061] Figure 2 A test method flow chart is provided in the embodiments of the present application; as shown in Figure 2 After a plurality of clients mount the shared directory of the storage system, the RSH passwordless access is uniformly controlled by a master client. The master client calls the Shell script to generate the mixed files of different sizes (for example, 20,000 4k files+20,000 1M files), and the client records the file name and the corresponding file size. After the data is written into the storage system, the client calls the storage system CLI command line to obtain the slice information of the file storage, and filters out the storage pool information to which the 0th object slice and the 1st object slice of the file belong. The client compares the record information of the written file with the storage pool information obtained from the 0th object slice and the 1st object slice of the file, analyzes the small files and the large files that are successfully identified, generates the small file identification accuracy and the large file identification accuracy, and outputs the list of the small files and the large files that are not successfully identified.
[0062] Figure 3 A file identification judgment flow chart is provided in the embodiments of the present application; as shown in Figure 3As shown, the client identifies whether the file is a small file from the data generation record. If the file is a small file, it is determined whether the storage pool to which the 0th object corresponding to the file returned from the storage system belongs is an SSD storage pool, and the result is no, and the small file identification fails. The result is yes, and it is further determined whether the 1st object exists, and the result is yes, and the small file identification fails. If the 1st object does not exist, the result is that the small file identification succeeds. If the file is a large file, it is determined whether the storage pool to which the 0th object corresponding to the file returned from the storage system belongs is an HDD storage pool, and the result is no, and the large file identification fails. The result is yes, and it is further determined whether the 1st object exists, and the result is no, and the large file identification succeeds. The result is yes, and it is further determined whether the storage pool to which the 1st object belongs is an HDD, and the result is no, and the large file identification fails, and the result is yes, and the large file identification succeeds. The identification success means that the actual type of the identified file is the same as the type determined by the storage record, that is, the identified file is stored correctly.
[0063] In addition, in actual application, the generation step of the storage record includes: mounting the shared directory of the storage system, and controlling the shell script to generate the mixed size to-be-stored file; recording the file size, file name and corresponding storage pool of the to-be-stored file, and generating the storage record. The user interface and command line are the additionally developed program, which is the "agent" at this level. In the virtual machine, the command line program is called Shell. Shell is an application program that connects users and virtual machine kernels, allowing users to use virtual machine kernels more efficiently, safely and at a lower cost, which is the essence of Shell. It should be noted that the script in the shell script is a pre-written program, and the generation and writing of the script are not specifically limited in this embodiment.
[0064] The generation method of the object slice in the above embodiment is not limited, and the preferred scheme is provided as follows. The generation of the object slice includes the following steps: after the to-be-stored file is written into the corresponding storage pool, the storage system CLI command line is called to slice the stored file; the sliced information is obtained, the slice is a plurality of slice objects with equal size obtained by dividing the stored file according to a preset size, when the size of the stored file cannot be divided by the preset size, the size of the last slice object is smaller than the preset size, and the object slices are numbered in sequence.
[0065] Solid State Drives (SSD), referred to as solid disk, is made of solid-state electronic storage chip array, which is composed of control unit and storage unit. The solid state disk is completely the same as the traditional hard disk in the specification and definition of interface, function and use method, and is completely consistent with the traditional hard disk in product shape and size, but the I / O performance is greatly improved compared with the traditional hard disk. Due to the higher cost, it is generally used to store important small files. Hard disk drive (HDD) is generally used for desktop applications, with large capacity and relatively low price, suitable for home hard disk, generally used for storing large files. In the above embodiment, the type of the preset storage pool is not limited, and in actual application, the preset storage pool includes an SSD pool and an HDD pool. For example, in this embodiment, the preset storage pool is at least two, one is an SSD pool and one is an HDD pool. Due to the difference between the performance and cost of the two, small files are generally stored in the SSD pool and large files are stored in the HDD pool.
[0066] In order to facilitate analysis, after obtaining the storage results of all the storage files in the preset storage pool, the accuracy of the storage files in the preset storage pool is obtained according to the storage results, and the identification files with storage errors are generated into an error file list. It should be noted that the accuracy of the storage files provided in this embodiment is obtained by comparing the record information of the written file with the storage pool information of the file 0 object slice and the file 1 object slice obtained from the storage pool on the client, analyzing the small files and large files that are successfully identified, generating small file identification accuracy and large file identification accuracy, and outputting the list of small files and large files that are not successfully identified.
[0067] Considering the adjustment of the final storage, a preferred solution is provided. After obtaining the accuracy of the storage files in the preset storage pool according to the storage results, and generating the error file list of the identification files with storage errors, if the accuracy is less than the preset accuracy, it is confirmed that there is a problem with the storage mode of the preset storage pool, and an alarm is sent to the user to adjust the storage mode of the preset storage pool in time.
[0068] In addition, the storage record generated in actual application can be obtained by querying the storage record of the storage file, and the file size, file name and corresponding storage pool of the file to be stored are recorded in the form of log, and finally the recorded log is used as the storage record.
[0069] In the above embodiment, the test method of the file storage position is described in detail, and the application also provides a corresponding embodiment of the test device of the file storage position. It should be noted that the embodiments of the device part are described from two angles, one is based on the function module, and the other is based on the hardware.
[0070] Based on the function module, the embodiment provides a file storage location testing device, Figure 4 A file storage location testing device provided by the embodiment of the application is shown in a schematic diagram, and the device comprises:
[0071] The acquisition module 10 is configured to acquire a storage record of a corresponding identification file of a preset storage pool.
[0072] The first determination module 11 is configured to determine the type of the identification file according to the storage record, wherein the type of the identification file comprises a large file and a small file, the large file is a file with a data volume greater than a threshold value, and the small file is a file with a data volume less than the threshold value.
[0073] The second determination module 12 is configured to determine the type of the identification file according to an object slice of the identification file, wherein the object slice is an object obtained by slicing the identification file when the identification file is stored.
[0074] The judgment module 13 is configured to determine whether the type determined according to the object slice is the same as the type determined according to the storage record, if the types are the same, it is determined that the identification file is stored correctly, and if the types are different, it is determined that the identification file is stored incorrectly.
[0075] Since the embodiments of the device part correspond to the embodiments of the method part, the embodiments of the device part are described in the description of the embodiments of the method part, and will not be described here.
[0076] The file storage location testing device provided by the embodiment corresponds to the above method, and therefore has the same beneficial effects as the above method.
[0077] Based on the hardware, the embodiment provides another file storage location testing device, Figure 5 A structural diagram of a file storage location testing device provided by another embodiment of the application is shown in FIG. 2, and the file storage location testing device comprises: Figure 5 A memory 20 for storing a computer program.
[0078] A processor 21 for executing the computer program to realize the steps of the file storage location testing method mentioned in the above embodiments.
[0079] The file storage location testing device provided by the embodiment can include but is not limited to a smart phone, a tablet computer, a notebook computer, a desktop computer, and the like.
[0080] The processor 21 can include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 can be implemented in at least one of a hardware form of a digital signal processor (DSP), a field-programmable gate array (FPGA), a programmable logic array (PLA), etc. The processor 21 can also include a main processor and a coprocessor. The main processor is a processor for processing data in a wake-up state, also referred to as a central processing unit (CPU). The coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 21 can be integrated with a graphics processor (GPU) for rendering and drawing content required to be displayed on the display screen. In some embodiments, the processor 21 can further include an artificial intelligence (AI) processor for processing computing operations related to machine learning.
[0081] The memory 20 can include one or more computer-readable storage media, which can be non-transitory. The memory 20 can further include a high-speed random access memory, and a nonvolatile memory such as one or more disk storage devices, flash storage devices. In the present embodiment, the memory 20 is at least used to store the following computer program 201, wherein the computer program is loaded and executed by the processor 21, and can implement the related steps of the file storage location testing method disclosed in any of the preceding embodiments. In addition, the resources stored in the memory 20 can further include an operating system 202 and data 203, etc., and the storage mode can be temporary storage or permanent storage. The operating system 202 can include Windows, Unix, Linux, etc. The data 203 can include but is not limited to data involved in the file storage location testing method, etc.
[0082] In some embodiments, the file storage location testing device can further include a display screen 22, an input / output interface 23, a communication interface 24, a power supply 25, and a communication bus 26.
[0083] Those skilled in the art can understand that, Figure 5 The structure shown in the above-mentioned embodiments does not constitute a limitation on the file storage location testing device, and can include more or fewer components than those shown in the drawings.
[0084] The file storage location testing device provided in the embodiment of the present application comprises a memory and a processor. When the processor executes the program stored in the memory, the following method can be realized: the file storage location testing method involved in the above embodiment.
[0085] Since the embodiments of the device part correspond to the embodiments of the method part, the embodiments of the device part are described in the description of the embodiments of the method part, and are not described here.
[0086] The file storage location testing device provided in the embodiment of the present application corresponds to the above method, and therefore has the same beneficial effects as the above method.
[0087] Finally, the present application also provides an embodiment corresponding to a computer readable storage medium. The computer readable storage medium stores a computer program. When the computer program is executed by a processor, the steps recorded in the above method embodiments are realized.
[0088] It can be understood that if the method in the above embodiment is realized in the form of a software function unit and is sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and executes all or part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, and various media that can store program codes.
[0089] Since the embodiments of the readable storage medium part correspond to the embodiments of the method part, the embodiments of the device part are described in the description of the embodiments of the method part, and are not described here.
[0090] The computer readable storage medium provided in the embodiment of the present application corresponds to the above method, and therefore has the same beneficial effects as the above method.
[0091] The above introduces in detail a file storage location test method, device and computer readable storage medium provided by the present application. The embodiments in the specification are described in a progressive manner, and each embodiment mainly explains the difference from other embodiments. The same or similar parts of each embodiment can be understood by referring to each other. For the device disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the related parts can be understood by referring to the method part. It should be pointed out that for ordinary skilled in the art, without departing from the principles of the present application, the present application can be improved and modified, and these improvements and modifications also fall within the protection scope of the claims of the present application.
[0092] It should also be noted that in the present specification, the relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the existence of other same elements in the process, method, article or device including the element.
Claims
1. A method of testing a file storage location, characterized by, The method comprises the following steps: acquiring a storage record of a corresponding identification file of a preset storage pool; confirming the type of the identification file according to the storage record, wherein the type of the identification file comprises a large file and a small file, the large file is a file with a data volume greater than a threshold value, and the small file is a file with a data volume less than the threshold value; determining the type of the identification file according to an object slice of the identification file, wherein the object slice is an object after the identification file is cut when stored; judging whether the type determined according to the object slice is the same as the type determined according to the storage record; if they are the same, determining that the identification file is stored correctly; if they are different, determining that the identification file is stored incorrectly; the judgment of whether the type determined according to the object slice is the same as the type determined according to the storage record comprises: judging whether the 0th object slice of the identification file in the information returned from the storage system belongs to the preset storage pool, wherein the 0th object slice is the first slice in the object slices of the identification file which are equally cut according to a preset storage size and sorted in order when stored; if the identification file is a small file in the storage record, and the 0th object slice is stored in an SSD storage pool, judging whether there is a 1st object slice of the identification file, if there is no 1st object slice, determining that the type of the identification file is the same as the type determined according to the storage record, wherein the 1st object slice is the second slice in the object slices of the identification file which are equally cut according to the preset storage size and sorted in order when stored; if the identification file is a large file in the storage record, and the 0th object slice is stored in an HDD storage pool, judging whether there is a 1st object slice of the identification file and the storage location of the 1st object slice, if there is the 1st object slice and the storage location of the 1st object slice is an HDD storage pool, determining that the type of the identification file is the same as the type determined according to the storage record.
2. The method of testing file storage locations of claim 1, wherein, The storage record is generated by the following steps: mounting a shared directory of a storage system, and controlling a shell script to generate a to-be-stored file with mixed sizes; recording the file size, file name and corresponding storage pool of the to-be-stored file, and generating the storage record.
3. The method of testing file storage locations of claim 2, wherein, The generation of the object slice comprises the following steps: after the to-be-stored file is written into the corresponding storage pool, a storage system CLI command line is called to cut the stored file; acquiring the stored slice information, wherein the slice is a plurality of slice objects with equal sizes obtained by cutting the stored file according to a preset size, when the size of the stored file cannot be divided by the preset size, the size of the last slice object is less than the preset size, and the object slices are numbered in order.
4. The method of testing file storage locations according to any one of claims 1 to 3, wherein, after all storage results of the storage files in the preset storage pool are acquired, the following steps are further included: according to each storage result, obtaining the accuracy rate of the storage files in the preset storage pool, and generating an error file list of the identification files with storage errors.
5. The method of testing file storage locations of claim 4, wherein, After the accuracy of the stored files in the preset storage pool is obtained according to the respective storage results, and the error file list of the identified files with storage errors is generated, the method further comprises: If the accuracy is less than a preset accuracy, it is determined that there is a problem with the storage mode of the preset storage pool, and an alarm is sent so that a user can timely adjust the storage mode of the preset storage pool.
6. The method of testing file storage locations of claim 2, wherein, The storage record of the identified file is obtained from the preset storage pool, and the storage record of the identified file is generated. The storage record of the identified file is queried, and the file size, file name and corresponding storage pool of the file to be stored are recorded in the form of a log, and the recorded log is taken as the storage record.
7. A test device for a file storage location, characterized by Comprise: The acquisition module is configured to acquire the storage record of the corresponding identified file of the preset storage pool; The first determination module is configured to determine the type of the identified file according to the storage record, wherein the type of the identified file comprises a large file and a small file, the large file is a file with a data volume greater than a threshold value, and the small file is a file with a data volume less than the threshold value; The second determination module is configured to determine the type of the identified file according to the object slice of the identified file; wherein the object slice is an object after the identified file is cut during storage; The judging module is configured to determine whether the type determined according to the object slice is the same as the type determined according to the storage record; if they are the same, it is determined that the identified file is stored correctly; if they are different, it is determined that the identified file is stored incorrectly; The judging module is specifically configured to: Determine whether the 0th object slice of the identified file in the return information from the storage system belongs to the preset storage pool, wherein the 0th object slice is the first slice in the object slices of the identified file which are equally cut according to a preset storage size and sorted in order during storage; If the identified file in the storage record is a small file, and the 0th object slice is stored in an SSD storage pool, it is determined whether there is a 1st object slice of the identified file; if there is no 1st object slice, it is determined that the type of the identified file is the same as the type determined according to the storage record, wherein the 1st object slice is the second slice in the object slices of the identified file which are equally cut according to a preset storage size and sorted in order during storage; If the identified file in the storage record is a large file, and the 0th object slice is stored in an HDD storage pool, it is determined whether there is a 1st object slice of the identified file and the storage location of the 1st object slice; if there is the 1st object slice and the storage location of the 1st object slice is an HDD storage pool, it is determined that the type of the identified file is the same as the type determined according to the storage record.
8. A test device for a file storage location, characterized by The memory is configured to store a computer program; The processor is configured to execute the computer program to implement the steps of the file storage location test method according to any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the file storage location test method according to any one of claims 1 to 6.
Citation Information
Patent Citations
Log file output method, system and device and readable storage medium
CN111125467A
Distributed storage method and device and medium
CN114756523A