File integrity verification method and device and computing device cluster

Abstract

The file integrity checking method comprises the steps that identifiers of N (N > = 1) file fragments and a first checksum of all the file fragments in the N file fragments are obtained from a server side, and the N file fragments are file fragments randomly sampled from a to-be-checked target file by the server side; based on the identifiers of the N file fragments, obtaining the N file fragments from the target file; determining a checksum of each file fragment in the obtained N file fragments to obtain N second checksums; and determining the integrity of the target file based on the N first checksums and the N second checksums. Therefore, when the integrity of the file is checked, the integrity of the whole file can be quickly obtained only by checking the checksum of the random file fragments, and the efficiency of checking the integrity of the file is improved. And meanwhile, the detected file fragments are randomly sampled, so that the file fragments are not easily tampered, and the security of the file is improved.

Description

Technical Field

[0001] The present application relates to the field of information technology (IT) technology, and in particular to a file integrity verification method, device and computing device cluster. Background Art

[0002] When files are being transmitted or stored, it is common practice to calculate the checksum of the entire file and then perform an integrity check. Regardless of the size of the file, a comparison is required to ensure the integrity of the file content and that it has not been tampered with. However, when the file is large, this verification method is very time-consuming and consumes a lot of network bandwidth and computing resources. It is basically impossible to complete the integrity check of massive large files. Summary of the invention

[0003] The present application provides a file integrity verification method, apparatus, computing device cluster, computer storage medium and computer product, which can improve the efficiency of file integrity verification.

[0004] In a first aspect, the present application provides a file integrity verification method, comprising: obtaining identifiers of N (N≥1) file fragments and a first checksum of each of the N file fragments from a server, wherein the N file fragments are file fragments randomly sampled by the server from a target file to be verified; based on the identifiers of the N file fragments, obtaining the N file fragments from the target file; determining the checksum of each of the N file fragments obtained to obtain N second checksums; and determining the integrity of the target file based on the N first checksums and the N second checksums.

[0005] In this way, when verifying the integrity of a file, you only need to verify the checksum of a random file fragment to quickly know the integrity of the entire file, which improves the efficiency of file integrity verification. At the same time, the verified file fragments are randomly sampled, so they are not easily tampered with, which improves the security of the file.

[0006] In a possible implementation, the integrity of the target file is determined based on the N first checksums and the N second checksums, including: if the first checksum of the first file segment is inconsistent with the second checksum of the first file segment, the target file is determined to be incomplete, and the first file segment is any one of the N file segments; if the first checksum of each file segment in the N file segments is consistent with the corresponding second checksum, the target file is determined to be complete. In this way, by comparing the first checksum and the second checksum of the file segment, the integrity of the target file can be known.

[0007] In a possible implementation, the N file fragments include the first file fragment. In this case, the method further includes: obtaining the first data amount contained in the first file fragment from the server; determining that the target file is incomplete when the first data amount contained in the first file fragment is inconsistent with the second data amount contained in the first file fragment, wherein the second data amount is the data amount contained in the first file fragment obtained from the target file; and determining whether the first checksum of the first file fragment is consistent with the second checksum of the first file fragment when the first data amount contained in the first file fragment is consistent with the second data amount contained in the first file fragment. In this way, when the first data amount and the second data amount are inconsistent, it can be quickly known that the target file is incomplete without the need to perform subsequent processes, thereby improving the efficiency of file integrity verification.

[0008] In a possible implementation, when N=1 or N=2, the file segment is a segment excluding the header content and the tail content of the target file.

[0009] In a possible implementation, the identifier of the file segment is the byte offset of the file segment.

[0010] In a second aspect, the present application provides a file integrity verification device, including: an acquisition module and a processing module. The acquisition module is used to obtain the identifiers of N (N≥1) file fragments and the first checksum of each of the N file fragments from the server, wherein the N file fragments are file fragments randomly sampled by the server from the target file to be verified, and, based on the identifiers of the N file fragments, the N file fragments are obtained from the target file. The processing module is used to determine the checksum of each of the N file fragments obtained to obtain N second checksums; and, based on the N first checksums and the N second checksums, determine the integrity of the target file.

[0011] In a possible implementation, when the processing module determines the integrity of the target file based on N first checksums and N second checksums, it is specifically used to: determine that the target file is incomplete when the first checksum of the first file fragment is inconsistent with the second checksum of the first file fragment, and the first file fragment is any one of the N file fragments; and determine that the target file is complete when the first checksum of each file fragment in the N file fragments is consistent with the corresponding second checksum.

[0012] In a possible implementation, the N file fragments include a first file fragment. In this case, the acquisition module is further used to obtain a first amount of data contained in the first file fragment from the server. The processing module is further used to determine that the target file is incomplete when the first amount of data contained in the first file fragment is inconsistent with the second amount of data contained in the first file fragment, wherein the second amount of data is the amount of data contained in the first file fragment obtained from the target file. The processing module is further used to determine whether a first checksum of the first file fragment is consistent with a second checksum of the first file fragment when the first amount of data contained in the first file fragment is consistent with the second amount of data contained in the first file fragment.

[0013] In a possible implementation, when N=1 or N=2, the file segment is a segment excluding the header content and the tail content of the target file.

[0014] In a possible implementation, the identifier of the file segment is the byte offset of the file segment.

[0015] In a third aspect, the present application provides a computing device cluster, comprising at least one computing device, each computing device comprising a processor and a memory; the processor of at least one computing device is used to execute instructions stored in the memory of at least one computing device, so that the computing device cluster performs the method described in the first aspect or any possible implementation of the first aspect.

[0016] In a fourth aspect, the present application provides a computer-readable storage medium, including computer program instructions, when the computer program instructions are executed by a computing device, the computing device executes the method described in the first aspect or any possible implementation of the first aspect; or, when the computer program instructions are executed by a computing device cluster, the computing device cluster executes the method described in the first aspect or any possible implementation of the first aspect. Exemplarily, the computing device cluster may include one or more computing devices.

[0017] In a fifth aspect, the present application provides a computer program product comprising instructions, which, when executed by a computing device, causes the computing device to perform the method described in the first aspect or any possible implementation of the first aspect, or, when executed by a computing device cluster, causes the computing device cluster to perform the method described in the first aspect or any possible implementation of the first aspect. Exemplarily, the computing device cluster may include one or more computing devices.

[0018] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1It is a schematic diagram of the architecture of a file integrity verification system provided by an embodiment of the present application;

[0020] Figure 2 It is a flowchart of a file integrity verification method provided by an embodiment of the present application;

[0021] Figure 3 It is a schematic diagram of a process for determining file integrity provided by an embodiment of the present application;

[0022] Figure 4 It is a structural schematic diagram of a file integrity verification device provided by an embodiment of the present application;

[0023] Figure 5 is a schematic diagram of the structure of a computing device provided in an embodiment of the present application;

[0024] Figure 6 is a schematic diagram of the structure of a computing device cluster provided in an embodiment of the present application;

[0025] Figure 7 It is a structural diagram of another computing device cluster provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] The term "and / or" in this article is a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The symbol " / " in this article indicates that the associated objects are in an or relationship, for example, A / B means A or B.

[0027] The terms "first" and "second" in the specification and claims herein are used to distinguish different objects rather than to describe a specific order of the objects. For example, a first response message and a second response message are used to distinguish different response messages rather than to describe a specific order of the response messages.

[0028] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0029] In the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more than two. For example, multiple processing units refer to two or more processing units, etc.; multiple elements refer to two or more elements, etc.

[0030] For example, Figure 1 FIG. 1 is a schematic diagram showing the architecture of a file integrity verification system provided by an embodiment of the present application. Figure 1 As shown, the file integrity verification system 100 may include: a file storage end 110 , a sampling and verification and calculation end 120 , and a file integrity verification end 130 .

[0031] Among them, the file storage end 110 is mainly used to store one end of the file. The file storage end 110 is usually deployed on a server (such as a cloud server or a physical server, etc.), and the server has certain computing and storage resources. Exemplarily, the stored files can be various different types of data, which can be read, written and processed by the computer system. The types of files can include: text files (i.e. files containing readable text content, such as files in formats such as TXT, CSV, XML, JSON, etc.), image files (i.e. files containing image data, such as files in formats such as JPEG, PNG, GIF, etc.), audio files (i.e. files containing audio data, such as files in formats such as MP3, WAV, FLAC, etc.), video files (i.e. files containing video data, such as files in formats such as MP4, AVI, MOV, etc.), compressed files, executable files (i.e. files containing executable code), database files (i.e. files containing structured data, such as files of databases such as SQLite, MySQL, MongoDB, etc.), log files (i.e. files containing log information generated by systems, applications or services) or configuration files (i.e. files containing application or system configuration information, such as files in formats such as INI, YAML, JSON, etc.). Of course, the files may also include other types of files, such as font files, CAD files, document files, spreadsheet files, etc.

[0032] The sampling and checksum calculation end 120 is mainly used to randomly sample files from the file storage end 110 and calculate the checksum of the sampled file fragments. The sampling and checksum calculation end 120 is usually deployed on a server (such as a cloud server or a physical server, etc.), which has certain computing and storage resources. In this embodiment, the sampling and checksum calculation end 120 can randomly sample files from the file storage end 110 to obtain at least one file fragment. At the same time, the sampling and checksum calculation end 120 can record the identification of each file fragment, such as the address of the data block contained in the file fragment or the location of the file fragment in the disk. In addition, the sampling and checksum calculation end 120 can also record the amount of data contained in each file fragment, that is, record the size of each file fragment. The sampling and checksum calculation end 120 can also calculate the checksum of each sampled file fragment through a checksum calculation algorithm (such as MD5 (message digest algorithm 5), SHA-1 (secure hashalgorithm 1), SHA-256 (secure hash algorithm 256) or Adler-32, etc.). In some embodiments, when the number of sampled file fragments is less than or equal to two, the sampled file fragments may not include the header content and / or the tail content of the file. In some embodiments, the server deploying the sampling and checksum calculation end 120 and the server deploying the file storage end 110 may be the same server or different servers. When both are deployed on the same server, the checksum of the file fragment calculated by the sampling and checksum calculation end 120 can be stored separately, that is, stored separately from the file stored in the file storage end 110, thereby preventing the checksum from being tampered with and improving the security of file storage.

[0033] The file integrity check end 130 is mainly used to check the integrity of the files stored in the file storage end 110. When it is necessary to perform integrity check on a file stored in the file storage end 110, the file integrity check end 130 can first obtain the identifier and checksum of the file fragment sampled by the sampling and checksum calculation end 120 from the file to be checked from the sampling and checksum calculation end 120. Then, the file integrity check end 130 can obtain each file fragment from the file storage end 110 based on the identifier of the file fragment obtained from the sampling and checksum calculation end 120, and calculate the checksum of these file fragments respectively through the checksum calculation algorithm. Finally, the file integrity check end 130 can compare the checksum of each file fragment calculated by it with the checksum of the corresponding file fragment obtained from the sampling and checksum calculation end 120. When the checksum of a file fragment calculated by the file integrity check end 130 is inconsistent with the checksum of the file fragment obtained from the sampling and checksum calculation end 120, it can be determined that the file to be checked is incomplete, that is, the file is damaged. When the checksum of the file segment calculated by the file integrity check end 130 is consistent with the checksum of the corresponding file segment obtained from the sampling and checksum calculation end 120, it can be determined that the file to be checked is complete.

[0034] For example, when the file segments sampled by the sampling and checksum calculation end 120 are file segment 1 and file segment 2, the checksum of file segment 1 is checksum a1, and the checksum of file segment 2 is checksum b1, if the checksum of file segment 1 calculated by the file integrity check end 130 is a1, and the checksum of file segment 2 is checksum b1, it can be determined that the file to be checked is complete. If the checksum of file segment 1 calculated by the file integrity check end 130 is a2, and the checksum of file segment 2 is checksum b1, it can be determined that the file to be checked is incomplete. If the checksum of file segment 1 calculated by the file integrity check end 130 is a1, and the checksum of file segment 2 is checksum b2, it can also be determined that the file to be checked is incomplete. If the checksum of file segment 1 calculated by the file integrity check end 130 is a2, and the checksum of file segment 2 is checksum b2, it can also be determined that the file to be checked is incomplete.

[0035] In some embodiments, the file integrity check end 130 may also obtain the amount of data contained in each file fragment from the sampling and checksum calculation end 120. Before comparing the checksum, the file integrity check end 130 may first compare the amount of data contained in each file fragment obtained by it with the amount of data contained in the corresponding file fragment obtained from the sampling and checksum calculation end 120 to see if they are the same. When the amount of data contained in a file fragment obtained by the file integrity check end 130 is different from the amount of data contained in the file fragment obtained from the sampling and checksum calculation end 120, it can be determined that the file to be checked is incomplete, that is, the file is damaged. When the amount of data contained in each file fragment obtained by the file integrity check end 130 is the same as the amount of data contained in the corresponding file fragment obtained from the sampling and checksum calculation end 120, the checksums of the file fragments can be compared. In this way, the efficiency of file integrity verification is improved.

[0036] For example, when the file segments sampled by the sampling and checksum calculation end 120 are file segment 1 and file segment 2, the amount of data contained in file segment 1 is m11, and the amount of data contained in file segment 2 is m12, the amount of data contained in file segment 1 obtained by the file integrity check end 130 is m21, and the amount of data contained in file segment 2 is m22. If m11 is different from m21, and / or m12 is different from m22, it can be determined that the file to be checked is complete. If m11 is the same as m21, and m12 is the same as m22, the checksum of file segment 1 and the checksum of file segment 2 can be determined, so as to determine whether the file to be checked is complete through the checksum.

[0037] In some embodiments, the file integrity check end 130 may also be a terminal for downloading files from the file storage end 110. In this case, the user may download the file required by the user from the file storage end 110 through the file integrity check end 130. After completing the file download, the file integrity check end 130 may obtain each file segment from the downloaded file through the identifier of the file segment obtained from the sampling and checksum calculation end 120, and then calculate the checksum of each file segment respectively.

[0038] The above is an introduction to the file integrity verification system 100 provided in the embodiment of the present application. Next, based on the above content, a file integrity verification method provided in the embodiment of the present application is introduced.

[0039] For example, Figure 2The flowchart of a file integrity verification method provided by an embodiment of the present application is shown. It can be understood that the method can be executed by any device, equipment, platform, or device cluster with computing and processing capabilities. In addition, the method can also be executed by any client, such as a client for downloading files. For ease of description, the following is an introduction using server execution as an example. Figure 2 As shown, the file integrity verification method may include the following steps:

[0040] S201. Obtain identifiers of N (N≥1) file segments and a first checksum of each of the N file segments from a server, wherein the N file segments are file segments randomly sampled by the server from a target file to be checked.

[0041] In this embodiment, when it is necessary to perform an integrity check on the target file to be checked, such as when the target file is downloaded locally or when the check period is reached, etc., the identifiers of N (N≥1) file fragments and the first checksum of each of the N file fragments can be obtained from the server. Among them, the N file fragments are file fragments randomly sampled from the target file by the server. A file fragment is associated with a first checksum. Exemplarily, the identifier of the file fragment can be the byte offset of the file fragment. In some embodiments, when N=1 or N=2, the file fragment is a fragment excluding the header content and the tail content of the target file. Exemplarily, the server can be the aforementioned Figure 1 The sampling, checking and calculation end 120 in the embodiment.

[0042] S202: Based on the identifiers of the N file segments, obtain the N file segments from the target file.

[0043] In this embodiment, after obtaining the identifiers of the N file fragments, the N file fragments can be obtained from the target file based on the identifiers of the N file fragments. For example, when verifying a file stored in a server, the server can be communicated with and the identifiers of the N file fragments can be sent to the server to obtain the N file fragments from the server; when verifying a file downloaded locally, the N file fragments can be extracted from the file downloaded locally based on the identifiers of the N file fragments.

[0044] S203: Determine the checksum of each file segment in the acquired N file segments to obtain N second checksums.

[0045] In this embodiment, after obtaining N file segments, each of the N file segments may be processed by a checksum calculation algorithm to obtain N second checksums, wherein one file segment is associated with one second checksum.

[0046] S204: Determine the integrity of the target file based on the N first checksums and the N second checksums.

[0047] In this embodiment, after obtaining N first checksums and N second checksums, these checksums may be compared to determine the integrity of the target file.

[0048] As a possible implementation, Figure 3 As shown, determining the integrity of the target file may include the following steps: S301, determining whether the first checksum of the i-th file fragment is consistent with the second checksum of the i-th file fragment, and the initial value of i is 1. By comparing the two checksums, it can be determined whether the two are consistent. When the two are inconsistent, S302 can be executed, that is, determining that the target file is incomplete. When the two are consistent, S303 can be executed, that is, setting i to i+1, and determining whether the value of i is greater than N. If i is less than or equal to N, return to execute S301. If i is greater than N, execute S304, that is, determine that the target file is complete.

[0049] As another possible implementation, it may be determined in parallel whether the first checksums of multiple file segments are consistent with the corresponding second checksums to determine the integrity of the target file.

[0050] In this way, when verifying the integrity of a file, you only need to verify the checksum of a random file fragment to quickly know the integrity of the entire file, which improves the efficiency of file integrity verification. At the same time, the verified file fragments are randomly sampled, so they are not easily tampered with, which improves the security of the file.

[0051] In some embodiments, before verifying whether the first checksum and the second checksum of a certain file fragment (hereinafter referred to as the "first file fragment") are consistent, the first data amount contained in the first file fragment can also be obtained from the server. Then, it is determined whether the second data amount contained in the first file fragment extracted from the target file is consistent with the first data amount. If they are inconsistent, it is determined that the target file is incomplete. If the two are consistent, it can be further determined whether the first checksum and the second checksum of the first file fragment are consistent. In this way, when the first data amount and the second data amount are inconsistent, it can be quickly learned that the target file is incomplete without performing subsequent processes, thereby improving the efficiency of file integrity verification.

[0052] It is understandable that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0053] The above is an introduction to the file integrity verification method provided by the embodiment of the present application. Next, based on the method in the above embodiment, a file integrity verification device provided by the embodiment of the present application is introduced.

[0054] For example, Figure 4 A schematic diagram of the structure of a file integrity verification device provided by an embodiment of the present application is shown. Figure 4 As shown, the file integrity verification device 400 may include: an acquisition module and a processing module. The acquisition module is used to obtain the identifiers of N (N≥1) file fragments and the first checksum of each file fragment in the N file fragments from the server, wherein the N file fragments are file fragments randomly sampled by the server from the target file to be verified. The acquisition module is also used to obtain N file fragments from the target file based on the identifiers of the N file fragments. The processing module is used to determine the checksum of each file fragment in the N file fragments obtained to obtain N second checksums. The processing module is also used to determine the integrity of the target file based on the N first checksums and the N second checksums.

[0055] In some embodiments, when the processing module determines the integrity of the target file based on N first checksums and N second checksums, it is specifically used to: determine that the target file is incomplete when the first checksum of the first file fragment is inconsistent with the second checksum of the first file fragment, and the first file fragment is any one of the N file fragments; determine that the target file is complete when the first checksum of each file fragment in the N file fragments is consistent with the corresponding second checksum.

[0056] In some embodiments, the N file fragments include a first file fragment. In this case, the acquisition module is further used to obtain a first amount of data contained in the first file fragment from the server. The processing module is further used to determine that the target file is incomplete when the first amount of data contained in the first file fragment is inconsistent with the second amount of data contained in the first file fragment, wherein the second amount of data is the amount of data contained in the first file fragment obtained from the target file. The processing module is further used to determine whether a first checksum of the first file fragment is consistent with a second checksum of the first file fragment when the first amount of data contained in the first file fragment is consistent with the second amount of data contained in the first file fragment.

[0057] In some embodiments, when N=1 or N=2, the file segments are segments excluding the header content and the tail content of the target file.

[0058] In some embodiments, the identification of the file segment is a byte offset of the file segment.

[0059] In some embodiments, Figure 4The acquisition module 401 and the processing module 402 shown in the figure can be implemented by software or by hardware. For example, the implementation of the acquisition module 401 is described below by taking the acquisition module 401 as an example. Similarly, the implementation of the processing module 402 can refer to the implementation of the acquisition module 401.

[0060] As an example of a software functional unit, the acquisition module 401 may include code running on a computing instance. Among them, the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the above-mentioned computing instance may be one or more. For example, the acquisition module 401 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region (region) or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including a data center or multiple data centers with close geographical locations. Among them, usually a region may include multiple AZs.

[0061] Similarly, multiple hosts / virtual machines / containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.

[0062] As an example of a hardware functional unit, the acquisition module 401 may include at least one computing device, such as a server, etc. Alternatively, the acquisition module 401 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0063] The multiple computing devices included in the acquisition module 401 can be distributed in the same region or in different regions. The multiple computing devices included in the acquisition module 401 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the acquisition module 401 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0064] It should be noted that, in other embodiments, the acquisition module 401 can be used to execute any step in the file integrity verification method described in the above embodiment, and the processing module 402 can be used to execute any step in the file integrity verification method described in the above embodiment. The steps that the acquisition module 401 and the processing module 402 are responsible for implementing can be specified as needed, and the acquisition module 401 and the processing module 402 respectively implement different steps in the file integrity verification method described in the above embodiment to achieve Figure 4 The entire functions of the file integrity verification device 400 are shown.

[0065] The present application also provides a computing device 500. Figure 5 As shown, computing device 500 includes: bus 502, processor 504, memory 506 and communication interface 508. Processor 504, memory 506 and communication interface 508 communicate through bus 502. Computing device 500 can be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in computing device 500.

[0066] The bus 502 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 The bus 504 is represented by only one line, but it does not mean that there is only one bus or one type of bus. The bus 504 may include a path for transmitting information between various components of the computing device 500 (eg, the memory 506, the processor 504, and the communication interface 508).

[0067] The processor 504 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0068] The memory 506 may include a volatile memory, such as a random access memory (RAM). The processor 504 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0069] The memory 506 stores executable program codes, and the processor 504 executes the executable program codes to respectively implement the aforementioned Figure 4 The functions of the acquisition module 401 and the processing module 402 shown in the above embodiment are implemented to realize the file integrity verification method described in the above embodiment. That is, the memory 506 stores instructions for executing the file integrity verification method described in the above embodiment.

[0070] Alternatively, the memory 506 stores executable codes, and the processor 504 executes the executable codes to respectively implement the aforementioned Figure 4 The functions of the file integrity verification device 400 shown in the above embodiment are implemented to realize the file integrity verification method described in the above embodiment. That is, the memory 506 stores instructions for executing the file integrity verification method described in the above embodiment.

[0071] The communication interface 503 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 500 and other devices or a communication network.

[0072] The embodiment of the present application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.

[0073] like Figure 6As shown, the computing device cluster includes at least one computing device 500. The memory 506 in one or more computing devices 500 in the computing device cluster may store the same instructions for executing the file integrity verification method described in the above embodiment.

[0074] In some possible implementations, the memory 506 of one or more computing devices 500 in the computing device cluster may also store some instructions for executing the file integrity verification method described in the above embodiment. In other words, the combination of one or more computing devices 500 can jointly execute the instructions for executing the file integrity verification method described in the above embodiment.

[0075] It should be noted that the memory 506 in different computing devices 500 in the computing device cluster may store different instructions, which are respectively used to execute the aforementioned Figure 4 The file integrity verification device 400 shown in FIG. 4 shows some functions. That is, the instructions stored in the memory 506 in different computing devices 500 can implement the functions of one or more modules in the acquisition module 401 and the processing module 402 .

[0076] In some possible implementations, one or more computing devices in the computing device cluster may be connected via a network, which may be a wide area network or a local area network. Figure 7 A possible implementation is shown. Figure 7 As shown, two computing devices 500A and 500B are connected via a network. Specifically, the network is connected via a communication interface in each computing device. In this type of possible implementation, the memory 506 in the computing device 500A stores instructions for executing the functions of the acquisition module 501. At the same time, the memory 506 in the computing device 500B stores instructions for executing the functions of the processing module 502.

[0077] It should be understood that Figure 7 The functions of the computing device 500A shown in FIG. 5A may also be completed by multiple computing devices 500. Similarly, the functions of the computing device 500B may also be completed by multiple computing devices 500.

[0078] The present application embodiment also provides another computing device cluster. The connection relationship between the computing devices in the computing device cluster can be similar to that of Figure 6 and Figure 7 The connection mode of the computing device cluster is different in that the memory 506 in one or more computing devices 500 in the computing device cluster may store the same instructions for executing the method in the above embodiment.

[0079] In some possible implementations, the memory 506 of one or more computing devices 500 in the computing device cluster may also store partial instructions for executing the aforementioned data processing method. In other words, the combination of one or more computing devices 500 may jointly execute instructions for executing the aforementioned data processing method.

[0080] Based on the method in the above embodiment, the embodiment of the present application provides a computer-readable storage medium, including computer program instructions. When the computer program instructions are executed by a computing device, the computing device executes the method in the above embodiment; or, when the computer program instructions are executed by a computing device cluster, the computing device cluster executes the method in the above embodiment. Exemplarily, the computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center that includes one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk), etc.

[0081] Based on the method in the above embodiment, an embodiment of the present application provides a computer program product containing instructions, which, when executed by a computing device, enables the computing device to execute the method in the above embodiment, or, when executed by a computing device cluster, enables the computing device cluster to execute the method in the above embodiment.

[0082] It is understood that the processor in the embodiments of the present application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0083] The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0084] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)), etc.

[0085] It should be understood that the various numerical numbers involved in the embodiments of the present application are only used for the convenience of description and are not used to limit the scope of the embodiments of the present application.

[0086] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present application.

Claims

1. A file integrity verification method, characterized in that: The method comprises: Obtaining identifiers of N (N≥1) file segments and a first checksum of each of the N file segments from the server, wherein the N file segments are file segments randomly sampled by the server from the target file to be checked; Based on the identifiers of the N file fragments, obtaining the N file fragments from the target file; Determining a checksum of each of the N acquired file fragments to obtain N second checksums; The integrity of the target file is determined based on the N first checksums and the N second checksums.

2. The method according to claim 1, characterized in that The determining the integrity of the target file based on the N first checksums and the N second checksums includes: In a case where a first checksum of a first file segment is inconsistent with a second checksum of the first file segment, determining that the target file is incomplete, the first file segment being any one of the N file segments; In a case where the first checksum of each of the N file segments is consistent with the corresponding second checksum, it is determined that the target file is complete.

3. The method according to claim 1 or 2, characterized in that: The N file segments include a first file segment; The method further comprises: Acquire a first amount of data included in the first file segment from the server; In a case where a first data amount included in the first file segment is inconsistent with a second data amount included in the first file segment, determining that the target file is incomplete, wherein the second data amount is the data amount included in the first file segment obtained from the target file; When a first data amount included in the first file segment is consistent with a second data amount included in the first file segment, it is determined whether a first checksum of the first file segment is consistent with a second checksum of the first file segment.

4. The method according to any one of claims 1 to 3, characterized in that: In the case of N=1 or N=2, the file segments are segments excluding the header content and the tail content of the target file.

5. The method according to any one of claims 1 to 4, characterized in that: The identifier of the file segment is the byte offset of the file segment.

6. A file integrity verification device, characterized in that: include: An acquisition module, used to acquire identifiers of N (N≥1) file segments and a first checksum of each of the N file segments from a server, wherein the N file segments are file segments randomly sampled by the server from a target file to be checked; The acquisition module is further used to acquire the N file fragments from the target file based on the identifiers of the N file fragments; A processing module, configured to determine a checksum of each of the N acquired file fragments to obtain N second checksums; The processing module is further used to determine the integrity of the target file based on the N first checksums and the N second checksums.

7. The device according to claim 6, characterized in that When the processing module determines the integrity of the target file based on the N first checksums and the N second checksums, it is specifically used to: In a case where a first checksum of a first file segment is inconsistent with a second checksum of the first file segment, determining that the target file is incomplete, the first file segment being any one of the N file segments; In a case where the first checksum of each of the N file segments is consistent with the corresponding second checksum, it is determined that the target file is complete.

8. The device according to claim 6 or 7, characterized in that The N file segments include a first file segment; The acquisition module is further used to acquire a first data amount included in the first file segment from the server; The processing module is further configured to determine that the target file is incomplete when a first data amount included in the first file segment is inconsistent with a second data amount included in the first file segment, wherein the second data amount is the data amount included in the first file segment obtained from the target file; The processing module is further configured to determine whether a first checksum of the first file segment is consistent with a second checksum of the first file segment when a first data amount included in the first file segment is consistent with a second data amount included in the first file segment.

9. The device according to any one of claims 6 to 8, characterized in that: In the case of N=1 or N=2, the file segments are segments excluding the header content and the tail content of the target file.

10. The device according to any one of claims 6 to 9, characterized in that: The identifier of the file segment is the byte offset of the file segment.

11. A computing device cluster, characterized in that: comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method according to any one of claims 1-5.

12. A computer-readable storage medium, characterized in that: The method comprises computer program instructions, and when the instructions are executed by a computing device cluster, the computing device cluster enables the computing device cluster to perform the method according to any one of claims 1 to 5, wherein the computing device cluster comprises at least one computing device.

13. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device cluster, the computing device cluster executes the method according to any one of claims 1 to 5, wherein the computing device cluster includes at least one computing device.

Images