Method, device, equipment and storage medium for determining data type in a file

By using preset bit arrays and multiple hash functions to calculate file data, the memory overflow and calculation time-consuming problems in large file processing are solved, and fast and accurate data type determination and low system resource occupation are achieved.

CN114490515BActive Publication Date: 2025-05-16JD DIGITS HAIYI INFORMATION TECHNOLOGY CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202210096091.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-01-26
Publication Date
2025-05-16
Estimated Expiration
2042-01-26

AI Technical Summary

Technical Problem

When processing large files, the prior art can easily cause system memory overflow, and the reading and calculating data types take a long time, affecting system performance.

Method used

By obtaining the data in the reference file and the target file, and using a preset bit array and multiple hash functions for calculation, the intersection or difference of the data can be quickly determined, reducing the amount of computing and system resource usage.

Benefits of technology

It realizes the rapid and accurate determination of data types in the file, reduces system resource usage and calculation time, and avoids memory overflow problems.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114490515B_ABST
    Figure CN114490515B_ABST
Patent Text Reader

Abstract

The present application discloses a method, device, equipment and storage medium for determining the data type in a file, the method may include: obtaining each data stored in a reference file and the target data stored in a target file; obtaining a preset bit array and N different hash functions, N being a positive integer greater than or equal to a preset value; based on the array length of the preset bit array and N different hash functions, calculating each data and the target data in turn, obtaining the reference operation result corresponding to each data and the judgment operation result of the target data; determining the data type of the target data relative to the reference file according to the reference operation result, the preset bit array and the judgment operation result; wherein the data type includes intersection data or difference data. The above scheme can quickly and accurately determine the data type of the target data relative to the reference file with a small amount of calculation, and the implementation method is not only simple in calculation, but also has the advantage of occupying less system resources.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present application relate to the field of data processing, and in particular, to a method, apparatus, device, and storage medium for determining a data type in a file. Background Art

[0002] At present, various industries mainly store data in the form of files. However, if the files storing data are too large, then when reading the data, it will cause excessive memory resources to be occupied. It will take a long time to calculate the data type of the data stored in the file, which can easily cause system memory overflow problems.

[0003] Among the common solutions, one is to read multiple files into memory. However, when multiple files are read into memory at the same time, the system memory usage rate will increase. Especially when multiple users operate at the same time, the system will frequently clean up the heap space (FULL GC), causing the system to be sluggish, or even overflow memory and unable to be used normally. Another solution is to split large files into multiple small files. Although this will reduce the system memory usage, the number of disk reads will increase and the time consumption will also increase. Summary of the invention

[0004] The embodiments of the present application provide a method, apparatus, device and storage medium for determining the data type in a file, which can quickly and accurately determine that the target data is the intersection data or difference data in the reference file and the target file with a small amount of calculation. This implementation method is not only simple in calculation, but also has the advantage of occupying less system resources.

[0005] In a first aspect, an embodiment of the present application further provides a method for determining a data type in a file, the method comprising:

[0006] Acquire each data stored in the reference file and the target data stored in the target file;

[0007] Obtain a preset bit array and N different hash functions, where N is a positive integer greater than or equal to a preset value;

[0008] Based on the array length of the preset bit array and N different hash functions, each data and the target data are calculated in turn to obtain the reference operation result corresponding to each data and the judgment operation result of the target data;

[0009] Determine the data type of the target data relative to the reference file based on the reference operation result, the preset bit array and the judgment operation result;

[0010] The data type includes intersection data or difference data.

[0011] In a second aspect, an embodiment of the present application further provides a device for determining a data type in a file, the device comprising:

[0012] An acquisition module, used for acquiring each data stored in the reference file and the target data stored in the target file;

[0013] The acquisition module is further used to acquire a preset bit array and N different hash functions, where N is a positive integer greater than or equal to a preset value;

[0014] A calculation module, used to calculate each data and target data in turn based on the array length of the preset bit array and N different hash functions, to obtain the reference calculation result corresponding to each data and the judgment calculation result of the target data;

[0015] A determination module, used to determine the data type of the target data relative to the reference file according to the reference operation result, the preset bit array and the judgment operation result;

[0016] The data type includes intersection data or difference data.

[0017] In a third aspect, an embodiment of the present application further provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, a method for determining the data type in a file as provided in any embodiment of the present application is implemented.

[0018] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for determining the data type in a file as provided in any embodiment of the present application is implemented.

[0019] The embodiment of the present application provides a method, device, equipment and storage medium for determining the data type in a file, the method may include: obtaining each data stored in a reference file and the target data stored in a target file; obtaining a preset bit array and N different hash functions, N being a positive integer greater than or equal to a preset value; based on the array length of the preset bit array and N different hash functions, calculating each data and the target data in turn, obtaining the reference operation result corresponding to each data and the judgment operation result of the target data; determining the data type of the target data relative to the reference file according to the reference operation result, the preset bit array and the judgment operation result; wherein the data type includes intersection data or difference data. In the above scheme, since the bit array and the number of hash functions are preset, the operation results of the data stored in the reference file and the target file can be effectively guaranteed to be discrete based on the preset bit array and the N hash functions, so that the target data can be quickly and accurately determined to be the intersection data or difference data in the reference file and the target file with a small amount of calculation, and this implementation method is not only simple in calculation, but also has the advantage of occupying less system resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A flow chart of a method for determining the data type in a file provided in an embodiment of the present application;

[0021] Figure 2 is a flow chart of a method for determining the data type of target data relative to a reference file provided in an embodiment of the present application;

[0022] Figure 3 It is a structural schematic diagram of a device for determining the data type in a file provided by an embodiment of the present application;

[0023] Figure 4 is a structural schematic diagram of another device for determining the data type in a file provided in an embodiment of the present application;

[0024] Figure 5 It is a schematic diagram of the structure of the computer device in the embodiment of the present application. DETAILED DESCRIPTION

[0025] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for ease of description, only the parts related to the present application, rather than all structures, are shown in the accompanying drawings.

[0026] In addition, in the embodiments of the present application, words such as "optionally" or "exemplarily" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "optionally" or "exemplarily" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "optionally" or "exemplarily" is intended to present related concepts in a specific way.

[0027] Figure 1A flowchart of a method for determining the data type in a file provided in an embodiment of the present application. The method can be applied to the scenario of quickly calculating the intersection data and difference data of multiple large files. The data stored in the reference file and the target file can be calculated using multiple hash functions and preset bit arrays, thereby ensuring that the calculation results of the data in the reference file and the target file are discrete. In this way, when judging the target data in the target file based on the calculation results of the data in the reference file, it is possible to quickly determine with a high accuracy that the target data is the intersection data or difference data between the reference file and the target file. This implementation method is not only simple in operation, but also has the advantage of occupying less system resources. The method can be executed by the device for determining the data type in a file provided in an embodiment of the present application, and the device can be implemented in software and / or hardware. In a specific embodiment, the device can be integrated in a computer device, such as a server. The following embodiments will be described by taking the device integrated in a computer device as an example, such as Figure 1 As shown, the method may include but is not limited to the following steps:

[0028] S101, acquiring each data stored in the reference file and the target data stored in the target file.

[0029] In an embodiment of the present application, a reference file and a target file are two files storing data, and the data stored in the two files may be the same or completely different. Further, it can be understood that the calculation order of the data stored in the reference file is before the calculation order of the data stored in the target file, that is, each data in the reference file is calculated first, and then the target data stored in the target file is calculated, wherein the target data in the target file can be any one or more data in the data stored in the target file. The embodiment of the present application takes the target data as one piece of data as an example to describe the scheme provided by the embodiment. Of course, in the case where the target data is multiple pieces of data, each piece of data can be calculated and processed one by one in the same manner.

[0030] It should be noted that the calculation order of the above data is not the same as the reading order of the data, that is, the data stored in the reference file and the target data stored in the target file can be obtained at the same time, or the target data stored in the target file can be obtained first, and then the data stored in the reference file can be obtained.

[0031] S102: Obtain a preset bit array and N different hash functions.

[0032] In an embodiment of the present application, the data length of the preset bit array can be set based on the actual application scenario (e.g., file data, the amount of data in each file, etc.). N different hash functions can be divided into two cases, one is that the function forms of the N hash functions are different, and the other is that the function forms of the N hash functions are the same, and the parameters of the N hash functions are different. Wherein, N is a positive integer greater than or equal to a preset value, and the preset value can also be set based on the actual application scenario, for example, set to 5, that is, N is required to be greater than or equal to 5.

[0033] In the embodiment of the present application, by setting the number of hash functions, it is possible to ensure that the calculation results of each data are discrete, that is, to avoid the situation where the calculation results corresponding to each data are not continuous or overlapping in the preset bit array. It can be understood that the more the number of hash functions and the larger the array length of the preset bit array, the better the discreteness of the calculation results of each data calculated.

[0034] S103, based on the array length of the preset bit array and N different hash functions, each data and the target data are calculated in turn to obtain the reference operation result corresponding to each data and the judgment operation result of the target data.

[0035] Exemplarily, the implementation method of this step may include: calculating each data and the target data in turn based on N different hash functions to obtain the hash results corresponding to each data and the hash results of the target data, respectively; performing modulo operations on the hash results corresponding to each data and the hash results of the target data in turn based on the array length of a preset bit array to obtain the modulo results corresponding to each data and the modulo results of the target data, respectively; determining the modulo results corresponding to each data and the modulo results of the target data as the reference operation results corresponding to each data and the judgment operation results of the target data, respectively.

[0036] For example, taking any piece of data stored in a reference file as an example, since there are N different hash functions, the data can be calculated based on the N different hash functions to obtain N hash results for the data. Based on the data length of the preset bit array, the N hash results of the data are respectively modulo-operated to obtain N modulo results of the data. The N modulo results are the N positions of the data in the preset bit array, and the N modulo results are determined as the reference operation results of the data in the reference file. Accordingly, the other data in the reference file and the target data in the target file can be calculated in the same manner as above to obtain the reference operation results corresponding to each data in the reference file and the judgment operation results of the target data in the target file.

[0037] That is to say, in the embodiment of the present application, any one of the data in the reference file corresponds to N reference operation results, and similarly, the target data in the target file also corresponds to N judgment operation results. The N operation results are respectively in correspondence with N different hash functions, and the N judgment operation results are also respectively in correspondence with N different hash functions.

[0038] S104: Determine the data type of the target data relative to the reference file according to the reference operation result, the preset bit array and the judgment operation result.

[0039] Based on the above process, it can be determined that the reference operation results respectively represent the positions of each data in the reference file in the preset bit array. Similarly, the judgment operation results also represent the positions of the target data in the target file in the preset bit array. Therefore, after obtaining the reference operation results of each data in the reference file and the judgment operation results of the target data in the target file, the data type of the target data relative to the reference file can be judged based on the positions of the reference operation results and the judgment operation results in the preset bit array respectively.

[0040] Exemplarily, the above data types may include intersection data or difference data. For example, when it is determined that the target data is intersection data relative to the reference file, it means that the target data is contained in both the reference file and the target file, and belongs to the intersection of the two files. When it is determined that the target data is difference data relative to the reference file, it means that the target data is only contained in the target file, but not in the reference file, and belongs to the difference of the two files. The above method can be used to quickly determine the data type of the target data relative to the reference file with less computational effort, and also avoid occupying more system resources.

[0041] The embodiment of the present application provides a method for determining the data type in a file, which may include: obtaining each data stored in a reference file and the target data stored in a target file; obtaining a preset bit array and N different hash functions, where N is a positive integer greater than or equal to a preset value; based on the array length of the preset bit array and N different hash functions, calculating each data and the target data in turn to obtain the reference operation result and the judgment operation result of the target data corresponding to each data; determining the data type of the target data relative to the reference file according to the reference operation result, the preset bit array and the judgment operation result; wherein the data type includes intersection data or difference data. In the above scheme, since the bit array and the number of hash functions are preset, the operation results of the data stored in the reference file and the target file can be effectively guaranteed to be discrete based on the preset bit array and the N hash functions, so that the target data can be quickly and accurately determined to be the intersection data or difference data in the reference file and the target file with a small amount of calculation. This implementation method is not only simple in calculation, but also has the advantage of occupying less system resources.

[0042] In one example, if Figure 2 As shown, the specific implementation method of determining the data type of the target data relative to the reference file in the above step S104 may include but is not limited to the following process:

[0043] S201 , updating the array content of the preset bit array according to the reference operation result.

[0044] Exemplarily, the updating method in this step may include: determining whether the numerical values ​​at N positions in the preset bit array included in the reference operation result are initialization values; if the numerical values ​​corresponding to the N positions are all initialization values, updating the initialization values ​​corresponding to the N positions to preset identification values.

[0045] It can be understood that, since a piece of data corresponds to N reference operation results, when determining the position of the value included in the reference operation result of any piece of data in the reference file in the preset bit array, the N positions of the reference operation result of the data can be determined in the preset bit array. Then, it is necessary to judge the N positions one by one. If the values ​​corresponding to the N positions are all initialization values, it means that the data has not been calculated before, or there is no data identical to the data in any previous file, then the N positions corresponding to the data are all updated to the preset identification value, for example, updated to 1. On the contrary, if the value corresponding to any one of the N positions is not the initialization value (that is, the preset identification value, and there are only two cases for the values ​​at each position in the preset bit array, that is, the value corresponding to the position is the initialization value or the preset identification value), it means that the data may exist in any previous reference file. At this time, there is no need to update the array content of the preset bit array.

[0046] S202: Match the judgment operation result with the updated preset bit array.

[0047] S203: When it is determined that the values ​​at the N positions included in the calculation result are the same as the values ​​at the corresponding N positions in the updated preset bit array, determine that the data type of the target data is intersection data.

[0048] Similarly, since the judgment operation result also includes the numerical values ​​at N positions, when matching the judgment operation result with the updated preset bit array, it is necessary to compare the numerical values ​​at the N positions contained in the judgment operation result with the numerical values ​​at the corresponding N positions in the updated preset bit array. If the numerical values ​​at the N positions included in the judgment operation result are the same as the numerical values ​​at the corresponding N positions in the updated preset bit array, it means that the target data corresponding to the judgment operation result has been calculated or stored in any previous reference file, then the data type of the target data relative to the reference file is intersection data.

[0049] S204: When it is determined that the values ​​at the N positions included in the calculation result are different from the values ​​at the corresponding N positions in the updated preset bit array, determine that the data type of the target data is difference data.

[0050] On the contrary, if the values ​​at the N positions included in the judgment operation result are different from the values ​​at the corresponding N positions in the updated preset bit array, for example, the values ​​at at least one position are different, it means that the target data corresponding to the judgment operation result is not stored in any previous reference file, then the data type of the target data relative to the reference file is difference data.

[0051] Optionally, when the reference file is a file calculated for the first time, before obtaining the preset bit array in the step S102, the embodiment of the present application further provides an implementation method including: initializing all positions of the preset bit array, for example, marking all positions of the preset bit array as 0.

[0052] For example, suppose there are three files, namely file A, file B and file C, and the data in the three files are calculated in the order of file A, file B and file C. Among them, file A is a reference file relative to file B and file C calculated later, and file B is a reference file relative to file C calculated later. For file A, file B and file C are target files, and for file B, file C is the target file. If the currently calculated file is file A, since file A is the file calculated for the first time, all positions in the preset position array can be initialized. On the contrary, if the currently calculated file is file B, since the array content of the preset position data has been updated before and based on file A, in this case, all positions of the preset position array do not need to be initialized.

[0053] Figure 3 A schematic diagram of a structure of a device for determining the data type in a file provided in an embodiment of the present application, such as Figure 3 As shown, the device may include: an acquisition module 301, a calculation module 302, and a determination module 303;

[0054] The acquisition module is used to acquire the data stored in the reference file and the target data stored in the target file;

[0055] The acquisition module is further used to acquire a preset bit array and N different hash functions, where N is a positive integer greater than or equal to a preset value;

[0056] A calculation module, used to calculate each data and target data in turn based on the array length of the preset bit array and N different hash functions, to obtain the reference calculation result corresponding to each data and the judgment calculation result of the target data;

[0057] A determination module, used to determine the data type of the target data relative to the reference file according to the reference operation result, the preset bit array and the judgment operation result;

[0058] The data type includes intersection data or difference data.

[0059] Optionally, the N hash functions have different functional forms;

[0060] Alternatively, the function forms of the N hash functions are the same, and the parameters of the N hash functions are different.

[0061] Furthermore, any one of the above data corresponds to N reference operation results;

[0062] One target data corresponds to N judgment operation results;

[0063] The N reference operation results and the N judgment operation results correspond to N different hash functions respectively.

[0064] In one example, the above-mentioned calculation module is used to calculate each data and the target data in sequence based on N different hash functions to obtain the hash results corresponding to each data and the hash results of the target data respectively; perform modulo operations on the hash results corresponding to each data and the hash results of the target data in sequence based on the array length of a preset bit array to obtain the modulo results corresponding to each data and the modulo results of the target data respectively; and determine the modulo results corresponding to each data and the modulo results of the target data as the reference operation results corresponding to each data and the judgment operation results of the target data respectively.

[0065] In one example, the above-mentioned determination module may include an updating unit and a determining unit;

[0066] Wherein, the updating unit can update the array content of the preset bit array according to the reference operation result;

[0067] A determination unit can match the judgment operation result with the updated preset bit array; wherein the reference operation result and the judgment operation result respectively include the values ​​at N positions in the preset bit array; when the values ​​at the N positions included in the judgment operation result are the same as the values ​​at the corresponding N positions in the updated preset bit array, the determination unit determines that the data type of the target data relative to the reference file is intersection data; when the values ​​at the N positions included in the judgment operation result are different from the values ​​at the corresponding N positions in the updated preset bit array, the determination unit determines that the data type of the target data relative to the reference file is difference data.

[0068] Exemplarily, the above-mentioned update unit can be used to determine whether the values ​​at N positions in the bit array included in the reference operation result are initialization values; and, when the corresponding values ​​of the N positions are all initialization values, the initialization values ​​corresponding to the N positions are updated to preset identification values.

[0069] Furthermore, if Figure 4 As shown, the above device may further include an initialization module 304;

[0070] In the case that the reference file is a file calculated for the first time, the initialization module is used to initialize all positions of the preset bit array.

[0071] The above-mentioned device for determining the data type in the file can execute Figure 1-Figure 2 The provided method for determining the data type in a file has corresponding devices and beneficial effects in the method.

[0072] Figure 5 A schematic diagram of the structure of a computer device provided in Embodiment 5 of the present invention is shown in FIG. Figure 5 As shown, the computer device includes a controller 501, a memory 502, an input device 503, and an output device 504; the number of controllers 501 in the computer device can be one or more. Figure 5 A controller 501 is taken as an example; the controller 501, the memory 502, the input device 503 and the output device 504 in the computer device can be connected by a bus or other means. Figure 5 The example of connecting through bus is taken in the following.

[0073] The memory 502 is a computer-readable storage medium that can be used to store software programs, computer executable programs, and modules, such as Figure 1 , Figure 2 , Figure 3The program instructions / modules corresponding to the method for determining the data type in a file in the embodiment (for example, the acquisition module 301, the calculation module 302, and the determination module 303 in the apparatus for determining the data type in a file). The controller 501 executes various functions and data processing of the computer device by running the software programs, instructions, and modules stored in the memory 502, that is, implements the above-mentioned method for determining the data type in a file.

[0074] The memory 502 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the terminal, etc. In addition, the memory 502 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 502 may further include a memory remotely arranged relative to the controller 501, and these remote memories may be connected to the terminal / server via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0075] The input device 503 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the computer device. The output device 504 may include a display device such as a display screen.

[0076] The present application also provides a storage medium containing computer executable instructions. When the computer executable instructions are executed by a computer controller, the method is used to perform a method for determining the data type in a file. The method includes: Figure 1-Figure 2 Steps shown.

[0077] Through the above description of the implementation method, the technicians in the relevant field can clearly understand that the present application can be implemented with the help of software and necessary general hardware, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application can be essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0078] It is worth noting that the modules included in the above-mentioned device for determining the data type in the file are only divided according to functional logic, but are not limited to the above-mentioned division method. As long as the corresponding functions can be achieved, it is not used to limit the protection scope of this application.

[0079] Note that the above are only preferred embodiments of the present application and the technical principles used. Those skilled in the art will understand that the present application is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present application. Therefore, although the present application is described in more detail through the above embodiments, the present application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the appended claims.

Claims

1. A method for determining the data type in a file, characterized in that: include: Acquire each data stored in the reference file and the target data stored in the target file; Obtain a preset bit array and N different hash functions, where N is a positive integer greater than or equal to a preset value; Based on the array length of the preset bit array and the N different hash functions, the data and the target data are calculated in sequence to obtain the reference operation results corresponding to the data and the judgment operation results of the target data; including: based on the N different hash functions, the data and the target data are calculated in sequence to obtain the hash results corresponding to the data and the hash results of the target data respectively; based on the array length of the preset bit array, the hash results corresponding to the data and the hash results of the target data are performed modulo operation in sequence to obtain the modulo results corresponding to the data and the modulo results of the target data respectively; the modulo results corresponding to the data and the modulo results of the target data are determined as the reference operation results corresponding to the data and the judgment operation results of the target data respectively; Determining the data type of the target data relative to the reference file according to the reference operation result, the preset bit array and the judgment operation result; The data type includes intersection data or difference data.

2. The method according to claim 1, characterized in that The functional forms of the N hash functions are different; Alternatively, the function forms of the N hash functions are the same, and the parameters of the N hash functions are different.

3. The method according to claim 1, characterized in that Any one of the data corresponds to N reference operation results; One piece of target data corresponds to N judgment operation results; The N reference operation results and the N judgment operation results are respectively in a corresponding relationship with the N different hash functions.

4. The method according to any one of claims 1 to 3, characterized in that: The step of determining the data type of the target data relative to the reference file according to the reference operation result, the preset bit array and the judgment operation result includes: Update the array content of the preset bit array according to the reference operation result; Matching the judgment operation result with the updated preset bit array; Wherein, the reference operation result and the judgment operation result respectively include the values ​​at N positions in the preset bit array; In a case where the values ​​at the N positions included in the judgment operation result are the same as the values ​​at the corresponding N positions in the updated preset bit array, determining that the data type of the target data relative to the reference file is intersection data; In the case that the values ​​at the N positions included in the judgment operation result are different from the values ​​at the corresponding N positions in the updated preset bit array, it is determined that the data type of the target data relative to the reference file is difference data.

5. The method according to claim 4, characterized in that The updating of the array content of the preset bit array according to the reference operation result comprises: Determine whether the values ​​at N positions in the preset bit array included in the reference operation result are initialization values; In a case where the numerical values ​​corresponding to the N positions are all initialization values, the initialization values ​​corresponding to the N positions are updated to preset identification values.

6. The method according to claim 1, characterized in that In the case where the reference file is a file calculated for the first time, before obtaining the preset bit array, the method further includes: Initialize all positions of the preset bit array.

7. A device for determining the data type in a file, characterized in that: include: An acquisition module, used for acquiring each data stored in the reference file and the target data stored in the target file; The acquisition module is further used to acquire a preset bit array and N different hash functions, where N is a positive integer greater than or equal to a preset value; A calculation module, for calculating each data and the target data in sequence based on the array length of the preset bit array and the N different hash functions, to obtain a reference operation result corresponding to each data and a judgment operation result of the target data; comprising: calculating each data and the target data in sequence based on the N different hash functions, to obtain a hash result corresponding to each data and a hash result of the target data respectively; performing a modulus operation on the hash result corresponding to each data and the hash result of the target data in sequence based on the array length of the preset bit array, to obtain a modulus result corresponding to each data and a modulus result of the target data respectively; determining the modulus result corresponding to each data and the modulus result of the target data as the reference operation result corresponding to each data and the judgment operation result of the target data respectively; A determination module, used for determining the data type of the target data relative to the reference file according to the reference operation result, the preset bit array and the judgment operation result; The data type includes intersection data or difference data.

8. A computer device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for determining the data type in a file according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for determining the data type in a file according to any one of claims 1 to 6 is implemented.

Citation Information

Patent Citations

  • Dynamic Hash computation method and device for block chain, node and storage medium

    CN108512650A

  • Method and verification method for intersection of private data for secure multi-party computation

    CN109104413A