Network resource data processing method and device and related product

Abstract

The invention discloses a network resource data processing method and device and a related product. The method comprises the following steps: acquiring source end network resource data, a source end target field corresponding to the source end network resource data, host end network resource data and a host end target field corresponding to the host end network resource data; generating a corresponding source end target fingerprint according to the source end target field, and obtaining a corresponding host end target fingerprint according to the host end target field; and performing data processing on the source end network resource data and / or the host end network data according to the source end target fingerprint and the corresponding host end target fingerprint. In the embodiment of the invention, through cooperative work of multiple types of fingerprints (comparison fingerprints, updating fingerprints, primary key fingerprints, sorting fingerprints and unique fingerprints), the storage and calculation overhead can be remarkably reduced, and the accuracy of a processing result is improved.

Description

Technical Field

[0001] This application relates to the field of information technology, and in particular to a method, apparatus and related products for processing network resource data. Background Technology

[0002] With the explosive growth of data volume, efficient management, storage, and verification of data has become a key challenge. Among these, resource acquisition and comparison technologies are core to applications such as data deduplication, version control, incremental synchronization, and integrity verification.

[0003] In related technologies, generating unique identifier fingerprints for data based on cryptographic hash algorithms and then comparing and matching them has become an industry standard practice. However, the shortcomings of these technologies in data compression, slicing, and fault tolerance mechanisms still lead to problems such as low computational efficiency and poor fault tolerance when processing massive amounts of data, resulting in inaccurate data processing results and even resource duplication or loss. Summary of the Invention

[0004] To address the aforementioned issues, this application provides a network resource data processing method, apparatus, and related products to improve data processing efficiency.

[0005] The embodiments of this application disclose the following technical solutions:

[0006] In a first aspect, embodiments of this application provide a method for processing network resource data, the method comprising:

[0007] Obtain source network resource data and the corresponding source target field, destination network resource data and the corresponding destination target field;

[0008] The source target fingerprint is generated based on the source target field. The source target fingerprint and the destination target fingerprint include at least one of the following: source comparison fingerprint, source unique fingerprint, source sorting fingerprint, source primary key fingerprint, and source update fingerprint. The destination target fingerprint is obtained based on the destination target field. The destination target fingerprint includes at least one of the following: destination comparison fingerprint, destination unique fingerprint, destination sorting fingerprint, destination primary key fingerprint, and destination update fingerprint.

[0009] Based on the source target fingerprint and the corresponding destination target fingerprint, perform data processing on the source network resource data and / or the destination network data.

[0010] In one possible implementation, the source network resource data and / or the destination network data are processed based on the source target fingerprint and the corresponding destination target fingerprint, including:

[0011] If the fingerprints compared at the source end and the fingerprints compared at the destination end are consistent, it indicates that the network resource data at the source end and the network resource data at the destination end contain the same network resource data.

[0012] In one possible implementation, the source network resource data and / or the destination network data are processed based on the source target fingerprint and the corresponding destination target fingerprint, including:

[0013] If the source-end comparison fingerprint is consistent with the destination-end comparison fingerprint, the priority of the source-end resource data and the destination-end resource data is determined based on the source-end sorted fingerprint and the destination-end sorted fingerprint. Among them, if the source-end network resource data has a higher priority, the source-end network resource data is used as the comparison benchmark, and if the destination-end network resource data has a higher priority, the destination-end network resource data is used as the comparison benchmark.

[0014] The source network resource data and the destination network resource data are compared according to priority.

[0015] In one possible implementation, the source network resource data and / or the destination network data are processed based on the source target fingerprint and the corresponding destination target fingerprint, including:

[0016] If the update fingerprint at the source end is inconsistent with the update fingerprint at the destination end, update the network resource data at either the source end or the network resource data at the destination end.

[0017] In one possible implementation, updating the source network resource data or the destination network resource data includes:

[0018] Source network resource data is located based on the source primary key fingerprint, and destination network resource data is located based on the destination primary key fingerprint.

[0019] Update the source network resource data or the destination network resource data.

[0020] In one possible implementation, if the source fingerprint matches the destination fingerprint, the method further includes:

[0021] If the fingerprints at the source end and the fingerprints at the destination end are consistent, compare the unique fingerprints at the source end and the unique fingerprints at the destination end.

[0022] If the unique fingerprint at the source end and the unique fingerprint at the destination end are the same, it indicates that the network resource data at the source end and the network resource data at the destination end contain the same network resource data.

[0023] In one possible implementation, the method also includes:

[0024] The source network resource data is divided into N source network resource data slices, and the destination network resource data is divided into N destination network resource slices; where N is an integer greater than or equal to 2.

[0025] The N computing nodes process N source network resource data slices and N destination network resource data slices; where each computing node corresponds to one source network resource data slice and one destination network resource data slice.

[0026] Secondly, embodiments of this application provide a network resource processing device, which includes: an acquisition module, a generation module, and a processing module;

[0027] The acquisition module is configured to acquire source network resource data and the source target field corresponding to the source network resource data, and destination network resource data and the destination target field corresponding to the destination network resource data.

[0028] The generation module is configured to generate a corresponding source-end target fingerprint based on the source-end target field, wherein the source-end target fingerprint and the destination-end target fingerprint include at least one of source-end comparison fingerprint, source-end unique fingerprint, source-end sorted fingerprint, source-end primary key fingerprint, and source-end update fingerprint; and to obtain a corresponding destination-end target fingerprint based on the destination-end target field, wherein the destination-end target fingerprint includes at least one of destination-end comparison fingerprint, destination-end unique fingerprint, destination-end sorted fingerprint, destination-end primary key fingerprint, and destination-end update fingerprint.

[0029] The processing module is configured to process the source network resource data and / or the destination network data based on the source target fingerprint and the corresponding destination target fingerprint.

[0030] Thirdly, embodiments of this application provide a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the network resource data processing method as described in any embodiment of the second aspect.

[0031] Fourthly, according to an embodiment of this application, a computer-readable storage medium stores instructions that, when executed on a terminal device, cause the terminal device to perform the network resource data processing method as described in any embodiment of the second aspect.

[0032] To improve the processing efficiency of network resource data, this application provides a network resource data processing method. The method includes: acquiring source network resource data and a corresponding source target field, destination network resource data and a corresponding destination target field; generating a corresponding source target fingerprint based on the source target field, wherein the source target fingerprint and destination target fingerprint include at least one of source comparison fingerprint, source unique fingerprint, source sorting fingerprint, source primary key fingerprint, and source update fingerprint; obtaining a corresponding destination target fingerprint based on the destination target field, wherein the destination target fingerprint includes at least one of destination comparison fingerprint, destination unique fingerprint, destination sorting fingerprint, destination primary key fingerprint, and destination update fingerprint; and performing data processing on the source network resource data and / or destination network data based on the source target fingerprint and the corresponding destination target fingerprint. In this application embodiment, by using multiple types of fingerprints (comparison fingerprint, update fingerprint, primary key fingerprint, sorting fingerprint, and unique fingerprint) in synergy, storage and computational overhead can be significantly reduced, and the accuracy of processing results can be improved. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 A flowchart illustrating a network resource data processing method provided in this application embodiment;

[0035] Figure 2 A schematic diagram illustrating the generation of a target fingerprint, provided as an embodiment of this application;

[0036] Figure 3 This is a schematic diagram illustrating another method for generating a target fingerprint, as provided in an embodiment of this application.

[0037] Figure 4 A schematic diagram illustrating yet another method of generating a target fingerprint, as provided in an embodiment of this application;

[0038] Figure 5 A schematic diagram illustrating data deduplication as provided in an embodiment of this application;

[0039] Figure 6 A schematic diagram of a network resource data processing device provided in an embodiment of this application;

[0040] Figure 7 This is a schematic diagram of a computer device provided in an embodiment of this application. Detailed Implementation

[0041] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0042] The terms "first" and "second," etc., used in the specification and claims of this application are used to distinguish different objects, not to describe a specific order of objects. For example, "first switching device" and "second switching device," etc., are used to distinguish different switching devices, not to describe a specific order of switching devices.

[0043] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0044] In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more, for example, multiple processing units means two or more processing units, multiple elements means two or more elements, etc.

[0045] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the bidirectional communication circuit will be described below in conjunction with the accompanying drawings.

[0046] See Figure 1 The figure is a flowchart of a network resource data processing method provided in an embodiment of this application.

[0047] like Figure 1 As shown, the method includes the following steps:

[0048] S100: Obtain source network resource data and the corresponding source target field, destination network resource data and the corresponding destination target field.

[0049] It should be understood that the source end, as described in the embodiments of this application, is the "provider" or "initiator" of the data / resource; the destination end is the "receiver" or "target" of the data / resource.

[0050] S200: Generate the corresponding source target fingerprint based on the source target field. The source target fingerprint and the destination target fingerprint include at least one of the following: source comparison fingerprint, source unique fingerprint, source sorting fingerprint, source primary key fingerprint, and source update fingerprint. Obtain the corresponding destination target fingerprint based on the destination target field. The destination target fingerprint includes at least one of the following: destination comparison fingerprint, destination unique fingerprint, destination sorting fingerprint, destination primary key fingerprint, and destination update fingerprint.

[0051] For example, such as Figure 1 As shown, the target field can be compare_key, unique_key, sort_key, id_key, and field (e.g., field1, field2, field3, ... and fieldn), where n is an integer greater than or equal to 2.

[0052] In this embodiment, a unique fingerprint, unique_key, is generated based on the Message-Digest Algorithm 5 (MD5) and the compare_key and unique_key fields. It should be noted that the method for generating the source-side unique fingerprint in this embodiment is the same as the method for generating the destination-side unique fingerprint; therefore, the generation processes for both will not be described in detail here.

[0053] For example, unique_key=md5{compare_key-1}#md5{unique_key-4}#md5{unique_key-5}#.

[0054] In this embodiment, the sort fingerprint `sort_key` is generated using the `sort_key` field. It should be noted that the method for generating the source-end sort fingerprint in this embodiment is the same as the method for generating the destination-end sort fingerprint; therefore, the generation processes for both will not be described in detail here.

[0055] For example, sort_key={sort_key-6}&{sort_key-7}&#.

[0056] In this embodiment, the primary key fingerprint id_key is generated using the id_key field. It should be noted that the method for generating the source primary key fingerprint in this embodiment is the same as the method for generating the destination primary key fingerprint; therefore, the generation processes for both will not be described in detail here.

[0057] For example, id_key={id_key}.

[0058] In this embodiment, based on the MD5 algorithm, an update fingerprint (update_key) is generated according to the field (e.g., field1, field2, field3, ..., and fieldn). It should be noted that the method for generating the source-end update fingerprint in this embodiment is the same as the method for generating the destination-end update fingerprint; therefore, the generation processes for both will not be described in detail here.

[0059] For example, update_key=md5{field1, field2, field3, ..., fieldn}.

[0060] It should be understood that the target fingerprints described above are merely illustrative, and customers can generate new fingerprints and rename them according to their own needs.

[0061] like Figure 2 As shown, in cases involving multiple comparison fields (compare_key field), this embodiment of the application can also generate a unique fingerprint (unique_key) based on the comparison_key field.

[0062] For example, unique_key=md5{compare_key-2}#.

[0063] Generate an update fingerprint, update_key, based on the ompare_key field.

[0064] update_key=md5{ compare_key-1}.

[0065] like Figure 3 As shown, in cases involving multiple comparison fields (compare_key field), this embodiment of the application can also generate a unique fingerprint (unique_key) based on the comparison_key field.

[0066] For example, unique_key=md5{compare_key-3}#.

[0067] S300: Perform data processing on source network resource data and / or destination network data based on the source target fingerprint and the corresponding destination target fingerprint.

[0068] In this embodiment, by using multiple types of fingerprints (compare fingerprints, update fingerprints, primary key fingerprints, sorted fingerprints, and unique fingerprints) in concert, storage and computational overhead can be significantly reduced, and the accuracy of processing results can be improved.

[0069] In one possible implementation, the source network resource data and / or the destination network data are processed based on the source target fingerprint and the corresponding destination target fingerprint, including: if the source comparison fingerprint and the destination comparison fingerprint are consistent, indicating that the source network resource data and the destination network resource data have the same network resource data.

[0070] The fingerprint is compared as the "existence identifier" of the resource to determine whether the source end (the resource to be synchronized) and the destination end (the target end receiving the synchronization) have the same resource.

[0071] If the destination does not have this comparison fingerprint, it means that the resource does not exist on the target. Perform the "Add" operation to synchronize the resource from the source to the destination. If the source does not have this comparison fingerprint, it means that the resource has been removed from the source. Perform the "Delete" operation to remove the corresponding resource from the destination. If both sides have this comparison fingerprint, proceed to the next step (update judgment and priority sorting).

[0072] In one possible implementation, the source network resource data and / or the destination network data are processed based on the source target fingerprint and the corresponding destination target fingerprint, including: if the source comparison fingerprint and the destination comparison fingerprint are consistent, determining the priority of the source resource data and the destination resource data based on the source sorting fingerprint and the destination sorting fingerprint; wherein, if the source network resource data has a higher priority, the source network resource data is used as the comparison benchmark, and if the destination network resource data has a higher priority, the destination network resource data is used as the comparison benchmark; and comparing the source network resource data and the destination network resource data according to the priority.

[0073] When the source and destination have the same comparison fingerprint (i.e., resources "coexist"), it is necessary to determine which side's resources have a higher priority to participate in subsequent comparisons by sorting the fingerprints.

[0074] In this embodiment of the application, by introducing sorted fingerprints, the problem of "resource conflict with the same identifier" can be solved, ensuring that resources with higher priority (such as update time updates or versions with higher weights) become the comparison benchmark, and avoiding invalid or low-priority resources from interfering with the synchronization logic.

[0075] In one possible implementation, the source network resource data and / or the destination network data are processed based on the source target fingerprint and the corresponding destination target fingerprint, including: updating the source network resource data or the destination network resource data when the source update fingerprint and the destination update fingerprint are inconsistent.

[0076] For example, such as Figure 5As shown, if there are differences between the first comparison key (A3) of the source end and the first comparison key (A31) of the destination end, and between the second comparison key (B2) of the source end and the second comparison key (B21) of the destination end, the data corresponding to the second row and the data corresponding to the third row are updated.

[0077] In addition, in the embodiments of this application, if the source fingerprint and the destination fingerprint are consistent, the source unique fingerprint and the destination unique fingerprint are compared; if the source unique fingerprint and the destination unique fingerprint are consistent, it is indicated that the source network resource data and the destination network resource data have the same network resource data.

[0078] For example, such as Figure 5 As shown, when the source network resource data includes unique_key#unique_key#unique_key#, sort_key&sort_key#, id_key, and update_key, the data columns are split using "#" to obtain unique_key, unique_key, unique_key, sort_key&sort_key, id_key, and update_key. Deduplication is performed on unique_key and sort_key to obtain unique_key, sort_key&sort_key, id_key, and update_key. Finally, unique_key, sort_key&sort_key, id_key, and update_key are separated by commas.

[0079] In this embodiment of the application, the deduplication operation can effectively eliminate the interference of duplicate data, thereby improving the accuracy and consistency of the comparison results.

[0080] In addition, the source network resource data is divided into N source network resource data slices, and the destination network resource data is divided into N destination network resource slices; where N is an integer greater than or equal to 2; the N source network resource data slices and N destination network resource data slices are processed by N computing nodes; where one computing node corresponds to one source network resource data slice and one destination network resource data slice.

[0081] In this embodiment of the application, by combining intelligent slicing technology and parallel computing, the system can significantly reduce the computational complexity of each comparison task, thereby improving the resource comparison efficiency by at least five times compared with traditional technologies.

[0082] Based on the foregoing embodiments, this application also provides a network resource data processing device, which includes: an acquisition module 100, a generation module 200, and a processing module 300;

[0083] The acquisition module 100 is configured to acquire source network resource data and the source target field corresponding to the source network resource data, destination network resource data and the destination target field corresponding to the destination network resource data.

[0084] The generation module 200 is configured to generate a corresponding source target fingerprint based on the source target field, wherein the source target fingerprint and the destination target fingerprint include at least one of the following: source comparison fingerprint, source unique fingerprint, source sorting fingerprint, source primary key fingerprint, and source update fingerprint; and to obtain a corresponding destination target fingerprint based on the destination target field, wherein the destination target fingerprint includes at least one of the following: destination comparison fingerprint, destination unique fingerprint, destination sorting fingerprint, destination primary key fingerprint, and destination update fingerprint.

[0085] The processing module 300 is configured to process source network resource data and / or destination network data based on the source target fingerprint and the corresponding destination target fingerprint.

[0086] In this embodiment, by using multiple types of fingerprints (compare fingerprints, update fingerprints, primary key fingerprints, sorted fingerprints, and unique fingerprints) in concert, storage and computational overhead can be significantly reduced, and the accuracy of processing results can be improved.

[0087] In one possible implementation, the processing module is configured to indicate that the source network resource data and the destination network resource data have the same network resource data if the source fingerprint matches the destination fingerprint.

[0088] In one possible implementation, the processing module is configured to determine the priority of source-end resource data and destination-end resource data based on the source-end sorted fingerprint and destination-end sorted fingerprint when the source-end comparison fingerprint matches. Specifically, if the source-end network resource data has a higher priority, it is used as the comparison benchmark; if the destination-end network resource data has a higher priority, it is used as the comparison benchmark. The source-end network resource data and destination-end network resource data are compared according to their priorities.

[0089] In one possible implementation, the processing module is configured to update either the source network resource data or the destination network resource data if the source update fingerprint and the destination update fingerprint are inconsistent.

[0090] In one possible implementation, the processing module is further configured to compare the source-side unique fingerprint and the destination-side unique fingerprint if the source-side fingerprint matches the destination-side fingerprint; and to indicate that the source-side network resource data and the destination-side network resource data have the same network resource data if the source-side unique fingerprint matches the destination-side unique fingerprint.

[0091] In one possible implementation, the processing module is further configured to divide the source network resource data into N source network resource data slices and the destination network resource data into N destination network resource slices; where N is an integer greater than or equal to 2; and to process the N source network resource data slices and N destination network resource data slices through N computing nodes; wherein one computing node corresponds to one source network resource data slice and one destination network resource data slice.

[0092] This application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the network resource data method described in this application.

[0093] In practical applications, the computer-readable storage medium can be any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. For example, a computer-readable storage medium can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0094] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0095] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0096] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0097] like Figure 7 The diagram shown is a structural schematic of a computer device provided in an embodiment of this application. Figure 7 The computer device 12 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present invention.

[0098] like Figure 7 As shown, the computer device 12 is represented in the form of a general-purpose computing device. The components of the computer device 12 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and a bus 18 connecting different system components (including system memory 28 and processing unit 16).

[0099] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0100] Computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by computer device 12, including volatile and non-volatile media, removable and non-removable media.

[0101] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (…). Figure 7 Not shown; usually referred to as a "hard drive"). Although Figure 7 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. Memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0102] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of the present invention.

[0103] Computer device 12 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), and with one or more devices that enable a user to interact with the computer device 12, and / or with any device that enables the computer device 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed through input / output (I / O) interface 22. Furthermore, computer device 12 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) through network adapter 20. Figure 7 As shown, network adapter 20 communicates with other modules of computer device 12 via bus 18. It should be understood that, although... Figure 7 As not shown, it can be used in conjunction with computer device 12 with other hardware and / or software modules, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0104] The processor unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing the network resource data method provided in the embodiments of this application.

[0105] The above description is merely one specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for processing network resource data, characterized in that, The method includes: Obtain source network resource data and the source target field corresponding to the source network resource data, as well as destination network resource data and the destination target field corresponding to the destination network resource data; A corresponding source-end target fingerprint is generated based on the source-end target field, wherein the source-end target fingerprint and the destination-end target fingerprint include at least one of the following: source-end comparison fingerprint, source-end unique fingerprint, source-end sorting fingerprint, source-end primary key fingerprint, and source-end update fingerprint; a corresponding destination-end target fingerprint is obtained based on the destination-end target field, wherein the destination-end target fingerprint includes at least one of the following: destination-end comparison fingerprint, destination-end unique fingerprint, destination-end sorting fingerprint, destination-end primary key fingerprint, and destination-end update fingerprint; Based on the source target fingerprint and the corresponding destination target fingerprint, data processing is performed on the source network resource data and / or the destination network data.

2. The network resource data processing method according to claim 1, characterized in that, The step of processing the source network resource data and / or the destination network data based on the source target fingerprint and the corresponding destination target fingerprint includes: If the source-end fingerprint matches the destination-end fingerprint, it indicates that the source-end network resource data and the destination-end network resource data have the same network resource data.

3. The network resource data processing method according to claim 1, characterized in that, The step of processing the source network resource data and / or the destination network data based on the source target fingerprint and the corresponding destination target fingerprint includes: If the source end comparison fingerprint and the destination end comparison fingerprint are consistent, the priority of the source end resource data and the destination end resource data is determined according to the source end sorting fingerprint and the destination end sorting fingerprint; wherein, if the source end network resource data has a higher priority, the source end network resource data is used as the comparison benchmark, and if the destination end network resource data has a higher priority, the destination end network resource data is used as the comparison benchmark. The source network resource data and the destination network resource data are compared according to the priority.

4. The network resource data processing method according to claim 1, characterized in that, The step of processing the source network resource data and / or the destination network data based on the source target fingerprint and the corresponding destination target fingerprint includes: If the source-end update fingerprint and the destination-end update fingerprint are inconsistent, the source-end network resource data or the destination-end network resource data shall be updated.

5. The network resource data processing method according to claim 4, characterized in that, Updating the source network resource data or the destination network resource data includes: The source network resource data is located based on the source primary key fingerprint, and the destination network resource data is located based on the destination primary key fingerprint. The source network resource data or the destination network resource data is updated.

6. The network resource data processing method according to claim 2, characterized in that, If the source fingerprint matches the destination fingerprint, the method further includes: If the source fingerprint and the destination fingerprint are identical, compare the unique fingerprint at the source and the unique fingerprint at the destination. If the source unique fingerprint matches the destination unique fingerprint, it indicates that the source network resource data and the destination network resource data have the same network resource data.

7. The network resource data processing method according to claim 1, characterized in that, The method further includes: The source network resource data is divided into N source network resource data slices, and the destination network resource data is divided into N destination network resource slices; where N is an integer greater than or equal to 2; The N source network resource data slices and N destination network resource data slices are processed by N computing nodes; wherein, one computing node corresponds to one source network resource data slice and one destination network resource data slice.

8. A network resource processing device, characterized in that, The network resource processing device includes: an acquisition module, a generation module, and a processing module; The acquisition module is configured to acquire source network resource data and the source target field corresponding to the source network resource data, and destination network resource data and the destination target field corresponding to the destination network resource data. The generation module is configured to generate a corresponding source-end target fingerprint based on the source-end target field, wherein the source-end target fingerprint and the destination-end target fingerprint include at least one of source-end comparison fingerprint, source-end unique fingerprint, source-end sorted fingerprint, source-end primary key fingerprint, and source-end update fingerprint; and to obtain a corresponding destination-end target fingerprint based on the destination-end target field, wherein the destination-end target fingerprint includes at least one of destination-end comparison fingerprint, destination-end unique fingerprint, destination-end sorted fingerprint, destination-end primary key fingerprint, and destination-end update fingerprint. The processing module is configured to process the source network resource data and / or the destination network data based on the source target fingerprint and the corresponding destination target fingerprint.

9. A computer device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the network resource data processing method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on a terminal device, cause the terminal device to perform the network resource data processing method as described in any one of claims 1-7.

Images