A pipeline data privacy calculation method and system based on distributed collaboration

By introducing a distributed collaboration mechanism in traditional systems, using coordination centers and distributed system nodes for data access and encryption processing, the problem of insufficient large-scale concurrent requests and fault processing capabilities of traditional systems is solved, and more efficient and secure Pipeline data access.

CN118101254BActive Publication Date: 2024-12-06BEIJING ZHICHENG JOINT TECH DEV CO LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202410150390.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-02-02
Publication Date
2024-12-06
Estimated Expiration
2044-02-02

AI Technical Summary

Technical Problem

The data access and calling methods in traditional systems lack flexibility, especially when facing large-scale concurrent requests or system node failures, the traditional system's processing capabilities of data are further reduced, affecting users' calls to pipeline data.

Method used

Adopt a pipeline data privacy calculation method based on distributed collaboration, and perform data access and encryption calculations on distributed system nodes through the coordination center, use common nodes and redundant nodes to perform data replication, storage and encryption processing, and troubleshoot when network failures are made and overhaul to ensure the reliability and security of data access.

Benefits of technology

Improve the reliability and security of data access, enhance the processing capability of large-scale concurrent requests and system failures, and significantly improve the speed of data access.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118101254B_ABST
    Figure CN118101254B_ABST
Patent Text Reader

Abstract

The present invention discloses a pipeline data privacy calculation method and system based on distributed collaboration, which relates to the technical field of pipeline data privacy calculation, including collecting pipeline data, and storing the data in distributed system nodes by classification; users call the data through a coordination center, and the coordination center accesses the pipeline data through a distributed system; the distributed system selects nodes to perform encryption calculations on the stored pipeline data and generates calculation results and sends them to the coordination center. The beneficial effects of the present invention are as follows: the present invention provides a pipeline data privacy calculation method based on distributed collaboration, which combines distributed storage and encryption technology, improves the reliability and security of data access, and at the same time greatly improves the data access speed by processing user needs through distributed system collaboration, which has significant advantages.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of pipeline data privacy computing, and in particular to a pipeline data privacy computing method and system based on distributed collaboration. Background Art

[0002] With the development of information technology and the popularization of big data, the management and protection of pipeline data has become increasingly important. In traditional data management systems, pipeline data is generally stored and processed centrally, which can easily lead to data bottlenecks and delays. At the same time, there is also a risk of data being stolen or tampered with in a single system. In order to enhance data privacy protection, various encryption technologies have also been incorporated into the data processing process, but this often increases the complexity of calculations. At the same time, the data access and calling methods in traditional systems lack sufficient flexibility, especially in the face of large-scale concurrent requests or system node failures. The traditional system's ability to process data is further reduced, affecting users' calls to pipeline data. Summary of the invention

[0003] The purpose of this section is to summarize some aspects of embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the specification abstract and the invention title of this application to avoid blurring the purpose of this section, the specification abstract and the invention title, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0004] In view of the above-mentioned and / or existing problems in the pipeline data privacy calculation method based on distributed collaboration, the present invention is proposed.

[0005] Therefore, the problem to be solved by the present invention is that the data access and calling methods in traditional systems lack sufficient flexibility, especially when facing large-scale concurrent requests or system node failures, the traditional system's data processing capabilities are further reduced, affecting users' calls to pipeline data.

[0006] To solve the above technical problems, the present invention provides the following technical solutions: a pipeline data privacy calculation method based on distributed collaboration, which includes collecting pipeline data and storing the data in distributed system nodes by classification;

[0007] Users call data through the coordination center, and the coordination center accesses pipeline data through the distributed system;

[0008] The distributed system selects nodes to perform encrypted calculations on the stored pipeline data and generates calculation results to be sent to the coordination center;

[0009] The coordination center summarizes the calculation results and sends the encrypted pipeline data to the user, and the user decrypts the encrypted data and restores it to the original data.

[0010] As a preferred solution of the pipeline data privacy calculation method based on distributed collaboration described in the present invention, wherein: after collecting the pipeline data, the data is classified according to data features, the distributed system is provided with common nodes and redundant nodes, and after classifying the pipeline data, the data is copied and stored in the common nodes and redundant nodes, and all distributed system nodes are connected through a network.

[0011] As a preferred solution of the pipeline data privacy calculation method based on distributed collaboration described in the present invention, wherein: the coordination center sends an access request to the distributed system through the network, including:

[0012] If the coordination center successfully accesses the distributed system, it means that the network status is good, and the coordination center and the distributed system exchange data through the network;

[0013] If the coordination center sends an access request but the access fails, the distributed system is troubleshooted and the access request is sent to the distributed system again;

[0014] If the coordination center times out when sending an access request, it means that the network status is poor. The coordination center performs troubleshooting and debugging on the network, and after the troubleshooting, sends a data access request to the distributed system again.

[0015] As a preferred solution of the pipeline data privacy calculation method based on distributed collaboration described in the present invention, the coordination center first queries the data to be called when accessing the pipeline data through the distributed system, and the query results include:

[0016] If the call data is completely stored in a single access node, the storage pipeline data is called directly;

[0017] If the called data is stored in multiple access nodes, all data storage locations are queried, and data access requests are synchronously sent to the storage data nodes through the distributed system;

[0018] If the call data does not exist in all distributed system nodes, the pipeline data selected by the user is collected again and recorded in the node according to the feature classification, and the user request is transferred to the node for data call.

[0019] As a preferred solution of the pipeline data privacy calculation method based on distributed collaboration described in the present invention, after the coordination center sends an access request to the distributed system, the distributed system needs to perform encryption calculation on the pipeline data, and the distributed system selects the calculation node:

[0020] If it is detected that the common nodes and redundant nodes of the distributed system are in an idle state, the distributed system will give priority to the common nodes to perform encryption calculations on pipeline data;

[0021] If it is detected that all common nodes of the distributed system are in working state and there are idle redundant nodes, the distributed system selects the idle redundant nodes to perform encryption calculation on the pipeline data;

[0022] If it is detected that all nodes of the distributed system are in working state or the idle common nodes and redundant nodes are damaged, all working nodes are sorted according to the remaining time of pipeline data encryption processing, and the node with the least remaining working time is selected according to the sorting to queue the task.

[0023] As a preferred solution of the pipeline data privacy calculation method based on distributed collaboration described in the present invention, when the distributed system node performs encryption calculation on pipeline data, it first generates a data encryption public key and a decryption private key:

[0024] n=p×q

[0025] λ=lcm(p-1,q-1)

[0026] Where p and q are two different prime numbers selected randomly, and lcm is the least common multiple;

[0027] μ=[L(g λ modn 2 )] -1 modn

[0028]

[0029] The random number g belongs to n 2 The multiplicative group of , from the above formula, the public key is (n, g) and the private key is (λ, μ).

[0030] As a preferred solution of the pipeline data privacy calculation method based on distributed collaboration described in the present invention, wherein: the distributed system node encrypts the pipeline data after obtaining the encryption public key:

[0031] c=g m × n mod n 2

[0032] Where m is the encrypted pipeline data plaintext, r is a randomly selected random number, and 0<r<n, and r and n are mutually prime. After the encryption calculation is completed, the distributed system node sends the encrypted data to the coordination center, and the coordination center transmits the encrypted data and the decryption private key to the user, and the user decrypts the data:

[0033] m=L(c λ mod n 2 )×μmod n

[0034] Where m is the pipeline data plaintext generated by re-decryption.

[0035] Another object of the present invention is to provide a pipeline data privacy computing system based on distributed collaboration, which includes a collection module, a transfer module, an identification module and a processing module;

[0036] The collection module includes a data collection submodule and a data cleaning submodule. The data collection submodule is used to collect pipeline data, and the data cleaning submodule is used to clean and filter a large amount of collected pipeline data to reduce redundant and erroneous pipeline data.

[0037] The transfer module is used to receive the user request and send it to the identification module and the processing module to process the user request;

[0038] The identification module is used to identify the characteristics of the pipeline data collected by the collection module, classify the pipeline data according to the characteristics, and also to identify and classify user requests to determine whether the user calls the data storage location to access the distributed system node;

[0039] The processing module is used to encrypt the pipeline data in the distributed system node and transmit the encrypted data to the transfer module.

[0040] A computer device comprises: a memory and a processor; the memory stores a computer program, characterized in that: when the processor executes the computer program, the steps of the pipeline data privacy calculation method based on distributed collaboration in any one of claims 1 to 7 are implemented.

[0041] A computer-readable storage medium having a computer program stored thereon, characterized in that: when the computer program is executed by a processor, the steps of any one of the pipeline data privacy calculation methods based on distributed collaboration in claims 1 to 7 are implemented.

[0042] The beneficial effects of the present invention are as follows: the present invention provides a pipeline data privacy calculation method based on distributed collaboration, which combines distributed storage and encryption technology, improves the reliability and security of data access, and at the same time greatly improves the data access speed by processing user needs through distributed system collaboration, which has significant advantages. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:

[0044] Figure 1 The flowchart of the pipeline data privacy calculation method based on distributed collaboration is shown in FIG.

[0045] Figure 2 The figure is a flow chart of the pipeline data privacy computing system based on distributed collaboration.

[0046] Figure 3 Schematic diagram of the equipment of the pipeline data privacy computing system based on distributed collaboration. DETAILED DESCRIPTION

[0047] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0048] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0049] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0050] Example 1

[0051] Reference Figure 1 , which is the first embodiment of the present invention, and provides a pipeline data privacy calculation method based on distributed collaboration. The pipeline data privacy calculation method based on distributed collaboration includes the following steps:

[0052] S1, collect pipeline data, and store the data in distributed system nodes;

[0053] It should be noted that after collecting pipeline data, the data is classified according to data features. The distributed system is provided with common nodes and redundant nodes. After the pipeline data is classified, the data is copied and stored in the common nodes and redundant nodes. All distributed system nodes are connected through a network.

[0054] S2. The user calls the data through the coordination center, and the coordination center accesses the pipeline data through the distributed system;

[0055] It should be noted that the coordination center sends access requests to the distributed system through the network, including:

[0056] If the coordination center successfully accesses the distributed system, it means that the network status is good, and the coordination center and the distributed system exchange data through the network;

[0057] If the coordination center sends an access request but the access fails, the distributed system is troubleshooted and the access request is sent to the distributed system again;

[0058] If the coordination center times out when sending an access request, it means that the network status is poor. The coordination center performs troubleshooting and debugging on the network, and after the troubleshooting, sends a data access request to the distributed system again.

[0059] It should also be noted that the coordination center first queries the data to be called when accessing the pipeline data through the distributed system. The query results include:

[0060] If the call data is completely stored in a single access node, the storage pipeline data is called directly;

[0061] If the called data is stored in multiple access nodes, all data storage locations are queried, and data access requests are synchronously sent to the storage data nodes through the distributed system;

[0062] If the call data does not exist in all distributed system nodes, the pipeline data selected by the user is collected again and recorded in the node according to the feature classification, and the user request is transferred to the node for data call.

[0063] S3, the distributed system selects a node to perform encryption calculation on the stored pipeline data and generates calculation results and sends them to the coordination center;

[0064] It should be noted that after the coordination center sends an access request to the distributed system, the distributed system needs to perform encryption calculations on the pipeline data, and the distributed system selects the computing nodes:

[0065] If it is detected that the common nodes and redundant nodes of the distributed system are in an idle state, the distributed system will give priority to the common nodes to perform encryption calculations on pipeline data;

[0066] If it is detected that all common nodes of the distributed system are in working state and there are idle redundant nodes, the distributed system selects the idle redundant nodes to perform encryption calculation on the pipeline data;

[0067] If it is detected that all nodes of the distributed system are in working state or the idle common nodes and redundant nodes are damaged, all working nodes are sorted according to the remaining time of pipeline data encryption processing, and the node with the least remaining working time is selected according to the sorting to queue the task.

[0068] It should also be noted that when the distributed system node performs encryption calculations on pipeline data, it first generates a data encryption public key and a decryption private key:

[0069] n=p×q

[0070] λ=lcm(p-1,q-1)

[0071] Where p and q are two different prime numbers selected randomly, and lcm is the least common multiple;

[0072] μ=[L(g λ modn 2 )] -1 modn

[0073]

[0074] The random number g belongs to n 2 The multiplicative group of , from the above formula, the public key is (n, g) and the private key is (λ, μ).

[0075] Furthermore, the distributed system node encrypts the pipeline data after obtaining the encryption public key:

[0076] c=g m × n mod n 2

[0077] Wherein m is the encrypted pipeline data plaintext, r is a randomly selected random number, 0<r<n, and r and n are mutually prime.

[0078] S4. The coordination center summarizes the calculation results and sends the encrypted pipeline data to the user, and the user decrypts the encrypted data and restores it to the original data;

[0079] It should be noted that the distributed system nodes send the encrypted data to the coordination center, and the coordination center transmits the encrypted data and the decryption private key to the user, who decrypts the data:

[0080] m=L(c λ mod n 2 )×μmod n

[0081] Where m is the pipeline data plaintext generated by re-decryption.

[0082] Example 2

[0083] Reference Figure 2 and Figure 3 , which is the second embodiment of the present invention, and which provides a pipeline data privacy computing system based on distributed collaboration, characterized in that it includes a collection module, a transfer module, an identification module and a processing module;

[0084] The collection module includes a data collection submodule and a data cleaning submodule. The data collection submodule is used to collect pipeline data, and the data cleaning submodule is used to clean and filter a large amount of collected pipeline data to reduce redundant and erroneous pipeline data.

[0085] The transfer module is used to receive the user request and send it to the identification module and the processing module to process the user request;

[0086] The identification module is used to identify the characteristics of the pipeline data collected by the collection module, classify the pipeline data according to the characteristics, and also to identify and classify user requests to determine whether the user calls the data storage location to access the distributed system node;

[0087] The processing module is used to encrypt the pipeline data in the distributed system node and transmit the encrypted data to the transfer module.

[0088] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0089] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0090] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0091] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0092] Example 3

[0093] This is the third embodiment of the present invention. This embodiment is different from the previous two embodiments. In order to verify the beneficial effects of the present invention, the method of the present invention and the traditional method are demonstrated through comparison and experiments. The comparison and experimental results are shown in the following table.

[0094] Table 1: Comparison between the method of the present invention and the traditional method

[0095]

[0096] Table 2: Experimental demonstration of the method of the present invention and the traditional method

[0097]

[0098]

[0099] It can be concluded from the above table that the method of the present invention has obvious advantages over the traditional method in data calling, processing, security assurance and data backup and recovery, and the user satisfaction is also higher than the traditional method, which is sufficient to show that the method of the present invention is superior to the traditional method in pipeline data privacy computing.

[0100] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A pipeline data privacy calculation method based on distributed collaboration, characterized by: include, Collect pipeline data and store the data in distributed system nodes in a classified manner; Users call data through the coordination center, and the coordination center accesses pipeline data through the distributed system; The distributed system selects nodes to perform encrypted calculations on the stored pipeline data and generates calculation results to be sent to the coordination center; The coordination center summarizes the calculation results and sends the encrypted pipeline data to the user, and the user decrypts the encrypted data and restores it to the original data; After collecting the pipeline data, the data is classified according to the data features. The distributed system is provided with common nodes and redundant nodes. After the pipeline data is classified, the data is copied and stored in the common nodes and redundant nodes. All the distributed system nodes are connected through a network. The coordination center sends an access request to the distributed system through the network, including: If the coordination center successfully accesses the distributed system, it means that the network status is good, and the coordination center and the distributed system exchange data through the network; If the coordination center sends an access request but the access fails, the distributed system is troubleshooted and the access request is sent to the distributed system again; If the coordination center times out when sending an access request, it means that the network status is poor. The coordination center performs troubleshooting and debugging on the network, and after the troubleshooting, sends a data access request to the distributed system again; The coordination center accesses the pipeline data through the distributed system and first queries the data to be called. The query results include: If the call data is completely stored in a single access node, the storage pipeline data is called directly; If the called data is stored in multiple access nodes, all data storage locations are queried, and data access requests are synchronously sent to the storage data nodes through the distributed system; If the called data does not exist in all distributed system nodes, the pipeline data selected by the user is collected again and recorded in the node according to the feature classification, and the user request is transferred to the node for data calling; After the coordination center sends an access request to the distributed system, the distributed system needs to perform encryption calculations on the pipeline data, and the distributed system selects the computing nodes: If it is detected that the common nodes and redundant nodes of the distributed system are in an idle state, the distributed system will give priority to the common nodes to perform encryption calculations on pipeline data; If it is detected that all common nodes of the distributed system are in working state and there are idle redundant nodes, the distributed system selects the idle redundant nodes to perform encryption calculation on the pipeline data; If it is detected that all nodes of the distributed system are in working state or the idle common nodes and redundant nodes are damaged, all working nodes are sorted according to the remaining time of pipeline data encryption processing, and the node with the least remaining working time is selected according to the sorting to queue the task.

2. The pipeline data privacy calculation method based on distributed collaboration as claimed in claim 1, characterized in that: When the distributed system node performs encryption calculation on pipeline data, it first generates a data encryption public key and a decryption private key: n=p×q λ=lcm(p-1,q-1) Where p and q are two different prime numbers selected randomly, and 1cm is the least common multiple; μ=[L(g λ towards n 2 )] -1 towards n The random number g belongs to n 2 The multiplication group of , from the above formula, the public key is (n, g) and the private key is (λ, μ).

3. The pipeline data privacy calculation method based on distributed collaboration as claimed in claim 2, characterized in that: The distributed system node encrypts the pipeline data after obtaining the encryption public key: c=g m ×r n mod n 2 Where m is the plaintext of the encrypted pipeline data, r is a randomly selected random number, and 0 < r ζ n, and r is relatively prime to n. After the encryption calculation is completed, the distributed system node sends the encrypted data to the coordination center, and the coordination center transmits the encrypted data and the decryption private key to the user, and the user decrypts the data: m=L(c λ mod n 2 )×μ mod n Where m is the plaintext of the pipeline data regenerated by re-decryption.

4. A distributed collaborative pipeline data privacy computing system based on the distributed collaborative pipeline data privacy computing method according to any one of claims 1 to 3, characterized in that: Including a collection module, a transfer module, an identification module, and a processing module; The collection module includes a data collection sub-module and a data cleaning sub-module. The data collection sub-module is used to collect pipeline data, and the data cleaning sub-module is used to clean and screen a large amount of collected pipeline data to reduce redundant and incorrect pipeline data; The transfer module is used to receive a user request and send it to the identification module and the processing module to process the user request; The identification module is used to perform feature identification on the pipeline data collected by the collection module, classify the pipeline data according to features, and is also used to identify and classify user requests, and judge the data storage location called by the user to access the distributed system node; The processing module is used to encrypt the pipeline data in the distributed system node and transmit the data to the transfer module after encryption.

5. A computer device comprising: A memory and a processor; The memory stores a computer program, characterized in that: when the processor executes the computer program, the steps of the method according to any one of claims 1 to 3 are implemented.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 3 are implemented.

Citation Information

Patent Citations

  • Ship virtual test storage and calculation separation system based on cloud platform

    CN116821239A