A distributed data sharing and exchange network abnormality monitoring and early warning method and system

Abstract

The present invention discloses a distributed data sharing and exchange network anomaly monitoring and early warning method and system. The method distributes main operations to different nodes for parallel processing based on distributed technology to complete data sharing and exchange network anomaly monitoring, thereby reducing information processing time and improving system operation efficiency. Data factors are stored in a tree structure, and hash values ​​are calculated layer by layer from bottom to top for the data factors in the tree structure using a hash function to ensure information integrity. A verification method combining public and private keys with digital signatures is proposed. The public and private keys are used for private verification and inspection of user information, and the digital signature is used for shared verification and inspection of platform information, thereby reducing the possibility of information being tampered with during transmission and ensuring information security.

Description

Technical Field

[0001] The present invention relates to the technical field of big data sharing network detection, and in particular to a method and system for monitoring and warning abnormalities in a distributed data sharing and exchange network. Background Art

[0002] Big data sharing and exchange platforms offer users vast storage capacity and inherent advantages unmatched by other platforms, such as low cost, easy scalability, and fast access to stored data. In recent years, big data sharing platforms, managed by data administrators, have become widely used by individuals and businesses. However, once data is stored on the platform, users and businesses lose direct control over the stored files. Therefore, database administrators are tasked with ensuring the security of user data and protecting it from malicious attacks or tampering. Database administrators regularly check the integrity of stored data. Instead of downloading large amounts of user information, they download partial files to verify the integrity of stored data. In addition to database administrators, big data sharing and exchange platforms also have their own monitoring programs. If stored data is maliciously modified or there are unusual data requests, the monitoring programs generate reports to notify database administrators and protect the platform.

[0003] At present, the network anomaly monitoring and early warning method of the big data sharing and exchange platform is widely used in real life. According to the method of detecting information integrity, the network anomaly monitoring and early warning of the big data sharing and exchange platform can be divided into private verification and shared verification. In the present invention, private verification is performed by the user who has the right to store data. The user uses the public and private keys generated by the system to verify the information by himself, with the purpose of facilitating the user to verify the information by himself. Shared verification is performed by the data administrator in the big data sharing and exchange platform on behalf of the user. The data administrator uses the digital signature information generated by the data management terminal to verify the information, with the purpose of improving information security and reducing the user burden. In real life, shared verification is generally used, which is convenient and has high security. The value of shared verification lies in protecting data information while performing integrity verification, allowing whitelisted users to add, delete, modify and query data, and also allowing data to be shared between whitelisted users. Shared verification can also detect abnormal behavior of whitelisted users and then determine whether to terminate the user's current behavior.

[0004] Big data sharing and exchange platforms represent a new paradigm that transforms how users work and process data. They are a new technology that is increasingly integrated into every aspect of people's lives. Sharing and exchange platforms enable convenient, on-demand access to shared resources, reduce costs, and operate efficiently, making them highly attractive. An increasing number of companies and individuals are choosing big data sharing and exchange platforms as their workspaces. In the past few years, big data sharing and exchange technology has grown rapidly, attracting widespread attention in areas such as healthcare, crowdsourcing, and network anomaly monitoring and early warning. For example, in the healthcare industry, with the widespread adoption of new smart devices, the industry is shifting in a new direction. With vast amounts of data being generated and exchanged freely and rapidly, medical professionals are leveraging big data sharing and exchange platforms to improve the efficiency of data exchange, increase the frequency of data operations, and enhance data analysis capabilities. While big data sharing and exchange platforms are improving people's lives, their distributed nature can also lead to security vulnerabilities and risks. For example, user-uploaded information may contain a large amount of sensitive personal information, making it vulnerable to malicious attacks by cybercriminals, who may exploit vulnerabilities in network servers for attacks and ransomware. These attacks can weaken the service capabilities of big data sharing and exchange platforms and compromise the confidentiality and integrity of information, necessitating strengthened security measures against cyberattacks. While various authentication methods and security strategies have been proposed for network security, identity verification and access control remain the most pressing issues for big data sharing and exchange platforms. Existing network anomaly monitoring and early warning systems rely heavily on database administrators to detect incidents, which is labor-intensive and expensive. These high operating costs make it difficult to sustain the operation of big data sharing and exchange platforms. Summary of the Invention

[0005] In view of this, one of the purposes of the present invention is to provide a distributed data sharing and exchange network abnormality monitoring and early warning method, which can overcome the problems existing in the background technology.

[0006] One of the objectives of the present invention is achieved through the following technical solutions:

[0007] A distributed data sharing and exchange network abnormality monitoring and early warning method includes the following steps:

[0008] Step S1: The user terminal obtains distributed system information and generates data factors;

[0009] Step S2: The data management terminal analyzes and detects the data information of the user terminal, and applies to the system server of the big data sharing platform to generate a public-private key pair for the information;

[0010] Step S3: The user end packages the analyzed data and transmits it to the data management end. The data management end generates corresponding verification tags based on the analysis results and uploads the tagged data to the big data sharing platform.

[0011] Step S4: The big data sharing platform verifies whether the data label is correct. If the verification is successful, the platform decides to store the data and stores the packaged data in the corresponding location; if the verification fails, the platform needs to check the data integrity.

[0012] Furthermore, the step S1 of the user end obtaining the distributed system information specifically includes:

[0013] Step S101: The user checks the monitoring program of the distributed system to see if there are any abnormalities in the big data sharing and exchange system. If there are no abnormalities in the system, the user prepares to upload the data. If there are abnormalities, the user needs to wait for the administrator to investigate the possible problems and proceed to the next step until the abnormalities are resolved.

[0014] Step S102: Pack the data and generate data factors, which are then sent to the data management end in the distributed system. Using the verification information formula, the information is first grouped and hashed. Then, the distributed technology is used to continue grouping and hashing the first group of processed information. The above steps are repeated until only one hash value remains, which is then compared and verified.

[0015] Step S103: If the verification is successful, the user terminal selects an information sending channel and uses the data factor to notify other users that the user will upload information to the big data sharing platform; if the verification is unsuccessful, the user terminal is notified through the data management terminal.

[0016] Furthermore, in step S103, when the verification information is an even number, the verification information formula is:

[0017] h 1= S ( T 1), h 2= S ( T 2), h 3= S ( T 3), h 4= S ( T 4), …, h 2i = S ( T 2i )

[0018] h 11 = S ( h 1, h 2), h 12 =S ( h 3, h 4), … , h 1i = S ( h (2i-1) , h 2i )

[0019] h 21 = S ( h 11 , h 12 ), h 22 = S ( h 13 , h 14 ), … , h 2(i / 2) = S ( h 1(i-1) , h 1i )

[0020] …

[0021] h i1 = S ( h (i-1)1 , h (i-1)2 )

[0022] in, T 1 to T 2i The information that needs to be verified during the interaction between the user and the platform is also called data factor. The present invention classifies the information that needs to be verified into a tree structure. T 1 to T 2i Calculate the hash value and get h function, and obtain different hash values ​​by calculation h 1 to h 2i This process uses a distributed system to calculate the hash value in groups. The distributed system can distribute these data factors to different nodes. These nodes are independent of each other and can calculate the hash value using the above algorithm at the same time. h 1 to h 2i . Then the calculated hash values ​​are assigned to {h 1, h 2},{ h 3, h 4},…,{ h (2i-1) , h 2i}, continue to send to different nodes of the distributed system, and calculate the hash value in parallel h 11 , h 12 ,…, h 1i After getting the second set of hash values, continue pairing them as { h 11 , h 12}, { h 13 , h 14},…,{ h 1(i-1) , h 1i}, and then send it to different nodes in the distributed system to calculate the hash value in parallel h 21 , h 22 ,…, h2 (i / 2) Then continue to assign the above hash values ​​in pairs and calculate the hash value until a unique hash value is obtained. h i1 .right h i1 Verify that the data has not been tampered with. If the result is the same as the previous one, the integrity of the information is guaranteed. If there is an odd number of information to be verified and the pairing is not satisfied, the last verification information is paired with itself. Then, the hash values ​​are calculated and paired with each other until a single hash value is calculated.

[0023] Furthermore, when the number of verification information is odd, the verification information formula is:

[0024] h 1= S ( T 1), h 2= S ( T 2), h 3= S ( T 3), h 4= S ( T 4), …, h 2i-1 = S (T 2i-1 )

[0025] h 11 = S ( h 1, h 2), h 12 = S ( h 3, h 4), … , h 1i = S ( h (2i-1) )

[0026] h 21 = S ( h 11 , h 12 ), h 22 = S ( h 13 , h 14 ), … , h 2(i / 2) = S ( h 1(i-1) , h 1i )

[0027] …

[0028] h i1 = S ( h (i-1)1 , h (i-1)2 )

[0029] in, T 1 to T 2i-1 The information that needs to be verified during the interaction between the user and the platform is also called data factor. The present invention classifies the information that needs to be verified into a tree structure. T 1 to T 2i-1 Calculate the hash value and get h function, and obtain different hash values ​​by calculation h 1 to h2i-1 This process uses a distributed system to calculate the hash value in groups. The distributed system can distribute these data factors to different nodes. These nodes are independent of each other and can calculate the hash value using the above algorithm at the same time. h 1 to h 2i-1 Because the information to be verified is an odd number and does not satisfy the pairing requirement, the last hash value is h 2i-1 Perform self-matching, that is, { h 1, h 2}, { h 3, h 4},…,{ h (2i-1)}, continue to send to different nodes of the distributed system. Calculate the hash value in parallel h 11 , h 12 ,…, h 1i After getting the second set of hash values, continue pairing them as { h 11 , h 12}, { h 13 , h 14},…,{ h 1(i-1) , h 1i}, and then send it to different nodes in the distributed system to calculate the hash value in parallel h 21 , h 22 ,…, h 2(i / 2) Then continue to assign the above hash values ​​in pairs and calculate the hash value until a unique hash value is obtained. h i1 .right h i1 Verify whether it has been tampered with. If the result is the same as the previous one, the information integrity can be guaranteed. The tree structure diagram of the verification information formula for odd and even numbers is as follows: Figure 4 shown.

[0030] Furthermore, the data management terminal in step S2 analyzes and detects the user's data flow and generates a public-private key pair, which specifically includes:

[0031] Step S201: Randomly sample the data packaged in step S102, verify the validity of the data and check whether the source is reliable;

[0032] Step S202: Request the system server of the big data sharing platform to generate public and private keys for the information, and package the user's access records together with the system server for verification; after the server has the user's access records, if the system encounters an abnormal request, it can accurately locate the IP address with the problem based on the system background user access records;

[0033] Step S203: The system server receives the information and then verifies the authenticity of the information using a public-private key authentication method. If the authentication passes, the information is sent to the data management terminal.

[0034] Furthermore, step S3 packages the analyzed data and uploads it to the data management terminal, which generates a corresponding verification tag based on the analysis results, specifically including:

[0035] Step S301: After the system server allows the data management terminal to generate a public and private key, it transmits the verified information to the data management terminal;

[0036] Step S302: The data management terminal receives the data system information and then generates a public and private key for the information to facilitate information verification;

[0037] Step S303: The data management terminal generates a corresponding verification tag and tags the information. The verification tag here is a tag marked with a digital signature for the public key to facilitate information management.

[0038] Step S304: The data management terminal uploads the tagged information to the big data sharing platform and waits for the big data sharing platform to receive the information;

[0039] Furthermore, the step S4 determines whether the data tag verified by the big data sharing platform is correct, specifically including:

[0040] Step S401: The big data sharing platform receives the information uploaded by the data management terminal and then verifies the label of the information;

[0041] Step S402: If the verification is successful, the platform decides to store the data and stores the packaged data in the corresponding location;

[0042] Step S403: If the verification fails, the data integrity is checked. The present invention queries the user's access records in the system background to check whether there are unfamiliar IP addresses or IP addresses with multiple access records in a short period of time. If so, the verification algorithm in step S2 is used again to verify whether the data has been tampered with. If data tampering is detected, the platform sends a data tampering report to the data management end. The report content includes the tampered information and abnormal data application records. The data management end checks the source of tampering based on the report and the abnormal data application records, and re-uploads the user information to the big data sharing platform.

[0043] Step S404: The platform receives the data uploaded by the user, performs normal data storage, and stores the packaged data in the corresponding location. K -Anonymity algorithms encrypt, hide and blur user information to protect user personal privacy data.

[0044] Furthermore, in step S1, the verification information formula relies on the distributed system to complete, and the distributed system distributes different formulas to different nodes for parallel calculation.

[0045] Furthermore, in step S203, the public-private key verification formula is:

[0046] E ( x , k )=( r , s )

[0047]

[0048]

[0049] in, p is a large prime number, g represents the generator, a is the private key, b is the public key, k is a random number, x For information that needs to be encrypted, E ( x , k ) is the encryption function, D ( r , s ) is the decryption function. If D ( r , s ) is equal to x , it means that the information has not been tampered with.

[0050] A second object of the present invention is to provide a distributed data sharing and exchange network anomaly monitoring and early warning system, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor implements the method described above when executing the computer program.

[0051] The beneficial effects of the present invention are:

[0052] (1) The present invention combines encryption and privacy protection technologies with a distributed system to detect anomalies and provide early warnings on shared exchange networks, thereby reducing the possibility of data being maliciously tampered with. This allows staff to promptly understand security issues encountered during the operation of the platform, facilitating subsequent operation and maintenance work.

[0053] (2) The present invention uses a method combining hash value calculation with a tree structure to check information integrity. The hash value has the characteristics of fast query of an array and can integrate the advantages of convenient and fast addition and deletion of elements of a linked list. The tree structure makes the calculation results more reliable and ensures information integrity.

[0054] (3) The present invention performs parallel computing through a distributed system. The distributed system uses the method of allocating nodes to speed up the computing speed, making up for the problem of long computing time of the tree structure, so that hash computing and tree structure can play a greater role and ensure the long-term and stable development of the monitoring and early warning system platform.

[0055] (4) This invention uses a combination of digital signatures and public and private keys to verify information. Digital signatures facilitate information verification by data administrators, while public and private keys facilitate information verification by users. The combination of digital signatures and public and private keys makes it easier for the system to maintain a secure environment and ensures the long-term and stable operation of the platform.

[0056] (5) The present invention is K -Anonymity algorithms hide and obscure key information of users and administrators, simplify and summarize user information, and convert it into a string in an encoded form for easy storage; the simplified data is then hidden and restored when needed, allowing users and administrators to use the system in a secure network environment.

[0057] (6) The present invention ensures the security and stability of the platform in its daily operations through a big data sharing and exchange system, an anomaly monitoring system, and an early warning system, combined with the daily operations of the data management end. The data management end can appropriately adjust and modify the system's decisions, allowing administrators to learn from past network anomalies detected, thereby achieving the effect of early warning and better handling of anomalies that may occur in the future. Other advantages, objectives, and features of the present invention will be explained to some extent in the subsequent description and, to some extent, will be obvious to those skilled in the art based on the following investigation and research, or can be taught from the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings, in which:

[0059] Figure 1 A schematic diagram of a distributed data sharing and exchange network anomaly monitoring and early warning method according to the present invention;

[0060] Figure 2 A schematic diagram of a distributed data sharing and exchange network anomaly monitoring and early warning system of the present invention;

[0061] Figure 3 This is a schematic diagram of a distributed data sharing and exchange network anomaly monitoring and early warning module of the present invention;

[0062] Figure 4 The present invention is a schematic diagram of distributed data sharing and exchange network abnormality monitoring information verification. DETAILED DESCRIPTION

[0063] The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the preferred embodiments are only for illustrating the present invention, and are not intended to limit the scope of protection of the present invention.

[0064] like Figure 1 As shown, a distributed data sharing and exchange network abnormality monitoring and early warning method of this embodiment includes the following steps:

[0065] Step S1: The user terminal obtains distributed system information and generates data factors;

[0066] Step S2: The data management terminal analyzes and detects the data flow information of the user terminal, and applies to the system server of the big data sharing platform to generate a public-private key pair for the information;

[0067] Step S3: The user end packages the analyzed data and transmits it to the data management end. The data management end generates corresponding verification tags based on the analysis results and uploads the tagged data to the big data sharing platform.

[0068] Step S4: The big data sharing platform verifies whether the data label is correct. If the verification is successful, the platform decides to store the data and stores the packaged data in the corresponding location; if the verification fails, the platform needs to check the data integrity.

[0069] It should be noted that any process or method description in the flowcharts in the accompanying drawings or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a specific logical function or process, and that the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may be performed in a manner not shown or discussed, including in a substantially simultaneous manner or in a reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present invention belong.

[0070] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and a portable compact disc read-only memory (CDROM). Furthermore, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing it in another suitable manner if necessary, and then storing it in a computer memory.

[0071] The following will further explain each step, the early warning module is as follows Figure 2 As shown in the figure, the information verification process is as follows Figure 3 shown.

[0072] In this embodiment, step S1 includes the following sub-steps:

[0073] Step S101: The user checks the monitoring program of the distributed system to see if there are any abnormalities in the big data sharing and exchange system. If there are no abnormalities in the system, the user prepares to upload the data. If there are abnormalities, the user needs to wait for the administrator to investigate the possible problems and proceed to the next step until the abnormalities are resolved.

[0074] Step S102: Pack the data and generate data factors, which are then sent to the data management end in the distributed system. Using the verification information formula, the information is first grouped and hashed. Then, the distributed technology is used to continue grouping and hashing the first group of processed information. The above steps are repeated until only one hash value remains, which is then compared and verified.

[0075] Step S103: Pack the data and generate data factors, which are defined as T ={ T 1, T 2, T 3.… , T n The data is sent to the data management end and verified using the verification information formula. If the verification is successful, the user end selects the information sending channel and uses the data factor to notify other users that the information will be uploaded to the big data sharing platform. If the verification fails, the user end is notified through the data management end.

[0076] When the verification information in step S1 is an even number, the verification information formula is:

[0077] h 1= S ( T 1), h 2= S ( T 2), h 3= S ( T 3), h 4= S ( T 4), h 5= S ( T 5), h 6= S ( T 6), h 7= S ( T 7), h 8= S ( T 8)

[0078] h a = S ( h 1, h 2), h b = S ( h 3, h 4), h c = S ( h 5, h 6), h d = S ( h 7, h 8)

[0079] h e = S ( h a , hb ), h f = S ( h c , h d )

[0080] h g = S ( h e , h f )

[0081] in, T 1 to T 8 represents the information that needs to be verified during the interaction between the user and the platform, which is also called data factor in this invention. This invention classifies the information that needs to be verified into a tree structure. T 1 to T 8 Calculate the hash value and get h Function, in this embodiment, SHA-1, SHA256 and other algorithms are used, that is, S (•) Function = {SHA-1, SHA256, etc.}, acts as h Function calculation obtains different hash values h 1 to h 8. This process uses a distributed system to calculate the hash value in groups. The distributed system can distribute these data factors to different nodes. These nodes are independent of each other and can calculate the hash value using the above algorithm at the same time. h 1 to h 8. Then assign the calculated hash values ​​to { h 1, h 2},{ h 3, h 4},{ h 5, h 6}, { h 7, h 8}, continue to send the hash value to different nodes in the distributed system and calculate it in parallel h a , h b , h c , h d After getting the second set of hash values, continue pairing them as { h a , h b}, {h c , h d}, then send it to different nodes in the distributed system to calculate the hash value in parallel h e , h f Finally, h e , h f}Calculate the hash value and get a unique hash value h g .right h g Verify that the information has not been tampered with. If the result is the same as the previous calculation, the information integrity is guaranteed. If there is an odd number of information to be verified and the two-by-two pairing is not satisfied, the last verification information is paired with itself. Then continue to calculate the hash value and pair it with two other values ​​until a single hash value is calculated.

[0082] The verification information formula in step S1 is odd-numbered:

[0083] h 1= S ( T 1), h 2= S ( T 2), h 3= S ( T 3), h 4= S ( T 4), h 5= S ( T 5), h 6= S ( T 6), h 7= S ( T 7)

[0084] h a = S ( h 1, h 2), h b = S ( h 3, h 4), h c =S ( h 5, h 6), h d = S ( h 7)

[0085] h e = S ( h a , h b ), h f = S ( h c , h d )

[0086] h g = S ( h e , h f )

[0087] in, T 1 to T 7 represents the information that needs to be verified during the interaction between the user and the platform, which is also called data factor in the present invention. This embodiment classifies the information that needs to be verified into a tree structure. T 1 to T 7 Calculate the hash value and get h Function, the present invention adopts SHA-1, SHA256 and other algorithms, that is, S (•) Function = {SHA-1, SHA256, etc.}, acts as h Function calculation obtains different hash values h 1 to h 7. This process uses a distributed system to calculate the hash value in groups. The distributed system can distribute these data factors to different nodes. These nodes are independent of each other and can calculate the hash value using the above algorithm at the same time. h 1 to h 7. Then assign the calculated hash values ​​to { h 1, h 2},{ h 3, h 4},{ h 5, h 6}, { h7}, because the information that needs to be verified is an odd number, it does not meet the pairing requirement, so the last verification information is paired with itself. The hash values ​​of the pairings above are sent to different nodes in the distributed system to calculate the hash values ​​in parallel. h a , h b , h c , h d After getting the second set of hash values, continue pairing them as { h a , h b}, { h c , h d}, and then send it to different nodes in the distributed system to calculate the hash value in parallel h e , h f Finally, h e , h f}Calculate the hash value and get a unique hash value h g .right h g Verify whether it has been tampered with. If the result is the same as the previous calculation, the information integrity can be guaranteed.

[0088] The tree structure diagrams for odd and even numbers are as follows Figure 4 As shown in the figure, the above steps are used by data factors and data administrators to verify the legitimacy of uploaded information and check for any anomalies. If any anomalies are found, the data management end will notify the user to re-upload the information. If the user engages in malicious uploads, the data administrator or the big data information exchange platform will block the user's IP address and prevent further uploads. A hash algorithm is also used to verify information integrity, ensuring greater security.

[0089] In step S2, the data management terminal analyzes and detects the user's data flow and generates a public-private key pair, specifically including:

[0090] Step S201: Randomly sample the data packaged in step S102, verify the validity of the data and check whether the source is reliable;

[0091] Step S202: Request the system server of the big data sharing platform to generate public and private keys for the information. The public key is a certificate issued by the system. In this embodiment, the private key is a string of 64 characters, such as 453dfc4e06c5f6a5927ca8996d53094f528948ec 39ca8ed12fb76ae3a532bbfe. The system also packages the retrieved records together with the information and sends them to the system server for verification.

[0092] Step S203: The system server receives the information and verifies it using the public-private key verification formula. If the verification is successful, the system server sends the information to the data management terminal, allowing the user to proceed to the next step. If the verification message fails, the system server does not return the information and issues a warning that the information is incorrect.

[0093] Furthermore, in step S203, the public-private key verification formula is:

[0094] E ( x , k )=( r , s )

[0095]

[0096]

[0097] in, p is a large prime number, g is the generator, a is the private key, b is the public key, k is a random number between [1, 10000], x For information that needs to be encrypted, E ( x , k ) is the encryption function, D ( r , s ) is the decryption function. If D ( r , s ) is equal to x , it means that the information has not been tampered with.

[0098] The above steps leverage the connection between data management and the system, allowing the data management client to request the system to generate public and private keys for the information. This allows the data management client to maintain a backend record of data processing, allowing for later querying of the corresponding data management client, facilitating data management. The data management client uses the data's hash value as a transaction record for new events and retrieval, preventing cybercriminals from modifying data records and increasing the security of the platform's operations.

[0099] In step S3, the analyzed data is packaged and uploaded to the data management terminal, which generates corresponding verification tags based on the analysis results, including:

[0100] Step S301: After the system server allows the data management terminal to generate a public and private key, it transmits the verified information to the data management terminal;

[0101] Step S302: The data management terminal receives the data system information and then generates a public and private key for the information to facilitate information verification;

[0102] Step S303: The data management terminal generates a corresponding verification tag and tags the information. The verification tag used in the present invention is a signature. The data management terminal uses the SHA-1 algorithm to sign the information to generate specific content data. The information can also be encoded using base64 encoding.

[0103] Step S304: The data management terminal uploads the tagged information to the big data sharing platform and waits for the big data sharing platform to receive the information. After receiving the information, the big data sharing platform can use the SHA-1 algorithm to verify the data signature; it can also use base64 to decode the encoded information to check whether the information is disguised;

[0104] In step S3 above, the data administrator generates a verification tag to facilitate the system to verify the data later. If the information verification is incorrect, the problem is solved according to the prescribed solution; if no problem is found after the information verification, the subsequent steps are carried out normally.

[0105] In step S4, the big data sharing platform verifies whether the data labels are correct, specifically including:

[0106] Step S401: The big data sharing platform receives the information uploaded by the data management terminal and then verifies the label of the information;

[0107] Step S402: If the verification is successful, the platform decides to store the data and stores the packaged data in the corresponding location;

[0108] Step S403: If the verification fails, the data integrity is checked. The present invention queries the user's access records in the system background to check whether there are unfamiliar IP addresses or IP addresses with multiple access records in a short period of time. If so, the verification algorithm in step S2 is used again to verify whether the data has been tampered with. If data tampering is detected, the platform sends a data tampering report to the data management end. The report content includes the tampered information and abnormal data application records. The data management end checks the source of tampering based on the report and the abnormal data application records, and re-uploads the user information to the big data sharing platform.

[0109] Step S404: The platform receives the data uploaded by the user, performs normal data storage, and stores the packaged data in the corresponding location. K - Anonymity algorithm protects the privacy of user information, which is divided into two stages: summarization and anonymization. First, the user data is summarized and simplified to make the data more streamlined while ensuring the integrity of the information; then the user data is used to K -The anonymity algorithm hides the simplified information and restores the information when the original data needs to be extracted.

[0110] The above steps verify the integrity of the data based on the data tags added by the data management end, ensuring the safe operation of the platform, thereby achieving platform maintenance in a reliable manner.

[0111] The present invention utilizes the data management terminal to regularly check the platform security of the system big data sharing platform and publish platform information to the platform monitoring system. The data management terminal regularly checks the platform security, and the platform security information that needs to be published to the platform monitoring system specifically includes:

[0112] Regularly check whether the load of the big data sharing platform is abnormal. If an abnormality occurs, a report will be issued to the platform monitoring program to notify users that the uploaded data may be abnormal at this time, and detect the source of the platform abnormality;

[0113] Regularly check the big data sharing platform for a large number of abnormal data applications. If there are any abnormalities, temporarily close the platform upload window and check the source of the abnormal data. If it is a malicious upload, block the IP address of the upload source and prohibit uploading information;

[0114] Regularly check whether the platform monitoring program displays normal information. If the platform displays no abnormalities but the monitoring program displays errors, modify the monitoring results of the monitoring program.

[0115] In summary, the method of the present invention includes the following contents: data storage on a big data sharing platform; group processing of user and administrator information, blurring the user's important information and placing it in a hidden location; verifying the user's identity tag and checking the integrity of the data provided by the user; recording maliciously modified data, tracking the source of the modification, and returning relevant reports.

[0116] The method of the present invention reduces the possibility of data being maliciously tampered with by combining a distributed system with privacy protection technology, so that the actions taken by the system in the face of network anomalies meet the expectations of users and managers. By hiding and blurring the key information of users and managers, the user's identity label and data integrity are verified, malicious data modifications are discovered in a timely manner and relevant instructions are uploaded, so that managers can understand the dangers at the first time and take corresponding actions, thereby ensuring the long-term and stable development of the monitoring and early warning system platform.

[0117] In this embodiment, the platform's security is ensured during daily operations through a big data sharing and exchange system, anomaly monitoring system, and early warning system, combined with the daily work of the data management side. Furthermore, based on the system's response to different anomalies, the data management side can appropriately adjust and modify the system's decisions, allowing the system to learn from past behavior and better cope with future anomalies, ensuring the platform's stable development under protection.

[0118] It should be appreciated that embodiments of the present invention can be implemented or practiced by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer-readable memory. The methods can be implemented in a computer program using standard programming techniques, including a non-transitory computer-readable storage medium configured with a computer program, wherein the storage medium so configured causes the computer to operate in a specific and predefined manner, according to the methods and figures described in the specific embodiments. Each program can be implemented in a high-level procedural or object-oriented programming language to communicate with the computer system. However, if desired, the program can be implemented in assembly or machine language. In any case, the language can be a compiled or interpreted language. In addition, the program can be run on a programmed application-specific integrated circuit for this purpose.

[0119] Furthermore, the operations of the processes described herein may be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by the context. The processes described herein (or variations and / or combinations thereof) may be performed under the control of one or more computer systems configured with executable instructions and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) that is executed collectively on one or more processors, by hardware, or a combination thereof. The computer program includes a plurality of instructions that can be executed by one or more processors.

[0120] Further, the methods can be implemented in any type of computing platform that is operably connected to a suitable computer, including but not limited to a personal computer, a minicomputer, a mainframe, a workstation, a network or distributed computing environment, a separate or integrated computer platform, or in communication with a charged particle tool or other imaging device, etc. Various aspects of the present invention can be implemented as machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, an optical read and / or write storage medium, RAM, ROM, etc., so that it can be read by a programmable computer, and when the storage medium or device is read by the computer, it can be used to configure and operate the computer to perform the processes described herein. In addition, the machine-readable code, or portions thereof, can be transmitted over a wired or wireless network. When such media includes instructions or programs that implement the steps described above in conjunction with a microprocessor or other data processor, the invention described herein includes these and other different types of non-transitory computer-readable storage media. When programmed according to the methods and techniques of the present invention, the present invention also includes the computer itself. Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solutions, which should all be included in the scope of the claims of the present invention.

Claims

1. A distributed data sharing and exchange network abnormality monitoring and early warning method, characterized by: The following steps are involved: Step S1: The user terminal obtains distributed system information and generates data factors. The data factors represent the information that needs to be verified during the interaction between the user and the platform. The step S1 specifically includes: Step S101: The user checks the monitoring program of the distributed system to see if there are any abnormalities in the big data sharing and exchange system. If there are no abnormalities in the system, the user prepares to upload the data. If there are abnormalities, the user needs to wait for the administrator to investigate the possible problems and proceed to the next step until the abnormalities are resolved. Step S102: Pack the data and generate data factors, which are then sent to the data management end in the distributed system. Using the verification information formula, the information is first grouped and hashed. Then, the distributed technology is used to continue grouping and hashing the first group of processed information. The above steps are repeated until only one hash value remains, which is then compared and verified. Step S103: If the verification is successful, the user terminal selects an information transmission channel and uses the data factor to notify other users that the user will upload information to the big data sharing platform; if the verification is unsuccessful, the user terminal is notified through the data management terminal; Step S2: The data management terminal analyzes and detects the data information of the user terminal, and applies to the system server of the big data sharing platform to generate a public-private key pair for the information; Step S2 specifically includes: Step S201: Randomly sample the data packaged in step S102, verify the validity of the data and check whether the source is reliable; Step S202: Request the system server of the big data sharing platform to generate public and private keys for the information, and package the user's access records together with the system server for verification; after the server has the user's access records, if the system encounters an abnormal request, it can accurately locate the IP address with the problem based on the system background user access records; Step S203: The system server receives the information and verifies the authenticity of the information using a public-private key verification formula. If the verification is successful, the system server sends the information to the data management terminal. Step S3: The user end packages the analyzed data and transmits it to the data management end. The data management end generates corresponding verification tags based on the analysis results and uploads the tagged data to the big data sharing platform. Step S4: The big data sharing platform verifies whether the data label is correct. If the verification is successful, the platform decides to store the data and stores the packaged data in the corresponding location; if the verification fails, the platform needs to check the data integrity.

2. The method for monitoring and early warning of abnormalities in a distributed data sharing and exchange network according to claim 1, characterized in that: In step S102, when the number of verification information is even, the verification information formula is: h 1= S ( T 1), h 2= S ( T 2), h 3= S ( T 3), h 4= S ( T 4), …, h 2i = S ( T 2i ) h 11 = S ( h 1, h 2), h 12 = S ( h 3, h 4), … , h 1i = S ( h (2i-1) , h 2i ) h 21 = S ( h 11 , h 12 ), h 22 = S ( h 13 , h 14 ), … , h 2(i / 2) = S ( h 1(i-1) , h 1i ) … h i1 = S ( h (i-1)1 , h (i-1)2 ) in, T 1 to T 2i Represents the information that needs to be verified during the interaction between the user and the platform, also called data factors. The information that needs to be verified is classified into a tree structure. First, T 1 to T 2i Calculate the hash value and get h Function value, used as a hash function to calculate different hash values h 1 to h 2i ; This process is to obtain hash values ​​by grouping the distributed system, so that the system can obtain the calculation results faster; after the first group of data factors are processed, the results calculated by the verification information are paired to obtain different hash values h 1 to h 1i If the number of pieces of information to be verified is odd and does not satisfy pairing, the last piece of verification information will be paired with itself; then, the hash values ​​will continue to be calculated and paired until a single hash value is calculated.

3. The method for monitoring and early warning of abnormalities in a distributed data sharing and exchange network according to claim 2, characterized in that: When the verification information is an odd number, the verification information formula is: h 1= S ( T 1), h 2= S ( T 2), h 3= S ( T 3), h 4= S ( T 4), …, h 2i-1 = S ( T 2i-1 ) h 11 = S ( h 1, h 2), h 12 = S ( h 3, h 4), … , h 1i = S ( h (2i-1) ) h 21 = S ( h 11 , h 12 ), h 22 = S ( h 13 , h 14 ), … , h 2(i / 2) = S ( h 1(i-1) , h 1i ) … h i1 = S ( h (i-1)1 , h (i-1)2 ) in, T 1 to T 2i-1 Represents the information that needs to be verified during the interaction between the user and the platform; the information that needs to be verified is classified into a tree structure. T 1 to T 2i-1 Calculate the hash value and get h Function value, used as a hash function to calculate different hash values h 1 to h 2i-1 ; Then pair the results calculated from the verification information to get different hash values h 1 to h 1i Because the number of pieces of information that need to be verified is an odd number and does not satisfy pairing, the last piece of verification information is paired with itself; then the hash values ​​are calculated and paired with each other until a single hash value is calculated.

4. The method for monitoring and early warning of abnormalities in a distributed data sharing and exchange network according to claim 1, characterized in that: The step S3 specifically includes: Step S301: After the system server allows the data management terminal to generate a public and private key, it transmits the verified information to the data management terminal; Step S302: The data management terminal receives the data system information and then generates a public and private key for the information to facilitate information verification; Step S303: The data management terminal generates a corresponding verification tag and tags the information. The verification tag here is a tag marked with a digital signature for the public key to facilitate information management. Step S304: The data management terminal uploads the tagged information to the big data sharing platform and waits for the big data sharing platform to receive the information.

5. The method for monitoring and warning abnormalities in a distributed data sharing and exchange network according to claim 4, characterized in that: The step S4 specifically includes: Step S401: The big data sharing platform receives the information uploaded by the data management terminal and then verifies the label of the information; Step S402: If the verification is successful, the platform decides to store the data and stores the packaged data in the corresponding location; Step S403: If the verification fails, the data integrity is checked; the user's access records in the system background are queried to check whether there are unfamiliar IP addresses or IP addresses with multiple access records in a short period of time. If so, the public-private key verification formula in step S2 is used again to verify whether the data has been tampered with; if data tampering is detected, the platform sends a data tampering report to the data management end, the report content includes the tampered information and abnormal data application records. The data management end checks the source of tampering based on the report and the abnormal data application records, and re-uploads the user information to the big data sharing platform; Step S404: The platform receives the data uploaded by the user, performs normal data storage, and stores the packaged data in the corresponding location; K -Anonymous algorithms encrypt, hide and blur user information to protect user personal privacy data.

6. The method for monitoring and early warning of abnormalities in a distributed data sharing and exchange network according to claim 1, characterized in that: The verification information formula in step S1 relies on the distributed system to complete, and the distributed system distributes different formulas to different nodes for parallel calculation.

7. The method for monitoring and early warning of abnormalities in a distributed data sharing and exchange network according to claim 1, characterized in that: In step S203, the public-private key verification formula is: E ( x , k )=( r , s ) in, p is a large prime number, g represents the generator, a is the private key, b is the public key, k is a random number, x For information that needs to be encrypted, E ( x , k ) is the encryption function, D ( r , s ) is the decryption function; if D ( r , s ) is equal to x , it means that the information has not been tampered with.

8. A distributed data sharing and exchange network anomaly monitoring and early warning system, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

Images