Data conjoint analysis method and device, computer equipment and storage medium

Abstract

The embodiment of the invention provides a data conjoint analysis method, a data conjoint analysis device, computer equipment and a computer storage medium, and relates to the technical field of data processing. The method comprises the following steps: in response to receiving a conjoint analysis task sent by a data analysis terminal, obtaining shared data which is released by each task participation terminal and contains a shared user entity from a joint block chain jointly constructed by each task participation terminal, the conjoint analysis task at least containing an initial conjoint analysis model; target gradient information is determined according to the shared data corresponding to the shared user entity, and the target gradient information is stored to the joint block chain, so that each task participation terminal updates the model parameters of the corresponding initial joint analysis model according to the target gradient information, and obtaining a conjoint analysis result based on the updated target conjoint analysis model. According to the method, the single-point fault problem can be solved, and the safety and privacy of model parameters in the model training process of federated learning can be improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular, to a data joint analysis method, a data joint analysis device, a computer device, and a computer storage medium. Background Art

[0002] With the advent of the big data era, various types of systems are becoming more and more complex, while the types of data collected by a single system are limited, and the corresponding analysis value is low. By jointly analyzing the data of multiple systems, their deep-level value can be efficiently mined. Taking the rail transit application scenario in the railway industry as an example, railway data covers passenger data, freight data, train operation data, equipment data and other types of data, which plays an increasingly important role in guiding railway operations, management, planning and other aspects. In order to improve the safety and rationality of railway transportation and promote railway development, it is usually necessary to jointly analyze railway data with external data such as traffic flow, weather conditions, and socio-economic data.

[0003] However, the railway data and external data in the above application scenarios are usually stored in different systems. The railway data is limited to the internal railway system, while the external data is stored in the external system, thus forming a data island problem. This undoubtedly increases the difficulty of data fusion and analysis between railway data and external data.

[0004] Traditional technical solutions usually use centralized federated learning methods to conduct joint data analysis between railway data and external data in different systems, thereby solving the above-mentioned data island problem. However, since the above method requires a trusted central server to achieve model aggregation, it is very easy to cause single point failure problems, and the credit security issues of model parameters arise during the model training process of federated learning. Summary of the invention

[0005] In the embodiments of the present application, a data joint analysis method, a data joint analysis device, a computer device and a computer storage medium are provided, thereby overcoming, at least to a certain extent, the single point failure problem that is easily generated in the cross-system data joint analysis process due to the limitations and defects of the relevant technology, and the technical problem of the credit security of the model parameters generated in the model training process of the federated learning.

[0006] In a first aspect of an embodiment of the present application, a data joint analysis method is provided, the method comprising: in response to receiving a joint analysis task sent by a data analysis terminal, obtaining shared data including a common user entity published by each task participating terminal from a joint blockchain jointly constructed by each task participating terminal, the joint analysis task including at least an initial joint analysis model; determining target gradient information according to the shared data corresponding to the common user entity, and storing the target gradient information in the joint blockchain, so that each task participating terminal can update the model parameters of the corresponding initial joint analysis model according to the target gradient information, and obtain the joint analysis result based on the updated target joint analysis model.

[0007] According to a second aspect of an embodiment of the present application, a data joint analysis device is provided, which includes: a data acquisition module, which is used to respond to a joint analysis task sent by a receiving data analysis terminal, and obtain shared data containing a common user entity published by each task participating terminal from a joint blockchain jointly constructed by each task participating terminal, wherein the joint analysis task at least includes an initial joint analysis model; an information determination module, which is used to determine target gradient information based on the shared data corresponding to the common user entity, and store the target gradient information in the joint blockchain, so that each task participating terminal can update the model parameters of the corresponding initial joint analysis model based on the target gradient information, and obtain the joint analysis result based on the updated target joint analysis model.

[0008] According to a third aspect of an embodiment of the present application, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of any of the above data joint analysis methods are implemented.

[0009] According to a fourth aspect of the embodiments of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above data joint analysis methods are implemented.

[0010] According to a fifth aspect of an embodiment of the present application, a computer program product is provided, including a computer program, which is executed by a processor to implement the steps of any one of the above data joint analysis methods.

[0011] The technical solution of this application has the following beneficial effects:

[0012] Through the data joint analysis method, in response to receiving a joint analysis task sent by a data analysis terminal, the shared data corresponding to the common user entity and published by each task participating terminal is obtained from the joint blockchain jointly constructed by each task participating terminal, and the joint analysis task at least includes an initial joint analysis model; the target gradient information is determined according to the shared data corresponding to the common user entity, and the target gradient information is stored in the joint blockchain, so that each task participating terminal can update the model parameters of the corresponding initial joint analysis model according to the target gradient information, and obtain the joint analysis result based on the updated target joint analysis model.

[0013] In this method, on the one hand, the shared data corresponding to the common user entity is stored in the constructed joint blockchain, so as to utilize the security of blockchain technology to connect the nodes in the federated learning network to the joint blockchain, realize the distributed storage and transmission of model update parameters, reduce the computational burden of the coordination analysis server, and improve the scalability and robustness of the system. On the other hand, during the entire data coordination and analysis process, the data between the task participating terminals are isolated from each other, so it is impossible to obtain the original data and features of other task participating terminals, and the training and updating process of model parameters can only obtain model parameters related to themselves, thus ensuring the security and privacy of data and model parameters. In addition, storing data in the joint blockchain avoids the single point failure problem of the system, thereby improving the stability and reliability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0015] Figure 1 An architecture diagram of a data joint analysis system provided for one embodiment of the present application;

[0016] Figure 2 A flowchart of a data joint analysis method provided in one embodiment of the present application;

[0017] Figure 3 A schematic diagram of a secure access mechanism provided for an embodiment of the present application;

[0018] Figure 4 A schematic diagram of the structure of a data joint analysis device provided in one embodiment of the present application;

[0019] Figure 5 A schematic diagram of the computer device structure provided for one embodiment of the present application. DETAILED DESCRIPTION

[0020] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as being limited to the examples set forth herein; on the contrary, these embodiments are provided so that the present disclosure will be more comprehensive and complete, and the concepts of the example embodiments are fully conveyed to those skilled in the art. The described features, structures, or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known technical solutions are not shown or described in detail to avoid obscuring various aspects of the present disclosure.

[0021] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and thus their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0022] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the steps. For example, some steps may be decomposed, while some steps may be combined or partially combined, so the actual execution order may change according to the actual situation. Summary of the invention:

[0024] With the advent of the big data era, various types of systems are becoming more and more complex, while the types of data collected by a single system are limited, and the corresponding analysis value is low. By jointly analyzing the data of multiple systems, its deep-level value can be efficiently mined. The embodiment of this application takes the rail transit application scenario of the railway industry as an example to illustrate the application scenario of the data joint analysis method provided by this application.

[0025] With the rapid development of the railway industry and the improvement of information level, the scale and complexity of railway data are increasing. These data cover passenger data, freight data, train operation data, equipment data and other aspects, and play an increasingly important role in railway operation, management, planning, etc. In order to improve the safety and rationality of railway transportation and promote railway development, it is usually necessary to jointly analyze railway data with other external data, such as traffic flow, weather conditions, socio-economic data, etc.

[0026] The inventors found that existing railway data and external data are usually stored in different systems, that is, railway data is often limited to its internal railway system, while external data is stored in other external systems, thus forming a data island problem, which will lead to the difficulty of data fusion and joint analysis between the railway data of the internal system and the external data of other external systems. In addition, the existing railway data sharing methods of the internal railway system are mainly data interfaces, offline file transfers, etc. These methods have certain limitations in ensuring data privacy, which brings data security issues and limits the discovery of the potential value of data. For example, railway data contains a large amount of sensitive information, such as passenger identity information, freight demand information, train operation information and other private data. Therefore, when jointly analyzing with external data, how to ensure the security and privacy of data has become a major challenge.

[0027] In order to solve the above technical problems, in the relevant technical solutions, the centralized federated learning method is usually used for data joint analysis. Although it can solve the above data island problem, it requires a trusted central server to achieve model aggregation. However, on the one hand, the above method is very prone to single point failure problems, and a single point of failure (SPOF) refers to a failure in a point in the system, which will cause the entire system to fail to operate. For example, if the above trusted central server fails, it will affect the stability and availability of the entire system. On the other hand, there is a credit security problem of using model parameters in the model training process in centralized federated learning, and the above related technical solutions are also unable to solve the security problems of privacy data brought about in the process of data joint analysis.

[0028] In summary, there is an urgent need to achieve joint analysis of railway data and external data while protecting data privacy, so as to empower railway development, which has become an important issue facing the current railway industry.

[0029] The exemplary implementation of the present application takes the above-mentioned problems into consideration, and the basic idea of ​​the present application is to provide a data joint analysis method, a data joint analysis device, a computer device and a computer storage medium, which stores the shared data corresponding to the common user entity in the constructed joint blockchain, thereby utilizing the security of blockchain technology to connect the nodes in the federated learning network to the joint blockchain, realize the distributed storage and transmission of model update parameters, reduce the computational burden of the coordination analysis server, and improve the scalability and robustness of the system. In addition, during the entire data coordination and analysis process, the data between the terminals participating in each task are all related and isolated, and it is impossible to understand the data and characteristics of other task participating terminals. The training and updating process of the model parameters can only obtain the model parameters related to themselves, thereby ensuring the security and privacy of the data and model parameters. In addition, storing the data in the joint blockchain avoids the single point failure problem of the system, thereby improving the stability and reliability of the system.

[0030] Application scenario overview:

[0031] The data joint analysis method provided in the embodiment of the present application can realize cross-system data joint analysis, such as joint analysis of railway data in the railway internal system and external data in other external systems, so as to play a positive role in railway operation, management and planning, thereby providing users with a better travel experience. Of course, it can also be applied to other application scenarios, such as shopping systems, express delivery systems, etc., and the embodiment of the present application does not impose any special restrictions on this.

[0032] It should be noted that the data joint analysis method provided in this application can also be applied to other fields, such as abnormal user identification in railway risk control services, high-value user identification in extended services, etc.

[0033] It should be noted that the above application scenarios are only shown to facilitate understanding of the spirit and principles of the present application. The implementation methods of the present application are not limited in this regard, but are applied to any applicable scenarios.

[0034] System architecture:

[0035] In order to solve the above problems, the present invention proposes a data joint analysis method and device, which can be applied to Figure 1 In the system architecture of the exemplary application environment shown.

[0036] like Figure 1As shown, the system architecture 100 may include one or more of the task participating terminals 101, 102, 103, 104, a data analysis terminal 105 and a joint analysis server 106, and a medium providing a communication link between the task participating terminals 101, 102, 103, 104, the data analysis terminal 105 and the joint analysis server 106 through a network. The network may include various connection types, such as wired, wireless communication links or optical fiber cables, etc. One or more of the task participating terminals 101, 102, 103, 104, and the data analysis terminal 105 may be, for example, a smart phone, a PDA (Personal Digital Assistant), a laptop, a server, a desktop computer or any other computing device with networking capabilities, but is not limited thereto. The joint analysis server 106 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), as well as big data and artificial intelligence platforms. The embodiments of the present disclosure do not impose any special restrictions on this.

[0037] It should be understood that Figure 1 The number of task participating terminals, data analysis terminals 105, joint analysis servers 106, and networks are merely illustrative. Depending on implementation requirements, there may be any number of terminal devices, networks, and servers, and the embodiments of the present disclosure do not impose any special restrictions on this.

[0038] For example, in an exemplary embodiment, in response to receiving a joint analysis task sent by a data analysis terminal, shared data including common user entities published by each task participating terminal is obtained from a joint blockchain jointly constructed by each task participating terminal, and the joint analysis task at least includes an initial joint analysis model; target gradient information is determined based on the shared data corresponding to the common user entity, and the target gradient information is stored in the joint blockchain, so that each task participating terminal can update the model parameters of the corresponding initial joint analysis model according to the target gradient information, and obtain the joint analysis results based on the updated target joint analysis model.

[0039] Exemplary methods:

[0040] After understanding the architecture diagram of the data joint analysis system provided in the embodiment of the present application, the following will be combined with Figure 2 The scheme of the data joint analysis method of the present application is described in detail.

[0041] The following uses the above-mentioned joint analysis server 105 as an execution subject and illustrates the application of the data joint analysis method to the above-mentioned joint analysis server 105 as an example. Figure 2 A flowchart schematically shows a data joint analysis method in this exemplary embodiment, see Figure 2 The data joint analysis method provided by the embodiment of the present disclosure includes the following steps 201-202:

[0042] Step 201: In response to receiving a joint analysis task sent by a data analysis terminal, shared data including common user entities published by each task participating terminal is obtained from a joint blockchain jointly constructed by each task participating terminal, and the joint analysis task at least includes an initial joint analysis model.

[0043] Step 202: determine the target gradient information according to the shared data corresponding to the common user entity, and store the target gradient information in the joint blockchain, so that each task participating terminal can update the model parameters of the corresponding initial joint analysis model according to the target gradient information, and obtain the joint analysis result based on the updated target joint analysis model.

[0044] In the technical solutions provided by some embodiments of the present disclosure, by responding to a joint analysis task sent by a receiving data analysis terminal, the shared data of the common user entities published by each task participating terminal is obtained from the joint blockchain jointly constructed by each task participating terminal, and the joint analysis task at least includes an initial joint analysis model; the target gradient information is determined according to the shared data corresponding to the common user entity, and the target gradient information is stored in the joint blockchain, so that each task participating terminal can update the model parameters of the corresponding initial joint analysis model according to the target gradient information, and obtain the joint analysis results based on the updated target joint analysis model.

[0045] In this method, on the one hand, the shared data corresponding to the common user entity is stored in the constructed joint blockchain, so as to utilize the security of blockchain technology to connect the nodes in the federated learning network to the joint blockchain, realize the distributed storage and transmission of model update parameters, reduce the computational burden of the coordination analysis server, and improve the scalability and robustness of the system. On the other hand, during the entire data coordination and analysis process, the data between the participating terminals of each task are all related and isolated, and the data and characteristics of the participating terminals of other tasks cannot be understood. The training and updating process of model parameters can only obtain model parameters related to themselves, thus ensuring the security and privacy of data and model parameters. In addition, storing data in the joint blockchain avoids the single point failure problem of the system, thereby improving the stability and reliability of the system.

[0046] The following will be combined with specific embodiments to Figure 2The specific implementation methods of each step in the embodiment shown are described in detail:

[0047] In step 201, in response to receiving a joint analysis task sent by a data analysis terminal, shared data containing common user entities published by each task participating terminal is obtained from a joint blockchain jointly constructed by each task participating terminal, and the joint analysis task at least includes an initial joint analysis model.

[0048] Among them, each task participating terminal can jointly build a joint blockchain, that is, a joint blockchain is formed by installing and configuring blockchain nodes on the server corresponding to each task participating terminal.

[0049] Exemplarily, the joint blockchain jointly constructed by each task participating terminal stores the shared data corresponding to the common user entity of each task participating terminal, and the coordination analysis server can obtain the shared data corresponding to the common user entity of each task participating terminal from the joint blockchain.

[0050] In an optional embodiment of the present disclosure, each task participating terminal and the joint analysis server comply with a consensus mechanism.

[0051] Among them, the consensus mechanism (also known as consensus algorithm) is a protocol used to achieve a single data value (consensus) in a distributed process or system. The consensus algorithm is designed to achieve consistency in a network involving multiple unreliable nodes. Its role is to ensure the correctness, credibility and security of the distributed system by solving the problems of network latency, fault tolerance and credibility in the distributed system.

[0052] For example, when the data analysis terminal needs to conduct joint analysis of data, it sends a federated learning task to each task participating terminal and the coordinated analysis server and determines a training task, which includes the target application participating in the federated learning task, the corresponding data requirements, etc. After receiving the federated learning task, each task participating terminal jointly builds a joint blockchain and selects a matching coordinated analysis server through a consensus mechanism.

[0053] It should be noted that the consensus mechanism method can be a proof-of-work algorithm, a proof-of-stake algorithm, a practical Byzantine fault-tolerance algorithm, an authority proof algorithm, and other consensus algorithms, etc., and the embodiments of the present disclosure do not impose any special restrictions on this.

[0054] In a distributed system, since there is no consensus mechanism between distributed nodes, it is easy to be attacked by malicious nodes, resulting in system crash or data tampering. Therefore, the consensus mechanism between each task participating terminal and the joint analysis server can further ensure the immutability and non-forgeability of the data, and further improve the data security in the process of joint data analysis.

[0055] In the data interaction process between the above-mentioned task participating terminals and the coordination and analysis server, the privacy and security of the data can be ensured from many aspects. The following will be illustrated in conjunction with specific embodiments:

[0056] Example 1: In order to ensure terminal security, in an optional embodiment of the present disclosure, a security application is installed in each task participating terminal.

[0057] Among them, security applications can be, for example, firewalls, anti-virus software, etc. Figure 1 As shown, each task participating terminal is installed with a firewall to ensure isolation and security protection between data.

[0058] Example 2: To ensure that each task participating terminal can achieve data unification without exposing the original shared data, before executing step 201 to obtain the shared data containing the common user entity published by each task participating terminal, in an optional embodiment of the present disclosure, the user entities in each task participating terminal may be aligned to obtain the common user entity.

[0059] Since the same object is expressed differently in different systems, in order to unify the meaning of data, the user entities in each task participating terminal may be physically aligned through this embodiment to obtain unified data.

[0060] Example 3: Encryption is performed during data transmission.

[0061] In some example embodiments of the present disclosure, an encryption policy is sent to each task participating terminal so that each task participating terminal encrypts the shared data corresponding to the common user entity based on the encryption policy and stores the encrypted shared data in the joint blockchain.

[0062] Among them, entity alignment: also known as entity matching or entity resolution, is the process of determining whether two entities in the same or different data sets point to the same object in the real world. For example, data set 1 contains a "terminal" entity, and data set 2 contains a "terminal device" entity, which point to the same object. Therefore, through entity alignment, the "terminal" and "terminal device" entities in data sets 1 and 2 can be merged, thereby associating different data sets.

[0063] For example, since the user groups corresponding to the task participating terminals belong to different systems, in order to ensure that the task participating terminals can align common users without exposing their respective corresponding original data, the user entities in the task participating terminals can be entity aligned.

[0064] In the above-mentioned process of entity alignment of user entities, each task participating terminal will not expose the users belonging to any party, thereby ensuring the privacy and security of railway data.

[0065] In order to enhance the security and privacy of data, when the step of sending an encryption policy to each task participating terminal is executed in the above embodiment so that each task participating terminal encrypts the shared data corresponding to the common user entity based on the encryption policy, one or more encryption policies need to be constructed in advance in the coordination analysis server for selection, thereby ensuring the privacy and security of data. The encryption policy will be exemplarily described in conjunction with a specific embodiment.

[0066] 1) Encryption strategy based on key pair:

[0067] In an optional embodiment of the present disclosure, a key pair is created, the key pair including a private key and a public key; the public key in the key pair is sent to each task participating terminal, so that each task participating terminal encrypts the shared data corresponding to the common user entity based on the public key of the key pair, so that the joint analysis server can decrypt the encrypted shared data based on the private key after obtaining the encrypted shared data.

[0068] Among them, in asymmetric encryption technology, there are two types of keys: private key (referred to as private key) and public key (referred to as public key), and the private key is held by the owner of the key pair and cannot be published, while the public key is published to others by the holder of the key pair and is used to encrypt shared data, and the shared data encrypted by the public key can only be decrypted using the private key. Conversely, by using the private key to encrypt the shared data, the coordination analysis server must use the public key to decrypt it after receiving the encrypted data.

[0069] Exemplarily, the coordination analysis server can create a key pair and send the public key in the key pair to each task participating terminal, so that each task participating terminal can encrypt the shared data corresponding to its common user entity and store it in the constructed joint blockchain. After the coordination analysis server obtains the corresponding shared data through the joint blockchain, it decrypts it according to the private key in the key pair, so as to calculate the gradient information through the decrypted shared data.

[0070] In this embodiment, the security and privacy of data transmission can be improved by encrypting the shared data for transmission.

[0071] 2) Encryption strategy based on digital signature and digital certificate:

[0072] Digital signature is a way to verify one's identity through identifiable digital information. A set of digital signatures usually defines two complementary operations, one for signing and one for verification. The sender holds a private key that can represent his or her identity (the private key cannot be disclosed), and the receiver holds the public key corresponding to the private key, which can verify the sender's identity when receiving information sent by the sender.

[0073] If a third-party attacker wants to deceive the recipient, he can tamper with the sender's public key stored by the recipient and change it to the third-party attacker's public key. At this time, the third-party attacker can impersonate the sender and send messages to the recipient. In this case, if the recipient wants to verify whether the public key he has stored belongs to the real sender, he can use a digital certificate for identity authentication.

[0074] For example, a digital certificate is issued by an authority, the CA Certificate Authority, and is used to identify the identity of the other party on the Internet.

[0075] In this embodiment, encryption technology based on digital certificates can encrypt and decrypt information transmitted on the network, digitally sign and verify signatures, ensure the confidentiality and integrity of information transmitted online, the authenticity of the identity of the transaction entity, and the non-repudiation of the signature information.

[0076] It should be understood that in addition to the encryption strategies shown in the above embodiments, other encryption strategies may also be used, and the embodiments of the present disclosure do not impose any special restrictions on this.

[0077] Example 3: Multi-factor authentication and smart contracts.

[0078] Exemplarily, when executing step 201, in an optional embodiment of the present disclosure, the user identity corresponding to the coordinating analysis party or the data analysis terminal is authenticated according to the smart contract in the joint blockchain, and the smart contract is constructed according to the access parameters for the shared data. The smart contract contains an access control list for the shared data, and the access parameters include one or more of the number of times the data is used and the data period; if the user identity corresponding to the coordinating analysis party or the data analysis terminal is successfully authenticated, the shared data corresponding to the shared user entity is obtained from the joint blockchain; or, if the user identity corresponding to the coordinating analysis party or the data analysis terminal fails to be authenticated, the data analysis terminal is denied to obtain the shared data corresponding to the shared user entity from the joint blockchain.

[0079] For example, Figure 3As shown in the figure, in the process of shared data release: the participating terminals in the task of ensuring terminal security must first undergo multi-factor authentication. Only after the authentication is passed can the metadata of the data to be shared be registered in the joint blockchain, and the circulation and use of the provided data be authorized, such as confirming the algorithm logic, number of uses and period of the data, in order to build a corresponding smart contract.

[0080] That is, identity authentication can be achieved through the constructed smart contract, permission control logic can be written in the smart contract, and an access control list or whitelist mechanism for shared data can be implemented to ensure that only authenticated users or contracts can access specific shared data or perform specific operations.

[0081] The multi-factor authentication described above means that users need to pass two or more authentication mechanisms before they can be authorized to use relevant data resources, such as the combined verification of the public key / private key and digital certificate in the above embodiment. The smart contract contains an access control list or whitelist mechanism for shared data. When the coordination analysis server and / or the data analysis terminal needs to obtain shared data corresponding to the shared user entity from the joint blockchain, it can be checked from the access control list or whitelist mechanism whether the user is included. If included, it means that the user has the authority to obtain the corresponding data, otherwise it does not.

[0082] In step 202, the target gradient information is determined according to the shared data corresponding to the common user entity, and the target gradient information is stored in the joint blockchain, so that each task participating terminal can update the model parameters of the corresponding initial joint analysis model according to the target gradient information, and obtain the joint analysis result based on the updated target joint analysis model.

[0083] Exemplarily, the coordination analysis server can determine the target gradient information based on the shared data corresponding to the common user entity obtained from the joint blockchain

[0084] In this embodiment, the security of blockchain technology is utilized, and the nodes in the federated learning network are connected to the blockchain to achieve distributed storage and transmission of model update parameters, reduce the computational burden of the coordination service party, and improve the scalability and robustness of the system.

[0085] In an optional embodiment of the present disclosure, the target gradient information and loss value are determined based on the shared data corresponding to the common user entity, and an additional mask is added to the target gradient information and loss value so that each task participating terminal can remove the additional mask of the target gradient information and loss value, and the model parameters of the initial joint analysis model are updated by removing the additional mask of the target gradient information and loss value.

[0086] Exemplarily, the coordination analysis server can determine the target gradient information and loss value based on the shared data corresponding to the common user entity, and add an additional mask when calculating the above target gradient information and loss value to enhance data security. The task participating terminal removes the mask on the gradient information and updates the model parameters based on these target gradient information and loss values. At the same time, the updated model parameters are published to the alliance blockchain to realize model updates on each terminal.

[0087] Furthermore, in an optional embodiment of the present disclosure, audit logs of different time periods are packaged into multiple connected data blocks according to time data, and the block header of the current data block in the multiple connected data blocks contains at least the hash value of the previous adjacent data block and the timestamp of the data block generation time; the historical records of data connection analysis are determined based on the multiple connected data blocks.

[0088] Exemplarily, the security audit log is packaged and organized into data blocks after node consensus. The block header of each new data block contains the encrypted hash value of the previous data block and a timestamp that records the time information of the data block generation. The hash value and timestamp can realize the tracking and verification of data blocks, ensure the integrity of audit data, and record the complete transaction history and records through blockchain technology, so that all activities in the federated learning process can be audited and traced.

[0089] It should be understood that, although the various steps in the flow chart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the figure may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0090] To implement the above data joint analysis method, please refer to Figure 4 An embodiment of the present application provides a data joint analysis device, and the data joint analysis device 400 may include: a data acquisition module 401 and an information determination module 402.

[0091] Among them, the data acquisition module 401 is used to respond to the joint analysis task sent by the receiving data analysis terminal, and obtain the shared data containing the common user entity published by each task participating terminal from the joint blockchain jointly constructed by each task participating terminal, and the joint analysis task at least includes an initial joint analysis model; the information determination module 402 is used to determine the target gradient information according to the shared data corresponding to the common user entity, and store the target gradient information in the joint blockchain, so that each task participating terminal can update the model parameters of the corresponding initial joint analysis model according to the target gradient information, and obtain the joint analysis result based on the updated target joint analysis model.

[0092] In an optional embodiment of the present disclosure, the device further includes a policy sending module and an entity alignment module, and the entity alignment module is used to perform entity alignment on user entities in each task participating terminal to obtain a common user entity.

[0093] In an optional embodiment of the present disclosure, the device further includes a policy sending module, wherein the policy sending module is used to send an encryption policy to each task participating terminal, so that each task participating terminal encrypts the shared data corresponding to the common user entity based on the encryption policy, and stores the encrypted shared data in the joint blockchain.

[0094] In an optional embodiment of the present disclosure, the policy sending module is used to create a key pair, which includes a private key and a public key; the public key in the key pair is sent to each task participating terminal, so that each task participating terminal encrypts the shared data corresponding to the common user entity based on the public key of the key pair, so that the joint analysis server can decrypt the encrypted shared data based on the private key after obtaining the encrypted shared data.

[0095] In an optional embodiment of the present disclosure, each task participating terminal and the joint analysis server comply with a consensus mechanism.

[0096] In an optional embodiment of the present disclosure, a data acquisition module 401 is used to authenticate the user identity corresponding to the coordinating analysis party or the data analysis terminal according to a smart contract in a joint blockchain. The smart contract is constructed based on access parameters for shared data. The smart contract includes an access control list for shared data. The access parameters include one or more of the number of times the data is used and the data period. If the user identity corresponding to the coordinating analysis party or the data analysis terminal is successfully authenticated, the shared data corresponding to the shared user entity is acquired from the joint blockchain. Alternatively, if the user identity corresponding to the coordinating analysis party or the data analysis terminal fails to be authenticated, the data analysis terminal is denied from acquiring the shared data corresponding to the shared user entity from the joint blockchain.

[0097] In an optional embodiment of the present disclosure, the information determination module 402 is used to determine the target gradient information and loss value based on the shared data corresponding to the common user entity, and add an additional mask to the target gradient information and loss value so that each task participating terminal can remove the additional mask of the target gradient information and loss value, and update the model parameters of the initial joint analysis model by removing the additional mask of the target gradient information and loss value.

[0098] In an optional embodiment of the present disclosure, the device may also include a data packaging module, which is used to package audit logs of different time periods into multiple connected data blocks according to time data, and the block header of the current data block in the multiple connected data blocks at least includes the hash value of the previous adjacent data block and the timestamp of the data block generation time; an information determination module 402 is used to determine the historical records of data connection analysis based on the multiple connected data blocks.

[0099] For the specific definition of the above-mentioned data joint analysis device, please refer to the definition of the data joint analysis method above, which will not be repeated here. Each module in the above-mentioned data joint analysis device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0100] In one embodiment, a computer device is provided, the internal structure diagram of which can be as follows: Figure 5 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a data joint analysis method as described above is implemented. It includes: a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, any step in the data joint analysis method as described above is implemented.

[0101] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, any step in the above data joint analysis method can be implemented.

[0102] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0103] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0104] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0105] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0106] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0107] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A data joint analysis method, characterized in that: Applied to the joint analysis server, the method includes: In response to receiving a joint analysis task sent by a data analysis terminal, obtaining shared data including a common user entity published by each task participating terminal from a joint blockchain jointly constructed by each task participating terminal, wherein the joint analysis task includes at least an initial joint analysis model; The target gradient information is determined according to the shared data corresponding to the common user entity, and the target gradient information is stored in the joint blockchain, so that each task participating terminal can update the model parameters of the corresponding initial joint analysis model according to the target gradient information, and obtain the joint analysis result based on the updated target joint analysis model.

2. The method according to claim 1, characterized in that: Before acquiring the shared data corresponding to the common user entity based on each task participating terminal, the method further includes: The user entities in each task participating terminal are entity aligned to obtain the common user entity.

3. The method according to claim 2, characterized in that The method further comprises: An encryption policy is sent to each task participating terminal, so that each task participating terminal encrypts the shared data corresponding to the common user entity based on the encryption policy, and stores the encrypted shared data in the joint blockchain.

4. The method according to claim 3, characterized in that The sending of the encryption policy to each task participating terminal so that each task participating terminal encrypts the shared data corresponding to the common user entity based on the encryption policy includes: Creating a key pair, wherein the key pair includes a private key and a public key; The public key in the key pair is sent to each task participating terminal, so that each task participating terminal encrypts the shared data corresponding to the common user entity based on the public key of the key pair, so that the joint analysis server can decrypt the encrypted shared data based on the private key after obtaining the encrypted shared data.

5. The method according to claim 1, characterized in that The task participating terminals and the joint analysis server comply with the consensus mechanism.

6. The method according to claim 1, characterized in that The obtaining of shared data including common user entities published by each task participating terminal from the joint blockchain jointly constructed by each task participating terminal includes: Authenticating the user identity of the coordinating analysis party or the data analysis terminal according to the smart contract in the joint blockchain, wherein the smart contract is constructed according to the access parameters for the shared data, and the smart contract includes an access control list for the shared data, and the access parameters include one or more of the number of times the data is used and the data period; If the user identity authentication corresponding to the coordinating analysis party or the data analysis terminal is successful, the shared data corresponding to the shared user entity is obtained from the joint blockchain; Alternatively, if the user identity authentication corresponding to the coordinating analysis party or the data analysis terminal fails, the data analysis terminal is denied access to the shared data corresponding to the shared user entity from the joint blockchain.

7. The method according to claim 1, characterized in that The method further comprises: The target gradient information and loss value are determined according to the shared data of the common user entity, and an additional mask is added to the target gradient information and loss value so that each task participating terminal can remove the additional mask of the target gradient information and loss value, and the model parameters of the initial joint analysis model are updated by the target gradient information and loss value after removing the additional mask.

8. The method according to claim 1, characterized in that The method further comprises: Packing audit logs of different time periods into multiple connected data blocks according to time data, wherein the block header of the current data block in the multiple connected data blocks at least includes a hash value of the previous adjacent data block and a timestamp of the generation time of each of the data blocks; A history of data connection analysis is determined based on multiple connected data blocks.

9. A data joint analysis device, characterized in that: Applied to the joint analysis server, the device comprises: A data acquisition module, configured to, in response to receiving a joint analysis task sent by a data analysis terminal, acquire shared data containing common user entities based on and published by each task participating terminal from a joint blockchain jointly constructed by each task participating terminal, wherein the joint analysis task at least includes an initial joint analysis model; An information determination module is used to determine the target gradient information according to the shared data corresponding to the common user entity, and store the target gradient information in the joint blockchain, so that each task participating terminal can update the model parameters of the corresponding initial joint analysis model according to the target gradient information, and obtain the joint analysis result based on the updated target joint analysis model.

10. A computer device comprising: It comprises a memory and a processor, wherein the memory stores a computer program, and is characterized in that when the processor executes the computer program, the steps of the data joint analysis method according to any one of claims 1 to 8 are implemented.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the data joint analysis method according to any one of claims 1 to 8 are implemented.

Images