Model generation method, device and product based on zero-knowledge proof and federated learning

Abstract

The embodiment of the application is suitable for the technical field of data processing, and provides a model generation method, device and product based on zero-knowledge proof and federated learning. The method comprises the following steps: obtaining first gradients and proof parameters sent by each participant; the first gradient comprises a gradient generated when the participant performs model training based on local data and a federated learning architecture; the proof parameter comprises a parameter used for verifying that the local data meets a preset constraint condition; the federated learning architecture comprises initial model parameters used for model training; a plurality of second gradients are determined; the second gradient is a first gradient corresponding to an effective parameter in the proof parameter; and the initial model parameters in a global model are updated based on the plurality of second gradients, so that an updated global model is obtained. By using the above method, the model training accuracy can be improved on the basis of ensuring data privacy.

Description

Technical Field

[0001] The present application belongs to the field of data processing technology, and in particular relates to a model generation method, device and product based on zero-knowledge proof and federated learning. Background Art

[0002] In the field of gas industry regulation, regulators often need to build analytical models that can accurately reflect the safety situation of the entire industry in real time. This is to achieve early warning of gas accident risks and optimize emergency response strategies. Typically, training such analytical models relies on massive amounts of historical data accumulated by gas operators, including private data such as the temporal and spatial distribution of safety incidents, equipment failure types, and the effectiveness of remediation measures.

[0003] However, the above-mentioned privacy data not only concerns the company's own competitive advantage (for example, commercial secrets such as equipment fault handling capabilities and risk prevention and control strategies), but may also cause public panic about gas safety due to leakage, leading to a crisis of social trust.

[0004] Therefore, data privacy and trade secret protection requirements make it difficult for companies to directly share raw data. Traditionally, only a small amount of data uploaded by companies can be used for training, resulting in low model training accuracy and difficulty meeting the regulatory authorities' requirements for refined risk management. For example, it is difficult to generate highly accurate risk assessment reports. Summary of the Invention

[0005] The embodiments of the present application provide a model generation method, device and product based on zero-knowledge proof and federated learning, which can solve the problems of low accuracy of models trained under traditional modes and the risk of data leakage.

[0006] In a first aspect, an embodiment of the present application provides a model generation method based on zero-knowledge proof and federated learning, which is applied to a node device of a blockchain, and the method includes:

[0007] Obtaining the first gradient and proof parameters sent by each participant; the first gradient includes the gradient generated by the participant when training the model based on local data and the federated learning architecture, and the proof parameters include parameters used to verify that the local data meets the preset constraints; the federated learning architecture includes the initial model parameters used for model training;

[0008] Determine a plurality of second gradients; the second gradients are first gradients corresponding to valid parameters in the certification parameters;

[0009] Initial model parameters in the global model are updated based on the multiple second gradients to obtain an updated global model.

[0010] In one embodiment, updating initial model parameters in the global model based on the plurality of second gradients to obtain an updated global model includes:

[0011] aggregating the plurality of second gradients to obtain an aggregated gradient;

[0012] Calculate the product of the aggregate gradient and the preset learning rate;

[0013] The difference between the initial model parameters and the product is determined as the target model parameters;

[0014] The target model parameters are used to replace the initial model parameters to obtain the updated global model.

[0015] In one embodiment, aggregating a plurality of second gradients to obtain an aggregated gradient comprises:

[0016] Determine the historical contribution of each participant. The historical contribution is used to quantify the effect of the first gradient sent by the participant at the historical moment on the improvement of the global model.

[0017] Determine the aggregation weight corresponding to each second gradient based on the historical contribution; the aggregation weight of a high historical contribution is greater than or equal to the aggregation weight of a low historical contribution;

[0018] Each second gradient is weighted and summed with the corresponding aggregation weight to obtain the aggregate gradient.

[0019] In one embodiment, the method further includes:

[0020] Count the total number of signatures obtained; the signature is proof that the participant requested to obtain the aggregate gradient;

[0021] If the total quantity is greater than or equal to the preset quantity, the aggregate gradient is published.

[0022] In a second aspect, an embodiment of the present application provides a model generation device based on zero-knowledge proof and federated learning, which is applied to a node device of a blockchain, and the device includes:

[0023] An acquisition module, configured to acquire first gradients and proof parameters sent by each participant; the first gradients include gradients generated by the participant when training the model based on local data and the federated learning architecture; the proof parameters include parameters used to verify that the local data meets preset constraints; the federated learning architecture includes initial model parameters used for model training;

[0024] A determination module, configured to determine a plurality of second gradients, wherein the second gradients are first gradients corresponding to valid parameters in the certification parameters;

[0025] An updating module is used to update initial model parameters in the global model based on the multiple second gradients to obtain an updated global model.

[0026] In a third aspect, an embodiment of the present application provides another model generation method based on zero-knowledge proof and federated learning, which is applied to electronic devices. The method includes:

[0027] Perform model training based on local data and a federated learning architecture to obtain the first gradient generated during the training process; the federated learning architecture includes initial model parameters used for model training;

[0028] Generate proof parameters for verifying that local data complies with pre-set constraints;

[0029] The first gradient and proof parameters are sent to a node device of the blockchain; the node device is used to determine multiple second gradients and update the initial model parameters of the global model based on the multiple second gradients to obtain an updated global model; the second gradient is the first gradient corresponding to the valid parameters in the proof parameters.

[0030] In one embodiment, model training is performed based on local data and a federated learning architecture to obtain a first gradient generated during the training process, including:

[0031] Obtain macro indicators and federated learning architecture sent by node devices;

[0032] Obtain training data corresponding to macro indicators from local data;

[0033] The model is trained based on the training data and federated learning architecture to obtain the first gradient.

[0034] In one embodiment, model training is performed based on local data and a federated learning architecture to obtain a first gradient generated during the training process, including:

[0035] When performing model training on initial model parameters in the model based on local data, obtaining a third gradient generated by the model during the training process; the third gradient includes at least one;

[0036] Perform noise processing on each third gradient respectively to obtain the corresponding noise gradient;

[0037] Aggregate each noise gradient to generate a first gradient.

[0038] In one embodiment, noise processing is performed on each third gradient to obtain a corresponding noise gradient, including:

[0039] For any third gradient, determining a sensitivity category of the corresponding local data used when generating the third gradient;

[0040] Determine the noise corresponding to the third gradient based on the sensitivity category; the noise corresponding to the high sensitivity category is greater than the noise corresponding to the low sensitivity category;

[0041] The third gradient is processed based on the noise to obtain a noisy gradient.

[0042] In a fourth aspect, an embodiment of the present application provides another model generation device based on zero-knowledge proof and federated learning, which is applied to an electronic device, and includes:

[0043] A training module is used to train the model based on local data and a federated learning architecture, and obtain the first gradient generated during the training process; the federated learning architecture includes initial model parameters used for model training;

[0044] A generation module, used to generate proof parameters for verifying that local data meets preset constraints;

[0045] A sending module is configured to send a first gradient and proof parameters to a node device of the blockchain; the node device is configured to determine multiple second gradients and update the initial model parameters of the global model based on the multiple second gradients to obtain an updated global model; the second gradient is the first gradient corresponding to the valid parameters in the proof parameters.

[0046] In a fifth aspect, an embodiment of the present application provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method of the first or third aspect described above is implemented.

[0047] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the method of the first or third aspect mentioned above.

[0048] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on a computer device, enables the computer device to execute the method of the first or third aspect above.

[0049] Compared to the prior art, the embodiments of the present application offer the following advantages: after obtaining the first gradient and proof parameters generated by a participant based on local data and a federated learning architecture, a second gradient can be determined from multiple first gradients based on the validity of the proof parameters. Specifically, the second gradient can be deemed "trusted," as the model update information it carries is obtained through training based on local data that meets preset constraints, rather than training based on false local data. This improves the accuracy of subsequent global model updates based on the second gradient. Furthermore, the first gradient is generated and uploaded by the participant when training the initial model parameters in the federated learning architecture based on local data. Therefore, it can be assumed that the participant only uploads the gradient information, not the original local data. Furthermore, this ensures that sensitive local data remains locally, avoiding the risk of commercial secrets being leaked or public panic caused by direct data sharing. Finally, the initial model parameters in the global model are updated based on the multiple second gradients, resulting in an updated global model. Therefore, since the multiple second gradients are generated by each participant based on their own local data training, they essentially incorporate the characteristics of a large amount of data. Furthermore, by updating the initial model parameters based on multiple second gradients, the global model can capture deep patterns that cannot be covered by the data of a single participant, thereby improving the training accuracy of the updated global model. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0051] Figure 1 This is a flowchart of an implementation of a model generation method based on zero-knowledge proof and federated learning provided in one embodiment of the present application;

[0052] Figure 2 This is a schematic diagram of an implementation method for generating a first gradient in a model generation method based on zero-knowledge proof and federated learning provided in one embodiment of the present application;

[0053] Figure 3 This is a schematic diagram of an implementation method for generating a first gradient in a model generation method based on zero-knowledge proof and federated learning provided in another embodiment of the present application;

[0054] Figure 4 This is a schematic diagram of an implementation method for generating noise gradients in a model generation method based on zero-knowledge proof and federated learning provided in one embodiment of the present application;

[0055] Figure 5This is a schematic diagram of an implementation method for updating a global model in a model generation method based on zero-knowledge proof and federated learning provided in one embodiment of the present application;

[0056] Figure 6 This is a schematic diagram of an implementation method for generating an aggregated gradient in a model generation method based on zero-knowledge proof and federated learning provided in one embodiment of the present application;

[0057] Figure 7 This is a flowchart of an implementation of a model generation method based on zero-knowledge proof and federated learning provided in another embodiment of the present application;

[0058] Figure 8 This is a schematic diagram of an application scenario in a model generation method based on zero-knowledge proof and federated learning provided in another embodiment of the present application;

[0059] Figure 9 This is a schematic diagram of the structure of a model generation device based on zero-knowledge proof and federated learning provided in one embodiment of the present application;

[0060] Figure 10 This is a structural diagram of a model generation device based on zero-knowledge proof and federated learning provided by another embodiment of the present application;

[0061] Figure 11 It is a structural diagram of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0062] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0063] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.

[0064] It should be noted that the information collection process (such as the facial image collection process, fingerprint information collection process, etc.) / feature extraction process involved in this application is performed with the user's knowledge and permission, that is, the information collection process / feature extraction process complies with the requirements of laws and regulations and does not constitute an act that harms the public interest.

[0065] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0066] In the field of gas industry regulation, regulators often need to build analytical models that can accurately reflect the safety situation of the entire industry in real time. This is to achieve early warning of gas accident risks and optimize emergency response strategies. Typically, training such analytical models relies on massive amounts of historical data accumulated by gas operators, including private data such as the temporal and spatial distribution of safety incidents, equipment failure types, and the effectiveness of remediation measures.

[0067] However, the above-mentioned privacy data not only concerns the company's own competitive advantage (for example, commercial secrets such as equipment fault handling capabilities and risk prevention and control strategies), but may also cause public panic about gas safety due to leakage, leading to a crisis of social trust.

[0068] Therefore, data privacy and trade secret protection requirements make it difficult for companies to directly share raw data. Traditionally, only a small amount of data uploaded by companies can be used for training, resulting in low model training accuracy and difficulty meeting the regulatory authorities' demand for refined risk management.

[0069] Based on this, in order to improve the training accuracy of the model, an embodiment of the present application provides a model generation method based on zero-knowledge proof and federated learning, which can be applied to node devices of the blockchain.

[0070] Blockchain is a distributed ledger technology that packages data into "blocks" in chronological order and links them into a chain structure using cryptography to achieve tamper-proof, traceable, and decentralized data storage. Its core features include:

[0071] Decentralization: There is no centralized management organization, and data is distributed across multiple node devices in the network to avoid single point of failure.

[0072] Unalterable: Each block contains the hash value of the previous block. Once the data is written, it is difficult to tamper with, ensuring integrity.

[0073] Transparency: Network participants can view on-chain data (depending on permissions), enhancing trust.

[0074] Security: Network security is ensured through encryption algorithms (such as SHA-256) and consensus mechanisms (such as PoW and PoS).

[0075] In federated learning scenarios, blockchain is primarily used for: Storing and verifying gradient data: Node devices receive gradients (e.g., first and second gradients) and proof parameters sent by each participant, verifying data validity (e.g., whether local data complies with privacy constraints). Ensuring trusted data flow: Smart contracts automatically enforce rules (e.g., aggregating only valid gradients) to prevent malicious data from contaminating the global model. Traceability and auditability: All operations are recorded on-chain, facilitating subsequent verification of participant behavior (e.g., who sent gradient data and when).

[0076] Node devices are participating entities in the blockchain network and can be hardware devices such as servers, computers, and mobile phones. Each node device must run the blockchain client software to participate in network communication, data verification, and maintain the blockchain ledger.

[0077] See also Figure 1 , Figure 1 The following is a flowchart of a model generation method based on zero-knowledge proof and federated learning provided in an embodiment of the present application. The method includes the following steps:

[0078] S101. Obtain the first gradient and proof parameters sent by each participant.

[0079] In one embodiment, the first gradient includes a gradient generated by a participant when performing model training based on local data and a federated learning architecture, and the federated learning architecture includes initial model parameters for model training.

[0080] The aforementioned participants refer to independent entities participating in the federated learning task (e.g., electronic devices such as mobile phones and tablets). Typically, each participant possesses their own local data (e.g., various types of data such as accident location, time, device model, and treatment measures). This local data can be, without limitation, gas or medical data. In this example, gas data is used as an example for the following explanation.

[0081] It should be noted that private data not only affects a company's competitive advantage but can also trigger public panic about gas safety and lead to a crisis of social trust if leaked. Therefore, collaborative training of the global model is necessary while protecting data privacy.

[0082] The first gradient is the direction and magnitude of the model parameter updates calculated by the participants when training the initial model based on local data and the federated learning architecture. The gradient is the core signal for model optimization, indicating the direction in which model parameter adjustments should minimize the loss function. In gas equipment failure prediction tasks, the first gradient may include the parameter updates for gas equipment operating parameters to improve the ability to identify safety features such as leaks and pressure anomalies.

[0083] The aforementioned federated learning architecture is a distributed machine learning framework that allows participants to collaboratively train a global model without sharing raw data. Typically, a federated learning architecture can include various definitions, such as initial model parameters and communication protocol security mechanisms.

[0084] Exemplarily, the initial model parameters are the initial state of the global model (e.g., randomly initialized neural network weights), which can be randomly set by the node device or actively set by the regulator, and then published to each participant.

[0085] Furthermore, the communication protocol can define information such as gradient transmission and model aggregation rules between participants and node devices.

[0086] Furthermore, security mechanisms may include information such as data encryption, privacy protection algorithms (e.g., differential privacy, secure multi-party computation), etc.

[0087] As an example, the electronic devices corresponding to the participants can be configured as follows: Figure 2 The steps S201-S203 shown generate the first gradient, which is described in detail as follows:

[0088] S201. Obtain macro indicators and federated learning architecture sent by node devices.

[0089] S202: Obtain training data corresponding to the macro indicators from local data.

[0090] In one embodiment, the node devices have been explained above and will not be further explained. Macro indicators refer to statistical characteristic values ​​extracted from local data such as gas equipment operation, user behavior, and security incidents, and are used to summarize the overall security situation, equipment operating efficiency, or user behavior patterns of an industry or region.

[0091] In one embodiment, macro indicators can be generated by aggregating statistics of raw data (such as counts, means, ratios, etc.), retaining only global features without including individual data details, and achieving "data available but invisible" in federated learning.

[0092] It should be noted that the corresponding to macro indicators are detail indicators, which are used to describe the specific details of the data.

[0093] As an example, macro indicators could be safety status indicators, reflecting the gas system's safety risk level and accident distribution. Corresponding detailed indicators could include the number of gas accidents per unit area (e.g., per square kilometer); the proportion of high-risk areas (the percentage of geographical areas where the number of accidents exceeds a threshold); and the recurrence rate (the percentage of accidents that occur twice or more in the same equipment / area within a certain period).

[0094] It is understandable that when the electronic device obtains corresponding training data from local data based on macro indicators, the obtained training data may not need to include detailed data such as the specific number of accidents, accident locations, user information, etc.

[0095] As an example, macro indicators may include, but are not limited to, accident frequency indicators, illegal operation indicators, and accident severity indicators. Correspondingly, detailed indicators corresponding to the accident frequency indicator may include the number of gas leak accidents, gas explosion accidents, gas poisoning accidents, daily accidents, weekly accidents, and monthly accidents. Detailed indicators corresponding to the illegal operation indicator may include the illegal equipment, time of violation, location of violation, and type of violation. Furthermore, detailed indicators corresponding to the accident severity indicator may include casualties, property damage, and other detailed indicators.

[0096] For example, using the accident frequency metric as an example, electronic devices can simply count the total number of accidents sent in a given week as training data, rather than the number of accidents of each type in that week. Furthermore, during the training process, node devices do not need to be aware of the specific accident details, further protecting the privacy of local data.

[0097] It's understandable that the training data for macro indicators is generated through calculations (e.g., summation or averaging), eliminating individual data identifiers (e.g., the number of gas leaks). Even if the training data for a macro indicator were leaked, it would be impossible to infer the data corresponding to the detailed indicators within the macro indicator. For example, "there were three gas accidents in a certain area within a week" does not refer to any specific types of accidents or the number of each type of accident.

[0098] In one embodiment, the aforementioned macro indicators can be predefined by a regulatory body (e.g., the "Federated Learning Macro Indicator Specification for the Gas Industry") to clearly define each macro indicator. The electronic device can automatically traverse local data to obtain training data corresponding to each macro indicator.

[0099] In another embodiment, in order to enable the regulator (node ​​device) to verify the authenticity of local data (for example, training data corresponding to macro indicators) through a smart contract, the electronic device can generate proof parameters for verifying that the local data meets preset constraints.

[0100] The preset constraint condition can be set according to actual conditions and is not limited thereto. For example, the preset constraint condition can be that the number of accidents in the past three months is ≤ 5.

[0101] It should be noted that preset constraints are predefined rules to ensure data compliance. These rules can be used to align data distribution and reflect local data characteristics.

[0102] For example, a preset constraint could be that the proportion of training data corresponding to "leakage accidents" must be within the range of [20%, 40%]. This allows the distribution of training data to be consistent with the training objective of the global model (for example, the proportion of "leakage accidents" in the accident data of each participant must be within the range of [20%, 40%].

[0103] In one embodiment, the electronic device can generate proof parameters for statistical indicators based on two zero-knowledge proof technologies, zk-SNARKs and zk-STARKs. Alternatively, the proof parameters can be generated based on a homomorphic encryption algorithm. In this embodiment, the method for generating the proof parameters is not limited.

[0104] Taking zk-SNARKs as an example, generating proof parameters is a technology for efficiently proving fixed statistical metrics, applicable to statistical metrics within a fixed time window (for example, "the number of accidents in the past three months is ≤ 5"). Electronic devices can pre-design a circuit structure to verify "the number of accidents is ≤ 5," and the data used to generate the proof is fixed. The electronic device then generates public parameters (such as a proving key and a verification key) and, based on the circuit structure, generates proof parameters corresponding to the local data. Finally, the electronic device uploads the public and proof parameters to the node device. At this point, the node device only needs to verify the validity of the proof parameters based on the public parameters.

[0105] Furthermore, zk-STARKs technology is applicable to dynamically updated data (for example, "number of real-time accidents ≤ threshold"), eliminating the need for trusted setup and enabling incremental proof updates. The method for generating proof parameters in zk-STARKs is similar to that of zk-SNARKs and will not be further explained.

[0106] It should be noted that for incremental update proofs, when a new event occurs in the local data record, the electronic device can generate proof parameters corresponding to the incremental training data based on zk-SNARKs technology: there is no need to generate proof parameters corresponding to the entire training data. This can further reduce the computing resources of the electronic device.

[0107] S203: Perform model training based on the training data and the federated learning architecture to obtain a first gradient.

[0108] In one embodiment, when performing model training, the electronic device can be pre-set with a neural network model, and the initial model parameters in the neural network model can be the initial model parameters in the global model set in the node device to achieve federated learning among multiple electronic devices.

[0109] In one embodiment, the neural network model set above may be a lightweight neural network model, for example, a long short-term memory network, a gated recurrent network, or the like, without limitation thereto.

[0110] As an example, taking a long short-term memory (LSTM) network (LSTM) as an example, during model training, the input layer is used to input training data, while the hidden layer extracts spatial features and captures the temporal dependencies between features. The output layer can perform classification (e.g., normal or faulty) based on the extracted features. For example, the output may be the probability of a normal state and the probability of a faulty state. The LSTM network can then use a loss function to calculate the training loss. Finally, backpropagation is performed based on the training loss to calculate the first gradient. For example, the gradients of the weights and bias parameters of each layer are calculated. The generated first gradient is used to iteratively update the model parameters of the current LSTM network.

[0111] The above content is only an example of obtaining the first gradient. In this embodiment, the process of obtaining the first gradient through model training is not limited.

[0112] It's understandable that by acquiring macro indicators sent by node devices and using a federated learning architecture, it's possible to extract corresponding training data from local data for model training, making training more targeted and helping to improve model training efficiency and convergence speed. Furthermore, relying on a federated learning architecture for model training ensures that data is trained locally, preventing the flow of raw local data across nodes and enhancing the security of private data.

[0113] It's worth noting that, in addition to the basic process described above, the following optimization scenarios can also be considered in practical applications. For example, when training a model based on training data, the training data can be desensitized to strike a balance between data utilization and privacy. By stripping sensitive information, model training requirements are met while complying with regulatory requirements and mitigating the risk of data misuse.

[0114] Exemplarily, the electronic device may generate desensitized features of the training data through desensitization technology, and delete the training data that meets the prohibited features.

[0115] Among them, desensitized features refer to features that are retained after processing and can be used for model training, but sensitive information has been removed or blurred, and only key dimensions at the business level are retained.

[0116] For example, the desensitized features may be features such as the accident type (eg, “pipeline leakage_Level 2”), the response time interval (eg, “1-2 hours”), and the like.

[0117] Specifically, specific accident types are abstracted into standardized codes, with "pipeline leak" representing the accident category and "Level 2" representing the severity level. The corresponding training data retains the business attributes of the accident type, making it easier for the model to learn patterns across accident types and levels while preventing the disclosure of sensitive information such as the accident location and responsible party.

[0118] Furthermore, dividing the actual response time into intervals rather than specific minutes can retain key indicators of response efficiency for use in model optimization of emergency response strategies while avoiding exposing time-sensitive details of specific events.

[0119] Furthermore, prohibited features may include specific coordinates, device serial numbers, operator IDs, and other features, thereby preventing private data from being leaked.

[0120] In another embodiment, it should be noted that in the deep learning model training process, multiple iterations of optimization are usually required to achieve model convergence. Since the model calculates and generates a gradient based on the current batch of training data at each iteration, if the gradient generated by each iteration is directly uploaded to the node device as the first gradient, it will cause network bandwidth consumption, affecting transmission efficiency and training costs. In addition, unprocessed gradients usually contain a large amount of model training information. If the gradient is uploaded directly, there may be a risk of privacy leakage. An attacker may use the gradient to infer the training data and obtain sensitive data.

[0121] Based on this, in order to reduce the network resources consumed in uploading the first gradient and ensure the security of private data, the electronic device can be configured as follows: Figure 3 The steps S301-S303 shown generate the first gradient, which is described in detail as follows:

[0122] S301. When performing model training on initial model parameters in a model based on local data, obtain a third gradient generated by the model during the training process.

[0123] In one embodiment, the local data has been explained above and will not be further described. It should be noted that when executing step S301, the local data can first be processed based on steps S201-S202 to obtain training data. Subsequently, the initial model parameters in the model are trained based on the training data, and the third gradient generated by the model during the training process is obtained.

[0124] The model may be the lightweight neural network model described above, which will not be further described.

[0125] Furthermore, the third gradient includes at least one. Each iteration described in step S203 generates a gradient. Therefore, when only one iteration is performed, one third gradient is obtained. Furthermore, when multiple iterations are performed, multiple third gradients are obtained. The third gradient can be considered to be the original, unprocessed density.

[0126] S302 : Perform noise processing on each third gradient to obtain a corresponding noise gradient.

[0127] In one embodiment, the electronic device may perform noise processing on the third gradient based on Laplace noise, Gaussian noise, or other methods to obtain a noise gradient, which is not limited thereto.

[0128] As an example, the electronic device may Figure 4 In steps S401-S403, the third gradient is subjected to noise processing to obtain a noise gradient. Detailed description is as follows:

[0129] S401: For any third gradient, determine a sensitivity category of the local data used when generating the third gradient.

[0130] In one embodiment, during the model training process, due to the large scale of local data, each model training process is usually only performed using a portion of the local data. It is understandable that each local data usually corresponds to a sensitivity category. For example, high sensitivity, general sensitivity, low sensitivity and other categories. Among them, the local data corresponding to high sensitivity usually has a higher privacy, and the local data corresponding to low sensitivity usually has a lower privacy. In this embodiment, the types of sensitivity categories are not limited. Among them, each local data can be pre-set with its corresponding sensitivity category.

[0131] For example, during the first model training, low-sensitivity local data may be used for training to obtain a first third gradient, and during the second model training, high-sensitivity local data may be used for training to obtain a second third gradient.

[0132] S402: Determine noise corresponding to the third gradient based on the sensitivity category.

[0133] In one embodiment, the noise corresponding to the high sensitivity category is greater than the noise corresponding to the low sensitivity category. Exemplarily, the electronic device may be pre-set with preset noises corresponding to multiple sensitivity categories, so as to determine the noise corresponding to the third gradient based on the corresponding relationship.

[0134] Specifically, the electronic device may associate the local data used in each model training process with the corresponding generated third gradient. Further, after determining the sensitivity category corresponding to the local data, the corresponding noise of the local data may be determined based on the above correspondence.

[0135] In another embodiment, the electronic device may also pre-set an initial noise for each third gradient, and then increase or decrease the initial noise based on the sensitivity category to obtain the noise corresponding to the third gradient.

[0136] For example, when the local data corresponding to the third gradient is highly sensitive data, the initial noise can be added to a preset value to obtain the noise corresponding to the third gradient. Furthermore, when the local data corresponding to the third gradient is of medium sensitivity, the initial noise can be determined as the noise corresponding to the third gradient. Furthermore, when the local data corresponding to the third gradient is of low sensitivity, the initial noise can be subtracted from the preset value to obtain the noise corresponding to the third gradient. In this embodiment, the method for determining the noise is not limited.

[0137] As an example, the electronic device may determine the noise according to the following formula, which is detailed as follows:

[0138] ;

[0139] in, represents the noise corresponding to the third gradient, Indicates the preset gradient sensitivity, represents the sensitivity value corresponding to the sensitivity category. The higher the sensitivity category, the lower the corresponding sensitivity value. Consequently, the noise calculated using the above formula will be higher, which will increase the noise subsequently added to the third gradient and improve the privacy protection of local data.

[0140] S403: Process the third gradient based on the noise to obtain a noise gradient.

[0141] In one embodiment, the electronic device may perform a calculation (e.g., add or multiply) the noise with the third gradient to obtain the noise gradient; or multiply the noise with the corresponding weight and then add the product to the third gradient to obtain the noise gradient. In this embodiment, the method for obtaining the noise gradient is not limited.

[0142] It should be noted that in the data processing process, the sensitivity category of the local data corresponding to the generation of the third gradient is first determined, and then the noise is matched to the gradient based on the principle that the noise corresponding to the high-sensitivity category is greater than the noise corresponding to the low-sensitivity category. Furthermore, when the noise gradient is obtained by processing the third gradient based on the determined noise, differential noise can be added to focus on protecting highly sensitive data to prevent privacy leakage, and to moderately process low-sensitivity data to avoid excessive interference with the original data characteristics, thereby avoiding excessive disturbance of the data. From the perspective of compliance management, operations based on data sensitivity classification are consistent with the requirements of many industry specifications for differentiated protection measures for data of different levels, and follow the hierarchical management specifications. Ultimately, hierarchical and enhanced privacy protection is achieved, which not only protects data privacy but also ensures the effectiveness of model training, improves system security and the trust of participants.

[0143] S303: Aggregate each noise gradient to generate a first gradient.

[0144] In one embodiment, if an electronic device undergoes multiple iterations during model training, a large number of noise gradients will be generated. Directly uploading all of these noise gradients to a node device would consume significant network resources and reduce the security of uploading the first gradient. Instead, the electronic device can aggregate each noise gradient to generate a single first gradient and upload it.

[0145] For example, the electronic device can weight the sum of multiple noise gradients to form a first gradient for upload, thereby reducing the amount of data required for upload and conserving network resources. Furthermore, since the first gradient is aggregated based on noise gradients, generating the first gradient based on the noise gradient after adding noise can also reduce potential privacy risks.

[0146] It should be noted that since each noise gradient covers the weights and bias parameters of each layer in the model, during the aggregation process, the weights of the same layer and the bias parameters of the same layer need to be aggregated separately, and finally the first gradient consisting of the aggregated weights and bias parameters of each layer is obtained.

[0147] To summarize, the above description provides an example of each participant generating the first gradient and proof parameters locally. Based on this example, participants can participate in global model training without sharing local data, thereby protecting local data privacy while improving the training accuracy of the updated global model.

[0148] After generating the first gradient and the proof parameters, in order to further ensure data security, the electronic device may first encrypt the first gradient and the proof parameters, and then upload the encrypted first gradient and the proof parameters.

[0149] S102: Determine a plurality of second gradients.

[0150] In one embodiment, the second gradient is the first gradient corresponding to the valid parameter in the proof parameter. Methods for verifying that the proof parameter is a valid parameter include, but are not limited to, cryptographic proof matching, homomorphic encryption verification, and other methods, which are not limited thereto.

[0151] Exemplarily, if the electronic device generates proof parameters based on a zero-knowledge proof (ZKP) circuit, the node device can use the verification key (including circuit constraints and hash function parameters) to perform the following steps for verification.

[0152] Specifically, node devices can use the verification key to parse the structure of the proof parameters and check whether they conform to the pre-set circuit logic (such as arithmetic constraints and threshold constraints). They then use the hash function in the public parameters to verify whether the proof parameters are correctly "bound" to the local data calculation process, ensuring that there has been no forgery or tampering. Finally, through a random challenge (such as the challenge value c in ZKP) and response mechanism, they verify whether the proof parameters can correctly answer questions about the authenticity of the local data, ensuring that the data has not been maliciously constructed.

[0153] Alternatively, when generating proof parameters based on a homomorphic encryption algorithm, the public parameters may include an encrypted public key and homomorphic operation rules. In this case, the node device can use the public key in the public parameters to verify the cryptographic legitimacy of the proof parameters (e.g., signature verification). Finally, based on the homomorphic operation rules, the node device can check whether the computation results (e.g., gradient aggregation, statistical values) in the proof parameters conform to the expected mathematical transformations (e.g., additive homomorphism, multiplicative homomorphism), indirectly verifying the validity of the local data.

[0154] In this embodiment, there is no limitation on the method of verifying that the parameters are valid parameters.

[0155] It should be noted that the first gradient that proves the parameter is invalid can be considered to be generated based on "contaminated" local data during training and is therefore of no reference value. Therefore, subsequent processing can be performed only on the second gradient.

[0156] S103: Update initial model parameters in the global model based on the multiple second gradients to obtain an updated global model.

[0157] In one embodiment, the node device may aggregate multiple second gradients to obtain an aggregated gradient. The aggregated gradient is then used to update initial model parameters (e.g., by adding or subtracting them) to obtain updated model parameters. At this point, the global model containing the updated model parameters is referred to as the updated global model.

[0158] As an example, the electronic device may Figure 5The steps S501-S504 shown update the global model. The details are as follows:

[0159] S501 , aggregating multiple second gradients to obtain an aggregated gradient.

[0160] In one embodiment, the method for aggregating the second gradients to obtain the aggregated gradient can be similar to the method for aggregating the noise gradients to obtain the first gradient. For example, because each second gradient includes the weights and bias parameters of each layer in the model, the aggregation process requires separately aggregating the weights and bias parameters of the same layer, ultimately obtaining an aggregated gradient consisting of the aggregated weights and bias parameters of each layer.

[0161] It should be noted that when aggregating the first gradients of each participant, if a simple average weighting or weighting by data volume is used for aggregation, there will be significant limitations.

[0162] On the one hand, this approach fails to consider the actual contribution of each participant's historical data and the historical first gradient to global model optimization, potentially underestimating the value of high-quality data and interfering with model convergence due to low-quality or "contaminated" data. For example, some gas companies may experience missing equipment monitoring data or sensor failures, so gradient updates not only fail to improve model performance but may actually reduce prediction accuracy.

[0163] On the other hand, the lack of an incentive mechanism may make participants lack the motivation to improve data quality and optimize gradients, which may easily form a vicious cycle of "bad money driving out good money", seriously restricting the application effect of federated learning in the field of gas safety and the coordinated development of the industry.

[0164] Based on this, in order to quantify the historical contributions of participants, optimize the aggregated gradient after aggregation, and improve the performance of the updated global model, electronic devices can be used according to Figure 6 The steps S601-S603 shown generate a polymer gradient. Detailed description is as follows:

[0165] S601: Determine the historical contribution of each participant.

[0166] In one embodiment, the above historical contribution is used to quantify the improvement effect of the first gradient sent by the participant at the historical moment on the global model. The first gradient at the historical moment can be one or more, and there is no limitation on this.

[0167] In one embodiment, the above-mentioned historical contribution can be set by the regulator, or the node device can determine the historical contribution of each participant from dimensions such as model performance improvement, gradient validity, and data quality.

[0168] Among them, model performance improvement is used to measure the improvement of the participant's gradient on the prediction indicators of the global model; gradient validity is used to evaluate the consistency of the participant's gradient direction with the global optimal update direction, reflecting the effectiveness of the gradient on model optimization; data quality is used to evaluate the data quality of the participant from the perspective of data integrity, timeliness, and compliance.

[0169] As an example, a node device can record the prediction indicators of the global model before each iteration (for example, the fault prediction accuracy indicator). After each participant uploads the gradient and aggregates it, the updated prediction indicator can be recorded and the corresponding improvement value of the prediction indicator can be calculated. For example, the above improvement value is obtained by subtracting the prediction indicator before the update from the updated prediction indicator. The improvement value is then normalized to the range [0, 1] to prevent the results from being affected by differences in the magnitude of different indicators.

[0170] For example, in the fifth iteration, Gas Company A improved its global model accuracy from 82% to 85%, a 3% improvement. Company B only improved its accuracy from 82% to 82.5%, a 0.5% improvement. After normalization, Company A's score (historical contribution) is 0.83, while Company B's score (historical contribution) is 0.

[0171] And, as another example, the node device can calculate the similarity between the second gradient of the participant and the aggregate gradient (for example, by calculating using cosine similarity, Euclidean distance similarity, etc.); then, if the similarity is lower than a preset threshold (for example, 0.5), the first gradient can be regarded as an invalid gradient and the score is set to 0; otherwise, the similarity is retained as the original value of the historical contribution.

[0172] For example, the cosine similarity between the second gradient of enterprise C and the aggregate gradient is 0.8, while the similarity between enterprise D and the aggregate gradient is only 0.2 due to data anomalies. Therefore, the historical contribution of enterprise C can be determined to be 0.8, and the historical contribution of enterprise D can be determined to be 0.

[0173] In one embodiment, the node device may determine the historical contribution based on any one of the above methods, or may first determine the contribution separately through the above multiple methods, and then perform weighted summation on the contribution determined by the multiple methods to obtain the historical contribution corresponding to each participant.

[0174] S602: Determine an aggregation weight corresponding to each second gradient based on historical contribution.

[0175] S603 : Perform weighted summation of each second gradient and the corresponding aggregation weight to obtain an aggregated gradient.

[0176] In one embodiment, the aggregation weight of a high historical contribution is greater than or equal to the aggregation weight of a low historical contribution.

[0177] As an example, the node device may pre-set aggregation weights corresponding to a plurality of contribution ranges, and then determine the aggregation weight corresponding to the second gradient based on the contribution range in which the historical contribution falls.

[0178] Alternatively, the node device may also set weight adjustment values ​​corresponding to multiple contribution ranges, as well as a preset weight for each participant. Then, based on the contribution range within which the historical contribution falls, a weight adjustment value corresponding to the second gradient is determined, and the preset weight is adjusted based on the weight adjustment value to obtain the aforementioned aggregate weight. In this embodiment, the method for determining the aggregate weight is not limited.

[0179] In one embodiment, after each second gradient is obtained, each second gradient may be weighted summed with the corresponding aggregation weight to obtain the above-mentioned aggregated gradient.

[0180] In this embodiment, the improvement effect of each participant's second gradient on the global model is quantified based on historical contribution. This allows for accurate identification of high-quality participants with strong gradient validity, giving participants with high historical contributions a higher weight during second gradient aggregation, incentivizing them to continue providing high-quality data. For participants with low historical contributions, their gradient aggregation weight is reduced, effectively suppressing interference with the model caused by invalid gradients due to missing data, anomalies, or malicious attacks. This differentiated weight allocation mechanism significantly reduces the risk of malicious nodes disrupting model training by uploading "contaminated" gradients, preventing low-quality data from misleading the model's optimization direction, thereby accelerating global model convergence and improving model prediction accuracy and generalization capabilities.

[0181] S502: Calculate the product of the aggregated gradient and the preset learning rate.

[0182] S503: Determine the difference between the initial model parameter and the product as the target model parameter.

[0183] S504: Replace the initial model parameters with the target model parameters to obtain an updated global model.

[0184] In one embodiment, the preset learning rate may determine the magnitude of adjustments to model parameters (e.g., weights, biases, etc.) based on the aggregated gradient at each iteration. The preset learning rate may be set by the supervisor based on actual circumstances and is not limited thereto.

[0185] It is understandable that if the learning rate is too large, the parameter update step may exceed the optimal solution position, causing the model to oscillate or even diverge (not converge). If the learning rate is too small, the parameter update is slow, the model convergence speed is greatly reduced, and it may fall into a local optimal solution or fail to converge for a long time.

[0186] The node device may determine the difference between the initial model parameter and the product as the updated target model parameter.

[0187] As an example, the node device can determine the target model parameters according to the following formula. The details are as follows:

[0188] ;

[0189] in, represents the target model parameters, represents the initial model parameters during the tth global model update, represents the preset learning rate, represents the aggregation gradient.

[0190] It is understandable that when the global model is updated t+1 times, the initial model parameters in the global model can be considered to be .

[0191] In one embodiment, the node device may update the global model according to the above method; alternatively, after obtaining the target model parameters, the node device may send the target model parameters to the supervisor's corresponding electronic device, so that the supervisor's corresponding electronic device updates the global model based on the target model parameters. In this embodiment, the method for updating the global model is not limited.

[0192] Among them, after the electronic device corresponding to the supervisor updates the global model based on the model parameters, the electronic device can generate a risk assessment report based on the global model, and display the corresponding macro indicators and risk assessment report on the dashboard.

[0193] It should be noted that during the gradient aggregation phase, since the second gradient comes from the local training of each participant, it can be integrated to incorporate the characteristics of multiple data sources, avoiding the limitations of a single data source and thus improving the model's generalization ability. Furthermore, the introduction of a preset learning rate allows for precise control of the parameter update step size, ensuring the stability and accuracy of model optimization. Finally, by determining the difference between the initial model parameters and the product as the target model parameters and updating them, the model can gradually adjust the parameters based on the direction and magnitude of the aggregated gradient to reduce the loss function value, resulting in a global model with better performance and greater adaptability, improving the model's prediction accuracy and reliability in practical application scenarios.

[0194] In this embodiment, after obtaining the first gradient and proof parameters generated by a participant based on local data and the federated learning framework, a second gradient can be determined from multiple first gradients based on the validity of the proof parameters. This means that the second gradient can be deemed "trusted" because the model update information it carries is obtained through training based on local data that meets preset constraints, rather than training based on false local data. This improves the accuracy of subsequent global model updates based on the second gradient. Furthermore, the first gradient is generated and uploaded by the participant when training the initial model parameters in the federated learning framework based on local data. Therefore, it can be assumed that the participant only uploads the gradient information, not the original local data. This ensures that sensitive local data remains locally, preventing the risk of commercial secrets being leaked or public panic caused by direct data sharing. Finally, the initial model parameters in the global model are updated based on the multiple second gradients, resulting in an updated global model. Because the multiple second gradients are generated by each participant based on their own local data, they essentially incorporate features from a large amount of data. Furthermore, by updating the initial model parameters based on multiple second gradients, the global model can capture deep patterns that cannot be covered by the data of a single participant, thereby improving the training accuracy of the updated global model.

[0195] In another embodiment, to ensure traceability and security, blockchains typically store aggregated gradients. Under this mechanism, participants must rely on aggregated gradients to update their local models, but arbitrarily sending aggregated gradients to all participants may lead to a trust crisis.

[0196] To ensure the legitimacy and credibility of the aggregated gradient, prevent blind dissemination of the aggregated gradient, and reduce trust risks among participants, node devices can count the total number of signatures obtained; the signatures serve as proof that the participants have requested the aggregated gradient. The aggregated gradient is then published if the total number is greater than or equal to a preset number. Otherwise, if the total number is less than the preset number, the aggregated gradient is prohibited from being published.

[0197] The preset number can be set according to actual conditions and is not limited. In addition, the signature can be considered as a proof of the authenticity and validity of the participant's active request to obtain the aggregated gradient.

[0198] It's understandable that by counting the total number of signatures from participants requesting the aggregated gradient and setting a preset threshold for release, the credibility of the process can be effectively strengthened: the release of the aggregated gradient is triggered only when the total number of signatures reaches or exceeds the preset threshold. Furthermore, the signature mechanism enables consensus among participants on the aggregation results, ensuring the legitimacy and credibility of the aggregated gradient, preventing blind dissemination of the aggregated gradient, and minimizing trust risks among participants.

[0199] Reference Figure 7, shows a schematic flow chart of another step of generating a global model provided by an embodiment of the present application, which may specifically include the following steps:

[0200] S701. Perform model training based on local data and a federated learning architecture to obtain a first gradient generated during the training process; the federated learning architecture includes initial model parameters for model training.

[0201] In this embodiment, the execution subject may be an electronic device. The electronic device may obtain the macro-indicator and the federated learning architecture sent by the node device, obtain training data corresponding to the macro-indicator from local data, and perform model training based on the training data and the federated learning architecture to obtain a first gradient.

[0202] Alternatively, when training the initial model parameters in the model based on local data, a third gradient generated by the model during the training process is obtained; the third gradient includes at least one; then, each third gradient is noise processed separately to obtain a corresponding noise gradient; and each noise gradient is aggregated to generate a first gradient.

[0203] Also, when obtaining the noise gradient, for any third gradient, determine the sensitivity category of the local data used when generating the third gradient; determine the noise corresponding to the third gradient based on the sensitivity category; the noise corresponding to the high sensitivity category is greater than the noise corresponding to the low sensitivity category; and process the third gradient based on the noise to obtain the noise gradient.

[0204] S702: Generate proof parameters for verifying that local data complies with preset constraints.

[0205] S703. Send the first gradient and proof parameters to a node device of the blockchain; the node device is used to determine multiple second gradients and update the initial model parameters of the global model based on the multiple second gradients to obtain an updated global model; the second gradient is the first gradient corresponding to the valid parameters in the proof parameters.

[0206] In one embodiment, the terms and processing procedures corresponding to S701-S703 are described in the above. Figures 2 to 4 The corresponding embodiments are explained and no further explanation is given.

[0207] In this embodiment, the electronic device can be trained based on local data and initial model parameters to obtain a first gradient, which not only fully utilizes the data value of the participants, but also avoids direct exposure of the original data and ensures data privacy and security. Afterwards, proof parameters are generated to verify that the local data meets the preset constraints, effectively ensuring data quality, compliance and integrity, and preventing "contaminated" data from interfering with model training. Finally, the first gradient and proof parameters are sent to the blockchain node device. The blockchain's tamper-proof characteristics can be used to store evidence, and the second gradient corresponding to the valid proof parameters can be screened through node verification to filter invalid or malicious gradients. Based on this, when the global model is updated based on the screened second gradient, the training accuracy of the updated global model can be improved, while avoiding data leakage and security risks, and promoting multi-party security.

[0208] In order to more clearly illustrate the solution in this application, the solution in this application is described below using specific examples. Figure 8 , Figure 8 This is a schematic diagram of an application scenario in a model generation method based on zero-knowledge proof and federated learning provided in another embodiment of the present application.

[0209] For example, a provincial gas safety management center collaborated with multiple gas companies to build a global equipment failure prediction model (a global model) based on federated learning and blockchain technology. Participants included: node devices: verification nodes in the blockchain network (deployed by the management center); electronic devices: local servers at each gas company, storing user gas equipment data; and pre-set constraints: desensitized equipment operating data that complies with the "Urban Gas Data Security Specification."

[0210] First, node devices can broadcast the federated learning architecture (including initial model parameters) and macro indicators (for example, abnormal device pressure data in the past three months).

[0211] Gas company A can select relevant training data from its local database. Using this local training data, company A trains a model, generating multiple layers of third gradients. Then, based on the sensitivity of the local data used to generate the third gradients, company A adds differentiated noise to the third gradients at different sensitivity layers, generating noise gradients. Finally, the noise gradients are aggregated to generate the first gradient.

[0212] At the same time, enterprise A can use zero-knowledge proof (ZKP) technology to generate proof parameters corresponding to the training data, and send the proof parameters and the first gradient to the node device.

[0213] The node device can verify the validity of the proof parameters uploaded by each enterprise respectively, and determine the second gradient corresponding to the valid proof parameters. Then, the weight corresponding to each second gradient can be determined based on the historical contribution degree of each participant, so as to obtain an aggregated gradient by weighted summation of the second gradients. For example, the historical contribution degree of enterprise A is determined to determine the weight of the second gradient uploaded by enterprise A, and the subsequent processing is participated.

[0214] Finally, the node device can calculate the product of the aggregated gradient and the preset learning rate, and determine the difference between the initial model parameters and the product as the target model parameters, so as to replace the initial model parameters with the target model parameters to obtain the updated global model.

[0215] Referring to Figure 9 , Figure 9 A schematic diagram of a model generation device based on zero-knowledge proof and federated learning is shown, which can be applied to a node device of a block chain. The model generation device based on zero-knowledge proof and federated learning 900 can include an acquisition module 910, a determination module 920, and an update module 930, wherein:

[0216] The acquisition module 910 is configured to acquire the first gradient and the proof parameter sent by each participant. The first gradient includes the gradient generated when the participant performs model training based on the local data and the federated learning architecture, and the proof parameter includes the parameter used to verify that the local data meets the preset constraint condition. The federated learning architecture includes the initial model parameters used for model training.

[0217] The determination module 920 is configured to determine a plurality of second gradients. The second gradient is the first gradient corresponding to the valid parameter in the proof parameter.

[0218] The update module 930 is configured to update the initial model parameters in the global model based on the plurality of second gradients to obtain an updated global model.

[0219] In an embodiment, the first update module 930 is further configured to:

[0220] aggregate the plurality of second gradients to obtain an aggregated gradient; calculate the product of the aggregated gradient and a preset learning rate; determine the difference between the initial model parameters and the product as target model parameters; and replace the initial model parameters with the target model parameters to obtain an updated global model.

[0221] In an embodiment, the first update module 930 is further configured to:

[0222] Determine the historical contribution corresponding to each participant; the historical contribution is used to quantify the improvement effect of the first gradient sent by the participant at the historical moment on the global model; determine the aggregation weight corresponding to each second gradient based on the historical contribution; the aggregation weight of the high historical contribution is greater than or equal to the aggregation weight of the low historical contribution; each second gradient is weighted and summed with the corresponding aggregation weight to obtain the aggregate gradient.

[0223] In one embodiment, the model generation device 900 based on zero-knowledge proof and federated learning further includes:

[0224] The statistics module is used to count the total number of signatures obtained; the signature is the proof that the participant requests to obtain the aggregate gradient.

[0225] The publishing module is used to publish the aggregate gradient if the total quantity is greater than or equal to the preset quantity.

[0226] When it is understood that Figure 9 In the structural diagram of the model generation device based on zero-knowledge proof and federated learning, each module is used to execute Figure 1 、 Figure 5 and Figure 6 The steps in the corresponding embodiment, and for Figure 1 、 Figure 5 and Figure 6 Each step in the corresponding embodiment has been explained in detail in the above embodiment. Figure 1 、 Figure 5 and Figure 6 as well as Figure 1 、 Figure 5 and Figure 6 The relevant descriptions in the corresponding embodiments will not be repeated here.

[0227] Reference Figure 10 , Figure 10 A schematic diagram of another model generation device based on zero-knowledge proof and federated learning provided in an embodiment of the present application is shown. The device can be applied to electronic devices. The model generation device 1000 based on zero-knowledge proof and federated learning may include a training module 1010, a generation module 1020, and a sending module 1030, wherein:

[0228] The training module 1010 is used to perform model training based on local data and a federated learning architecture to obtain a first gradient generated during the training process; the federated learning architecture includes initial model parameters for model training.

[0229] The generation module 1020 is used to generate proof parameters for verifying that the local data meets the preset constraint conditions.

[0230] The sending module 1030 is configured to send the first gradient and the proof parameter to a node device of a block chain; the node device is configured to determine a plurality of second gradients, and update initial model parameters of a global model based on the plurality of second gradients to obtain an updated global model; the second gradient is the first gradient corresponding to an effective parameter in the proof parameter.

[0231] In an embodiment, the training module 1010 is further configured to:

[0232] obtain macroscopic indicators and a federated learning architecture sent by the node device; obtain training data corresponding to the macroscopic indicators from the local data; and perform model training based on the training data and the federated learning architecture to obtain the first gradient.

[0233] In an embodiment, the training module 1010 is further configured to:

[0234] In the model training based on the local data on the initial model parameters in the model, a third gradient generated in the training process of the model is obtained; the third gradient includes at least one; each third gradient is processed with noise to obtain a corresponding noise gradient; and each noise gradient is aggregated to generate the first gradient.

[0235] In an embodiment, the training module 1010 is further configured to:

[0236] For any third gradient, a sensitivity category of the local data used to generate the third gradient is determined; a noise corresponding to the third gradient is determined based on the sensitivity category; the noise corresponding to the high sensitivity category is greater than the noise corresponding to the low sensitivity category; and the third gradient is processed based on the noise to obtain the noise gradient.

[0237] It is understood that, Figure 10 The structure diagram of the model generation device based on the zero-knowledge proof and the federated learning is shown, each module is configured to perform Figure 2-Figure 4 and Figure 7 each step in the corresponding embodiment, and for Figure 2-Figure 4 and Figure 7 each step in the corresponding embodiment has been explained in detail in the above embodiments, please refer to Figure 2-Figure 4 and Figure 7 and Figure 2-Figure 4 and Figure 7 the related description in the corresponding embodiment, which will not be repeated here.

[0238] Figure 11 is a structure diagram of a computer device provided by an embodiment of the present application. As Figure 11As shown, the computer device 1100 of this embodiment includes: a processor 1110, a memory 1120, and a computer program 1130 stored in the memory 1120 and executable on the processor 1110, such as a program for a model generation method based on zero-knowledge proof and federated learning. When the processor 1110 executes the computer program 1130, the steps of each embodiment of the model generation method based on zero-knowledge proof and federated learning are implemented, such as Figure 1 S101 to S103 shown or Figure 7 Alternatively, the processor 1110 executes the computer program 1130 to implement the above Figure 9 or Figure 10 The functions of each module in the corresponding embodiment are, for example, Figure 9 or Figure 10 For details on the functions of each module, please refer to Figure 9 or Figure 10 Related description in the corresponding embodiment.

[0239] Exemplarily, the computer program 1130 can be divided into one or more modules, one or more of which are stored in the memory 1120 and executed by the processor 1110 to implement the model generation method based on zero-knowledge proof and federated learning provided in the embodiment of the present application. One or more modules can be a series of computer program instruction segments that can perform specific functions, and the instruction segments are used to describe the execution process of the computer program 1130 in the computer device 1100. For example, the computer program 1130 can implement the model generation method based on zero-knowledge proof and federated learning provided in the embodiment of the present application.

[0240] The computer device 1100 may include, but is not limited to, a processor 1110 and a memory 1120. Those skilled in the art will appreciate that Figure 10 This is merely an example of the computer device 1100 and does not constitute a limitation of the computer device 1100 . The computer device 1100 may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the computer device may also include input and output devices, network access devices, buses, etc.

[0241] The processor 1110 may be a central processing unit, or other general-purpose processor, digital signal processor, application-specific integrated circuit, off-the-shelf programmable gate array or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0242] The memory 1120 may be an internal storage unit of the computer device 1100, such as a hard disk or memory of the computer device 1100. The memory 1120 may also be an external storage device of the computer device 1100, such as a plug-in hard disk, smart memory card, flash memory card, etc. equipped on the computer device 1100. Furthermore, the memory 1120 may include both an internal storage unit of the computer device 1100 and an external storage device.

[0243] An embodiment of the present application provides a computer-readable storage medium, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, a model generation method based on zero-knowledge proof and federated learning as described in the above-mentioned embodiments is implemented.

[0244] An embodiment of the present application provides a computer program product. When the computer program product is run on a computer device, the computer device executes the model generation method based on zero-knowledge proof and federated learning in the above-mentioned embodiments.

[0245] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A model generation method based on zero-knowledge proof and federated learning, characterized in that: A node device applied to a blockchain, the method comprising: Obtaining first gradients and proof parameters sent by each participant; the first gradients include gradients generated by the participant when training a model based on local data and a federated learning architecture; the proof parameters include parameters used to verify that the local data meets preset constraints; the federated learning architecture includes initial model parameters used for model training; Determine a plurality of second gradients; wherein the second gradients are the first gradients corresponding to the valid parameters in the certification parameters; updating the initial model parameters in the global model based on the plurality of second gradients to obtain the updated global model; The updating of the initial model parameters in the global model based on the plurality of second gradients to obtain the updated global model includes: aggregating a plurality of the second gradients to obtain an aggregated gradient; Calculating the product of the aggregated gradient and a preset learning rate; Determine the difference between the initial model parameter and the product as the target model parameter; Replacing the initial model parameters with the target model parameters to obtain the updated global model; Determine a historical contribution corresponding to each participant; the historical contribution is used to quantify the improvement effect of the first gradient sent by the participant at a historical moment on the global model; Determine an aggregation weight corresponding to each second gradient based on the historical contribution; the aggregation weight with a high historical contribution is greater than or equal to the aggregation weight with a low historical contribution; respectively performing weighted summation of each second gradient and the corresponding aggregation weight to obtain the aggregated gradient; Determine the historical contribution of each participant, including: For any of the participants, based on the difference between the prediction index of the updated global model and the prediction index of the global model before the update, determine the model performance improvement effect of the updated global model; the model performance improvement effect is used to measure the improvement of the prediction index of the global model by the first gradient of the participant; Calculating the similarity between the first gradient corresponding to the participant and the aggregated gradient, and generating a gradient validity of the first gradient based on the similarity; the gradient validity is used to evaluate the consistency of the gradient direction of the participant and the optimal update direction of the global model; Obtaining the data quality of the training data corresponding to the participant; the data quality is evaluated from the perspectives of data integrity, timeliness, and compliance; The historical contribution of the participant is determined based on the model performance improvement effect, the gradient validity and the data quality.

2. The method according to claim 1, characterized in that The method further comprises: Counting the total number of acquired signatures; the signatures are proof that the participant requested to acquire the aggregate gradient; If the total number is greater than or equal to a preset number, the aggregate gradient is published.

3. A model generation method based on zero-knowledge proof and federated learning, characterized in that: Applied to electronic equipment, the method includes: Performing model training based on local data and a federated learning architecture to obtain a first gradient generated during the training process; the federated learning architecture includes initial model parameters for model training; Generating a proof parameter for verifying that the local data complies with a preset constraint condition; The first gradient and the proof parameter are sent to a node device of the blockchain; the node device is configured to determine a plurality of second gradients, and update the initial model parameters of the global model based on the plurality of second gradients to obtain an updated global model; the second gradient is the first gradient corresponding to the valid parameter in the proof parameter; The node device updates the initial model parameters in the global model based on the plurality of second gradients to obtain the updated global model, including: aggregating a plurality of the second gradients to obtain an aggregated gradient; Calculating the product of the aggregated gradient and a preset learning rate; Determine the difference between the initial model parameter and the product as the target model parameter; Replacing the initial model parameters with the target model parameters to obtain the updated global model; Determine the historical contribution corresponding to each participant; the historical contribution is used to quantify the improvement effect of the first gradient sent by the participant at the historical moment on the global model; Determine an aggregation weight corresponding to each second gradient based on the historical contribution; the aggregation weight with a high historical contribution is greater than or equal to the aggregation weight with a low historical contribution; respectively performing weighted summation of each second gradient and the corresponding aggregation weight to obtain the aggregated gradient; Determine the historical contribution of each participant, including: For any of the participants, based on the difference between the prediction index of the updated global model and the prediction index of the global model before the update, determine the model performance improvement effect of the updated global model; the model performance improvement effect is used to measure the improvement of the prediction index of the global model by the first gradient of the participant; Calculating the similarity between the first gradient corresponding to the participant and the aggregated gradient, and generating a gradient validity of the first gradient based on the similarity; the gradient validity is used to evaluate the consistency of the gradient direction of the participant and the optimal update direction of the global model; Obtaining the data quality of the training data corresponding to the participant; the data quality is evaluated from the perspectives of data integrity, timeliness, and compliance; The historical contribution of the participant is determined based on the model performance improvement effect, the gradient validity and the data quality.

4. The method according to claim 3, characterized in that The model training is performed based on local data and the federated learning architecture to obtain the first gradient generated during the training process, including: Obtaining the macro indicators and the federated learning architecture sent by the node device; Obtaining training data corresponding to the macro indicator from the local data; Model training is performed based on the training data and the federated learning architecture to obtain the first gradient.

5. The method according to claim 4 or 3, characterized in that The model training is performed based on local data and the federated learning architecture to obtain the first gradient generated during the training process, including: When the initial model parameters in the model are trained based on the local data, a third gradient generated by the model during the training process is obtained; the third gradient includes at least one; performing noise processing on each of the third gradients to obtain a corresponding noise gradient; Aggregating each of the noise gradients generates the first gradient.

6. The method according to claim 5, characterized in that The performing noise processing on each of the third gradients to obtain a corresponding noise gradient includes: For any of the third gradients, determining a sensitivity category of the local data used when generating the third gradient; Determine the noise corresponding to the third gradient based on the sensitivity category; the noise corresponding to the high sensitivity category is greater than the noise corresponding to the low sensitivity category; The third gradient is processed based on the noise to obtain the noise gradient.

7. A computer device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the computer device implements the method according to any one of claims 1 to 2 or 3 to 6.

8. A computer program product, characterized in that The invention comprises a computer program, which, when being executed, enables the method according to any one of claims 1 to 2 or 3 to 6 to be performed.

Images