Method, System, Computing Node and Medium for Scaling in Data Product Circulation

Abstract

An embodiment of the present invention provides a method, system, computing node, and medium for scaling in and out during the circulation of data products. The method includes: as a newly added participant in a federated learning task of a federated learning data product, after receiving the federated learning data product release information sent by the data trading platform, applying to the data trading platform for a data trading certificate, so that the data trading platform audits the newly added participant based on the configuration information of the federated learning data product, and issues a data trading certificate and an instruction to join the federated learning task to the newly added participant that passes the audit; if receiving the instruction to join the federated learning task issued by the data trading platform, participating in the federated learning task to obtain the output result of the federated learning scenario, so that the data trading platform sends the output result to the target participant. Using this method, dynamic scaling in and out of federated learning participants is achieved during the circulation of data products, reducing the impact on the business.

Description

Technical Field

[0001] The present invention relates to the technical field of data circulation, and in particular, to a method, system, computing node and medium for scaling in and out in data product circulation. Background Art

[0002] Different from the traditional data product's definition method of sharing data entities, the data product based on federated learning provides a new method for data product management and circulation through data product definition, data flow process management and data version management applicable to federated learning, and realizes the innovation of the data element trading process while ensuring data security and privacy.

[0003] However, the data circulation based on federated learning has its particularity. In order to improve the effect of the federated learning model, it may be necessary to introduce data of other participants during the operation of the data product. Some participants need to withdraw from the federated learning process due to data quality or other reasons. However, the current data products based on federated learning cannot support the dynamic scaling in and out during these data transfer processes. It is necessary to stop the running data circulation instance, update the data product, and then cold start and run a new data circulation instance, which affects the efficiency of data circulation and cannot reuse the previous model and needs to relearn, resulting in poor user experience. Summary of the Invention

[0004] Embodiments of the present invention provide a method, system, computing node and medium for scaling in and out in data product circulation, realizing the dynamic scaling in and out of federated learning participants during the data product circulation process and reducing the impact on the business process.

[0005] In a first aspect, an embodiment of the present invention provides a method for scaling in and out in data product circulation, which is applied to a participant in data product circulation in a data product circulation system. The method includes:

[0006] As a new participant in the federated learning task of the federated learning data product, after receiving the federated learning data product release information sent by the data trading platform, apply to the data trading platform for a data trading certificate, so that the data trading platform audits the new participant based on the configuration information of the federated learning data product, and issues a data trading certificate and an instruction to join the federated learning task to the newly added participant that passes the audit;

[0007] If receiving the instruction to join the federated learning task sent by the data trading platform, participate in the federated learning task to obtain the output result of the federated learning scenario, so that the data trading platform sends the output result to the target participant.

[0008] Second aspect, an embodiment of the present invention provides a method for scaling in and out in data product circulation, which is applied to a data trading platform in a data product circulation system. The method includes:

[0009] After receiving a data trading voucher application sent by a newly added participant, the newly added participant is audited based on the configuration information of the federated learning data product;

[0010] Issue a data trading voucher and a federated learning task instruction to join the federated learning data product to the newly added participant that passes the audit, so that the corresponding newly added participant can participate in the federated learning task to obtain the output result of the federated learning scenario;

[0011] Send the output result to the target participant.

[0012] Third aspect, an embodiment of the present invention further provides a data product circulation system, including: a data trading platform and participants for data circulation. The participants for data circulation include participants as the initiator of the federated learning task, newly added participants, and existing participants; among them,

[0013] The participant as the initiator of the federated learning task is used to complete the configuration information of the federated learning data product locally and send a federated learning data product registration request to the data trading platform. The federated learning data product registration request includes the configuration information of the federated learning data product;

[0014] The data trading platform is used to manage the federated learning data product and then send the federated learning data product release information to each participant;

[0015] As a newly added participant in the federated learning task, it is used to apply for a data trading voucher from the data trading platform after receiving the federated learning data product release information sent by the data trading platform;

[0016] The data trading platform is used to audit the newly added participant based on the configuration information of the federated learning data product, and issue a data trading voucher and a federated learning task instruction to join the federated learning task to the newly added participant that passes the audit;

[0017] The newly added participant is used to participate in the federated learning task to obtain the output result of the federated learning scenario if it receives the federated learning task instruction to join issued by the data trading platform;

[0018] The data trading platform is used to send the output result to the target participant;

[0019] The existing participant as the federated learning task is used to send an application to withdraw from the federated learning task to the data trading platform during the operation of the federated learning task;

[0020] The data trading platform is further configured to audit the existing participants based on the configuration information of the federated learning data product, and issue an instruction to exit the federated learning task to the existing participants who pass the audit;

[0021] The existing participant is further configured to exit the federated learning task if it receives the instruction to exit the federated learning task issued by the data trading platform.

[0022] Fourthly, an embodiment of the present invention further provides a computing node, which includes:

[0023] At least one processor; and

[0024] A memory communicatively connected to the at least one processor; wherein,

[0025] The memory stores a computer program executable by the at least one processor, and when the computer program is executed by the at least one processor, the at least one processor is enabled to execute the method for scaling in and out during the circulation of data products according to the embodiments of the first aspect or the second aspect.

[0026] Fifthly, an embodiment of the present invention further provides a storage medium containing computer-executable instructions, and when the computer-executable instructions are executed by a computer processor, they are used to execute the method for scaling in and out during the circulation of data products according to the embodiments of the first aspect or the second aspect.

[0027] An embodiment of the present invention provides a method, a system, a computing node, and a medium for scaling in and out during the circulation of data products. The method includes: as a new participant in a federated learning task that is a federated learning data product, after receiving the federated learning data product release information sent by the data trading platform, applying to the data trading platform for a data trading certificate, so that the data trading platform audits the new participant based on the configuration information of the federated learning data product, and issues a data trading certificate and an instruction to join the federated learning task to the new participant who passes the audit; if receiving the instruction to join the federated learning task issued by the data trading platform, participating in the federated learning task to obtain the output result of the federated learning scenario, so that the data trading platform sends the output result to the target participant. Different from the prior art in which data products based on federated learning cannot support dynamic scaling in and out during these data transfer processes, it is necessary to stop the running data circulation instance and update the data product; the above technical solution expands based on the configuration items of the federated learning data product to make it adapt to the dynamic scaling in and out process, and actively manages the dynamic joining / exit of participants from the data element circulation process after the federated learning data product runs. Dynamic scaling in and out of federated learning participants is realized during the data product circulation process, reducing the impact on data circulation based on federated learning.

[0028] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it used to limit the scope of the present invention. Other features of the present invention will become readily understood from the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention, and those of ordinary skill in the art can obtain other drawings based on these drawings without creative efforts.

[0030] Figure 1 It is a schematic flowchart of a method for scaling in and out in data product circulation provided in Embodiment 1 of the present invention;

[0031] Figure 2 It is a schematic flowchart of a method for scaling in and out in data product circulation provided in Embodiment 2 of the present invention;

[0032] Figure 3 It is a schematic flowchart of a method for scaling in and out in data product circulation provided in Embodiment 3 of the present invention;

[0033] Figure 4 It is a schematic flowchart of a method for scaling in and out in data product circulation provided in Embodiment 4 of the present invention;

[0034] Figure 4a It is a schematic data circulation flowchart of dynamic scaling in of data products in a horizontal federated learning scenario provided in Embodiment 4 of the present invention;

[0035] Figure 4b It is a schematic data circulation flowchart of dynamic scaling out of data products in a horizontal federated learning scenario provided in Embodiment 4 of the present invention;

[0036] Figure 5 It is a schematic structural diagram of a data product circulation system provided in Embodiment 5 of the present invention;

[0037] Figure 6 It is a schematic data circulation flowchart of a data product learning scenario for federated learning provided in an exemplary embodiment of the present invention;

[0038] Figure 7 It is a schematic structural diagram of a computing node of the data product circulation method according to the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0039] To enable those skilled in the art to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0040] It should be noted that the terms "original", "target", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances, so that the embodiments of the present invention described here can be implemented in an order other than those illustrated or described here. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0041] Embodiment 1

[0042] Figure 1 FIG. is a schematic flowchart of a method for scaling in and out in data product circulation provided in Embodiment 1 of the present invention. This method is applicable to the situation of managing dynamic scaling in and out of participants in data product circulation. This method can be executed by a computing node as a participant, where the computing node can be implemented by software and / or hardware and is generally integrated into a data product circulation system. In this system, any participant can be both a data provider and a data consumer, and any participant can complete a federated learning task based on the data it owns and the data owned by other participants.

[0043] As Figure 1 shown, the method for scaling in and out in data product circulation provided in Embodiment 1 specifically may include the following steps:

[0044] S110. As a new participant in the federated learning task of the federated learning data product, after receiving the federated learning data product release information sent by the data trading platform, apply to the data trading platform for a data trading certificate, so that the data trading platform audits the new participant based on the configuration information of the federated learning data product, and issues a data trading certificate and an instruction to join the federated learning task to the new participant that passes the audit.

[0045] Considering that in the process of data flow based on federated learning, sometimes in order to improve the effect of the federated learning model, it may be necessary to introduce data from other participants during the operation of the data product. Some participants need to withdraw from the federated learning process due to data quality or other reasons. The application scenario of this embodiment can be specifically understood as a scenario where participants dynamically join or withdraw during the data flow process based on federated learning, without stopping the running data circulation instance.

[0046] Among them, federated learning is a technology in which each participant with data uses encryption technology to protect data privacy and achieve multi-party collaborative optimization of the model without the data leaving the original library. A federated learning data product can be understood as a data product for federated learning tasks; a data product is a product form produced based on data. It should be noted that the data product does not include the real original data. A data trading platform can be understood as a platform for data circulation and management, which can be used for the registration, subscription, review, release, update, and status management of data products. The data trading platform can also be responsible for the status information management of federated learning data products.

[0047] Among them, the federated learning data product release information can include the data trading platform sending a basic information subscription message of the federated learning data product to the participating party as the data consumer; it can also include the data trading platform sending a successful release message of the federated learning data product to the participating party as the data provider. A data trading certificate is a certificate issued by the data trading platform to the participating party to prove that the participating party has the qualification for data trading. There is no limitation on the way to apply for a data trading certificate to the data trading platform here.

[0048] In this embodiment, a new participating party who wants to join the federated learning task during the process from the release of the federated learning data product to the end of the task is denoted as a newly added participating party. It can be understood that the newly added participating party refers to the data consumer. There is no specific limit on the timing for the newly added participating party to apply to join the federated learning task, which can be any moment during the process from the release of the federated learning data product to the end of the task.

[0049] Among them, the configuration information of the federated learning data product is different from the definition method of traditional data products that share data entities. The configuration information can include defining the basic information of the federated learning data product and the change information of the federated learning data product; the basic information of the federated learning data product includes one or more of the following: the basic information of the federated learning data product, the participating party data information, the federated learning task information, and the status information of the federated learning data product; the change information of the federated learning data product includes one or more of the following: the attribution policy of the federated learning output result, the participating party management policy, and the security policy. There is no specific limitation on the process of auditing the participating party based on the configuration information of the federated learning data product here.

[0050] In this embodiment, the circulation of data products may involve multiple participants, among which there may be a task initiator for initiating a federated learning task of the federated learning data product. The initiator needs to complete the configuration of the federated learning data product locally, that is, define the federated data product, and send the configuration information to the data trading platform, so that the data trading platform can publish the federated learning data product and review whether the participants are eligible for trading according to the configuration information of the federated learning data product. It should be noted that the federated learning data product published by the data trading platform only contains logical and attribute configurations and does not include source data; in the process of data circulation, the local data of each participant will not leave the domain, avoiding data leakage.

[0051] In the process of the circulation of the federated learning data product, each participant can learn about the basic information of the federated learning data product, such as the data source format requirements, quantity requirements, and the corresponding federated learning task, through the federated learning data product release information sent by the data trading platform; if a new participant wants to participate in the federated learning task of the federated learning data product, they need to apply for a trading certificate from the data product trading platform, and the data trading platform will review the applications for trading certificates submitted by each participant according to the configuration information of the federated learning data product sent by the task initiator. If the application for the trading certificate submitted by the new participant passes the review, the participant can receive the trading certificate issued by the data trading platform and the instruction to join the federated learning task.

[0052] The configuration information may include the basic information of the federated learning data product and the change information of the federated learning data product. The basic information of the federated learning data product may include the basic information of the federated learning data product, the participant data information, the federated learning task information, and the status information of the federated learning data product. The change information of the federated learning data product may include the attribution policy of the federated learning output result, the participant management policy, and the security policy. It should be noted that in order to achieve dynamic scaling in the process of the circulation of the federated learning data product, that is, there are participants joining or leaving during the circulation process, relevant settings need to be made to the configuration information of the federated learning data product. The joining or leaving of participants mainly involves the participant management policy in the configuration information.

[0053] In this embodiment, the participant management strategy is configured. The participant management strategy includes a static participant management strategy and a dynamic participant management strategy. Static participant management strategy: During the training process of the federated learning model, the participants are fixed, and no new participants are allowed to join or existing participants are allowed to withdraw. Dynamic participant management strategy: During the training process of the federated learning model, within the range of the maximum and minimum number of participants, participants can dynamically join or withdraw. Before the formal start of the data circulation task, any participant meeting the data requirements is allowed to apply for a data transaction certificate. Once the task starts, no new participants are allowed to join and no existing participants are allowed to withdraw. When performing a federated learning task, the participant management strategy needs to be set in advance, and only one of the participant management strategies can be set. Exemplarily, the participant management strategy can be set to the static participant management strategy or the dynamic participant management strategy.

[0054] Continuing with the above description, when the data trading platform reviews new participants based on the configuration information of the federated learning data product, it is necessary to review the new participants according to the current participant management strategy to determine whether the new participants meet the relevant settings in the participant management strategy. If they meet the requirements, the review of the new participants will continue based on other content in the configuration information. If the review is passed, the new participant can receive the data transaction certificate and the instruction to join the federated learning task issued by the data trading platform.

[0055] Exemplarily, if the participant management strategy is the static participant management strategy 1: From the release of the federated learning data product to the end of the task, only the agreed participants are allowed to participate throughout the process, and the participant information and the number of parameter parties remain fixed. If such a configuration is adopted, for a new participant applying to join the federated learning task, the data trading platform will reject the application, and the new participant is not allowed to join during the data flow process. If the participant management strategy is the dynamic participant management strategy: During the training process of the federated learning model, within the range of the maximum and minimum number of participants, participants can dynamically join or withdraw. If such a configuration is adopted, for a new participant applying to join the federated learning task, the data trading platform will review whether the number of new participants after joining exceeds the maximum number of participants. If it meets the requirement, the new participant is allowed to join.

[0056] S120. If an instruction to join the federated learning task is received from the data trading platform, then participate in the federated learning task to obtain the output result of the federated learning scenario, so that the data trading platform can send the output result to the target participant.

[0057] It can be understood that for new participating parties, they can participate in the federated task of the federated learning data product only after receiving the instruction to join the federated learning task issued by the trading platform. In this embodiment, the federated learning scenario and the specific content of the federated learning task can be determined according to the configuration information of the federated data product. Exemplarily, if the federated learning scenario is a learning scenario, the participating party receives an instruction for the federated learning model learning task, and the federated learning task that can be executed is the federated learning model learning task, and the corresponding output result can be the final federated learning model.

[0058] Among them, the target participating party can be the participating party that can receive the output result. The target participating party can be determined according to the federated learning output result attribution policy in the configuration information. All participating parties can be used as the target participating party, or the data requester can be used as the target participating party. There is no specific limitation here.

[0059] The embodiment of the present invention provides a method for scaling in and out in data product circulation. The method includes: as a new participating party in the federated learning task of the federated learning data product, after receiving the federated learning data product release information sent by the data trading platform, applying to the data trading platform for a data trading voucher, so that the data trading platform audits the new participating party based on the configuration information of the federated learning data product, and issues a data trading voucher and an instruction to join the federated learning task to the new participating party that passes the audit; if receiving the instruction to join the federated learning task issued by the data trading platform, participating in the federated learning task to obtain the output result of the federated learning scenario, so that the data trading platform sends the output result to the target participating party. Different from the prior art in which data products based on federated learning cannot support dynamic scaling in and out during these data transfer processes and need to stop the running data circulation instance and update the data product; the above technical solution expands based on the configuration items of the data product based on federated learning to make it adapt to the dynamic scaling in and out process, and actively manages the dynamic joining / withdrawal of participating parties from the data element circulation process after the federated learning data product runs. Realize the dynamic scaling in and out of federated learning participating parties during the data product circulation process, reduce the impact on data circulation based on federated learning, and reduce the impact on the business.

[0060] Embodiment Two

[0061] Figure 2 FIG. is a schematic flowchart of another method for scaling in and out in data product circulation provided by Embodiment Two of the present invention. This embodiment is a further optimization of the above embodiment. For the content not detailed in this embodiment, please refer to Embodiment One.

[0062] As Figure 2 shown, Embodiment Two of the present invention provides a method for scaling in and out in data product circulation, which specifically includes the following steps:

[0063] S210. The participating party that is the initiator of the federated learning task completes the configuration information of the federated learning data product locally.

[0064] It should be noted that steps S210 - S220 describe the process of releasing the federated learning data product, which occurs before the newly added participating party receives the federated learning data product release information. It can also be understood that after the newly added participating party receives the federated learning data product release information, it can apply for a data transaction voucher from the data trading platform at any time. It can be seen that the executor of this step is the participating party of the federated learning task initiator.

[0065] Among them, the participating party can include a data provider and a data consumer. The initiator of the federated learning task can act as the data provider, and the initiator of the federated learning task needs to complete the configuration information of the federated learning data product locally.

[0066] Specifically, the functions of the configuration information mainly include: enabling the data trading platform to determine whether to release the federated learning product to the agreed participating party based on the basic information of the federated learning data product and the participating party data information in the configuration information; enabling the participating party to determine whether to participate in the federated learning task of the federated learning data product based on the basic information of the federated learning data product in the configuration information, that is, whether to apply for a data transaction voucher from the data trading platform; enabling the data trading platform to review whether the participating party has the qualification to participate in the federated learning task based on the participating party data information, federated learning task information, and participating party management strategy in the configuration information, that is, whether to issue a data transaction voucher to the participating party that submits a data transaction voucher application; enabling the data trading platform to determine which participating party to send the output result to based on the federated learning output result attribution strategy in the configuration information, that is, to determine the target participating party.

[0067] Optionally, the configuration information of the federated learning data product includes the basic information of the federated learning data product and the change information of the federated learning data product.

[0068] Optionally, the basic information of the federated learning data product includes one or more of the following: the basic information of the federated learning data product, the participating party data information, the federated learning task information, and the state information of the federated learning data product; among them, the basic information of the federated learning data product includes the product name, product introduction, and product adaptation scenario; the participating party data information includes the data source description and data field description; the federated learning task information includes algorithm and parameter information, federated learning scenario, and federated learning type.

[0069] Among them, the basic information of the federated learning data product describes the name, introduction, model classification, adaptation scenario, etc. of the federated learning data product, helping the demand side to quickly retrieve the required data product and understand the federated learning scenario suitable for the data product.

[0070] The participating party data information can be understood as a detailed description of the data sources and data fields of each participating party, which is used to clarify the data requirements of the federated learning model for the participating parties. Among them, the data source description includes data source type, data storage format, data classification, etc.; the data field description includes column name, Chinese name, field type, field description, and whether it is a sensitive type, etc. It should be noted that in the horizontal federated learning scenario, all data fields of the participating parties must be the same. Any participating party that wants to join the horizontal federated learning needs to ensure that the local data meets the requirements of the data fields in the logical model of the federated learning data product; in the vertical federated learning scenario, the data fields of each participating party are different, and only one party is allowed to hold the label field. Therefore, in the vertical federated learning data product, it is also necessary to determine the allocation scheme of data fields for all participating parties according to the local data situation of the participating parties.

[0071] The federated learning task information refers to the description of the algorithm information selected for the federated learning data product, including: algorithm name, algorithm type (classification / clustering / regression, etc.), federated learning type, federated learning scenario, input feature name and number, label name, label type, output type, etc. The participation of the federated learning algorithm can include general configuration parameters, hyperparameters, and model parameters. Among them, the federated learning type includes horizontal federated learning and vertical federated learning; the federated learning scenario is divided into learning scenario and inference scenario.

[0072] The status information of the federated learning data product includes: in submission, registered, published, running, ended, etc.

[0073] Optionally, the federated learning data product change information includes one or more of the following: federated learning output result attribution policy, participating party management policy, and security policy; among them, the participating party management policy includes the fixed number requirement of participating parties, the range of the number of participating parties, the static management policy of participating party management information, and the dynamic management policy of participating parties; the security policy includes transaction voucher application and authorization policy, participating party authentication policy, and encryption protocol.

[0074] Among them, the federated learning output result attribution policy stipulates which participating parties the output results belong to. The main attribution policies include being owned by all participating parties and being owned by the data requester, etc.

[0075] The participant management strategy may include participant information management, fixed number requirements for participants, the range of the number of participants (i.e., the minimum and maximum limits of the number of participants), static participant management strategies, and dynamic participant management strategies, etc. Among them, participant information management may include saving initial participant information and maintaining participant information, such as adding or deleting; the fixed number of participants refers to the number of specified participants in the federated learning data product; the range of the number of participants refers to the minimum and maximum numbers of participants in the federated learning task. Beyond this range, new participants are not allowed to be added or existing participants are not allowed to withdraw. The static participant management strategy means that during the training process of the federated learning model, the participants are fixed and no new participants are allowed to join or existing participants are allowed to withdraw in the middle. The dynamic participant management strategy means that during the training process of the federated learning model, within the range of the maximum and minimum numbers of participants, participants can dynamically join or withdraw.

[0076] The security strategy may include the application and authorization strategy for participant transaction vouchers, participant authentication strategies, encryption protocols, etc. Among them, the application and authorization strategy for participant transaction vouchers includes: when a participant wants to join a federated learning task, it needs to first apply for a transaction voucher from the data trading platform. The data trading platform will review whether the participant's data meets the data requirements in the logical model, as well as the impact on the current federated learning task and the business of other participants, to decide whether to issue a transaction voucher. Only participants holding a transaction voucher have the right to join the federated learning task; the participant authentication strategy means that before the start of the federated learning task, the data trading platform needs to authenticate and authorize all participants. The data trading platform needs to check with the participant information defined in the participant information management strategy. Participants with mismatched information will fail the authentication. In addition, all participants need to hold a transaction voucher; the encryption protocol refers to an explanation of the data, model encryption protocol, and transmission link encryption protocol used for the federated learning data product, such as homomorphic encryption, DH algorithm, and secret sharing and other encryption protocols.

[0077] S220. Initiate a registration request for the federated learning data product to the data trading platform so that the data trading platform manages the federated learning data product and sends the federated learning data product release information to each participant; among them, the registration request for the federated learning data product includes the configuration information of the federated learning data product.

[0078] In this embodiment, the registration request for the federated learning data product can be understood as a request to register the federated learning data product on the data trading platform. Managing the federated learning data product may include registering, reviewing, releasing, etc. of the federated learning data product.

[0079] In this embodiment, after the participating party, i.e., the data provider, which is the initiator of the federated learning task, completes the configuration information of the federated learning data product locally, it can initiate a registration request for the federated learning data product to the data trading platform. After receiving the registration request for the federated learning data product, the data trading platform can manage the registration, review, release, etc. of the federated learning data product, and send a successful federated learning data product release message to the participating party of the federated learning task initiator, and send a subscription message for the basic information of the released federated learning data product to the participating party that is not the federated learning task initiator, i.e., the data consumer. Among them, the participating party that is not the federated learning task initiator may include the data consumers specified by the federated learning task initiator, and may also include the data consumers who send subscription messages for the basic information of the federated learning data product to the data trading platform. Optionally, the subscription message can be sent in an active push manner or in a subscription request manner.

[0080] It can be understood that since the registration request for the federated learning data product includes the configuration information of the federated learning data product, after receiving the registration request for the federated learning data product, the data trading platform can register, review, and release the configuration information of the federated learning data product, so that all participating parties can obtain the configuration information of the federated learning data product.

[0081] S230. As a new participating party in the federated learning task of the federated learning data product, after receiving the federated learning data product release information sent by the data trading platform, it applies to the data trading platform for a data trading certificate, so that the data trading platform can review the new participating party based on the configuration information of the federated learning data product, and issue a data trading certificate and an instruction to join the federated learning task to the new participating party that passes the review.

[0082] Specifically, as a new participating party in the federated learning task, after receiving the federated learning data product release information sent by the data trading platform, it can apply to the data trading platform for a data trading certificate at any time, requesting the data trading platform to issue a data trading certificate, so that the new participating party holding the data trading certificate can participate in the federated learning task of the federated learning data product.

[0083] S240. If receiving the instruction to join the federated learning task sent by the data trading platform, participate in the learning process of the federated learning model to obtain the final federated learning model.

[0084] Specifically, if the data trading platform approves the new participant, the new participant will receive an instruction from the data trading platform to join the federated learning task. Then, the new participant holding the data trading certificate can participate in the learning process of the federated learning model. The new participant participates in the federated learning model learning task of the federated learning data product to obtain the final federated learning model. The learning process of the federated learning model is a prior art and will not be elaborated here.

[0085] Among them, the learning process of the federated learning model may include: combining the product data of multiple participants and completing the parameter learning of the algorithm while protecting privacy.

[0086] S250. Use the final federated learning model as the output result of the federated data product, so that the data trading platform sends the output result to the target participant.

[0087] Specifically, use the final federated learning model as the output result of the federated data product, so that the data trading platform sends the output result to the target participant. The target participant can be determined according to the federated learning output result attribution policy in the configuration information. All participants can be used as the target participant, or the data requester can be used as the target participant.

[0088] A method for expanding, contracting, and accommodating the circulation of data products provided in the second embodiment of the present invention. The definition of the configuration information of the federated learning data product in this method is different from the definition method of traditional data products that share data entities. The data product definition based on federated learning is adopted, which can avoid data leakage during the circulation of data products. At the same time, the release process of the federated learning data product is specified. After receiving the federated learning data product release information, the new participant can apply for a data trading certificate at any time, realizing dynamic expansion during data circulation.

[0089] As an optional embodiment of the embodiment of the present invention, based on the above optional embodiment, it further includes: as an existing participant in the federated learning task, sending an application to withdraw from the federated learning task to the data trading platform during the operation of the federated learning task, so that the data trading platform audits the existing participant based on the configuration information of the federated learning data product and issues an instruction to withdraw from the federated learning task to the existing participant who passes the audit; if receiving the instruction to withdraw from the federated learning task issued by the data trading platform, then withdraw from the federated learning task.

[0090] Among them, the existing participants can be specifically understood as the participants who have participated in the federated learning task. In this alternative embodiment, it mainly describes the scenario where an existing participant wants to withdraw from the federated learning task during the operation of the federated learning task. As an existing participant in the federated learning task, when wanting to withdraw from the federated learning task, an application for withdrawing from the federated learning task needs to be sent to the data trading platform. The data trading platform needs to review the existing participants based on the configuration information of the federated learning data product.

[0091] Continuing with the above description, when the data trading platform reviews the new participants based on the configuration information of the federated learning data product, it needs to review the existing participants according to the current participant management policy to determine whether the withdrawal of the existing participants meets the relevant settings in the participant management policy. If it meets the requirements, that is, the review is passed, an instruction to withdraw from the federated learning task will be issued to the existing participant. When the existing participant receives the instruction to withdraw from the federated learning task issued by the data trading platform, it can then withdraw from the federated learning task.

[0092] Exemplarily, if the participant management policy is Participant Static Management Policy 1: Throughout the process from the release of the federated learning data product to the end of the task, only the agreed-upon participants are allowed to participate, the participant information and the number of parameter parties are fixed and unchanged, and once the data circulation task is started, no new participants or withdrawals are allowed for all participants. If it is such a configuration, for an existing participant's application to withdraw from the federated learning task, the data trading platform will reject the review and not allow it to withdraw during the data flow process. If the participant management policy is the Participant Dynamic Management Policy: It means that during the training process of the federated learning model, within the range of the maximum and minimum number of participants, participants can dynamically join or withdraw. If it is such a configuration, for an existing participant's application to withdraw from the federated learning task, the data trading platform reviews whether the remaining number of participants after the withdrawal of the existing participant is not less than the minimum number of participants. If it meets the requirements, the existing participant is allowed to withdraw.

[0093] This alternative embodiment specifies the process of how an existing participant withdraws from the federated learning task during the operation of the federated learning task, realizing dynamic capacity reduction during the data flow process.

[0094] Embodiment III

[0095] Figure 3 It is a schematic flowchart of a data product circulation method provided by Embodiment III of the present invention. This method can be applicable to the situation of managing and circulating data products. This method can be executed by a computing node acting as a data trading platform, where the computing node can be implemented by software and / or hardware and is generally integrated into the data product circulation system. In this system, any participant can act as both a data provider and a data consumer, and any participant can complete the federated learning task based on the data it owns and the data owned by other participants.

[0096] As Figure 3 shown, a data product circulation method provided in Embodiment 3 of the present invention includes the following steps:

[0097] S310. After receiving a data transaction voucher application sent by a newly added participant, audit the newly added participant based on the configuration information of the federated learning data product.

[0098] In this embodiment, after the data trading platform receives a data transaction voucher application sent by a newly added participant, it can audit the participant who sent the data transaction voucher application according to the configuration information of the federated learning data product configured by the initiator of the federated learning task, so as to determine whether the newly added participant meets the requirements.

[0099] Exemplarily, the data trading platform queries the participant management policy of the federated learning data product. Audit whether the newly added participant has the permission to join according to the participant management policy. For another example, the data trading platform audits whether the local data of the newly added participant meets the data field requirements of the federated learning data product. In the horizontal federated learning scenario, the local data fields of all participants must be exactly the same as the defined data fields; in the vertical federated learning scenario, the local data fields of the participant need to be the same as the data fields allocated to this participant in the federated learning data product. If the participant does not meet the data requirements defined by the data product, the request of the participant to apply for a transaction voucher is rejected.

[0100] Alternatively, the data trading platform initiates a federated data quality assessment to the newly added participant. After the newly added participant jointly completes the data quality assessment, the assessment report is sent to the data trading platform. The data trading platform audits the data quality report of the newly added participant. If the data quality report of the newly added participant does not meet the requirements, the request of the participant to apply for a transaction voucher is rejected.

[0101] Optionally, the configuration information of the federated learning data product includes the basic information of the federated learning data product and the change information of the federated learning data product.

[0102] Among them, the basic information of the federated learning data product includes one or more of the following:

[0103] The basic information of the federated learning data product, participant data information, federated learning task information, and federated learning data product status information;

[0104] Among them, the basic information of the federated learning data product includes the product name, product introduction, and product adaptation scenario; the participant data information includes the data source description and data field description; the federated learning task information includes the algorithm and parameter information, federated learning scenario, and federated learning type; the federated learning data product status information includes submitted, registered, published, running, or ended.

[0105] Among them, the federal learning data product change information includes one or more of the following:

[0106] The federal learning output result attribution policy, the participant management policy, and the security policy;

[0107] Among them, the participant management policy includes the fixed number requirement of participants, the range of the number of participants, the participant information management, the participant static management policy, and the participant dynamic management policy; the security policy includes the trading voucher application and authorization policy, the participant authentication policy, and the encryption protocol.

[0108] S320. Issue data trading vouchers and federal learning task instructions for joining the federal learning data product to the newly added participants that have passed the review, so that the corresponding newly added participants can participate in the federal learning task to obtain the output results of the federal learning scenario.

[0109] In this embodiment, if the application for the data trading voucher sent by the newly added participant passes the review, the data trading platform can send the data trading voucher to the newly added participant, and send the federal learning task instruction for joining the federal learning task to the newly added participant holding the data trading voucher, so that the participant holding the data trading voucher can participate in the federal learning task of the federal learning data product to obtain the corresponding output results.

[0110] S330. Send the output results to the target participants.

[0111] Specifically, the target participants can be determined according to the federal learning output result attribution policy in the configuration information. All participants can be used as the target participants, or the data demanders can also be used as the target participants. Send the output results to the target participants.

[0112] An embodiment of the present invention provides a method for scaling in and out in data product circulation, which is applied to a data trading platform in a data product circulation system. The method includes: after receiving a data trading voucher application sent by a newly added participant, auditing the participant based on the configuration information of the federated learning data product; issuing a data trading voucher and a federated learning task instruction to join the federated learning data product to the newly added participant that has passed the audit, so that the corresponding newly added participant can participate in the federated learning task to obtain the output result of the federated learning scenario; sending the output result to the target participant. Different from the prior art, the data product based on federated learning cannot support the dynamic scaling in and out during these data transfer processes, and it is necessary to stop the running data circulation instance and update the data product. The above technical solution expands based on the configuration items of the federated learning data product to make it adapt to the dynamic scaling in and out process. After the federated learning data product runs, it actively manages the dynamic joining / withdrawal of participants from the data element circulation process. Realize the dynamic scaling in and out of federated learning participants during the data product circulation process, and reduce the impact on the data circulation based on federated learning.

[0113] Embodiment 4

[0114] Figure 4 FIG. is a schematic flowchart of a method for scaling in and out in data product circulation provided by Embodiment 4 of the present invention. Embodiment 4 is optimized based on the above embodiments. For the content not detailed in this embodiment, please refer to Embodiment 3.

[0115] As Figure 4 shown, a method for scaling in and out in data product circulation provided by Embodiment 4 of the present invention includes the following steps:

[0116] S410. Receive a federated learning data product registration request initiated by a participant who is the initiator of the federated learning task. The federated learning data product registration request includes the configuration information of the federated learning data product.

[0117] In this embodiment, after the participant who is the initiator of the federated learning task completes the configuration information of the federated learning data product locally, it can send a federated learning data product registration request including the configuration information to the data trading platform.

[0118] There is no specific limitation on the manner in which the data trading platform receives the federated learning data product registration request here.

[0119] S420. Manage the federated learning data product and then send the federated learning data product release information to each participant.

[0120] In this embodiment, after receiving the federated learning data product registration request, the data trading platform can perform management such as registration, auditing, and release on the configuration information included in the federated learning data product registration request.

[0121] It should be noted that the information on the release of federated learning data products sent by the data trading platform to the participating parties who are the initiators of the federated learning tasks and the participating parties who are not the initiators of the federated learning tasks can be different. Since the participating parties who are the initiators of the federated learning tasks are the configurators of the federated learning data products, the data trading platform only needs to notify the initiators of the federated learning tasks that the federated learning data products have been released. And the information on the release of the federated learning data products sent by the data trading platform to the participating parties who are not the initiators of the federated learning tasks needs to include the basic information of the federated learning data products for the participating parties who are not the initiators of the federated learning tasks to understand the federated learning data products. Among them, the participating parties who are not the initiators of the federated learning tasks can include the data consumers specified by the initiators of the federated learning tasks, and can also include the data consumers who send subscription messages for the basic information of the federated learning data products to the data trading platform.

[0122] S430. Based on the participant management policy in the configuration information of the federated learning product, review whether the new participants who send data trading voucher applications have the participation permission.

[0123] In this embodiment, for the federated data product learning scenario, the review of the participants can include reviewing the participant permissions, reviewing whether the participants' data meets the data field requirements, and initiating a federated data quality assessment. This step is used to query the participant management policy in the configuration information of the federated learning product. Based on the configured participant management policy, review the new participants who send data trading voucher applications to determine whether the new participants have the participation permission. If the new participants do not have the participation permission, the data trading platform rejects the request of the participants to apply for trading vouchers.

[0124] Among them, there are two ways of the participant management policy, including the participant static management policy and the participant dynamic management policy. The participant static management policy includes static management policy one and static management policy two. Participant static management policy one: From the release of the federated learning data product to the end of the task, only the agreed participants are allowed to participate throughout the process, and the participant information and the number of parameter parties remain fixed. Participant static management policy two: From the release of the federated learning data product to the end of the task, the number of parameter parties remains fixed throughout the process. Before the formal start of the data circulation task, all participants who meet the data requirements are allowed to apply for data trading vouchers. Once started, no new participants are allowed to be added or existing participants are allowed to withdraw.

[0125] Among them, the participant dynamic management strategy means that during the training process of the federated learning model, within the range of the maximum and minimum number of participants, participants can dynamically join or withdraw. Considering that the types of federated learning include horizontal federated learning and vertical federated learning, for horizontal federated learning, according to the participant dynamic management strategy, participants can dynamically join or withdraw. For vertical federated learning, since vertical federated learning requires dividing the computing logic of vertical federated learning according to algorithm attributes, participant attributes, and data attributes. Whether a participant is added or decreased, the computing logic needs to be re-partitioned, which also affects the allocation scheduling and execution of the computing logic. Therefore, once the participants change, the vertical federated data product needs to interrupt the original business and restart the model training with the newly partitioned computing logic as the new federated learning data product.

[0126] S440. If the participation permission is available, verify whether the local data of the newly added participant meets the participant data information in the configuration information of the federated learning product.

[0127] Specifically, if the newly added participant has the participation permission, the data trading platform verifies whether the local data of the newly added participant meets the data field requirements of the federated learning data product. In the scenario of horizontal federated learning, the local data fields of all participants must be exactly the same as the defined data fields; in the scenario of vertical federated learning, the local data fields of the participant need to be the same as the data fields allocated to this participant in the federated learning data product. If the participant does not meet the data requirements defined by the data product, the request for a trading certificate from this participant is rejected.

[0128] S450. If the verification is passed, initiate a federated data quality assessment request to the newly added participant, so that after the newly added participant who receives the federated data quality assessment request completes the federated data quality assessment locally, the generated federated data quality assessment report is sent to the data trading platform.

[0129] Specifically, if the local data of the newly added participant meets the participant data information in the configuration information of the federated learning product, the data trading platform initiates a federated data quality assessment to the newly added participant. After the newly added participant completes the data quality assessment, the assessment report is sent to the data trading platform. The data trading platform verifies the data quality reports of all participants. If the data quality report of a participant does not meet the requirements, the request for a trading certificate from this participant is rejected.

[0130] It should be noted that this step can be an optional item. When the configuration information is the static management strategy, the federated data quality assessment verification for the verification of newly added participants can be skipped. For the dynamic management strategy, however, this step needs to be executed for the federated data quality assessment verification.

[0131] S460. Receive the federated data quality assessment report sent by the newly added participant, and review the federated data quality assessment report.

[0132] Specifically, the data trading platform receives the federated data quality assessment report sent by the newly added participant, and reviews the federated data quality assessment report of the newly added participant.

[0133] S470. Issue a data trading certificate and a federated learning task instruction to join the federated learning data product to the newly added participant that has passed the review, so that the corresponding newly added participant can participate in the federated learning task to obtain the output result of the federated learning scenario.

[0134] Specifically, if the data quality report of the newly added participant does not meet the requirements, the request for a data trading certificate from the newly added participant is rejected. The data trading platform issues a data trading certificate and a federated learning task instruction to the newly added participant that has passed the review. When the data trading platform issues a data trading certificate to the newly added participant that has passed the review, it also needs to update the participant information locally, that is, update the participant information of the federated learning data product.

[0135] S480. Send the output result to the target participant.

[0136] Among them, according to the federated learning output result attribution policy in the configuration information, it can be determined which participants the output result belongs to. In this embodiment, it is recorded as the target participant. The federated learning output result attribution policy can be that all participants have it or the data requester has it.

[0137] The technical solution of the embodiment of the present invention, through the management of the application for a data trading certificate, the newly added participant needs to have a data trading certificate to apply to participate in the federated learning task. It specifies which reviews need to be carried out on the newly added participant to issue a data trading certificate to it, realizes the effect of the newly added participant in data circulation, and realizes the expansion in data circulation.

[0138] As an optional embodiment of the embodiment of the present invention, on the basis of the above embodiment, the step of reviewing whether the newly added participant who sends the data trading certificate application has the participation permission based on the participant management policy in the configuration information of the federated learning product may specifically include:

[0139] a1. Query the participant management policy in the configuration information of the federated learning product.

[0140] In this alternative embodiment, it specifically describes how to review newly added participants according to the participant management strategy in the configuration information of the federated learning product. It can be understood that when defining the logical model of the federated learning data product, the participant management strategy has been set. The participant management strategy includes the first static management strategy, the second static management strategy, the horizontal federated dynamic management strategy for horizontal federated learning, and the vertical federated dynamic management strategy for vertical federated learning. The following steps will specifically describe the definition of each participant management strategy. For a single federated learning task, only one participant management strategy can be set for it.

[0141] b1. If the participant management strategy is the first static management strategy, then review whether the newly added participant has the participation permission according to the first static discrimination condition.

[0142] In this step, the first static management strategy is also denoted as the first participant static management strategy. This strategy requires that during the entire process from the release of the federated learning data product to the end of the task, only the agreed participants are allowed to participate, and the participant information and the number of parameter parties remain fixed. Among them, the first static discrimination condition is determined by the first static management strategy, and specifically may include whether the status information of the federated learning data product is running, whether the information of the newly added participant conforms to the participant information management defined in the configuration information of the federated learning product, and whether the number of participants is within the range of the number of participants after adding the newly added participant.

[0143] It is clear that when defining the logical model of the federated learning data product, the fixed participant information is set, the number of participants is determined, and the participant management strategy is set as the first participant static management strategy, and then the federated learning data product is released and subscribed.

[0144] Specifically, when a newly added participant initiates a data trading voucher application to join the federated learning task on the data trading platform, if it is queried that the participant management strategy in the configuration information of the federated learning product is the first static management strategy, then review whether the newly added participant has the participation permission according to the first static discrimination condition.

[0145] Furthermore, reviewing whether the newly added participant has the participation permission according to the first static discrimination condition includes:

[0146] b11. Obtain the current status information of the federated learning data product.

[0147] Specifically, the data trading platform reads the current status information of the federated learning data product. The product status information may be in the process of submission, registered, released, running, or ended, etc.

[0148] b12. If the status information of the federated learning data product is running, then determine that the newly added participant does not have the participation permission.

[0149] Specifically, if the status information of the federated learning data product is "running", since the first static management policy requires that once the data circulation task is started, no new participants are allowed to be added or existing participants are allowed to withdraw, it is determined that the newly added participant does not have the permission to participate, and the data trading platform rejects the request of the newly added participant for a trading certificate.

[0150] b13. Otherwise, obtain the participant information and the range of the number of participants defined in the configuration information of the federated learning product.

[0151] Specifically, if the status information of the federated learning data product is not "running", the data trading platform reads the participant information and the range of the number of participants specified in the logical model of the federated learning data product.

[0152] b14. If the information of the newly added participant is consistent with the participant information management defined in the configuration information of the federated learning product and the number of participants does not exceed the range of the number of participants after adding the newly added participant, it is determined that the newly added participant has the permission to participate.

[0153] Specifically, the data trading platform checks whether the newly added participant is consistent with the participant information defined in the logical model. If not, the request of the data consumer for a trading certificate is rejected. If so, the data trading platform checks whether adding a new participant exceeds the range of the number of participants constrained by the logical model. If it exceeds, the request of the data consumer for a trading certificate is rejected. If both conditions are met, it is determined that the participant has the permission to participate.

[0154] b15. Otherwise, it is determined that the newly added participant does not have the permission to participate.

[0155] If any item in step b14 is not satisfied, it is determined that the newly added participant does not have the permission to participate.

[0156] c1. If the participant management policy is the second static management policy, verify whether the newly added participant has the permission to participate according to the second static discrimination condition.

[0157] In this step, the second static management policy is also denoted as the participant static management policy two. This policy requires that the number of parameter parties remains fixed throughout the process from the release of the federated learning data product to the end of the task. Before the data circulation task is officially started, any participant meeting the data requirements is allowed to apply for a data trading certificate. Once started, no new participants are allowed to be added or existing participants are allowed to withdraw. Among them, the second static discrimination condition is determined by the second static management policy, and specifically may include whether the status information of the federated learning data product is "running" and whether the number of participants is within the range of the number of participants after adding the newly added participant.

[0158] It is clear that when defining the logical model of the federated learning data product, the participant management policy is set to the second static participant management policy, and then the federated learning data product is published and subscribed.

[0159] Specifically, when a new participant initiates a data trading voucher application to join the federated learning task on the data trading platform, if the participant management policy in the configuration information of the federated learning product is the second static management policy, the new participant's eligibility to participate is reviewed according to the second static discrimination condition.

[0160] Furthermore, reviewing whether the new participant has the right to participate according to the second static discrimination condition includes:

[0161] c11. Obtain the current status information of the federated learning data product.

[0162] Specifically, the data trading platform reads the current status information of the federated learning data product. The product status information may be in the process of submission, registered, published, running, or ended, etc.

[0163] c12. If the status information of the federated learning data product is "running", it is determined that the new participant does not have the right to participate.

[0164] Specifically, if the status information of the federated learning data product is "running", since the second static management policy requires that once the data circulation task is started, no new participants are allowed to be added or existing participants are allowed to withdraw, it is determined that the new participant does not have the right to participate, and the data trading platform rejects the new participant's request for a trading voucher.

[0165] c13. Otherwise, obtain the range of the number of participants restricted in the configuration information of the federated learning product.

[0166] Specifically, if the status information of the federated learning data product is not "running", the data trading platform reads the range of the number of participants specified in the logical model of the federated learning data product.

[0167] c14. If the number of participants does not exceed the range after adding the new participant, it is determined that the new participant has the right to participate.

[0168] Specifically, the data trading platform checks whether adding a new participant exceeds the range of the number of participants restricted by the logical model. If it exceeds, the request for a trading voucher from the data consumer is rejected. If all conditions are met, it is determined that the participant has the right to participate.

[0169] c15. Otherwise, it is determined that the new participant does not have the right to participate.

[0170] If the condition in step c14 is not met, it is determined that the new participant does not have the right to participate.

[0171] d1. If the participant management strategy is a horizontal federated dynamic management strategy, the newly added participant's participation permission is verified according to the first dynamic discrimination condition.

[0172] Among them, the first static discrimination condition is determined by the first static management strategy, the second static discrimination condition is determined by the second static management strategy, and the first dynamic discrimination condition is determined by the horizontal federated dynamic management strategy.

[0173] Among them, the dynamic management strategy means that during the training process of the federated learning model, within the range of the maximum and minimum number of participants, participants can dynamically join or withdraw. Considering that the current federated learning scenario may be a horizontal federated learning scenario or a vertical federated learning scenario. In the horizontal federated learning scenario, all data fields of the participants must be the same. Any participant who wants to join the horizontal federated learning needs to ensure that the local data meets the requirements of the data fields in the federated learning data product logic model. In the vertical federated learning scenario, the data fields of each participant are different, and only one party is allowed to hold the label field. Therefore, in the vertical federated learning data product, it is also necessary to determine the allocation scheme of all participants' data fields according to the local data situation of the participants. For the sake of distinction, in this embodiment, the dynamic management strategy corresponding to the horizontal federated learning scenario is denoted as the horizontal federated dynamic management strategy, and the dynamic management strategy corresponding to the vertical federated learning scenario is denoted as the vertical federated dynamic management strategy.

[0174] It should be noted that the federated learning data product provided in this embodiment not only supports the participant static management strategy, but also supports the dynamic scaling of the horizontal federated learning data product and the scaling processing of the vertical federated learning data product. That is: during the federated learning process, the number of participants can change, and participants can choose to join or withdraw. In the horizontal federated learning scenario, for a participant, as long as the data format and data quality of the participant meet the requirements of the data product for the input data, the participant can be allowed to dynamically participate in the federated learning model learning.

[0175] Specifically, when a newly added participant initiates a data transaction voucher application to the data trading platform to join the federated learning task, if the current learning scenario is a horizontal federated learning scenario and the participant management strategy in the configuration information of the federated learning product is the first dynamic management strategy, the newly added participant's participation permission is verified according to the first dynamic discrimination condition.

[0176] Further, verifying whether the newly added participant has the participation permission according to the first dynamic discrimination condition includes:

[0177] d11. Obtain the participant information management and the participant quantity range defined in the configuration information of the federated learning product.

[0178] Specifically, the data trading platform reads the participant information management and the range of the number of participants defined in the configuration information of the federated learning product.

[0179] d12. If the information of the newly added participant conforms to the participant information management defined in the configuration information of the federated learning product and the newly added participant does not exceed the range of the number of participants, it is determined that the newly added participant has the participation permission.

[0180] Specifically, check whether the newly added data consumer is a participant defined by the data product. And check whether adding a new participant is within the range of the number of participants restricted by the data product. The federated learning data product has constraints on the maximum and minimum number of participants. Once the number of participants exceeds the threshold, no participant is allowed to join or withdraw. If both are satisfied, it is determined that the newly added participant has the participation permission.

[0181] d13. Otherwise, it is determined that the newly added participant does not have the participation permission.

[0182] Specifically, if one of the above steps d12 is not satisfied, it is determined that the newly added participant does not have the participation permission.

[0183] Exemplarily, Figure 4a FIG. is a schematic diagram of the data circulation process for dynamic inclusion of data products in a horizontal federated learning scenario provided in Embodiment 4 of the present invention. As Figure 4a shown, the process of dynamic inclusion of data products in a horizontal federated learning scenario can be described as follows:

[0184] Step 1: During the learning process of the horizontal federated learning model, a new data consumer sends a data trading voucher application request to the data trading platform to join the learning of the horizontal federated learning model.

[0185] Step 2: Manage the data trading vouchers of the newly added data consumer and evaluate the data quality.

[0186] Step 201: The data trading platform queries the participant management policy of the federated learning data product. Only for the dynamic policy, the addition or withdrawal of participants is allowed. This step conducts conditional review on the newly added data demand side.

[0187] Condition 1: Confirm whether the participant management policy is a dynamic management policy.

[0188] Condition 2: Check whether the newly added data consumer is a participant defined by the data product.

[0189] Condition 3: Check whether adding a new participant is within the range of the number of participants restricted by the data product. The federated learning data product has constraints on the maximum and minimum number of participants. Once the number of participants exceeds the threshold, no participant is allowed to join or withdraw.

[0190] Condition 4: Check whether the local data of the new data consumer meets all the data field requirements defined in the logical model.

[0191] If at least one of the above 4 conditions is not met, the application for the data transaction voucher of the new data requester will be rejected.

[0192] Step 202: The data trading platform initiates a local data quality assessment request to the data consumer; the new data consumer conducts a quality assessment on the local data and sends the data quality assessment report to the data trading platform. The data trading platform reviews the data quality assessment report of the new data requester. If it does not meet the data quality requirements of federated learning, the application for the data transaction voucher of the new data requester will be rejected.

[0193] Step 203: After the new data consumer passes the review and authentication, the data trading platform sends a transaction voucher to the new data consumer and synchronously updates the participant information in the federated learning data product.

[0194] Furthermore, if the participant management strategy is a horizontal federated dynamic management strategy, after issuing the data transaction voucher to the newly added approved participant, it also includes:

[0195] Send a notice of change in the horizontal federated calculation logic to the federated learning coordinator, so that the federated learning coordinator suspends the current model aggregation operation, sends the encrypted aggregation model of the current iteration round to the newly added participant, and modifies the horizontal federated calculation logic based on the gradient information sent by the newly added participant.

[0196] Continue to refer to Figure 4a , specifically, the data trading platform sends a notice of change in the calculation logic to the federated learning coordinator; after receiving the notice of change in the calculation logic, the federated learning coordinator suspends the existing model aggregation operation; the federated learning coordinator sends the encrypted aggregation model of the current iteration round to the newly added data requester; the newly added participant completes one local iteration training based on the received aggregation model and encrypts and transmits the gradient to the federated learning coordinator according to the normal horizontal federated learning process; after receiving the gradient information sent by the newly added data requester, the federated learning coordinator initiates the modification of the horizontal federated learning calculation logic, adds the gradient data of the newly added party when calculating the gradient mean, and adds the newly added participant node in the processes such as calculation logic distribution and scheduling; continue according to the normal horizontal federated learning processing process.

[0197] e1. If the participant management strategy is a vertical federated dynamic management strategy, interrupt the current federated learning task and restart the federated learning task with the newly segmented calculation logic as the new federated learning data product.

[0198] Due to vertical federated learning, it is necessary to divide the computing logic of vertical federated learning according to algorithm attributes, participant attributes, and data attributes. Whether new participants are added or existing participants are reduced, the computing logic needs to be re-segmented, which also affects the allocation scheduling and execution of the computing logic. Therefore, once the participants change, the vertical federated data product needs to interrupt the original business and restart the model training with the newly segmented computing logic as the new federated learning data product.

[0199] To shorten the system time consumption for scaling, in this embodiment, a data product template technology can be introduced, specifically: directly reference the basic information of the previous product in the data product configuration, and only the changed information needs to be modified. Further, in the processes of data trading voucher application and data quality assessment, for the participants whose computing logic and data have not changed, directly reuse the data trading vouchers and data quality assessment report results of the previous federated learning data product. Thus, the business interruption duration can be minimized to the greatest extent.

[0200] As an alternative embodiment of the present invention, based on the above embodiment, the method further includes: when receiving an application from an existing participant in the federated learning task to withdraw from the federated learning task, auditing the existing participant based on the participant management strategy of the federated learning data product, and issuing an instruction to withdraw from the federated learning task to the existing participant who passes the audit.

[0201] In this alternative embodiment, it mainly describes the scenario where an existing participant in the federated learning task wants to withdraw from the federated learning task. As an existing participant in the federated learning task, when wanting to withdraw from the federated learning task, it needs to send an application to withdraw from the federated learning task to the data trading platform. The data trading platform needs to audit the existing participant based on the configuration information of the federated learning data product.

[0202] As described above, when the data trading platform audits a new participant based on the configuration information of the federated learning data product, it needs to audit the existing participant according to the current participant management strategy to determine whether the new participant meets the relevant settings in the participant management strategy. If it meets the requirements, that is, the audit is passed, an instruction to withdraw from the federated learning task is issued to the existing participant. When the existing participant receives the instruction to withdraw from the federated learning task issued by the data trading platform, it can withdraw from the federated learning task. It should be clear that only with the participant dynamic management strategy can existing participants be allowed to withdraw.

[0203] Further, auditing the existing participant based on the participant management strategy of the federated learning data product includes:

[0204] a2. Query the participant management strategy of the federated learning data product.

[0205] Specifically, the data trading platform reads the participant management policy in the configuration information of the federated learning data product.

[0206] b2. If the participant management policy is the first static management policy or the second static management policy, it is determined that the existing participants do not have the permission to withdraw.

[0207] Specifically, if the participant management policy is a static management policy, it is determined that the existing participants do not have the permission to withdraw, and the application of the existing participants to withdraw from the federated learning task is rejected.

[0208] c2. If the participant management policy is a horizontal federated dynamic management policy, it is judged whether the number of participants is within the range after reducing the existing participants.

[0209] Specifically, if the current is a horizontal federated learning scenario and it is a dynamic management policy, it is necessary to judge whether reducing one participant is within the number of participants restricted by the data product.

[0210] d2. If so, it is determined that the existing participants have the permission to withdraw; otherwise, it is determined that the existing participants do not have the permission to withdraw.

[0211] Specifically, if reducing one participant is within the number of participants restricted by the data product, it is determined that the existing participants have the permission to withdraw; otherwise, the existing participants do not have the permission to withdraw.

[0212] Exemplarily, Figure 4b FIG. is a schematic diagram of the data circulation process for dynamic scaling down of a data product in a horizontal federated learning scenario provided in Embodiment 4 of the present invention. As Figure 4b shown, the process of dynamic scaling down of a data product in a horizontal federated learning scenario can be described as follows:

[0213] Step 1: During the learning process of the horizontal federated learning model, the participant sends a request to withdraw from the federated learning to the data trading platform.

[0214] Step 2: Participant withdrawal management.

[0215] Step 201: The data trading platform queries the participant management policy of the federated learning data product to confirm whether the participant management policy is a dynamic management policy. Only for the dynamic policy, the participant is allowed to withdraw. At the same time, the data trading platform checks whether reducing one participant is within the number of participants restricted by the data product. If at least one of the above two conditions is not met, the participant's withdrawal application is rejected.

[0216] Step 202: The data trading platform sends a message indicating that the withdrawal application from the federated learning has been approved to the participant, and synchronously updates the participant information in the federated learning data product.

[0217] It should be noted here that during the learning process of the horizontal federated learning model, if a participant withdraws, the federated learning calculation logic also needs to be changed. The specific change steps are as follows.

[0218] Step 3: Change the federated learning calculation logic.

[0219] Step 301: The data trading platform sends a calculation logic change notice to the federated learning coordinator;

[0220] Step 302: The federated learning coordinator starts to modify the horizontal federated learning calculation logic, removes the gradient data of the withdrawing participant when calculating the gradient mean, and removes the withdrawing participant node in the processes such as calculation logic distribution and scheduling.

[0221] Step 303: Continue according to the normal horizontal federated learning processing flow.

[0222] e2. If the participant management policy is the vertical federated dynamic management policy, interrupt the original business and use the newly segmented calculation logic as the new federated learning data product to restart the federated learning task.

[0223] In vertical federated learning, the calculation logic of vertical federated learning needs to be divided according to algorithm attributes, participant attributes, and data attributes. Whether a participant is added or decreased, the calculation logic needs to be re-segmented, which also affects the allocation, scheduling, and execution of the calculation logic. Therefore, once the participants change, the vertical federated data product needs to interrupt the original business and use the newly segmented calculation logic as the new federated learning data product to restart model training.

[0224] To shorten the system time consumption of scaling, the data product template technology can be introduced in this embodiment. Specifically: directly reference the basic information of the previous product in the data product configuration, and only need to modify the changed information. Further, in the processes of data trading voucher application and data quality assessment, for the participants whose calculation logic and data have not changed, directly reuse the data trading vouchers and data quality assessment report results of the previous federated learning data product. Thus, the business interruption duration can be minimized to the greatest extent.

[0225] This optional embodiment supports not only the participant static management policy but also the dynamic scaling management policy, based on the participant joining / withdrawing policies of horizontal federated learning and vertical federated learning. In specific implementation, in addition to the innovative method of dynamic scaling, to realize the dynamic joining or withdrawing of participants, it also supports control policies such as participant number constraints to minimize the impact on normal business.

[0226] Embodiment 5

[0227] Figure 5The figure is a schematic structural diagram of a data product circulation system provided in Embodiment 5 of the present invention. As Figure 5 shown, the system includes a data trading platform 120 and participants 110 involved in data circulation. The participants 110 involved in data circulation include participants 111 as the initiator of the federated learning task, new participants 112, and existing participants 113; among them,

[0228] The participant 111 as the initiator of the federated learning task is used to complete the configuration information of the federated learning data product locally and send a registration request for the federated learning data product to the data trading platform 120, and the registration request for the federated learning data product includes the configuration information of the federated learning data product;

[0229] The data trading platform 120 is used to manage the federated learning data product and then send the release information of the federated learning data product to each participant;

[0230] The new participant 112 as the federated learning task is used to apply for a data trading voucher from the data trading platform after receiving the release information of the federated learning data product sent by the data trading platform;

[0231] The data trading platform 120 is used to review the new participant based on the configuration information of the federated learning data product, and issue a data trading voucher and an instruction to join the federated learning task to the new participant that passes the review;

[0232] The new participant 112 is used to participate in the federated learning task to obtain the output result of the federated learning scenario if it receives the instruction to join the federated learning task issued by the data trading platform;

[0233] The data trading platform 120 is used to send the output result to the target participant;

[0234] The existing participant 113 as the federated learning task is also used to send an application to withdraw from the federated learning task to the data trading platform during the operation of the federated learning task;

[0235] The data trading platform 120 is also used to review the existing participant based on the configuration information of the federated learning data product, and issue an instruction to withdraw from the federated learning task to the existing participant that passes the review;

[0236] The existing participant 113 is also used to withdraw from the federated learning task if it receives the instruction to withdraw from the federated learning task issued by the data trading platform.

[0237] For the parts not described in this embodiment, reference can be made to the content in the method embodiment, and no repeated description will be made here.

[0238] The data product circulation system provided by the embodiments of the present invention can circulate the product safely and compliantly while realizing the sharing of federated learning data products, avoiding data leakage, and improving the data security and privacy of each participating party.

[0239] Based on the technical solutions of the above embodiments, the embodiments of the present invention provide several specific implementation manners.

[0240] Figure 6 It is a schematic diagram of the data circulation process of a federated learning data product learning scenario provided by an exemplary embodiment of the present invention. As Figure 6 shown, it includes the following steps:

[0241] Step 1: Publication of federated learning data products.

[0242] Step 101: The data provider, i.e., the participating party that is the initiator of the federated learning task, completes the definition of the federated learning data product locally, i.e., completes the configuration information of the federated learning data product locally, and sends a registration request for the federated learning data product to the data trading platform.

[0243] Step 102: The data trading platform is responsible for the management of the federated learning data product. After receiving the registration request for the federated learning data product submitted by the data provider, it completes the storage of the federated learning data product and sets the status to: registered; after registration, after being reviewed by the data trading platform, it is published, and a subscription message for the basic information of the federated learning data product is sent to the data product agreed data consumer, i.e., the participating party that is not the initiator of the federated learning task, and the product status is updated to: published; the data trading platform sends a successful message for the publication of the federated learning data product to the data provider to complete the entire data product publication. Among them, the subscription message can be sent in either an active push manner or a subscription request manner.

[0244] Step 2: Management of data trading vouchers and data quality assessment.

[0245] Step 201: The participating party applies to the data trading platform for a data trading voucher. Among them, after receiving the successful message for the publication of the federated learning data product, the data provider will immediately start the process of applying for a trading voucher and automatically send a trading voucher application to the data trading platform; while after receiving the subscription message for the federated learning data product, the data consumer needs to start the process of applying for a trading voucher, and the triggering condition can be set according to the specific business process, and a data trading voucher application is sent to the data trading platform.

[0246] Step 202: The data trading platform queries the management strategy of the participating parties of the federated learning data product. If it is the static management strategy one, step 203 is executed; if it is the static management strategy two or the dynamic management strategy, step 204 is executed.

[0247] Step 203: Under the condition that the participant management policy is the static management policy 1, the data trading platform audits the permissions of the participants.

[0248] Condition 1: Check whether the newly added data consumer is a participant defined for the data product.

[0249] Condition 2: Check whether the current number of participants is within the range of the number of participants restricted by the data product. The federated learning data product has maximum and minimum constraints on the number of participants. Once the number of participants exceeds the threshold, participants are not allowed to join or withdraw.

[0250] If at least one of the above two conditions is not met, the data trading platform rejects the request of the participant to apply for a trading certificate.

[0251] Step 204: Under the condition that the participant management policy is the static management policy 2 or the dynamic management policy, the data trading platform audits the permissions of the participants. Check whether the current number of participants is within the range of the number of participants restricted by the data product. If it exceeds, the request of the participant to apply for a trading certificate is rejected.

[0252] Step 205: The data trading platform audits whether the local data of the participants meets the data field requirements of the federated learning data product. In the horizontal federated learning scenario, the local data fields of all participants must be exactly the same as the defined data fields; in the vertical federated learning scenario, the local data fields of the participants need to be consistent with the data fields allocated to the participants in the federated learning data product. If the participants do not meet the data requirements defined by the data product, the request of the participants to apply for a trading certificate is rejected.

[0253] Step 206: The data trading platform initiates a federated data quality assessment to all participants. After all participants jointly complete the data quality assessment, they send the assessment report to the data trading platform. The data trading platform audits the data quality reports of all participants. If the data quality report of a participant does not meet the requirements, the request of the participant to apply for a trading certificate is rejected.

[0254] Step 207: The data trading platform issues data trading certificates to all participants and updates the participant information of the federated learning data product at the same time.

[0255] Step 3: The data circulation and model attribution process based on federated learning.

[0256] The data trading platform sends a message to start the federated learning model learning task to all participants and updates the status of the federated learning data product to: running. All participants complete the federated learning data product model learning task, and finally, according to the federated learning result attribution policy, the final federated learning output result is output to the participants.

[0257] Step 4: The data trading platform sends a message indicating the end of data circulation to the data provider, and the data trading platform updates the status of the federated learning data product to: Ended.

[0258] Embodiment Six

[0259] Figure 7 FIG. shows a schematic structural diagram of a computing node 10 that can be used to implement an embodiment of the present invention. The computing node is intended to represent various forms of digital computers, such as, for example, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The computing node can also represent various forms of mobile devices, such as, for example, personal digital processors, cellular telephones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0260] As Figure 7 shown, the computing node 10 includes at least one processor 11, and a memory communicatively connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc. The memory stores a computer program executable by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the computing node 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. The input / output (I / O) interface 15 is also connected to the bus 14.

[0261] A plurality of components in the computing node 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the computing node 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0262] The processor 11 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the scaling method in data product circulation applied to the parties involved in data product circulation in the data product circulation system, and the scaling method in data product circulation applied to the data trading platform in the data product circulation system.

[0263] In some embodiments, the scaling method in data product circulation can be implemented as a computer program, which is tangibly included in a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the scaling method in data product circulation described above can be executed. Alternatively, in other embodiments, the processor 11 can be configured to execute the scaling method in data product circulation by any other suitable means (e.g., by means of firmware).

[0264] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special-purpose or general-purpose programmable processor, and can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0265] A computer program for implementing the method of the present invention may be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer programs are executed by the processor, the functions / operations specified in the flowchart and / or block diagram are implemented. The computer programs may be executed entirely on the machine, partially on the machine, as a stand-alone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0266] In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, the computer-readable storage medium may be a machine-readable signal medium. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0267] To provide interaction with a user, the systems and techniques described herein may be implemented on a computing node that has: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computing node. Other kinds of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, speech input, or tactile input).

[0268] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected to each other by digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), blockchain network, and the Internet.

[0269] A computing system can include a client and a server. The client and the server are generally remote from each other and typically interact via a communication network. The client-server relationship is created by computer programs running on respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system, and solves the defects of difficult management and weak business scalability existing in traditional physical hosts and VPS services.

[0270] It should be understood that various forms of the processes shown above can be used, steps can be reordered, added, or deleted. For example, the steps recited in the present invention can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved, and no limitation is made herein.

[0271] The above specific embodiments do not constitute a limitation on the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for scaling in and out in data product circulation, characterized in that, it is applied to the participants in the data product circulation system for data product circulation, and the method includes: As a new participant in the federated learning task of the federated learning data product, after receiving the federated learning data product release information sent by the data trading platform, apply to the data trading platform for a data trading certificate, so that the data trading platform audits the new participant based on the configuration information of the federated learning data product, and issues a data trading certificate and an instruction to join the federated learning task to the new participant who passes the audit; If receiving the instruction to join the federated learning task issued by the data trading platform, then participate in the federated learning task to obtain the output result of the federated learning scenario, so that the data trading platform sends the output result to the target participant.

2. The method according to claim 1, characterized in that, before receiving the federated learning data product release information sent by the data trading platform, it further includes: As a participant who is the initiator of the federated learning task, complete the configuration information of the federated learning data product locally; Initiate a registration request for the federated learning data product to the data trading platform, so that the data trading platform manages the federated learning data product and sends the federated learning data product release information to each participant; wherein, the registration request for the federated learning data product includes the configuration information of the federated learning data product.

3. The method according to claim 1, characterized in that, the federated learning task includes a federated learning model learning task, and the federated learning scenario includes a federated data product learning scenario; correspondingly, participating in the federated learning task of the federated learning data product to obtain the output result of the federated learning scenario includes: Participate in the learning process of the federated learning model to obtain the final federated learning model; Use the final federated learning model as the output result of the federated data product.

4. The method according to claim 1, characterized in that, it further includes: As an existing participant in the federated learning task, send an application to withdraw from the federated learning task to the data trading platform during the operation of the federated learning task, so that the data trading platform audits the existing participant based on the configuration information of the federated learning data product, and issues an instruction to withdraw from the federated learning task to the existing participant who passes the audit; If receiving the instruction to withdraw from the federated learning task issued by the data trading platform, then withdraw from the federated learning task.

5. A method for scaling in and out in data product circulation, characterized in that, it is applied to the data trading platform in the data product circulation system, and the method includes: After receiving the data trading certificate application sent by the new participant, audit the new participant based on the configuration information of the federated learning data product; Issue a data trading certificate and an instruction to join the federated learning task of the federated learning data product to the new participant who passes the audit, so that the corresponding new participant participates in the federated learning task to obtain the output result of the federated learning scenario; Send the output result to the target participant.

6. The method according to claim 5, It is characterized in that before receiving a data transaction voucher application sent by a newly added participant, it further includes: receiving a federated learning data product registration request initiated by a participant who is the initiator of the federated learning task, where the federated learning data product registration request includes configuration information of the federated learning data product; after managing the federated learning data product, sending federated learning data product release information to each participant.

7. The method according to claim 5, It is characterized in that the configuration information of the federated learning data product includes basic information of the federated learning data product and change information of the federated learning data product.

8. The method according to claim 7, It is characterized in that the basic information of the federated learning data product includes one or more of the following: basic information of the federated learning data product, participant data information, federated learning task information, and federated learning data product status information; wherein, the basic information of the federated learning data product includes product name, product introduction, and product adaptation scenarios; the participant data information includes data source description and data field description; the federated learning task information includes algorithm and parameter information, federated learning scenarios, and federated learning types; the federated learning data product status information includes in submission, registered, released, in operation, or ended.

9. The method according to claim 7, It is characterized in that the change information of the federated learning data product includes one or more of the following: federated learning output result attribution policy, participant management policy, and security policy; wherein, the participant management policy includes requirements for the fixed number of participants, the range of the number of participants, participant information management, static participant management policy, and dynamic participant management policy; the security policy includes data transaction voucher application and authorization policy, participant authentication policy, and encryption protocol.

10. The method according to claim 6, It is characterized in that after receiving the data transaction voucher application sent by the newly added participant, auditing the newly added participant based on the configuration information of the federated learning data product includes: auditing whether the newly added participant who sends the data transaction voucher application has the participation permission based on the participant management policy in the configuration information of the federated learning product; if having the participation permission, auditing whether the local data of the newly added participant meets the participant data information in the configuration information of the federated learning product; if the audit is passed, initiating a federated data quality assessment request to the newly added participant, so that after the newly added participant who receives the federated data quality assessment request completes the federated data quality assessment locally, sending the generated federated data quality assessment report to the data trading platform; receiving the federated data quality assessment report sent by the newly added participant and auditing the federated data quality assessment report.

11. The method according to claim 10, It is characterized in that auditing whether the newly added participant who sends the data transaction voucher application has the participation permission based on the participant management policy in the configuration information of the federated learning product includes: querying the participant management policy in the configuration information of the federated learning product; If the participant management policy is the first static management policy, verify whether the newly added participant has the participation permission according to the first static discrimination condition; If the participant management policy is the second static management policy, verify whether the newly added participant has the participation permission according to the second static discrimination condition; If the participant management policy is the horizontal federated dynamic management policy, verify whether the newly added participant has the participation permission according to the first dynamic discrimination condition; If the participant management policy is the vertical federated dynamic management policy, interrupt the current federated learning task, and use the newly segmented calculation logic as the new federated learning data product to restart the federated learning task; Among them, the first static discrimination condition is determined by the first static management policy, the second static discrimination condition is determined by the second static management policy, and the first dynamic discrimination condition is determined by the horizontal federated dynamic management policy.

12. The method according to claim 11, wherein, verifying whether the newly added participant has the participation permission according to the first static discrimination condition includes: Obtain the current status information of the federated learning data product; If the status information of the federated learning data product is "running", it is determined that the newly added participant does not have the participation permission; Otherwise, obtain the participant information and the range of the number of participants defined in the configuration information of the federated learning product; If the information of the participant is consistent with the participant information management defined in the configuration information of the federated learning product and the number of participants does not exceed the range after adding the newly added participant, it is determined that the participant has the participation permission; Otherwise, it is determined that the newly added participant does not have the participation permission.

13. The method according to claim 11, wherein, verifying whether the newly added participant has the participation permission according to the second static discrimination condition includes: Obtain the current status information of the federated learning data product; If the status information of the federated learning data product is "running", it is determined that the participant does not have the participation permission; Otherwise, obtain the range of the number of participants restricted in the configuration information of the federated learning product; If the number of participants does not exceed the range after adding the newly added participant, it is determined that the participant has the participation permission; Otherwise, it is determined that the newly added participant does not have the participation permission.

14. The method according to claim 11, wherein, verifying whether the newly added participant has the participation permission according to the first dynamic discrimination condition includes: Obtain the participant information management and the range of the number of participants defined in the configuration information of the federated learning product; If the information of the newly added participant is consistent with the participant information management defined in the configuration information of the federated learning product and the newly added participant does not exceed the range of the number of participants, it is determined that the participant has the participation permission; Otherwise, it is determined that the participant does not have the participation permission.

15. The method according to claim 5, if the participant management policy is the horizontal federated dynamic management policy, after issuing a data trading certificate to the newly added participant who has passed the review, further includes: Send a notice of horizontal federated computing logic change to the federated learning coordinator to cause the federated learning coordinator to suspend the current model aggregation operation, send the encrypted aggregated model of the current iteration round to the newly added participant, and modify the horizontal federated computing logic based on the gradient information sent by the newly added participant.

16. The method according to claim 5, wherein, further comprising: When receiving an application from an existing participant in the federated learning task to withdraw from the federated learning task, audit the existing participant based on the participant management policy of the federated learning data product, and issue an instruction to withdraw from the federated learning task to the existing participant that passes the audit.

17. The method according to claim 16, wherein auditing the existing participant based on the participant management policy of the federated learning data product, comprises: Query the participant management policy of the federated learning data product; If the participant management policy is the first static management policy or the second static management policy, determine that the existing participant does not have the right to withdraw; If the participant management policy is the horizontal federated dynamic management policy, determine whether the number of participants after reducing the existing participant is within the range of the number of participants constrained in the configuration information of the federated learning product; If so, determine that the existing participant has the right to withdraw, otherwise, determine that the existing participant does not have the right to withdraw; If the participant management policy is the vertical federated dynamic management policy, interrupt the original business, and restart the federated learning task with the newly segmented computing logic as the new federated learning data product.

18. A data product circulation system, wherein, comprising: A data trading platform and participants for data circulation. The participants for data circulation include participants as the initiator of the federated learning task, newly added participants, and existing participants; wherein, The participant as the initiator of the federated learning task is used to complete the configuration information of the federated learning data product locally and send a registration request for the federated learning data product to the data trading platform, and the registration request for the federated learning data product includes the configuration information of the federated learning data product; The data trading platform is used to manage the federated learning data product and send the federated learning data product release information to each participant; The newly added participant as the federated learning task is used to apply for a data trading certificate from the data trading platform after receiving the federated learning data product release information sent by the data trading platform; The data trading platform is used to audit the newly added participant based on the configuration information of the federated learning data product, and issue a data trading certificate and an instruction to join the federated learning task to the newly added participant that passes the audit; The newly added participant is used to participate in the federated learning task to obtain the output result of the federated learning scenario if it receives the instruction to join the federated learning task issued by the data trading platform; The data trading platform is used to send the output result to the target participant; The existing participant as the federated learning task is used to send an application to withdraw from the federated learning task to the data trading platform during the operation of the federated learning task. The data trading platform is further configured to audit the existing participants based on the configuration information of the federated learning data product, and issue an instruction to withdraw from the federated learning task to the existing participants who pass the audit; The existing participants are further configured to withdraw from the federated learning task if they receive the instruction to join the federated learning task issued by the data trading platform.

19. A computing node, Characterized in that, The computing node includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method for scaling in and out in the circulation of data products according to any one of claims 1-4 or 5-17.

20. A computer-readable storage medium, Characterized in that, The computer-readable storage medium stores computer instructions, and the computer instructions are used to implement the method for scaling in and out in the circulation of data products according to any one of claims 1-4 or 5-17 when executed by a processor.

Images