Federal learning reasoning enhancement method and device, electronic equipment and storage medium

Abstract

The invention provides a federated learning reasoning enhancement method and device, electronic equipment and a storage medium, and the method comprises the steps: a reasoning client obtains a target feature of target task data, encrypts the target feature, and sends the encrypted target feature to each non-reasoning client; then determining a second distance between the target feature and data of each data category included in the reasoning client, and determining a second data volume corresponding to each data category, and determining a reasoning enhancement result of the target task data based on the first target distance, the second distance, the first data volume and the second data volume, and performing distance matching with each category of data included in each non-reasoning client, thereby giving play to the advantages of global data resources, and improving the reasoning enhancement efficiency. And meanwhile, the calculation cost during ciphertext matching of the client is reduced, effective utilization of global heterogeneous data is realized, and the efficiency and accuracy of federal learning reasoning enhancement are improved.

Description

Technical Field

[0001] The present application relates to the technical field of reasoning enhancement of federated learning, and in particular to a method, device, electronic device and storage medium for reasoning enhancement of federated learning. Background Art

[0002] As an emerging machine learning method, federated learning is rapidly promoting the development of artificial intelligence. Under the premise of protecting data privacy, it allows multiple distributed devices or institutions to collaboratively train models, thereby solving the problem of data security and privacy compliance. It has shown great potential in the fields of medicine and finance, and provides a safe and efficient collaborative training path. At present, the optimization of federated learning models is mainly concentrated in the training stage. However, for the federated learning reasoning stage, there is a lack of efficient and accurate federated learning reasoning enhancement methods. Summary of the invention

[0003] In view of this, the purpose of this application is to propose a reasoning enhancement method, device, electronic device and storage medium for federated learning to solve or partially solve the above-mentioned technical problems.

[0004] Based on the above purpose, the present application provides a method for enhancing reasoning in federated learning. The device is applied to a federated learning system, and the system includes multiple clients. The method includes:

[0005] The inference client obtains the target feature of the target task data, encrypts the target feature, and sends it to each non-inference client; wherein the inference client is any one of the multiple clients, and the non-inference client is a client other than the inference client among the multiple clients;

[0006] The non-inference client receives the encrypted target feature, determines the homomorphic encryption distance between each category of data included in the non-inference client and the encrypted target feature through homomorphic encryption technology, determines a first data volume of the data category corresponding to the homomorphic encryption distance, multiplies the homomorphic encryption distance by the first data volume to obtain a first target distance; and returns the first target distance and the encrypted first data volume to the inference client;

[0007] The inference client determines a second distance between the target feature and data of each data category included in the inference client, and determines a second data volume corresponding to each data category, and determines an inference enhancement result of the target task data based on the first target distance, the second distance, the first data volume and the second data volume.

[0008] Based on the same concept, the present application also proposes a federated learning reasoning enhancement device, which is applied to a federated learning system, wherein the system includes multiple clients; including:

[0009] The task sending module, the inference client obtains the target feature of the target task data, encrypts the target feature, and sends it to each non-inference client; wherein the inference client is any one of the multiple clients, and the non-inference client is a client among the multiple clients except the inference client;

[0010] The determination module, wherein the non-inference client receives the encrypted target feature, determines the homomorphic encryption distance between the data of each data category included in the non-inference client and the encrypted target feature through homomorphic encryption technology, and determines the first data volume of the data category corresponding to the homomorphic encryption distance, multiplies the homomorphic encryption distance by the first data volume to obtain a first target distance; and returns the first target distance and the encrypted first data volume to the inference client;

[0011] An inference module, wherein the inference client determines a second distance between the target feature and data of each data category included in the inference client, and determines a second data volume corresponding to each data category, and determines an inference enhancement result of the target task data based on the first target distance, the second distance, the first data volume and the second data volume.

[0012] Based on the same concept, the present application also proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the reasoning enhancement method of federated learning as described above is implemented.

[0013] Based on the same concept, the present application also proposes a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to execute the reasoning enhancement method of federated learning as described above.

[0014] Based on the same concept, the present application also proposes a computer program product, which includes a computer program, and the computer program is executed by one or more processors so that the processors execute the reasoning enhancement method of federated learning as described above.

[0015] From the above, it can be seen that the reasoning enhancement method, device, electronic device and storage medium for federated learning provided by the present application, the reasoning client obtains the target feature of the target task data, encrypts the target feature, and sends it to each non-reasoning client; wherein the reasoning client is any one of the multiple clients, and the non-reasoning client is a client among the multiple clients other than the reasoning client; the non-reasoning client receives the encrypted target feature, determines the homomorphic encryption distance between the data of each data category included in the non-reasoning client and the encrypted target feature through homomorphic encryption technology, and determines the first data amount of the data category corresponding to the homomorphic encryption distance, and compares the homomorphic encryption distance with the first data amount. The first target distance is obtained by multiplying the first data volume by the first data volume after encryption; and the first target distance and the encrypted first data volume are returned to the inference client; the inference client determines the second distance between the target feature and the data of each data category included in the inference client, and determines the second data volume corresponding to each data category, and determines the inference enhancement result of the target task data based on the first target distance, the second distance, the first data volume and the second data volume. By performing distance matching with the data of each data category included in each non-inference client, the advantages of global data resources are brought into play, and the computational cost of client ciphertext matching is reduced, thereby achieving effective utilization of global heterogeneous data and improving the efficiency and accuracy of federated learning inference enhancement. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present application or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 A flowchart of a method for enhancing reasoning in federated learning according to an embodiment of the present application;

[0018] Figure 2 A schematic diagram of the structure of a federated learning system according to an embodiment of the present application;

[0019] Figure 3 A schematic diagram of a process for updating synthetic data according to an embodiment of the present application;

[0020] Figure 4 A schematic diagram of the structure of a reasoning enhancement device for federated learning according to an embodiment of the present application;

[0021] Figure 5 A schematic diagram of a specific electronic device hardware structure of an embodiment of the present application. DETAILED DESCRIPTION

[0022] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0023] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be the usual meanings understood by people with general skills in the field to which the present application belongs. The "first", "second" and similar words used in the present application do not represent any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connecting" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, which can be direct connections, indirect connections, wired connections, and wireless connections. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly. In addition, in the description of the present application, unless otherwise specified, the term "multiple" refers to two or more. The term "and / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0024] The principles and spirit of the present application will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided only to enable those skilled in the art to better understand and implement the present application, and are not intended to limit the scope of the present application in any way. On the contrary, these embodiments are provided to make the present application more thorough and complete, and to fully convey the scope of the present application to those skilled in the art.

[0025] It is understandable that before using the technical solutions of each embodiment of the present application, the type, scope of use, usage scenarios, etc. of the personal information involved will be informed to the user in an appropriate manner, and the user's authorization will be obtained.

[0026] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly remind the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can independently choose whether to provide personal information to the electronic device, application, server, storage medium or other software or hardware that performs the operation of the technical solution of the present application according to the prompt message.

[0027] As an optional but non-limiting implementation, in response to receiving the user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0028] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation method of this application. Other methods that meet relevant laws and regulations may also be applied to the implementation method of this application.

[0029] It should be understood herein that any number of elements in the drawings is for illustration rather than limitation, and any naming is only for distinction rather than having any limiting meaning.

[0030] As mentioned in the background technology section, the optimization of federated learning models is currently mainly focused on the training phase. However, for the federated learning reasoning phase, there is a lack of efficient and accurate federated learning reasoning enhancement methods. In federated learning, each client usually holds heterogeneous data that is unevenly distributed and large in scale. In addition, the client's communication bandwidth, computing power, and storage space are often strictly limited. Therefore, under the conditions of large client data volumes and limited resources, it is of great practical significance to study the security and efficiency of federated learning during reasoning.

[0031] In the reasoning stage of federated learning, using the full amount of data of the client to assist in reasoning enhancement can improve the accuracy of the model, but it will significantly increase the amount of computation and communication costs in the reasoning process, resulting in low reasoning efficiency. Data screening is used to reduce the computational burden of the client. However, although this method reduces the communication and computing costs, it will inevitably lose a lot of information, resulting in large errors in the reasoning results of some clients. Therefore, how to balance computing efficiency and model accuracy while ensuring data privacy has become a technical problem that needs to be solved urgently.

[0032] A similarity query technology for local feature graphs is proposed in the related technology. The model extracts the features of local private data and uses it as auxiliary information to enhance the client's decision-making ability. However, it matches all local data and the data cannot be used to enhance the reasoning results of other clients. The accuracy and efficiency of auxiliary reasoning are low.

[0033] In summary, in order to solve the above problems, the present application embodiment proposes a method for enhancing reasoning of federated learning, including:

[0034] The inference client obtains the target feature of the target task data, encrypts the target feature, and sends it to each non-inference client; wherein the inference client is any one of the multiple clients, and the non-inference client is a client other than the inference client among the multiple clients; the non-inference client receives the encrypted target feature, determines the homomorphic encryption distance between the data of each data category included in the non-inference client and the encrypted target feature through homomorphic encryption technology, and determines the first data volume of the data category corresponding to the homomorphic encryption distance, multiplies the homomorphic encryption distance by the first data volume to obtain a first target distance; and The distance and the encrypted first data volume are returned to the inference client; the inference client determines the second distance between the target feature and the data of each data category included in the inference client, and determines the second data volume corresponding to each data category, and determines the inference enhancement result of the target task data based on the first target distance, the second distance, the first data volume and the second data volume. By performing distance matching with the data of each data category included in each non-inference client, the advantages of global data resources are brought into play, and the computational cost of client ciphertext matching is reduced, thereby achieving effective use of global heterogeneous data and improving the efficiency and accuracy of federated learning reasoning enhancement.

[0035] In some specific application scenarios, the reasoning enhancement method of federated learning of the present application can be applied to various platforms or systems involving federated learning.

[0036] The following describes the reasoning enhancement method of federated learning according to the exemplary embodiment of the present application in combination with specific application scenarios. It should be noted that the above application scenarios are only shown to facilitate understanding of the spirit and principles of the present application, and the implementation of the present application is not limited in this regard. On the contrary, the implementation of the present application can be applied to any applicable scenario.

[0037] refer to Figure 1 , is a flow chart of a method for enhancing reasoning of federated learning according to an embodiment of the present application, the method is applied to a federated learning system, the system includes multiple clients; the method includes the following steps:

[0038] S101, the inference client obtains the target features of the target task data, encrypts the target features, and sends them to each non-inference client; wherein the inference client is any one of the multiple clients, and the non-inference client is a client among the multiple clients except the inference client.

[0039] When implementing it, refer to Figure 2, is a federated learning system in an embodiment of the present application, the federated learning system includes multiple clients, and any client can be used as an inference client, the inference client is the client that currently needs to process the target task data; the non-inference client is the client other than the inference client. Optionally, after obtaining the target task data, the inference client can further extract the target features of the target task data, and then encrypt the target features, and send the encrypted target features to each non-inference client, so as to ensure that the target task data of the inference client is not leaked. Optionally, the inference client can generate a public key and a private key in advance, the public key is used to encrypt the target features to ensure the security of subsequent data transmission, and, in order to facilitate homomorphic encryption by the non-inference client, the public key can be sent to each non-inference client.

[0040] S102, the non-inference client receives the encrypted target feature, determines the homomorphic encryption distance between the data of each data category included in the non-inference client and the encrypted target feature through homomorphic encryption technology, and determines the first data volume of the data category corresponding to the homomorphic encryption distance, multiplies the homomorphic encryption distance by the first data volume to obtain a first target distance; and returns the first target distance and the encrypted first data volume to the inference client.

[0041] In specific implementation, after the non-inference client receives the encrypted target feature, it can use the above-mentioned public key and homomorphic encryption technology to determine the homomorphic encryption distance between each category of data included in the non-inference client and the encrypted target feature, and determine the first data volume of the data category corresponding to the homomorphic encryption distance, multiply the homomorphic encryption distance by the first data volume to obtain the first target distance; finally, the first target distance and the encrypted first data volume are returned to the inference client.

[0042] It should be noted that in order to prevent the distance between each data category of the non-inference client and the target feature from being leaked, in this embodiment, the homomorphic encryption distance is not directly returned to the inference client, but the homomorphic encryption distance is first multiplied by the first data volume to obtain the first target distance, and then the first target distance is returned. For example, if the distance between a certain type of data of a non-inference client and the target feature is 5, and the amount of data of this type is 1000, then the corresponding first target distance is 5000. When the inference client receives the value 5000, it cannot directly obtain the information that the distance between a certain type of data and the target feature is 5.

[0043] In order to further protect the first amount of data from being leaked, in some embodiments, returning the encrypted first amount of data to the inference client specifically includes:

[0044] The non-inference client returns the first data volume to the inference client through a secret sharing technology.

[0045] In specific implementation, the non-inference client can divide the first amount of data into multiple parts through secret sharing technology to generate corresponding shared shards. Each shard will be distributed to different nodes or participants to ensure the privacy and security of the data. On these nodes, simple calculations and processing can be performed on the shared shards without leaking the information of the original data. After that, each node sends the shards they own to the inference client. After receiving the shards from multiple nodes, the inference client recombines these shards through a reconstruction algorithm to obtain the encrypted data volume of each category.

[0046] In order to improve the efficiency and accuracy of the non-inference client in calculating the homomorphic encryption distance between each data category and the encrypted target feature, in some embodiments, the method further includes:

[0047] For any client among the multiple clients, the data included therein is divided into multiple data sets by clustering; wherein each data set corresponds to a data category; for each data category, representative data is determined from the data set, and the representative data is updated by gradient matching technology to obtain synthetic data; the synthetic data is used to represent the characteristics of the original data in the data set.

[0048] In specific implementation, each client includes a large amount of data, which can be divided into multiple data sets through clustering, each data set corresponds to a sub-cluster data, and each data set corresponds to a data category. Then for each data category, representative data can be determined from the data set, and then the representative data can be updated through the gradient matching technology to obtain synthetic data; the synthetic data is used to represent the characteristics of the original data in the data set.

[0049] refer to Figure 3 , use the clustering algorithm in the client to cluster the feature graphs in each category c and generate N C sub-clusters, where N c is a predefined hyperparameter. The center point of the sub-cluster is used as the representative data of the category The sample space of the entire category can be evenly covered while providing sufficient diversity. Representative data of each sub-cluster is selected as synthetic data Then the synthetic data and the data in the corresponding sub-cluster Input the client model for forward propagation to obtain the corresponding inference results of the two. Calculate the classification loss and its corresponding gradient, match the gradient according to the gradient difference function D(·), and update the synthetic data As shown in the following formula, this process enables the synthetic data to better preserve the commonality and feature information within the category.

[0050]

[0051] In some embodiments, when local data is updated, the client first uses the same feature extraction layer to generate a corresponding feature map for the new data, and then recalculates the distance between the feature maps based on the relationship between the pre-generated feature map and the new feature map, and clusters in the same way as described above; when the center point of a sub-cluster changes, the synthetic data of the sub-cluster is re-optimized. The dataset distillation process has fused the original data information into the synthetic data. When new data needs to be inferred, the distance to the inference data feature map is calculated based on the updated synthetic data feature map.

[0052] S103, the inference client determines a second distance between the target feature and the data of each data category included in the inference client, and determines a second data volume corresponding to each data category, and determines an inference enhancement result of the target task data based on the first target distance, the second distance, the first data volume and the second data volume.

[0053] In specific implementation, after obtaining the first target distance and the first data volume from the non-inference client, the inference client determines the second distance between the target feature and each data category locally included in the inference client, and determines the second data volume corresponding to each data category, and then determines the inference enhancement result of the target task data according to the first target distance, the second distance, the first data volume and the second data volume. That is, the inference enhancement is realized by using the data in all clients included in the federated learning system.

[0054] In some embodiments, after determining the reasoning enhancement result of the target task data based on the first target distance, the second distance, the first data volume, and the second data volume, the method further includes:

[0055] The inference client inputs the target task data into the task processing model corresponding to the federated learning system to obtain an initial output result; and determines an enhanced output result of the target task data based on the inference enhancement result and the initial output result.

[0056] During specific implementation, after determining the reasoning enhancement result of the target task data, the result can be used together with the initial output result of the task processing model to obtain the final enhanced output result of the target task data. Optionally, weights can be configured for the reasoning enhancement result and the initial output result respectively, and then the enhanced output result of the target task data can be comprehensively determined based on the configured weights. Optionally, in general, the weight of the initial output result is larger. For example, the weight ratio of the reasoning enhancement result to the initial output result can be set to 2:8. When the various clients included in the federated learning system continuously add new data, considering that the task processing model corresponding to the federated learning system is trained based on previous historical data, the weight ratio of the reasoning enhancement result can be appropriately increased.

[0057] In some embodiments, determining the reasoning enhancement result of the target task data based on the first target distance, the second distance, the first data volume, and the second data volume specifically includes:

[0058] multiplying the second distance by the second data amount to obtain a second target distance;

[0059] For each data category, determine the total quantity of the quantity category on all clients based on the first data quantity and the second data quantity, determine the average distance of the data category based on the first target distance, the second target distance and the total quantity, and determine the confidence of the data category based on the average distance;

[0060] The reasoning enhancement result of the target task data is determined based on the confidence of all the data categories.

[0061] In specific implementation, in order to unify the dimensions of the second distance and the first target distance, the second distance is multiplied by the second data volume to obtain the second target distance, and then the total data volume of each data category on all clients is used to determine the average distance of the data category, and the confidence of the data category is determined based on the average distance. Optionally, when determining the confidence of the data category based on the average distance, the average distance can be made inversely proportional to the confidence, that is, the smaller the average distance, the higher the corresponding confidence. In order to accurately obtain the confidence corresponding to each average distance, a preset mapping function can be directly set to obtain the confidence. Optionally, the preset mapping function can be designed as a monotonically decreasing curve mapping function to ensure that the smaller the distance, the higher the mapping value. After determining the confidence corresponding to each data category, the confidence can be normalized first, and then the normalized result can be used as the reasoning enhancement result of the target task data, that is, the reasoning enhancement result can be the confidence that the target task data belongs to each data category.

[0062] In some embodiments, determining the average distance of the data category based on the first target distance, the second target distance, and the total number specifically includes:

[0063] Decrypting the first target distance;

[0064] For each of the first target distances sent by the non-inference client, dividing the decrypted first target distance by the total number to obtain a first weighted distance corresponding to the non-inference client;

[0065] Dividing the second target distance by the total number to obtain a second weighted distance corresponding to the inference client;

[0066] An average distance of the data categories is determined based on the first weighted distance and the second weighted distance.

[0067] In specific implementation, in order to unify the first target distance and the second target distance, the first target distance can be first decrypted by the above-mentioned private key, and then the first weighted distance corresponding to the non-inference client and the second weighted distance corresponding to the inference client are calculated respectively. Finally, the average of all first weighted distances and second weighted distances is calculated as the average distance of the data category.

[0068] In some embodiments, determining the average distance of the data category based on the first weighted distance and the second weighted distance specifically includes:

[0069] In response to determining that the data category includes a plurality of synthetic data, each synthetic data corresponds to a weighted distance; wherein the first weighted distance and the second weighted distance each include a plurality of weighted distances;

[0070] Determine the target weighted distances that belong to the preset ranking from the smallest to the largest from each weighted distance;

[0071] An average distance for the data classes is determined based on the target weighted distance.

[0072] During the specific implementation, considering that in some embodiments, each data category may correspond to multiple synthetic data, this time, the distance between each data category of each non-inference client and the target feature may not be unique. For example, a data category includes two synthetic data, A1 and A2. At this time, A1 and A2 have a distance from the target feature respectively. For the convenience of calculation, in some embodiments, multiple synthetic data can be further unified into one data to represent the data category. However, in this embodiment, in order to retain the distances between multiple synthetic data and the target feature, the distance between each synthetic data and the target feature can be calculated first, and then the distances corresponding to multiple synthetic data with smaller distances can be selected, and these distances are used for the final average calculation to obtain the average distance of the data category.

[0073] For example, a federated learning system includes three clients, namely, an inference client 1 and two non-inference clients 2 and 3, wherein each client includes three data categories. The synthetic data corresponding to each data category included in client 1 is: (A11, A12), (B11, B12, B13), (C11, C12); the synthetic data corresponding to each data category included in client 2 is: (A21, A22), (B21, B22, B23), (C21, C22); the synthetic data corresponding to each data category included in client 3 is: (A31, A32, A33), (B31, B32, B33), (C31, C32). Optionally, the weighted distance of each synthetic data can be determined by the following formula:

[0074]

[0075] Among them, d represents the weighted distance of a synthetic data of the current client, x represents the distance between the synthetic data and the target feature, n represents the amount of data corresponding to the synthetic data on the current client, and N 总 Indicates the total data corresponding to the synthetic data in all clients. For example, the distance between the synthetic data A11 and the target feature is 5, and the amount of data corresponding to A11 in client 1 is 1000, the amount of data corresponding to A21 in client 2 is 500, and the amount of data corresponding to A31 in client 3 is 500. Then the weighted distance corresponding to A11 is (3*1000) / (1000+500+500)=1.5. Similarly, assuming that the weighted distance of the synthetic data corresponding to each data category included in client 1 is calculated as: (1.5,0.7),

[0076] (0.8, 1.5, 3), (0.3, 0.7,); the weighted distances of the synthetic data corresponding to each data category included in client 2 are: (0.8, 1.2), (0.3, 0.9, 1.5), (0.5, 0.9); the weighted distances of the synthetic data corresponding to each data category included in client 3 are: (0.9, 1.9, 1.5), (0.8, 2, 1.2), (0.2, 0.7). Assuming that the preset ranking is the first with the smallest distance, the average distance corresponding to the first data category is: (0.7 + 0.8 + 0.9) / 3 = 0.8. Assuming that the preset ranking is the top two with the smallest distance, the average distance corresponding to the first data category is: (0.7 + 1.5 + 0.8 + 1.2 + 0.9 + 1.5) / 6 = 1.1.

[0077] The present application provides an inference enhancement method for federated learning, in which an inference client obtains a target feature of target task data, encrypts the target feature, and sends the target feature to each non-inference client; wherein the inference client is any one of the multiple clients, and the non-inference client is a client other than the inference client among the multiple clients; the non-inference client receives the encrypted target feature, determines the homomorphic encryption distance between the data of each data category included in the non-inference client and the encrypted target feature through homomorphic encryption technology, and determines a first data volume of the data category corresponding to the homomorphic encryption distance, and multiplies the homomorphic encryption distance by the first data volume to obtain a first target distance ; and return the first target distance and the first encrypted data volume to the reasoning client; the reasoning client determines the second distance between the target feature and the data of each data category included in the reasoning client, and determines the second data volume corresponding to each data category, and determines the reasoning enhancement result of the target task data based on the first target distance, the second distance, the first data volume and the second data volume, and performs distance matching with the synthetic data of each category included in each non-reasoning client, thereby giving play to the advantages of global data resources, and at the same time reducing the computational cost of client ciphertext matching, achieving effective utilization of global heterogeneous data, and improving the efficiency and accuracy of federated learning reasoning enhancement. In addition, the present application selects representative data through a clustering algorithm, obtains synthetic data of the client data set through trajectory matching; performs secure query protection of the client's data distribution information through homomorphic encryption and secret sharing technology; maps the query data to the final query result according to the query data, thereby achieving efficient reasoning enhancement of the federated learning model under privacy protection, and performing good fluency and grammatical correctness.

[0078] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a reasoning enhancement device for federated learning, which is applied to a federated learning system, and the system includes multiple clients.

[0079] refer to Figure 4 , the reasoning enhancement device of the federated learning comprises:

[0080] In the task generation module 201, the inference client obtains the target feature of the target task data, encrypts the target feature, and sends it to each non-inference client; wherein the inference client is any one of the multiple clients, and the non-inference client is a client other than the inference client among the multiple clients;

[0081] Determination module 202, the non-inference client receives the encrypted target feature, determines the homomorphic encryption distance between the data of each data category included in the non-inference client and the encrypted target feature through homomorphic encryption technology, and determines the first data volume of the data category corresponding to the homomorphic encryption distance, multiplies the homomorphic encryption distance by the first data volume to obtain a first target distance; and returns the first target distance and the encrypted first data volume to the inference client;

[0082] Inference module 203, the inference client determines the second distance between the target feature and the data of each data category included in the inference client, and determines the second data volume corresponding to each data category, and determines the inference enhancement result of the target task data based on the first target distance, the second distance, the first data volume and the second data volume.

[0083] For the convenience of description, the above device is described in terms of functions divided into various modules. Of course, when implementing the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0084] The reasoning enhancement device for federated learning in the above-mentioned embodiment is used to implement the corresponding reasoning enhancement method for federated learning in any of the above-mentioned embodiments, and has the beneficial effects of the corresponding embodiment of the reasoning enhancement method for federated learning, which will not be repeated here.

[0085] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the reasoning enhancement method for federated learning described in any of the above embodiments is implemented.

[0086] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the described method.

[0087] It should be noted that the above describes some embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0088] Figure 5 A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown. The device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040 and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030 and the communication interface 1040 are connected to each other in communication within the device through the bus 1050.

[0089] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0090] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0091] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0092] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and the power equipment. The communication module can realize communication through a wired method (such as USB, network cable, etc.) or a wireless method (such as mobile network, WIFI, Bluetooth, etc.).

[0093] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).

[0094] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0095] The electronic device of the above embodiment is used to implement the corresponding federated learning reasoning enhancement method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0096] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present disclosure also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the reasoning enhancement method of federated learning as described in any of the above embodiments.

[0097] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0098] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the reasoning enhancement method of federated learning as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0099] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present application also provides a computer program product, which includes a computer program. In some embodiments, the computer program instructions can be executed by one or more processors of a computer so that the computer and / or the processor execute the reasoning enhancement method of federated learning described in the above-mentioned embodiments. Corresponding to the execution subject corresponding to each step in each embodiment of the reasoning enhancement method of federated learning, the processor that executes the corresponding step may belong to the corresponding execution subject.

[0100] To simplify the description and discussion, and in order not to make the embodiments of the present application difficult to understand, the known power supply / ground connection with the integrated circuit (IC) chip and other components may or may not be shown in the provided drawings. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present application difficult to understand, and this also takes into account the following fact, that is, the details of the implementation of these block diagram devices are highly dependent on the platform to be implemented in the embodiments of the present application (that is, these details should be fully within the scope of understanding of those skilled in the art). In the case of elaborating specific details (e.g., circuit) to describe exemplary embodiments of the present application, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered to be illustrative rather than restrictive.

[0101] Although the present application has been described in conjunction with specific embodiments of the present application, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0102] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application (including the claims) is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the present application as described above, which are not provided in detail for the sake of simplicity.

[0103] The embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of the present application.

Claims

1. A method for enhancing reasoning in federated learning, characterized in that: The method is applied to a federated learning system, wherein the system includes a plurality of clients; the method includes: The inference client obtains the target feature of the target task data, encrypts the target feature, and sends it to each non-inference client; wherein the inference client is any one of the multiple clients, and the non-inference client is a client other than the inference client among the multiple clients; The non-inference client receives the encrypted target feature, determines the homomorphic encryption distance between the data of each data category included in the non-inference client and the encrypted target feature through homomorphic encryption technology, determines the first data volume of the data category corresponding to the homomorphic encryption distance, multiplies the homomorphic encryption distance by the first data volume to obtain a first target distance; and returns the first target distance and the encrypted first data volume to the inference client; The inference client determines a second distance between the target feature and data of each data category included in the inference client, and determines a second data volume corresponding to each data category, and determines an inference enhancement result of the target task data based on the first target distance, the second distance, the first data volume and the second data volume.

2. The method according to claim 1, characterized in that After determining the reasoning enhancement result of the target task data based on the first target distance, the second distance, the first data volume, and the second data volume, the method further includes: The inference client inputs the target task data into the task processing model corresponding to the federated learning system to obtain an initial output result; and determines an enhanced output result of the target task data based on the inference enhancement result and the initial output result.

3. The method according to claim 1, characterized in that Determining the reasoning enhancement result of the target task data based on the first target distance, the second distance, the first data volume, and the second data volume specifically includes: multiplying the second distance by the second data amount to obtain a second target distance; For each data category, determine the total number of the data category on all clients based on the first data amount and the second data amount, determine the average distance of the data category based on the first target distance, the second target distance and the total number, and determine the confidence of the data category based on the average distance; The reasoning enhancement result of the target task data is determined based on the confidence of all the data categories.

4. The method according to claim 3, characterized in that Determining the average distance of the data category based on the first target distance, the second target distance and the total number specifically includes: Decrypting the first target distance; For each of the first target distances sent by the non-inference client, dividing the decrypted first target distance by the total number to obtain a first weighted distance corresponding to the non-inference client; Dividing the second target distance by the total number to obtain a second weighted distance corresponding to the inference client; An average distance of the data categories is determined based on the first weighted distance and the second weighted distance.

5. The method according to claim 4, characterized in that Determining the average distance of the data category based on the first weighted distance and the second weighted distance specifically includes: In response to determining that the data category includes a plurality of synthetic data, each synthetic data corresponds to a weighted distance; wherein the first weighted distance and the second weighted distance each include a plurality of weighted distances; Determine the target weighted distances that belong to the preset ranking from the smallest to the largest from each weighted distance; An average distance for the data classes is determined based on the target weighted distance.

6. The method according to claim 1, characterized in that Returning the encrypted first amount of data to the inference client specifically includes: The non-inference client returns the first data volume to the inference client through a secret sharing technology.

7. The method according to claim 1, characterized in that The method further comprises: For any client among the multiple clients, the data included therein is divided into multiple data sets by clustering; wherein each data set corresponds to a data category; for each data category, representative data is determined from the data set, and the representative data is updated by gradient matching technology to obtain synthetic data; the synthetic data is used to represent the characteristics of the original data in the data set.

8. A federated learning reasoning enhancement device, characterized in that: The device is applied to a federated learning system, the system includes multiple clients; the device includes: The task sending module, the inference client obtains the target features of the target task data, encrypts the target features, and sends them to each non-inference client; wherein the inference client is any one of the multiple clients, and the non-inference client is a client other than the inference client among the multiple clients; The determination module, wherein the non-inference client receives the encrypted target feature, determines the homomorphic encryption distance between the data of each data category included in the non-inference client and the encrypted target feature through homomorphic encryption technology, and determines the first data volume of the data category corresponding to the homomorphic encryption distance, multiplies the homomorphic encryption distance by the first data volume to obtain a first target distance; and returns the first target distance and the encrypted first data volume to the inference client; An inference module, wherein the inference client determines a second distance between the target feature and data of each data category included in the inference client, and determines a second data volume corresponding to each data category, and determines an inference enhancement result of the target task data based on the first target distance, the second distance, the first data volume, and the second data volume.

9. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 7.

Images