Privacy protection data screening method and device, equipment and storage medium
By adopting a lightweight proxy model and a fragmented transmission strategy, the problem of high computational cost in privacy protection data screening schemes for large language models is solved, achieving efficient screening and secure transmission, and expanding the applicability of the model.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-24
- Publication Date
- 2026-04-07
AI Technical Summary
Existing privacy-preserving data filtering schemes are computationally expensive, cannot be generalized to larger-scale language models, thus limiting their applicability, and pose a risk of data leakage.
A lightweight proxy model is used to replace the large language model. The information density and distribution characteristics of candidate data are quantified by entropy prediction, which can quickly identify high-value data. A fragmented transmission strategy is adopted to reduce the risk of data leakage and achieve on-demand transmission.
It improves the efficiency of privacy-preserving data screening, shortens the screening cycle, reduces computing power consumption and time costs, expands the applicability of large language models, and reduces the risk of data leakage.
Smart Images
Figure CN121808835A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of big data processing technology, and in particular to privacy-preserving data screening methods, apparatus, devices, and storage media. Background Technology
[0002] As publicly available data dwindles during the training of Large Language Models (LLMs), model developers are forced to procure privacy-preserving data specific to their domains to improve LLM performance in particular professional scenarios. For developers, this means selecting data that offers the greatest performance boost to LLMs while ensuring that model parameters are not leaked. For data owners, the goal is to maximize their data ownership, disclosing only limited privacy information about the data for sale.
[0003] The process of privacy-preserving data filtering schemes in related technologies is as follows: All candidate data are used as input, and a forward propagation process of LLM is executed using Secure Multi-party Computation (MPC). The entropy value of each candidate data point on the model is calculated, and the candidate data are then sorted in descending order of entropy value to filter out the candidate data with higher entropy values. However, due to the large number of parameters in LLM and the inclusion of numerous non-linear operations that are not friendly to MPC, the computational cost of the privacy-preserving forward propagation process based on MPC is extremely high. This makes it impossible to generalize to larger-scale LLMs, thus limiting its applicability. Summary of the Invention
[0004] The main objective of this application is to propose a method, apparatus, device, and storage medium for filtering privacy-preserving data, thereby improving the efficiency of filtering privacy-preserving data and expanding the applicability of large language models.
[0005] To achieve the above objectives, a first aspect of this application proposes a privacy-preserving data filtering method, applied to a data ownership device, the method comprising: Obtain the current proxy model from the data-using device, input the candidate privacy-preserving data into the proxy model for entropy prediction, and obtain the prediction mean and prediction variance corresponding to the candidate privacy-preserving data; Based on the predicted mean and the predicted variance, a first number of candidate target data are obtained from the candidate privacy protection data, a first data slice and a second data slice are generated for each candidate target data, and the first data slice is sent to the data usage device. Obtain data indication information, and send at least one second data slice corresponding to the candidate target data to the data usage device according to the data indication information.
[0006] In one embodiment, obtaining a first number of candidate target data from the candidate privacy-preserving data based on the predicted mean and the predicted variance includes: The prediction mean and the prediction variance are input into the acquisition function to calculate the confidence level, thereby obtaining the upper confidence bound of each candidate privacy-preserving data. The confidence upper bound is sorted from largest to smallest, and the first number of candidate privacy-preserving data are selected as the candidate target data.
[0007] In one embodiment, the acquisition of data indication information includes: The second data slice is sent to the target home device so that the target home device and the data user device perform deduplication based on the first data slice and the second data slice to obtain a second number of candidate target data after deduplication as deduplicated data. The target home device receives at least one second data slice sent by the data home device, and the data user device receives at least one first data slice sent by the data home device. Obtain the data indication information based on the deduplicated data.
[0008] In one embodiment, when the target attribution device and the data using device perform deduplication based on the first data slice and the second data slice to obtain a second number of candidate target data after deduplication as deduplicated data, the following steps are performed: The data is aggregated, including all the first data segments acquired by the data usage device and all the second data segments received by the target home device. Based on the mapping relationship, the first data segments and the second data segments are recombined into target filtering data. The target filtering data is deduplicated to obtain the deduplicated data.
[0009] In one embodiment, obtaining the data indication information based on the deduplicated data includes: The deduplicated data is sorted, and a preset number of the deduplicated data are selected as data to be confirmed. Based on the data source of the data to be confirmed, data indication information corresponding to the data home device is generated. The data indication information is used to instruct the data home device to send the second data segment corresponding to the data to be confirmed to the data user device.
[0010] To achieve the above objectives, a second aspect of this application proposes a privacy-preserving data filtering method, applied to a data-using device, the method comprising: Acquire training data, pre-train a large language model using the training data to obtain the entropy value corresponding to each training data, use the training data and the corresponding entropy value as input data, construct a proxy model based on the input data, and send the proxy model to at least one data ownership device. Receive at least one first data segment sent from at least one of the data home devices, receive at least one second data segment sent from at least one of the data home devices based on corresponding data indication information, and reconstruct the first data segment and the second data segment according to the mapping relationship to obtain at least one target data; After fine-tuning the large language model using the target data, the fine-tuning entropy value of the training data is obtained. The input data is then updated based on the fine-tuning entropy value, and the proxy model is updated as well.
[0011] In one embodiment, the process of generating the data indication information includes the following steps: All the first data segments and all the second data segments from the target home device are recombined into target filtering data based on the mapping relationship; The target filtering data is deduplicated to obtain the deduplicated data. The deduplicated data is sorted, and a preset number of the deduplicated data are selected as the data to be confirmed. Based on the data source of the data to be confirmed, data indication information corresponding to the data home device is generated. The data indication information is used to instruct the data home device to send the second data segment corresponding to the data to be confirmed to the data user device.
[0012] To achieve the above objectives, a third aspect of this application provides a privacy-preserving data filtering device applied to a data ownership device, the device comprising: Entropy prediction module: used to obtain the current proxy model from the data-using device, input the candidate privacy-preserving data into the proxy model to perform entropy prediction, and obtain the prediction mean and prediction variance corresponding to the candidate privacy-preserving data; Data splitting module: used to obtain a first number of candidate target data from the candidate privacy protection data according to the predicted mean and the predicted variance, generate a first data slice and a second data slice corresponding to each candidate target data, and send the first data slice to the data usage device; Data transmission module: used to acquire data indication information and send at least one second data slice corresponding to the candidate target data to the data usage device according to the data indication information.
[0013] To achieve the above objectives, a fourth aspect of the present application provides an electronic device, the electronic device including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method described in the first or second aspect.
[0014] To achieve the above objectives, a fifth aspect of the present application provides a storage medium storing a computer program that, when executed by a processor, implements the method described in the first or second aspect.
[0015] The privacy-preserving data filtering method, apparatus, device, and storage medium proposed in this application obtain the current proxy model from the data-using device, input candidate privacy-preserving data into the proxy model for entropy prediction, and obtain the prediction mean and prediction variance corresponding to the candidate privacy-preserving data; based on the prediction mean and prediction variance, obtain a first number of candidate target data from the candidate privacy-preserving data, generate a first data slice and a second data slice corresponding to each candidate target data, and send the first data slice to the data-using device; obtain data indication information, and send at least one second data slice corresponding to candidate target data to the data-using device according to the data indication information. This application uses a lightweight proxy model to replace the large language model with a large number of parameters to directly participate in data filtering. By using entropy prediction to quantify the information density and distribution characteristics of candidate data, it can quickly identify privacy-preserving data with high value for training or inference of the large language model. Compared with the method of full data traversal filtering, it can reduce the processing of worthless data, effectively shorten the filtering cycle, improve the filtering efficiency of privacy-preserving data, and avoid the high computing power consumption and time cost caused by the large language model directly processing massive candidate data. After selecting candidate target data, a fragmented transmission strategy is used to split the data into multiple segments for transmission: the first data segment is sent to the data-using device first, and then the second data segment is sent according to the data indication information. Since a single data segment cannot reconstruct complete privacy information, the risk of data leakage can be reduced at the transmission level. Furthermore, the data-using device can determine the data indication information corresponding to each data-owning device based on actual needs, only acquiring the second data segment of the required candidate target data, achieving on-demand data transmission. This avoids the privacy and security risks associated with full-volume transmission and reduces unnecessary data transmission overhead. This improves the efficiency of privacy-preserving data screening and expands the applicability of large language models. Attached Figure Description
[0016] Figure 1 This is a flowchart of the privacy-protected data filtering method provided in the embodiments of this application.
[0017] Figure 2This is a flowchart provided in an embodiment of the present application for obtaining a first number of candidate target data from candidate privacy-preserving data based on the predicted mean and predicted variance.
[0018] Figure 3 This is a flowchart of the process for obtaining data indication information provided in the embodiments of this application.
[0019] Figure 4 This is a flowchart illustrating how the target attribution device and the data usage device, based on the first data slice and the second data slice, perform deduplication to obtain a second number of candidate target data as deduplicated data.
[0020] Figure 5 This is a flowchart of how data indication information is obtained based on deduplicated data in this application.
[0021] Figure 6 This is an optional flowchart of the privacy-protected data filtering method provided in the embodiments of this application applied to a data-using device.
[0022] Figure 7 This is an overall flowchart of the privacy-protected data filtering method according to an embodiment of this application.
[0023] Figure 8 This is a structural block diagram of a privacy-protected data filtering device provided in another embodiment of this application.
[0024] Figure 9 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation
[0025] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0026] It should be noted that although functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart.
[0027] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.
[0028] First, let's analyze some of the terms used in this application: Artificial Intelligence (AI) is a new branch of computer science that studies, develops, and applies theories, methods, technologies, and systems to simulate, extend, and expand human intelligence. It aims to understand the essence of intelligence and produce intelligent machines that can react in a way similar to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing, and expert systems. AI can simulate the information processes of human consciousness and thought. Furthermore, AI utilizes digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceiving the environment, acquiring knowledge, and using that knowledge to achieve optimal results.
[0029] Currently, Large Language Models (LLMs), such as DeepSeek-R1, GPT-4O, and LLaMa-3, have permeated all aspects of production and daily life, continuously enriching people's daily work and lifestyles. However, as publicly available data gradually dwindles, the training of large language models faces a data shortage dilemma. Model developers are forced to purchase private data from specialized fields to improve the performance of LLMs in specific professional scenarios. This trend not only drives up the value of data in specialized fields but also turns it into a commodity with tradable value. For model developers, they hope to maximize the performance of LLMs at the lowest cost, thus needing to select the data with the highest value for improving LLM performance. At the same time, given that LLM parameters are their core commercial competitiveness, developers also need to ensure the privacy of these parameters is not leaked. For data owners, they hope to maximize the protection of their data ownership, that is, to only disclose relevant private information about the data to be sold.
[0030] For transactions involving privacy-preserving data, delivery can only be completed after a purchase agreement is reached between the parties. However, the unique nature of privacy-preserving data—the binding of ownership and usage rights—makes it impossible for model developers to assess its value before a transaction is finalized. Therefore, it is necessary to screen candidate privacy-preserving data owners while meeting the privacy needs of both parties, obtaining the data most beneficial to improving LLM performance, and thus helping both parties reach a purchase agreement.
[0031] The process of privacy-preserving data filtering schemes in related technologies is as follows: All candidate data are used as input, and a forward propagation process of LLM is executed using Secure Multi-party Computation (MPC). The entropy value of each candidate data point on the model is calculated, and the candidate data are then sorted in descending order of entropy value to filter out the candidate data with higher entropy values. However, due to the large number of parameters in LLM and the inclusion of numerous non-linear operations that are not friendly to MPC, the computational cost of the privacy-preserving forward propagation process based on MPC is extremely high. This makes it impossible to generalize to larger-scale LLMs, thus limiting its applicability.
[0032] Based on this, embodiments of this application provide a privacy-preserving data filtering method, apparatus, device, and storage medium. A lightweight proxy model replaces a large language model with a massive number of parameters in directly participating in data filtering. Entropy prediction is used to quantify the information density and distribution characteristics of candidate data, enabling rapid identification of privacy-preserving data with high value for training or inference with the large language model. Compared to full-data traversal filtering, this reduces the processing of worthless data, effectively shortens the filtering cycle, and improves the efficiency of privacy-preserving data filtering. It also avoids the high computational cost and time expense of large language models directly processing massive amounts of candidate data. After filtering out candidate target data, a segmented transmission strategy is used to split the data into multiple fragments for transmission: the first data fragment is sent to the data-using device, and then the second data fragment is sent according to the data indication information. Since a single data fragment cannot restore complete privacy information, the risk of data leakage can be reduced at the transmission level. Furthermore, the data-using device can determine the data indication information corresponding to each data-owning device according to actual needs, only obtaining the second data fragment of the required candidate target data, achieving on-demand data transmission. This avoids the privacy and security risks of full-data transmission and reduces unnecessary data transmission overhead. This improves the efficiency of privacy-preserving data screening and expands the applicability of large language models.
[0033] This application provides a method, apparatus, device, and storage medium for filtering privacy-protected data, which will be described in detail through the following embodiments. First, the privacy-protected data filtering method in this application embodiment is described.
[0034] This application's embodiments can acquire and process relevant data based on artificial intelligence (AI) technology. AI is the theory, methods, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can react in a way similar to human intelligence. AI also studies the design principles and implementation methods of various intelligent machines, enabling them to possess perception, reasoning, and decision-making capabilities.
[0035] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.
[0036] The privacy-preserving data filtering method provided in this application relates to the field of big data processing technology. This method can be applied to a terminal, a server, or a computer program running on either the terminal or the server. For example, the computer program can be a native program or software module in an operating system; it can be a native application (APP), i.e., a program that needs to be installed in the operating system to run, such as a client that supports privacy-preserving data filtering, i.e., a program that only needs to be downloaded to a browser environment to run; or it can be a small program that can be embedded in any APP. In short, the above-mentioned computer program can be any form of application, module, or plugin. The terminal communicates with the server via a network. This privacy-preserving data filtering method can be executed by the terminal or the server, or by the terminal and the server working together.
[0037] In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, or smartwatch, etc. The server can be a standalone server, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms; it can also be a service node in a blockchain system, where the service nodes form a peer-to-peer (P2P) network. The P2P protocol is an application layer protocol running on top of the Transmission Control Protocol (TCP). The terminal and server can connect via Bluetooth, Universal Serial Bus (USB), or a network, etc., and this embodiment does not impose any limitations.
[0038] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.
[0039] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when embodiments of this application require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirection to confirmation pages. Only after obtaining the user's separate permission or consent is the necessary user-related data required for the proper functioning of these embodiments acquired.
[0040] The privacy-protected data filtering method in the embodiments of this application is described below.
[0041] First, let's describe the overall framework of this application's embodiment. The large language model M to be fine-tuned is deployed on a data-using device, which can be considered the model developer. There are multiple data owners, each of which can be considered a data-attributing device, and each data-attributing device possesses multiple candidate privacy-preserving data. To fine-tune the large language model M, the model developer needs to purchase or obtain corresponding candidate privacy-preserving data from at least one data owner as training data. However, the model developer cannot directly send the large language model's parameters to the data owners, and they need to obtain the training data that best performs the training of the large language model. Similarly, the data owners cannot directly send all the candidate privacy-preserving data to the model developer; they need to send the data that the model developer needs to purchase or obtain while ensuring privacy and security. This application's embodiment addresses the privacy-preserving data selection process in this scenario, which will be described in detail below.
[0042] Figure 1 This is an optional flowchart of the privacy-protected data filtering method provided in the embodiments of this application. Figure 1 The method described herein, when applied to a data ownership device, may include, but is not limited to, steps 110 to 130. It is also understood that this embodiment... Figure 1 The order of steps 110 to 130 is not specifically limited. The order of steps can be adjusted or some steps can be reduced or added according to actual needs.
[0043] Step 110: Obtain the current proxy model from the data-using device, input the candidate privacy-preserving data into the proxy model for entropy prediction, and obtain the prediction mean and prediction variance corresponding to the candidate privacy-preserving data.
[0044] In one embodiment, the data-using device sends the current proxy model to each data-owning device. This proxy model differs from the deployed large language model, and its construction process will be detailed later in the flow description focusing on the data-using device. Each data-owning device then obtains the current proxy model and uses it to process its local candidate privacy-preserving data. This eliminates the need for the data-using device to externalize the parameters of the large language model, and also eliminates the need for the data-owning device to externalize the entire dataset. Both devices can process the data locally using the proxy model. The proxy model directly accesses the candidate privacy-preserving data, replacing the large language model and avoiding the risk of privacy leaks.
[0045] Specifically, a surrogate model is used to estimate the entropy value of local candidate privacy-preserving data. In privacy-preserving data scenarios, entropy prediction can assess the effectiveness of privacy protection processing; the higher the entropy value, the stronger the data uncertainty, and generally the higher the degree of privacy protection. The surrogate model extracts features and calculates entropy values from the input candidate privacy-preserving data based on its trained feature mapping relationships, obtaining the predicted mean and predicted variance. The predicted mean refers to the arithmetic mean of the entropy value predictions of the candidate privacy-preserving data by the surrogate model, reflecting the central tendency of the entropy values of this batch of data and is a core indicator for measuring the overall level of data privacy protection. The predicted variance refers to the dispersion of the entropy prediction results of the surrogate model, used to measure the difference in entropy values between individual candidate privacy-preserving data.
[0046] Step 120: Obtain a first number of candidate target data from the candidate privacy protection data based on the predicted mean and predicted variance, generate a first data slice and a second data slice corresponding to each candidate target data, and send the first data slice to the data usage device.
[0047] In one embodiment, reference is made to Figure 2 , Figure 2 This is a flowchart provided in an embodiment of the present application for obtaining a first number of candidate target data from candidate privacy-preserving data based on the predicted mean and predicted variance, specifically including the following steps: Step 210: Input the predicted mean and predicted variance into the acquisition function to calculate the confidence level and obtain the upper confidence bound for each candidate privacy-preserving data.
[0048] In one embodiment, the acquisition function can be a Gaussian process-confidence upper bound function. This function combines the predicted mean and predicted variance to comprehensively evaluate the potential value and reliability of candidate privacy-preserving data. The input to the acquisition function is the predicted mean and predicted variance calculated by the surrogate model described above, and the output is a quantitative score, such as a confidence upper bound, used to rank the candidate privacy-preserving data. This allows for the efficient selection of the most valuable data for fine-tuning the large language model from a vast amount of candidate privacy-preserving data.
[0049] Step 220: Sort the confidence upper bounds from largest to smallest and select the first number of candidate privacy-preserving data as candidate target data.
[0050] In one embodiment, the data ownership device sorts all candidate data by confidence upper bounds, and selects the first number of candidate privacy-preserving data with the largest confidence upper bounds as candidate target data based on the sorting results. For example, the data owner... Sort all candidate privacy-preserving data by confidence upper bounds, and select h candidate privacy-preserving data with the largest confidence upper bounds as candidate target data. .
[0051] Next, based on the selected candidate target data, a first data slice and a second data slice are generated for each candidate target data. This data sharding is performed according to preset sharding rules, such as sharding by data feature dimensions, data content segmentation, or encrypted sharding algorithms. This generates corresponding first and second data slices for each candidate target data. For example, for candidate target data... Data owner generate ,in, for The first data slice, for The second data slice.
[0052] Understandably, after fragmentation, fragment verification is required to ensure that the two data fragments can be completely reconstructed to obtain the candidate target data, and that no information is lost or corrupted after fragmentation. Then, the first data fragment is sent to the data-using device through a secure transmission channel, while the second data fragment is either stored locally or transmitted to a designated secure storage node according to a privacy protection policy.
[0053] Step 130: Obtain data indication information, and send the second data slice corresponding to at least one candidate target data to the data usage device according to the data indication information.
[0054] In one embodiment, reference is made to Figure 3 , Figure 3 This is a flowchart of the process for obtaining data indication information provided in an embodiment of this application, which specifically includes the following steps: Step 310: Send the second data slice to the target home device so that the target home device and the data user device can perform deduplication based on the first data slice and the second data slice to obtain a second number of candidate target data after deduplication as deduplicated data.
[0055] In one embodiment, while the data-using device receives first data fragments sent by all data-owning devices, all data-owning devices also need to select one data-owning device as a secure storage node, referred to as the target data-owning device. The target data-owning device receives second data fragments sent by at least one data-owning device and aggregates the second data fragments from all data-owning devices, including its own second data fragments. Simultaneously, the data-using device also aggregates the first data fragments received from at least one data-owning device. Therefore, the target data-owning device contains all the second data fragments, and the data-using device contains all the first data fragments.
[0056] In one embodiment, reference is made to Figure 4 , Figure 4This is a flowchart illustrating how the target attribution device and data usage device, based on a first data slice and a second data slice, perform deduplication to obtain a second number of candidate target data as deduplicated data. The flowchart specifically includes the following steps: Step 410: Summarize all the first data slices obtained from the data usage device and all the second data slices received from the target home device, and recombine the first data slices and second data slices into target filtering data based on the mapping relationship.
[0057] In one embodiment, since the first and second data slices obtained from splitting the same candidate target data can be recombined into candidate target data, the relationship between the first data slice, the second data slice, and the candidate target data is called a mapping relationship. Therefore, each group of first and second data slices can be recombined according to the mapping relationship, and the combined data is called target screening data.
[0058] Step 420: Perform a deduplication operation on the target filtered data to obtain deduplicated data.
[0059] In one embodiment, a deduplication operation is performed on all target screening data. For example, a hash operation is performed on the target screening data to generate a unique fingerprint. Duplicate data is identified by comparing fingerprint values, and then deduplicated data is obtained, which contains multiple target screening data.
[0060] Step 320: Obtain data indication information based on deduplicated data.
[0061] In one embodiment, once the deduplicated data is available, data indication information for each data-owning device can be obtained. (See also...) Figure 5 , Figure 5 This is a flowchart illustrating how the data indication information is obtained based on deduplicated data, specifically including the following steps: Step 510: Sort the deduplicated data and select a preset number of deduplicated data as the data to be confirmed.
[0062] In one embodiment, the deduplicated data obtained after deduplication is sorted according to a preset sorting rule, which may be sorting by entropy value. Then, a preset number of deduplicated data items are selected from the top of the list and designated as data to be confirmed, thus entering the subsequent data distribution and confirmation process.
[0063] Step 520: Generate data indication information corresponding to the data-owning device based on the data source of the data to be confirmed.
[0064] In one embodiment, after obtaining the data to be confirmed, it is determined which data-owning device each piece of data belongs to. Then, data indication information is generated for each data-owning device based on the data source. This data indication information instructs the corresponding data-owning device to send the second data segment corresponding to the data to be confirmed to the data-using device. At this time, the data-using device includes all the first data segments and some of the second data segments. Therefore, the corresponding first data segment is selected based on the second data segment for data reconstruction, resulting in multiple target data sets.
[0065] As described above, this application employs a lightweight proxy model to replace the large language model with a massive number of parameters in direct data screening. By leveraging entropy prediction to quantify the information density and distribution characteristics of candidate data, it can quickly identify privacy-preserving data with high value for training or inference of the large language model. Compared to the method of screening through all data, this reduces the processing of worthless data, effectively shortens the screening cycle, and improves the screening efficiency of privacy-preserving data. It also avoids the high computational cost and time expense of the large language model directly processing massive amounts of candidate data. After screening candidate target data, a segmented transmission strategy is used to split the data into multiple fragments for transmission: the first data fragment is sent to the data-using device, and then the second data fragment is sent according to the data indication information. Since a single data fragment cannot restore complete privacy information, the risk of data leakage can be reduced at the transmission level. Furthermore, the data-using device can determine the data indication information corresponding to each data-owning device according to actual needs, only obtaining the second data fragment of the required candidate target data, achieving on-demand data transmission. This avoids the privacy and security risks of full transmission and reduces unnecessary data transmission overhead. This improves the screening efficiency of privacy-preserving data and expands the applicability of the large language model.
[0066] The following describes the execution process of the data using the device provided in the embodiments of this application.
[0067] In one embodiment, Figure 6 This is an optional flowchart of the privacy-protected data filtering method provided in this application embodiment applied to a data-using device. Figure 6 The method may include, but is not limited to, steps 610 to 630. It is also understood that this embodiment... Figure 6 The order of steps 610 to 630 is not specifically limited, and the order of steps can be adjusted or some steps can be reduced or added according to actual needs.
[0068] Step 610: Obtain training data, pre-train the large language model using the training data, obtain the entropy value corresponding to each training data, use the training data and the corresponding entropy value as input data, construct a proxy model based on the input data, and send the proxy model to at least one data ownership device.
[0069] In one embodiment, multiple data points can be sampled from a public dataset as training data. This training data is then used as input to a large language model for plaintext inference, yielding the entropy value for each training data point. Here, the entropy value represents the uncertainty of the probability distribution output by the large language model after one forward pass. Higher entropy indicates a more uniform probability distribution, meaning the model is less confident in its predictions. Therefore, the model can learn more from this type of training data, making the training data more valuable. Conversely, lower entropy indicates a more concentrated probability distribution, meaning the model is more confident in its predictions and has fully grasped the patterns in this type of data. Consequently, the training data has lower learning value.
[0070] Next, the training data and their corresponding entropy values are correlated and used together as input data to construct a proxy model. The proxy model's complexity needs to be significantly lower than that of the large language model; it can be a linear regression model, a lightweight Transformer model, a ResNet-18 network, etc. At this point, the proxy model constructed from the input data can achieve the mapping between the training relationships and the entropy values.
[0071] The proxy model is then sent to at least one data ownership device.
[0072] Step 620: Receive at least one first data segment sent from at least one data home device, receive at least one second data segment sent from at least one data home device based on corresponding data indication information, and reconstruct the first data segment and the second data segment according to the mapping relationship to obtain at least one target data.
[0073] In one embodiment, after the data home device obtains multiple first data fragments based on the proxy model, it sends all the first data fragments to the data user device. Then, the data user device and the target home device jointly generate data indication information for each data home device.
[0074] Specifically, the data indication information generation process includes the following steps: recombining all the first data fragments and all the second data fragments from the target home device into target filtering data based on the mapping relationship; performing a deduplication operation on the target filtering data to obtain deduplicated data; sorting the deduplicated data; selecting a preset number of deduplicated data as data to be confirmed; and generating data indication information corresponding to the data home device based on the data source of the data to be confirmed. The data indication information is used to instruct the data home device to send the second data fragment corresponding to the data to be confirmed to the data user device.
[0075] Next, the data-using device receives at least one second data slice from at least one data-owning device based on corresponding data indication information, and reconstructs the first and second data slices according to the mapping relationship to obtain at least one target data.
[0076] Step 630: After fine-tuning the large language model using the target data, obtain the fine-tuning entropy value of the training data, update the input data based on the fine-tuning entropy value, and update the proxy model.
[0077] In one embodiment, after obtaining the target data, the data-using device uses the target data to fine-tune the large language model, and then uses the fine-tuned large language model to re-obtain the fine-tuning entropy value of the training data. The fine-tuning entropy value is used to update the input data, and the proxy model is updated accordingly, thus entering a new round of iteration.
[0078] The following reference Figure 7 , Figure 7 This is an overall flowchart of the privacy-preserving data filtering method according to an embodiment of this application, which describes the execution flow of the data using device and the data-owning device. (Refer to...) Figure 7 This includes one data-using device, namely the model developer S, and multiple data-attributing devices, namely the data owners. Specifically, taking the GP Gaussian process model as an example, the entire execution process includes two stages: surrogate model preconstruction and privacy-preserving data filtering, and surrogate model optimization.
[0079] The first step is proxy model pre-construction. The main purpose of the proxy model pre-construction stage is for model developers to obtain training data using publicly available data and initialize the GP proxy model locally in plaintext. The surrogate model is constructed to predict the entropy value of the inference result of the large language model (LLM) to be fine-tuned for a given data. The specific steps are as follows: 1. The model developer samples m data points from a publicly available dataset. As training data.
[0080] 2. The model developers will As a large language model that needs fine-tuning Given the input, perform a plaintext inference process and output the entropy value representing the uncertainty of each data point. .
[0081] 3. The model developers will As input data, the GP model building process is executed, and the constructed GP proxy model is output. .
[0082] 4. The model developers will Send to each data owner .
[0083] The next step is the process of privacy-preserving data screening and proxy model optimization. The main purpose of the privacy-preserving data screening and proxy model optimization phase is for the data owner to utilize the initial... Local candidate privacy-preserving data is filtered, and the selected candidate target data is sent to the model developers, who then use the selected data to fine-tune the large language model. and to Update.
[0084] In the specific steps, for Model developer S and data owner Together, they cycle through the following processes: 1. Data owner Using candidate privacy-preserving data as the current proxy model The input is used to execute the GP model prediction process, which yields the prediction mean and prediction variance for all candidate privacy-preserving data.
[0085] 2. Data Owner By using the predicted mean and predicted variance as inputs to a collection function, such as a Gaussian process with upper confidence limit, the upper confidence bound for each candidate privacy-preserving data is calculated.
[0086] 3. Data Owner Sort all candidate privacy-preserving data by confidence upper bounds, and select h candidate privacy-preserving data with the largest confidence upper bounds as candidate target data. .
[0087] 4. Data Owner Generate candidate target data First data slice and second data slice and the first data slice Send to model developer S.
[0088] 5. Data Owner , the second data slice Send to the target device, such as the data owner. .
[0089] 6. Model developer S and data owner Will As input, the first and second data slices are recombined into target filtering data based on the mapping relationship. The target filtering data is then deduplicated to obtain deduplicated data. ,in, Here we assume that each data owner sends h data entries to the model developer.
[0090] 7. Model developer S and data owner Deduplicated data As input, the deduplicated data is sorted, and a preset number k of deduplicated data is selected as the data to be confirmed. It is understandable that both deduplication and sorting can be performed in an encrypted state to avoid data security risks.
[0091] 8. Based on the data source of the data to be confirmed, generate data instruction information corresponding to the data owner. The data owner then sends the second data fragment corresponding to the data to be confirmed to the model developer S according to the data instruction information.
[0092] 9. Model developer S reconstructs the first and second data slices based on the mapping relationship to obtain the purchased k target data items. And use them as input to large language models. Fine-tuning was performed to obtain .
[0093] 10. Model developer S utilizes Reacquire training data The fine-tuned entropy value is obtained. .
[0094] 11. Model developer S will Using the new input data, a new proxy model is constructed. And send it to each data owner, for example .
[0095] The process described above involves multiple rounds of continuous screening to improve the quality of data selection. Specifically, model developers screen candidate data in multiple rounds, using the data obtained in each round to fine-tune the large language model. This allows developers to update the proxy model using the fine-tuned large language model and re-screen the model, thus avoiding the purchase of unnecessary data.
[0096] As described above, in this embodiment of the application, model developers with model parameters and computing power, and data owners with privacy-preserving candidate data and computing power, can complete data filtering while protecting the privacy of model parameters and candidate data, obtaining the data most helpful for improving the performance of large language models. Furthermore, by utilizing a parameter-independent proxy model, the efficiency of privacy-preserving data filtering is improved, avoiding the huge overhead caused by the large language model's privacy-preserving forward propagation in privacy data filtering, thereby greatly improving the efficiency of privacy-preserving data filtering for large language models and enabling it to be generalized to larger-scale models.
[0097] The technical solution provided in this application involves obtaining the current proxy model from the data-using device, inputting candidate privacy-preserving data into the proxy model for entropy prediction, and obtaining the prediction mean and prediction variance corresponding to the candidate privacy-preserving data. Based on the prediction mean and prediction variance, a first number of candidate target data are obtained from the candidate privacy-preserving data, generating a first data slice and a second data slice corresponding to each candidate target data, and sending the first data slice to the data-using device. Data indication information is obtained, and based on the data indication information, at least one second data slice corresponding to a candidate target data is sent to the data-using device. This application uses a lightweight proxy model to replace the large language model with a large number of parameters to directly participate in data screening. By leveraging entropy prediction to quantify the information density and distribution characteristics of candidate data, it can quickly identify privacy-preserving data with high value for training or inference of the large language model. Compared to the method of traversing and screening all data, it can reduce the processing steps of worthless data, effectively shorten the screening cycle, improve the screening efficiency of privacy-preserving data, and avoid the high computational power consumption and time cost caused by the large language model directly processing massive candidate data. After selecting candidate target data, a fragmented transmission strategy is used to split the data into multiple segments for transmission: the first data segment is sent to the data-using device first, and then the second data segment is sent according to the data indication information. Since a single data segment cannot reconstruct complete privacy information, the risk of data leakage can be reduced at the transmission level. Furthermore, the data-using device can determine the data indication information corresponding to each data-owning device based on actual needs, only acquiring the second data segment of the required candidate target data, achieving on-demand data transmission. This avoids the privacy and security risks associated with full-volume transmission and reduces unnecessary data transmission overhead. This improves the efficiency of privacy-preserving data screening and expands the applicability of large language models.
[0098] This application also provides a privacy-protected data filtering device that can implement the above-described privacy-protected data filtering method, see reference. Figure 8 The device is used in data ownership equipment and includes: Entropy prediction module 810: used to obtain the current proxy model from the data-using device, input the candidate privacy-preserving data into the proxy model to perform entropy prediction, and obtain the prediction mean and prediction variance corresponding to the candidate privacy-preserving data.
[0099] Data splitting module 820: used to obtain a first number of candidate target data from the candidate privacy-preserving data based on the predicted mean and predicted variance, generate a first data slice and a second data slice corresponding to each candidate target data, and send the first data slice to the data-using device.
[0100] Data transmission module 830: used to acquire data indication information and send the second data slice corresponding to at least one candidate target data to the data usage device according to the data indication information.
[0101] The specific implementation of the privacy-protected data filtering device in this embodiment is basically the same as the specific implementation of the privacy-protected data filtering method described above, and will not be repeated here.
[0102] This application also provides an electronic device, including: At least one memory; at least one processor; at least one program; The program is stored in a memory, and the processor executes the at least one program to implement the privacy-protected data filtering method described above. The electronic device can be any smart terminal, including mobile phones, tablets, personal digital assistants (PDAs), and in-vehicle computers.
[0103] Please see Figure 9 , Figure 9 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes: The processor 901 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 902 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 902 and is called and executed by the processor 901 using the privacy-protected data filtering method of the embodiments of this application. The input / output interface 903 is used to implement information input and output; The communication interface 904 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 905 transmits information between various components of the device (e.g., processor 901, memory 902, input / output interface 903, and communication interface 904); The processor 901, memory 902, input / output interface 903, and communication interface 904 are connected to each other within the device via bus 905.
[0104] This application embodiment also provides a storage medium that stores a computer program, which, when executed by a processor, implements the above-described privacy-protected data filtering method.
[0105] Memory, as a non-transitory storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.
[0106] The privacy-preserving data filtering method, apparatus, device, and storage medium proposed in this application obtain the current proxy model from the data-using device, input candidate privacy-preserving data into the proxy model for entropy prediction, and obtain the prediction mean and prediction variance corresponding to the candidate privacy-preserving data; based on the prediction mean and prediction variance, obtain a first number of candidate target data from the candidate privacy-preserving data, generate a first data slice and a second data slice corresponding to each candidate target data, and send the first data slice to the data-using device; obtain data indication information, and send at least one second data slice corresponding to candidate target data to the data-using device according to the data indication information. This application uses a lightweight proxy model to replace the large language model with a large number of parameters to directly participate in data filtering. By using entropy prediction to quantify the information density and distribution characteristics of candidate data, it can quickly identify privacy-preserving data with high value for training or inference of the large language model. Compared with the method of full data traversal filtering, it can reduce the processing of worthless data, effectively shorten the filtering cycle, improve the filtering efficiency of privacy-preserving data, and avoid the high computing power consumption and time cost caused by the large language model directly processing massive candidate data. After selecting candidate target data, a fragmented transmission strategy is used to split the data into multiple segments for transmission: the first data segment is sent to the data-using device first, and then the second data segment is sent according to the data indication information. Since a single data segment cannot reconstruct complete privacy information, the risk of data leakage can be reduced at the transmission level. Furthermore, the data-using device can determine the data indication information corresponding to each data-owning device based on actual needs, only acquiring the second data segment of the required candidate target data, achieving on-demand data transmission. This avoids the privacy and security risks associated with full-volume transmission and reduces unnecessary data transmission overhead. This improves the efficiency of privacy-preserving data screening and expands the applicability of large language models.
[0107] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.
[0108] Those skilled in the art will understand that the technical solutions illustrated in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than illustrated, or combine certain steps, or different steps. The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the solution in this embodiment according to actual needs.
[0109] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.
[0110] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0111] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.
[0112] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.
[0113] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, the functional units in the various embodiments of this application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The integrated units described above can be implemented in hardware or as software functional units.
[0114] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0115] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.
Claims
1. A method for filtering privacy-preserving data, characterized in that, Applied to data ownership devices, the method includes: Obtain the current proxy model from the data-using device, input the candidate privacy-preserving data into the proxy model to perform entropy prediction, and obtain the prediction mean and prediction variance corresponding to the candidate privacy-preserving data; Based on the predicted mean and the predicted variance, a first number of candidate target data are obtained from the candidate privacy protection data, a first data slice and a second data slice are generated for each candidate target data, and the first data slice is sent to the data usage device. Obtain data indication information, and send at least one second data slice corresponding to the candidate target data to the data usage device according to the data indication information.
2. The privacy-protected data filtering method according to claim 1, characterized in that, The step of obtaining a first number of candidate target data from the candidate privacy-preserving data based on the predicted mean and the predicted variance includes: The predicted mean and the predicted variance are input into the acquisition function to calculate the confidence level, thereby obtaining the upper confidence bound of each candidate privacy-preserving data. The confidence upper bound is sorted from largest to smallest, and the first number of candidate privacy-preserving data are selected as the candidate target data.
3. The privacy-protecting data filtering method according to claim 1, characterized in that, The data acquisition instruction information includes: The second data slice is sent to the target home device so that the target home device and the data user device perform deduplication based on the first data slice and the second data slice to obtain a second number of candidate target data after deduplication as deduplicated data. The target home device receives at least one second data slice sent by the data home device, and the data user device receives at least one first data slice sent by the data home device. Obtain the data indication information based on the deduplicated data.
4. The privacy-protected data filtering method according to claim 3, characterized in that, When the target attribution device and the data using device perform deduplication based on the first data slice and the second data slice to obtain a second number of candidate target data after deduplication, the following steps are performed: The data is aggregated, including all the first data segments obtained from the data usage device and all the second data segments received by the target home device. Based on the mapping relationship, the first data segments and the second data segments are recombined into target filtering data. The target filtered data is deduplicated to obtain the deduplicated data.
5. The privacy-protected data filtering method according to claim 4, characterized in that, The data indication information is obtained based on the deduplicated data, including: Sort the deduplicated data and select a preset number of the deduplicated data as data to be confirmed. Based on the data source of the data to be confirmed, data indication information corresponding to the data home device is generated. The data indication information is used to instruct the data home device to send the second data segment corresponding to the data to be confirmed to the data user device.
6. A method for filtering privacy-preserving data, characterized in that, Applied to data-using devices, the method includes: Acquire training data, pre-train a large language model using the training data to obtain the entropy value corresponding to each training data, use the training data and the corresponding entropy value as input data, construct a proxy model based on the input data, and send the proxy model to at least one data ownership device. Receive at least one first data segment sent from at least one of the data home devices, receive at least one second data segment sent from at least one of the data home devices based on corresponding data indication information, and reconstruct the first data segment and the second data segment according to the mapping relationship to obtain at least one target data; After fine-tuning the large language model using the target data, the fine-tuning entropy value of the training data is obtained. The input data is then updated based on the fine-tuning entropy value, and the proxy model is updated as well.
7. The privacy-protected data filtering method according to claim 6, characterized in that, The process of generating the data indication information includes the following steps: All the first data slices and all the second data slices from the target home device are recombined into target filtering data based on the mapping relationship; The target filtering data is deduplicated to obtain the deduplicated data. The deduplicated data is sorted, and a preset number of the deduplicated data are selected as the data to be confirmed. Based on the data source of the data to be confirmed, data indication information corresponding to the data home device is generated. The data indication information is used to instruct the data home device to send the second data segment corresponding to the data to be confirmed to the data user device.
8. A privacy-protecting data filtering device, characterized in that, Applied to data attribution equipment, the device includes: Entropy prediction module: used to obtain the current proxy model from the data-using device, input the candidate privacy-preserving data into the proxy model to perform entropy prediction, and obtain the prediction mean and prediction variance corresponding to the candidate privacy-preserving data; Data splitting module: used to obtain a first number of candidate target data from the candidate privacy protection data according to the predicted mean and the predicted variance, generate a first data slice and a second data slice corresponding to each candidate target data, and send the first data slice to the data usage device; Data transmission module: used to acquire data indication information and send at least one second data slice corresponding to the candidate target data to the data usage device according to the data indication information.
9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the privacy-protected data filtering method according to any one of claims 1 to 7.
10. A storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the privacy-protected data filtering method according to any one of claims 1 to 7.