Text processing method, data processing method and equipment

Abstract

One or more embodiments of the invention provide a text processing method, a data processing method and equipment. The text processing method comprises the steps that multiple word replacement operations are conducted on an original text, a replacement text corresponding to each word replacement operation is obtained, and the word replacement operations indicate that at least one word in the original text is subjected to synonym replacement; determining semantic similarity between the original text and each replacement text; performing differential privacy processing on the semantic similarity between the original text and each replacement text; selecting at least one target replacement text from the plurality of replacement texts based on the semantic similarity after differential privacy processing corresponding to each replacement text; wherein the target replacement text is used for replacing the original text to participate in the task with the privacy protection requirement.

Description

Technical Field

[0001] One or more embodiments of the present specification relate to the field of computer software technology, and in particular, to a text processing method, a data processing method, an electronic device, a computer-readable storage medium, and a computer program product. Background Art

[0002] In today's information age, text analysis and processing technology plays an important role in the field of natural language processing. With the increasing awareness of data privacy protection, how to effectively protect user privacy during text analysis and processing has become one of the research hotspots.

[0003] Taking the application scenario of Large Language Model (LLM) as an example, large language models have achieved great success in recent years and have been widely used in various fields. Data privacy in the application process of large language models has also attracted attention. There are serious data privacy risks in the fine-tuning process of large language models. During fine-tuning, the downstream task party needs to use the data set provided by itself to fine-tune a large language model. Since fine-tuning a large language model has high requirements for computing resources, it is also difficult for downstream task parties with limited computing power. Such downstream task parties may choose to entrust a third-party organization to help them fine-tune the model, but this requires passing the data to a third party, which poses a risk of data privacy leakage. In addition, directly using the original data to fine-tune the large language model is also prone to the threat of black box attacks on the fine-tuned large language model. Summary of the invention

[0004] In view of this, one or more embodiments of the present specification provide a text processing method, a data processing method, an electronic device, a computer-readable storage medium, and a computer program product.

[0005] To achieve the above objectives, one or more embodiments of this specification provide the following technical solutions:

[0006] According to a first aspect of one or more embodiments of this specification, a text processing method is proposed, including:

[0007] Performing multiple word replacement operations on the original text to obtain replacement text corresponding to each word replacement operation, wherein the word replacement operation indicates replacing at least one word in the original text with a synonym;

[0008] Determining the semantic similarity between the original text and each of the replacement texts;

[0009] Performing differential privacy processing on the semantic similarity between the original text and each of the replacement texts;

[0010] Based on the semantic similarity after differential privacy processing corresponding to each of the replacement texts, at least one target replacement text is selected from the multiple replacement texts; wherein the target replacement text is used to replace the original text to participate in tasks with privacy protection requirements.

[0011] According to a second aspect of an embodiment of this specification, a data processing method is provided, including:

[0012] Performing multiple replacement operations on the original data item to obtain replacement data items corresponding to each replacement operation, wherein the replacement operation indicates performing an approximate replacement on at least one of all sub-data items included in the original data item;

[0013] Determining semantic similarity between the original data item and each of the replacement data items;

[0014] Performing differential privacy processing on the semantic similarity between the original data item and each of the replacement data items;

[0015] Based on the semantic similarity after differential privacy processing corresponding to each of the replacement data items, at least one target replacement data item is selected from the multiple replacement data items; wherein the target replacement data item is used to replace the original data item to participate in tasks with privacy protection requirements.

[0016] According to a third aspect of the embodiments of this specification, there is provided an electronic device, including:

[0017] processor;

[0018] a memory for storing processor-executable instructions;

[0019] When the processor executes the executable instructions, it is used to implement the method described in the first aspect or the second aspect.

[0020] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the steps of the method described in the first aspect or the second aspect are implemented.

[0021] According to a fifth aspect of the embodiments of this specification, a computer program product is provided, including a computer program, which implements the steps of the method described in the first aspect or the second aspect when executed by a processor.

[0022] The technical solutions provided by the embodiments of this specification may have the following beneficial effects:

[0023] The text processing method provided in the embodiments of this specification can perform multiple word replacement operations on the original text to obtain replacement text corresponding to each word replacement operation. The generation of multiple replacement texts can provide a variety of expressions, thereby reducing the risk of directly exposing the original text, and then determine the semantic similarity between the original text and each replacement text. Considering that directly using the semantic similarity determined above may expose sensitive information, the semantic similarity is processed by differential privacy to effectively reduce the risk of the replacement text leaking the original content, thereby effectively protecting the sensitive information of the original text. Finally, based on the semantic similarity after differential privacy processing corresponding to each replacement text, at least one target replacement text is selected from the multiple replacement texts. The target replacement text can be used to replace the original text to participate in tasks with privacy protection requirements, thereby ensuring that user privacy is not exposed during the task.

[0024] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present specification. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a flowchart of a text processing method provided by an exemplary embodiment.

[0026] Figure 2 It is a schematic diagram of a text processing flow provided by an exemplary embodiment.

[0027] Figure 3 It is a schematic diagram of a replacement text classification provided by an exemplary embodiment.

[0028] Figure 4 It is a flowchart of a data processing method provided by an exemplary embodiment.

[0029] Figure 5 It is a schematic structural diagram of an electronic device provided by an exemplary embodiment. DETAILED DESCRIPTION

[0030] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with one or more embodiments of this specification. Instead, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0031] It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0032] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this manual are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0033] Based on the privacy leakage problem in text analysis and processing scenarios, the embodiments of this specification provide a text processing method, which can be executed by electronic devices, including but not limited to servers, server clusters, virtual servers, PCs (Personal Computers), tablet devices, laptops, PDAs (Personal Digital Assistants), smart phones / mobile phones, etc. Please refer to Figure 1 as well as Figure 2 , the text processing method comprises:

[0034] In S101, multiple word replacement operations are performed on the original text to obtain replacement text corresponding to each word replacement operation, and the word replacement operation indicates that at least one word in the original text is replaced with a synonym.

[0035] This step performs multiple word replacement operations on the original text. Each operation selects one or more words in the original text and replaces them with synonyms to generate different replacement texts. Generating multiple replacement texts can provide a variety of expressions, thereby reducing the risk of directly exposing the original text. In addition, multiple replacement texts also provide more options for privacy protection processing, and the most suitable replacement version can be selected in subsequent steps. For example, see Figure 2 , the original text is subjected to X word replacement operations to obtain X replacement texts, where X is an integer greater than 0.

[0036] Exemplarily, the original text may be obtained from a preset text set, and the original text may be randomly extracted from the text set multiple times to perform the text processing method provided in the embodiment of this specification, but the method is not limited thereto.

[0037] Here is an example of the process of selecting at least one word in the original text:

[0038] In one possible implementation, the word replacement operation may instruct to randomly select at least one word in the original text for synonym replacement, so that the generated replacement text has higher diversity. Because the vocabulary selection is random, the replacement content may also cover multiple types of vocabulary, which is suitable for scenarios that require diversified replacement to protect privacy.

[0039] In another possible implementation, the word replacement operation may indicate the replacement of at least one word in the original text that belongs to a specified category; the specified category includes at least one of the following: nouns, verbs, and vocabulary related to a specific subject or field, but is not limited thereto. Replacing nouns in the original text can change the subject or content body of the text; replacing verbs in the original text will affect the action or intention of the sentence, which is suitable for changing the behavior description in the original text; replacing vocabulary related to a specific subject or field (such as medicine, law, computers, etc.) in the original text can hide privacy-sensitive subjects while retaining the semantic information of the original text. This embodiment ensures that the replacement operation is only performed on certain categories of words, thereby effectively retaining the overall semantics and structure of the text. By specifying categories, sensitive words can be selectively replaced, making the replacement process more targeted.

[0040] Take the medical field as an example:

[0041] Suppose the original text is "The patient was diagnosed with acute pneumonia, and the doctor recommended antibiotic treatment with azithromycin, supplemented with oxygen therapy and bed rest."

[0042] Replacement text 1 is "The patient was diagnosed with respiratory tract infection, and the doctor recommended the use of antibiotics for treatment, supplemented by oxygen inhalation and bed rest." Among them, replacement text 1 replaces "acute pneumonia" with "respiratory tract infection" and "azithromycin" with the general term "antibiotics". This replacement text blurs the specific symptoms and drug names, but retains the treatment method.

[0043] Replacement text 2 is "The patient suffers from lung infection. The doctor recommends the use of specific anti-infective drugs, and recommends oxygen inhalation and bed rest." Among them, replacement text 2 replaces "acute pneumonia" with the broader "lung infection" and "azithromycin" with "anti-infective drugs", further blurring the symptoms and drugs, making the text more suitable for application scenarios that require high privacy protection.

[0044] Replacement text 3 is "The patient has respiratory problems, and the doctor recommends the use of anti-infective drugs for treatment, supplemented by supportive care." Replacement text 3 replaces "acute pneumonia" with "respiratory problems," "azithromycin" with "anti-infective drugs," and "oxygen therapy and bed rest" with the broader "supportive care." This replacement text widely obscures specific symptoms and treatment methods and is used in scenarios with extremely high privacy needs.

[0045] Replacement text 4 is "The patient is diagnosed with acute respiratory infection, and the doctor recommends the use of anti-infective drugs and respiratory care." Among them, replacement text 4 replaces "acute pneumonia" with "acute respiratory infection", "azithromycin" with "anti-infective drugs", and "oxygen therapy" with "respiratory care". Replacement text 4 still retains certain medical terms, but removes specific drug information, which is suitable for use under general privacy protection needs.

[0046] Replacement text 5 is "The patient has symptoms of respiratory infection, and the doctor recommends the use of appropriate medications and enhanced care." Among them, replacement text 5 replaces specific symptoms and medications with more vague statements, using "appropriate medications" and "enhanced care" instead of specific treatment methods.

[0047] After determining the words to be replaced from the original text, the following is an exemplary description of the process of selecting synonyms corresponding to the words to be replaced:

[0048] In a possible implementation, the electronic device can determine at least one word to be replaced from the original text according to any of the above strategies, and then for each word to be replaced, obtain a synonym set corresponding to the word to be replaced, and the words in this set are semantically similar to the original word; finally, randomly select a synonym from the synonym set corresponding to the word to be replaced to replace the word to be replaced. The replacement process described in this embodiment is simple and direct, and the word replacement process can be quickly performed without complex semantic calculations.

[0049] For example, the original text is: "The doctor recommends that the patient take antihypertensive drugs every day and pay attention to a light diet." Assume that "antihypertensive drugs" is determined as the word to be replaced. Obtain a synonym set, for example, the synonym set includes "antihypertensive drugs", "blood pressure control drugs", etc. The electronic device randomly selects one of the synonyms (assuming it is "antihypertensive drugs") for replacement to form a replacement text. Assume that the replacement text is: "The doctor recommends that the patient take antihypertensive drugs every day and pay attention to a light diet."

[0050] In another possible implementation, the electronic device can determine at least one word to be replaced from the original text according to any of the above strategies, and then for each word to be replaced, obtain a set of synonyms corresponding to the word to be replaced, and determine the semantic similarity between the word to be replaced and each synonym in the synonym set; it can be understood that the embodiment of this specification does not impose any restrictions on the specific calculation method of semantic similarity, and can be specifically set according to the actual application scenario; for example, a trained semantic similarity classification model (such as a BERT model) can be used to score word pairs, and the higher the score, the more similar the semantics. Then the electronic device performs differential privacy processing on the semantic similarity between the word to be replaced and each synonym in the synonym set, so as to use the result after differential privacy processing as the selection probability of each synonym; finally, a synonym is selected from the synonym set according to the selection probability of each synonym to replace the word to be replaced. In this embodiment, the semantic similarity after differential privacy processing actually becomes the probabilistic basis for selecting synonyms. The semantic similarity score is protected by differential privacy to avoid leakage of true similarity information. Moreover, the selection probability is based on semantic similarity, which makes the replacement more semantically consistent and retains the accuracy of the original information.

[0051] For example, suppose we want to replace the word "fast" and find the following synonym set S = {fast, fast, quick, agile}, and calculate each synonym (denoted by w i , 0<i≤4) (denoted as u(w i )) are as follows: fast: 0.9; fast: 0.8; swift: 0.7; agile: 0.6. The selection probability of each synonym is calculated by the exponential mechanism (the exponential mechanism is a differential privacy method specifically used to select discrete options, which guides the selection by defining the score of each option). Assuming that the privacy budget parameter ∈ is 0.5 and the sensitivity Δu is 1, the initial selection probability of each synonym is calculated by the following formula Then the initial selection probability of each synonym is: P(quickly) P(fast)≈1.221, P(quick)≈1.191, P(agile)≈1.162.

[0052] Then, the initial selection probability of each synonym is normalized to determine the selection probability of each synonym. The selection probability of each synonym can reflect the probability distribution of synonyms in the synonym set, and we can get P(rapid)≈0.259; P(fast)≈0.253; P(swift)≈0.247; P(agile)≈0.241. Then, we can randomly sample a synonym according to the probability distribution. Assume that a weight-based sampling method is used to randomly generate a number between 0 and 1: (1) If the generated number is in the interval [0,0.259), select "rapid"; (2) If the generated number is in the interval [0.259,0.512), select "fast"; (3) If the generated number is in the interval [0.512,0.759), select "swift"; (4) If the generated number is in the interval [0.759,1], select "agile". Assume that the generated random number is 0.3, which falls in the interval [0.259, 0.512), and the electronic device selects "Fast" as the replacement result. In this example, after the probability distribution is calculated through the exponential mechanism, each replacement result will be generated based on the probability distribution, making high-scoring words more likely to be selected, and each selection has a certain degree of randomness, achieving privacy protection while ensuring replacement quality.

[0053] In some embodiments, considering that there is no correlation between different word replacement operations, there is a situation where two replacement texts generated by any two word replacement operations on the same original text are the same. Then, after performing multiple word replacement operations on the same original text, the multiple replacement texts corresponding to the multiple word replacement operations can be deduplicated to avoid the duplicate replacement texts affecting the subsequent replacement text selection process.

[0054] In summary, the above word replacement operation can ensure that some words in the original text are replaced in a controllable manner, replace sensitive information in the original text, and retain the semantic information of the original text as much as possible. This ensures that the replacement text has similar semantics to the original text, while avoiding the leakage of sensitive information and ensuring the usability of the replacement text.

[0055] In S102, the semantic similarity between the original text and each replacement text is determined.

[0056] In this step, please also refer to Figure 2 ,Through the calculation of semantic similarity, the semantic closeness between each ,replacement text and the original text can be determined to ensure the ,semantic accuracy of the replacement text, which provides basic data for the ,subsequent differential privacy processing steps.

[0057] It is understandable that the embodiments of this specification do not impose any restrictions on the specific calculation method of semantic similarity, and can be specifically set according to actual application scenarios.

[0058] For example, the semantic similarity between the original text and each replacement text may be determined by a trained semantic similarity classification model, and the semantic similarity classification model may be a semantic similarity two-classification model, a semantic similarity three-classification model, or a semantic similarity N-classification model, where N≥4, and so on.

[0059] The semantic similarity binary classification model can be used to determine whether the original text and the replacement text are semantically similar, usually outputting two categories: "similar" or "dissimilar". This model is simple and direct, and is suitable for tasks with clear similarity requirements.

[0060] The semantic similarity three-classification model can add a "partial similarity" category on the basis of the two-classification model, allowing the replacement text to have partial semantic consistency with the original text. This classification can capture the semantic relationship between the replacement word and the original word more finely, and is suitable for scenarios that require differentiation of partial similarity.

[0061] The semantic similarity N-classification model (N≥4) can provide a more fine-grained semantic similarity classification. The N-classification model divides the similarity between the replacement text and the original text into multiple levels or semantic categories. This method is suitable for application scenarios that require more accurate quantification of semantic similarity.

[0062] For another example, the semantic similarity between the original text and each replacement text may be measured by using the cosine similarity based on the word vectors. The cosine similarity of the word vectors of the original text and the word vectors of the replacement text may be calculated.

[0063] In one possible implementation, for each replacement text, the electronic device inputs a text pair consisting of the original text and the replacement text into a trained semantic similarity binary classification model, obtains a first probability value indicating the degree of similarity and a second probability value indicating the degree of dissimilarity output by the semantic similarity binary classification model; and then determines the semantic similarity between the original text and the replacement text based on the probability difference between the first probability value and the second probability value.

[0064] In another possible implementation, for each replacement text, the electronic device inputs the text pair consisting of the original text and the replacement text into a trained semantic similarity N classification model (N≥4), obtains N probability values ​​corresponding to N categories (assuming that the N categories include "completely similar", "partially similar", "slightly similar", "dissimilar", etc.) output by the semantic similarity N classification model, and determines the highest probability value among the N probability values ​​as the semantic similarity between the original text and the replacement text.

[0065] In S103, the semantic similarity between the original text and each replacement text is subjected to differential privacy processing.

[0066] In this step, see Figure 2,Considering that directly using the semantic similarity data determined by S102 may expose sensitive information, differential privacy is performed on the semantic similarity data.,Differential privacy can effectively prevent the leakage of individual information while providing useful information, and effectively protect the sensitive information of the original text by perturbing the semantic similarity.,That is to say, by processing semantic similarity with differential privacy, the risk of the replacement text leaking the original content can be effectively reduced.

[0067] Exemplarily, the electronic device may determine the differential privacy noise to be added, and then add the differential privacy noise to the semantic similarity between the original text and each replacement text.

[0068] Among them, the electronic device can determine the noise intensity of the differential privacy noise based on the ratio between the specified sensitivity of the semantic similarity and the specified privacy budget parameter; then sample the differential privacy noise from the Laplace distribution or the Gaussian distribution based on the noise intensity of the differential privacy noise. In differential privacy, sensitivity is used to measure the maximum impact of the output of a query (or function) on a single data point; specifically, it represents the maximum possible change in the query output when only one data point is changed; the higher the sensitivity, the stronger the dependence of the query on a single data, and therefore stronger noise is required to protect privacy. The privacy budget is used to quantify the degree of data privacy protection. In differential privacy, the intensity of added noise is positively correlated with sensitivity and negatively correlated with the privacy budget.

[0069] Of course, other mechanisms may also be used to determine the differential privacy noise, and this embodiment is not limited to this.

[0070] Exemplarily, the semantic similarity between the original text and each replacement text is determined by a trained semantic similarity classification model. Then, in the differential privacy processing process, after determining the differential privacy noise to be added, for each replacement text, the electronic device can scale the semantic similarity corresponding to the replacement text based on the difference between the value range of the semantic similarity obtained by the semantic similarity classification model and the value range of the specified sensitivity indication, and then obtain the semantic similarity after differential privacy processing corresponding to the replacement text based on the sum of the scaled semantic similarity and the differential privacy noise. In this embodiment, the scaling process causes the value range of the semantic similarity to fluctuate within the specified sensitivity range, thereby reducing the risk of the sensitivity exceeding the control range. The scaled semantic similarity is more adaptable to the addition of differential privacy noise, so that the sensitivity is always maintained within the preset range, effectively improving the control of the differential privacy mechanism.

[0071] For example, assuming that the semantic similarity classification model includes a semantic similarity binary classification model, the specified sensitivity of the semantic similarity is set to 1, the value range of the semantic similarity obtained by the semantic similarity binary classification model is [-1,1], and the value range of the specified sensitivity indication is [-0.5,0.5]. Therefore, it is necessary to scale the semantic similarity corresponding to the replacement text, that is, divide the semantic similarity corresponding to the replacement text by two, and add differential privacy noise to the semantic similarity after scaling. Through scaling, the semantic similarity is adjusted to the range of [-0.5,0.5], so that the effect of the differential privacy noise is within a controllable range, which helps to reduce the excessive deviation caused by the addition of noise, avoid excessive influence of noise on the original similarity, and make the similarity after privacy processing more stable.

[0072] In S104, at least one target replacement text is selected from the multiple replacement texts based on the semantic similarity after differential privacy processing corresponding to each replacement text; wherein the target replacement text is used to replace the original text to participate in the task with privacy protection requirements.

[0073] In this step, see Figure 2 , based on the semantic similarity score after differential privacy processing, at least one replacement text can be selected from multiple replacement texts as the target replacement text. The selection criteria can be set according to specific task requirements, and this embodiment does not impose any restrictions on this.

[0074] For example, see Figure 3 ( Figure 3 (where Y is an integer greater than 1), after obtaining the semantic similarity after differential privacy processing corresponding to each replacement text, the electronic device can classify multiple replacement texts based on the semantic similarity after differential privacy processing corresponding to each replacement text; wherein the difference between the semantic similarity after differential privacy processing corresponding to any two replacement texts belonging to the same category is less than the difference between the semantic similarity after differential privacy processing corresponding to any two replacement texts belonging to different categories. This embodiment effectively ensures that the texts in each category have similar semantic features by classifying replacement texts with close semantic similarity into the same category; in this way, the selected target replacement text can better maintain semantic consistency with the original text, thereby improving the usability of the replacement text. Moreover, the above-mentioned differential privacy processing will introduce noise, thereby causing fluctuations in the semantic similarity value. Dividing the similarity after noise processing into multiple categories can effectively reduce the selection bias caused by noise, thereby ensuring the stability of classification and the rationality of subsequent random selection.

[0075] It is understandable that this embodiment does not impose any restrictions on the specific classification algorithm of the above application, and specific settings can be made according to actual application scenarios.

[0076] For example, the electronic device can perform local sensitive hashing on the semantic similarity after differential privacy processing corresponding to each replacement text to map it into a hash bucket; wherein any two replacement texts belonging to the same category are mapped to the same hash bucket, and any two replacement texts belonging to different categories are mapped to different hash buckets. Local sensitive hashing is a technology that can map similar data to the same bucket. Since local sensitive hashing can ensure that similar data is mapped to the same bucket in a high-dimensional space, it can efficiently group and retain semantic similarity, which is particularly applicable in scenarios with large amounts of data or complex similarity distributions. Moreover, the hash bucket after local sensitive hashing will not directly expose the specific value of the similarity, further enhancing privacy protection. Even if differential privacy noise causes individual numerical deviations, similar replacement texts can still be classified into the same category after local sensitive hashing to ensure consistency in replacement text selection.

[0077] For another example, the electronic device can use a preset series of piecewise functions to process the semantic similarity after differential privacy processing corresponding to each replacement text, so as to divide multiple replacement texts into different intervals, each interval representing a different level of semantic similarity; wherein, any two replacement texts belonging to the same category are mapped to the same interval, and any two replacement texts belonging to different categories are mapped to different intervals. This embodiment uses a custom piecewise function to divide the replacement text into more detailed categories according to application requirements. For example, in an interval with a similarity of [0,1], it can be divided into multiple sub-intervals, such as 0-0.25, 0.25-0.5, 0.5-0.75, and 0.75-1, to achieve more accurate text replacement selection.

[0078] In short, whether it is the local sensitive hashing or the piecewise function method, both effectively improve the rationality of classification and ensure that in the environment after differential privacy processing, the replacement content can be selected from the semantically more similar text. Of course, other known classification algorithms in the relevant technology can also be used, and this embodiment does not limit this.

[0079] For example, see Figure 3 After classifying the multiple replacement texts, the electronic device randomly selects at least one target replacement text from the replacement text set corresponding to at least one category. The target replacement text selection method can be specifically set according to the actual application scenario, and this embodiment does not impose any limitation on this.

[0080] For example, one or more replacement texts can be randomly selected from each category as the target replacement text, which can ensure that replacement texts from different categories have the opportunity to be selected, increasing the diversity of the target replacement text. Randomly selecting replacement texts from multiple categories can effectively reduce the risk of inferring the original text, which helps to enhance text privacy. This random selection method is suitable for a variety of application scenarios and does not rely on the high similarity of a specific category, thus providing a wider range of text selection.

[0081] For another example, one or more replacement texts can be selected from categories with relatively high similarity as target replacement texts. Selecting replacement texts from high-similarity categories can ensure that the replacement content is more semantically consistent with the original text, which is suitable for scenarios that require high consistency in text content. In addition, texts in high-similarity categories still retain high semantic consistency after differential privacy processing, so this method is more effective in balancing privacy and text quality.

[0082] For example, one or more replacement texts may be selected from categories with relatively low similarity as target replacement texts. Selecting texts with relatively low similarity can effectively reduce the recognizability of the original text, thereby better protecting the privacy of users and reducing the risk of leaking sensitive information.

[0083] The text processing method provided by the embodiments of this specification has low computational overhead, and the semantic similarity calculation process can use a simple structured model (such as a semantic similarity binary classification model), and does not need to be trained, only reasoning is required. The entire processing process can be conveniently deployed locally on the electronic device and is easy to use.

[0084] The target replacement text obtained by the embodiments of this specification can replace the original text to participate in tasks with privacy protection needs. For example, using replacement text in medical research or data sharing tasks can not only convey the patient's need to pay attention to health, but also protect specific medical record information. For another example, in social media data analysis tasks, when performing sentiment analysis, the use of replacement text can effectively protect the user's privacy and avoid leaking sensitive personal emotional states. For another example, in market research tasks, using replacement text in research reports can maintain the effectiveness of product feedback while avoiding directly quoting customers' negative emotions.

[0085] The various technical features in the above embodiments can be arbitrarily combined as long as there is no conflict or contradiction between the combinations of features. However, due to space limitations, they are not described one by one. Therefore, any combination of the various technical features in the above embodiments also falls within the scope of this specification.

[0086] In some embodiments, considering that in addition to the privacy protection needs in the text analysis and processing field, other data fields (such as image analysis and processing field, audio analysis and processing field, video analysis and processing field, etc.) also have privacy protection needs, the ideas of vocabulary replacement and selection in the above text processing method are also applicable to other data fields. Therefore, please refer to Figure 4 The embodiment of this specification also provides a data processing method, which can be executed by an electronic device, and the method includes:

[0087] In S401, multiple replacement operations are performed on the original data item to obtain replacement data items corresponding to each replacement operation, where the replacement operation indicates to perform an approximate replacement on at least one of all sub-data items included in the original data item.

[0088] In this step, the original data item undergoes multiple replacement operations to generate corresponding replacement data items. The replacement operation involves approximate replacement of at least one sub-data item in the original data, which can effectively blur the data content, protect user privacy, and ensure data availability.

[0089] In one implementation, the original data item includes at least one of the following types: original text, original audio, original image, and original video. In the original text, the sub-data item in the original data item is a word. In the original audio, the sub-data item in the original data item is an audio segment. In the original image, the sub-data item in the original data item is a pixel. In the original video, the sub-data item in the original data item is at least one of an image frame and an audio segment.

[0090] In S402, the semantic similarity between the original data item and each replacement data item is determined.

[0091] In this step, the semantic similarity between the original data item and each replacement data item is calculated to evaluate how close the replacement data item is to the original data item in terms of semantics. This process helps ensure that the replacement data item can still be useful in a specific task while maintaining privacy.

[0092] In S403, the semantic similarity between the original data item and each replacement data item is subjected to differential privacy processing.

[0093] In this step, the semantic similarity between the original data item and the replacement data item is differentially private, which ensures that any analysis of the similarity will not leak individual data, enhancing the security of data processing. By processing semantic similarity with differential privacy noise, the risk of the replacement data item leaking the original content can be effectively reduced.

[0094] In S404, at least one target replacement data item is selected from the multiple replacement data items based on the semantic similarity after differential privacy processing corresponding to each replacement data item; wherein the target replacement data item is used to replace the original data item to participate in the task with privacy protection requirements.

[0095] In this step, at least one target replacement data item is selected from multiple replacement data items based on the semantic similarity after differential privacy processing. This target data item will be used instead of the original data item in tasks with privacy protection requirements to ensure that the data does not expose user privacy during the analysis process.

[0096] In this embodiment, the above steps can effectively protect personal privacy and prevent data leakage, while maintaining the semantic integrity of the data. It is suitable for a variety of scenarios that require privacy protection, such as medical data analysis, social media data processing, etc., ensuring that both data security and practicality are given equal importance.

[0097] In one implementation, determining the semantic similarity between an original data item and each replacement data item includes: for each replacement data item, inputting a data item pair consisting of the original data item and the replacement data item into a trained semantic similarity binary classification model, obtaining a first probability value indicating a degree of similarity and a second probability value indicating a degree of dissimilarity output by the semantic similarity binary classification model; and determining the semantic similarity between the original data item and the replacement data item based on a probability difference between the first probability value and the second probability value.

[0098] In one implementation, the semantic similarity between the original data item and each replacement data item is differentially privately processed, including: determining the differential privacy noise to be added; and adding the differential privacy noise to the semantic similarity between the original data item and each replacement data item.

[0099] In one implementation, the semantic similarity between the original data item and each replacement data item is determined by a trained semantic similarity classification model. The noise intensity of the differential privacy noise is determined based on a ratio between a specified sensitivity of the semantic similarity and a specified privacy budget parameter.

[0100] Adding differential privacy noise to the semantic similarity between the original data item and each replacement data item includes: for each replacement data item, scaling the semantic similarity corresponding to the replacement data item based on the difference between the value range of the semantic similarity obtained by the semantic similarity classification model and the value range of the specified sensitivity indication; and obtaining the semantic similarity after differential privacy processing corresponding to the replacement data item based on the sum of the semantic similarity after scaling and the differential privacy noise.

[0101] In one implementation, the semantic similarity classification model includes a semantic similarity binary classification model, the specified sensitivity of the semantic similarity is 1, the value range of the semantic similarity obtained by the semantic similarity binary classification model is [-1, 1], and the value range of the specified sensitivity indication is [-0.5, 0.5].

[0102] Scaling the semantic similarity corresponding to the replacement data item includes: dividing the semantic similarity corresponding to the replacement data item by two.

[0103] In one implementation, the replacement operation indicates replacing at least one sub-data item belonging to a specified category in the original data item.

[0104] In one implementation, performing a replacement operation on an original data item includes: determining at least one sub-data item to be replaced from the original data item; for each sub-data item to be replaced, obtaining a set of approximate items corresponding to the sub-data item to be replaced, and randomly selecting an approximate item from the set of approximate items corresponding to the sub-data item to be replaced to replace the sub-data item to be replaced.

[0105] In one implementation, a replacement operation is performed on an original data item, including: determining at least one sub-data item to be replaced from the original data item; for each sub-data item to be replaced, obtaining a set of approximate items corresponding to the sub-data item to be replaced, and determining the semantic similarity between the sub-data item to be replaced and each approximate item in the approximate item set; performing differential privacy processing on the semantic similarity between the sub-data item to be replaced and each approximate item in the approximate item set, so as to use the result after the differential privacy processing as the selection probability of each approximate item; and selecting an approximate item from the approximate item set according to the selection probability of each approximate item to replace the sub-data item to be replaced.

[0106] In one implementation, before determining the semantic similarity between the original data item and each replacement data item, the method further includes: performing deduplication processing on a plurality of replacement data items respectively corresponding to the plurality of replacement operations.

[0107] In one implementation, at least one target replacement data item is selected from multiple replacement data items based on the semantic similarity after differential privacy processing corresponding to each replacement data item, including: classifying the multiple replacement data items based on the semantic similarity after differential privacy processing corresponding to each replacement data item; wherein the difference between the semantic similarities after differential privacy processing corresponding to any two replacement data items belonging to the same category is smaller than the difference between the semantic similarities after differential privacy processing corresponding to any two replacement data items belonging to different categories; and randomly selecting at least one target replacement data item from a set of replacement data items corresponding to at least one category.

[0108] In one implementation, multiple replacement data items are classified based on the semantic similarity after differential privacy processing corresponding to each replacement data item, including: performing local sensitive hashing on the semantic similarity after differential privacy processing corresponding to each replacement data item to map it into a hash bucket; wherein any two replacement data items belonging to the same category are mapped to the same hash bucket, and any two replacement data items belonging to different categories are mapped to different hash buckets.

[0109] The various technical features in the above embodiments can be arbitrarily combined as long as there is no conflict or contradiction between the combinations of features. However, due to space limitations, they are not described one by one. Therefore, any combination of the various technical features in the above embodiments also falls within the scope of this specification.

[0110] In some embodiments, the embodiments of this specification also provide an electronic device, including: a processor; a memory for storing processor executable instructions; wherein the processor implements any of the above methods by running the executable instructions.

[0111] Figure 5 is a schematic structural diagram of a device provided by an exemplary embodiment. Figure 5 At the hardware level, the device includes a processor 502, an internal bus 504, a network interface 506, a memory 508, and a non-volatile memory 510, and may also include hardware required for other functions. One or more embodiments of this specification may be implemented based on software, such as the processor 502 reading the corresponding computer program from the non-volatile memory 510 into the memory 508 and then running it. Of course, in addition to the software implementation, one or more embodiments of this specification do not exclude other implementations, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0112] In some embodiments, the text processing device can be applied to Figure 5 The device shown in the figure is used to implement the technical solution of this specification. The text processing device may include:

[0113] The word replacement operation module is used to perform multiple word replacement operations on the original text to obtain replacement text corresponding to each word replacement operation. The word replacement operation indicates that at least one word in the original text is replaced with a synonym.

[0114] The semantic similarity determination module is used to determine the semantic similarity between the original text and each replacement text.

[0115] The differential privacy processing module is used to perform differential privacy processing on the semantic similarity between the original text and each replacement text.

[0116] A replacement text selection module is used to select at least one target replacement text from multiple replacement texts based on the semantic similarity after differential privacy processing corresponding to each replacement text; wherein the target replacement text is used to replace the original text to participate in tasks with privacy protection requirements.

[0117] In one implementation, the semantic similarity determination module is specifically used to: for each replacement text, input the text pair consisting of the original text and the replacement text into a trained semantic similarity binary classification model, obtain the first probability value indicating the degree of similarity and the second probability value indicating the degree of dissimilarity output by the semantic similarity binary classification model; based on the probability difference between the first probability value and the second probability value, determine the semantic similarity between the original text and the replacement text.

[0118] In one implementation, the differential privacy processing module is specifically used to determine the differential privacy noise to be added; and add the differential privacy noise for the semantic similarity between the original text and each replacement text.

[0119] In one implementation, the semantic similarity between the original text and each replacement text is determined by a trained semantic similarity classification model; the noise intensity of the differential privacy noise is determined based on the ratio between the specified sensitivity of the semantic similarity and the specified privacy budget parameter.

[0120] The differential privacy processing module is specifically used to scale the semantic similarity corresponding to the replacement text for each replacement text based on the difference between the value range of the semantic similarity obtained by the semantic similarity classification model and the value range of the specified sensitivity indication; based on the sum of the semantic similarity after scaling and the differential privacy noise, obtain the semantic similarity after differential privacy processing corresponding to the replacement text.

[0121] In one implementation, the semantic similarity classification model includes a semantic similarity binary classification model, the specified sensitivity of the semantic similarity is 1, the value range of the semantic similarity obtained by the semantic similarity binary classification model is [-1, 1], and the value range of the specified sensitivity indication is [-0.5, 0.5]. The differential privacy processing module is specifically used to divide the semantic similarity corresponding to the replacement text by two.

[0122] In one implementation, the word replacement operation indicates replacing at least one word belonging to a specified category in the original text; wherein the specified category includes at least one of the following: nouns, verbs, and vocabulary related to a specific topic or field.

[0123] In one implementation, performing a word replacement operation on the original text includes:

[0124] Determining at least one word to be replaced from the original text;

[0125] For each word to be replaced, a synonym set corresponding to the word to be replaced is obtained, and a synonym is randomly selected from the synonym set corresponding to the word to be replaced to replace the word to be replaced.

[0126] In one implementation, the word replacement operation module is specifically used to determine at least one word to be replaced from the original text; for each word to be replaced, obtain a synonym set corresponding to the word to be replaced, and determine the semantic similarity between the word to be replaced and each synonym in the synonym set; perform differential privacy processing on the semantic similarity between the word to be replaced and each synonym in the synonym set, so as to use the result after the differential privacy processing as the selection probability of each synonym; select a synonym from the synonym set according to the selection probability of each synonym to replace the word to be replaced.

[0127] In one implementation, the text processing apparatus further includes a deduplication module for performing deduplication processing on a plurality of replacement texts corresponding to the plurality of word replacement operations.

[0128] In one implementation, the replacement text selection module is specifically used to classify multiple replacement texts based on the semantic similarity after differential privacy processing corresponding to each replacement text; wherein the difference between the semantic similarity after differential privacy processing corresponding to any two replacement texts belonging to the same category is smaller than the difference between the semantic similarity after differential privacy processing corresponding to any two replacement texts belonging to different categories; and randomly select at least one target replacement text from a set of replacement texts corresponding to at least one category.

[0129] In one implementation, the replacement text selection module is specifically used to perform local sensitive hashing on the semantic similarity after differential privacy processing corresponding to each replacement text to map it into a hash bucket; wherein, any two replacement texts belonging to the same category are mapped to the same hash bucket, and any two replacement texts belonging to different categories are mapped to different hash buckets.

[0130] In some embodiments, the embodiments of this specification further provide a data processing device, including:

[0131] A replacement operation module, used to perform multiple replacement operations on the original data item to obtain replacement data items corresponding to each replacement operation, wherein the replacement operation indicates to perform an approximate replacement on at least one of all the sub-data items included in the original data item;

[0132] A semantic similarity determination module, used to determine the semantic similarity between the original data item and each replacement data item;

[0133] A differential privacy processing module, used to perform differential privacy processing on the semantic similarity between the original data item and each replacement data item;

[0134] A replacement data item selection module is used to select at least one target replacement data item from multiple replacement data items based on the semantic similarity after differential privacy processing corresponding to each replacement data item; wherein the target replacement data item is used to replace the original data item to participate in tasks with privacy protection requirements.

[0135] The implementation process of the functions and effects of each module in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, which will not be repeated here.

[0136] Based on the same concept as the above method, this specification also provides a computer-readable storage medium on which computer instructions are stored. When the instructions are executed by a processor, the steps of the method described in any of the above embodiments are implemented.

[0137] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0138] Based on the same concept as the above method, this specification also provides a computer program product, including a computer program / instruction, which implements the steps of the method described in any of the above embodiments when executed by a processor.

[0139] The above description is merely a preferred embodiment of one or more embodiments of the present specification and is not intended to limit one or more embodiments of the present specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of one or more embodiments of the present specification shall be included in the scope of protection of one or more embodiments of the present specification.

Claims

1. A text processing method, comprising: Performing multiple word replacement operations on the original text to obtain replacement text corresponding to each word replacement operation, wherein the word replacement operation indicates replacing at least one word in the original text with a synonym; Determining the semantic similarity between the original text and each of the replacement texts; Performing differential privacy processing on the semantic similarity between the original text and each of the replacement texts; Based on the semantic similarity after differential privacy processing corresponding to each of the replacement texts, at least one target replacement text is selected from the multiple replacement texts; wherein the target replacement text is used to replace the original text to participate in tasks with privacy protection requirements.

2. The method according to claim 1, wherein determining the semantic similarity between the original text and each of the replacement texts comprises: For each replacement text, input a text pair consisting of the original text and the replacement text into a trained semantic similarity binary classification model to obtain a first probability value indicating a similarity degree and a second probability value indicating a dissimilarity degree output by the semantic similarity binary classification model; Based on the probability difference between the first probability value and the second probability value, a semantic similarity between the original text and the replacement text is determined.

3. According to the method of claim 1, the step of performing differential privacy processing on the semantic similarity between the original text and each of the replacement texts comprises: Determine the differential privacy noise to be added; The differential privacy noise is added to the semantic similarity between the original text and each of the replacement texts.

4. The method according to claim 3, wherein the semantic similarity between the original text and each of the replacement texts is determined by a trained semantic similarity classification model; The noise intensity of the differential privacy noise is determined based on a ratio between a specified sensitivity of the semantic similarity and a specified privacy budget parameter; The adding the differential privacy noise to the semantic similarity between the original text and each of the replacement texts includes: For each replacement text, scaling the semantic similarity corresponding to the replacement text based on the difference between the value range of the semantic similarity obtained by the semantic similarity classification model and the value range indicated by the specified sensitivity; Based on the sum of the semantic similarity after scaling and the differential privacy noise, the semantic similarity after differential privacy processing corresponding to the replacement text is obtained.

5. According to the method of claim 4, the semantic similarity classification model comprises a semantic similarity binary classification model, the specified sensitivity of the semantic similarity is 1, the value range of the semantic similarity obtained by the semantic similarity binary classification model is [-1, 1], and the value range of the specified sensitivity indication is [-0.5, 0.5]; The scaling process of the semantic similarity corresponding to the replacement text includes: The semantic similarity corresponding to the replacement text is divided by two.

6. The method according to claim 1, wherein the word replacement operation indicates replacing at least one word belonging to a specified category in the original text; wherein, The designated category includes at least one of the following: nouns, verbs, and words related to a specific topic or field; and / or Before determining the semantic similarity between the original text and each of the replacement texts, the method further includes: Deduplication processing is performed on the multiple replacement texts corresponding to the multiple word replacement operations.

7. The method according to claim 1, performing a word replacement operation on the original text, comprising: Determining at least one word to be replaced from the original text; For each of the words to be replaced, a synonym set corresponding to the word to be replaced is obtained, and a synonym is randomly selected from the synonym set corresponding to the word to be replaced to replace the word to be replaced; or, Perform word replacement operations on the original text, including: Determining at least one word to be replaced from the original text; For each of the words to be replaced, a synonym set corresponding to the word to be replaced is obtained, and a semantic similarity between the word to be replaced and each synonym in the synonym set is determined; Performing differential privacy processing on the semantic similarity between the word to be replaced and each synonym in the synonym set, so as to use the result after the differential privacy processing as the selection probability of each synonym; A synonym is selected from the synonym set according to the selection probability of each synonym to replace the word to be replaced.

8. The method according to claim 1, wherein the step of selecting at least one target replacement text from the plurality of replacement texts based on the semantic similarity after differential privacy processing corresponding to each replacement text comprises: Based on the semantic similarities after differential privacy processing corresponding to each of the replacement texts, the multiple replacement texts are classified; wherein the difference between the semantic similarities after differential privacy processing corresponding to any two replacement texts belonging to the same category is smaller than the difference between the semantic similarities after differential privacy processing corresponding to any two replacement texts belonging to different categories; At least one target replacement text is randomly selected from a replacement text set corresponding to at least one category.

9. A data processing method, comprising: Performing multiple replacement operations on the original data item to obtain replacement data items corresponding to each replacement operation, wherein the replacement operation indicates performing an approximate replacement on at least one of all sub-data items included in the original data item; Determining semantic similarity between the original data item and each of the replacement data items; Performing differential privacy processing on the semantic similarity between the original data item and each of the replacement data items; Based on the semantic similarity after differential privacy processing corresponding to each of the replacement data items, at least one target replacement data item is selected from the multiple replacement data items; wherein the target replacement data item is used to replace the original data item to participate in tasks with privacy protection requirements.

10. An electronic device, comprising: processor; A memory for storing processor-executable instructions; wherein the processor implements the steps of the method according to any one of claims 1 to 9 by executing the executable instructions.

Images