Text processing method and device, computer equipment and storage medium

Abstract

The invention relates to a text processing method and device, computer equipment, a storage medium and a computer program product. The method can be applied to the field of artificial intelligence. The method comprises the steps of obtaining a target text and a text type of the target text; the text type comprises normal and abnormal; performing punctuation mark perturbation on the target text according to multiple perturbation modes to obtain multiple candidate samples; processing each candidate sample through a defense model to obtain an abnormal score of each candidate sample; determining an adversarial sample from the candidate samples according to the abnormal score, and determining a label of the adversarial sample according to the text type of the target text; the adversarial samples and the corresponding labels are used for retraining the defense model. The method can improve the efficiency of obtaining the adversarial sample.

Description

Technical Field

[0001] The present application relates to the field of computer technologies, and in particular, to a text processing method, apparatus, computer device, storage medium, and computer program product. Background Art

[0002] Natural language processing models need a large amount of labeled data for training. Training natural language models with adversarial samples can improve the in-depth understanding of semantics by natural language processing models. In related technologies, perturbations at the word level or sentence level are usually performed on texts to generate adversarial samples. However, for perturbations at the word level or sentence level, it is necessary to determine the perturbed words or sentences through complex calculations, and to determine synonyms or synonymous sentences through semantic analysis for replacing the perturbed words or sentences, which takes a long time and results in low efficiency in obtaining adversarial samples. Summary of the Invention

[0003] Based on this, in view of the above technical problems, it is necessary to provide a text processing method, apparatus, computer device, computer-readable storage medium, and computer program product that can improve the efficiency of obtaining adversarial samples.

[0004] In a first aspect, the present application provides a text processing method. The method includes:

[0005] Obtaining a target text and the text type of the target text; the text type includes normal and abnormal; performing punctuation perturbations on the target text according to multiple perturbation methods to obtain multiple candidate samples; respectively processing each candidate sample through a defense model to obtain the abnormal score of each candidate sample; determining an adversarial sample from the candidate samples according to the abnormal score, and determining the label of the adversarial sample according to the text type of the target text; the adversarial sample and the corresponding label are used to retrain the defense model.

[0006] In a second aspect, the present application further provides a text processing apparatus. The apparatus includes:

[0007] An obtaining module, configured to obtain a target text and the text type of the target text; the text type includes normal and abnormal;

[0008] A perturbation module, configured to perform punctuation perturbations on the target text according to multiple perturbation methods to obtain multiple candidate samples;

[0009] A first determination module, configured to respectively process each candidate sample through a defense model to obtain the abnormal score of each candidate sample;

[0010] A second determination module, configured to determine an adversarial sample from the candidate samples according to the abnormal score, and determine the label of the adversarial sample according to the text type of the target text; the adversarial sample and the corresponding label are used to retrain the defense model.

[0011] In some embodiments, the multiple perturbation methods include at least two of an insertion method, a displacement method, a deletion method, or a replacement method.

[0012] In some embodiments, the candidate sample includes a first candidate sample; the perturbation module is further configured to obtain at least one preset punctuation mark; and insert each preset punctuation mark into the target text to obtain the first candidate sample.

[0013] In some embodiments, the candidate sample includes a second candidate sample; in the case where the perturbation method includes a displacement method, the perturbation module is further configured to obtain a first position where the original punctuation mark is located in the target text; move the original punctuation mark from the first position to a second position in the target text to obtain the second candidate sample; the second position is different from the first position.

[0014] In some embodiments, the candidate sample includes a third candidate sample; in the case where the perturbation method includes a deletion method, the perturbation module is further configured to delete at least one original punctuation mark in the target text to obtain the third candidate sample.

[0015] In some embodiments, the candidate sample includes a fourth candidate sample; in the case where the perturbation method includes a replacement method, the perturbation module is further configured to determine a punctuation mark to be processed from the original punctuation marks in the target text; and replace the punctuation mark to be processed with a punctuation mark different from the punctuation mark to be processed to obtain the fourth candidate sample.

[0016] In some embodiments, the first determination module is further configured to, for any candidate sample, construct input data based on the candidate sample targeted; perform feature extraction on the input data through a defense model to obtain a feature vector corresponding to the candidate sample targeted, and perform classification based on the feature vector to obtain an anomaly score of the candidate sample targeted.

[0017] In some embodiments, the first determination module is further configured to, for any candidate sample, compare the target text with the candidate sample targeted to obtain the perturbed punctuation mark and the perturbation position of the candidate sample targeted; and determine the input data corresponding to the candidate sample targeted based on the target text, the perturbed punctuation mark of the candidate sample targeted, and the perturbation position.

[0018] In some embodiments, the first determination module is further configured to perform feature extraction on the target text and the perturbed punctuation mark through a defense model to obtain a text vector and a perturbed punctuation mark vector; determine a perturbation position vector based on the text vector and the perturbation position; and fuse the text vector, the perturbed punctuation mark vector, and the perturbation position vector to obtain a feature vector corresponding to the candidate sample targeted.

[0019] In some embodiments, the first determination module is further configured to fuse the text vector of the target text and the feature vector of the targeted candidate sample to obtain a fused vector; perform a classification process on the fused vector to obtain the anomaly score of the targeted candidate sample.

[0020] In some embodiments, the second determination module is further configured to, when the target text is normal text, use the candidate samples whose anomaly scores belong to the first interval as adversarial samples and use the normal type as the label of the adversarial samples; when the target text is abnormal text, use the candidate samples whose anomaly scores belong to the second interval as adversarial samples and use the abnormal type as the label of the adversarial samples; wherein, the minimum value of the first interval is greater than the maximum value of the second interval.

[0021] In some embodiments, the text processing device further includes a defense module, configured to, in response to a text sending request, obtain the text to be processed in the text sending request; process the text to be processed through a retrained defense model to obtain a prediction score; intercept the text sending request when the prediction score meets the defense condition.

[0022] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0023] Obtain a target text and the text type of the target text; the text type includes normal and abnormal; perform punctuation perturbation on the target text according to multiple perturbation methods to obtain multiple candidate samples; process each candidate sample through a defense model to obtain the anomaly score of each candidate sample; determine adversarial samples from the candidate samples according to the anomaly scores, and determine the labels of the adversarial samples according to the text type of the target text; the adversarial samples and the corresponding labels are used to retrain the defense model.

[0024] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0025] Obtain a target text and the text type of the target text; the text type includes normal and abnormal; perform punctuation perturbation on the target text according to multiple perturbation methods to obtain multiple candidate samples; process each candidate sample through a defense model to obtain the anomaly score of each candidate sample; determine adversarial samples from the candidate samples according to the anomaly scores, and determine the labels of the adversarial samples according to the text type of the target text; the adversarial samples and the corresponding labels are used to retrain the defense model.

[0026] Fifth aspect, the present application also provides a computer program product. The computer program product includes a computer program which, when executed by a processor, implements the following steps:

[0027] Obtain a target text and the text type of the target text; the text type includes normal and abnormal; perform punctuation perturbation on the target text according to multiple perturbation methods to obtain multiple candidate samples; process each candidate sample through a defense model to obtain the abnormal score of each candidate sample; determine an adversarial sample from the candidate samples based on the abnormal score, and determine the label of the adversarial sample according to the text type of the target text; the adversarial sample and the corresponding label are used to retrain the defense model.

[0028] The above text processing method, device, computer device, storage medium and computer program product perform punctuation perturbation on the target text according to multiple perturbation methods to obtain multiple candidate samples, process the candidate samples through a defense model to obtain the abnormal scores of the candidate samples, select the adversarial samples that the defense model fails to correctly identify from the candidate samples based on the abnormal scores, and determine the labels of the adversarial samples according to the text type of the target text, so as to retrain the defense model through the adversarial samples and the corresponding labels; performing punctuation-level perturbation on the target text is simple in operation, and the perturbation process does not require calculation, greatly saving the time required to generate adversarial samples and improving the efficiency of obtaining adversarial samples; in addition, retraining the defense model through the adversarial samples and the corresponding labels improves the recognition ability of the retrained defense model for the text after punctuation perturbation, and thus improves the defense ability of the retrained defense model. Description of the Drawings

[0029] Figure 1 It is an application environment diagram of the text processing method in an embodiment;

[0030] Figure 2 It is a flowchart of the text processing method in an embodiment;

[0031] Figure 3 It is a flowchart of determining whether a candidate sample is an adversarial sample in an embodiment;

[0032] Figure 4 It is a flowchart of determining whether a candidate sample is an adversarial sample in another embodiment;

[0033] Figure 5 It is a schematic diagram of performing punctuation perturbation by the insertion method in an embodiment;

[0034] Figure 6 It is a schematic diagram of performing punctuation perturbation by the displacement method in an embodiment;

[0035] Figure 7 Schematic diagram of punctuation perturbation by deletion in an embodiment;

[0036] Figure 8 Another schematic diagram of punctuation perturbation by deletion in an embodiment;

[0037] Figure 9 Schematic diagram of punctuation perturbation by replacement in an embodiment;

[0038] Figure 10 Flow schematic diagram of four stages included in the text processing method in a scenario embodiment;

[0039] Figure 11 Flow schematic diagram of the text processing method in another embodiment;

[0040] Figure 12 Structural block diagram of a text processing device in an embodiment;

[0041] Figure 13 Internal structure diagram of a computer device in an embodiment. Detailed implementation manners

[0042] In order to make the objectives, technical solutions, and advantages of this application clearer, the following further elaborates on this application in combination with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not used to limit this application.

[0043] Natural Language Processing (NLP) is an important direction in the fields of computer science and artificial intelligence. It studies various theories and methods that can achieve effective communication between humans and computers in natural language. Natural language processing involves natural language, that is, the language people use daily, and is closely related to linguistics research; at the same time, it involves computer science and mathematics. The pre-trained model, an important technology for model training in the field of artificial intelligence, is developed from the large language model (LLM) in the NLP field. After fine-tuning, the large language model can be widely applied to downstream tasks. Natural language processing technologies generally include text processing, semantic understanding, machine translation, robot question answering, knowledge graphs, and other technologies.

[0044] The text processing method provided by the embodiments of this application can be applied to, for example Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through a network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or can be placed on the cloud or other network servers; the text processing method can be executed by the terminal 102 or the server 104, or can be executed collaboratively by the terminal 102 and the server 104.

[0045] Taking the text processing method executed by the terminal 102 as an example, the terminal 102 can obtain the target text and the text type of the target text; the text type includes normal and abnormal; the terminal 102 can perform punctuation perturbation on the target text according to multiple perturbation methods to obtain multiple candidate samples; the terminal 102 can process each candidate sample through a defense model respectively to obtain the abnormal score of each candidate sample; the terminal 102 can also determine the adversarial sample from the candidate samples according to the abnormal score, and determine the label of the adversarial sample according to the text type of the target text; the adversarial sample and the corresponding label are used to retrain the defense model.

[0046] Among them, the terminal 102 can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, an Internet of Things device, and a portable wearable device. The Internet of Things device can be a smart speaker, a smart TV, a smart air conditioner, and a smart vehicle-mounted device, etc. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, etc.

[0047] The server 104 can be an independent physical server, or can be a service node in a blockchain system, and a peer-to-peer network is formed among the service nodes in the blockchain system.

[0048] In addition, the server 104 can also be a server cluster composed of multiple physical servers, and can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms.

[0049] The connection between the terminal 102 and the server 104 can be made through communication connection methods such as Bluetooth, USB (Universal Serial Bus), or network, and this application does not make any restrictions here.

[0050] In some embodiments, as Figure 2 shown, a text processing method is provided, and this method is executed by Figure 1 the server or the terminal in Figure 1The server and the terminal in Figure 1 cooperate and execute. Taking the example that this method is executed by a computer device, the computer device can be

[0051] Step 202: Obtain the target text and the text type of the target text; the text type includes normal and abnormal.

[0052] Among them, the target text can be a statement, specifically a Chinese statement or an English statement; the text type is used to reflect whether the target text includes sensitive information, and the sensitive information can be information that misleads users, may affect user safety or social safety; the target text of the normal type does not include sensitive information, and the target text of the abnormal type includes sensitive information.

[0053] In some embodiments, the computer device obtains the target text, processes the target text through a defense model, and obtains the text type of the target text.

[0054] The defense model can be used to detect whether the text includes sensitive information, and the defense model can be obtained by pre-training an initial defense model with a training sample set.

[0055] In practical applications, perform word-level perturbations on the preset text to obtain the first training sample, and train the initial defense model based on the first training sample to obtain the defense model; or perform sentence-level perturbations on the preset text to obtain the second training sample, and train the initial defense model based on the second training sample to obtain the defense model; or train the initial defense model based on the first training sample and the second training sample to obtain the defense model.

[0056] Specifically, the computer device inputs the target text into the defense model, extracts the text representation of the target text through the defense model, and performs classification processing on the text representation to obtain the probability value of the target text. If the probability value belongs to the first interval, it is determined that the text type of the target text is the abnormal type. If the probability value belongs to the second interval, it is determined that the text type of the target text is the normal type.

[0057] For the first probability value belonging to the normal type and the second probability value belonging to the abnormal type, select the higher probability value from the first probability value and the second probability value, and use the text type corresponding to the selected probability value as the text type of the target text.

[0058] Exemplarily, input the target text into the defense model, and the probability value of the target text obtained through the defense model is 0.3, it can be determined that the target text belongs to the normal type.

[0059] Step 204: Perform punctuation perturbations on the target text according to multiple perturbation methods to obtain multiple candidate samples.

[0060] Among them, the perturbation method is used to perturb the punctuation marks of the target text, that is, to adjust the target text through punctuation marks; punctuation marks include but are not limited to: Chinese punctuation marks, English punctuation marks and numerical punctuation marks; exemplarily, punctuation marks include: Chinese comma, English comma, Chinese period, English period, Chinese ellipsis, English ellipsis, percent sign, brackets, pound sign, asterisk, etc.

[0061] The perturbation method can be: adding punctuation marks, deleting punctuation marks, replacing punctuation marks, moving punctuation marks, etc.; the candidate sample is obtained after the punctuation marks are perturbed on the target text; for example, the perturbation method is adding symbols, and the computer device selects a punctuation mark and adds the punctuation mark to the target text to obtain a candidate sample.

[0062] In some embodiments, the plurality of perturbation modes include at least two of an insertion mode, a displacement mode, a deletion mode, or a replacement mode.

[0063] Among them, the insertion method refers to inserting punctuation marks in the target text, such as inserting a comma between the first word and the second word of the target text; the displacement method refers to moving the position of punctuation marks in the target text, such as moving a semicolon between the first word and the second word of the target text to between the third word and the fourth word of the target text; the deletion method refers to deleting punctuation marks in the target text, such as deleting a comma between the first word and the second word of the target text; the replacement method refers to replacing punctuation marks in the target text with other punctuation marks, such as replacing a comma between the first word and the second word of the target text with a semicolon; the above-mentioned words in the target text can also be single words.

[0064] Exemplarily, the multiple perturbation modes include at least two of the insertion mode, the displacement mode, the deletion mode or the replacement mode, which means that the multiple perturbation modes may include two perturbation modes, or three perturbation modes, or all perturbation modes among the insertion mode, the displacement mode, the deletion mode or the replacement mode.

[0065] For example, multiple perturbation modes may include insertion mode and displacement mode, and multiple perturbation modes may include displacement mode, deletion mode and replacement mode, and multiple perturbation modes may also include insertion mode, displacement mode, deletion mode and replacement mode.

[0066] In some embodiments, the multiple candidate samples may include one or more candidate samples for each perturbation method, that is, one or more candidate samples for a perturbation method can be obtained through a perturbation method; for example, the multiple perturbation methods include an insertion method and a displacement method. The punctuation of the target text is perturbed by the insertion method to obtain one or more first candidate samples, and the punctuation of the target text is perturbed by the displacement method to obtain one or more second candidate samples.

[0067] In some embodiments, the multiple candidate samples may include multiple candidate samples corresponding to multiple perturbations. Each perturbation may be to perturb the punctuation of the target text sequentially through at least two perturbation methods; for example, the multiple perturbation methods include an insertion method and a displacement method. In one perturbation, the punctuation of the target text can be perturbed by the insertion method to obtain an intermediate sample, and the punctuation of the intermediate sample can be perturbed by the displacement method to obtain a candidate sample. After multiple perturbations, multiple candidate samples are obtained.

[0068] In the above embodiments, the punctuation of the target text is perturbed in different ways by various perturbation methods. Using multiple perturbation methods to perturb the punctuation of the target text improves the richness and diversity of the obtained candidate samples, so that when the defense model is retrained later, the defense ability of the defense model can be improved.

[0069] Step 206: Process each candidate sample through the defense model to obtain the anomaly score of each candidate sample.

[0070] The anomaly score is used to represent the probability that the candidate sample detected by the defense model includes sensitive information; the larger the anomaly score, the greater the probability that the defense model believes that the candidate sample includes sensitive information, and the smaller the anomaly score, the smaller the probability that the defense model believes that the candidate sample includes sensitive information.

[0071] The computer device inputs the candidate sample into the defense model, and the defense model outputs the anomaly score of the candidate sample. Specifically, for each candidate sample, the computer device inputs the candidate sample to be targeted into the defense model, the defense model extracts features of the candidate sample to be targeted to obtain the candidate sample representation of the candidate sample to be targeted, and classifies the candidate sample representation to obtain the probability value of the candidate sample to be targeted, and this probability value is the anomaly score.

[0072] It should be noted that the defense model can be used to detect whether the text includes sensitive information. The multiple candidate samples are obtained by perturbing the punctuation of the target text according to multiple perturbation methods, and the multiple candidate samples are different. The defense model may have different understandings of different candidate samples, and thus the anomaly scores determined by the defense model for each candidate sample may be different.

[0073] Step 208: Determine adversarial samples from the candidate samples according to the anomaly scores, and determine the labels of the adversarial samples according to the text type of the target text; the adversarial samples and the corresponding labels are used to retrain the defense model.

[0074] Among them, the adversarial sample is a candidate sample for which the defense model has a detection error. The defense model having a detection error means that the detection result obtained by the defense model for detecting the target sample is different from the detection result obtained by detecting the candidate sample corresponding to the target sample.

[0075] Specifically, in the case where the target text includes sensitive information, the candidate samples obtained by perturbing the punctuation of the target text also include sensitive information. When the defense model detects the target text, it is determined that the target text includes sensitive information, while the anomaly score obtained by detecting the candidate sample indicates that the candidate sample does not include sensitive information, which means that the defense model has a detection error.

[0076] Correspondingly, in the case where the target text does not include sensitive information, the candidate samples obtained by perturbing the punctuation of the target text also do not include sensitive information. When the defense model detects the target text, it is determined that the target text does not include sensitive information, while the anomaly score obtained by detecting the candidate sample indicates that the candidate sample includes sensitive information, which means that the defense model has a detection error.

[0077] Based on the text type of the target text and the anomaly scores of the candidate samples, the computer device can determine whether there is an error in the detection of the candidate samples by the defense model. If there is an error, the candidate sample is used as an adversarial sample, and the text type of the target text is used as the label of the adversarial sample. The defense model is retrained with the adversarial sample and the label of the adversarial sample.

[0078] In some embodiments, determining adversarial samples from the candidate samples according to the anomaly scores and determining the labels of the adversarial samples according to the text type of the target text includes: when the target text is normal text, the candidate samples with anomaly scores belonging to the first interval are used as adversarial samples, and the normal type is used as the label of the adversarial samples; when the target text is abnormal text, the candidate samples with anomaly scores belonging to the second interval are used as adversarial samples, and the abnormal type is used as the label of the adversarial samples; wherein, the minimum value of the first interval is greater than the maximum value of the second interval.

[0079] Among them, the target text being normal text means that the text type of the target text is the normal type, and the target text being abnormal text means that the text type of the target text is the abnormal type.

[0080] Specifically, the computer device obtains the text type of the target text. When the text type is the normal type, it determines that the target text is a normal text, selects adversarial samples with abnormal scores belonging to the first interval from multiple candidate samples, and uses the normal type as the label of the adversarial samples; when the text type of the target text is the abnormal type, it determines that the target text is an abnormal text, selects adversarial samples with abnormal scores belonging to the second interval from multiple candidate samples, and uses the abnormal type as the label of the adversarial samples. Among them, the values of the first interval and the second interval can be set according to actual needs, and the embodiments of the present application do not limit the values of the first interval and the second interval.

[0081] Exemplarily, the first interval is [0.7, 1], and the second interval is [0, 0.7); as Figure 3 shown, the text type of the target text is the normal type, and the abnormal score of candidate sample k1 is 0.1, that is, the abnormal score of candidate sample k1 does not belong to the first interval, so candidate sample k1 is not an adversarial sample; the abnormal score of candidate sample k2 is 0.8, that is, the abnormal score of candidate sample k2 belongs to the first interval, so candidate sample k2 is an adversarial sample.

[0082] As Figure 4 shown, the text type of the target text is the abnormal type, the abnormal score of candidate sample k3 is 0.5, that is, the abnormal score of candidate sample k3 belongs to the second interval, so candidate sample k3 is an adversarial sample; the abnormal score of candidate sample k4 is 0.9, that is, the abnormal score of candidate sample k4 does not belong to the second interval, so candidate sample k4 is not an adversarial sample.

[0083] In some embodiments, the process of retraining the defense model includes: inputting the adversarial samples into the defense model, determining the predicted abnormal scores of the adversarial samples through the defense model, determining the loss value based on the labels and predicted abnormal scores of the adversarial samples, and adjusting the defense model according to the loss value until the defense model converges to obtain the retrained defense model.

[0084] Among them, the specific method for determining the loss value based on the labels and predicted abnormal scores of the adversarial samples can be set according to actual needs. For example, it can be to calculate the cross-entropy loss value between the labels and predicted abnormal scores of the adversarial samples. The embodiments of the present application do not limit the specific method for determining the loss value based on the labels and predicted abnormal scores of the adversarial samples.

[0085] It should be noted that when the target text is a normal text, the adversarial samples selected from the candidate samples can be used as positive samples for retraining the defense model. When the target text is an abnormal sample, the adversarial samples selected from the candidate samples can be used as negative samples for retraining the adversarial defense model. By retraining the defense model with positive samples, when the text contains sensitive information, after perturbing the punctuation of the text, the retrained defense model can still identify that the perturbed text contains sensitive information, improving the defense model's ability to identify sensitive information. By retraining the defense model with negative samples, when the text does not contain sensitive information, after perturbing the punctuation of the text, the retrained defense model can still identify that the perturbed text does not contain sensitive information, avoiding overfitting of the defense model.

[0086] In the related art, keywords in the text can be determined, and the keywords can be replaced with synonyms (perturbation at the word level), or key sentences in the text can be determined, and the key sentences can be replaced with synonymous sentences (perturbation at the sentence level). The perturbation operations at the word level and the sentence level are complex. When determining keywords and synonyms, or when determining key sentences and synonymous sentences, a large amount of calculation is required, which takes a long time and results in low efficiency in obtaining adversarial samples.

[0087] In the above text processing method, the punctuation of the target text is perturbed in multiple perturbation ways to obtain multiple candidate samples. The candidate samples are processed by the defense model to obtain the anomaly scores of the candidate samples. Adversarial samples that the defense model fails to correctly identify are selected from the candidate samples according to the anomaly scores, and the labels of the adversarial samples are determined according to the text type of the target text, so that the defense model can be retrained with the adversarial samples and the corresponding labels. Perturbing the punctuation at the punctuation level for the target text is simple, and no calculation is required during the perturbation process, greatly saving the time required to generate adversarial samples and improving the efficiency of obtaining adversarial samples. In addition, by retraining the defense model with the adversarial samples and the corresponding labels, the recognition ability of the retrained defense model for the text after punctuation perturbation is improved, and thus the defense ability of the retrained defense model is improved.

[0088] In some embodiments, the candidate samples include first candidate samples. When the perturbation method includes the insertion method, perturbing the punctuation of the target text includes: obtaining at least one preset punctuation; inserting each preset punctuation into the target text to obtain the first candidate samples.

[0089] Among them, the preset punctuation can be Chinese punctuation, English punctuation or numerical punctuation, and the number of preset punctuation can be one or more; the first candidate samples are obtained by perturbing the punctuation of the target text by the insertion method; the number of the first candidate samples can be one or more.

[0090] Specifically, the computer device can randomly obtain at least one preset punctuation mark from the punctuation mark set, determine the insertion position corresponding to the preset punctuation mark in the target text, and insert at least one preset punctuation mark into its corresponding insertion position to obtain a first candidate sample.

[0091] The insertion position can be the position between two adjacent words in the target text. That is to say, a preset punctuation mark can be inserted between two adjacent words without punctuation marks in the target text to obtain a first candidate sample; it should be noted that the insertion position corresponding to the preset punctuation mark can also be randomly determined in the target text.

[0092] In some embodiments, inserting each preset punctuation mark into the target text includes the following specific insertion methods:

[0093] (1) The number of preset punctuation marks is one, and the number of insertion positions is one; through one perturbation, insert the preset punctuation mark into the insertion position to obtain a first candidate sample.

[0094] (2) The number of preset punctuation marks is one, and the number of insertion positions is multiple; each perturbation inserts the preset punctuation mark into one insertion position to obtain a first candidate sample corresponding to that perturbation, and through multiple perturbations, obtain multiple first candidate samples.

[0095] (3) The number of preset punctuation marks is one, and the number of insertion positions is multiple; through one perturbation, insert the preset punctuation mark into multiple insertion positions respectively to obtain a first candidate sample corresponding to that perturbation, and through multiple perturbations, obtain multiple first candidate samples.

[0096] (4) The number of preset punctuation marks is multiple, and the number of insertion positions is multiple, including multiple insertion positions corresponding to each preset punctuation mark; each perturbation inserts one preset punctuation mark into one of its corresponding insertion positions to obtain a first candidate sample corresponding to that perturbation, and through multiple perturbations, obtain multiple first candidate samples.

[0097] (5) The number of preset punctuation marks is multiple, and the number of insertion positions is multiple, including multiple insertion positions corresponding to each preset punctuation mark; each perturbation inserts one preset punctuation mark into multiple insertion positions to obtain multiple first candidate samples corresponding to that perturbation; through multiple perturbations, obtain multiple first candidate samples.

[0098] (6) The number of preset punctuation marks is multiple, and the number of positions to be inserted is multiple, including the positions to be inserted corresponding to each of the multiple preset punctuation marks; through one perturbation, the multiple preset punctuation marks are respectively inserted into their corresponding positions to be inserted, obtaining a first candidate sample.

[0099] Exemplarily, as Figure 5 shown, the second insertion method described above is illustrated; the target text is: "Most people probably consider, even though the courts didn't actually find, Klaus guilty of murder." The preset punctuation mark is a comma, and the multiple positions to be inserted include a first position to be inserted and a second position to be inserted. Inserting the comma into the first position to be inserted in the target text, the obtained first candidate sample is: "Most ,people probably consider, even though the courts didn't actually find, Klaus guilty of murder."; inserting the comma into the second position to be inserted in the target text, the obtained first candidate sample is: "Most people probably consider, even though the, courts didn't actually find, Klaus guilty of murder.".

[0100] Exemplarily, the fourth insertion method described above is illustrated; the target text is as shown in the above example, the preset punctuation marks include a comma and a percent sign, and the multiple positions to be inserted include a first position to be inserted and a second position to be inserted. Four first candidate samples can be obtained according to the fourth method above. Among them, inserting the comma into the first position to be inserted in the target text, a first candidate sample is obtained; inserting the comma into the second position to be inserted in the target text, a first candidate sample is obtained (see the above example); inserting the percent sign into the first position to be inserted in the target text, the obtained first candidate sample is: "Most%people probably consider, even though the courts didn't actually find, Klaus guilty of murder."; inserting the percent sign into the second position to be inserted in the target text, the obtained first candidate sample is: "Most people probably consider, even though the%courts didn't actually find, Klaus guilty of murder.".

[0101] In the above embodiments, punctuation perturbation is performed on the target text in an insertion manner to obtain a first candidate sample. Compared with the related art in which candidate samples are generated by synonym replacement of words, performing punctuation perturbation on the target text in an insertion manner does not require determining important words and synonyms, thus improving the efficiency of generating candidate samples. In addition, the first candidate sample has additional preset punctuation marks compared with the target text, which may cause a deviation in the semantic understanding of the first candidate sample by the defense model. Retraining the defense model based on the first candidate sample can improve the semantic understanding ability of the defense model.

[0102] In some embodiments, the candidate sample includes a second candidate sample. When the perturbation method includes a displacement method, performing punctuation perturbation on the target text includes: obtaining a first position where the original punctuation mark in the target text is located; moving the original punctuation mark from the first position to a second position in the target text to obtain a second candidate sample; the second position is different from the first position.

[0103] Wherein, the original punctuation mark is the punctuation mark included in the target text, and the displacement method refers to moving the position of the original punctuation mark in the target text; the second candidate sample is obtained by performing punctuation perturbation on the target text in the displacement method.

[0104] In one scenario, the number of original punctuation marks is one. The computer device obtains the first position where the original punctuation mark is located in the target text, obtains each adjacent word group in the target text, determines the position between two words in each adjacent word group, and selects a second position different from the first position among the determined positions. When the number of second positions is one, the computer device moves the original punctuation mark at the first position to the second position to obtain a second candidate sample. When the number of second positions is multiple, the computer device moves the original punctuation mark at the first position to the second position through one perturbation to obtain the second candidate sample corresponding to this perturbation, and through multiple perturbations, moves the original punctuation mark at the first position to multiple second positions respectively to obtain multiple second candidate samples.

[0105] Exemplarily, such as Figure 6As shown, taking the number of the second position as one for example, the target text is: "Most people probably consider, even though the courts didn't actually find, Klaus guilty of murder." Moving the original punctuation mark ",", from the first position to the second position, the second candidate sample obtained is: "Most people probably consider even though the courts didn't, actually find, Klaus guilty of murder."

[0106] In another scenario, when the number of original punctuation marks is multiple, for each original punctuation mark in the target text, the computer device can perturb the original punctuation mark in the same unique way as in the previous scenario to obtain one or more second candidate samples corresponding to the original punctuation mark; in this way, one or more second candidate samples corresponding to each of the multiple original punctuation marks can be obtained.

[0107] In the above embodiment, by perturbing the punctuation marks of the target text in the displacement manner to obtain the second candidate samples, moving the positions of the original punctuation marks is simple in operation, and the candidate samples can be obtained quickly, which improves the efficiency of obtaining the candidate samples; in addition, compared with the target text, the second candidate samples change the positions of the original punctuation marks, which may cause a deviation in the semantic understanding of the second candidate samples by the defense model. Retraining the defense model based on the second candidate samples can improve the semantic understanding ability of the defense model.

[0108] In some embodiments, the candidate samples include third candidate samples; when the perturbation method includes the deletion method, perturbing the punctuation marks of the target text includes: deleting at least one original punctuation mark in the target text to obtain the third candidate samples.

[0109] Among them, the deletion method refers to deleting the original punctuation marks in the target text, and the third candidate samples are obtained by perturbing the punctuation marks of the target text in the deletion method.

[0110] Specifically, the computer device obtains the original punctuation marks in the target text. When the number of original punctuation marks is one, the original punctuation mark is deleted to obtain the third candidate samples; when the number of original punctuation marks is multiple, at least one of the multiple original punctuation marks is deleted to obtain the third candidate samples.

[0111] At least one of the multiple original punctuation marks is deleted, including multiple deletion methods. For example, by traversing the multiple original punctuation marks and perturbing to delete one original punctuation mark each time, multiple third candidate samples can be obtained through multiple perturbations. For example, any two original punctuation marks in the multiple original punctuation marks are combined to form an original punctuation mark group, and one original punctuation mark group is deleted each time through multiple perturbations to obtain multiple third candidate samples. For example, all the multiple original punctuation marks are deleted through one perturbation to obtain a third candidate sample.

[0112] Exemplarily, the target text is: "Most people probably consider, even though the courts didn't actually find, Klaus guilty of murder.", and the target text includes 4 punctuation marks, namely: ",", ",", "'", and ".", as Figure 7 shown, ",", and "," can be deleted, and the obtained third candidate sample is: "Most people probably consider even though the courts didn't actually find Klaus guilty of murder."; as Figure 8 shown, "'" can also be deleted, and the obtained third candidate sample is: "Most people probably consider, even though the courts didnt actually find, Klaus guilty of murder.".

[0113] In the above embodiment, the punctuation marks of the target text are perturbed by the deletion method to obtain third candidate samples. The position of deleting the original punctuation marks is simple and fast to operate, and candidate samples can be obtained quickly, improving the efficiency of obtaining candidate samples. In addition, compared with the target text, the third candidate samples have fewer original punctuation marks, which may cause the defense model to have a deviation in semantic understanding of the third candidate samples. Retraining the defense model based on the third candidate samples can improve the semantic understanding ability of the defense model.

[0114] [[ID=,13]]In some embodiments, the candidate samples include fourth candidate samples; in the case where the perturbation method includes the replacement method, perturbing the punctuation marks of the target text includes: determining the punctuation mark to be processed from the original punctuation marks of the target text; using a punctuation mark different from the punctuation mark to be processed to replace the punctuation mark to be processed to obtain a fourth candidate sample.

[0115] Among them, the fourth candidate sample is obtained by perturbing the punctuation marks of the target text in a replacement manner, and the replacement manner refers to replacing the original punctuation marks in the target text with other punctuation marks.

[0116] Specifically, the computer device obtains the original punctuation marks of the target text, and obtains at least one punctuation mark different from the original punctuation mark in the punctuation mark set. The punctuation mark different from the original punctuation mark can be a preset punctuation mark.

[0117] In a possible scenario, the number of original punctuation marks included in the target text is one. Obtain multiple punctuation marks different from the original punctuation mark. Through one perturbation, use one punctuation mark to replace the original punctuation mark to obtain one fourth candidate sample; through multiple perturbations, in order to achieve using multiple punctuation marks to replace the original punctuation mark in turn, multiple fourth candidate samples are obtained.

[0118] In another possible scenario, the number of original punctuation marks included in the target text is multiple. For each original punctuation mark, multiple punctuation marks different from the original punctuation mark can be obtained. In the same way as the previous scenario, the original punctuation mark targeted can be perturbed multiple times to obtain multiple fourth candidate samples corresponding to the original punctuation mark targeted.

[0119] Exemplarily, as Figure 9 shown, the target text is: "Most people probably consider, eventhough the courts didn't actually find, Klaus guilty of murder.", and the target text includes 4 original punctuation marks, namely: ",", ",", "'" and "."; taking the replacement of the original punctuation mark "'" as an example for illustration, obtain multiple punctuation marks different from the original punctuation mark: "!" and "="; use "!" to replace the "'" in the target text, and the obtained fourth candidate sample is: "Most people probably consider, even though the courts didn!tactually find, Klaus guilty of murder."; use "=" to replace the "'" in the target text, and the obtained fourth candidate sample is: "Most people probably consider, even though the courts didn=tactually find, Klaus guilty of murder."

[0120] In the above embodiments, the punctuation of the target text is perturbed by replacement to obtain the fourth candidate sample. The operations involved in replacing the original punctuation are simple, and the candidate sample can be obtained quickly, improving the efficiency of obtaining the candidate sample. In addition, the second candidate sample changes the position of the original punctuation compared with the target text, which may cause the defense model to have a deviation in semantic understanding of the second candidate sample. Retraining the defense model based on the second candidate sample can improve the semantic understanding ability of the defense model.

[0121] In some embodiments, the punctuation of the target text is perturbed according to multiple perturbation methods to obtain multiple candidate samples, including: traversing to obtain the target perturbation method among multiple perturbation methods, perturbing the punctuation of the target text according to the traversed target perturbation method to obtain multiple intermediate samples. When the traversal of multiple perturbation methods is completed, the multiple intermediate samples are used as multiple candidate samples. When the traversal of multiple perturbation methods is not completed, continue to traverse to obtain the next target perturbation method, and perturb the punctuation of the multiple intermediate samples according to the traversed target perturbation method to update the multiple intermediate samples until the traversal of multiple perturbation methods is completed, and the updated multiple intermediate samples are used as multiple candidate samples.

[0122] Exemplarily, the perturbation methods include displacement method and replacement method. The target text is: "Most people probably consider, even though the courts didn't actually find, Klaus guilty of murder." By perturbing the punctuation of the target text by the displacement method, the first intermediate sample obtained is: "Most people probably consider even though the courts didn't, actually find, Klaus guilty of murder."; by perturbing the punctuation of the first intermediate sample by the replacement method, the candidate sample obtained is: "Most people probably consider even though the courts didn=t, actually find, Klaus guilty of murder."

[0123] In the above embodiments, in one perturbation, multiple perturbation methods are used to perturb the punctuation of the same target text in sequence, which improves the difference between the candidate sample obtained by perturbation and the target text. By retraining the defense model based on the candidate sample, the semantic understanding ability of the defense model can be further improved.

[0124] In some embodiments, each candidate sample is processed by a defense model to obtain the anomaly score of each candidate sample, including: for any candidate sample, input data is constructed based on the candidate sample targeted; the defense model extracts features from the input data to obtain the feature vector corresponding to the candidate sample targeted, and classification is performed based on the feature vector to obtain the anomaly score of the candidate sample targeted.

[0125] In a possible scenario, the input data is the candidate sample targeted, that is, the candidate sample targeted is used as the input data. The computer device extracts the feature vector of the candidate sample targeted (input data) through the defense model, and then the defense model performs classification processing on the feature vector corresponding to the candidate sample targeted to obtain the anomaly score of the candidate sample targeted.

[0126] In another possible scenario, the input data can reflect the difference between the target text and the candidate sample targeted, or can represent the situation of punctuation perturbation to the target text; for example, when the perturbation method is the insertion method, the input data can be the preset punctuation to be inserted into the target text and the insertion position.

[0127] Specifically, for any candidate sample, constructing the input data based on the candidate sample targeted includes: for any candidate sample, comparing the target text with the candidate sample targeted to obtain the perturbed punctuation and the perturbation position of the candidate sample targeted; based on the target text, the perturbed punctuation and the perturbation position of the candidate sample targeted, determining the input data corresponding to the candidate sample targeted.

[0128] After determining the input data of the candidate sample targeted, the computer device extracts features from the input data of the candidate sample targeted through the defense model to obtain the feature vector of the candidate sample targeted, and then the defense model performs classification processing on the feature vector corresponding to the candidate sample targeted to obtain the anomaly score of the candidate sample targeted.

[0129] Among them, the perturbed punctuation is the punctuation that is different in the candidate sample compared with the target text, and the perturbation position is the position of the perturbed punctuation in the target text. For example, if the perturbed punctuation is in the first position of the text, the perturbation position is the 1st position in the text; the input data includes the target text, the perturbed punctuation and the perturbation position.

[0130] Specifically, when the perturbation method is the insertion method, the perturbed punctuation mark is the preset punctuation mark to be inserted, and the perturbation position is the position where the perturbed punctuation mark is inserted into the target text; when the perturbation method is the deletion method, the perturbed punctuation mark is the punctuation mark to be deleted, and the perturbation position is the position of the punctuation mark to be deleted in the target text; when the perturbation method is the replacement method, the perturbed punctuation mark is the punctuation mark used to replace the original punctuation mark, and the perturbation position is the position of the original punctuation mark in the target text; when the perturbation method is the displacement method, the perturbed punctuation mark is the original punctuation mark whose position has been moved, and the perturbation position is the position after the original punctuation mark is moved.

[0131] Taking the perturbation method as the insertion method as an example; exemplarily, inserting the preset punctuation mark between the first word (or single character) and the second word (or single character) of the target text to obtain a candidate sample. The computer device compares the candidate sample with the target text, determines that the perturbed symbol is the preset punctuation mark, and the perturbation position is the position where the perturbed symbol is inserted in the target text, that is, the perturbation position is the second position in the target text.

[0132] Comparing the target text with the candidate sample, the perturbed punctuation mark and the perturbation position of the candidate sample can be obtained. The input data includes the target text, the perturbed punctuation mark and the perturbation position, which can improve the efficiency of the subsequent defense model in processing feature extraction of the input data.

[0133] In the above embodiment, the defense model performs feature extraction and classification processing on the input data of the candidate sample to obtain the anomaly score of the candidate sample, so as to select the adversarial samples that the defense model fails to correctly identify from the candidate samples according to the anomaly score, and determine the labels of the adversarial samples according to the text type of the target text, so that the defense model can be retrained with the adversarial samples and the corresponding labels, and the defense ability of the retrained defense model can be improved.

[0134] In some embodiments, the defense model performs feature extraction on the input data to obtain the feature vector corresponding to the candidate sample to be targeted, including: performing feature extraction on the target text and the perturbed punctuation mark through the defense model to obtain a text vector and a perturbed punctuation mark vector; determining a perturbation position vector based on the text vector and the perturbation position; and fusing the text vector, the perturbed punctuation mark vector and the perturbation position vector to obtain the feature vector corresponding to the candidate sample to be targeted.

[0135] Among them, the text vector is obtained by performing feature extraction on the target text, and the perturbed punctuation mark vector is obtained by performing feature extraction on the perturbed punctuation mark.

[0136] Specifically, the defense model includes a feature extractor and a classifier. The computer device inputs the target text into the feature extractor to obtain a text vector, and inputs the perturbed punctuation into the feature extractor to obtain a perturbed punctuation vector.

[0137] In some embodiments, the candidate samples to be targeted are obtained by traversing multiple candidate samples corresponding to the target text, and the corresponding feature vectors of each traversed candidate sample are determined in sequence. When the candidate sample to be targeted is the first traversed candidate sample, the text vector obtained by the defense model for feature extraction of the target text is saved; when the candidate sample to be targeted is not the first traversed candidate sample, the text vector of the target text is obtained.

[0138] There are multiple candidate samples for the same target text. When determining the feature vectors of multiple candidate samples, the text vector of the target text is required. Therefore, the text vector extracted when determining the feature vector of the first candidate sample can be saved for determining the feature vectors of other candidate samples. In this way, the text vector of the target text can be reused in the process of determining the feature vectors of multiple candidate samples, improving the efficiency of determining the feature vectors of multiple candidate samples.

[0139] In some embodiments, the computer device obtains the feature vectors of each preset punctuation in the punctuation set, and obtains the perturbed punctuation vector of the perturbed punctuation from the feature vectors of each preset punctuation.

[0140] Since the number of punctuation marks is limited and not large, all preset punctuation marks involved in punctuation perturbation can be obtained in advance, and the feature vectors of all preset punctuation marks can be extracted. After determining the perturbed punctuation, the perturbed punctuation vector of the perturbed punctuation is obtained from the pre-determined feature vectors of the preset punctuation marks, avoiding repeated execution of the process of extracting the features of the perturbed punctuation and improving the efficiency of determining the feature vectors of multiple candidate samples.

[0141] Based on the text vector and the perturbation position, the perturbation position vector can be determined. It can be to determine the dimension of the text vector, and determine the perturbation position vector according to the perturbation position and the position of the text vector.

[0142] Exemplarily, as shown in formulas (1) and (2).

[0143] (1)

[0144] (2)

[0145] Where, is the perturbation position vector, is the position serial number of the perturbation position, is a preset parameter, is the dimension of the text vector. When the position serial number of the perturbation position is an even number, it is equal to the position serial number, and the perturbation position vector is determined by formula (1); when the position serial number of the perturbation position is an odd number, it is equal to the position serial number, and the perturbation position vector is determined by formula (2); wherein, the preset parameter can be set according to actual needs. For example, the preset parameter can be set to 10000.

[0146] In some embodiments, the computer device determines the position vectors of each preset position in the target text. After determining the perturbation position, the perturbation position vector of the perturbation position is obtained from the position vectors of each preset position.

[0147] Since in the process of determining the feature vectors of multiple candidate samples corresponding to the target text, the dimension of the target text remains unchanged and the positions involved in punctuation perturbation are limited, the position vectors of each preset position can also be determined in advance. After determining the perturbation position, the perturbation position vector of the perturbation position is determined among the positions of each preset position, avoiding repeated execution of the process of determining the perturbation position vector and improving the efficiency of determining the feature vectors of multiple candidate samples.

[0148] Fusing the text vector, the perturbed punctuation symbol vector, and the perturbed position vector can be to superimpose the text vector, the perturbed punctuation symbol vector, and the perturbed position vector, or to perform weighted summation according to the text vector, the perturbed punctuation symbol, the perturbed position vector, and their respective corresponding weights. Among them, the text vector, the perturbed punctuation symbol, the perturbed position vector, and their respective corresponding weights can be set according to actual needs.

[0149] In the above embodiments, the text vector of the target text, the perturbed punctuation symbol vector of the perturbed punctuation symbol, and the perturbed position vector of the perturbed position are determined, and the text vector, the perturbed punctuation symbol vector, and the perturbed position vector are fused to obtain the feature vector of the candidate sample. Thus, in the process of determining the feature vectors of multiple candidate samples corresponding to the target text, due to the same text vector, and some candidate samples may have the same perturbed punctuation symbols, and some candidate samples may have the same perturbed positions, the text vector, the perturbed punctuation symbol vector, and the perturbed position vector can be reused, improving the efficiency of determining the feature vectors of multiple candidate samples.

[0150] In some embodiments, classification is performed based on the feature vector to obtain the anomaly score for the target candidate sample, including: fusing the text vector of the target text and the feature vector of the target candidate sample to obtain a fused vector; performing classification processing on the fused vector to obtain the anomaly score for the target candidate sample.

[0151] Specifically, the text vector of the target text and the feature vector of the corresponding candidate sample are fused to obtain a fusion vector. It can be splicing the text vector and the feature vector of the corresponding candidate sample to obtain the fusion vector. Classifying the fusion vector can be understood as a paraphrasing task, that is, determining whether the target text is a paraphrase of the candidate sample (or whether the candidate sample is a paraphrase of the target text) based on the text vector and the feature vector of the candidate sample. The anomaly score obtained by the defense model performing the paraphrasing task can reflect the matching degree between the target text and the candidate sample determined by the defense model; the higher the anomaly score, the lower the matching degree, and the lower the anomaly score, the higher the matching degree.

[0152] It should be noted that when the matching degree is low, it can indicate that the defense model detects that the target text contains sensitive information while the candidate sample does not contain sensitive information, or the defense model detects that the target text does not contain sensitive information while the candidate sample contains sensitive information. That is, when the matching degree corresponding to the candidate sample is low (when the anomaly score is high), it means that the defense model fails to accurately identify whether the candidate sample includes sensitive information. Such a candidate sample can be used as an adversarial sample for retraining the defense model.

[0153] Therefore, after classifying the fusion vector to obtain the anomaly score of the corresponding candidate sample, it further includes: when the anomaly score of the candidate sample is greater than a preset threshold, using the candidate sample as an adversarial sample; when the anomaly score of the candidate sample is greater than the preset threshold, it means the anomaly score is high, and when the anomaly score of the candidate sample is not greater than the preset threshold, it means the anomaly score is low.

[0154] Fusing the text vector of the target text and the feature vector of the corresponding candidate sample to obtain a fusion vector can also be superimposing the text vector and the feature vector of the corresponding candidate sample, or performing weighted summation on the text vector and the feature vector of the corresponding candidate sample according to the weights of the text vector and the feature vector.

[0155] In the above embodiment, the text vector and the feature vector are fused to obtain a fusion vector, and the fusion vector is classified to determine the matching degree between the text vector and the feature vector, and then the anomaly score of the candidate sample is obtained. In this way, adversarial samples can be directly screened out through the anomaly score, improving the efficiency of determining adversarial samples.

[0156] In some embodiments, the text processing method further includes: in response to a text sending request, obtaining the text to be processed in the text sending request; processing the text to be processed by the defense model obtained through retraining to obtain a prediction score; and intercepting the text sending request when the prediction score meets the defense condition.

[0157] Among them, the text sending request can be used to implement information interaction between communication objects. For example, the first communication object can send messages, emails, messages, etc. to the second communication object through the text sending request; the text sending request can be used to implement content publishing by communication objects. For example, a communication object can send bullet screen messages, comments, etc. in a public interaction area through the text sending request.

[0158] The predicted score is used to reflect the probability that the text to be processed detected by the retrained defense model includes sensitive information. The higher the predicted score, the higher the probability that the text to be processed includes sensitive information, and the lower the predicted score, the lower the probability that the text to be processed includes sensitive information.

[0159] The predicted score meeting the defense condition indicates that the text to be processed may include sensitive information; the predicted score meeting the defense condition can be that the predicted score belongs to the first interval. That is to say, the predicted score of the text to be processed belonging to the first interval indicates that the text to be processed may include sensitive information.

[0160] Specifically, the computer device receives the text sending request sent by the first communication object, obtains the text to be processed in the text sending request, inputs the text to be processed into the retrained defense model, obtains the predicted score of the text to be processed. When the predicted score belongs to the first interval, it is determined that the predicted score meets the defense condition, indicating that the text to be processed may include sensitive information, and the computer device intercepts the text sending request.

[0161] In some embodiments, the text processing method further includes: when the predicted score meets the defense condition, performing the operation corresponding to the text sending request. Among them, the operation corresponding to the text sending request can be to send the text to be processed to another communication object, or to publish the text to be processed in a specified area.

[0162] In practical applications, when the first communication object sends a text including sensitive information, in order to bypass the interception of the defense model, the text is usually modified at the punctuation level. For example, adding punctuation marks to the text, moving punctuation marks in the text, etc., so that the modified text can bypass the defense model. And modifying the text at the punctuation level has only subtle differences visually, enabling the second communication object receiving the modified text to understand the original semantics of the text. It can be seen that modifying the text at the punctuation level will cause the defense model to fail to recognize that the modified text contains sensitive information, and the defense ability of the defense model is weak.

[0163] In the above embodiments, the defense model is retrained with adversarial samples and labels. After the text including sensitive information is modified at the punctuation level, the retrained defense model can also recognize that the modified text contains sensitive information, improving the defense ability of the defense model.

[0164] In some embodiments, the text processing method can be applied to scenarios for detecting sensitive information in information transmission, such as Figure 10 shown. Taking the server executing the text processing method as an example for illustration, it includes the following stages:

[0165] First, perform punctuation-level perturbation on the target text to obtain multiple candidate texts.

[0166] The server obtains the target text and performs punctuation-level perturbation on the target text through at least two of the insertion method, displacement method, deletion method, or replacement method to obtain multiple candidate texts.

[0167] Second, screen out adversarial samples that the defense model fails to correctly identify among the multiple candidate texts.

[0168] The server processes the multiple candidate texts through the defense model respectively to obtain the anomaly scores of the multiple candidate texts; when the text type of the target text is a normal type, if the anomaly score of any candidate text belongs to the first interval, then take this candidate text as an adversarial sample, and when the text type of the target text is an abnormal type, if the anomaly score of any candidate text belongs to the second interval, then take this candidate text as an adversarial sample.

[0169] Third, retrain the defense model with the adversarial samples.

[0170] The server uses the text type of the target text as the label of the adversarial sample and retrains the defense model with the adversarial sample and its label to obtain the retrained defense model.

[0171] Fourth, perform sensitive information control on information transmission through the retrained defense model.

[0172] The communication terminal sends a text sending request to the server. The server obtains the text to be processed included in the text sending request and processes the text to be processed through the retrained defense model to obtain a prediction score. Under the condition that the prediction score meets the defense condition, intercept the text sending request; under the condition that the prediction score does not meet the defense condition, send the text to be processed to another communication terminal specified in the text sending request.

[0173] In some embodiments, such as Figure 11 shown, the text processing method includes:

[0174] Step 1101, obtain the target text and the text type of the target text; the text type includes normal and abnormal;

[0175] Step 1102, when the perturbation method includes the insertion method, obtain at least one preset punctuation mark; insert each preset punctuation mark into the target text to obtain the first candidate sample; the candidate samples include the first candidate sample;

[0176] Step 1103, when the perturbation method includes the displacement method, obtain the first position where the original punctuation mark is located in the target text; move the original punctuation mark from the first position to the second position in the target text to obtain the second candidate sample; the second position is different from the first position; the candidate samples include the second candidate sample;

[0177] Step 1104, when the perturbation method includes the deletion method, delete at least one original punctuation mark in the target text to obtain the third candidate sample, and the candidate samples include the third candidate sample;

[0178] Step 1105, when the perturbation method includes the replacement method, determine the punctuation mark to be processed from the original punctuation marks in the target text; use a punctuation mark different from the punctuation mark to be processed to replace the punctuation mark to be processed to obtain the fourth candidate sample, and the candidate samples include the fourth candidate sample;

[0179] Step 1106, for any candidate sample, compare the target text with the candidate sample targeted to obtain the perturbed punctuation mark and the perturbed position of the candidate sample targeted, and based on the target text, the perturbed punctuation mark and the perturbed position of the candidate sample targeted, determine the input data corresponding to the candidate sample targeted;

[0180] Step 1107, extract features from the target text and the perturbed punctuation mark through the defense model to obtain a text vector and a perturbed punctuation mark vector; determine a perturbed position vector based on the text vector and the perturbed position; fuse the text vector, the perturbed punctuation mark vector and the perturbed position vector to obtain the feature vector corresponding to the candidate sample targeted;

[0181] Step 1108, fuse the text vector of the target text and the feature vector of the candidate sample targeted to obtain a fused vector; perform classification processing on the fused vector to obtain the anomaly score of the candidate sample targeted;

[0182] Step 1109, when the target text is a normal text, use the candidate samples whose anomaly scores belong to the first interval as adversarial samples, and use the normal type as the label of the adversarial samples; when the target text is an abnormal text, use the candidate samples whose anomaly scores belong to the second interval as adversarial samples, and use the abnormal type as the label of the adversarial samples; wherein, the minimum value of the first interval is greater than the maximum value of the second interval; the adversarial samples and the corresponding labels are used to retrain the defense model;

[0183] Step 1110: In response to a text sending request, obtain the text to be processed in the text sending request; process the text to be processed through a defense model obtained by retraining to obtain a prediction score; intercept the text sending request when the prediction score meets the defense condition.

[0184] In the above text processing method, punctuation perturbations are performed on the target text in multiple perturbation manners to obtain multiple candidate samples. The defense model is used to process the candidate samples to obtain the anomaly scores of the candidate samples. Adversarial samples that the defense model fails to correctly identify are selected from the candidate samples according to the anomaly scores, and the labels of the adversarial samples are determined according to the text type of the target text, so that the defense model is retrained with the adversarial samples and the corresponding labels; performing punctuation-level perturbations on the target text is simple in operation, and the perturbation process does not require calculations, greatly saving the time required to generate adversarial samples and improving the efficiency of obtaining adversarial samples; in addition, retraining the defense model with the adversarial samples and the corresponding labels improves the recognition ability of the retrained defense model for the text after punctuation perturbations, and thus improves the defense ability of the retrained defense model.

[0185] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least a part of other steps or steps or stages in other steps.

[0186] Based on the same inventive concept, an embodiment of the present application also provides a text processing device for implementing the above-mentioned text processing method. The implementation solutions provided by this device to solve problems are similar to the implementation solutions described in the above method. Therefore, the specific limitations in one or more text processing device embodiments provided below can refer to the limitations on the text processing method in the above text and will not be repeated here.

[0187] In some embodiments, as Figure 12 shown, a text processing device is provided, including: an acquisition module 1201, a perturbation module 1202, a first determination module 1203, and a second determination module 1204, where:

[0188] An acquisition module 1201, configured to acquire a target text and the text type of the target text; the text type includes normal and abnormal;

[0189] A perturbation module 1202, configured to perform punctuation perturbation on the target text in multiple perturbation manners to obtain a plurality of candidate samples;

[0190] A first determination module 1203, configured to process each candidate sample through a defense model to obtain the abnormal score of each candidate sample;

[0191] A second determination module 1204, configured to determine adversarial samples from the candidate samples according to the abnormal scores, and determine the labels of the adversarial samples according to the text type of the target text; the adversarial samples and the corresponding labels are used to retrain the defense model.

[0192] In some embodiments, the multiple perturbation manners include at least two of an insertion manner, a displacement manner, a deletion manner, or a replacement manner.

[0193] In some embodiments, the candidate samples include first candidate samples; the perturbation module 1202 is further configured to acquire at least one preset punctuation mark; and insert each preset punctuation mark into the target text to obtain the first candidate samples.

[0194] In some embodiments, the candidate samples include second candidate samples; when the perturbation manner includes a displacement manner, the perturbation module 1202 is further configured to acquire the first position where the original punctuation mark in the target text is located; move the original punctuation mark from the first position to the second position in the target text to obtain the second candidate samples; the second position is different from the first position.

[0195] In some embodiments, the candidate samples include third candidate samples; when the perturbation manner includes a deletion manner, the perturbation module 1202 is further configured to delete at least one original punctuation mark in the target text to obtain the third candidate samples.

[0196] In some embodiments, the candidate samples include fourth candidate samples; when the perturbation manner includes a replacement manner, the perturbation module 1202 is further configured to determine a punctuation mark to be processed from the original punctuation marks in the target text; and use a punctuation mark different from the punctuation mark to be processed to replace the punctuation mark to be processed to obtain the fourth candidate samples.

[0197] In some embodiments, the first determination module 1203 is further configured to, for any candidate sample, construct input data based on the candidate sample to be targeted; perform feature extraction on the input data through the defense model to obtain a feature vector corresponding to the candidate sample to be targeted, and perform classification based on the feature vector to obtain the abnormal score of the candidate sample to be targeted.

[0198] In some embodiments, the first determination module 1203 is further configured to, for any candidate sample, compare the target text with the candidate sample targeted, to obtain the perturbed punctuation and the perturbation position of the candidate sample targeted; and determine the input data corresponding to the candidate sample targeted based on the target text, the perturbed punctuation and the perturbation position of the candidate sample targeted.

[0199] In some embodiments, the first determination module 1203 is further configured to extract features from the target text and the perturbed punctuation through a defense model, to obtain a text vector and a perturbed punctuation vector; determine a perturbation position vector based on the text vector and the perturbation position; and fuse the text vector, the perturbed punctuation vector and the perturbation position vector to obtain the feature vector corresponding to the candidate sample targeted.

[0200] In some embodiments, the first determination module 1203 is further configured to fuse the text vector of the target text and the feature vector of the candidate sample targeted to obtain a fusion vector; and perform classification processing on the fusion vector to obtain the anomaly score of the candidate sample targeted.

[0201] In some embodiments, the second determination module 1204 is further configured to, when the target text is normal text, use the candidate samples whose anomaly scores belong to the first interval as adversarial samples, and use the normal type as the label of the adversarial samples; when the target text is abnormal text, use the candidate samples whose anomaly scores belong to the second interval as adversarial samples, and use the abnormal type as the label of the adversarial samples; wherein, the minimum value of the first interval is greater than the maximum value of the second interval.

[0202] In some embodiments, the text processing device further includes a defense module, configured to, in response to a text sending request, obtain the text to be processed in the text sending request; process the text to be processed through a retrained defense model to obtain a prediction score; and intercept the text sending request when the prediction score meets the defense condition.

[0203] The above text processing device perturbs the punctuation of the target text in multiple perturbation manners to obtain multiple candidate samples, processes the candidate samples through a defense model to obtain the anomaly scores of the candidate samples, selects the adversarial samples that the defense model fails to correctly identify from the candidate samples according to the anomaly scores, and determines the labels of the adversarial samples according to the text type of the target text, so as to retrain the defense model through the adversarial samples and the corresponding labels; perturbing the target text at the punctuation level is simple in operation, and the perturbation process does not require calculation, greatly saving the time required to generate adversarial samples and improving the efficiency of obtaining adversarial samples; in addition, retraining the defense model through the adversarial samples and the corresponding labels improves the recognition ability of the retrained defense model for the text after punctuation perturbation, and further improves the defense ability of the retrained defense model.

[0204] Each module in the above text processing device can be implemented in whole or in part by software, hardware, and their combination. Each of the above modules can be embedded in the processor of the computer device in hardware form or be independent of it, or be stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each of the above modules.

[0205] In some embodiments, a computer device is provided. The computer device can be a server or a terminal. Taking the computer device being a server as an example, its internal structure diagram can be as Figure 13 shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O), and a communication interface. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data related to the text processing method. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals through a network connection. When the computer program is executed by the processor, it implements a text processing method.

[0206] Those skilled in the art can understand that Figure 13 the structure shown in

[0207] is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0208] Obtain the target text and the text type of the target text; the text type includes normal and abnormal; perform punctuation perturbation on the target text according to multiple perturbation methods to obtain multiple candidate samples; process each candidate sample through a defense model to obtain the abnormal score of each candidate sample; determine adversarial samples from the candidate samples based on the abnormal scores, and determine the labels of the adversarial samples according to the text type of the target text; the adversarial samples and the corresponding labels are used to retrain the defense model.

[0209] In some embodiments, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0210] Obtain a target text and the text type of the target text; the text type includes normal and abnormal; perform punctuation perturbation on the target text according to multiple perturbation methods to obtain multiple candidate samples; process each candidate sample through a defense model to obtain the abnormal score of each candidate sample; determine an adversarial sample from the candidate samples according to the abnormal score, and determine the label of the adversarial sample according to the text type of the target text; the adversarial sample and the corresponding label are used to retrain the defense model.

[0211] In some embodiments, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the following steps are implemented:

[0212] Obtain a target text and the text type of the target text; the text type includes normal and abnormal; perform punctuation perturbation on the target text according to multiple perturbation methods to obtain multiple candidate samples; process each candidate sample through a defense model to obtain the abnormal score of each candidate sample; determine an adversarial sample from the candidate samples according to the abnormal score, and determine the label of the adversarial sample according to the text type of the target text; the adversarial sample and the corresponding label are used to retrain the defense model.

[0213] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant regulations.

[0214] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.

[0215] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0216] The above-described embodiments only represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A text processing method, characterized in that, The method includes: Obtaining a target text and the text type of the target text; the text type includes normal and abnormal; Performing punctuation perturbation on the target text according to multiple perturbation methods to obtain multiple candidate samples; Processing each of the candidate samples through a defense model to obtain the abnormal scores of each of the candidate samples; Determining adversarial samples from the candidate samples according to the abnormal scores, and determining the labels of the adversarial samples according to the text type of the target text; the adversarial samples and the corresponding labels are used to retrain the defense model.

2. The method according to claim 1, characterized in that The multiple perturbation methods include at least two of an insertion method, a displacement method, a deletion method, or a replacement method.

3. The method according to claim 2, characterized in that The candidate samples include first candidate samples; in the case where the perturbation method includes an insertion method, performing punctuation perturbation on the target text includes: Obtaining at least one preset punctuation; Inserting each of the preset punctuations into the target text to obtain first candidate samples.

4. The method according to claim 2, wherein The candidate samples include second candidate samples; in the case where the perturbation method includes a displacement method, performing punctuation perturbation on the target text includes: Obtaining the first position where the original punctuation in the target text is located; Moving the original punctuation from the first position to the second position of the target text to obtain second candidate samples; the second position is different from the first position.

5. The method according to claim 2, wherein The candidate samples include third candidate samples; in the case where the perturbation method includes a deletion method, performing punctuation perturbation on the target text includes: Deleting at least one original punctuation in the target text to obtain third candidate samples.

6. The method according to claim 2, characterized in that The candidate samples include fourth candidate samples; in the case where the perturbation method includes a replacement method, performing punctuation perturbation on the target text includes: Determining a punctuation to be processed from the original punctuations of the target text; Replacing the punctuation to be processed with a punctuation different from the punctuation to be processed to obtain fourth candidate samples.

7. The method according to claim 1, characterized in that The processing each of the candidate samples through a defense model to obtain the abnormal scores of each of the candidate samples includes: For any candidate sample, constructing input data based on the candidate sample to be targeted; Performing feature extraction on the input data through a defense model to obtain a feature vector corresponding to the candidate sample to be targeted, and performing classification based on the feature vector to obtain the abnormal score of the candidate sample to be targeted.

8. The method according to claim 7, characterized in that For any candidate sample, constructing input data based on the candidate sample to be targeted includes: For any candidate sample, comparing the target text with the candidate sample to be targeted to obtain the perturbed punctuation and the perturbed position of the candidate sample to be targeted; Determining the input data corresponding to the candidate sample to be targeted based on the target text, the perturbed punctuation of the candidate sample to be targeted, and the perturbed position.

9. The method according to claim 8, characterized in that Performing feature extraction on the input data through a defense model to obtain a feature vector corresponding to the candidate sample to be targeted includes: Feature extraction is performed on the target text and the perturbed punctuation marks through a defense model to obtain a text vector and a perturbed punctuation mark vector; A perturbation position vector is determined based on the text vector and the perturbation position; The text vector, the perturbed punctuation mark vector, and the perturbation position vector are fused to obtain a feature vector corresponding to the candidate sample; 10. The method according to claim 7, characterized in that, Classifying based on the feature vector to obtain an anomaly score for the candidate sample includes: Fusing the text vector of the target text and the feature vector of the candidate sample to obtain a fused vector; Performing classification processing on the fused vector to obtain an anomaly score for the candidate sample; 11. The method according to claim 1, wherein Determining an adversarial sample from the candidate samples according to the anomaly score and determining a label for the adversarial sample according to the text type of the target text includes: When the target text is a normal text, the candidate samples whose anomaly scores belong to the first interval are used as adversarial samples, and the normal type is used as the label of the adversarial samples; When the target text is an abnormal text, the candidate samples whose anomaly scores belong to the second interval are used as adversarial samples, and the abnormal type is used as the label of the adversarial samples; wherein, the minimum value of the first interval is greater than the maximum value of the second interval; 12. The method according to any one of claims 1 to 11, characterized in that, The method further includes: In response to a text sending request, obtaining the text to be processed in the text sending request; Processing the text to be processed through a defense model obtained by retraining to obtain a prediction score; Intercepting the text sending request when the prediction score meets the defense condition; 13. A text processing device, characterized in that, The device includes: An acquisition module, configured to acquire a target text and the text type of the target text; the text type includes normal and abnormal; A perturbation module, configured to perform punctuation perturbation on the target text in multiple perturbation manners to obtain multiple candidate samples; A first determination module, configured to process each of the candidate samples through a defense model to obtain an anomaly score for each of the candidate samples; A second determination module, configured to determine an adversarial sample from the candidate samples according to the anomaly score and determine a label for the adversarial sample according to the text type of the target text; the adversarial sample and the corresponding label are used to retrain the defense model; 14. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the method according to any one of claims 1 to 12 are implemented; 15. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 12 are implemented; 16. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 12 are implemented;

Images