Sample leakage detection methods, equipment, media and program products

Abstract

The present invention provides a sample leakage detection method, device, medium and program product, which can be applied to the field of artificial intelligence technology. The method includes: based on the attribute information of each of the multiple sample words in the sample to be tested, rewriting the text of the sample to be tested to obtain at least one similar sample; using a preset model to respectively determine the first probability distribution feature of the sample to be tested and the second probability distribution feature of each of the at least one similar sample, wherein the first probability distribution feature is used to characterize the distribution feature of the first predicted probabilities of the multiple sample words predicted by the preset model, and the second probability distribution feature is used to characterize the distribution feature of the second predicted probabilities of the multiple target words in the similar samples predicted by the preset model; based on the sample distribution distance between the first probability distribution feature and the at least one second probability distribution feature, leakage detection is performed on the sample to be tested to obtain a detection result.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a sample leakage detection method, device, equipment, medium and program product. Background Art

[0002] With the rapid development of artificial intelligence and natural language processing technologies, the performance requirements for deep learning models are gradually increasing. However, sample leakage causes deviations in the evaluation results of deep learning models, and model performance cannot be effectively evaluated.

[0003] Sample leakage usually refers to the inclusion of test samples used to evaluate model performance in training samples. Related technologies usually use the method of calculating the overlap rate between test samples and training samples to identify whether the test samples are leaked training samples. However, the overlap rate method relies on the disclosure of training samples. The training samples of many deep learning models are not disclosed, making it difficult to detect sample leakage. Summary of the invention

[0004] In view of the above problems, the present invention provides a sample leakage detection method, device, equipment, medium and program product.

[0005] According to one aspect of the present invention, a sample leakage detection method is provided, comprising: based on the attribute information of each of a plurality of sample words in the sample to be tested, rewriting the text of the sample to be tested to obtain at least one similar sample; using a preset model to respectively determine a first probability distribution feature of the sample to be tested and a second probability distribution feature of each of at least one similar sample, wherein the first probability distribution feature is used to characterize the distribution feature of the first predicted probabilities of the plurality of sample words predicted by the preset model, and the second probability distribution feature is used to characterize the distribution feature of the second predicted probabilities of the plurality of target words in the similar samples predicted by the preset model; based on the sample distribution distance between the first probability distribution feature and the at least one second probability distribution feature, leakage detection is performed on the sample to be tested to obtain a detection result.

[0006] Another aspect of the present invention provides a sample leakage detection device, including: a rewriting module, used to rewrite the text of the sample to be tested based on the attribute information of each of multiple sample words in the sample to be tested, so as to obtain at least one similar sample; a determination module, used to use a preset model to respectively determine the first probability distribution feature of the sample to be tested and the second probability distribution feature of each of at least one similar sample, wherein the first probability distribution feature is used to characterize the distribution feature of the first predicted probabilities of multiple sample words predicted by the preset model, and the second probability distribution feature is used to characterize the distribution feature of the second predicted probabilities of multiple target words in the similar samples predicted by the preset model; a detection module, used to perform leakage detection on the sample to be tested based on the sample distribution distance between the first probability distribution feature and at least one second probability distribution feature, so as to obtain a detection result.

[0007] Another aspect of the present invention provides an electronic device, comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0008] Another aspect of the present invention further provides a computer-readable storage medium having a computer program or instructions stored thereon, wherein the computer program or instructions implement the steps of the above method when executed by a processor.

[0009] Another aspect of the present invention further provides a computer program product, including a computer program or instructions, which implement the steps of the above method when executed by a processor.

[0010] According to the sample leakage detection method of the present invention, when the preset model is used to infer the trained training samples, even if the content of the training samples changes slightly, the probability distribution characteristics of the model for the training samples before and after the change may also change significantly. The attribute information of multiple sample words in the sample to be tested is used to rewrite the sample to be tested, and a similar sample similar to the sample to be tested but with disturbance is obtained, and the first probability distribution characteristics of the sample to be tested and the second probability distribution characteristics of each similar sample are determined based on the preset model to determine the sample distribution distance between the sample to be tested and at least one test sample. And by observing the sample distribution distance, leakage detection of the sample to be tested is achieved. Therefore, at least part of the technical problem that it is difficult to perform sample leakage detection for models that are not public to the training samples in the related art is solved, and the technical effect of improving the versatility of sample leakage detection and improving detection accuracy is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The above contents and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings.

[0012] Figure 1 An application scenario diagram of a sample leakage detection method, apparatus, device, medium, and program product according to an embodiment of the present invention is shown.

[0013] Figure 2 A flow chart of a sample leakage detection method according to an embodiment of the present invention is shown.

[0014] Figure 3 A flow chart for determining a first probability distribution feature according to an embodiment of the present invention is shown.

[0015] Figure 4 A data flow diagram of a sample leakage detection method according to another embodiment of the present invention is shown.

[0016] Figure 5 A structural block diagram of a sample leakage detection device according to an embodiment of the present invention is shown.

[0017] Figure 6 A block diagram of an electronic device suitable for implementing a sample leakage detection method according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0018] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0019] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0020] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0021] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0022] In the technical solution of the present invention, the user information (including but not limited to user personal information, user image information, user device information, such as location information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0023] In the scenario of using personal information for automated decision-making, the methods, devices, and systems provided by the embodiments of the present invention provide users with corresponding operation portals for users to choose to agree or reject the automated decision-making results; if the user chooses to reject, the expert decision-making process will be entered. The expression "automated decision-making" here refers to the activity of automatically analyzing and evaluating an individual's behavioral habits, interests and hobbies, or economic, health, credit status, etc. through computer programs, and making decisions. The expression "expert decision-making" here refers to the activity of making decisions by people who specialize in a certain field, have specialized experience, knowledge and skills, and have reached a certain level of professionalism.

[0024] A large language model usually refers to a "large parameter" model trained using large-scale data and powerful computing power. These models are usually highly versatile and generalizable, and can be applied to natural language processing, image recognition, speech recognition and other fields. They can be divided into large language models, large visual models, large multimodal models, and basic large models.

[0025] Sample leakage usually means that the training sample contains the content of the test set, which causes the model's score on the test set to exceed the original ability of the large model. That is, the model sees the test set data during the training process, so it is too optimistic during the evaluation and cannot accurately reflect its true generalization ability. Sample leakage will cause the model evaluation results to be distorted and affect the actual application effect of the model. Therefore, in machine learning projects, it is necessary to strictly avoid the test set data from appearing in the training set.

[0026] During the research process, it was found that the significant progress of deep learning models such as large language models in natural language processing tasks depends largely on the continuous expansion of the scale of training samples. With the widespread application of models, the quality of training samples has become increasingly prominent, among which sample leakage has become a key challenge. Sample leakage refers to the intentional or unintentional inclusion of test samples in the training set, which leads to deviations in model evaluation results and makes it difficult to obtain a truly effective model. The purpose of model evaluation is to objectively and accurately evaluate the generalization ability of the model. When training samples are mixed into test samples, the performance of the model on these sample data will be overestimated, making the evaluation results unable to truly reflect the actual ability of the model on unknown data.

[0027] For example, when evaluating the question-answering capabilities of a language model, if the training samples contain questions and answers from some of the test samples, the model can easily give the correct answer when encountering these duplicate data during the evaluation, which will make the evaluation indicators inflated and mislead researchers and developers about the performance of the model. The training data of many large language models comes from a wide range of sources. In the process of collecting and organizing these data, it is easy to mix in inappropriate data, and the risk of sample leakage is also increasing.

[0028] At present, some methods for sample leakage detection are to identify leaked samples by calculating the overlap rate between test samples and training samples. These methods compare test samples with known training samples and count the frequency and overlap of N consecutive words or symbols in the test sample in the training sample. If the overlap rate exceeds a certain threshold, the test sample is determined to be a leaked sample. The overlap rate method relies on the disclosure of training corpora, but the training corpora of many large language models are not public. Therefore, it is difficult to accurately calculate the overlap rate between test samples and training data. On the other hand, sample data of text types are highly diverse, and texts in different fields and styles have great differences in usage, grammatical structure, etc. The overlap method is difficult to adapt to this diversity and change, and it is difficult to adjust the detection standards in a timely manner, which reduces the accuracy and reliability of detection.

[0029] An embodiment of the present invention provides a sample leakage detection method, comprising: based on the attribute information of each of multiple sample words in the sample to be tested, rewriting the text of the sample to be tested to obtain at least one similar sample; using a preset model to respectively determine a first probability distribution feature of the sample to be tested and a second probability distribution feature of each of at least one similar sample, wherein the first probability distribution feature is used to characterize the distribution feature of the first predicted probabilities of multiple sample words predicted by the preset model, and the second probability distribution feature is used to characterize the distribution feature of the second predicted probabilities of multiple target words in the similar samples predicted by the preset model; based on the sample distribution distance between the first probability distribution feature and the at least one second probability distribution feature, leakage detection is performed on the sample to be tested to obtain a detection result.

[0030] Figure 1 An application scenario diagram of a sample leakage detection method, apparatus, device, medium, and program product according to an embodiment of the present invention is shown.

[0031] like Figure 1 As shown, the application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0032] The user can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only for example).

[0033] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0034] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0035] It should be noted that the sample leakage detection method provided in the embodiment of the present invention can generally be executed by the server 105. Accordingly, the sample leakage detection device provided in the embodiment of the present invention can generally be set in the server 105. The sample leakage detection method provided in the embodiment of the present invention can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Correspondingly, the sample leakage detection device provided in the embodiment of the present invention can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105.

[0036] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0037] The following will be based on Figure 1 The scene described by Figure 2~Figure 4 The sample leakage detection method of the embodiment of the invention is described in detail.

[0038] Figure 2 A flow chart of a sample leakage detection method according to an embodiment of the present invention is shown.

[0039] like Figure 2 As shown, the method includes operations S210 to S230.

[0040] In operation S210, based on the attribute information of each of the plurality of sample words in the sample to be tested, the sample to be tested is rewritten to obtain at least one similar sample.

[0041] In operation S220, a preset model is used to respectively determine a first probability distribution feature of the sample to be tested and a second probability distribution feature of each of at least one similar sample, wherein the first probability distribution feature is used to characterize the distribution features of the first predicted probabilities of multiple sample words predicted by the preset model, and the second probability distribution feature is used to characterize the distribution features of the second predicted probabilities of multiple target words in similar samples predicted by the preset model.

[0042] In operation S230, based on the sample distribution distance between the first probability distribution feature and the at least one second probability distribution feature, leakage detection is performed on the sample to be detected to obtain a detection result.

[0043] According to an embodiment of the present invention, the sample to be tested may be a test sample used to test a preset model.

[0044] According to an embodiment of the present invention, a preset model may be used to perform word segmentation processing on a sample to be tested, thereby determining a plurality of sample words.

[0045] According to an embodiment of the present invention, a sample word may be a basic unit of text processing, and a preset model may divide a sample to be tested into sample words for understanding and generation during processing. The sample words may be words, subwords, or characters, etc. For example, a sample word in an English sample to be tested usually corresponds to a word or subword, while a sample word in a Chinese sample to be tested may correspond to a Chinese character or phrase.

[0046] According to the embodiment of the present invention, the attribute information is not limited and may be information used to define attributes of sample words, such as semantic information, position information, and the like.

[0047] According to an embodiment of the present invention, the attribute information of the multiple sample words can be used to reorder, replace synonyms, modify tone, change double negation to affirmation, and other text rewriting of the multiple sample words in the sample to be tested, so as to obtain at least one similar sample.

[0048] According to an embodiment of the present invention, there is no limitation on the implementation method of the preset model, which may be any deep learning model, such as a large language model (LLM).

[0049] According to an embodiment of the present invention, the first probability distribution feature may be obtained by extracting distribution features from the first predicted probabilities of each of the multiple sample words predicted by the preset model in the process of reasoning the sample to be tested. The second probability distribution feature may be obtained by extracting distribution features from the second predicted probabilities of each of the multiple target words predicted by the preset model in the process of reasoning the similar sample in the same manner as the multiple first predicted probabilities.

[0050] According to an embodiment of the present invention, the reasoning conditions of the preset model on the sample to be tested and similar samples can be determined respectively through the first probability distribution feature and the second probability distribution feature.

[0051] According to an embodiment of the present invention, by determining the gap between the first probability distribution feature and at least one second probability distribution feature, it is possible to determine the magnitude of change predicted by the preset model for the sample to be tested when the sample to be tested changes slightly, thereby enabling leakage detection of the sample to be tested.

[0052] According to the sample leakage detection method of the present invention, when the preset model is used to infer the trained training samples, even if the content of the training samples changes slightly, the probability distribution characteristics of the model for the training samples before and after the change may also change significantly. The attribute information of multiple sample words in the sample to be tested is used to rewrite the sample to be tested, and a similar sample similar to the sample to be tested but with disturbance is obtained, and the first probability distribution characteristics of the sample to be tested and the second probability distribution characteristics of each similar sample are determined based on the preset model to determine the sample distribution distance between the sample to be tested and at least one test sample. And by observing the sample distribution distance, leakage detection of the sample to be tested is achieved. Therefore, at least part of the technical problem that it is difficult to perform sample leakage detection for models that are not public to the training samples in the related art is solved, and the technical effect of improving the versatility of sample leakage detection and improving detection accuracy is achieved.

[0053] According to an embodiment of the present invention, for trained training samples and untrained data, the probability distribution characteristics during reasoning determined by the weights of the preset model itself will show different characteristics. For trained sample data, especially when the preset model overfits the training data, even if the content of the sample changes slightly, the probability prediction distribution of the model for the samples before and after the change may also change significantly. Therefore, by adjusting the sample to be tested to generate similar samples, and calculating the distance between the probability distribution characteristics of the similar samples and the sample to be tested, it is possible to mine the characteristics of whether the sample to be tested is leaked from the probability information input and output of the preset model without accessing the training samples, and determine whether the sample to be tested is a leaked sample. The model for the training corpus that is not publicly trained can also realize data leakage detection, and at the same time, it can at least partially solve the problem of low detection accuracy existing in the related technology.

[0054] According to an embodiment of the present invention, using a preset model to respectively determine a first probability distribution feature of a sample to be tested and a second probability distribution feature of at least one similar sample may include the following operations.

[0055] Using a preset model, the first prediction probability of each of the multiple sample words and the second prediction probability of each of the multiple target words included in each similar sample are determined respectively; distribution features of the multiple first prediction probabilities are extracted to determine the first probability distribution features of the sample to be tested; for each similar sample, distribution features of the second prediction probability of each of the multiple target words included in the similar sample are extracted to determine the second probability distribution features of the similar sample.

[0056] According to an embodiment of the present invention, the preset model can be used to predict the probability of each of the multiple sample words being located at the current position in the sample to be tested, thereby obtaining the first predicted probability of each of the multiple sample words. The preset model can also be used to predict the probability of each of the multiple target words being located at the current position in the similar sample, thereby obtaining the second predicted probability of each of the multiple target words.

[0057] According to the embodiment of the present invention, there is no limitation on the method of extracting distribution features, and different methods can be used for extraction according to different numbers of sample words or target words. For example, when the number of sample words is large, some samples to be tested can be selected from multiple samples to be tested to calculate distribution features. When the number of sample words is small, all sample words can be used and multiple methods can be used to calculate distribution features.

[0058] According to an embodiment of the present invention, distribution feature analysis is performed on the first predicted probability of multiple sample words and the second predicted probability of the target word of each similar sample in the same manner, so that the first probability distribution feature and the second probability distribution feature are in the same dimension, so as to facilitate subsequent analysis of whether the sample to be tested is a leaked sample.

[0059] According to an embodiment of the present invention, it is characterized in that using a preset model to respectively determine the first prediction probability of each of a plurality of sample words and the second prediction probability of each of a plurality of target words included in each similar sample may include the following operations.

[0060] The sample to be tested is input into the preset model to obtain the first position prediction probabilities of multiple preset words located at the position of each sample word in the sample to be tested, wherein the first position prediction probability is obtained by the preset model predicting the probability of the preset word being located at the position of the sample word based on the semantic information of the previous sample word that is located before the sample word in the sample to be tested; for each sample word, the first position prediction probability of the preset word that is the same as the sample word is used as the first prediction probability of the sample word; for each similar sample, the similar sample is input into the preset model to obtain the second position prediction probabilities of multiple preset words located at the position of each target word in the similar sample, wherein the second position prediction probability is obtained by the preset model predicting the probability of the preset word being located at the position of the target word based on the semantic information of the previous target word that is located before the target word in the similar sample; for each target word, the second position prediction probability of the preset word that is the same as the target word is used as the second prediction probability of the target word.

[0061] According to an embodiment of the present invention, a plurality of preset words may be stored in a preset word list of a preset model.

[0062] According to an embodiment of the present invention, the plurality of preset words include a plurality of sample words.

[0063] According to an embodiment of the present invention, the previous sample word may be a sample word located before the current sample word in the sample to be tested. Each first position prediction probability may be obtained by predicting the probability of the preset word being located at the position after the preset model performs context analysis based on the semantic information of the previous sample word.

[0064] According to an embodiment of the present invention, the previous target word may be a target word located before the current target word in a similar sample. Each second position prediction probability may be obtained by predicting the probability of the preset word being located at the position after a preset model performs context analysis based on the semantic information of the previous target word.

[0065] According to an embodiment of the present invention, by matching a sample word with a plurality of preset words, the first predicted probability of the sample word being at its current position in the sample to be tested can be used to determine the first predicted probability of the sample word being at its current position.

[0066] According to an embodiment of the present invention, for example, the multiple sample word sequences in the sample to be tested s are , where j represents the jth sample word, and len(s) represents the length of the sample word sequence. The preset model L predicts that multiple preset words are located in the sample word t j The first position prediction probability of the position is determined by matching the preset word with the sample word. j The first predicted probability is That is, when predicting the probability of the preset word being located at the position of the jth sample word, the probability of the preset word being located at the position of the sample word t is taken into account. j Previous sample words .

[0067] According to an embodiment of the present invention, by matching the target word with multiple preset words, the second predicted probability of the target word being at its current position can be determined based on the second position predicted probabilities of the multiple preset words being at the position of the target word in similar samples.

[0068] According to an embodiment of the present invention, the first position prediction probability of each sample word at the position where the preset model predicts multiple preset words is located in the sample to be tested is determined, and the sample word is matched with each preset word to determine the first prediction probability of the sample word. Similar samples also use the above-mentioned distribution feature extraction method, thereby realizing the determination of key information from a large number of position prediction probabilities output by the preset model to determine the probability distribution characteristics, thereby realizing feature dimensionality reduction and improving the determination rate of the probability distribution characteristics.

[0069] According to an embodiment of the present invention, extracting distribution features of multiple first prediction probabilities to determine first probability distribution features of the sample to be tested may include the following operations.

[0070] When it is determined that the number of sample words of multiple sample words is greater than a preset number threshold, the multiple first prediction probabilities are arranged according to numerical values ​​to obtain a prediction probability sequence; based on multiple selected prediction probabilities determined by the arrangement order of the multiple first prediction probabilities in the prediction probability sequence, the first probability distribution feature of the sample to be tested is determined.

[0071] According to an embodiment of the present invention, the number of sample words may be the total number of multiple sample words. There is no limitation on the preset number threshold, which may be set according to actual needs.

[0072] According to the embodiment of the present invention, there is no limitation on the arrangement of the first prediction probabilities, and the first prediction probabilities may be arranged in a manner such as from large to small or from small to large according to numerical values.

[0073] According to an embodiment of the present invention, there is no limitation on the method for determining the selected prediction probability. It can be selected by the order of arrangement of each first prediction probability, or it can be determined by the position of each first prediction probability in the arrangement order, for example: selecting an arrangement order of odd digits, even digits, or the median, etc.

[0074] According to an embodiment of the present invention, different methods may be used to determine the first probability distribution feature through different ways of determining the selected prediction probability.

[0075] According to an embodiment of the present invention, the distribution feature extraction method is determined by judging the number of sample words. When the number of sample words is large, part of the first prediction probability is selected to calculate the first probability distribution feature of the sample to be tested, thereby achieving data dimensionality reduction and improving the calculation speed of the first probability distribution feature.

[0076] According to an embodiment of the present invention, the multiple selected prediction probabilities include the first M first prediction probabilities and the last N first prediction probabilities from large to small in the prediction probability sequence, where M and N are both integers greater than or equal to 1 and are both determined based on the number of sample words; wherein, determining the first probability distribution feature of the sample to be tested based on the multiple selected prediction probabilities determined by the respective arrangement order of the multiple first prediction probabilities in the prediction probability sequence may include the following operations.

[0077] Determine a first probability mean of the first M first prediction probabilities and a second probability mean of the last N first prediction probabilities; and determine a first probability distribution feature of the sample to be tested based on the first probability mean and the second probability mean.

[0078] According to an embodiment of the present invention, the prediction probability sequence may be selected to be the first M first prediction probabilities and the last N first prediction probabilities arranged from large to small.

[0079] According to an embodiment of the present invention, the specific values ​​of M and N can be determined by the number of sample words and the preset percentage. For example, k1% first prediction probabilities are selected from multiple preset probability sequences as the first M first prediction probabilities, and k 2 % first prediction probabilities are used as the next N first prediction probabilities.

[0080] According to an embodiment of the present invention, for example, when k1=5, k2=30, and the number of sample words is 100, M can be obtained to be 5 and N to be 30.

[0081] According to an embodiment of the present invention, the first M first prediction probabilities are a portion of first prediction probabilities with larger values, and the last N first prediction probabilities are a portion of first prediction probabilities with smaller values.

[0082] According to an embodiment of the present invention, the average value of the first predicted probabilities of the larger numerical part can be obtained by calculating the first probability mean of the first M first predicted probabilities. The average value of the first predicted probabilities of the smaller numerical part can be obtained by calculating the second probability mean of the last N first predicted probabilities.

[0083] According to an embodiment of the present invention, the first probability distribution feature may be obtained by subtracting the first probability mean from the second probability mean.

[0084] According to an embodiment of the present invention, a first probability mean of the first M first prediction probabilities and a second probability mean of the last N first prediction probabilities are determined; based on the first probability mean and the second probability mean, a first probability distribution feature is determined, which can be shown in the following formula (1).

[0085] ; (1)

[0086] in, Characterizes the first probability distribution characteristics of the sample to be tested, s is the sample to be tested, L is the preset model, M represents the number of the first M predicted probabilities, N represents the number of the last N first predicted probabilities, is a set of sample words corresponding to the first M first prediction probabilities, is a set of sample words corresponding to the last N first prediction probabilities, is the i-th sample word The first predicted probability.

[0087] According to an embodiment of the present invention, when the first probability distribution feature is determined by using formula (1), the second probability distribution feature of the similar sample may be obtained by using the following formula (2).

[0088] ; (2)

[0089] in, Characterize the second probability distribution characteristics of similar samples, s' is a similar sample, M represents the number of the first M second prediction probabilities, N represents the number of the last N second prediction probabilities, is a set of sample words corresponding to the first M second prediction probabilities, is a set of sample words corresponding to the last N second prediction probabilities, is the i-th target word The second predicted probability of .

[0090] According to an embodiment of the present invention, by determining the first M larger first prediction probabilities and the last N smaller first prediction probabilities according to the number of sample words, it is possible to achieve dynamic data selection for the samples to be tested including different numbers of sample words, thereby improving the versatility and accuracy of the method. And by calculating the first probability distribution feature through the first probability mean and the second probability mean, the two ends of the probability distribution can be understood more quickly. The first probability mean reflects the central tendency of the high probability area, and the second probability mean reflects the central tendency of the low probability area, which helps to outline the shape of the probability distribution. And by comparing the first probability mean and the second probability mean, the difference between the high and low ends of the probability distribution can be clearly shown, thereby better characterizing the first probability distribution feature of the sample to be tested.

[0091] According to an embodiment of the present invention, extracting distribution features of multiple first prediction probabilities to determine first probability distribution features of the sample to be tested may also include the following operations.

[0092] When it is determined that the number of sample words of multiple sample words is less than or equal to a preset number threshold, a change characteristic value is calculated based on multiple first prediction probabilities, wherein the change characteristic value is determined based on a concentration index and a dispersion index used to measure the first prediction probabilities of each of the multiple sample words; and the change characteristic value is used as a first probability distribution feature.

[0093] According to an embodiment of the present invention, the concentration index used to measure the plurality of first prediction probabilities may be the mean of the plurality of first prediction probabilities, and the dispersion index used to measure the variance, standard deviation, etc. of the plurality of first prediction probabilities.

[0094] According to an embodiment of the present invention, the variation characteristic value may include at least one of a concentration index and a dispersion index.

[0095] According to an embodiment of the present invention, when the change characteristic value includes a concentration index and a dispersion index, when calculating the sample distribution distance between the first probability distribution feature and the second probability distribution feature, when there is only one similar sample, the distance between the concentration index of the first probability distribution feature and the second probability distribution feature and the distance between the dispersion index of the first probability distribution feature and the second probability distribution feature can be calculated respectively, so as to determine the final sample distribution distance. When there are multiple similar samples, the mean of the dispersion index and the concentration index of the multiple similar samples can be calculated, and then the distance between the mean of the dispersion index and the concentration index and the concentration index of the sample to be tested is calculated.

[0096] According to an embodiment of the present invention, for example: the square of the difference between the first concentration index included in the first probability distribution feature and the second concentration index included in the second probability distribution feature can be calculated to obtain a first square value; and the square of the difference between the first discreteness index included in the first probability distribution feature and the second discreteness index included in the second probability distribution feature can be calculated to obtain a second square value; and the 1 / 2 root operation of the sum of the first square value and the second square value can be performed to obtain the sample distribution distance.

[0097] According to an embodiment of the present invention, the distribution feature extraction method is determined by judging the number of sample words. When the number of sample words is small, most or all of the first prediction probabilities are used to calculate the first probability distribution feature of the sample to be tested, thereby improving the accuracy of the distribution feature calculation, and obtaining the first probability distribution feature through the concentration index and the dispersion index. This realizes the comprehensive determination of the first probability distribution feature from multiple aspects when the number of sample words is small, thereby improving the accuracy of the distribution feature calculation.

[0098] According to an embodiment of the present invention, the sample distribution distance is determined in the following manner.

[0099] Based on at least one second probability distribution feature and the number of similar samples, a distribution mean is determined; and an absolute value of a difference between the first probability distribution feature and the distribution mean is used as a sample distribution distance.

[0100] According to an embodiment of the present invention, the sample distribution distance may be calculated as shown in the following formula (3).

[0101] ; (3)

[0102] Among them, d is the sample distribution distance, K is the number of similar samples, B is the sample space of similar samples, s is the sample to be tested, s' is the similar sample, is the first probability distribution feature, is the second probability distribution feature.

[0103] According to an embodiment of the present invention, by taking the absolute value of the difference between the distribution mean of at least one second probability distribution and the first probability distribution as the sample distribution distance, it is possible to determine the average distribution distance between the sample to be tested and at least one similar sample, thereby achieving the technical effect of obtaining a more accurate sample distribution distance.

[0104] According to an embodiment of the present invention, based on the sample distribution distance between the first probability distribution feature and at least one second probability distribution feature, performing leakage detection on the sample to be tested to obtain the detection result may include the following operations.

[0105] The sample distribution distance is compared with a preset distance threshold to obtain a comparison result; based on the comparison result, the detection result of the sample to be tested is determined.

[0106] According to the embodiment of the present invention, the preset distance threshold is not limited and may be determined according to actual needs, for example, based on experience or multiple experiments.

[0107] According to an embodiment of the present invention, the preset distance threshold may be determined in the following manner, such as by determining the historical sample distribution distances between a plurality of historical samples to be tested and respective historical similar samples of the plurality of historical samples to be tested. The plurality of historical sample distribution distances are clustered to obtain at least two cluster regions, wherein the cluster region is a region where a cluster is formed. The preset distance threshold is determined from the sample distribution distance between the at least two cluster regions. Thus, the preset distance threshold is quantitatively determined, and a suitable preset distance threshold can be determined for different preset models.

[0108] According to the embodiment of the present invention, there is no limitation on the clustering method, which may be DBSCAN (Density-Based Spatial Clustering of Applications with Noise), Hierarchical Clustering Algorithm, etc.

[0109] According to an embodiment of the present invention, the comparison result between the sample distribution distance and the preset distance threshold is used as the leakage detection result of the sample to be detected, so as to realize the quantification of the detection process of whether the sample to be detected is leaked. For a large model of undisclosed training data, the detection of whether the training sample is leaked can also be realized, thereby improving the versatility of detection. Compared with introducing other detection models with uncertain capabilities, the accuracy of leakage detection is improved by utilizing the distance between the determined probability distribution of the sample to be tested and similar samples in the reasoning process of the preset model as a response indicator.

[0110] According to an embodiment of the present invention, determining the detection result of the sample to be tested based on the comparison result may include the following operations.

[0111] When it is determined that the comparison result indicates that the sample distribution distance is greater than a preset distance threshold, the detection result is determined that the sample to be tested is a leaked sample, wherein the leaked sample indicates that the sample to be tested is a training sample of a preset model.

[0112] According to an embodiment of the present invention, the preset distance threshold may be a boundary value of the sample distribution distance between the sample to be tested and similar samples, and when the sample distribution distance is greater than the preset distance threshold, the sample to be tested may be considered a leaked sample. When the sample distribution distance is less than or equal to the preset distance threshold, the sample to be tested may be considered a non-leaked sample.

[0113] According to an embodiment of the present invention, the leaked sample may be a sample that has appeared in a training set for training a preset model.

[0114] According to the embodiment of the present invention, the comparison result can be used to determine whether the sample to be tested is a leaked sample, thereby further illustrating the relationship between the comparison result, the sample to be tested, and the detection result.

[0115] According to an embodiment of the present invention, the sample leakage detection method may further include the following operations.

[0116] When it is determined that the sample to be tested is a leaked sample, the sample to be tested is marked with a preset mark to obtain a marked sample; the similarity between the marked sample and other samples to be tested in the sample set to be tested except the marked sample is calculated to obtain a calculation result, wherein the sample to be tested is obtained from the sample set to be tested; and other samples to be tested whose calculation results represent a similarity greater than a preset similarity threshold are filtered out.

[0117] According to an embodiment of the present invention, when it is determined that the sample to be tested is a leaked sample, the sample to be tested can be marked to facilitate subsequent distinction between the leaked sample and the non-leaked sample. When testing the preset model, the non-leaked sample is used to perform the performance test of the preset model.

[0118] According to the embodiment of the present invention, there is no limitation on the marking method, and special identification, special attribute information, etc. may be added.

[0119] According to an embodiment of the present invention, when the set of samples to be tested includes other samples to be tested except the current sample to be tested, the other samples to be tested may be filtered according to the similarity between the sample to be tested and the other samples to be tested.

[0120] According to an embodiment of the present invention, in order to improve filtering accuracy, the domain of the sample to be tested can be determined, so that similarity is calculated between the marked sample and other samples to be tested with the same domain, so as to avoid false filtering situations where the similarity is high but the actual samples are different.

[0121] According to an embodiment of the present invention, the fields of the sample to be tested and other samples to be tested may be predetermined.

[0122] According to an embodiment of the present invention, when it is determined that the sample to be tested is a leaked sample, the sample to be tested is marked, and its similarity with other samples to be tested is calculated, and other samples to be tested with higher similarity are filtered, thereby improving the detection speed of whether the sample to be tested is a leaked sample.

[0123] According to an embodiment of the present invention, the attribute information includes position information; based on the attribute information of each of the multiple sample words in the sample to be tested, rewriting the text of the sample to be tested to obtain at least one similar sample may include the following operations.

[0124] Based on the position information of each of the multiple sample words in the sample to be tested, adjacent sample words at adjacent positions are determined; based on at least one preset exchange order, the adjacent sample words at adjacent positions are exchanged to obtain at least one similar sample.

[0125] According to an embodiment of the present invention, the preset exchange order may include the exchange order of every two adjacent sample words located at various positions. The preset exchange order may be randomly generated.

[0126] According to an embodiment of the present invention, the preset exchange order and the similar samples may be in a one-to-one correspondence. By pre-randomly generating multiple preset exchange orders and exchanging the positions of adjacent sample words of the sample to be tested based on the preset exchange order, multiple different similar samples may be obtained.

[0127] According to an embodiment of the present invention, for example, when the sample to be tested is: deep learning is an important technology in the field of artificial intelligence, by swapping the positions of adjacent sample words, a similar sample that can be obtained is: learning artificial intelligence is an important technology in the deep field.

[0128] According to an embodiment of the present invention, since the positions of adjacent words may occasionally be reversed in the use of natural language, by exchanging adjacent sample words, it is possible to simulate such accidental variations in natural language, thereby generating a similar sample that is similar to the sample to be tested but with perturbations.

[0129] According to an embodiment of the present invention, the attribute information includes semantic information; based on the attribute information of each of the multiple sample words in the sample to be tested, rewriting the text of the sample to be tested to obtain at least one similar sample may include the following operations.

[0130] Based on the semantic information of each of multiple sample words, determine a sample word to be replaced from the multiple sample words; perform a synonym replacement on the sample word to be replaced to obtain at least one similar sample.

[0131] According to an embodiment of the present invention, the semantic information of each of the multiple sample words can be determined in advance by means such as semantic dictionary matching.

[0132] According to an embodiment of the present invention, the sample word to be replaced can be a sample word with clear semantic information, such as: nouns, adjectives, verbs, etc.

[0133] According to an embodiment of the present invention, by determining a sample word to be replaced with clear semantic information from multiple sample words, it is possible to avoid replacing function words with unclear semantic information or functionality such as "de" (的) and "le" (了), thereby improving the effectiveness of similar text generation.

[0134] According to an embodiment of the present invention, by selecting different sample words to be replaced or combinations of sample words to be replaced for synonym replacement, multiple similar samples are obtained.

[0135] According to an embodiment of the present invention, by determining a more effective sample word to be replaced from the semantic information of each sample word and performing a synonym replacement to obtain a similar sample, similar sample generation can be achieved without changing the sentence semantics.

[0136] Figure 3 Shows a flowchart for determining a first probability distribution feature according to an embodiment of the present invention.

[0137] As Figure 3 shown, determining the first probability distribution feature includes operations S301 to S307.

[0138] In operation S301, obtain the number of sample words in the sample to be measured.

[0139] In operation S302, determine whether the number of sample words is greater than a preset number threshold. If the number of sample words is greater than the preset number threshold, perform operation S303. If the number of sample words is less than or equal to the preset number threshold, perform operation S307.

[0140] In operation S303, arrange multiple first prediction probabilities in ascending or descending order of numerical value to obtain a prediction probability sequence.

[0141] In operation S304, determine the top M first prediction probabilities and the bottom N first prediction probabilities from the prediction probability sequence in descending order.

[0142] In operation S305 , a first probability mean of the first M first prediction probabilities and a second probability mean of the last N first prediction probabilities are determined.

[0143] In operation S306 , a first probability distribution feature of the sample to be tested is determined based on the first probability mean and the second probability mean.

[0144] In operation S307, a change feature value calculated based on the plurality of first prediction probabilities is used as a first probability distribution feature, wherein the change feature value is determined based on a concentration index and a dispersion index for measuring the first prediction probabilities of the plurality of sample words.

[0145] Figure 4 A data flow diagram of a sample leakage detection method according to another embodiment of the present invention is shown.

[0146] like Figure 4 As shown, based on the attribute information of each of the multiple sample words in the sample to be tested 401, the sample to be tested 401 is rewritten to obtain at least one similar sample 402. Using the preset model, the first prediction probability 403 of each of the multiple sample words and the second prediction probability of each of the multiple target words included in each similar sample are determined respectively, so as to obtain at least one second prediction probability set 404 corresponding to the at least one similar sample. Distribution feature extraction is performed on the first prediction probability 403 of each of the multiple sample words to determine the first probability distribution feature 405 of the sample to be tested. Distribution feature extraction is performed on the at least one second prediction probability set 404 to obtain at least one second probability distribution feature 406.

[0147] According to an embodiment of the present invention, a distribution mean 408 is determined based on at least one second probability distribution feature 406 and the number of similar samples 407; the absolute value of the difference between the first probability distribution feature 405 and the distribution mean 408 is used as the sample distribution distance 409.

[0148] According to an embodiment of the present invention, the sample distribution distance 409 is compared with a preset distance threshold 410 to obtain a comparison result; based on the comparison result, a detection result 411 of the sample to be tested is determined.

[0149] Based on the above sample leakage detection method, the present invention also provides a sample leakage detection device. Figure 5 The device is described in detail.

[0150] Figure 5 A structural block diagram of a sample leakage detection device according to an embodiment of the present invention is shown.

[0151] like Figure 5As shown, the sample leakage detection device 500 includes a rewriting module 510 , a determination module 520 and a detection module 530 .

[0152] The rewriting module 510 is used to rewrite the text of the sample to be tested based on the attribute information of each of the multiple sample words in the sample to be tested, so as to obtain at least one similar sample.

[0153] Determination module 520 is used to use a preset model to respectively determine the first probability distribution characteristics of the sample to be tested and the second probability distribution characteristics of at least one similar sample, wherein the first probability distribution characteristics are used to characterize the distribution characteristics of the first predicted probabilities of multiple sample words predicted by the preset model, and the second probability distribution characteristics are used to characterize the distribution characteristics of the second predicted probabilities of multiple target words in similar samples predicted by the preset model.

[0154] The detection module 530 is used to perform leakage detection on the sample to be tested based on the sample distribution distance between the first probability distribution feature and at least one second probability distribution feature to obtain a detection result.

[0155] According to an embodiment of the present invention, the determination module 520 includes: a first determination submodule, a second determination submodule and a third determination submodule.

[0156] The first determination submodule is used to determine the first prediction probability of each of the multiple sample words and the second prediction probability of each of the multiple target words included in each similar sample by using a preset model.

[0157] The second determination submodule is used to extract distribution features of multiple first prediction probabilities to determine the first probability distribution features of the sample to be tested.

[0158] The third determination submodule is used to extract distribution features of the second predicted probabilities of each of the multiple target words included in the similar sample for each similar sample, and determine the second probability distribution features of the similar sample.

[0159] According to an embodiment of the present invention, the first determining submodule includes: a first input unit, a first determining unit, a second input unit and a second determining unit.

[0160] A first input unit is used to input the sample to be tested into a preset model to obtain a first position prediction probability of multiple preset words at the position of each sample word in the sample to be tested, wherein the first position prediction probability is obtained by predicting the probability of the preset word being located at the position of the sample word based on the semantic information of the previous sample word that precedes the sample word in the sample to be tested.

[0161] The first determining unit is configured to, for each sample word, use the first position prediction probability of the preset word that is the same as the sample word as the first prediction probability of the sample word.

[0162] The second input unit is used to input the similar sample into the preset model for each similar sample, so as to obtain the second position prediction probability of multiple preset words being located at the position of each target word in the similar sample, wherein the second position prediction probability is obtained by predicting the probability of the preset word being located at the position of the target word based on the semantic information of the previous target word that is located before the target word in the similar sample by the preset model.

[0163] The second determining unit is configured to, for each target word, use the second position prediction probability of the preset word identical to the target word as the second prediction probability of the target word.

[0164] According to an embodiment of the present invention, the second determination submodule includes: an arrangement submodule and a feature determination submodule.

[0165] The arrangement submodule is used to arrange the multiple first prediction probabilities according to numerical values ​​to obtain a prediction probability sequence when it is determined that the number of sample words of the multiple sample words is greater than a preset number threshold.

[0166] The feature determination submodule is used to determine the first probability distribution feature of the sample to be tested based on multiple selected prediction probabilities determined by the arrangement order of each of the multiple first prediction probabilities in the prediction probability sequence.

[0167] According to an embodiment of the present invention, the plurality of selected prediction probabilities include the first M first prediction probabilities and the last N first prediction probabilities from large to small in the prediction probability sequence, where M and N are both integers greater than or equal to 1 and are determined based on the number of sample words. The feature determination submodule includes: a mean determination unit and a first distribution determination unit.

[0168] The mean determination unit is used to determine the first probability mean of the first M first prediction probabilities and the second probability mean of the last N first prediction probabilities.

[0169] The first distribution determination unit is used to determine a first probability distribution feature of the sample to be tested based on the first probability mean and the second probability mean.

[0170] According to an embodiment of the present invention, the second determination submodule further includes: a variation feature calculation unit and a second distribution determination unit.

[0171] A change feature calculation unit is used to calculate a change feature value based on multiple first prediction probabilities when it is determined that the number of sample words of multiple sample words is less than or equal to a preset number threshold, wherein the change feature value is determined based on a concentration index and a dispersion index used to measure the first prediction probabilities of each of the multiple sample words.

[0172] The second distribution determination unit is used to use the change characteristic value as the first probability distribution feature.

[0173] According to an embodiment of the present invention, the sample leakage detection device 500 further includes: a distribution mean determination module and a distance determination module.

[0174] The distribution mean determination module is used to determine the distribution mean based on at least one second probability distribution feature and the number of similar samples.

[0175] The distance determination module is used to use the absolute value of the difference between the first probability distribution feature and the distribution mean as the sample distribution distance.

[0176] According to an embodiment of the present invention, the detection module 530 includes: a comparison submodule and a result determination submodule.

[0177] The comparison submodule is used to compare the sample distribution distance with a preset distance threshold to obtain a comparison result.

[0178] The result determination submodule is used to determine the detection result of the sample to be tested based on the comparison result.

[0179] According to an embodiment of the present invention, the result determination submodule includes: a first result determination unit.

[0180] The first result determination unit is used to determine the detection result that the sample to be tested is a leaked sample when it is determined that the comparison result indicates that the sample distribution distance is greater than a preset distance threshold, wherein the leaked sample indicates that the sample to be tested is a training sample of a preset model.

[0181] According to an embodiment of the present invention, the sample leakage detection device 500 further includes: a marking module, a similarity calculation module and a filtering module.

[0182] The marking module is used to mark the sample to be tested with a preset mark to obtain a marked sample when it is determined that the sample to be tested is a leaked sample.

[0183] The similarity calculation module is used to calculate the similarity between the marked sample and other samples to be tested in the sample set to be tested except the marked sample, and obtain a calculation result, wherein the sample to be tested is obtained from the sample set to be tested.

[0184] The filtering module is used to filter out other samples to be tested whose calculation results are greater than a preset similarity threshold.

[0185] According to an embodiment of the present invention, the attribute information includes location information. The rewriting module 510 includes: a neighbor determination module and an exchange module.

[0186] The neighbor determination module is used to determine adjacent sample words at adjacent positions based on the position information of each of the multiple sample words in the sample to be tested.

[0187] The interchange module is used to interchange the positions of adjacent sample words at adjacent positions based on at least one preset interchange order to obtain at least one similar sample.

[0188] According to an embodiment of the present invention, the attribute information includes semantic information. The rewriting module 510 includes: a word determination module and a replacement module.

[0189] The word determination module is used to determine the sample word to be replaced from the multiple sample words based on the semantic information of each of the multiple sample words.

[0190] The replacement module is used to replace the sample word to be replaced with a synonym to obtain at least one similar sample.

[0191] According to an embodiment of the present invention, any multiple modules among the rewriting module 510, the determination module 520 and the detection module 530 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the rewriting module 510, the determination module 520 and the detection module 530 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in any appropriate combination of any of them. Alternatively, at least one of the rewriting module 510, the determination module 520 and the detection module 530 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be executed.

[0192] Figure 6 A block diagram of an electronic device suitable for implementing a sample leakage detection method according to an embodiment of the present invention is shown.

[0193] like Figure 6As shown, the electronic device 600 according to an embodiment of the present invention includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 608 to a random access memory (RAM) 603. The processor 601 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include an onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0194] In RAM 603, various programs and data required for the operation of electronic device 600 are stored. Processor 601, ROM 602 and RAM 603 are connected to each other via bus 604. Processor 601 performs various operations of the method flow according to the embodiment of the present invention by executing the program in ROM 602 and / or RAM 603. It should be noted that the program can also be stored in one or more memories other than ROM 602 and RAM 603. Processor 601 can also perform various operations of the method flow according to the embodiment of the present invention by executing the program stored in the one or more memories.

[0195] According to an embodiment of the present invention, the electronic device 600 may further include an input / output (I / O) interface 605, which is also connected to the bus 604. The electronic device 600 may further include one or more of the following components connected to the input / output (I / O) interface 605: an input portion 606 including a keyboard, a mouse, etc.; an output portion 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 608 including a hard disk, etc.; and a communication portion 609 including a network interface card such as a LAN card, a modem, etc. The communication portion 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output (I / O) interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage portion 608 as needed.

[0196] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0197] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 602 and / or RAM 603 described above and / or one or more memories other than ROM 602 and RAM 603.

[0198] The embodiment of the present invention also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the sample leakage detection method provided by the embodiment of the present invention.

[0199] The computer program executes the above functions defined in the system / device of the embodiment of the present invention when it is executed by the processor 601. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0200] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 609, and / or installed from a removable medium 611. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0201] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the processor 601, the above functions defined in the system of the embodiment of the present invention are performed. According to the embodiment of the present invention, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0202] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0203] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0204] It will be appreciated by those skilled in the art that the features described in the various embodiments of the present invention may be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features described in the various embodiments of the present invention may be combined and / or combined in various ways. All of these combinations and / or combinations fall within the scope of the present invention.

[0205] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination advantageously. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A sample leakage detection method, characterized in that: The method comprises: Based on the attribute information of each of the multiple sample words in the sample to be tested, rewrite the text of the sample to be tested to obtain at least one similar sample; Determine the first probability distribution feature of the sample to be tested and the second probability distribution feature of at least one of the similar samples respectively by using a preset model, wherein the first probability distribution feature is used to characterize the distribution feature of the first predicted probabilities of the plurality of sample words predicted by the preset model, and the second probability distribution feature is used to characterize the distribution feature of the second predicted probabilities of the plurality of target words in the similar samples predicted by the preset model; Based on the sample distribution distance between the first probability distribution feature and at least one of the second probability distribution features, performing leakage detection on the sample to be tested to obtain a detection result; The first probability distribution feature is determined in the following manner: When it is determined that the number of sample words of the plurality of sample words is less than or equal to a preset number threshold, a change characteristic value is calculated based on the plurality of first prediction probabilities, wherein the change characteristic value is determined based on a concentration index and a dispersion index used to measure the first prediction probabilities of the plurality of sample words; The change characteristic value is used as the first probability distribution feature.

2. The method according to claim 1, characterized in that The method of using a preset model to respectively determine the first probability distribution feature of the sample to be tested and the second probability distribution feature of at least one of the similar samples includes: Using the preset model, respectively determining a first prediction probability of each of the plurality of sample words and a second prediction probability of each of the plurality of target words included in each of the similar samples; Extracting distribution features of the plurality of first prediction probabilities to determine first probability distribution features of the sample to be tested; For each of the similar samples, distribution features are extracted for the second predicted probabilities of the plurality of target words included in the similar sample to determine the second probability distribution features of the similar sample.

3. The method according to claim 2, characterized in that The using the preset model to respectively determine the first prediction probability of each of the plurality of sample words and the second prediction probability of each of the plurality of target words included in each of the similar samples comprises: Inputting the sample to be tested into the preset model, obtaining a first position prediction probability of a plurality of preset words located at the position where each sample word is located in the sample to be tested, wherein the first position prediction probability is obtained by predicting the probability of the preset word being located at the position where the sample word is located based on the semantic information of a previous sample word located before the sample word in the sample to be tested by the preset model; For each of the sample words, taking the first position prediction probability of the preset word that is the same as the sample word as the first prediction probability of the sample word; For each of the similar samples, the similar samples are input into the preset model to obtain a plurality of second position prediction probabilities of the preset words being located at the position of each of the target words in the similar samples, wherein the second position prediction probability is obtained by predicting the probability of the preset word being located at the position of the target word by the preset model based on the semantic information of the previous target word that precedes the target word in the similar samples; For each of the target words, the second position prediction probability of the preset word that is the same as the target word is used as the second prediction probability of the target word.

4. The method according to claim 2, characterized in that: The extracting distribution features of the plurality of first prediction probabilities to determine the first probability distribution features of the sample to be tested includes: When it is determined that the number of sample words of the plurality of sample words is greater than a preset number threshold, the plurality of first prediction probabilities are arranged according to numerical values ​​to obtain a prediction probability sequence; Based on a plurality of selected prediction probabilities determined by respective arrangement orders of a plurality of the first prediction probabilities in the prediction probability sequence, a first probability distribution feature of the sample to be tested is determined.

5. The method according to claim 4, characterized in that The multiple selected prediction probabilities include the first M first prediction probabilities and the last N first prediction probabilities from large to small in the prediction probability sequence, where M and N are both integers greater than or equal to 1 and are both determined based on the number of sample words; Wherein, determining the first probability distribution feature of the sample to be tested based on a plurality of selected prediction probabilities determined by respective arrangement orders of a plurality of the first prediction probabilities in the prediction probability sequence includes: Determine a first probability mean of the first M first predicted probabilities and a second probability mean of the last N first predicted probabilities; Based on the first probability mean and the second probability mean, a first probability distribution feature of the sample to be tested is determined.

6. The method according to claim 1, characterized in that The sample distribution distance is determined by: Determining a distribution mean based on at least one of the second probability distribution characteristics and the number of the similar samples; The absolute value of the difference between the first probability distribution feature and the distribution mean is used as the sample distribution distance.

7. The method according to claim 1, characterized in that The step of performing leakage detection on the sample to be tested based on the sample distribution distance between the first probability distribution feature and at least one of the second probability distribution features to obtain a detection result includes: Comparing the sample distribution distance with a preset distance threshold to obtain a comparison result; Based on the comparison result, a detection result of the sample to be tested is determined.

8. The method according to claim 7, characterized in that Determining the detection result of the sample to be tested based on the comparison result includes: When it is determined that the comparison result indicates that the sample distribution distance is greater than the preset distance threshold, the detection result is determined that the sample to be tested is a leaked sample, wherein the leaked sample indicates that the sample to be tested is a training sample of the preset model.

9. The method according to claim 8, characterized in that The method further comprises: In the case where it is determined that the sample to be tested is the leaked sample, marking the sample to be tested with a preset mark to obtain a marked sample; Calculating the similarity between the labeled sample and other samples to be tested in the sample set to be tested except the labeled sample, and obtaining a calculation result, wherein the sample to be tested is obtained from the sample set to be tested; Other samples to be tested whose calculation results represent a similarity greater than a preset similarity threshold are filtered out.

10. The method according to claim 1, characterized in that The attribute information includes position information; the text rewriting of the sample to be tested based on the attribute information of each of the multiple sample words in the sample to be tested to obtain at least one similar sample includes: Determine adjacent sample words at adjacent positions based on position information of each of the plurality of sample words in the sample to be tested; Based on at least one preset exchange order, the adjacent sample words at adjacent positions are exchanged to obtain at least one similar sample.

11. The method according to claim 1, characterized in that: The attribute information includes semantic information; the text rewriting of the sample to be tested based on the attribute information of each of the multiple sample words in the sample to be tested to obtain at least one similar sample includes: Based on the semantic information of each of the plurality of sample words, determining a sample word to be replaced from the plurality of sample words; The sample words to be replaced are replaced with synonyms to obtain at least one similar sample.

12. An electronic device comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 11.

13. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

14. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

Images