Data leakage detection method and device, electronic equipment and storage medium

Abstract

The invention provides a data leakage detection method and device, electronic equipment and a storage medium, and relates to the technical field of computers. The method comprises the following steps: acquiring to-be-detected data, and performing disturbance processing on the to-be-detected data to generate at least two derivative data sets; obtaining a to-be-detected model, and inputting each derivative data set into the to-be-detected model to generate a probability data set; and performing outlier detection based on the probability data set, and generating a detection result indicating whether data leakage occurs between the to-be-detected data and the to-be-detected model. According to the invention, the problem of low accuracy of data leakage detection in the prior art is solved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology. Specifically, this application relates to a data leakage detection method, apparatus, electronic device, and storage medium. Background Art

[0002] During the training process of large language models (LLMs), due to data leakage problems between the pre-training data and the benchmark test data sets, the reliability of the test results of the model will be reduced. Therefore, it is necessary to detect whether there is data leakage between the model and the test data.

[0003] The data leakage detection methods in the current technology include using post hoc n-gram overlap analysis between the benchmark and the pre-training corpus to measure data leakage; or using benchmark perturbation and synthetic data to detect data leakage; and determining whether there is data leakage by comparing the losses of the model on the training, validation, and test sets. However, these methods detect the differences between data sets, are computationally complex during the leakage detection process, and have poor detection effects when there are out-of-order problems in the test data, resulting in low accuracy.

[0004] As can be seen from the above, how to improve the accuracy of data leakage detection remains to be solved. Summary of the Invention

[0005] This application provides a data leakage detection method, apparatus, electronic device, and storage medium, which can solve the problem of low accuracy in data leakage detection in the related technology. The technical solutions are as follows:

[0006] According to one aspect of this application, a data leakage detection method is characterized by including:

[0007] Obtain the data to be detected, and perform a scrambling process on the data to be detected to generate at least two derivative data sets;

[0008] Obtain the model to be detected, and input each of the derivative data sets into the model to be detected to generate a probability data set;

[0009] Based on the probability data set, perform an outlier detection to generate a detection result indicating whether there is data leakage between the data to be detected and the model to be detected.

[0010] According to one aspect of this application, a data leakage detection apparatus is characterized by including:

[0011] A data acquisition module, configured to obtain the data to be detected, and perform a scrambling process on the data to be detected to generate at least two derivative data sets;

[0012] A score calculation module, configured to obtain a model to be detected, and input each of the derivative data sets into the model to be detected to generate a probability data set;

[0013] A detection module, configured to perform outlier detection based on the probability data set to generate a detection result indicating whether data leakage has occurred between the data to be detected and the model to be detected.

[0014] In an exemplary embodiment, the data to be detected includes problem information and initial option information corresponding to the problem information.

[0015] In an exemplary embodiment, the data acquisition module includes:

[0016] A perturbation unit, configured to randomly perturb the initial option information corresponding to the problem information to obtain at least two pieces of derivative option information corresponding to each piece of problem information;

[0017] A derivative data set generation unit, configured to generate a derivative data set based on the problem information and the corresponding derivative option information.

[0018] In an exemplary embodiment, the score calculation module includes:

[0019] A probability calculation unit, configured to input each of the derivative data sets into the model to be detected for logarithmic probability distribution calculation to obtain the logarithmic probability corresponding to each derivative data set;

[0020] A probability data set generation unit, configured to generate a probability data set based on the logarithmic probability corresponding to each derivative data set.

[0021] In an exemplary embodiment, the detection module includes:

[0022] A score calculation unit, configured to calculate the outlier score of each data point in the probability data set based on an outlier algorithm;

[0023] A detection unit, configured to perform outlier detection on each of the outlier scores based on a preset threshold to determine whether there is an outlier in the outlier scores; if there is an outlier, generate a detection result determining that data leakage has occurred, and if there is no outlier, generate a detection result indicating that there is no data leakage.

[0024] In an exemplary embodiment, the outlier algorithm is the Isolation Forest algorithm.

[0025] In an exemplary embodiment, the model to be detected is a large language model.

[0026] According to one aspect of the present application, an electronic device includes at least one processor and at least one memory, wherein computer-readable instructions are stored on the memory; the computer-readable instructions are executed by one or more of the processors, so that the electronic device implements the data leakage detection method as described above.

[0027] According to one aspect of the present application, a storage medium stores computer-readable instructions, and the computer-readable instructions are executed by one or more processors to implement the data leakage detection method as described above.

[0028] According to one aspect of the present application, a computer program product includes computer-readable instructions. The computer-readable instructions are stored in a storage medium, and one or more processors of an electronic device read the computer-readable instructions from the storage medium, load and execute the computer-readable instructions, so that the electronic device implements the data leakage detection method as described above.

[0029] The beneficial effects brought by the technical solution provided by the present application are:

[0030] In the above technical solution, a derivative data set is generated by disturbing the data to be detected, and then a probability data set corresponding to the derivative data set is generated according to the model to be detected. Whether data leakage occurs between the data to be detected and the model to be detected is determined by detecting the outlier detection of the probability data set. By introducing disordered derivative data into the data set and judging data leakage through outliers, the detection process is not only simple and effective, but also the calculation efficiency is improved, and the detection accuracy of information leakage is guaranteed, so that the problem of low accuracy of data leakage detection in the related technology can be effectively solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments of the present application. Obviously, the following described drawings are only some embodiments of the present application, and those skilled in the art can obtain other drawings without creative efforts based on these drawings.

[0032] Figure 1 is a schematic diagram of the implementation environment related to the present application;

[0033] Figure 2 is a flowchart of a data leakage detection method shown according to an exemplary embodiment;

[0034] Figure 3 is Figure 2 a flowchart of step 210 in the corresponding embodiment in one embodiment;

[0035] Figure 4 Yes Figure 2 It is a flowchart of step 230 in a corresponding embodiment in one embodiment;

[0036] Figure 5 Yes Figure 2 It is a flowchart of step 250 in a corresponding embodiment in one embodiment;

[0037] Figure 6 It is a schematic diagram of a specific implementation of a data leakage detection method in an application scenario;

[0038] Figure 7 It is a structural block diagram of a data leakage detection device shown according to an exemplary embodiment;

[0039] Figure 8 It is a hardware structure diagram of a server shown according to an exemplary embodiment;

[0040] Figure 9 It is a structural block diagram of an electronic device shown according to an exemplary embodiment. Detailed implementation manners

[0041] The embodiments of the present application will be described in detail below. The examples of the embodiments are shown in the drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions from beginning to end. The embodiments described below with reference to the drawings are exemplary and are only used to explain the present application and cannot be construed as a limitation to the present application.

[0042] Those skilled in the art of the present technology can understand that unless specifically stated otherwise, the singular forms "a", "an", "the" and "said" used herein may also include the plural forms. It should be further understood that the term "including" used in the specification of the present disclosure means the presence of the described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or their groups. It should be understood that when we say an element is "connected" or "coupled" to another element, it can be directly connected or coupled to other elements, or there may also be intermediate elements. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The phrase "and / or" used herein includes all or any unit and all combinations of one or more related listed items.

[0043] As described above, in the existing data leakage detection methods, the differences between data sets are detected, which is computationally complex during the leakage detection process and has poor detection effects when there are out-of-order problems in the test data, resulting in low accuracy.

[0044] As can be seen from the above, there are still defects in the accuracy of data leakage detection in the related art.

[0045] To this end, the data leakage detection method provided by this application can effectively improve the accuracy of data leakage detection. Correspondingly, this data leakage detection method is applicable to a data leakage detection device, which can be deployed on an electronic device. The electronic device can be a computer device configured with a von Neumann architecture. For example, the computer device includes a desktop computer, a laptop computer, a server, etc.; the electronic device can also be an electronic device with a central control function. For example, the electronic device includes a gateway, etc.; the electronic device can also refer to a portable mobile electronic device. For example, the electronic device includes a smart phone, a tablet computer, etc.

[0046] To make the objectives, technical solutions, and advantages of this application clearer, the following will further describe the embodiments of this application in detail with reference to the accompanying drawings.

[0047] Figure 1 It is a schematic diagram of an implementation environment involved in an image processing method. It should be noted that this implementation environment is only an example adapted to the present invention and should not be considered as providing any limitation to the scope of use of the present invention.

[0048] This implementation environment includes a collection end 110 and a server end 130.

[0049] Specifically, the collection end 110 can be considered as an information collection device, including but not limited to electronic devices with information collection functions such as cameras and locators, or can also be considered as a device integrating information collection and data leakage detection, including but not limited to laptop computers and desktop computers with both information collection functions and data leakage detection functions. That is to say, in some embodiments, the collection end 110 can collect data to be detected.

[0050] The server end 130 can also be considered as a data leakage detection device, including but not limited to electronic devices with data leakage detection functions such as smart phones, desktop computers, laptop computers, and servers, and can also be a computer cluster composed of multiple servers, or even a cloud computing center composed of multiple servers. Among them, the server end 130 is used to provide background services. For example, the background services include but are not limited to data leakage detection services, etc.

[0051] A network communication connection is pre-established between the server end 130 and the collection end 110 by means of wired or wireless, etc., and data transmission between the server end 130 and the collection end 110 is realized through this network communication connection. The transmitted data includes but is not limited to: data to be detected, etc.

[0052] In an application scenario, through the interaction between the acquisition end 110 and the server end 130, the acquisition end 110 acquires the data to be detected, and uploads the data to be detected to the server end 130 to request the server end 130 to provide a data leakage detection service.

[0053] At this time, for the server end 130, after receiving the data to be detected uploaded by the acquisition end 110, it calls the data leakage detection service, and detects whether there is data leakage between the data to be detected and the model to be detected, solving the problem of low accuracy of data leakage detection in the related art.

[0054] Please refer to Figure 2 , the embodiment of the present application provides a data leakage detection method, which is applicable to an electronic device, and the electronic device can be Figure 1 the server end 130 in the shown implementation environment, and can also be a desktop computer, a laptop computer, a server, etc.

[0055] In the following method embodiments, for the convenience of description, the execution subject of each step of the method is taken as an example of an electronic device for illustration, but it is not specifically limited thereto.

[0056] As Figure 2 shown, the method may include the following steps:

[0057] Step 210, obtain the data to be detected, and perform a scrambling process on the data to be detected to generate at least two derivative data sets.

[0058] Among them, the data to be detected and the model to be detected are the detection objects of data leakage detection. The derivative data sets generated by scrambling the data to be detected can introduce a disordered relationship into the data set to be detected, so that the generated derivative data sets can more comprehensively reflect the characteristics of the data to be detected.

[0059] In an exemplary embodiment, the data to be detected includes problem information and initial option information corresponding to the problem information.

[0060] Step 230, obtain the model to be detected, and input each derivative data set into the model to be detected to generate a probability data set.

[0061] Among them, the model to be detected is a trained deep learning model with the ability to perform logarithmic probability calculation on the derivative data set.

[0062] In an exemplary embodiment, the model to be detected is a large language model.

[0063] Step 250, perform an outlier detection based on the probability data set to generate a detection result indicating whether there is data leakage between the data to be detected and the model to be detected.

[0064] It should be noted that when the inventor notices data leakage between the model to be detected and the data to be detected, for example, if the model to be detected has trained a certain data to be detected, the log probability of the model to be detected predicting this data will be significantly higher than other data. At this time, there will be a maximum and outlier value in the generated log probability set. If the model has not trained the data to be detected, then in the case of disorder, for example, swapping the content between options, the log probability may increase or decrease, and no obvious outlier will appear.

[0065] Therefore, by performing outlier detection to determine whether there are outliers in the probability dataset, it can be determined whether data leakage has occurred between the data to be detected and the model to be detected.

[0066] Through the above process, by introducing disordered derivative data into the dataset and judging data leakage through outliers, it not only ensures that the detection process is simple and effective, but also improves the calculation efficiency and ensures the detection accuracy of information leakage, thus effectively solving the problem of low accuracy in data leakage detection in the related art.

[0067] In an exemplary embodiment, as Figure 3 shown, step 210 includes the following steps:

[0068] Step 211, randomly disrupt the initial option information corresponding to the problem information to obtain at least two pieces of derivative option information corresponding to each problem information.

[0069] Specifically, by randomly disrupting the order of the initial option information corresponding to each problem information and randomly combining them, derivative option information including the initial option information and its derivatives can be generated.

[0070] Step 213, generate a derivative dataset based on the problem information and the corresponding derivative option information.

[0071] After derivative option information has been generated for all the problem information in the data to be detected, a derivative dataset is generated by integrating each problem information and its corresponding derivative option information.

[0072] Through the above process, the generation of derivative information for the data to be detected is realized, ensuring the generalization of the detection process.

[0073] In an exemplary embodiment, as Figure 4 shown, step 230 may include the following steps:

[0074] Step 231, input each derivative dataset into the model to be detected for log probability distribution calculation to obtain the log probabilities corresponding to each derivative dataset.

[0075] Specifically, the logarithmic probability of each derivative dataset is calculated by the model to be detected, and the corresponding logarithmic probability is generated through the prediction of the derivative dataset by the model to be detected.

[0076] Step 233: Generate a probability dataset based on the logarithmic probabilities corresponding to each derivative dataset.

[0077] Among them, after calculating the logarithmic probability distribution of each derivative dataset, the logarithmic probabilities of each derivative dataset are used as data points to generate a probability dataset.

[0078] Through the above process, the degree of data leakage of the derivative dataset is quantified as a logarithmic probability, thereby improving the accuracy and detection efficiency of data leakage detection.

[0079] In an exemplary embodiment, as Figure 5 shown, step 250 may include the following steps:

[0080] Step 251: Calculate the outlier scores of each data point in the probability dataset based on an outlier algorithm.

[0081] In a possible implementation, the outlier algorithm is the Isolation Forest algorithm.

[0082] Step 253: Perform outlier detection on each outlier score based on a preset threshold to determine whether there are outliers in the outlier scores. If there are outliers, generate a detection result indicating that data leakage has occurred; if there are no outliers, generate a detection result indicating that there is no data leakage.

[0083] In a possible implementation, the preset threshold is a threshold set according to the outlier algorithm and user requirements. For example, when the outlier algorithm is the Isolation Forest algorithm, since the smaller the outlier score in the Isolation Forest represents more outlier, the minimum value of the outlier score is set. When there is a situation where each outlier score is less than the preset threshold, it is determined that data leakage has occurred.

[0084] Through the above process, it is determined whether there is data leakage through the outlier scores, improving the detection effectiveness and detection accuracy.

[0085] Figure 6 is a specific implementation schematic diagram of a data leakage detection method in an application scenario. In this application scenario, the data to be tested is obtained, which includes problem information All of the following tissues are connective tissues, except the following, and the corresponding initial option information, namely ligaments, muscles, blood, and cartilage.

[0086] By randomly scrambling the option order in the original data x, 24 different derivative datasets X = {x1, x2,..., x n!} are generated, where n is the number of options.

[0087] Input each piece of derivative data into the large language model to obtain, for each derivative data set x i The log probability logp output by model M i ;

[0088] Then obtain the log probability set of the entire derivative data

[0089] Use the Isolation Forest algorithm to calculate the outlier score for each data point and determine the outlier score s of the derivative data with the maximum log probability out .

[0090] According to the preset threshold δ, judge s out Whether it is less than δ. If s out <δ, it is considered that the data x is leaked; otherwise, it is considered that x is not leaked. Finally, output the result of whether there is data leakage between the model to be detected and the data to be detected

[0091] In this application scenario, the verification and detection accuracy rate of the present invention is greater than 70%, achieving focus on the leakage problem of multiple-choice question format data in the large language model. The method is simple and effective, and can effectively detect data leakage even when the option content in the test data is scrambled

[0092] Please refer to Figure 7 , in the embodiment of the present application, a data leakage detection device 700 is provided, including but not limited to: a data acquisition module 710, a score calculation module 730, and a detection module 750

[0093] Among them, the data acquisition module 710 is used to acquire the data to be detected and perform a scrambling process on the data to be detected to generate at least two derivative data sets

[0094] The score calculation module 730 is used to obtain the model to be detected and input each derivative data set into the model to be detected to generate a probability data set

[0095] The detection module 750 is used to perform outlier detection based on the probability data set and generate a detection result indicating whether there is data leakage between the data to be detected and the model to be detected

[0096] It should be noted that when the above-mentioned data leakage detection device performs data leakage detection, only the above-mentioned division of each functional module is used for illustration. In actual applications, the above-mentioned functions can be allocated to different functional modules according to needs, that is, the internal structure of the data leakage detection device will be divided into different functional modules to complete all or part of the functions described above

[0097] In addition, the data leakage detection device and the embodiments of the data leakage detection method provided in the above embodiments belong to the same concept. The specific manners in which each module performs operations have been described in detail in the method embodiments and will not be elaborated here.

[0098] Figure 8 The structural schematic diagram of a server shown according to an exemplary embodiment. This server is applicable to Figure 1 the server 130 in the shown implementation environment.

[0099] It should be noted that this server is only an example adapted to this application and cannot be considered as providing any limitation to the scope of use of this application. This server cannot be interpreted as requiring dependence on or necessarily having Figure 8 one or more components in the shown exemplary server 2000.

[0100] The hardware structure of the server 2000 may vary greatly due to different configurations or performances. For example, Figure 8 as shown, the server 2000 includes: a power supply 210, an interface 230, at least one memory 250, and at least one central processing unit (CPU) 270.

[0101] Specifically, the power supply 210 is used to provide working voltage for each hardware device on the server 2000.

[0102] The interface 230 includes at least one wired or wireless network interface 231 for interacting with external devices. For example, for Figure 1 the interaction between the terminal 100 and the server 200 in the shown implementation environment.

[0103] Of course, in other examples adapted to this application, the interface 230 may further include at least one serial-parallel conversion interface 233, at least one input-output interface 235, and at least one USB interface 237, etc. Figure 8 As shown, this is not a specific limitation here.

[0104] The memory 250, as a carrier for resource storage, can be a read-only memory, a random access memory, a magnetic disk, an optical disk, etc. The resources stored thereon include an operating system 251, application programs 253, and data 255, etc. The storage method can be short-term storage or permanent storage.

[0105] Among them, the operating system 251 is used to manage and control each hardware device and application program 253 on the server 2000, so as to implement the operation and processing of the massive data 255 in the memory 250 by the central processing unit 270. It can be Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSD TM, etc.

[0106] The application program 253 is a computer-readable instruction that completes at least one specific task based on the operating system 251. It can include at least one module ( Figure 8 (not shown), and each module can separately contain computer-readable instructions for the server 2000. For example, the data leakage detection device can be regarded as an application program 253 deployed on the server 2000.

[0107] The data 255 can be photos, pictures, etc. stored in the disk, or recommendation information, etc., and is stored in the memory 250.

[0108] The central processing unit 270 can include one or more than one processors, and is set to communicate with the memory 250 through at least one communication bus, so as to read the computer-readable instructions stored in the memory 250, and then implement the operation and processing of the massive data 255 in the memory 250. For example, the data leakage detection method is completed in the form of reading a series of computer-readable instructions stored in the memory 250 by the central processing unit 270.

[0109] In addition, the present application can also be implemented by a hardware circuit or a combination of a hardware circuit and software. Therefore, the implementation of the present application is not limited to any specific hardware circuit, software, and the combination of the two.

[0110] Please refer to Figure 9 , in the embodiments of the present application, an electronic device 4000 is provided. The electronic device 400 can include: a desktop computer, a notebook computer, a server, etc.

[0111] In Figure 9 , the electronic device 4000 includes at least one processor 4001 and at least one memory 4003.

[0112] Among them, the data interaction between the processor 4001 and the memory 4003 can be realized through at least one communication bus 4002. The communication bus 4002 may include a path for transmitting data between the processor 4001 and the memory 4003. The communication bus 4002 can be a PCI (Peripheral Component Interconnect) bus, an EISA (Extended Industry Standard Architecture) bus, or the like. The communication bus 4002 can be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience of representation, Figure 9 only a thick line is used to represent it in the figure, but it does not mean that there is only one bus or one type of bus.

[0113] Optionally, the electronic device 4000 may further include a transceiver 4004, and the transceiver 4004 can be used for data interaction between the electronic device and other electronic devices, such as data sending and / or data receiving. It should be noted that in practical applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present application.

[0114] The processor 4001 can be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute various exemplary logic blocks, modules, and circuits described in combination with the disclosure of the present application. The processor 4001 can also be a combination that realizes computing functions, such as a combination including one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0115] The memory 4003 can be a ROM (Read Only Memory), or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory), or other types of dynamic storage devices that can store information and instructions. It can also be an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory), or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media, or other magnetic storage devices, or any other medium that can be used to carry or store the desired program instructions or code in the form of instructions or data structures and can be accessed by the electronic device 400, but is not limited thereto.

[0116] Computer-readable instructions are stored on the memory 4003, and the processor 4001 can read the computer-readable instructions stored in the memory 4003 through the communication bus 4002.

[0117] The computer-readable instructions are executed by one or more processors 4001 to implement the data leakage detection method in the above embodiments.

[0118] In addition, an embodiment of the present application provides a storage medium on which computer-readable instructions are stored, and the computer-readable instructions are executed by one or more processors to implement the data leakage detection method as described above.

[0119] An embodiment of the present application provides a computer program product. The computer program product includes computer-readable instructions. The computer-readable instructions are stored in a storage medium, and one or more processors of the electronic device read the computer-readable instructions from the storage medium, load and execute the computer-readable instructions, so that the electronic device implements the data leakage detection method as described above.

[0120] It should be understood that although the steps in the flowchart of the accompanying drawings are shown sequentially according to the indication of the arrows, these steps are not necessarily executed sequentially according to the order indicated by the arrows. Unless otherwise clearly stated in this article, the execution of these steps is not strictly limited in order, and they can be executed in other orders. Moreover, at least some of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed alternately or alternately with at least a part of other steps or sub-steps or stages of other steps.

[0121] The above are only some embodiments of the present application. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present application, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present application.

Claims

1. A data leakage detection method, characterized in that, Including: Obtain the data to be detected, and perform perturbation processing on the data to be detected to generate at least two derivative data sets; Obtain the model to be detected, and input each of the derivative data sets into the model to be detected to generate a probability data set; Based on the probability data set, perform outlier detection to generate a detection result indicating whether data leakage has occurred between the data to be detected and the model to be detected.

2. The method according to claim 1, wherein The data to be detected includes problem information and initial option information corresponding to the problem information.

3. The method according to claim 2, wherein The obtaining the data to be detected and performing perturbation processing on the data to be detected to generate at least two derivative data sets includes: Perform random perturbation on the initial option information corresponding to the problem information to obtain at least two types of derivative option information corresponding to each problem information; Based on the problem information and the corresponding derivative option information, generate a derivative data set.

4. The method according to claim 1, characterized in that, The inputting each of the derivative data sets into the model to be detected to generate a probability data set includes: Input each of the derivative data sets into the model to be detected for logarithmic probability distribution calculation to obtain the logarithmic probability corresponding to each derivative data set; Based on the logarithmic probabilities corresponding to each of the derivative data sets, generate a probability data set.

5. The method according to claim 1, characterized in that The performing outlier detection based on the probability data set to generate a detection result indicating whether data leakage has occurred between the data to be detected and the model to be detected includes: Calculate the outlier scores of each data point in the probability data set based on an outlier algorithm; Based on a preset threshold, perform outlier detection on the outlier scores to determine whether there is an outlier among the outlier scores; if there is an outlier, generate a detection result determining that data leakage has occurred, and if there is no outlier, generate a detection result indicating that there is no data leakage.

6. The method according to claim 5, wherein The outlier algorithm is the Isolation Forest algorithm.

7. The method according to claim 1, characterized in that, The model to be detected is a large language model.

8. A data leakage detection device, characterized in that, Including: A data acquisition module for obtaining the data to be detected and performing perturbation processing on the data to be detected to generate at least two derivative data sets; A score calculation module for obtaining the model to be detected and inputting each of the derivative data sets into the model to be detected to generate a probability data set; A detection module for performing outlier detection based on the probability data set to generate a detection result indicating whether data leakage has occurred between the data to be detected and the model to be detected.

9. An electronic device, characterized in that, Including: At least one processor and at least one memory, wherein, The memory stores computer-readable instructions.

10. A storage medium having computer-readable instructions stored thereon, characterized in that, The computer-readable instructions are executed by one or more processors to implement the data leakage detection method according to any one of claims 1 to 9.

Images