Model sensitive data detection method and related equipment

Abstract

The present application provides a model sensitive data detection method, including loading a model file of a model to be detected, determining a framework type of the model file, obtaining model graph structure data corresponding to the model file, and obtaining a model parameter set corresponding to the model file from the model graph structure data; performing sensitive data detection on the model parameters in the model parameter set to obtain a first detection result; when the first detection result indicates that the model parameter set does not contain sensitive data, performing abnormal hidden data detection on the weight parameters and bias parameters corresponding to the model to be detected in the model parameter set to obtain a second detection result; when the second detection result indicates that the model parameter set does not contain abnormal hidden data, performing a safe operation detection on the model to be detected in an isolated environment to obtain an operation detection result, so as to realize multi-level sensitive data detection, ensure the comprehensiveness and depth of detection, and improve the detection efficiency and detection accuracy of sensitive data detection.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of data processing technology, and in particular to a model-sensitive data detection method and related equipment. Background Art

[0002] Existing machine learning models may involve user sensitive data, such as user privacy information and commercial confidential information, when processing and storing data, which may lead to the risk of leakage or abuse of these data, and may cause serious damage to individuals or organizations. Therefore, it is currently necessary to detect model sensitive data to determine the security of the model.

[0003] However, existing security detection methods generally require formal verification of parameters such as model accuracy based on specific verification tools and mathematical models to confirm whether sensitive data exists. The complex detection process leads to low detection efficiency. In addition, since current neural network models have strong robustness, the model will not suffer a large loss of accuracy due to the inclusion of private data. As a result, the existing security detection methods are not comprehensive and in-depth, and are prone to false positives or omissions, affecting detection accuracy and unable to ensure the security of model operation. Summary of the Invention

[0004] An embodiment of the present application provides a model sensitive data detection method that can quickly and effectively perform multi-level sensitive data detection on the model files of the model to be detected, so as to ensure the comprehensiveness and depth of the detection, thereby improving the detection efficiency and detection accuracy of sensitive data to ensure the security of model operation.

[0005] To achieve the above-mentioned objectives, the first aspect of an embodiment of the present application provides a model sensitive data detection method, comprising: loading a model file of a model to be detected, and reading a file data stream in the model file; performing model parsing processing on the file data stream to determine the framework type of the model file; obtaining model graph structure data corresponding to the model file according to the framework type and the file data stream, and obtaining a model parameter set corresponding to the model file from the model graph structure data; performing sensitive data detection on the model parameters in the model parameter set based on a preset sensitive word library to obtain a first detection result; when the first detection result indicates that the model parameter set does not contain sensitive data, performing abnormal hidden data detection on the weight parameters and bias parameters corresponding to the model to be detected in the model parameter set to obtain a second detection result; when the second detection result indicates that the model parameter set does not contain abnormal hidden data, performing a safe operation detection on the model to be detected in an isolated environment to obtain an operation detection result.

[0006] In some embodiments, the model parsing processing is performed on the file data stream to determine the framework type of the model file, including: obtaining an extension name dataset, the extension name dataset including file extensions corresponding to multiple framework types; parsing the file extensions of multiple file data from the file data stream, and matching the framework type corresponding to the model file from the extension name dataset based on the file extensions of the multiple file data.

[0007] In some embodiments, the obtaining of the model graph structure data corresponding to the model file according to the framework type and the file data stream includes: performing file screening processing on the file data stream based on a preset exception regular library to obtain a plurality of file data in the file data stream that conforms to the expected file format; obtaining a plurality of target file types according to a plurality of file extensions corresponding to the plurality of file data, and selecting corresponding model renderers for the plurality of target file types according to the framework type; decoding processing on the plurality of file data based on the plurality of model renderers to obtain a plurality of image object data and node object data; and obtaining the model graph structure data corresponding to the model file according to the plurality of image object data and the plurality of node object data.

[0008] In some embodiments, before the file data stream is subjected to file screening processing based on a preset exception regular library, the method further includes: performing compressed data detection on the file data stream; and when the file data stream contains compressed data, decompressing the compressed data to obtain file data corresponding to the compressed data.

[0009] In some embodiments, after obtaining the model graph structure data corresponding to the model file according to the framework type and the file data stream, it also includes: obtaining a model visualization tool corresponding to the framework type from a preset algorithm library; and generating a visual model view corresponding to the model graph structure data based on the model visualization tool.

[0010] In some embodiments, the abnormal hidden data detection is performed on the weight parameters and bias parameters corresponding to the model to be detected in the model parameter set to obtain a second detection result, including: performing model anomaly analysis based on the weight parameters and bias parameters corresponding to the model to be detected in the model parameter set to obtain a model anomaly index corresponding to the model to be detected, and the model anomaly index includes at least one of the following: uniform distribution coefficient, output difference coefficient, similarity coefficient and injection detection result; determining whether the model parameter set contains abnormal hidden data based on the model anomaly index to obtain a second detection result.

[0011] In some embodiments, the process of obtaining the uniform distribution coefficient is as follows: obtaining the least significant bit distribution information of the weight parameters and bias parameters corresponding to the model to be detected in the model parameter set; and calculating the uniform distribution coefficient corresponding to the model parameter set based on the least significant bit distribution information.

[0012] In some embodiments, the process of obtaining the output difference coefficient is as follows: generating multiple test samples according to the weight parameters and bias parameters corresponding to the model to be detected in the model parameter set, and generating multiple adversarial samples based on the multiple test samples and a preset loss function; respectively inputting the multiple test samples and the multiple adversarial samples into the model to be detected to calculate the output difference coefficients of the test samples and the adversarial samples.

[0013] In some embodiments, the process of obtaining the similarity coefficient is as follows: performing federated learning training based on the weight parameters and bias parameters corresponding to the model to be detected in the model parameter set to obtain a sample model corresponding to the model to be detected; and calculating the similarity coefficient between the model to be detected and the sample model.

[0014] In some embodiments, the process of obtaining the injection detection result is as follows: obtaining binary data corresponding to the weight parameters and bias parameters of the model to be detected in the model parameter set; performing injection detection on the operation code field in the binary data to obtain the injection detection result.

[0015] To achieve the above-mentioned purpose, the second aspect of an embodiment of the present application provides a model sensitive data detection device, comprising: a model parsing module, used to load the model file of the model to be detected and read the file data stream in the model file; perform model parsing processing on the file data stream to determine the framework type of the model file; obtain the model graph structure data corresponding to the model file according to the framework type and the file data stream, and obtain the model parameter set corresponding to the model file from the model graph structure data; a sensitive data detection module, used to perform sensitive data inclusion detection on the model parameters in the model parameter set based on a preset sensitive word library to obtain a first detection result; a hidden data detection module, used to perform abnormal hidden data detection on the weight parameters and bias parameters corresponding to the model to be detected in the model parameter set when the first detection result indicates that the model parameter set does not contain sensitive data, to obtain a second detection result; a model operation module, used to perform safe operation detection on the model to be detected in an isolated environment when the second detection result indicates that the model parameter set does not contain abnormal hidden data, to obtain an operation detection result.

[0016] To achieve the above-mentioned objectives, the third aspect of an embodiment of the present application provides an electronic device, comprising: at least one processor; at least one memory for storing at least one program; and when at least one of the programs is executed by at least one of the processors, a model-sensitive data detection method as described in any one of the first aspects is implemented.

[0017] To achieve the above-mentioned objectives, the fourth aspect of an embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the model-sensitive data detection method as described in any one of the first aspects.

[0018] The model sensitive data detection method and related equipment proposed in the embodiments of the present application have at least the following beneficial effects: the embodiments of the present application load the model file of the model to be detected, read the file data stream in the model file, and perform model parsing processing on the file data stream to determine the framework type of the model file, thereby quickly and effectively obtaining the model graph structure data corresponding to the model file according to the framework type and the file data stream, and obtaining the model parameter set corresponding to the model file from the model graph structure data; further, a preliminary sensitive data inclusion detection is performed on the model parameter set through a preset sensitive word library to ensure that the model parameter set does not directly contain sensitive data, and on the basis of the preliminary detection, abnormal hidden data detection is performed on the weight parameters and bias parameters corresponding to the model to be detected in the model parameter set to ensure that the model parameter set does not contain any form of sensitive data, and then, when the second detection result indicates that the model parameter set does not contain abnormal hidden data, a safe operation detection is performed on the model to be detected in an isolated environment to obtain an operation detection result, so as to realize multi-level detection of sensitive data and ensure the comprehensiveness and depth of the detection, thereby improving the detection efficiency and detection accuracy of sensitive data detection to ensure the security of model operation. In addition, in the embodiment of the present application, the corresponding detection sub-module can be loaded from the preset detection sub-module set according to user demand information, so that the detection method can be adjusted according to different security requirements and data protection standards, effectively improving the flexibility and configurability of the detection method. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A flowchart of a method for detecting model-sensitive data provided in one embodiment of the present application;

[0020] Figure 2 A schematic diagram of obtaining a model parameter set in a model sensitive data detection method provided in an embodiment of the present application;

[0021] Figure 3 A flowchart of a method for performing model parsing processing in a model-sensitive data detection method provided in an embodiment of the present application;

[0022] Figure 4 A flowchart of a method for determining a framework type in a model-sensitive data detection method provided in an embodiment of the present application;

[0023] Figure 5 A flowchart of a method for performing decompression processing in a model-sensitive data detection method provided in an embodiment of the present application;

[0024] Figure 6 A flowchart of a method for visualizing model-sensitive data detection provided in an embodiment of the present application;

[0025] Figure 7 A flowchart of a method for detecting abnormal hidden data in a model-sensitive data detection method provided in one embodiment of the present application;

[0026] Figure 8 This is a flowchart of an example of a model-sensitive data detection method provided in one embodiment of the present application;

[0027] Figure 9 A schematic diagram of the system structure corresponding to the model-sensitive data detection method provided in one embodiment of the present application;

[0028] Figure 10 A schematic diagram of the structure of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0030] In some embodiments, although the system diagrams illustrate functional module divisions and the flowcharts illustrate logical sequences, in some cases, the steps shown or described may be performed in a sequence that differs from the module divisions in the system or the sequence in the flowcharts. The terms "first," "second," and so on in the specification, claims, and drawings are used to distinguish similar items and are not necessarily used to describe a specific sequence or precedence.

[0031] In addition, unless otherwise clearly specified and limited, the term "connection / connection" should be understood in a broad sense. For example, it can be a fixed connection or a movable connection, a detachable connection or a non-detachable connection, or an integral connection; it can be a mechanical connection, an electrical connection, or mutual communication; it can be a direct connection or an indirect connection through an intermediate medium.

[0032] In the description of the embodiments of the present application, the reference terms "one embodiment / implementation", "another embodiment / implementation" or "certain embodiments / implementations", "in the above-mentioned embodiments / implementations" and the like mean that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least two embodiments or implementations disclosed in the present application. In the disclosure of this application, the schematic representations of the above terms are not necessarily the same embodiments or implementations. It should be noted that although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in an order different from that in the flowchart.

[0033] Existing machine learning models may involve user sensitive data, such as user privacy information and commercial confidential information, when processing and storing data, which may lead to the risk of leakage or abuse of these data, which may cause serious damage to individuals or organizations. Therefore, it is currently necessary to detect model sensitive data to determine the security of the model; however, existing security detection methods generally require formal verification of parameters such as model accuracy based on specific verification tools and mathematical models to confirm whether sensitive data exists. The complex detection process leads to low detection efficiency. In addition, since current neural network models have strong robustness, the model will not suffer a large loss of accuracy due to the inclusion of privacy data, resulting in poor comprehensiveness and depth of existing security detection methods, and prone to false positives or omissions, which affects detection accuracy and cannot ensure the security of model operation.

[0034] Based on this, an embodiment of the present application provides a model sensitive data detection method, which can quickly and effectively perform multi-level sensitive data detection on the model files of the model to be detected to ensure the comprehensiveness and depth of the detection, thereby improving the detection efficiency and detection accuracy of sensitive data to ensure the security of model operation.

[0035] The present application scheme is described below with reference to the accompanying drawings.

[0036] refer to Figure 1 , Figure 1 A method flow chart of a model-sensitive data detection method provided in an embodiment of the present application; a first aspect of an embodiment of the present application provides a model-sensitive data detection method, comprising at least the following steps:

[0037] Step S110: Load the model file of the model to be tested and read the file data stream in the model file; perform model parsing on the file data stream to determine the framework type of the model file;

[0038] Step S120, obtaining model graph structure data corresponding to the model file according to the framework type and the file data stream, and obtaining a model parameter set corresponding to the model file from the model graph structure data;

[0039] Step S130: performing sensitive data detection on the model parameters in the model parameter set based on a preset sensitive word library to obtain a first detection result;

[0040] Step S140: When the first detection result indicates that the model parameter set does not contain sensitive data, abnormal hidden data detection is performed on the weight parameters and bias parameters corresponding to the model to be detected in the model parameter set to obtain a second detection result;

[0041] Step S150: When the second detection result indicates that the model parameter set does not contain abnormal hidden data, a safe operation detection is performed on the model to be detected in an isolated environment to obtain an operation detection result.

[0042] Among them, the model file can be a file saved after the machine learning model is trained, which can contain the model's structural information, weight parameters, and other necessary metadata; the file data stream can be a continuous sequence of bytes read from the model file, which can contain various information about the model, such as structure, parameters, etc. The model parsing process can be a series of processing actions, the purpose of which is to parse the content of the model file and understand the model's structure and parameters. This process may include reading the model file header information, parsing the model architecture, extracting parameters, etc.; the model framework type can be a software framework or platform type used to build and train machine learning models, such as TensorFlow, PyTorch, etc. Each framework has a different file format and data organization method; the model graph structure data can be a representation of the internal structure of the model, which can be composed of multiple nodes, each node representing an operation in the model (such as convolution, fully connected layer, etc.), and the lines between the nodes represent the direction of data flow.

[0043] It is understandable that the model parameter set can be a set of all parameters that constitute the machine learning model, including but not limited to weight parameters, bias parameters, etc. The sensitive vocabulary is a database containing a series of predefined sensitive words or patterns, which is used to detect whether the text or data contains sensitive information, such as personal identity information, telephone numbers, email addresses, etc.; the first detection result: refers to the result of whether sensitive information is contained after detecting the model parameter set through the sensitive vocabulary. The weight parameter and bias parameter are parameters that constitute the neural network model. The weight parameter determines the degree of influence of the input signal on the output, and the bias parameter is the threshold of the neuron activation function.

[0044] It is worth noting that abnormal hidden data detection is used to detect whether the model parameters contain abnormal, potential privacy data or other illegal information. For example, by checking whether the least significant bits of the model parameters contain regular patterns, or by analyzing the model behavior to find abnormal patterns. The second detection result refers to the result of whether the model parameters contain abnormal hidden data after performing abnormal hidden data detection on the model parameters.

[0045] Furthermore, the isolated environment can be an execution environment isolated from the outside world, such as a sandbox, in which unverified code can be safely run without affecting external systems. Running the model in such an environment can prevent potential malicious code from causing damage to the real system. The model to be tested can then be run in the isolated environment for safe operation detection and its behavior can be monitored to assess whether the model contains potential security risks.

[0046] In some embodiments, the present application loads the model file of the model to be detected, reads the file data stream in the model file, and performs model parsing processing on the file data stream to determine the framework type of the model file, so as to quickly and effectively obtain the model graph structure data corresponding to the model file according to the framework type and the file data stream, and obtain the model parameter set corresponding to the model file from the model graph structure data; further, a preliminary sensitive data inclusion detection is performed on the model parameter set through a preset sensitive word library to ensure that the model parameter set does not directly contain sensitive data, and on the basis of the preliminary detection, the weight parameters and bias parameters corresponding to the model to be detected in the model parameter set are detected for abnormal hidden data to ensure that the model parameter set does not contain any form of sensitive data, and then, when the second detection result indicates that the model parameter set does not contain abnormal hidden data, a safe operation detection is performed on the model to be detected in an isolated environment to obtain an operation detection result, so as to realize multi-level detection of sensitive data and ensure the comprehensiveness and depth of detection, thereby improving the detection efficiency and detection accuracy of sensitive data detection to ensure the security of model operation. In addition, in the embodiment of the present application, the corresponding detection sub-module can be loaded from the preset detection sub-module set according to user demand information, so that the detection method can be adjusted according to different security requirements and data protection standards, effectively improving the flexibility and configurability of the detection method.

[0047] In some embodiments, when the first detection result indicates that the model parameter set contains sensitive data, it means that the audit system corresponding to the method of the present application has detected personal information or other confidential information that should not exist in the model parameters. At this time, the system will record this abnormal situation and generate an abnormal record file, indicating that sensitive information exists in the model. Corresponding measures can be taken subsequently to prevent the release or use of the model.

[0048] In some embodiments, when the second detection result indicates that the model parameter set contains abnormal hidden data, it indicates that the audit system corresponding to the method of the present application has detected that hidden sensitive data is entrained or embedded in the model parameters. For example, the data may be embedded in the least significant bit of the model parameter, or exist in the hidden layer neurons in an encrypted form. This will also be recorded and used as part of the abnormality record, that is, the statistical record of all abnormal analysis results of the malicious entrainment detection model.

[0049] In some embodiments, when the second detection result indicates that the model parameter set does not contain abnormal hidden data, the model to be tested is tested for safe operation in an isolated environment to obtain an operation test result; it is understandable that the present application provides an automated sandbox creation and destruction process by providing a temporary and completely isolated environment, so that users can safely run the tested model, while monitoring the resource usage and network activity in the sandbox in real time, and using load balancing to reasonably allocate CPU / NPU, memory, disk I / O and other resource limits to prevent resource abuse. All operations and events in the sandbox are recorded to ensure that the operations are traceable, and logs are regularly reviewed to check for abnormal behavior, further eliminating the occurrence of malicious smuggling.

[0050] refer to Figure 2 , Figure 2 A schematic diagram of obtaining a model parameter set in a model sensitive data detection method provided in an embodiment of the present application; in some embodiments, the model file of the model to be detected is loaded, and the file data stream in the model file is read; the file data stream is subjected to model parsing processing to determine the framework type of the model file. The process of obtaining the model graph structure data corresponding to the model file according to the framework type and the file data stream, and obtaining the model parameter set corresponding to the model file from the model graph structure data may include: running a model parsing and visualization module, which is responsible for loading the model and obtaining the string data of the model. By using a model visualization warehouse, such as the Python open source warehouse Netron, to assist in model visualization and obtain the string data of the model, the model file is read according to the framework type of the model file, a model graph structure is generated, and the model parameter set is output as the data stream basis for privacy-sensitive data review.

[0051] Among them, the function of the model parsing and visualization module is to load the specified model file and read the string data therein. By using specific tools or libraries (such as Netron), it obtains the string data of the model, parses the model file to determine its framework type, and generates the graph structure data of the model; the model visualization warehouse can help developers or researchers view and understand the internal structure of complex models, and then create a graphical representation to show the architecture of the model, including the connection relationship between each layer.

[0052] Further, Figure 2 The corresponding model parameter set acquisition process may include: when the user initiates a request, the service will start, read the data stream and file name of the model file, and create a model factory as an intermediate link for data stream and file name processing. The model factory can be understood as a processing center, which receives instructions from the model parsing and visualization service and is responsible for subsequent data processing tasks; the model factory will store a comparison table of the model framework and its file extension, so as to select the correct parsing tool (model renderer) according to the file extension. The model renderer is used to convert the model file data into an understandable form, such as a model image object and a node object; further, by including a series of regular tables Dashi's exception regularization library detects whether the file data stream meets the expected format requirements. If the file content does not meet the expected format, the system will mark the file as illegal. If the file is in a compressed format, it needs to be decompressed before anomaly detection. Furthermore, by parsing the model data layer by layer, the model's graph structure is constructed to display the model's hierarchical relationship. Each component of the model, such as layers and nodes, contains the model parameters of that part, resulting in a model parameter set: a set of all parameters in the model. These parameters will be used for subsequent sensitive data audits, and tools such as Netron are used to convert the model graph structure into a graphical representation to facilitate users' understanding of the model structure and parameter configuration.

[0053] It can be understood that through the above steps, the present application can effectively extract the model graph structure data from the model file to be tested, and then obtain the model parameter set, preparing for the subsequent privacy-sensitive data review.

[0054] refer to Figure 3 , Figure 3 A flowchart of a method for performing model parsing in a model-sensitive data detection method provided in one embodiment of the present application; in some embodiments, performing model parsing on a file data stream to determine the framework type of the model file includes at least the following steps:

[0055] Step S310, obtaining an extension name dataset, where the extension name dataset includes file extension names corresponding to multiple framework types;

[0056] Step S320 , parsing the file extensions of the plurality of file data from the file data stream, and matching the framework type corresponding to the model file from the extension dataset according to the file extensions of the plurality of file data.

[0057] The extension dataset contains a collection of file extensions corresponding to various machine learning frameworks. For example, a TensorFlow model file might be in .pb or .ckpt format, while a PyTorch model might be in .pt or .pth format. This dataset helps identify the specific framework type to which the model file belongs.

[0058] It can be understood that the file data stream is a series of bytes read from the model file, representing the content of the model file. The file extension is the string following the file name, which is used to indicate the type of file. In the case of a model file, the extension can help identify which framework the file was created by. Then, based on the parsed file extension, the corresponding entry can be found in the extension dataset to determine which framework the model file belongs to.

[0059] refer to Figure 4 , Figure 4 A flowchart of a method for determining a framework type in a model-sensitive data detection method provided in an embodiment of the present application; in some embodiments, obtaining model graph structure data corresponding to a model file based on the framework type and the file data stream includes at least the following steps:

[0060] Step S410: performing file screening processing on the file data stream based on a preset abnormal regular library to obtain a plurality of file data in the file data stream that conform to the expected file format;

[0061] Step S420, obtaining multiple target file types according to multiple file extensions corresponding to the multiple file data, and selecting corresponding model renderers for the multiple target file types according to the framework type;

[0062] Step S430: decoding the multiple file data based on the multiple model renderers to obtain multiple image object data and node object data;

[0063] Step S440 , obtaining model graph structure data corresponding to the model file according to the plurality of image object data and node object data.

[0064] Among them, the preset exception regular library can be a collection of a series of regular expressions. These regular expressions are used to match legal file formats. Any file data that does not conform to these regular expressions will be regarded as abnormal or erroneous file data. Then, the file data stream can be filtered by applying regular expressions to ensure that only file data that conforms to the expected format is retained.

[0065] In some embodiments, the target file type can be a set of file types that are associated with the framework type of the model file. The target file type is determined by identifying the file extension. Then, a suitable tool or program (i.e., a model renderer) is selected for each target file type to correctly parse and display the file data of these types.

[0066] In some embodiments, a model renderer is a tool or program used to process a specific type of model file, responsible for converting file data into an understandable form, such as image object data and node object data, by using the model renderer to convert the file data into a format that is easy to understand and process.

[0067] In some embodiments, the image object data may be visual image data representing the model structure, and the node object data may be detailed information about each component or operation in the model graph, such as the function, input and output of each layer, etc. The model graph structure data is a representation of the internal structure of the model, which describes the hierarchical relationship of the model, the connection between nodes, and the way data flows.

[0068] In some embodiments, sensitive data detection is performed on the model parameters in the model parameter set based on a preset sensitive word library to obtain a first detection result, which may include performing sensitive data detection through a sensitive data review module, and the sensitive data review module receives a model parameter set extracted from a model file. These parameters may include key numerical information in the model such as weight matrices and bias terms, and special data detection tools are used to process these model parameter data. These tools generally include but are not limited to text processing functions, which can identify and mark potential sensitive information; further, the extracted model string information is preprocessed to remove redundant symbols that may interfere with sensitive information detection, including removing spaces, brackets "[]", commas ",", slashes "\" and carriage returns, to ensure that during the detection process, sensitive information will not be missed due to the presence of these symbols.

[0069] Furthermore, after preprocessing is complete, a pre-set sensitive word library can be loaded. The sensitive word library is dynamically configurable and can be updated at any time to adapt to new privacy protection requirements. The sensitive word library can contain a series of predefined keywords or patterns used to identify personal information such as names, phone numbers, and ID numbers. The preprocessed model parameter data can be searched through the sensitive word library to detect whether it contains sensitive information. This step involves scanning each element in the model parameter to find information that matches the patterns defined in the sensitive word library.

[0070] Furthermore, after completing the above-mentioned detection, a first detection result can be obtained, indicating whether the model parameter set contains sensitive information. If sensitive information is detected, the system will record this information and may generate an exception record file for subsequent processing or auditing. If no sensitive information is detected, the model parameter set is considered safe and can continue to be reviewed or used subsequently.

[0071] It can be understood that through the above steps, the sensitive data review module can effectively identify sensitive information that may exist in the model parameters and make corresponding processing decisions accordingly, thereby ensuring the security and privacy of the model.

[0072] refer to Figure 5 , Figure 5 A flowchart of a method for performing decompression processing in a model-sensitive data detection method provided in an embodiment of the present application; in some embodiments, before performing file screening processing on a file data stream based on a preset abnormal regularization library, at least the following steps are further included:

[0073] Step S510, performing compressed data detection on the file data stream;

[0074] Step S520: When the file data stream includes compressed data, decompress the compressed data to obtain file data corresponding to the compressed data.

[0075] In some embodiments, compressed data detection is used to search for compressed data in a file data stream. Compressed data may include files in common compression formats such as ZIP and TAR. Detecting compressed data is to ensure that subsequent processing is of decompressed data for more accurate model parsing. If compressed data is detected in a file data stream, it is necessary to use an appropriate decompression tool to decompress this data to obtain the original uncompressed file data. The purpose of this is to ensure that all subsequently processed data exists in an uncompressed format for further analysis and processing.

[0076] refer to Figure 6 , Figure 6 A flowchart of a method for visualizing model-sensitive data detection provided in an embodiment of the present application; in some embodiments, after obtaining the model graph structure data corresponding to the model file according to the framework type and the file data stream, at least the following steps are further included:

[0077] Step S610, obtaining a model visualization tool corresponding to the framework type from a preset algorithm library;

[0078] Step S620: Generate a visual model view corresponding to the model graph structure data based on the model visualization tool.

[0079] In some embodiments, the preset algorithm library is a pre-configured algorithm collection, which may include model visualization tools for different framework types. The framework type here refers to the development framework used by the machine learning model, such as TensorFlow, PyTorch, etc. The model visualization tool is used to convert the model structure into a visual form to facilitate users to understand the structure and parameters of the model. Commonly used model visualization tools include Netron, etc. The model visualization tool can then be used to convert the model graph structure data into a graphical interface, so that users can intuitively see the structure of the model and the relationship between each component.

[0080] refer to Figure 7 , Figure 7 A flowchart of a method for detecting abnormal hidden data in a model-sensitive data detection method provided in an embodiment of the present application is provided. In some embodiments, abnormal hidden data detection is performed on weight parameters and bias parameters corresponding to the model to be detected in the model parameter set to obtain a second detection result, which further includes at least the following steps:

[0081] Step S710: performing model anomaly analysis based on the weight parameters and bias parameters corresponding to the model to be detected in the model parameter set to obtain a model anomaly index corresponding to the model to be detected. The model anomaly index includes at least one of the following: uniform distribution coefficient, output difference coefficient, similarity coefficient, and injection detection result;

[0082] Step S720: Determine whether the model parameter set contains abnormal hidden data according to the model abnormality index to obtain a second detection result.

[0083] In some embodiments, the weight parameters and bias parameters of the model are statistically analyzed to check whether there are any abnormalities. For example, the least significant bit (LSB) can be analyzed to detect whether there is a non-random pattern, or the mean and variance of the parameters can be calculated to detect whether there is a deviation from the normal distribution. Model abnormality indicators include but are not limited to the uniform distribution coefficient (used to detect whether the LSB is uniformly distributed), the output difference coefficient (used to detect whether the model's response to the adversarial sample is abnormal), the similarity coefficient (used to detect the similarity between the model and the normal model) and the injection detection result (used to detect whether there is malicious code inserted in the model). Based on the analysis of the model abnormality indicators, it is determined whether there is abnormal hidden data in the model parameter set. If an abnormality is found, it indicates that the model may have hidden data and needs to be further processed or excluded. If no abnormality is found, it indicates that the model parameter set is relatively normal and can continue to be reviewed or used subsequently.

[0084] In some embodiments, the process of obtaining the uniform distribution coefficient is as follows: obtaining the least significant bit distribution information of the weight parameters and bias parameters corresponding to the model to be detected in the model parameter set; calculating the uniform distribution coefficient corresponding to the model parameter set based on the least significant bit distribution information, wherein the model parameter set refers to all parameters extracted from the model, including weight parameters and bias parameters, which constitute the basic components of the model; the least significant bit (Least Significant Bit, LSB) refers to the rightmost bit of the binary number in digital representation, which is the part with the smallest value. When detecting entrained data, the LSB can be used to hide information because it has the least impact on the value and is not easy to attract attention; the least significant bit distribution information is the least significant bit taken for each model parameter for subsequent statistical analysis.

[0085] It can be understood that the uniform distribution coefficient corresponding to the model parameter set is calculated based on the least significant bit distribution information, the least significant bit (LSB) values ​​of all model parameters are recorded, the frequency of occurrence of 0 and 1 is counted, and the actual number of occurrences of each LSB value (0 or 1) is recorded. In the absence of LSB steganography, it is assumed that the number of occurrences of 0 and 1 should be equal, that is, E0=E1=N / 2, where N is the total number of LSBs. The chi-square test is used to detect the uniformity of the LSB distribution. According to the chi-square value χ2 and the set significance level α, the critical value χ2_(α) in the critical value table is found. If the calculated chi-square value is greater than the critical value χ2_(α), the null hypothesis is rejected, and it is believed that the LSB distribution of the model parameters is not random, and there may be entrained data behavior. Based on the statistical analysis, the uniform distribution coefficient can be calculated based on the chi-square value. This coefficient reflects whether the LSB distribution is uniform. If the LSB distribution is close to uniform, the uniform distribution coefficient is close to 1; if it deviates from uniformity, the coefficient is far away from 1.

[0086] Through the above steps, it is possible to effectively detect whether there is any behavior of data entrainment by modifying the least significant bit in the model parameters. The uniform distribution coefficient, as a quantitative indicator to measure whether the LSB distribution is uniform, can help this application determine whether the model may be used to hide information. If the test results show that the uniform distribution coefficient deviates from the ideal value, it indicates that the model may be at risk of data entrainment.

[0087] In some embodiments, specifically, a chi-square test is used to detect the uniformity of the LSB distribution. (where Ok is the actual frequency and Ek is the expected frequency). Without LSB steganography: Record the actual frequency and expected frequency to calculate the chi-square statistic By statistically analyzing the distribution of model parameters, finding outliers and calculating the mean of model parameters and standard deviation Then use the normality test to check whether the parameters conform to the normal distribution. Detect whether there are any deviations from the normal distribution.

[0088] In some embodiments, the process of obtaining the output difference coefficient is as follows: generating multiple test samples according to the weight parameters and bias parameters corresponding to the model to be tested in the model parameter set, and generating multiple adversarial samples based on the multiple test samples and the preset loss function; inputting the multiple test samples and the multiple adversarial samples into the model to be tested respectively to calculate the output difference coefficients of the test samples and the adversarial samples; wherein the test samples are a series of representative input data generated, which should cover various scenarios that the model may encounter, and the selection of test samples is representative to ensure that the behavior of the model can be fully detected; the weight parameters and bias parameters are extracted from the model file and are used to define the calculation logic within the model. The test samples can be generated according to the characteristics and application scenarios of the model to ensure that the main functions of the model are covered.

[0089] It can be understood that adversarial examples are generated by adding small perturbations to the original input data. These perturbations are added along the gradient of the loss function, aiming to cause the model to make incorrect predictions. Adversarial examples can be generated using adversarial sample techniques such as FGSM or PGD.

[0090] A pre-set loss function can make the model's output on adversarial examples as close as possible to certain sensitive data or features. The choice of loss function should be relevant to the model's task and reflect the model's sensitivity to input changes. For example, by adding small perturbations along the gradient of the loss function, a series of adversarial examples X′={X1′,X2′,…,Xn′} can be generated. (X is the original input, X′ is the adversarial sample, ∈ is the perturbation size, J(θ,x,y) is the loss function, is the gradient of the loss function with respect to the input).

[0091] Furthermore, multiple test samples and multiple adversarial samples are fed into the model to be tested to calculate the output difference coefficient between the test samples and the adversarial samples. The generated test samples and adversarial samples are then fed into the model to be tested, and the model is asked to make predictions. The model's output for the test samples and adversarial samples is recorded, and the difference between the two is calculated. The output difference can be the absolute difference in the prediction results or a distance metric. The mean and variance of the output differences between all test samples and adversarial samples are calculated. The output difference coefficient can be defined using these statistics. For example, the mean can represent the average difference, and the variance can represent the degree of fluctuation in the difference. If the output difference exceeds a preset threshold, the model is considered abnormal and may contain sensitive data or features.

[0092] In some embodiments, the output difference ΔOutput between the original input and the adversarial example is calculated i =f(X ′ i )-f(X i ), and calculate the mean of all output differences and variance Then conduct statistical analysis to check whether there are any abnormalities Set the threshold; To detect the entrainment behavior of encrypted embedded hidden neurons, gradient analysis of hidden layer neurons can be performed to check whether there is an abnormal gradient distribution or pattern. Assume that L is the loss function, h i Is the activation value vector of the hidden layer neurons, and calculates the gradient of the loss function with respect to the activation value of the hidden layer neurons. The mean gradient of the hidden layer neuron j is: The gradient variance of hidden layer neuron j By calculating the mean and variance of the gradients of the hidden layer neurons and performing statistical analysis on them, we can detect whether there are abnormal gradient patterns.

[0093] Through the above steps, we can effectively detect the behavior of the model when facing adversarial samples, especially whether there are abnormal output patterns. The output difference coefficient, as a quantitative indicator, can help this application evaluate the stability of the model output and whether there is a potential risk of data entrainment.

[0094] In some embodiments, the process of obtaining the similarity coefficient is as follows: Federated learning training is performed based on the weight parameters and bias parameters corresponding to the model to be tested in the model parameter set to obtain a sample model corresponding to the model to be tested; the similarity coefficient between the model to be tested and the sample model is calculated, wherein federated learning training refers to a federated learning environment in which multiple participants (clients) jointly train a shared model without directly exchanging data. Each participant uses its own local data for model training and only shares model updates (such as weight updates); the sample model refers to a model obtained by training in accordance with the federated learning protocol without any private data, which can be used as a baseline model for comparative analysis with the model to be tested. The sample model should be generated under the same training protocol and should not contain any private data or abnormal hidden data; the model to be tested refers to the model that needs to be tested. The model may be trained by other means or obtained from the federated learning process, and it is necessary to verify whether it contains private data.

[0095] It can be understood that the weight similarity between the detection model and the sample model is determined by calculating the cosine similarity of the model weights, the model output is verified to see if there is a significant change by comparing the output KL divergence on the same input, and the gradient similarity of the model is verified by comparing the gradient distance on the same input. According to the results of the above similarity calculations, a similarity coefficient can be obtained to determine the degree of similarity between the model to be detected and the sample model. If the similarity coefficient is high, it indicates that the difference between the two models is small and there may be no data entrainment behavior; if the similarity coefficient is low, it indicates that there is a large difference between the two models and there may be data entrainment behavior, thereby effectively comparing the similarity between the model to be detected and the sample model, thereby determining whether the model to be detected has the risk of data entrainment. The similarity coefficient, as a comprehensive indicator, can help this application evaluate the similarity of the model from multiple angles and ensure the safety and reliability of the model.

[0096] Specifically, this application can calculate the cosine similarity of the model weights To determine the weight similarity between the detection model and the sample model, where ||wi|| and ||wj|| are the norms of the weight vectors of the detection model and the sample model respectively; by comparing the output KL divergence on the same input Verify whether the model output has significant changes, where y1 and y2 are the probability output vectors of the two models;

[0097] By comparing the gradient distance on the same input, Verify the gradient similarity of the model, where are the gradients of the two models on the input sample data Xi, and n is the dimension of the input.

[0098] In some embodiments, the process of obtaining the injection detection result is as follows: obtaining binary data corresponding to the weight parameters and bias parameters of the model to be detected in the model parameter set; performing injection detection on the operation code field in the binary data to obtain the injection detection result, wherein the model parameter set refers to all parameters extracted from the model file, including weight parameters (weights) and bias parameters (biases), which constitute the basic components of the model.

[0099] In some embodiments, the model file is typically stored in a binary format, which means that the data in the file exists in the form of bits and bytes. Obtaining the binary data corresponding to the model parameters means reading the model file and extracting the binary representation of the weight parameters and bias parameters from it. In computer science, an opcode refers to a part of a machine language instruction set that instructs the processor to perform a specific operation. In the binary representation of the model file, the opcode is used to indicate how to process the data stream. Therefore, this application checks the opcode field in the binary data to ensure that these opcodes conform to the expected behavior pattern and do not contain any operations that may lead to malicious behavior. The specific detection steps include:

[0100] Traverse the PROTO opcodes of the parsed model file. If duplicate opcodes are found, this may mean that the model file has been tampered with. Ensure that the version numbers of all opcodes are consistent. Otherwise, it may indicate that there is a problem with the model file. Check whether the position of the PROTO opcode meets the requirements of its protocol version. The position and order of the opcodes should comply with the provisions of the protocol version used by the model file. Any non-compliance may indicate that the file has been tampered with to obtain injection detection results.

[0101] Furthermore, the AST nodes of the parsed model file are traversed. AST nodes: Abstract Syntax Tree (Abstract Syntax Tree) is a tree representation of the source code structure. The purpose of traversing the AST nodes is to check whether the code in the model file calls certain dangerous functions, such as eva l, exec, compile, etc. These functions can execute arbitrary code and therefore should not appear in the model file. If calls to these functions are detected, then the model file can be considered to be almost malicious, so as to obtain injection detection results.

[0102] Furthermore, the interpreter is used to traverse the opcodes and check for unused variable assignments: the interpreter is used to traverse the opcodes in the model file to check whether there are unused variable assignments. If unused variable assignments are found, then these variables may be part of malicious code, making the model file suspicious, so as to obtain injection detection results.

[0103] Based on the results of the above detection steps, a conclusion can be drawn as to whether there is malicious injection behavior in the model file. If any unexpected behavior is found during the detection process, such as repeated opcodes, calls to dangerous functions, or unused variable assignments, the model file will be considered suspicious and may contain malicious injection behavior. This can effectively detect whether there is malicious injection behavior in the model file, thereby ensuring the security and reliability of the model. The injection detection results serve as the final judgment basis to help this application identify whether the model file has been tampered with or contains potential malicious code.

[0104] refer to Figure 8 and Figure 9 , Figure 8 This is an example flow chart of a model-sensitive data detection method provided in one embodiment of the present application. Figure 9 Schematic diagram of the system structure corresponding to the model sensitive data detection method provided in one embodiment of the present application; in some embodiments, the present application can load the model file to be detected through the model parsing and visualization module, and read the data stream in the model file, parse the data stream of the model file, determine the framework type used by the model, and generate the graph structure data of the model according to the framework type, which includes the various components of the model (such as layers, nodes, etc.) and their connection relationships. Use tools to visualize the model graph structure to facilitate users to view the model structure and parameter characteristics; through the sensitive data review module, extract the model parameter set from the model graph structure data, use a dynamically configurable sensitive word library, perform sensitive data detection on the model parameters, identify whether there is sensitive information such as ID card, email address, telephone number, etc., pre-process the extracted model string information, remove redundant symbols, facilitate sensitive information detection, overload the dynamic sensitive word matching library, check whether there is sensitive data, and record the detection results; through the malicious entrainment detection module: Perform statistical analysis on the least significant bits (LSBs) of the model parameters to detect whether there are non-random patterns or distribution anomalies to determine whether there is entrained data. Provide a series of representative inputs and analyze whether there are abnormal patterns in the model output. For example, use adversarial sample technology to detect whether the model produces abnormal responses to specific inputs, compare with the sample model without privacy data entrainment trained in accordance with the federated learning protocol, calculate the similarity of weights, outputs and gradients, detect whether there is significant deviation in the model, perform binary serialization and deserialization on the model file, traverse the opcodes and AST nodes, and detect whether there are dangerous function calls or unused variable assignments; provide a temporary and completely isolated environment through the sandbox environment running module, allowing users to safely run the detected model, monitor resource usage and network activity in the sandbox in real time, ensure reasonable resource allocation, and prevent resource abuse, record all operations and events in the sandbox to ensure traceability of operations, and regularly audit logs to check for abnormal behavior.

[0105] In some embodiments, the present application can implement a multi-level audit mechanism. By combining multiple protection mechanisms such as model analysis, sensitive word detection, malicious inclusion detection, and sandbox environment operation, the solution of the present invention comprehensively improves the security and privacy protection capabilities of the machine learning model. It also intelligently extracts model parameters through model analysis and visualization modules, and uses a dynamically configurable sensitive word library for sensitive data detection, which can quickly and accurately discover sensitive information in model parameters. In addition, the solution of the present invention has high flexibility and configurability, and users can dynamically adjust detection strategies according to different security requirements and data protection standards, thereby improving practicality and adaptability.

[0106] A second aspect of an embodiment of the present application provides a model sensitive data detection device, comprising: a model parsing module, configured to load a model file of a model to be detected and read a file data stream in the model file; perform model parsing processing on the file data stream to determine a framework type of the model file; obtain model graph structure data corresponding to the model file according to the framework type and the file data stream, and obtain a model parameter set corresponding to the model file from the model graph structure data; a sensitive data detection module, configured to perform sensitive data inclusion detection on model parameters in a model parameter set based on a preset sensitive vocabulary library to obtain a first detection result; a hidden data detection module, configured to perform abnormal hidden data detection on weight parameters and bias parameters corresponding to the model to be detected in the model parameter set when the first detection result indicates that the model parameter set does not contain sensitive data to obtain a second detection result; and a model operation module, configured to perform a safe operation detection on the model to be detected in an isolated environment to obtain an operation detection result when the second detection result indicates that the model parameter set does not contain abnormal hidden data.

[0107] refer to Figure 10 , Figure 10 This is a structural diagram of an electronic device provided in an embodiment of the present application. The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the model-sensitive data detection method of any of the above embodiments is implemented. For example, the above-described Figure 1 Steps S110 to S150 of the method are performed as described above. Figure 3 Steps S310 to S320 of the method, Figure 4 Steps S410 to S440 of the method, Figure 5 Steps S510 to S520 of the method, Figure 6 Steps S610 to S620 of the method, Figure 7 Method steps S710 to S720.

[0108] The electronic device 1000 of the embodiment of the present application includes one or more processors 1010 and a memory 1020. Figure 10 In the figure, a processor 1010 and a memory 1020 are taken as an example.

[0109] The processor 1010 and the memory 1020 may be connected via a bus or other means. Figure 10 The bus connection is taken as an example.

[0110] The memory 1020 is a non-transitory computer-readable storage medium that can be used to store non-transitory software programs and non-transitory computer executable programs. In addition, the memory 1020 may include a high-speed random access memory and may also include a non-transitory memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 1020 may optionally include a memory 1020 remotely located relative to the processor 1010. These remote memories can be connected to the electronic device 1000 via a network. At the same time, examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0111] In some embodiments, when the processor executes the computer program, the model-sensitive data detection method of any one of the above embodiments is performed at a preset interval.

[0112] Those skilled in the art will understand that Figure 10 The device structure shown in the figure does not constitute a limitation on the electronic device 1000, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0113] exist Figure 10 In the electronic device 1000 shown, the processor 1010 can be used to call the model-sensitive data detection method stored in the memory 1020, thereby implementing the model-sensitive data detection method.

[0114] Based on the hardware structure of the above-mentioned electronic device 1000, various embodiments of the model-sensitive data detection device of the present application are proposed. At the same time, the non-transient software programs and instructions required to implement the model-sensitive data detection method of the above-mentioned embodiments are stored in the memory. When executed by the processor, the model-sensitive data detection method of the above-mentioned embodiments is executed.

[0115] The embodiment of the present application also provides a computer-readable storage medium, which stores computer-executable instructions, which are used to execute the above-mentioned model-sensitive data detection method, and can enable the above-mentioned one or more processors to execute the model-sensitive data detection method of any of the above-mentioned embodiments, for example, to execute the above-mentioned Figure 1Steps S110 to S150 of the method are performed as described above. Figure 3 Steps S310 to S320 of the method, Figure 4 Steps S410 to S440 of the method, Figure 5 Steps S510 to S520 of the method, Figure 6 Steps S610 to S620 of the method, Figure 7 Method steps S710 to S720.

[0116] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network nodes. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0117] Those skilled in the art will appreciate that all or some of the steps and systems in the disclosed method above can be implemented as software, firmware, hardware, and appropriate combinations thereof. Some physical components or all physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, and the computer-readable medium can include a computer-readable storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer-readable storage medium is included in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data) and is volatile and non-volatile, removable, and non-removable. Computer-readable storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory, or other memory technology, CD to ROM, digital versatile disk (DVD), or other optical disk storage, magnetic cassette, magnetic tape, disk storage, or other magnetic storage device, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media may contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0118] The above is a specific description of the preferred implementation of the present application, but the present application is not limited to the above implementation mode. Technical personnel familiar with the field can also make various equivalent modifications or substitutions without violating the spirit of the present application. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present application.

Claims

1. A method for detecting model sensitive data, characterized in that: include: Load the model file of the model to be detected and read the file data stream in the model file; Performing model parsing processing on the file data stream to determine the framework type of the model file; Obtaining model graph structure data corresponding to the model file according to the framework type and the file data stream, and acquiring a model parameter set corresponding to the model file from the model graph structure data; Performing sensitive data detection on the model parameters in the model parameter set based on a preset sensitive word library to obtain a first detection result; When the first detection result indicates that the model parameter set does not contain sensitive data, performing abnormal hidden data detection on the weight parameters and bias parameters corresponding to the model to be detected in the model parameter set to obtain a second detection result; When the second detection result indicates that the model parameter set does not contain abnormal hidden data, a safe operation detection is performed on the model to be detected in an isolated environment to obtain an operation detection result.

2. The model sensitive data detection method according to claim 1, characterized in that: The performing model parsing processing on the file data stream to determine the framework type of the model file includes: Acquire an extension name dataset, where the extension name dataset includes file extension names corresponding to multiple framework types; The file extensions of a plurality of file data are parsed from the file data stream, and the framework type corresponding to the model file is matched from the extension dataset according to the file extensions of the plurality of file data.

3. The model sensitive data detection method according to claim 1, characterized in that: The obtaining of the model graph structure data corresponding to the model file according to the framework type and the file data stream includes: Performing file screening processing on the file data stream based on a preset abnormal regular library to obtain a plurality of file data in the file data stream that conform to an expected file format; Obtaining a plurality of target file types according to a plurality of file extensions corresponding to the plurality of file data, and selecting corresponding model renderers for the plurality of target file types according to the framework type; Decoding the plurality of file data based on the plurality of model renderers to obtain a plurality of image object data and node object data; The model graph structure data corresponding to the model file is obtained according to the plurality of image object data and the node object data.

4. The model sensitive data detection method according to claim 3, characterized in that: Before performing file screening processing on the file data stream based on the preset abnormal regular library, the method further includes: Performing compressed data detection on the file data stream; In the case that the file data stream includes compressed data, the compressed data is decompressed to obtain file data corresponding to the compressed data.

5. The model sensitive data detection method according to any one of claims 1 to 4, characterized in that: After obtaining the model graph structure data corresponding to the model file according to the framework type and the file data stream, the method further includes: Obtain a model visualization tool corresponding to the framework type from a preset algorithm library; A visual model view corresponding to the model graph structure data is generated based on the model visualization tool.

6. The model sensitive data detection method according to claim 1, characterized in that: The performing abnormal hidden data detection on the weight parameters and bias parameters corresponding to the to-be-detected model in the model parameter set to obtain a second detection result includes: Performing model anomaly analysis based on weight parameters and bias parameters corresponding to the model to be detected in the model parameter set to obtain a model anomaly indicator corresponding to the model to be detected, wherein the model anomaly indicator includes at least one of the following: a uniform distribution coefficient, an output difference coefficient, a similarity coefficient, and an injection detection result; Determine whether the model parameter set contains abnormal hidden data according to the model abnormality indicator to obtain a second detection result.

7. The model sensitive data detection method according to claim 6, characterized in that: The process of obtaining the uniform distribution coefficient is as follows: Obtaining least significant bit distribution information of weight parameters and bias parameters corresponding to the model to be detected in the model parameter set; A uniform distribution coefficient corresponding to the model parameter set is calculated according to the least significant bit distribution information.

8. The model sensitive data detection method according to claim 6, characterized in that: The process of obtaining the output difference coefficient is as follows: Generate multiple test samples according to the weight parameters and bias parameters corresponding to the model to be tested in the model parameter set, and generate multiple adversarial samples based on the multiple test samples and a preset loss function; The multiple test samples and the multiple adversarial samples are respectively input into the model to be detected to calculate the output difference coefficients of the test samples and the adversarial samples.

9. The model sensitive data detection method according to claim 6, characterized in that: The process of obtaining the similarity coefficient is as follows: Performing federated learning training according to the weight parameters and bias parameters corresponding to the model to be detected in the model parameter set to obtain a sample model corresponding to the model to be detected; Calculate the similarity coefficient between the model to be detected and the sample model.

10. The model sensitive data detection method according to claim 6, characterized in that: The process of obtaining the injection detection result is as follows: Obtaining binary data corresponding to weight parameters and bias parameters of the model to be detected in the model parameter set; An injection detection is performed on the operation code field in the binary data to obtain an injection detection result.

11. A model sensitive data detection device, characterized in that: include: A model parsing module is used to load the model file of the model to be detected and read the file data stream in the model file; Performing model parsing processing on the file data stream to determine the framework type of the model file; obtaining model graph structure data corresponding to the model file according to the framework type and the file data stream, and obtaining a model parameter set corresponding to the model file from the model graph structure data; a sensitive data detection module, configured to perform a sensitive data inclusion detection on the model parameters in the model parameter set based on a preset sensitive word library to obtain a first detection result; a hidden data detection module, configured to, when the first detection result indicates that the model parameter set does not contain sensitive data, perform abnormal hidden data detection on the weight parameters and bias parameters corresponding to the model to be detected in the model parameter set to obtain a second detection result; The model operation module is used to perform a safe operation test on the model to be tested in an isolated environment to obtain an operation test result when the second detection result indicates that the model parameter set does not contain abnormal hidden data.

12. An electronic device, characterized in that: include: at least one processor; at least one memory for storing at least one program; When at least one of the programs is executed by at least one of the processors, the model-sensitive data detection method according to any one of claims 1 to 10 is implemented.

13. A computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the model-sensitive data detection method according to any one of claims 1 to 10.

Images