Model sensitive data detection method and related equipment

Abstract

The invention provides a model sensitive data detection method which comprises the following steps: loading a model file of a to-be-detected model, determining a frame type of the model file, obtaining model graph structure data corresponding to the model file, and obtaining a model parameter set corresponding to the model file from the model graph structure data; performing sensitive data detection on the model parameters in the model parameter set to obtain a first detection result; when the first detection result represents that the model parameter set does not contain sensitive data, abnormal hidden data detection is carried out on a weight parameter and an offset parameter corresponding to the to-be-detected model in the model parameter set, and a second detection result is obtained; and when the second detection result represents that the model parameter set does not contain abnormal hidden data, performing safe operation detection on the to-be-detected model in the isolation environment to obtain an operation detection result, so as to realize multi-level sensitive data detection, ensure the comprehensiveness and depth of detection, and improve the detection efficiency and detection precision of sensitive data detection.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of data processing technology, and in particular to a model-sensitive data detection method and related equipment. Background Art

[0002] Existing machine learning models may involve user sensitive data, such as user privacy information, commercial confidential information, etc. when processing and storing data, which may lead to the risk of leakage or abuse of these data, and may cause serious damage to individuals or organizations. Therefore, it is necessary to detect model sensitive data to determine the security of the model.

[0003] However, existing security detection methods generally require formal verification of parameters such as model accuracy based on specific verification tools and mathematical models to confirm whether sensitive data exists. The complex detection process leads to low detection efficiency. In addition, since current neural network models have strong robustness, the model will not suffer a large loss of accuracy due to the inclusion of private data. As a result, existing security detection methods are not comprehensive and in-depth, and are prone to false positives or omissions, which affects detection accuracy and cannot ensure the security of model operation. Summary of the invention

[0004] The embodiment of the present application provides a model sensitive data detection method, which can quickly and effectively perform multi-level sensitive data detection on the model files of the model to be detected, so as to ensure the comprehensiveness and depth of the detection, thereby improving the detection efficiency and detection accuracy of sensitive data to ensure the security of model operation.

[0005] To achieve the above-mentioned purpose, the first aspect of an embodiment of the present application provides a model sensitive data detection method, comprising: loading a model file of a model to be detected, and reading a file data stream in the model file; performing model parsing processing on the file data stream to determine the framework type of the model file; obtaining model graph structure data corresponding to the model file according to the framework type and the file data stream, and obtaining a model parameter set corresponding to the model file from the model graph structure data; performing sensitive data detection on the model parameters in the model parameter set based on a preset sensitive word library to obtain a first detection result; when the first detection result indicates that the model parameter set does not contain sensitive data, performing abnormal hidden data detection on the weight parameters and bias parameters corresponding to the model to be detected in the model parameter set to obtain a second detection result; when the second detection result indicates that the model parameter set does not contain abnormal hidden data, performing a safe operation detection on the model to be detected in an isolated environment to obtain an operation detection result.

[0006] In some embodiments, the model parsing processing is performed on the file data stream to determine the framework type of the model file, including: obtaining an extension name data set, the extension name data set including file extensions corresponding to multiple framework types; parsing the file extensions of multiple file data from the file data stream, and matching the framework type corresponding to the model file from the extension name data set according to the file extensions of the multiple file data.

[0007] In some embodiments, the method of obtaining the model graph structure data corresponding to the model file according to the framework type and the file data stream includes: performing file screening processing on the file data stream based on a preset exception regular library to obtain multiple file data in the file data stream that conform to the expected file format; obtaining multiple target file types according to multiple file extensions corresponding to the multiple file data, and selecting corresponding model renderers for the multiple target file types according to the framework type; decoding processing on the multiple file data based on multiple model renderers to obtain multiple image object data and node object data; obtaining the model graph structure data corresponding to the model file according to the multiple image object data and the node object data.

[0008] In some embodiments, before the file data stream is subjected to file screening processing based on a preset exception regular library, it also includes: performing compressed data detection on the file data stream; when the file data stream contains compressed data, decompressing the compressed data to obtain file data corresponding to the compressed data.

[0009] In some embodiments, after obtaining the model graph structure data corresponding to the model file according to the framework type and the file data stream, it also includes: obtaining a model visualization tool corresponding to the framework type from a preset algorithm library; and generating a visualization model view corresponding to the model graph structure data based on the model visualization tool.

[0010] In some embodiments, the abnormal hidden data detection is performed on the weight parameters and bias parameters corresponding to the model to be detected in the model parameter set to obtain a second detection result, including: performing model anomaly analysis according to the weight parameters and bias parameters corresponding to the model to be detected in the model parameter set to obtain model anomaly indicators corresponding to the model to be detected, and the model anomaly indicators include at least one of the following: uniform distribution coefficient, output difference coefficient, similarity coefficient and injection detection result; determining whether the model parameter set contains abnormal hidden data according to the model anomaly indicators to obtain a second detection result.

[0011] In some embodiments, the process of obtaining the uniform distribution coefficient is as follows: obtaining the least significant bit distribution information of the weight parameters and bias parameters corresponding to the model to be detected in the model parameter set; and calculating the uniform distribution coefficient corresponding to the model parameter set based on the least significant bit distribution information.

[0012] In some embodiments, the process of obtaining the output difference coefficient is as follows: generating multiple test samples according to the weight parameters and bias parameters corresponding to the model to be detected in the model parameter set, and generating multiple adversarial samples based on the multiple test samples and a preset loss function; respectively inputting the multiple test samples and the multiple adversarial samples into the model to be detected to calculate the output difference coefficients of the test samples and the adversarial samples.

[0013] In some embodiments, the process of obtaining the similarity coefficient is as follows: performing federated learning training according to the weight parameters and bias parameters corresponding to the model to be detected in the model parameter set to obtain a sample model corresponding to the model to be detected; and calculating the similarity coefficient between the model to be detected and the sample model.

[0014] In some embodiments, the process of obtaining the injection detection result is as follows: obtaining binary data corresponding to weight parameters and bias parameters corresponding to the model to be detected in the model parameter set; performing injection detection on the operation code field in the binary data to obtain the injection detection result.

[0015] To achieve the above-mentioned purpose, the second aspect of an embodiment of the present application provides a model sensitive data detection device, comprising: a model parsing module, used to load a model file of a model to be detected, and read a file data stream in the model file; perform model parsing processing on the file data stream to determine the framework type of the model file; obtain the model graph structure data corresponding to the model file according to the framework type and the file data stream, and obtain the model parameter set corresponding to the model file from the model graph structure data; a sensitive data detection module, used to perform sensitive data inclusion detection on the model parameters in the model parameter set based on a preset sensitive word library to obtain a first detection result; a hidden data detection module, used to perform abnormal hidden data detection on the weight parameters and bias parameters corresponding to the model to be detected in the model parameter set when the first detection result indicates that the model parameter set does not contain sensitive data, and obtain a second detection result; a model operation module, used to perform a safe operation detection on the model to be detected in an isolated environment when the second detection result indicates that the model parameter set does not contain abnormal hidden data, and obtain an operation detection result.

[0016] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application provides an electronic device, comprising: at least one processor; at least one memory for storing at least one program; when at least one of the programs is executed by at least one of the processors, a model-sensitive data detection method as described in any one of the first aspects is implemented.

[0017] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the model-sensitive data detection method as described in any one of the first aspects.

[0018] According to the model sensitive data detection method and related equipment proposed in the embodiment of the present application, there are at least the following beneficial effects: the embodiment of the present application loads the model file of the model to be detected, reads the file data stream in the model file, performs model parsing processing on the file data stream to determine the framework type of the model file, and then can quickly and effectively obtain the model graph structure data corresponding to the model file according to the framework type and the file data stream, and obtain the model parameter set corresponding to the model file from the model graph structure data; further, a preliminary sensitive data inclusion detection is performed on the model parameter set through a preset sensitive word library to ensure that the model parameter set does not directly contain sensitive data, and on the basis of the preliminary detection, the weight parameters and bias parameters corresponding to the model to be detected in the model parameter set are detected for abnormal hidden data to ensure that the model parameter set does not contain any form of sensitive data, and then when the second detection result indicates that the model parameter set does not contain abnormal hidden data, a safe operation detection is performed on the model to be detected in an isolated environment to obtain an operation detection result, so as to realize multi-level detection of sensitive data and ensure the comprehensiveness and depth of detection, so as to improve the detection efficiency and detection accuracy of sensitive data detection to ensure the security of model operation. In addition, in the embodiments of the present application, the corresponding detection sub-module can be loaded from the preset detection sub-module set according to user demand information, so that the detection method can be adjusted according to different security requirements and data protection standards, effectively improving the flexibility and configurability of the detection method. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A method flow chart of a model sensitive data detection method provided in an embodiment of the present application;

[0020] Figure 2 A schematic diagram of obtaining a model parameter set in a model sensitive data detection method provided in an embodiment of the present application;

[0021] Figure 3 A flow chart of a method for performing model parsing processing in a model sensitive data detection method provided in an embodiment of the present application;

[0022] Figure 4 A flow chart of a method for determining a framework type in a model sensitive data detection method provided in an embodiment of the present application;

[0023] Figure 5 A flow chart of a method for performing decompression processing in a model sensitive data detection method provided in an embodiment of the present application;

[0024] Figure 6 A flowchart of a method for visualizing a model-sensitive data detection method provided in an embodiment of the present application;

[0025] Figure 7 A flow chart of a method for detecting abnormal hidden data in a model sensitive data detection method provided in an embodiment of the present application;

[0026] Figure 8 An example flow chart of a model sensitive data detection method provided in one embodiment of the present application;

[0027] Fig. 9 A schematic diagram of the system structure corresponding to the model sensitive data detection method provided in one embodiment of the present application;

[0028] Fig.10 A schematic diagram of the structure of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0030] In some embodiments, although the functional modules are divided in the system schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the system or the order in the flowchart. The terms first, second, etc. in the specification, claims and the above drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0031] In addition, unless otherwise clearly specified and limited, the term "connected" should be understood in a broad sense. For example, it can be a fixed connection or a movable connection, a detachable connection or a non-detachable connection, or an integral connection; it can be a mechanical connection, an electrical connection, or the two can communicate with each other; it can be a direct connection or an indirect connection through an intermediate medium.

[0032] In the description of the embodiments of the present application, the description with reference to the terms "one embodiment / implementation", "another embodiment / implementation" or "certain embodiments / implementations", "in the above embodiments / implementations", etc. means that the specific features, structures, materials or characteristics described in combination with the implementation or example are included in at least two embodiments or implementations disclosed in the present application. In the disclosure of the present application, the schematic representation of the above terms may not necessarily be the same embodiment or implementation. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in an order different from that in the flowchart.

[0033] Existing machine learning models may involve user sensitive data, such as user privacy information, commercial confidential information, etc. when processing and storing data, which may lead to the risk of leakage or abuse of these data, which may cause serious damage to individuals or organizations. Therefore, it is currently necessary to detect model sensitive data to determine the security of the model; however, existing security detection methods generally require formal verification of parameters such as model accuracy based on specific verification tools and mathematical models to confirm whether there is sensitive data. The detection process is complicated, resulting in low detection efficiency. In addition, since current neural network models have strong robustness, the model will not suffer a large loss of accuracy due to the inclusion of privacy data, resulting in poor comprehensiveness and depth of existing security detection methods, and prone to false positives or omissions, which affects the detection accuracy and cannot ensure the security of model operation.

[0034] Based on this, an embodiment of the present application provides a model sensitive data detection method, which can quickly and effectively perform multi-level sensitive data detection on the model files of the model to be detected, so as to ensure the comprehensiveness and depth of the detection, thereby improving the detection efficiency and detection accuracy of sensitive data to ensure the security of model operation.

[0035] The present application scheme is described below with reference to the accompanying drawings.

[0036] refer to Figure 1 , Figure 1 A method flow chart of a model sensitive data detection method provided in an embodiment of the present application; a first aspect of an embodiment of the present application provides a model sensitive data detection method, comprising at least the following steps:

[0037] Step S110, loading the model file of the model to be detected, and reading the file data stream in the model file; performing model parsing processing on the file data stream to determine the framework type of the model file;

[0038] Step S120, obtaining model graph structure data corresponding to the model file according to the framework type and the file data stream, and obtaining a model parameter set corresponding to the model file from the model graph structure data;

[0039] Step S130, performing sensitive data detection on the model parameters in the model parameter set based on a preset sensitive word library to obtain a first detection result;

[0040] Step S140, when the first detection result indicates that the model parameter set does not contain sensitive data, abnormal hidden data detection is performed on the weight parameters and bias parameters corresponding to the model to be detected in the model parameter set to obtain a second detection result;

[0041] Step S150, when the second detection result indicates that the model parameter set does not contain abnormal hidden data, a safe operation detection is performed on the model to be detected in an isolated environment to obtain an operation detection result.

[0042] Among them, the model file can be a file saved after the machine learning model is trained, which can contain the model's structural information, weight parameters and other necessary metadata information; the file data stream can be a continuous sequence of bytes read from the model file, which can contain various information of the model, such as structure, parameters, etc. The model parsing process can be a series of processing actions, the purpose of which is to parse the content of the model file and understand the structure and parameters of the model. This process may include reading the model file header information, parsing the model architecture, extracting parameters, etc.; the model framework type can be a software framework or platform type used to build and train machine learning models, such as TensorFlow, PyTorch, etc. Each framework has a different file format and data organization method; the model graph structure data can be a representation of the internal structure of the model, which can be composed of multiple nodes, each node represents an operation in the model (such as convolution, fully connected layer, etc.), and the lines between nodes represent the direction of data flow.

[0043] It can be understood that the model parameter set can be a set of all parameters that constitute the machine learning model, including but not limited to weight parameters, bias parameters, etc. The sensitive word library is a database containing a series of predefined sensitive words or patterns, which is used to detect whether the text or data contains sensitive information, such as personal identity information, telephone numbers, email addresses, etc.; the first detection result: refers to the result of whether sensitive information is contained after detecting the model parameter set through the sensitive word library. The weight parameter and bias parameter are parameters that constitute the neural network model. The weight parameter determines the degree of influence of the input signal on the output, and the bias parameter is the threshold of the neuron activation function.

[0044] It is worth noting that abnormal hidden data detection is used to detect whether the model parameters contain abnormal, potential privacy data or other illegal information, for example, by checking whether the least significant bits of the model parameters contain regular patterns, or by analyzing the model behavior to find abnormal patterns. The second detection result refers to the result of whether the model parameters contain abnormal hidden data after performing abnormal hidden data detection on the model parameters.

[0045] Furthermore, the isolated environment can be an execution environment isolated from the outside world, such as a sandbox, in which unverified code can be safely run without affecting external systems. Running models in such an environment can prevent potential malicious code from causing harm to the real system, and then running the model to be tested in the isolated environment for safe operation detection, and monitoring its behavior to evaluate whether the model contains potential security risks.

[0046] In some embodiments, the present application loads the model file of the model to be detected, reads the file data stream in the model file, performs model parsing processing on the file data stream to determine the framework type of the model file, and then can quickly and effectively obtain the model graph structure data corresponding to the model file according to the framework type and the file data stream, and obtain the model parameter set corresponding to the model file from the model graph structure data; further, a preliminary sensitive data inclusion detection is performed on the model parameter set through a preset sensitive word library to ensure that the model parameter set does not directly contain sensitive data, and on the basis of the preliminary detection, the weight parameters and bias parameters corresponding to the model to be detected in the model parameter set are detected for abnormal hidden data to ensure that the model parameter set does not contain any form of sensitive data, and then when the second detection result indicates that the model parameter set does not contain abnormal hidden data, a safe operation detection is performed on the model to be detected in an isolated environment to obtain an operation detection result, so as to realize multi-level detection of sensitive data and ensure the comprehensiveness and depth of detection, so as to improve the detection efficiency and detection accuracy of sensitive data detection to ensure the security of model operation. In addition, in the embodiments of the present application, the corresponding detection sub-module can be loaded from the preset detection sub-module set according to user demand information, so that the detection method can be adjusted according to different security requirements and data protection standards, effectively improving the flexibility and configurability of the detection method.

[0047] In some embodiments, when the first detection result indicates that the model parameter set contains sensitive data, it means that the audit system corresponding to the method of the present application has detected personal information or other confidential information that should not exist in the model parameters. At this time, the system will record this abnormal situation and generate an abnormal record file, indicating that sensitive information exists in the model. Corresponding measures can be taken subsequently to prevent the release or use of the model.

[0048] In some embodiments, when the second detection result represents that the model parameter set contains abnormal hidden data, it represents that the audit system corresponding to the method of the present application has detected that hidden sensitive data is carried or embedded in the model parameters. For example, the data may be embedded in the least significant bit of the model parameter, or exist in the hidden layer neurons in an encrypted form. This will also be recorded and used as part of the abnormal record, that is, the record statistics of all abnormal analysis results of the malicious inclusion detection model.

[0049] In some embodiments, when the second detection result characterizes that the model parameter set does not contain abnormal hidden data, the model to be detected is tested for safe operation in an isolated environment to obtain an operation test result; it is understandable that the present application provides an automated sandbox creation and destruction process by providing a temporary and completely isolated environment, so that users can safely run the detected model, and at the same time monitor the resource usage and network activities in the sandbox in real time, and use load balancing to reasonably allocate CPU / NPU, memory, disk I / O and other resource restrictions to prevent resource abuse. All operations and events in the sandbox are recorded to ensure that the operations are traceable, and logs are regularly reviewed to check for abnormal behavior, further eliminating malicious smuggling.

[0050] refer to Figure 2 , Figure 2 A schematic diagram of obtaining a model parameter set in a model sensitive data detection method provided in an embodiment of the present application; in some embodiments, a model file of a model to be detected is loaded, and a file data stream in the model file is read; the file data stream is subjected to model parsing processing to determine the framework type of the model file. The process of obtaining the model graph structure data corresponding to the model file according to the framework type and the file data stream, and obtaining the model parameter set corresponding to the model file from the model graph structure data may include: running a model parsing and visualization module, which is responsible for loading the model and obtaining the string data of the model. By using a model visualization warehouse, such as the Python open source warehouse Netron, to assist in the visualization of the model and obtain the string data of the model, the model file is read according to the framework type of the model file, a model graph structure is generated, and the model parameter set is output as the data stream basis for privacy-sensitive data review.

[0051] Among them, the function of the model parsing and visualization module is to load the specified model file and read the string data therein, obtain the string data of the model by using specific tools or libraries (such as Netron), parse the model file to determine its framework type, and generate the graph structure data of the model; the model visualization warehouse can help developers or researchers view and understand the internal structure of complex models, and then create a graphical representation to show the architecture of the model, including the connection relationship between each layer.

[0052] Furthermore, Figure 2 The corresponding model parameter set acquisition process in may include: when a user initiates a request, the service will start, read the data stream and file name of the model file, and create a model factory as an intermediate link for data stream and file name processing. The model factory can be understood as a processing center, which receives instructions from the model parsing and visualization service and is responsible for subsequent data processing tasks; the model factory will store a comparison table of the model framework and its file extension, so as to select the correct parsing tool (model renderer) according to the file extension. The model renderer is used to convert the model file data into an understandable form, such as a model image object and a node object; further, by including a series of regular tables Dashi's exception regularization library detects whether the file data stream meets the expected format requirements. If the file content does not meet the expected format, the system will mark the file as illegal. If the file is in a compressed format, it needs to be decompressed before anomaly detection. Furthermore, by parsing the model data layer by layer, the graph structure of the model is constructed to display the hierarchical relationship of the model. Each component of the model, such as layers and nodes, each node contains the model parameters of this part, and the model parameter set is obtained: the set of all parameters in the model. These parameters will be used for subsequent sensitive data audits, and tools such as Netron are used to convert the model graph structure into a graphical representation to facilitate users to understand the model structure and parameter configuration.

[0053] It can be understood that through the above steps, the present application can effectively extract the model graph structure data from the model file to be tested, and then obtain the model parameter set, so as to prepare for the subsequent privacy-sensitive data review.

[0054] refer to Figure 3 , Figure 3 A flow chart of a method for performing model parsing in a model sensitive data detection method provided in an embodiment of the present application; in some embodiments, performing model parsing on a file data stream to determine a framework type of a model file includes at least the following steps:

[0055] Step S310, obtaining an extension name dataset, where the extension name dataset includes file extension names corresponding to multiple framework types;

[0056] Step S320, parsing the file extensions of the plurality of file data from the file data stream, and matching the framework type corresponding to the model file from the extension data set according to the file extensions of the plurality of file data.

[0057] The extension dataset can contain a data set of file extensions corresponding to various machine learning frameworks. For example, a TensorFlow model file may be in .pb or .ckpt format, while a PyTorch model may be in .pt or .pth format. This dataset helps identify the specific framework type to which the model file belongs.

[0058] It can be understood that the file data stream is a series of bytes read from the model file, representing the content of the model file. The file extension is the string after the file name, which is used to indicate the type of file. In the case of a model file, the extension can help identify which framework the file was created by. Then, based on the parsed file extension, the corresponding entry can be found in the extension dataset to determine which framework the model file belongs to.

[0059] refer to Figure 4 , Figure 4 A flow chart of a method for determining a framework type in a model sensitive data detection method provided in an embodiment of the present application; in some embodiments, obtaining model graph structure data corresponding to a model file according to the framework type and the file data stream includes at least the following steps:

[0060] Step S410, performing file screening processing on the file data stream based on a preset abnormal regular library to obtain a plurality of file data in the file data stream that conform to an expected file format;

[0061] Step S420, obtaining multiple target file types according to multiple file extensions corresponding to the multiple file data, and selecting corresponding model renderers for the multiple target file types according to the framework type;

[0062] Step S430, decoding the multiple file data based on the multiple model renderers to obtain multiple image object data and node object data;

[0063] Step S440, obtaining model graph structure data corresponding to the model file according to the plurality of image object data and node object data.

[0064] Among them, the preset exception regular library can be a collection of a series of regular expressions, which are used to match legal file formats. Any file data that does not conform to these regular expressions will be regarded as abnormal or erroneous file data. Then, the file data stream can be filtered by applying regular expressions to ensure that only file data that conforms to the expected format is retained.

[0065] In some embodiments, the target file type can be a set of file types that are associated with the framework type of the model file. The target file type is determined by identifying the file extension. Then, a suitable tool or program (i.e., a model renderer) is selected for each target file type so that the file data of these types can be correctly parsed and displayed.

[0066] In some embodiments, a model renderer is a tool or program used to process a specific type of model file, responsible for converting file data into an understandable form, such as image object data and node object data, by using the model renderer to convert the file data into a format that is easy to understand and process.

[0067] In some embodiments, the image object data may be visual image data representing the model structure, and the node object data may be detailed information on each component or operation in the model graph, such as the function, input and output of each layer, etc. The model graph structure data is a representation of the internal structure of the model, which describes the hierarchical relationship of the model, the connection between nodes, and the way data flows.

[0068] In some embodiments, based on a preset sensitive word library, sensitive data detection is performed on the model parameters in the model parameter set to obtain a first detection result, which may include performing sensitive data detection through a sensitive data review module, and the sensitive data review module receives a model parameter set extracted from a model file. These parameters may include key numerical information in the model such as weight matrices and bias items, and special data detection tools are used to process these model parameter data. These tools generally include but are not limited to text processing functions, which can identify and mark potential sensitive information; further, the extracted model string information is preprocessed to remove redundant symbols that may interfere with sensitive information detection, including removing spaces, brackets "[]", commas "," slashes "\" and carriage returns, etc., to ensure that during the detection process, sensitive information will not be missed due to the presence of these symbols.

[0069] Furthermore, after preprocessing is completed, the pre-set sensitive word library can be loaded. The sensitive word library is dynamically configurable and can update the sensitive word list at any time according to needs to adapt to new privacy protection needs. The sensitive word library can contain a series of predefined keywords or patterns for identifying personal information such as name, phone number, ID number, etc.; the pre-processed model parameter data can be searched through the sensitive word library to detect whether it contains sensitive information. This step involves scanning each element in the model parameter to find out whether there is information that matches the pattern defined in the sensitive word library.

[0070] Furthermore, after completing the above-mentioned tests, a first test result can be obtained, indicating whether the model parameter set contains sensitive information. If sensitive information is detected, the system will record this information and may generate an exception record file for subsequent processing or auditing. If no sensitive information is detected, the model parameter set is considered safe and subsequent review or use can continue.

[0071] It can be understood that through the above steps, the sensitive data review module can effectively identify sensitive information that may exist in the model parameters and make corresponding processing decisions based on this, thereby ensuring the security and privacy of the model.

[0072] refer to Figure 5 , Figure 5 A flowchart of a method for decompression processing in a model sensitive data detection method provided in an embodiment of the present application; in some embodiments, before performing file screening processing on a file data stream based on a preset abnormal regular library, at least the following steps are included:

[0073] Step S510, performing compressed data detection on the file data stream;

[0074] Step S520: When the file data stream contains compressed data, decompress the compressed data to obtain file data corresponding to the compressed data.

[0075] In some embodiments, compressed data detection is used to search for compressed data in a file data stream. Compressed data may include files in common compression formats such as ZIP and TAR. Detecting compressed data is to ensure that subsequent processing is of decompressed data for more accurate model parsing. Furthermore, if compressed data is detected in the file data stream, it is necessary to use an appropriate decompression tool to decompress the data to obtain the original uncompressed file data. The purpose of this is to ensure that all subsequently processed data is in an uncompressed format for further analysis and processing.

[0076] refer to Figure 6 , Figure 6 A method flow chart for visualization in a model sensitive data detection method provided in an embodiment of the present application; in some embodiments, after obtaining the model graph structure data corresponding to the model file according to the framework type and the file data stream, at least the following steps are included:

[0077] Step S610, obtaining a model visualization tool corresponding to the framework type from a preset algorithm library;

[0078] Step S620: Generate a visualized model view corresponding to the model graph structure data based on a model visualization tool.

[0079] In some embodiments, the preset algorithm library is a pre-configured algorithm collection, which may include model visualization tools for different framework types. The framework type here refers to the development framework used by the machine learning model, such as TensorFlow, PyTorch, etc. The model visualization tool is used to convert the model structure into a visual form to facilitate the user to understand the structure and parameters of the model. Commonly used model visualization tools include Netron, etc., and the model visualization tool can be used to convert the model graph structure data into a graphical interface, so that the user can intuitively see the structure of the model and the relationship between each component.

[0080] refer to Figure 7 , Figure 7 A flow chart of a method for abnormal hidden data detection in a model sensitive data detection method provided in an embodiment of the present application; in some embodiments, abnormal hidden data detection is performed on weight parameters and bias parameters corresponding to the model to be detected in the model parameter set to obtain a second detection result, which at least includes the following steps:

[0081] Step S710, performing model anomaly analysis according to the weight parameters and bias parameters corresponding to the model to be detected in the model parameter set, and obtaining a model anomaly index corresponding to the model to be detected, wherein the model anomaly index includes at least one of the following: uniform distribution coefficient, output difference coefficient, similarity coefficient and injection detection result;

[0082] Step S720, determining whether the model parameter set contains abnormal hidden data according to the model abnormality index, and obtaining a second detection result.

[0083] In some embodiments, the weight parameters and bias parameters of the model are statistically analyzed to check whether there are abnormal conditions. For example, the least significant bit (LSB) can be analyzed to detect whether there is a non-random pattern, or the mean and variance of the parameters can be calculated to detect whether there is a deviation from the normal distribution. Model abnormality indicators include but are not limited to uniform distribution coefficients (used to detect whether the LSB is uniformly distributed), output difference coefficients (used to detect whether the model's response to adversarial samples is abnormal), similarity coefficients (used to detect the similarity between the model and the normal model) and injection detection results (used to detect whether malicious code is inserted into the model). Based on the analysis of model abnormality indicators, it is determined whether there is abnormal hidden data in the model parameter set. If an abnormality is found, it indicates that the model may have hidden data, which needs to be further processed or excluded. If no abnormality is found, it indicates that the model parameter set is relatively normal and can continue to be reviewed or used.

[0084] In some embodiments, the process of obtaining the uniform distribution coefficient is as follows: obtaining the least significant bit distribution information of the weight parameters and bias parameters corresponding to the model to be detected in the model parameter set; calculating the uniform distribution coefficient corresponding to the model parameter set according to the least significant bit distribution information, wherein the model parameter set refers to all parameters extracted from the model, including weight parameters and bias parameters, which constitute the basic components of the model; the least significant bit (Least Significant Bit, LSB) refers to the rightmost bit of a binary number in digital representation, which is the part with the smallest value. When detecting entrained data, LSB can be used to hide information because it has the least impact on the value and is not easy to attract attention; the least significant bit distribution information is the least significant bit taken for each model parameter for subsequent statistical analysis.

[0085] It can be understood that the uniform distribution coefficient corresponding to the model parameter set is calculated according to the least significant bit distribution information, the least significant bit (LSB) values ​​of all model parameters are recorded, the frequency of occurrence of 0 and 1 is counted, and the actual number of occurrences of each LSB value (0 or 1) is recorded. In the absence of LSB steganography, it is assumed that the number of occurrences of 0 and 1 should be equal, that is, E0=E1=N / 2, where N is the total number of LSBs. The chi-square test is used to detect the uniformity of the LSB distribution. According to the chi-square value χ2 and the set significance level α, the critical value χ2_(α) in the critical value table is found. If the calculated chi-square value is greater than the critical value χ2_(α), the original hypothesis is rejected, and it is believed that the LSB distribution of the model parameters is not random, and there may be entrained data behavior. Based on the statistical analysis, the uniform distribution coefficient can be calculated according to the chi-square value. The coefficient reflects whether the LSB distribution is uniform. If the LSB distribution is close to uniform, the uniform distribution coefficient is close to 1; if it deviates from uniformity, the coefficient is far away from 1.

[0086] Through the above steps, it is possible to effectively detect whether there is any behavior of data entrainment by modifying the least significant bit in the model parameters. The uniform distribution coefficient, as a quantitative indicator to measure whether the LSB distribution is uniform, can help the present application determine whether the model may be used to hide information. If the test result shows that the uniform distribution coefficient deviates from the ideal value, it indicates that the model may be at risk of data entrainment.

[0087] In some embodiments, specifically, a chi-square test is used to detect the uniformity of the LSB distribution. (where Ok is the actual frequency and Ek is the expected frequency). Without LSB steganography: Record the actual frequency and expected frequency to calculate the chi-square statistic By statistically analyzing the distribution of model parameters, finding outliers and calculating the mean of model parameters and standard deviation Then use the normality test to check whether the parameters conform to the normal distribution. Detect whether there are any deviations from the normal distribution.

[0088] In some embodiments, the process of obtaining the output difference coefficient is as follows: generate multiple test samples according to the weight parameters and bias parameters corresponding to the model to be tested in the model parameter set, and generate multiple adversarial samples based on the multiple test samples and the preset loss function; input the multiple test samples and the multiple adversarial samples into the model to be tested respectively to calculate the output difference coefficients of the test samples and the adversarial samples; wherein the test samples are a series of representative input data generated, which should cover various scenarios that the model may encounter, and the selection of test samples is representative to ensure that the behavior of the model can be fully detected; the weight parameters and bias parameters are extracted from the model file and are used to define the calculation logic within the model. The test samples can be generated according to the characteristics of the model and the application scenarios to ensure that the main functions of the model are covered.

[0089] It can be understood that adversarial samples are samples generated by adding small perturbations to the original input data. These perturbations are added along the gradient direction of the loss function, aiming to make the model produce wrong predictions. Adversarial sample generation can use adversarial sample techniques such as FGSM or PGD.

[0090] The preset loss function can make the output of the model on the adversarial sample as close as possible to certain sensitive data or features. The choice of loss function should be related to the task of the model and reflect the sensitivity of the model to input changes. For example, a series of adversarial samples X′={X1′,X2′,…,Xn′} can be generated by adding small perturbations along the gradient direction of the loss function. (X is the original input, X′ is the adversarial sample, ∈ is the perturbation size, J(θ,x,y) is the loss function, is the gradient of the loss function with respect to the input).

[0091] Furthermore, multiple test samples and multiple adversarial samples are input into the model to be tested respectively to calculate the output difference coefficient of the test samples and the adversarial samples, and the generated test samples and adversarial samples are input into the model to be tested respectively, and the model is allowed to make predictions, and the output of the model for the test samples and adversarial samples is recorded, and the difference between the two is calculated. The output difference can be the absolute difference of the prediction results or a certain distance metric to calculate the mean and variance of the output differences of all test samples and adversarial samples. The output difference coefficient can be defined by these statistics. For example, the mean can represent the average difference size, and the variance can represent the degree of fluctuation of the difference. If the output difference exceeds the preset threshold, it is considered that the model is abnormal and may contain sensitive data or features.

[0092] In some embodiments, the output difference ΔOutput between the original input and the adversarial sample is calculated i =f(X ′ i )-f(X i ), and calculate the mean of all output differences and variance Then conduct statistical analysis to check whether there are any abnormalities Set a threshold; To detect the entrainment behavior of encrypted embedded hidden neurons, gradient analysis of hidden layer neurons can be performed to check whether there is an abnormal gradient distribution or pattern. Assume that L is the loss function, h i Is the activation value vector of the hidden layer neurons, and calculates the gradient of the loss function to the activation value of the hidden layer neurons. The mean gradient of the hidden layer neuron j is: Variance of the gradient of hidden layer neuron j By calculating the mean and variance of the gradients of the hidden layer neurons and performing statistical analysis on them, it is possible to detect whether there are abnormal gradient patterns.

[0093] Through the above steps, the behavior of the model when facing adversarial samples can be effectively detected, especially whether there are abnormal output patterns. The output difference coefficient, as a quantitative indicator, can help this application evaluate the stability of the model output and whether there is a potential risk of entrained data.

[0094] In some embodiments, the process of obtaining the similarity coefficient is as follows: perform federated learning training according to the weight parameters and bias parameters corresponding to the model to be detected in the model parameter set to obtain a sample model corresponding to the model to be detected; calculate the similarity coefficient between the model to be detected and the sample model, wherein federated learning training refers to a federated learning environment in which multiple participants (clients) jointly train a shared model without directly exchanging data. Each participant uses its own local data for model training and only shares model updates (such as weight updates); the sample model refers to a model obtained by training in accordance with the federated learning protocol without any private data, which can be used as a benchmark model for comparison and analysis with the model to be detected. The sample model should be generated under the same training protocol and does not contain any private data or abnormal hidden data; the model to be detected refers to the model that needs to be detected. The model may be trained by other means or obtained from the federated learning process, and it is necessary to verify whether it contains private data.

[0095] It can be understood that the weight similarity between the detection model and the sample model is determined by calculating the cosine similarity of the model weights, the output KL divergence on the same input is compared to verify whether the model output has significant changes, and the gradient similarity of the model is verified by comparing the gradient distance on the same input. According to the results of the above similarity calculation, a similarity coefficient can be obtained to determine the similarity between the model to be detected and the sample model. If the similarity coefficient is high, it indicates that the difference between the two models is small, and there may be no data entrainment behavior; if the similarity coefficient is low, it indicates that there is a large difference between the two models, and there may be data entrainment behavior, and then effectively compare the similarity between the model to be detected and the sample model, so as to determine whether the model to be detected has the risk of data entrainment. The similarity coefficient, as a comprehensive indicator, can help this application evaluate the similarity of the model from multiple angles and ensure the safety and reliability of the model.

[0096] Specifically, the present application can calculate the cosine similarity of the model weights To determine the weight similarity between the detection model and the sample model, where ||wi|| and ||wj|| are the norms of the weight vectors of the detection model and the sample model respectively; by comparing the output KL divergence on the same input Verify whether the model output has significant changes, where y1 and y2 are the probability output vectors of the two models;

[0097] By comparing the gradient distance on the same input, Verify the gradient similarity of the model, where are the gradients of the two models on the input sample data Xi, and n is the dimension of the input.

[0098] In some embodiments, the process of obtaining the injection detection result is as follows: obtaining binary data corresponding to weight parameters and bias parameters corresponding to the model to be detected in the model parameter set; performing injection detection on the operation code field in the binary data to obtain the injection detection result, wherein the model parameter set refers to all parameters extracted from the model file, including weight parameters (weights) and bias parameters (biases), which constitute the basic components of the model.

[0099] In some embodiments, the model file is usually stored in binary format, which means that the data in the file exists in the form of bits and bytes. Obtaining the binary data corresponding to the model parameters means reading the model file and extracting the binary representation of the weight parameters and bias parameters from it. In computer science, an opcode refers to a part of a machine language instruction set that instructs the processor to perform a specific operation. In the binary representation of the model file, the opcode is used to indicate how to process the data stream. Therefore, this application checks the opcode field in the binary data to ensure that these opcodes conform to the expected behavior pattern and do not contain any operations that may lead to malicious behavior. The specific detection steps include:

[0100] Traverse the PROTO opcodes of the parsed model file. If duplicate opcodes are found, this may mean that the model file has been tampered with. Make sure that the version numbers of all opcodes are consistent. Otherwise, it may indicate that there is a problem with the model file. Check whether the location of the PROTO opcode meets the requirements of its protocol version. The location and order of the opcodes should comply with the provisions of the protocol version used by the model file. Any non-compliance may indicate that the file has been tampered with to obtain injection detection results.

[0101] Furthermore, the AST nodes of the parsed model file are traversed. AST nodes: Abstract Syntax Tree (AbstractSyntaxTree) is a tree representation of the source code structure. The purpose of traversing the AST nodes is to check whether the code in the model file calls certain dangerous functions, and check whether dangerous functions such as eva l, exec, compile, etc. are called. These functions can execute arbitrary code, so they should not appear in the model file. If the call of these functions is detected, then the model file can be considered to be almost malicious, so as to obtain the injection detection result.

[0102] Furthermore, the interpreter is used to traverse the opcodes and check for unused variable assignments: the interpreter is used to traverse the opcodes in the model file to check whether there are unused variable assignments. If unused variable assignments are found, then these variables may be part of the malicious code, making the model file suspicious, so as to obtain injection detection results.

[0103] Based on the results of the above detection steps, a conclusion can be drawn as to whether there is malicious injection behavior in the model file. If any unexpected behavior is found during the detection process, such as repeated opcodes, calls to dangerous functions, or unused variable assignments, the model file will be considered suspicious and may contain malicious injection behavior. This can effectively detect whether there is malicious injection behavior in the model file, thereby ensuring the security and reliability of the model. The injection detection results serve as the final judgment basis to help this application identify whether the model file has been tampered with or contains potential malicious code.

[0104] refer to Figure 8 and Fig. 9 , Figure 8 This is an example flow chart of a model sensitive data detection method provided in an embodiment of the present application. Fig. 9 A schematic diagram of the system structure corresponding to the model sensitive data detection method provided in an embodiment of the present application; in some embodiments, the present application can load the model file to be detected through the model parsing and visualization module, read the data flow in the model file, parse the data flow of the model file, determine the framework type used by the model, and generate the graph structure data of the model according to the framework type, which includes the various components of the model (such as layers, nodes, etc.) and their connection relationships, and use tools to visualize the model graph structure to facilitate users to view the model structure and parameter characteristics; through the sensitive data review module, extract the model parameter set from the model graph structure data, use a dynamically configurable sensitive word library, perform sensitive data detection on the model parameters, identify whether there is sensitive information such as identity card, email address, telephone number, etc., pre-process the extracted model string information, remove redundant symbols, facilitate sensitive information detection, overload the dynamic sensitive word matching library, check whether there is sensitive data, and record the detection results; through the malicious entrainment detection module: Perform statistical analysis on the least significant bits (LSB) of the model parameters to detect whether there are non-random patterns or distribution anomalies to determine whether there is entrained data, provide a series of representative inputs, and analyze whether there are abnormal patterns in the model output, such as using adversarial sample technology to detect whether the model has an abnormal response to specific inputs, and compare it with the sample model without privacy data entrainment trained in accordance with the federated learning protocol, calculate the similarity of weights, outputs, and gradients, detect whether there is significant deviation in the model, binary serialize and deserialize the model file, traverse the opcodes and AST nodes, and detect whether there are dangerous function calls or unused variable assignments; provide a temporary and completely isolated environment through the sandbox environment running module, allowing users to safely run the detected model, monitor resource usage and network activity in the sandbox in real time, ensure reasonable resource allocation, and prevent resource abuse, record all operations and events in the sandbox to ensure that operations are traceable, and regularly audit logs to check for abnormal behavior.

[0105] In some embodiments, the present application can implement a multi-level audit mechanism. By combining multiple protection mechanisms such as model analysis, sensitive word detection, malicious inclusion detection, and sandbox environment operation, the solution of the present invention comprehensively improves the security and privacy protection capabilities of the machine learning model. It also intelligently extracts model parameters through the model analysis and visualization module, and uses a dynamically configurable sensitive word library to perform sensitive data detection, which can quickly and accurately discover sensitive information in the model parameters. In addition, the solution of the present invention has high flexibility and configurability, and users can dynamically adjust the detection strategy according to different security requirements and data protection standards, thereby improving practicality and adaptability.

[0106] A second aspect of an embodiment of the present application provides a model sensitive data detection device, including: a model parsing module, used to load a model file of a model to be detected, and read a file data stream in the model file; perform model parsing processing on the file data stream to determine the framework type of the model file; obtain model graph structure data corresponding to the model file according to the framework type and the file data stream, and obtain a model parameter set corresponding to the model file from the model graph structure data; a sensitive data detection module, used to perform sensitive data inclusion detection on model parameters in a model parameter set based on a preset sensitive word library to obtain a first detection result; a hidden data detection module, used to perform abnormal hidden data detection on weight parameters and bias parameters corresponding to the model to be detected in the model parameter set when the first detection result indicates that the model parameter set does not contain sensitive data, to obtain a second detection result; a model running module, used to perform a safe operation detection on the model to be detected in an isolated environment to obtain an operation detection result when the second detection result indicates that the model parameter set does not contain abnormal hidden data.

[0107] refer to Fig.10 , Fig.10 This is a structural diagram of an electronic device provided in an embodiment of the present application. The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the model sensitive data detection method of any of the above embodiments is implemented. For example, the above described Figure 1 Steps S110 to S150 of the method described above are performed. Figure 3 Steps S310 to S320 of the method, Figure 4 Steps S410 to S440 of the method, Figure 5 Steps S510 to S520 of the method, Figure 6 Steps S610 to S620 of the method, Figure 7 Method steps S710 to S720.

[0108] The electronic device 1000 of the embodiment of the present application includes one or more processors 1010 and a memory 1020. Fig.10 In the figure, a processor 1010 and a memory 1020 are taken as an example.

[0109] The processor 1010 and the memory 1020 may be connected via a bus or other means. Fig.10 The example of connecting through bus is taken in the following.

[0110] The memory 1020, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer executable programs. In addition, the memory 1020 may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 1020 may optionally include a memory 1020 remotely disposed relative to the processor 1010, and these remote memories may be connected to the electronic device 1000 via a network. At the same time, examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0111] In some embodiments, when the processor executes the computer program, the model sensitive data detection method of any one of the above embodiments is performed at preset intervals.

[0112] Those skilled in the art will understand that Fig.10 The device structure shown in the figure does not constitute a limitation on the electronic device 1000, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0113] exist Fig.10 In the electronic device 1000 shown, the processor 1010 can be used to call the model sensitive data detection method stored in the memory 1020, so as to implement the model sensitive data detection method.

[0114] Based on the hardware structure of the above-mentioned electronic device 1000, various embodiments of the model-sensitive data detection device of the present application are proposed. At the same time, the non-transient software programs and instructions required to implement the model-sensitive data detection method of the above-mentioned embodiments are stored in the memory. When executed by the processor, the model-sensitive data detection method of the above-mentioned embodiments is executed.

[0115] The present application also provides a computer-readable storage medium storing computer-executable instructions for executing the above-mentioned model-sensitive data detection method, which can enable the above-mentioned one or more processors to execute the model-sensitive data detection method of any of the above-mentioned embodiments, for example, to execute the above-mentioned Figure 1Steps S110 to S150 of the method described above are performed. Figure 3 Steps S310 to S320 of the method, Figure 4 Steps S410 to S440 of the method, Figure 5 Steps S510 to S520 of the method, Figure 6 Steps S610 to S620 of the method, Figure 7 Method steps S710 to S720.

[0116] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network nodes. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0117] It will be appreciated by those skilled in the art that all or some of the steps and systems in the disclosed method above may be implemented as software, firmware, hardware and appropriate combinations thereof. Some physical components or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer-readable storage medium (or a non-transitory medium) and a communication medium (or a temporary medium). As known to those skilled in the art, the term computer-readable storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer-readable storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD to ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that may be used to store desired information and may be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media may contain computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0118] The above is a specific description of the preferred implementation of the present application, but the present application is not limited to the above implementation mode. Technical personnel familiar with the field can also make various equivalent modifications or substitutions without violating the spirit of the present application. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present application.

Claims

1. A model sensitive data detection method, characterized in that: include: Load the model file of the model to be detected, and read the file data stream in the model file; Performing model parsing processing on the file data stream to determine the framework type of the model file; Obtaining model graph structure data corresponding to the model file according to the framework type and the file data stream, and acquiring a model parameter set corresponding to the model file from the model graph structure data; Performing sensitive data detection on the model parameters in the model parameter set based on a preset sensitive word library to obtain a first detection result; When the first detection result indicates that the model parameter set does not contain sensitive data, performing abnormal hidden data detection on weight parameters and bias parameters corresponding to the model to be detected in the model parameter set to obtain a second detection result; When the second detection result indicates that the model parameter set does not contain abnormal hidden data, a safe operation detection is performed on the model to be detected in an isolated environment to obtain an operation detection result.

2. The model sensitive data detection method according to claim 1, characterized in that: The performing model parsing processing on the file data stream to determine the framework type of the model file includes: Acquire an extension name dataset, where the extension name dataset includes file extension names corresponding to multiple framework types; The file extensions of a plurality of file data are parsed from the file data stream, and the framework type corresponding to the model file is matched from the extension data set according to the file extensions of the plurality of file data.

3. The model sensitive data detection method according to claim 1, characterized in that: The obtaining of the model graph structure data corresponding to the model file according to the framework type and the file data stream includes: Performing file screening processing on the file data stream based on a preset abnormal regular library to obtain a plurality of file data in the file data stream that conform to an expected file format; Obtaining a plurality of target file types according to a plurality of file extensions corresponding to the plurality of file data, and selecting corresponding model renderers for the plurality of target file types respectively according to the framework type; Decoding the plurality of file data based on the plurality of model renderers to obtain a plurality of image object data and node object data; The model graph structure data corresponding to the model file is obtained according to the plurality of image object data and the node object data.

4. The model sensitive data detection method according to claim 3, characterized in that: Before the file data stream is subjected to file screening processing based on the preset abnormal regular library, the method further includes: Performing compressed data detection on the file data stream; In the case that the file data stream includes compressed data, the compressed data is decompressed to obtain file data corresponding to the compressed data.

5. The model sensitive data detection method according to any one of claims 1 to 4, characterized in that: After obtaining the model graph structure data corresponding to the model file according to the framework type and the file data stream, the method further includes: Obtain a model visualization tool corresponding to the framework type from a preset algorithm library; A visualization model view corresponding to the model graph structure data is generated based on the model visualization tool.

6. The model sensitive data detection method according to claim 1, characterized in that: The performing abnormal hidden data detection on the weight parameters and bias parameters corresponding to the to-be-detected model in the model parameter set to obtain a second detection result includes: Performing model anomaly analysis according to the weight parameters and bias parameters corresponding to the model to be detected in the model parameter set to obtain a model anomaly index corresponding to the model to be detected, wherein the model anomaly index includes at least one of the following: a uniform distribution coefficient, an output difference coefficient, a similarity coefficient, and an injection detection result; Determine whether the model parameter set contains abnormal hidden data according to the model abnormality indicator to obtain a second detection result.

7. The model sensitive data detection method according to claim 6, characterized in that: The process of obtaining the uniform distribution coefficient is as follows: Obtaining least significant bit distribution information of weight parameters and bias parameters corresponding to the model to be detected in the model parameter set; The uniform distribution coefficient corresponding to the model parameter set is calculated according to the least significant bit distribution information.

8. The model sensitive data detection method according to claim 6, characterized in that: The process of obtaining the output difference coefficient is as follows: Generate multiple test samples according to the weight parameters and bias parameters corresponding to the model to be detected in the model parameter set, and generate multiple adversarial samples based on the multiple test samples and a preset loss function; The multiple test samples and the multiple adversarial samples are respectively input into the model to be detected to calculate the output difference coefficients of the test samples and the adversarial samples.

9. The model sensitive data detection method according to claim 6, characterized in that: The process of obtaining the similarity coefficient is as follows: Performing federated learning training according to the weight parameters and bias parameters corresponding to the model to be detected in the model parameter set to obtain a sample model corresponding to the model to be detected; Calculate the similarity coefficient between the model to be detected and the sample model.

10. The model sensitive data detection method according to claim 6, characterized in that: The process of obtaining the injection detection result is as follows: Obtaining binary data corresponding to weight parameters and bias parameters corresponding to the model to be detected in the model parameter set; An injection detection is performed on the operation code field in the binary data to obtain an injection detection result.

11. A model sensitive data detection device, characterized in that: include: A model parsing module is used to load the model file of the model to be detected and read the file data stream in the model file; Performing model parsing processing on the file data stream to determine the framework type of the model file; obtaining model graph structure data corresponding to the model file according to the framework type and the file data stream, and obtaining a model parameter set corresponding to the model file from the model graph structure data; A sensitive data detection module, configured to perform a sensitive data inclusion detection on the model parameters in the model parameter set based on a preset sensitive word library to obtain a first detection result; a hidden data detection module, configured to perform abnormal hidden data detection on weight parameters and bias parameters corresponding to the model to be detected in the model parameter set when the first detection result indicates that the model parameter set does not contain sensitive data, to obtain a second detection result; The model operation module is used to perform a safe operation test on the model to be tested in an isolated environment to obtain an operation test result when the second test result indicates that the model parameter set does not contain abnormal hidden data.

12. An electronic device, characterized in that: include: at least one processor; at least one memory for storing at least one program; When at least one of the programs is executed by at least one of the processors, the model-sensitive data detection method according to any one of claims 1 to 10 is implemented.

13. A computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the model-sensitive data detection method as described in any one of claims 1 to 10.

Images