Sensitive data detection method and device based on intelligent agent and intelligent agent

Abstract

The invention provides an agent-based sensitive data detection method and device, an agent, electronic equipment, a storage medium and a computer program product, relates to the technical field of computers, in particular to the technical fields of agents, artificial intelligence large models, sensitive data detection and the like, and can be applied to a sensitive data detection scene. According to the specific implementation scheme, an analysis strategy of a to-be-detected file is determined according to a cognitive state of a data format of the to-be-detected file by utilizing an artificial intelligence large model through a decision-making layer of an intelligent agent; analyzing the to-be-detected file according to the analysis strategy through a decision-making layer to obtain target analysis data; and detecting whether the target analysis data comprises sensitive data or not through an execution layer of the intelligent agent to obtain a sensitive data detection result. According to the method and the device, the coverage range of the analysis scene by the analysis operation and the precision and the efficiency of the analysis process are improved, so that the comprehensiveness and the accuracy of a sensitive data detection result are improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, in particular to the technical field of agent, artificial intelligence large model, sensitive data detection, and the like, and more particularly to a sensitive data detection method and device based on an agent, an agent, an electronic device, a storage medium, and a computer program product, which can be applied to a sensitive data detection scenario. BACKGROUND

[0002] In the fields of natural resources, intelligent driving, and industrial manufacturing, enterprises often involve sensitive information processing in the process of data collection, transmission, storage, and delivery. With the continuous improvement of data compliance laws and regulations such as the Data Security Law, the Network Security Law, the Notice of the Ministry of Natural Resources on Promoting the Development of Intelligent Networked Vehicles and Safeguarding the Security of Surveying and Geographical Information, and the Provisions on the Management of Automobile Data Security, relevant enterprises are facing increasingly strict regulatory requirements in data compliance management. SUMMARY

[0003] The present disclosure provides a sensitive data detection method and device based on an agent, an agent, an electronic device, a storage medium, and a computer program product.

[0004] According to a first aspect, a sensitive data detection method based on an agent is provided, comprising: determining, by a decision layer of the agent, an analysis strategy of a to-be-detected file according to a cognitive state of a data format of the to-be-detected file by using an artificial intelligence large model; analyzing, by the decision layer, the to-be-detected file according to the analysis strategy to obtain target analysis data; and detecting, by an execution layer of the agent, whether the target analysis data includes sensitive data to obtain a sensitive data detection result.

[0005] According to a second aspect, a sensitive data detection device based on an agent is provided, comprising: a strategy determination unit configured to determine, by a decision layer of the agent, an analysis strategy of a to-be-detected file according to a cognitive state of a data format of the to-be-detected file by using an artificial intelligence large model; a data analysis unit configured to analyze, by the decision layer, the to-be-detected file according to the analysis strategy to obtain target analysis data; and a data detection unit configured to detect, by an execution layer of the agent, whether the target analysis data includes sensitive data to obtain a sensitive data detection result.

[0006] According to a third aspect, an agent for detecting sensitive data is provided, comprising: a perception layer configured to determine a to-be-detected file; a decision layer configured to determine an analysis strategy of the to-be-detected file according to a cognitive state of a data format of the to-be-detected file by using an artificial intelligence large model, and analyze the to-be-detected file according to the analysis strategy to obtain target analysis data; and an execution layer configured to detect whether the target analysis data includes sensitive data to obtain a sensitive data detection result.

[0007] According to a fourth aspect, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any implementation manner of the first aspect.

[0008] According to a fifth aspect, a non-transitory computer-readable storage medium storing computer instructions is provided, the computer instructions being used to cause a computer to perform the method according to any implementation manner of the first aspect.

[0009] According to a sixth aspect, a computer program product is provided, comprising: a computer program, the computer program implementing the method according to any implementation manner of the first aspect when executed by a processor.

[0010] According to the technology of the present disclosure, a sensitive data detection method and device based on an agent are provided, which utilize an artificial intelligence large model, determine a parsing strategy of a to-be-detected file according to a cognitive state of a data format of the to-be-detected file, and parse the to-be-detected file according to the parsing strategy to obtain target parsing data, thereby improving the coverage range of a parsing operation on a parsing scene, the accuracy and efficiency of a parsing process, and the comprehensiveness and accuracy of a sensitive data detection result.

[0011] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present disclosure, nor to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0012] The accompanying drawings are used to better understand the present scheme and do not limit the present disclosure. Among them: Figure 1 is an exemplary system architecture diagram to which an embodiment according to the present disclosure can be applied; Figure 2 is a flowchart of an embodiment of the sensitive data detection method based on an agent according to the present disclosure; Figure 3 is a schematic diagram of an application scenario of the sensitive data detection method based on an agent according to the present embodiment; Figure 4 is a flowchart of another embodiment of the sensitive data detection method based on an agent according to the present disclosure; Figure 5 is a structural diagram of an embodiment of the sensitive data detection device based on an agent according to the present disclosure; Figure 6 is a structural diagram of an embodiment of the agent for detecting sensitive data according to the present disclosure; Figure 7 is a structural schematic diagram of a computer system suitable for implementing embodiments of the present disclosure. DETAILED DESCRIPTION

[0013] Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, which include various details of the embodiments of the present disclosure to assist in understanding, which should be considered in a descriptive sense only. Thus, it will be apparent to one of ordinary skill in the art that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, for the sake of brevity and clarity, descriptions of well-known functions and constructions are omitted from the following description.

[0014] In the technical solutions of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved in the technical solutions comply with relevant laws and regulations and do not violate public order and good customs.

[0015] Figure 1 An exemplary architecture 100 of an agent-based sensitive data detection method and device to which the present disclosure can be applied is shown.

[0016] As shown in Figure 1 The system architecture 100 can include terminal devices 101, 102, 103, a network 104 and a server 105. The terminal devices 101, 102, 103 are communicatively connected to form a topological network, and the network 104 is used as a medium to provide communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 can include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0017] The terminal devices 101, 102, 103 can be hardware devices or software that support network connection for data interaction and data processing. When the terminal devices 101, 102, 103 are hardware, they can be various electronic devices that support network connection, information acquisition, interaction, display, processing, etc., including but not limited to smartphones, tablet computers, e-book readers, laptop computers and desktop computers, etc. When the terminal devices 101, 102, 103 are software, they can be installed in the above-mentioned electronic devices. They can be implemented as multiple software or software modules for providing distributed services, or as a single software or software module. No specific limitation is made herein.

[0018] Server 105 can be a server providing various services, such as a background processing server that receives files to be detected sent by the target object through terminal devices 101, 102, and 103, and detects whether the files contain sensitive data based on an intelligent agent. Optionally, the server can feed back the sensitive data detection results to the terminal devices. As an example, server 105 can be a cloud server.

[0019] It should be noted that a server can be either hardware or software. When the server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When the server is software, it can be implemented as multiple software programs or software modules (such as software programs or software modules used to provide distributed services), or as a single software program or software module. No specific limitations are made here.

[0020] It should also be noted that the sensitive data detection method based on intelligent agents provided in the embodiments of this disclosure is generally executed by a server, but the possibility of it being executed by a terminal device, or by the server and the terminal device cooperating with each other, is not excluded. Accordingly, the various parts (e.g., various units) included in the sensitive data detection device based on intelligent agents can be all set in the server, all set in the terminal device, or set in the server and the terminal device respectively.

[0021] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included. When the electronic device on which the agent-based sensitive data detection method runs does not need to transmit data with other electronic devices, the system architecture may only include the electronic device (e.g., a terminal device or server) on which the agent-based sensitive data detection method runs.

[0022] Please refer to Figure 2 , Figure 2 A flowchart illustrating a sensitive data detection method based on an intelligent agent, provided in this embodiment of the disclosure. Flowchart 200 includes the following steps: Step 201: Through the decision-making layer of the intelligent agent, the large artificial intelligence model is used to determine the parsing strategy of the file to be detected based on the cognitive state of the data format of the file to be detected.

[0023] In this embodiment, the executing entity of the sensitive data detection method based on the intelligent agent (e.g., Figure 1 The server in the system can obtain the file to be detected from a remote location or from a local location via a wired or wireless network connection. Then, through the decision-making layer of the intelligent agent, it uses a large artificial intelligence model to determine the parsing strategy of the file to be detected based on its cognitive state of the data format of the file.

[0024] The file to be detected is a file to be detected for sensitive data, which is sent to the above-mentioned execution subject by a target object, such as a person, other intelligent devices, artificial intelligence assistants, and the like. Sensitive data refers to information that may cause harm to the legitimate rights and interests of individuals, organizations, enterprises, or countries (such as privacy leakage, property loss, reputation damage, security risk, etc.) if it is accessed, used, disclosed, modified, or destroyed without authorization. Such data usually has the characteristics of privacy, confidentiality, or sensitivity, and its protection must comply with relevant laws and regulations (such as the Personal Information Protection Law, the Data Security Law, etc.) and industry standards. Sensitive data can cover multiple dimensions, such as: Personal sensitive data: ID number, bank account number, biometric information (fingerprint, facial data), medical records, location trajectory, etc. Enterprise sensitive data: business secrets (such as core technology solutions, customer lists), financial statements, unpublished strategic plans, etc. National sensitive data: classified information, critical infrastructure operation data, national defense-related data, etc.

[0025] The above-mentioned execution subject is deployed with an intelligent agent for detecting sensitive data, and the decision layer in the intelligent agent can call an artificial intelligence large model to determine the analysis strategy of the file to be detected according to the cognitive state of the artificial intelligence large model for the data format of the file to be detected.

[0026] An artificial intelligence large model (referred to as a "large model") is a type of artificial intelligence model constructed by artificial neural networks with a large number of parameters, including large language models, visual large models, multi-modal large models, and basic scientific large models. In the present embodiment, the artificial intelligence large model is a large language model, which is a large-scale language model based on deep learning technology, mainly used for processing natural language processing tasks. It learns the patterns and structures of language by training on large-scale data and can generate natural language text or understand natural language input. The present embodiment specifically uses a multi-modal large language model, which usually includes the following modules: Input module: receives the file to be detected input by the user, which can be at least one of text, image, video, and voice.

[0027] Preprocessing module: pre-processes the input file to be detected.

[0028] Encoding module: encodes the pre-processed multi-modal data into vector form so that the model can understand and process. Common encoding methods include word embedding and encoders in Transformer architecture. Among them, word embedding includes, for example, Word2Vec (Words to Vector), GloVe (Global Vectors for Word Representation).

[0029] Model module: the core part, usually based on deep learning architecture (such as Transformer), responsible for processing encoded data vectors, language understanding and generation. The model learns the complex patterns and semantic relationships of language through multi-layer neural network structure.

[0030] Decoding module: decodes the output vector of the model into natural language text, image, video, voice, generates a reply or processing result to the user input. Decoding methods can include greedy decoding, beam search, etc.

[0031] Output module: outputs the decoded text in a user-readable form.

[0032] Data format includes encoding rules (such as character encoding, compression algorithm, encryption method, etc.) and data structure (such as field distribution, hierarchical relationship, separation identifier, etc.). The cognitive state of artificial intelligence large model to the data format of the file to be detected includes known and unknown. In the known state, the knowledge base of the artificial intelligence large model includes the data format followed by the file to be detected, there is an explicit and reusable parsing strategy (parsing rule), and no additional inference or guess is needed for the file to be detected. The file to be detected in the known state includes, for example: 1. General standard format, such as JSON (JavaScript Object Notation), XML (Extensible Markup Language), PNG (Portable Network Graphics), H.264, etc., whose encoding rules and structure definitions are public, and the parsing strategy is, for example, to parse according to the standard parsing process corresponding to the data format.

[0033] 2. Industry-standardized private protocol, such as DICOM (Digital Imaging and Communications in Medicine) format in the medical field (if the system has mastered the meaning of its metadata fields).

[0034] The core of its analysis process is to follow the format specification "structured parsing + verification" process, as follows: First, read the binary stream of the file to be detected, check if it conforms to the basic specification of the target format (such as JSON needs to verify the matching of parentheses, the closing of quotes; XML needs to check the legality of tag nesting; PNG needs to verify the file header magic number "89 50 4E 47" and CRC check), if not, return format error.

[0035] Then, according to the syntax rules of the format, the data is disassembled. For JSON parsing library, according to the syntax of "key-value pair", "array", "nested object", etc., the text stream is converted into structured data such as dictionary, list, etc. in memory (such as Python's json.loads()); For XML parser, according to the structure of "tag level", "attribute", "text node", etc., build DOM (Document Object Model, Document Object Model) tree or SAX (Simple API for XML, XML Simple Application Program Interface) event stream, map the parent-child relationship between tags (such as Java's DocumentBuilder); For PNG decoder: according to the block structure of "file header -> IHDR (Image Header) block (image size / format) -> IDAT (Image Data) block (compressed pixel data) -> IEND (Image End) block", split and parse the metadata and payload of each block.

[0036] Then, the parsed structured data is converted into a form that can be directly used by the application. For JSON / XML data, extract specific field values (such as extracting id as 123 from <user> <id> 123< / id> < / user> ), process data types (such as converting JSON numbers and boolean values to corresponding programming language types); for PNG data, decompress the compressed data (using Deflate algorithm) in the IDAT block, combine the bit depth and color type in the IHDR, and restore it to an RGB / A pixel matrix for image rendering.

[0037] Finally, output the parsed structured data (such as memory object, pixel array), and handle the exceptions that may occur during parsing (such as JSON syntax error, XML unclosed tag, PNG block verification failure), return specific error information to assist debugging.

[0038] In the unknown state, the knowledge base of the artificial intelligence large model does not include the data format followed by the file to be detected, there is no ready-made parsing strategy, and the data format information such as data structure and coding logic needs to be dynamically explored.

[0039] The unknown state to be detected file includes, for example: 1. A private protocol standardized in the industry, but the data format knowledge thereof is not included in the knowledge base, or the flexibility of the data contained therein cannot clearly know which encoding format the data includes, for example, a self-defined channel MF4 (Measurement Data Format 4).

[0040] 2. A completely self-defined private protocol, such as a sensor data format (unpublished structure definition) self-developed in an enterprise, a special file (encoding rule is not transparent) encrypted or obfuscated.

[0041] 3. A mixed format file, such as a file containing multiple unlabeled sub-protocols (such as a binary stream containing unknown encoded text and image data, and no explicit separation identifier).

[0042] For the unknown data format to be detected file, the artificial intelligence large model formulates an analysis strategy for the to-be-detected file according to the file characteristics of the to-be-detected file. For example, first, the file is segmented into fixed-size segments, and the highest frequency byte sequence of each segment is extracted as a candidate separator. The file is split using the separator, and if the length of the sub-blocks is similar, the byte distribution in the sub-blocks is counted, and if a certain type of sub-block contains continuous 0x00, it is presumed to be a padding field, and the remaining part is analyzed after being removed. For segments without obvious separators, use a sliding window (such as 100 bytes) to scan, record the special values (such as 0xFF, 0x0A) in the window, associate the known format marker bits (such as image frame start), and use the corresponding format analysis logic to try to analyze. If the trial analysis fails, adjust the window size and scan again, and combine the time sequence relationship (such as increasing numerical value) between sub-blocks to infer the data dimension, and gradually optimize the analysis strategy.

[0043] In some optional implementations of the embodiment, the above-mentioned execution subject can execute the above-mentioned step 201 by: in response to the cognitive state of the artificial intelligence large model to the to-be-detected file being unknown, using the artificial intelligence large model to determine the analysis strategy according to the file type of the to-be-detected file.

[0044] In response to the cognitive state of the artificial intelligence large model to the to-be-detected file being unknown, using the artificial intelligence large model to determine the data storage mode of the to-be-detected file according to the file type of the to-be-detected file; and determining the analysis strategy according to the data storage mode.

[0045] Taking an MF4 file as an example, the data storage mode thereof is that data is stored according to channels (Channel), and each channel represents a physical quantity or an ECU (Electronic Control Unit) signal, such as engine speed, vehicle speed, temperature, and the like. According to the channel storage mode, the analysis strategy thereof can be determined as channel-by-channel analysis with a channel as an analysis unit.

[0046] Taking a PCAP (Packet Capture Library) file as an example, the sequential storage mode based on data packets is adopted. According to the data packet storage mode, the analysis strategy thereof can be determined as data packet-by-data packet analysis with a data packet as an analysis unit.

[0047] In the implementation mode, for a to-be-detected file whose data format is unknown, the corresponding analysis strategy is determined according to the file type, which improves the accuracy of the analysis strategy and the degree of adaptation to the to-be-detected file, and helps to improve the file analysis speed and accuracy.

[0048] In step 202, the to-be-detected file is analyzed according to the analysis strategy through the decision layer, and target analysis data is obtained.

[0049] In the embodiment, the above execution subject can analyze the to-be-detected file according to the analysis strategy through the decision layer in the intelligent agent, and obtain target analysis data.

[0050] For a to-be-detected file whose data format is known, the to-be-detected file is analyzed according to the “structured analysis + verification” process complying with the format specification.

[0051] For a to-be-detected file whose data format is unknown, the to-be-detected file is analyzed according to the analysis strategy specified by the artificial intelligence large model. For example, the file is first segmented into fixed-size segments, and the byte sequence with the highest frequency in each segment is extracted as a candidate delimiter. The file is split by the delimiter, and if sub-blocks with similar lengths are obtained, the byte distribution in the sub-blocks is counted, and if a certain type of sub-block contains continuous 0x00, it is presumed to be a padding field, and the remaining part is analyzed after being removed. For a segment without obvious delimiters, a sliding window (such as 100 bytes) is used for scanning, and special values (such as 0xFF and 0x0A) appearing in the window are recorded. The known format marker bits (such as the start of a picture frame) are associated, and the corresponding format analysis logic is used for trial analysis. If the trial analysis fails, the window size is adjusted for re-scanning, and the data dimension is inferred from the time sequence relationship (such as an increasing numerical value) between the sub-blocks, and the analysis strategy is gradually optimized.

[0052] In some optional implementation modes of the embodiment, the above execution subject can execute the above step 202 in the following manner: In a first step, the analysis unit in the to-be-detected file is determined according to the analysis strategy.

[0053] A parsing unit refers to a basic unit in a file parsing process, which has independent metadata (such as type, format, and range), can be independently located and processed, and does not depend on the complete parsing of other units. It generally has clear boundaries (such as data start / end position, identification field), contains independent format information (such as encoding method and data structure), and can be parsed as an independent step (without waiting for the parsing results of other units).

[0054] Taking the MF4 file as an example, the parsing strategy is to parse each channel as a parsing unit, and the parsing unit in the file to be detected is determined as a channel.

[0055] Taking the PCAP file as an example, the parsing strategy is to parse each data packet as a parsing unit, and the parsing unit in the file to be detected is determined as a data packet.

[0056] In the second step, the encoding format of the parsing unit is determined according to the data characteristics of the parsing unit, and the parsing unit is parsed according to the encoding format to obtain unit parsing data.

[0057] For each parsing unit, the encoding format of the parsing unit is determined according to the data characteristics of the parsing unit, and the parsing unit is parsed according to the encoding format to obtain unit parsing data.

[0058] The file to be detected is generally stored in binary, and the binary characteristics of the parsing unit are extracted first, such as the first 16 bytes of the file header, the overall byte entropy value (to determine whether it is compressed or encrypted), and the repeated byte pattern. The file header is compared with the known format signature library, and if it matches the approximate format (such as 6 bytes of the PNG file header), the corresponding format parsing tool is called for trial parsing, and if a logical structure (such as image size and pixel value) is parsed, the strategy is followed.

[0059] In the third step, the target parsing data is obtained by combining the unit parsing data of each parsing unit.

[0060] For each parsing unit, the execution subject may successfully parse the parsing unit to obtain the unit parsing data of the parsing unit, or the execution subject may fail to parse the parsing unit to obtain the unit parsing data of the parsing unit.

[0061] For each successfully parsed parsing unit, the target parsing data is obtained by combining the unit parsing data of each parsing unit.

[0062] In this implementation, a way of parsing a file to be detected based on a parsing strategy is provided, each parsing unit is tried to be parsed according to the parsing strategy, and the coverage of the parsing operation on the parsing scene, and the accuracy and efficiency of the parsing process are further improved.

[0063] In some optional implementations of the embodiment, the execution subject can perform the second step in the following manner: The following parsing operation is iteratively performed until a preset ending condition is reached: First, the encoding format of the parsing unit is determined according to the data characteristics of the parsing unit; then, the parsing unit is parsed according to the encoding format to obtain unit parsing data; and then, in response to the unit parsing data not being in the target data modality, the next parsing operation is performed.

[0064] As an example, for each parsing unit in the file to be detected, first, the encoding format of the parsing unit is predicted according to the data characteristics of the parsing unit; then, the corresponding parser is determined according to the encoding format, and the parsing unit is parsed to obtain unit parsing data; then, it is determined whether the unit parsing data is in the target data modality, and the data in the target data modality is generally human-readable data, such as text, image, etc. Target data modality. When the unit parsing data is in the form of human-unreadable data, such as garbled code, it indicates that the parsing process for the parsing unit fails. Finally, the next parsing operation is performed, that is, the encoding format of the parsing unit is continuously predicted according to the data characteristics of the parsing unit.

[0065] The preset ending condition is, for example, that a parsing unit is successfully parsed, that is, the unit parsing data is in the target data modality, or that the possible encoding format of the parsing unit predicted based on the data characteristics has been tried and the parsing unit is still not successfully parsed.

[0066] In the implementation, the unit parsing data obtained by the parsing operation when the preset ending condition is reached can be used as the unit parsing data corresponding to the parsing unit.

[0067] In the implementation, a way of trying to parse the parsing unit in the file to be detected based on a loop is provided, which helps to improve the success rate of the parsing process and thus helps to improve the comprehensiveness of the sensitive data detection result.

[0068] In some optional implementations of the embodiment, the execution subject can perform the second step in the following manner: using an artificial intelligence large model, determining the encoding format of the parsing unit according to the file characteristics of the parsing unit, and parsing the parsing unit according to the encoding format to obtain unit parsing data.

[0069] Specifically, using an artificial intelligence large model, the following parsing operation is iteratively performed until a preset ending condition is reached: first, the encoding format of the parsing unit is determined according to the data characteristics of the parsing unit; then, the parsing unit is parsed according to the encoding format to obtain unit parsing data; and then, in response to the unit parsing data not being in the target data modality, the next parsing operation is performed.

[0070] In this implementation, the artificial intelligence large model serves as a determinator of the coding format, a multi-modal parser of the analysis unit, a determinator of whether the analysis is successful, and iteratively performs the analysis process of the analysis unit. With the powerful logical analysis capability and data processing capability of the artificial intelligence large model, the success rate and efficiency of the analysis operation can be further improved.

[0071] In some optional implementations of the present embodiment, in response to the cognitive state of the artificial intelligence large model for the to-be-detected file being known, the execution subject can perform the step 202 by using a standard parser corresponding to the data format to analyze the to-be-detected file to obtain the target analysis data.

[0072] First, according to the data format of the to-be-detected file, a standard parser for analyzing the to-be-detected file is used to analyze the to-be-detected file to obtain the target analysis data.

[0073] For example, for a JSON format file, a JSON parsing library is used to convert the text stream into structured data such as dictionaries, lists, etc. in memory according to the syntax of "key-value pair", "array", "nested object" (such as Python's json.loads ()); for an XML format file, an XML parser is used to construct a DOM tree or SAX event stream according to the structure of "tag level", "attribute", "text node", and map the parent-child relationship between tags (such as Java's DocumentBuilder); for a PNG format file, a PNG decoder is used to split and analyze the metadata and payload of each block according to the block structure of "file header -> IHDR block (image size / format) -> IDAT block (compressed pixel data) -> IEND block".

[0074] In this implementation, for the to-be-detected file whose data format is known, the decoding strategy is to use a standard parser corresponding to the data format to analyze the to-be-detected file, which helps to further improve the analysis efficiency and success rate of the to-be-detected file.

[0075] Step 203: Detect whether the target analysis data includes sensitive data through the execution layer of the intelligent agent to obtain a sensitive data detection result.

[0076] In the present embodiment, the execution subject can detect whether the target analysis data includes sensitive data through the execution layer of the intelligent agent to obtain a sensitive data detection result.

[0077] As an example, the target resolution data can be compared with sensitive data in the sensitive database, and in response to the similarity between the target resolution data and the sensitive data being higher than a preset similarity threshold, it is indicated that the sensitive data is included in the target resolution data; otherwise, it is indicated that the sensitive data is not included in the target resolution data, and finally a sensitive data detection result is obtained.

[0078] In some optional implementations of the embodiment, the execution subject can execute the step 203 in the following manner: First, determine a target tool chain for detecting whether the sensitive data is included in the target resolution data according to the target data modality to which the target resolution data belongs.

[0079] The target tool chain includes at least one tool, and the at least one tool executes a detection task of the sensitive data in sequence. For each data modality, a corresponding target tool chain can be preset, so that after determining the target data modality to which the target resolution data belongs, the target tool chain for detecting whether the sensitive data is included in the target resolution data is determined according to the preset corresponding relationship.

[0080] For example, for the text modality, the corresponding target tool chain includes a text preprocessing tool and a text scanner. The text preprocessing tool is used for preprocessing the text data such as word segmentation, and the text scanner is used for determining whether the preprocessed text data includes the sensitive data.

[0081] For example, for the image modality, the corresponding target tool chain includes an OCR (Optical Character Recognition) engine, a text preprocessing tool, a text scanner, and an image processing engine. The OCR engine is used for recognizing text in image data, and the image processing engine is used for understanding image content.

[0082] Second, use the tools in the target tool chain to detect whether the sensitive data is included in the target resolution data, and obtain a sensitive data detection result.

[0083] Each tool in the target tool chain is called in sequence to perform data processing with the data output by the associated previous tool as input, and finally a sensitive data detection task is completed to obtain a sensitive data detection result.

[0084] In some implementations, the tools in the tool chain can be implemented based on an artificial intelligence large model. As an example, different tools are implemented based on an artificial intelligence large model using preset prompt words corresponding to different functional tools. For example, the preset prompt word is "you are an OCR engine, please recognize the text in the image", and the preset prompt word is input into the artificial intelligence large model to instruct the artificial intelligence large model to run as an OCR engine.

[0085] In this implementation, based on the target data modality to which the target analysis data belongs, the target tool chain for detecting whether the sensitive data is included in the target analysis data is determined, so as to perform the sensitive data detection task of the analysis data in a targeted manner, which helps to improve the accuracy and determination efficiency of the sensitive data detection result.

[0086] In some optional implementations of the embodiment, the target analysis data includes data of multiple target data modalities. For example, the target analysis data includes image data and text data.

[0087] In this implementation, the execution subject can perform the second step by the following manner to determine the sensitive data detection result: First, for the data of multiple target data modalities associated in the target analysis data, the tools in the target tool chain corresponding to the target data modality are used to detect whether the sensitive data is included in the data of the target data modality, and an initial detection result is obtained.

[0088] In this implementation, the data of different modalities with the same timestamp or adjacent timestamp in the to-be-detected file can be used as the associated data; the data of different modalities with semantic relationship can be used as the associated data, for example, the image content and the text content in the image modality data are used as the associated data.

[0089] For the data of multiple target data modalities associated, the tools in the target tool chain corresponding to the target data modality are used to detect whether the sensitive data is included in the data of the target data modality, and an initial detection result and a result confidence are obtained.

[0090] Then, the initial detection results of the multiple target data modalities are cross-verified to obtain the sensitive data detection results corresponding to the multiple target data modalities respectively.

[0091] The initial detection results of the multiple target data modalities are cross-verified according to the confidence thereof. For example, in response to the fact that the sensitive data is included in the text modality data and the confidence is high, and the sensitive data is included in the associated image data and the confidence is high, it is determined that the sensitive data detection results corresponding to the multiple target data modalities respectively are all the sensitive data.

[0092] In some implementations, in order to further improve the accuracy of the cross-verification result, the initial detection results of the multiple target data modalities are cross-verified according to the types of the sensitive data therein, for example, the sensitive data included in the image data and the text data represents the same position and other sensitive entities, and the sensitive data detection results corresponding to the multiple target data modalities respectively are obtained, so as to ensure the association of the sensitive data in the associated different modalities.

[0093] In this implementation, cross-validation based on detection results from different modal data improves the accuracy of the final sensitive data detection results.

[0094] In some implementations, in response to the sensitive data detection results indicating the presence of sensitive data in the data to be detected, a data anonymization method is determined based on the data type of the sensitive data; the sensitive data is then processed according to the data anonymization method to obtain the anonymized data. For example, for video data, the data anonymization method is to replace the video frame containing the sensitive data with a black frame; for text data, sensitive words and other sensitive data parts are deleted.

[0095] See also Figure 3 , Figure 3 This is a schematic diagram 300 illustrating an application scenario of the agent-based sensitive data detection method according to this embodiment. First, an agent 302 is deployed in the server 301. The agent includes a perception layer 3021, a decision layer 3022, and an execution layer 3023. The target object 303 sends the file to be detected to the server through the corresponding terminal device 304. The server, through the agent's decision layer, utilizes a large-scale artificial intelligence model to determine the parsing strategy for the file to be detected based on its understanding of the file's data format. The decision layer then parses the file according to the parsing strategy to obtain the target parsed data. Finally, the agent's execution layer detects whether the target parsed data contains sensitive data, thus obtaining the sensitive data detection result.

[0096] This embodiment provides a sensitive data detection method based on an intelligent agent. By utilizing a large artificial intelligence model, the method determines the parsing strategy for the file to be detected based on the cognitive state of the data format of the file to be detected, and parses the file to be detected according to the parsing strategy to obtain the target parsed data. This improves the coverage of the parsing operation on the parsing scenario, as well as the accuracy and efficiency of the parsing process, thereby improving the comprehensiveness and accuracy of the sensitive data detection results.

[0097] In some optional implementations of this embodiment, the execution entity may also perform the following operations: through the feedback and learning layer in the intelligent agent, standardize the processing of the file to be detected for unknown data formats, and generate a standard parser corresponding to the data format of the file to be detected.

[0098] As an example, firstly, after completing the iterative parsing of the unknown data format file, the agent automatically extracts all valid operation trajectories during this parsing process, including successful encoding format predictions (such as encoding method, field separator), called toolchains (such as specific decoders, feature extractors), parameter configurations (such as byte order, block size), and manual feedback correction records, forming the original parsing rule set.

[0099] Then, the original rule set is structurally abstracted, separating "encoding layer rules" (such as compression algorithms and character sets) and "structural layer rules" (such as field length and hierarchical nesting relationships). Through feature tagging tools, core features such as file header signatures and key delimiters are bound to the rules to form a preliminary format description template.

[0100] Then, the format description template is validated using multiple unknown files of the same type within the same domain (those not involved in the initial parsing). If the parsing success rate exceeds a preset threshold, the core rules of the template are retained. If there are failed cases, the rules that are not covered (such as special field variants) are located by comparing differences and added to the template to form a candidate parser rule base.

[0101] Then, a standardized parser code framework is generated based on the candidate rule base, encapsulating input and output interfaces (such as receiving binary streams and outputting structured dictionaries), integrating tool calling modules (such as automatically associating with the corresponding decoders), and embedding error handling logic (such as a fallback strategy when field parsing fails) to ensure that the standard parser can be directly called by the agent.

[0102] Finally, the newly generated standard parser is incorporated into the system knowledge base, and dynamic optimization trigger conditions are set. For example, when multiple new errors occur in parsing similar files, the parsing rules of the new cases are automatically extracted and merged with the existing parser rules to achieve continuous iteration of the parser and gradually form a standardized parsing capability covering a class of unknown formats.

[0103] The feedback and learning layer is a functional layer independent of the perception, decision-making, and execution layers, but it relies on the outputs of these three layers to achieve its core function, forming a closed-loop collaborative relationship. The feedback and learning layer does not directly participate in data perception (perception layer function), strategy formulation (decision layer function), or operation execution (execution layer function). Instead, it independently undertakes the function of "generating a standard parser based on the normalization of collected data," possessing its own rule base and learning module. However, the operation of the feedback and learning layer depends on the perception, decision-making, and execution layers. The data it collects includes the data to be processed determined by the perception layer, historical parsing process data from the decision-making layer, and operational process data from the execution layer (such as execution process data based on the toolchain). Through the aforementioned normalization process, the feedback and learning layer extracts the standard parser, which is then fed back to the decision-making layer to guide the iteration of subsequent parsing strategies, ultimately achieving the autonomous evolution of the entire system's parsing capabilities.

[0104] In this implementation, based on the standardized processing of files to be detected with unknown data formats, a standard parser corresponding to the data format of the file to be detected is generated. This gradually enriches the understanding of the data formats and the types of standard parsers, which helps to improve the efficiency and accuracy of the agent in the subsequent parsing process.

[0105] In some optional implementations of this embodiment, the execution entity may also perform the following operations: First, for the remaining data in the file to be tested that failed to be parsed, the remaining data is parsed according to the received parsing operation to obtain auxiliary parsing data.

[0106] The parsing operation can be a target object, such as an experienced technician, to assist in parsing the remaining data and obtain auxiliary parsing data.

[0107] Then, based on the target data modality to which the auxiliary parsing data belongs, an auxiliary toolchain is determined for detecting whether the auxiliary parsing data includes sensitive data.

[0108] Finally, tools from the auxiliary toolchain are used to detect whether the auxiliary parsing data includes sensitive data.

[0109] In this implementation, you can refer to the relevant examples of the target toolchain mentioned above.

[0110] Furthermore, for data from multiple target data modalities associated with the auxiliary parsing data, tools from the auxiliary toolchain corresponding to the target data modal are used to detect whether the data of the target data modal includes sensitive data, and initial detection results are obtained. Cross-validation is then performed on the initial detection results of each of the multiple target data modalities to obtain the sensitive data detection results corresponding to each of the multiple target data modalities.

[0111] In this implementation, the remaining data in the file to be detected that failed to be parsed is parsed based on the received parsing operation, which improves the comprehensiveness and flexibility of the parsing process.

[0112] In some optional implementations of this embodiment, the execution entity may also perform the following operations: through the feedback and learning layer in the agent, standardize the processing of the remaining data and generate a standard parser corresponding to the data format of the remaining data.

[0113] As an example, firstly, based on the manual review results and parsing logs during the processing of the remaining data, the agent automatically extracts all valid operation trajectories during this parsing process, including successful encoding format predictions (such as encoding method, field separator), called toolchains (such as specific decoders, feature extractors), parameter configurations (such as byte order, block size), and manual feedback correction records, forming the original parsing rule set.

[0114] Then, the original rule set is structurally abstracted, separating "encoding layer rules" (such as compression algorithms and character sets) and "structural layer rules" (such as field length and hierarchical nesting relationships). Through feature tagging tools, core features such as file header signatures and key delimiters are bound to the rules to form a preliminary format description template.

[0115] Then, the template is validated using multiple unknown files of the same type within the same domain (those not involved in the initial parsing). If the parsing success rate exceeds a preset threshold, the core rules of the template are retained. If there are failed cases, the rules that are not covered (such as special field variants) are located by comparing differences and added to the template to form a candidate parser rule base.

[0116] Then, a standardized parser code framework is generated based on the candidate rule base, encapsulating input and output interfaces (such as receiving binary streams and outputting structured dictionaries), integrating tool calling modules (such as automatically associating with the corresponding decoders), and embedding error handling logic (such as a fallback strategy when field parsing fails) to ensure that the standard parser can be directly called by the agent.

[0117] Finally, the newly generated standard parser is incorporated into the system knowledge base, and dynamic optimization trigger conditions are set. For example, when multiple new errors occur in parsing similar files, the parsing rules of the new cases are automatically extracted and merged with the existing parser rules to achieve continuous iteration of the parser and gradually form a standardized parsing capability covering a class of unknown formats.

[0118] In this implementation, based on the standardized processing of remaining data for unknown data formats, a standard parser corresponding to the data format of the file to be detected is generated. This gradually enriches the understanding of the data format and the standard parser, which helps to improve the efficiency and accuracy of the agent in the subsequent parsing process.

[0119] Continue to refer to Figure 4 This illustrates an illustrative flow 400 of yet another embodiment of the agent-based sensitive data detection method according to the present disclosure. Flow 400 includes the following steps: Step 401: In response to the fact that the AI ​​model's understanding of the data format of the file to be detected is unknown, the AI ​​model is used to determine the parsing strategy for the file to be detected based on its file type.

[0120] Step 402: Determine the parsing units in the file to be detected according to the parsing strategy.

[0121] Step 403: Using a large-scale artificial intelligence model, iteratively execute the following parsing operations until the preset termination condition is met: Step 4031: Determine the encoding format of the parsing unit based on its data characteristics; Step 4032: Parse the parsing unit according to the encoding format to obtain the unit parsing data; Step 4033: In response to the fact that the parsed data of the cell is not presented as the target data modality, perform the next parsing operation.

[0122] Step 404: Combine the unit parsing data of each parsing unit to obtain the target parsing data.

[0123] Step 405: Determine the target toolchain for detecting whether the target parsing data includes sensitive data, based on the target data modality to which the target parsing data belongs.

[0124] Step 406: Use tools from the target toolchain to detect whether the target parsed data includes sensitive data, and obtain the sensitive data detection results.

[0125] Step 407: Standardize the processing procedure for files to be tested with unknown data formats, and generate a standard parser corresponding to the data format of the files to be tested.

[0126] The process 400 of the agent-based sensitive data detection method in this embodiment specifically illustrates the parsing process of the file to be detected, the detection process of sensitive data, and the generation process of the standard parser. This improves the coverage of the parsing operation on the parsing scenario, as well as the accuracy and efficiency of the parsing process, thereby improving the comprehensiveness and accuracy of the sensitive data detection results.

[0127] Continue to refer to Figure 5 As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of a sensitive data detection device based on an intelligent agent. This system embodiment is similar to... Figure 2 Corresponding to the method embodiments shown, the system can be specifically applied to various electronic devices.

[0128] like Figure 5 As shown, the sensitive data detection device 500 based on intelligent agents includes: a strategy determination unit 501, configured to determine the parsing strategy of the file to be detected by using an artificial intelligence big data model through the decision layer of the intelligent agent, based on the cognitive state of the data format of the file to be detected; a data parsing unit 502, configured to parse the file to be detected according to the parsing strategy through the decision layer to obtain target parsed data; and a data detection unit 503, configured to detect whether the target parsed data contains sensitive data through the execution layer of the intelligent agent to obtain a sensitive data detection result.

[0129] In some optional implementations of this embodiment, the strategy determination unit 501 is further configured to: in response to the fact that the cognitive state of the artificial intelligence big model of the file to be detected is unknown, use the artificial intelligence big model to determine the parsing strategy according to the file type of the file to be detected.

[0130] In some optional implementations of this embodiment, the data parsing unit 502 is further configured to: determine the parsing units in the file to be detected according to the parsing strategy; determine the encoding format of the parsing units according to the data characteristics of the parsing units, and parse the parsing units according to the encoding format to obtain unit parsing data; and combine the unit parsing data of each parsing unit to obtain target parsing data.

[0131] In some optional implementations of this embodiment, the data parsing unit 502 is further configured to iteratively execute the following parsing operations until a preset termination condition is met: determine the encoding format of the parsing unit based on the data characteristics of the parsing unit; parse the parsing unit according to the encoding format to obtain the parsed data of the unit; and execute the next parsing operation in response to the fact that the parsed data of the unit does not present as the target data modality.

[0132] In some optional implementations of this embodiment, the data parsing unit 502 is further configured to: use an artificial intelligence big data model to determine the encoding format of the parsing unit based on the file characteristics of the parsing unit, and parse the parsing unit according to the encoding format to obtain the parsed data of the unit.

[0133] In some optional implementations of this embodiment, the data parsing unit 502 is further configured to: in response to the cognitive state being known, use a standard parser corresponding to the data format to parse the file to be detected and obtain the target parsing data.

[0134] In some optional implementations of this embodiment, the data detection unit 503 is further configured to: determine a target toolchain for detecting whether the target parsing data includes sensitive data based on the target data modality to which the target parsing data belongs; use the tools in the target toolchain to detect whether the target parsing data includes sensitive data, and obtain a sensitive data detection result.

[0135] In some optional implementations of this embodiment, the target parsing data includes data of multiple target data modalities, and the data detection unit 503 is further configured to: for the data of multiple target data modalities associated in the target parsing data, use the tools in the target toolchain corresponding to the target data modal to detect whether the data of the target data modal includes sensitive data, and obtain an initial detection result; perform cross-validation on the initial detection results of each of the multiple target data modalities to obtain the sensitive data detection results corresponding to each of the multiple target data modalities.

[0136] In some optional implementations of this embodiment, the above apparatus further includes: a feedback learning unit (not shown in the figure), configured to: standardize the processing of a file to be detected with an unknown data format through the feedback and learning layer in the agent, and generate a standard parser corresponding to the data format of the file to be detected.

[0137] In some optional implementations of this embodiment, the above apparatus further includes: an auxiliary parsing unit (not shown in the figure), configured to parse the remaining data in the file to be detected that failed to be parsed, according to the received parsing operation, to obtain auxiliary parsing data; and the data detection unit 503 is further configured to: determine an auxiliary toolchain for detecting whether the auxiliary parsing data includes sensitive data according to the target data modality to which the auxiliary parsing data belongs; and use the tools in the auxiliary toolchain to detect whether the auxiliary parsing data includes sensitive data.

[0138] In some optional implementations of this embodiment, the feedback learning unit (not shown in the figure) is further configured to: standardize the processing of the remaining data through the feedback and learning layer in the agent, and generate a standard parser corresponding to the data format of the remaining data.

[0139] This embodiment provides a sensitive data detection device based on an intelligent agent. By utilizing a large artificial intelligence model, the device determines the parsing strategy for the file to be detected based on the cognitive state of the data format of the file to be detected, and parses the file to be detected according to the parsing strategy to obtain the target parsed data. This improves the coverage of the parsing operation on the parsing scenario, as well as the accuracy and efficiency of the parsing process, thereby improving the comprehensiveness and accuracy of the sensitive data detection results.

[0140] Continue to refer to Figure 6 This disclosure provides an embodiment of an intelligent agent for detecting sensitive data, and the system embodiment is similar to... Figure 2 Corresponding to the method embodiments shown, this intelligent agent can be specifically applied to various electronic devices.

[0141] like Figure 6 As shown, an intelligent agent for detecting sensitive data includes: a perception layer 601, used to determine the file to be detected; a decision layer 602, used to use an artificial intelligence big data model to determine the parsing strategy of the file to be detected based on the cognitive state of the data format of the file to be detected, and to parse the file to be detected according to the parsing strategy to obtain the target parsed data; and an execution layer 603, used to detect whether the target parsed data includes sensitive data and to obtain the sensitive data detection result.

[0142] In some optional implementations of this embodiment, the decision layer 602 is further used to: in response to the fact that the cognitive state of the artificial intelligence big model of the file to be detected is unknown, to use the artificial intelligence big model to determine the parsing strategy according to the file type of the file to be detected.

[0143] In some optional implementations of this embodiment, the decision layer 602 is further used to: determine the parsing units in the file to be detected according to the parsing strategy; determine the encoding format of the parsing units according to the data characteristics of the parsing units, and parse the parsing units according to the encoding format to obtain unit parsing data; and combine the unit parsing data of each parsing unit to obtain target parsing data.

[0144] In some optional implementations of this embodiment, the decision layer 602 is further used to: iteratively execute the following parsing operations until a preset termination condition is met: determine the encoding format of the parsing unit based on the data characteristics of the parsing unit; parse the parsing unit according to the encoding format to obtain the unit parsing data; and execute the next parsing operation in response to the unit parsing data not being presented as the target data modality.

[0145] In some optional implementations of this embodiment, the decision layer 602 is further used to: utilize a large artificial intelligence model to determine the encoding format of the parsing unit based on the file characteristics of the parsing unit, and parse the parsing unit according to the encoding format to obtain the unit parsing data.

[0146] In some optional implementations of this embodiment, the decision layer 602 is further used to: in response to the known cognitive state, use a standard parser corresponding to the data format to parse the file to be detected and obtain the target parsing data.

[0147] In some optional implementations of this embodiment, the execution layer 603 is further used to: determine a target toolchain for detecting whether the target parsed data includes sensitive data based on the target data modality to which the target parsed data belongs; use the tools in the target toolchain to detect whether the target parsed data includes sensitive data, and obtain a sensitive data detection result.

[0148] In some optional implementations of this embodiment, the target parsing data includes data of multiple target data modalities, and the execution layer 603 is further used to: for the data of multiple target data modalities associated in the target parsing data, use the tools in the target toolchain corresponding to the target data modal to detect whether the data of the target data modal includes sensitive data, and obtain an initial detection result; perform cross-validation on the initial detection results of each of the multiple target data modalities to obtain the sensitive data detection results corresponding to each of the multiple target data modalities.

[0149] In some optional implementations of this embodiment, the intelligent agent further includes a feedback learning layer (not shown in the figure), which is used to: standardize the processing of the file to be detected for unknown data formats through the feedback and learning layer in the intelligent agent, and generate a standard parser corresponding to the data format of the file to be detected.

[0150] In some optional implementations of this embodiment, the decision layer 602 is further configured to: an auxiliary parsing unit (not shown in the figure) is configured to parse the remaining data in the file to be detected that failed to be parsed, according to the received parsing operation, to obtain auxiliary parsing data; and the execution layer 603 is further configured to: determine an auxiliary toolchain for detecting whether the auxiliary parsing data includes sensitive data according to the target data modality to which the auxiliary parsing data belongs; and use the tools in the auxiliary toolchain to detect whether the auxiliary parsing data includes sensitive data.

[0151] In some optional implementations of this embodiment, the feedback learning layer (not shown in the figure) is used to: standardize the processing of the remaining data through the feedback and learning layer in the agent, and generate a standard parser corresponding to the data format of the remaining data.

[0152] In this embodiment, an intelligent agent for detecting sensitive data is provided. Utilizing a large artificial intelligence model, the agent determines the parsing strategy for the file to be detected based on its cognitive state of the data format of the file to be detected, and parses the file to be detected according to the parsing strategy to obtain the target parsed data. This improves the coverage of the parsing operation for the parsing scenario, as well as the accuracy and efficiency of the parsing process, thereby improving the comprehensiveness and accuracy of the sensitive data detection results.

[0153] According to embodiments of this disclosure, this disclosure also provides an electronic device, the electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to implement the agent-based sensitive data detection method described in any of the above embodiments.

[0154] According to embodiments of this disclosure, this disclosure also provides a readable storage medium storing computer instructions that enable a computer to implement the agent-based sensitive data detection method described in any of the above embodiments when executed.

[0155] This disclosure provides a computer program product that, when executed by a processor, can implement the agent-based sensitive data detection method described in any of the above embodiments.

[0156] Figure 7A schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0157] like Figure 7 As shown, device 700 includes a computing unit 701, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 702 or a computer program loaded into random access memory (RAM) 703 from storage unit 708. The RAM 703 may also store various programs and data required for the operation of device 700. The computing unit 701, ROM 702, and RAM 703 are interconnected via bus 704. Input / output (I / O) interface 705 is also connected to bus 704.

[0158] Multiple components in device 700 are connected to I / O interface 705, including: input unit 706, such as keyboard, mouse, etc.; output unit 707, such as various types of monitors, speakers, etc.; storage unit 708, such as disk, optical disk, etc.; and communication unit 709, such as network card, modem, wireless transceiver, etc. Communication unit 709 allows device 700 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0159] The computing unit 701 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above, such as the agent-based sensitive data detection method. For example, in some embodiments, the agent-based sensitive data detection method can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed on device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by the computing unit 701, one or more steps of the agent-based sensitive data detection method described above can be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform an agent-based sensitive data detection method by any other suitable means (e.g., by means of firmware).

[0160] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0161] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to the processor or controller of a general-purpose computer, special-purpose computer, or other programmable agent-based sensitive data detection device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0162] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0163] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0164] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0165] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, also known as cloud computing servers or cloud hosts, which are hosting products within the cloud computing service system to address the management difficulties and weak business scalability inherent in traditional physical hosts and Virtual Private Servers (VPS) services; they can also be servers for distributed systems or servers incorporating blockchain technology.

[0166] According to the technical solution of the embodiments of this disclosure, a sensitive data detection method and apparatus based on intelligent agents are provided. By utilizing a large artificial intelligence model, the parsing strategy of the file to be detected is determined based on the cognitive state of the data format of the file to be detected, and the file to be detected is parsed according to the parsing strategy to obtain the target parsed data. This improves the coverage of the parsing operation on the parsing scenario, as well as the accuracy and efficiency of the parsing process, thereby improving the comprehensiveness and accuracy of the sensitive data detection results.

[0167] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution provided in this disclosure can be achieved, and this is not limited herein.

[0168] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A sensitive data detection method based on intelligent agents, comprising: Through the decision-making layer of the intelligent agent, using a large artificial intelligence model, the parsing strategy of the file to be detected is determined based on the cognitive state of the data format of the file to be detected; The decision layer parses the file to be detected according to the parsing strategy to obtain the target parsing data. The execution layer of the intelligent agent detects whether the target parsed data includes sensitive data, and obtains the sensitive data detection result.

2. The method according to claim 1, wherein, The method of utilizing a large-scale artificial intelligence model to determine the parsing strategy for the file to be detected based on the understanding of the data format of the file to be detected includes: In response to the fact that the AI ​​model's perception of the file to be detected is unknown, the AI ​​model is used to determine the parsing strategy based on the file type of the file to be detected.

3. The method according to claim 1, wherein, The step of parsing the file to be detected according to the parsing strategy to obtain target parsing data includes: Based on the parsing strategy, the parsing units in the file to be detected are determined; The encoding format of the parsing unit is determined based on the data characteristics of the parsing unit, and the parsing unit is parsed according to the encoding format to obtain the unit parsing data; The target parsing data is obtained by combining the unit parsing data of each parsing unit.

4. The method according to claim 3, wherein, The step of determining the encoding format of the parsing unit based on its data characteristics, and parsing the parsing unit according to the encoding format to obtain unit parsing data, includes: The following parsing operations are performed iteratively until the preset termination condition is met: The encoding format of the parsing unit is determined based on the data characteristics of the parsing unit; The parsing unit is parsed according to the encoding format to obtain the parsed data of the unit; If the parsed data from the unit does not present the target data modality, the next parsing operation is performed.

5. The method according to claim 3 or 4, wherein, The step of determining the encoding format of the parsing unit based on its data characteristics, and parsing the parsing unit according to the encoding format to obtain unit parsing data, includes: Using the aforementioned large-scale artificial intelligence model, the encoding format of the parsing unit is determined based on the file characteristics of the parsing unit, and the parsing unit is parsed according to the encoding format to obtain the parsed data of the unit.

6. The method according to claim 1, wherein, In response to the known cognitive state, the step of parsing the file to be detected according to the parsing strategy to obtain target parsing data includes: The file to be detected is parsed using the standard parser corresponding to the data format to obtain the target parsing data.

7. The method according to any one of claims 1-6, wherein, The process of detecting whether the target parsed data includes sensitive data and obtaining sensitive data detection results includes: Based on the target data modality to which the target parsed data belongs, determine the target toolchain used to detect whether the target parsed data includes sensitive data; Using the tools in the target toolchain, detect whether the target parsed data includes sensitive data, and obtain the sensitive data detection result.

8. The method according to claim 7, wherein, The target parsing data includes data from multiple target data modalities, and The step of using tools from the target toolchain to detect whether the target parsed data includes sensitive data, and obtaining the sensitive data detection result, includes: For the data of multiple target data modalities associated with the target parsing data, the tools in the target toolchain corresponding to the target data modal are used to detect whether the data of the target data modal includes sensitive data, and an initial detection result is obtained; Cross-validation is performed on the initial detection results of each of the various target data modalities to obtain the sensitive data detection results corresponding to each of the various target data modalities.

9. The method according to claim 3 or 4, wherein, Also includes: Through the feedback and learning layers in the intelligent agent, the processing procedure for files to be detected with unknown data formats is standardized, and a standard parser corresponding to the data format of the files to be detected is generated.

10. The method according to claim 1, wherein, Also includes: For the remaining data in the file to be detected that failed to be parsed, the remaining data is parsed according to the received parsing operation to obtain auxiliary parsing data; Based on the target data modality to which the auxiliary parsing data belongs, determine an auxiliary toolchain for detecting whether the auxiliary parsing data includes sensitive data; The tools in the auxiliary toolchain are used to detect whether the auxiliary parsing data includes sensitive data.

11. The method according to claim 10, wherein, Also includes: Through the feedback and learning layers in the intelligent agent, the processing procedure for the remaining data is standardized, and a standard parser corresponding to the data format of the remaining data is generated.

12. A sensitive data detection device based on intelligent agents, comprising: The strategy determination unit is configured to determine the parsing strategy of the file to be detected by using a large artificial intelligence model through the decision layer of the intelligent agent, based on the cognitive state of the data format of the file to be detected. The data parsing unit is configured to parse the file to be detected through the decision layer according to the parsing strategy to obtain target parsing data; The data detection unit is configured to detect whether the target parsed data includes sensitive data through the execution layer of the intelligent agent, and obtain the sensitive data detection result.

13. An intelligent agent for detecting sensitive data, comprising: The perception layer is used to identify the file to be detected; The decision layer is used to utilize a large artificial intelligence model to determine the parsing strategy for the file to be detected based on the cognitive state of the data format of the file to be detected, and to parse the file to be detected according to the parsing strategy to obtain the target parsed data; The execution layer is used to detect whether the target parsed data includes sensitive data and obtain the sensitive data detection result.

14. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-11.

15. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-11.

16. A computer program product comprising: A computer program that, when executed by a processor, implements the method according to any one of claims 1-11.

Images