Sensitive data processing method and device, electronic equipment, storage medium and product

Abstract

The invention provides a sensitive data processing method and device, electronic equipment, a storage medium and a product, and relates to the technical field of data processing. The sensitive field type of the prompt word is determined according to the input instruction, the sensitive field type can be flexibly determined according to user requirements, and the sensitive data processing flexibility is improved. According to the method, the callable data is generated by calling the large language model, accurate understanding of semantics of the prompt words is achieved, the accuracy of generation of the callable data is guaranteed, and then the accuracy of generation of the data processing function is guaranteed. According to the method, the data processing function is generated according to the function template library and the callable data, the data processing function matched with the user demand can be flexibly generated, and the flexibility of sensitive data processing is improved.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a sensitive data processing method, apparatus, electronic device, storage medium, and product. Background Technology

[0002] With the rapid development of artificial intelligence, especially large-scale language model technology, the role of big data platforms is becoming increasingly crucial. Big data platforms typically possess a range of core functions, including but not limited to data acquisition, data aggregation, data cleaning, data sharing, and data services. Among these, the data acquisition module receives mostly unprocessed raw data, which may contain various sensitive information, requiring de-identification processing.

[0003] Existing methods for de-identifying sensitive data require specific strategies and tools, which are inconvenient for scenarios requiring temporary de-identification. If a different de-identification method is desired each time, the de-identification rules need to be rewritten for the data before the de-identification process can be performed.

[0004] In summary, the existing rules and fields for de-identifying sensitive data are fixed in advance, which makes the processing of sensitive data inflexible. Summary of the Invention

[0005] This application provides a sensitive data processing method, apparatus, electronic device, storage medium, and product to address the shortcomings of inflexible sensitive data processing in the prior art and to improve the flexibility of sensitive data processing.

[0006] Firstly, this application provides a method for processing sensitive data, including: Based on the type and complexity of the user's input command, generate a prompt message for the input command, including sensitive field types; The large language model matching the prompts is invoked to generate callable prompt data; Based on the function template library and callable data, generate a data processing function for the prompt message; The data processing function is executed to de-identify the target sensitive fields in the data to be processed. The target sensitive fields belong to the sensitive field type.

[0007] In one embodiment, a prompt is generated based on the type and complexity of the user's input instruction, including: Based on the type and complexity of the input command, a prompt template for the input command is matched from the prompt template library; Perform semantic analysis on the input commands to obtain the types of sensitive fields and the methods for desensitizing them; Based on the prompt template, words are recombined according to the sensitive field type and desensitization method to obtain the prompt.

[0008] In one embodiment, the prompt also includes feature extraction instructions, invokes a large language model matching the prompt, and generates callable data for the prompt, including: Perform semantic understanding on the prompts to obtain at least one sensitive field type, at least one desensitization method, and feature extraction description; Based on the feature extraction description, taking each sensitive field type as the main body and each desensitization method as the processing method, at least one standardized information group is generated; Obtain explanations and descriptions of each standardized information group; Based on the standardized information groups and explanations, callable data is obtained.

[0009] In one embodiment, a standardized information group consists of a subject and a processing method. The data processing function that generates the prompt based on a function template library and callable data includes: Based on the processing method of callable data, the processing function template is matched from the function template library; Generate the initial data processing function based on the processing function template, main body, processing method, and explanation; The initial data processing function is optimized to obtain the final data processing function.

[0010] In one embodiment, there are multiple prompts. After generating the prompt for the input command based on its type and complexity, the system further includes: Provide the user with confirmation information regarding the receipt of the input command; Based on the priority and generation order of the prompts, sort multiple prompts to obtain a prompt sequence; Within the prompt sequence, the large language model matching the sorted prompts is called sequentially to generate sorted, callable prompt data.

[0011] In one embodiment, after executing the data processing function to de-identify the target sensitive fields in the data to be processed, the method further includes: Store input instructions, obtain historical input instructions, and store the pending data of historical input instructions; When the input command for the next time period matches the historical input command, the pending data for the input command for the next time period is obtained based on the pending data of the historical input command.

[0012] Secondly, this application provides a sensitive data processing apparatus, comprising: The prompt generation module is used to generate prompts for input commands based on the type and complexity of the user's input commands. The prompts include sensitive field types. The callable data generation module is used to call the large language model that matches the prompts and generate callable data for the prompts. The function generation module is used to generate data processing functions for prompt messages based on the function template library and callable data; The data masking module is used to execute data processing functions to mask the target sensitive fields in the data to be processed. The target sensitive fields belong to the sensitive field type.

[0013] Thirdly, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any of the sensitive data processing methods described above.

[0014] Fourthly, this application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the sensitive data processing methods described above.

[0015] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements any of the sensitive data processing methods described above.

[0016] The sensitive data processing method, apparatus, electronic device, storage medium, and product provided in this application generate a prompt message for the user's input command based on the type and complexity of the input command. The prompt message includes sensitive field types. A matching large language model is invoked to generate callable data for the prompt message. A data processing function for the prompt message is generated based on a function template library and the callable data. The data processing function is executed to de-identify target sensitive fields in the data to be processed, where the target sensitive fields belong to the sensitive field type. This application determines the sensitive field type of the prompt message based on the input command, enabling flexible determination of sensitive field types according to user needs, thus improving the flexibility of sensitive data processing. By invoking a large language model to generate callable data, this application achieves accurate semantic understanding of the prompt message, ensuring the accuracy of the generated callable data, and consequently ensuring the accuracy of the generated data processing function. This application generates a data processing function based on a function template library and the callable data, enabling flexible generation of data processing functions that match user needs, further improving the flexibility of sensitive data processing. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is one of the flowcharts illustrating the sensitive data processing method provided in this application.

[0019] Figure 2 This is the second flowchart of the sensitive data processing method provided in this application.

[0020] Figure 3 This is a schematic diagram of the sensitive data processing device provided in this application.

[0021] Figure 4 This is a schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] The process of handling sensitive data includes data identification and classification, development of desensitization strategies, and implementation of desensitization processing.

[0024] Data identification and classification involves using automated tools to scan databases, data warehouses, or other data within big data platforms to identify potentially sensitive data, and then labeling and classifying that sensitive data.

[0025] Developing a desensitization strategy involves defining specific desensitization rules for each type of sensitive data based on the identified, labeled, and classified data, and then implementing these rules using code.

[0026] Implementing data anonymization involves using tools or scripts, configuring tool or script parameters according to predefined anonymization rules, initiating the anonymization process, anonymizing the identified data, and storing the anonymized data.

[0027] Existing methods for sensitive data governance, particularly for unstructured long texts, often fail to accurately identify the location of sensitive words in batches. The common approach is to encrypt the entire text, requiring users to decrypt it using a key. This prevents the public use of such anonymized data. This application leverages the ability of a large language model to understand the context of long texts, enabling sensitive data governance for unstructured data and significantly expanding the scope of its application.

[0028] Meanwhile, existing sensitive data governance solutions typically configure given sensitive field types and de-identification methods, which cannot adapt to dynamically changing usage needs. Furthermore, users need to be familiar with the system representation of sensitive field types to configure sensitive data governance. This application lowers the barrier to entry for sensitive data governance and makes governance methods more flexible by providing configurations of sensitive fields and governance methods (de-identification methods) described in natural language.

[0029] The following is combined with Figures 1-4 This application describes the sensitive data processing methods, apparatus, and electronic devices.

[0030] Figure 1 This is one of the flowcharts illustrating the sensitive data processing method provided in this application, such as... Figure 1 As shown, the sensitive data processing method includes steps S100 to S400, and the specific steps are as follows.

[0031] S100: Generate prompts for input commands based on the type and complexity of the user's input commands.

[0032] The prompt includes information about sensitive field types.

[0033] The executing entities of this application include sensitive data processing devices, processors, servers, etc. This application's embodiments use a server as an example for illustration.

[0034] Users input processing instructions (input instructions) at the server's front end. During user input, intelligent input based on historical input and common task patterns is provided to reduce input errors and improve efficiency. Users select the data to be anonymized (used later to determine the data to be processed), the type of field to be anonymized, and the anonymization method, and an input instruction is generated. The input instruction is then sent to the server's front end.

[0035] Input commands support various input modes, such as text input, voice input, image input, and gesture input. For example, data processing personnel at Telecom Group A need to regularly process large amounts of user data, including masking user IDs and obfuscating usernames. When a data processing personnel begins typing "use for student ID" into the input interface, the system server immediately and intelligently prompts for complete command options based on historical input records and common task patterns, such as "use mask to replace student ID". This allows data processing personnel to quickly select and complete the input, reducing input errors and improving work efficiency.

[0036] For example, when a data processor receives a report containing sensitive user information that needs immediate attention, due to time constraints, they might choose to input processing instructions via voice. They might say into the system microphone, "Mask all user phone numbers, keeping only the last four digits." The server-side voice recognition system accurately captures this instruction and converts it into text for subsequent processing. This multimodal input method is particularly effective when handling urgent tasks because it allows data processors to input instructions in the fastest possible way.

[0037] Furthermore, basic validation of input commands is performed during the input phase, such as checking syntax and the existence of key fields, providing feedback in advance. For example, when a data processing staff member attempts to process a batch of user data, they may omit the key field "processing object" in their input command due to unfamiliarity with the system. When they enter "use mask instead" (input command) and submit, the system server immediately performs input validation, discovers the missing processing object, and returns an error message: "Please specify the processing object (e.g., user number, mobile phone number, etc.)." The data processing staff member corrects the input command based on the prompt and successfully completes the data processing task. This real-time input validation mechanism helps novice employees (data processing staff) quickly familiarize themselves with the system and ensures that all input commands are complete and valid.

[0038] The system dynamically generates prompts based on the type and complexity of the user's input command. For example, it dynamically retrieves a corresponding prompt template based on the type and complexity of the input command. The input command is then restructured using the prompt template to generate the final prompt.

[0039] Furthermore, when the server's front-end receives an input command that cannot be parsed, it may be due to a typing error or a syntax error. When the server's front-end attempts to parse the input command, it finds it cannot be converted into a valid prompt. To enhance system compatibility, the server does not directly output an error (false), but instead provides a possible correction suggestion, such as "Your command appears to have a syntax error; please check and re-enter." At this point, the server provides a link or button to guide the user to re-enter the command, allowing the user to quickly correct the error and resubmit the input. This application ensures that the server can still provide user-friendly feedback and assistance when encountering unparseable commands.

[0040] S200: Invokes the large language model that matches the prompt and generates callable data for the prompt.

[0041] Large Language Models (LLMs) are deep learning models trained on massive amounts of text data. They can not only generate natural language text, but also deeply understand the meaning of text and handle various natural language tasks, such as text summarization, question answering, and translation.

[0042] Callable data refers to data that can be directly called when generating data processing functions, and is used for subsequent data processing functions.

[0043] like Figure 2 As shown, the server calls a large language model to process the prompt. First, the server dynamically selects the most suitable large language model based on the type and complexity of the prompt, thus obtaining a matching large language model.

[0044] Prompts can include text information, voice recordings, image data, etc. For example, when the server receives a text-based prompt, it dynamically selects the most suitable matching large language model to process the prompt based on the type and complexity (difficulty) of the task within the prompt. This matching large language model includes models adept at processing natural language text, possessing strong text understanding and generation capabilities. For instance, when the server receives an instruction to process image data, it will select a suitable large language model for image processing as the matching large language model. In this way, the server can ensure that the optimal model is selected for each prompt (processing task), thereby improving processing efficiency and quality.

[0045] The server invokes a large language model that matches the prompt to generate callable data for the prompt. This callable data is used by subsequent data processing functions. The matching large language model is invoked to standardize key information in the prompt (e.g., sensitive fields and de-identification methods) to obtain callable data in a standardized format. The standardized format includes a lightweight data exchange format (JSON).

[0046] S300: A data processing function that generates prompts based on the function template library and callable data.

[0047] First, a function template library for common processing functions is built. Based on the callable data generated by the large language model, the corresponding processing function template is quickly matched from the function template library. Then, based on the processing function template and the callable data, the data processing function for the prompt message is generated.

[0048] S400: Executes a data processing function to de-identify the target sensitive fields in the data to be processed. The target sensitive fields belong to the sensitive field type.

[0049] The data to be processed was obtained in the following way: It dynamically adapts to data sources of different data types and formats to obtain raw data to be processed containing sensitive fields; the data source includes the data source specified in the input command (the data that needs to be de-sensitized selected by the user).

[0050] The original data to be processed is filtered, completed, deduplicated, and converted in format to obtain the data to be processed.

[0051] The sensitive field type in the prompt message includes a general overview of a category of sensitive fields that need to be processed, such as "name," "phone number," or "ID card number." The target sensitive field includes the specific sensitive field that belongs to the sensitive field type that needs to be processed, such as "Zhang San" or "Li Si."

[0052] This application supports dynamic data source adaptation for various data types and formats when acquiring data to be processed, eliminating the need to specify data types in advance. For example, Group A's data processing center needs to integrate user data from different departments and systems. This user data may be stored in various data sources, such as relational databases, non-relational (Not Only SQL, NoSQL) databases, comma-separated values ​​(CSV) files, and application programming interfaces (APIs). To achieve unified data processing and analysis, the server employs dynamic data source adaptation technology. When the server receives a data processing request, it first identifies the data source type and data format specified in the request. Then, based on the identification result, the server dynamically loads the corresponding data source adapter for connecting to and reading data. In this way, regardless of the data source type or data format, the server can accurately and efficiently read the raw data to be processed and perform subsequent processing operations. Through this method, the server achieves dynamic adaptation to multiple data source types and formats without needing to specify the database type in advance, improving the system's flexibility and scalability.

[0053] The server filters, completes, removes duplicates, and converts the format of the raw data to be processed. This raw data contains outliers, missing values, and duplicates. To ensure the accuracy and consistency of the raw data, the server performs necessary preprocessing operations before reading it. First, the server detects and handles outliers, such as correcting or removing values ​​outside the reasonable range. Then, for missing values, the server uses interpolation or padding methods to ensure data integrity. Furthermore, the server removes duplicates to avoid their impact on subsequent processing. Through these preprocessing operations, the server ensures that the data it reads is accurate, consistent, and reliable, providing a solid foundation for subsequent data processing and analysis.

[0054] The data to be processed includes unstructured data, such as long unstructured text. Because this long unstructured text lacks a fixed format, has a scattered information distribution, and its semantics depend on context, sensitive fields (such as personal information, trade secrets, and health data) may appear anywhere and are highly mixed with ordinary content. Traditional rule-based or simple keyword matching methods cannot accurately identify the location of these sensitive fields (potentially misidentifying or omitting them). Therefore, de-identification methods often employ a conservative strategy of "entire-segment encryption," sacrificing data usability (making it unusable publicly).

[0055] This application flexibly generates data processing functions based on sensitive field types. By executing these functions, only the target sensitive fields in the data to be processed are de-identified in batches, generating de-identified data. This application's sensitive data processing method not only ensures accurate processing of the target sensitive fields but also guarantees that the de-identified data can be publicly used normally, thus improving the utilization rate of the de-identified data.

[0056] Furthermore, determine whether the data anonymization process was successful. For example, determine if the anonymization process was abnormal. Determine if the dimensions and volume of the anonymized data deviate from the expected dimensions and volume, and whether the magnitude of the deviation is normal.

[0057] If the data anonymization process is successful, the original sensitive fields are replaced with the anonymized fields, and the entire dataset is output as a whole. If the data anonymization process is unsuccessful, considering the non-idempotent nature of the large language model output, steps S100-S400 are repeated.

[0058] Furthermore, while outputting the processing results of the input commands, the server also outputs a parsing of the processing procedures. This parsing helps users better understand the extraction process and results of the input commands. It details the processing method for each field, the comparison before and after processing, and the meaning of the results. In this way, users can more clearly understand the processing procedures and results, thus better evaluating the processing effectiveness and adjusting their input commands.

[0059] The sensitive data processing method provided in this application generates a prompt message for the user's input command based on the type and complexity of the input command. The prompt message includes sensitive field types. It then calls a matching large language model to generate callable data for the prompt message. Based on a function template library and the callable data, it generates a data processing function for the prompt message. Finally, it executes the data processing function to de-identify target sensitive fields in the data to be processed, where the target sensitive fields belong to a sensitive field type. This application determines the sensitive field type of the prompt message based on the input command, enabling flexible determination of sensitive field types according to user needs, thus improving the flexibility of sensitive data processing. By calling a large language model to generate callable data, this application achieves accurate semantic understanding of the prompt message, ensuring the accuracy of the generated callable data, and consequently ensuring the accuracy of the generated data processing function. Finally, by generating a data processing function based on a function template library and the callable data, this application can flexibly generate data processing functions that match user needs, further improving the flexibility of sensitive data processing.

[0060] Based on the above embodiments, generating prompts for input commands according to the type and complexity of the user's input commands includes the following steps: Based on the type and complexity of the input command, a prompt template for the input command is matched from the prompt template library; Perform semantic analysis on the input commands to obtain the types of sensitive fields and the methods for desensitizing them; Based on the prompt template, words are recombined according to the sensitive field type and desensitization method to obtain the prompt.

[0061] The prompt template library includes prompt templates for various input commands. It includes the type and complexity of the input command, the prompt templates, and the mapping relationships between them.

[0062] Based on the type and complexity of the input command, a prompt template is matched from the prompt template library. Semantic analysis is performed on the input command to obtain the types of sensitive fields and the desensitization methods. Based on the prompt template, words are recombined according to the sensitive field types and desensitization methods to obtain the prompt.

[0063] Furthermore, the prompt template also includes more detailed and clear contextual information about the input instructions, so that the large language model can complete the task more accurately and efficiently when processing prompts.

[0064] For example, a data processor inputs a processing instruction (input instruction) into the server front-end: "Enigma the user's name and address." Upon receiving this input instruction, the server uses natural language processing (NLP) technology to perform semantic analysis and optimization. The server first identifies key information in the input instruction: "Username" and "Address" are sensitive field types that need to be processed, and "Enigma" is the processing method (de-sensitization method). Then, based on this key information and a prompt template, the server generates a more accurate and clear prompt: "Please enigma the 'Username' and 'Address' fields to protect user privacy." In this way, the server can ensure that the large language model can more accurately understand the intent and requirements of the input instruction when processing it.

[0065] This application generates prompts by dynamically matching prompt templates. It can dynamically generate prompts based on user-defined sensitive field types and desensitization methods, facilitating personalized sensitive data processing according to user needs. By generating prompts, this application improves the accuracy of large language models in understanding the intent and requirements of input commands.

[0066] Based on the above embodiments, the prompt also includes feature extraction instructions, calling a large language model that matches the prompt, and generating callable data for the prompt, including the following steps: Perform semantic understanding on the prompts to obtain at least one sensitive field type, at least one desensitization method, and feature extraction description; Based on the feature extraction description, taking each sensitive field type as the main body and each desensitization method as the processing method, at least one standardized information group is generated; Obtain explanations and descriptions of each standardized information group; Based on the standardized information groups and explanations, callable data is obtained.

[0067] The matching large language model performs semantic understanding on the prompt, obtaining at least one sensitive field type, at least one desensitization method, and feature extraction instructions. For example, the prompt: 'Use a mask to replace student IDs and blur the name' contains one or more pairs of data processing subjects and their corresponding processing methods. Please extract these from the original text and output them as a list. Each item in the list contains an object of a set of processing methods, with the object structure: key: subject (processing object), method: processing method. Here, "student ID" and "name" are the sensitive field types that need to be desensitized. "Mask replacement" and "blurring" are the desensitization methods. The instruction "This sentence contains one or more pairs of data processing subjects and their corresponding processing methods. Please extract these from the original text and output them as a list. Each item in the list contains an object of a set of processing methods, with the object structure: key: subject (processing object), method: processing method" is the feature extraction instruction.

[0068] Based on the feature extraction instructions, at least one standardized information group is generated, using each sensitive field type as the main body and each de-identification method as the processing method. For example, "key: name, method: fuzzing" is one standardized information group, and "key: student ID, method: masking replacement" is another standardized information group. All standardized information groups are listed in a list format. The list is as follows. [ { "key": "student ID" "method": mask substitution }, { "key": "Name" "method": "fuzzing" } ].

[0070] Obtain explanations for each standardized information group. For example, the explanation might be: "Each item in this list represents an object of a set of processing methods, where key indicates the subject to be processed (sensitive field type)," and method indicates the processing method (de-sensitization method) for this subject.

[0071] Retrieve callable data based on the standardized information groups and explanations.

[0072] Furthermore, if the matching large language model cannot generate a standardized information group based on the prompt, an error message is generated, along with detailed error information and possible solutions.

[0073] This application extracts features from prompts to obtain standardized information groups and explanations. When generating data processing functions, these standardized information groups and callable data can be directly called, which helps improve the efficiency and accuracy of generating data processing functions.

[0074] Furthermore, the server can determine two or more matching large language models based on the prompt. It then acquires and compares the callable data for each matching large language model. The most accurate callable data is used as the final callable data for that prompt.

[0075] For example, when processing user data, the data processing center encountered ambiguous user address information. Due to input errors or non-standard formatting, the initial processing result had low confidence. To improve the accuracy of the processing results, the server introduced a multi-round verification mechanism. First, the server used a large language model, adept at processing user address information, to perform preliminary processing on this address, but the resulting callable data was not ideal. Therefore, the server used another large language model, adept at processing text information, to cross-validate the user address information. By comparing the processing results of the two models, the server found that the callable data generated by the second model was more accurate and reliable. Ultimately, the server adopted the callable data from the second model and returned it to the user. This multi-round verification improved the accuracy of the callable data.

[0076] Based on the above embodiments, a standardized information group consists of a subject and a processing method. The data processing function that generates the prompt message based on the function template library and callable data includes: Based on the processing method of callable data, the processing function template is matched from the function template library; Generate the initial data processing function based on the processing function template, main body, processing method, and explanation; The initial data processing function is optimized to obtain the final data processing function.

[0077] This project constructs common processing function templates and builds a function template library based on these templates. Common processing function templates include those for data cleaning, transformation, and aggregation. Each common processing function template defines the basic structure of input parameters, output results, and processing logic.

[0078] Based on the processing methods in the callable data provided by the matching large language model, a processing function template is matched from the function template library. A customized initial data processing function is generated based on the processing function template, the body, the processing method, and the explanation.

[0079] Because some initial processing functions have low execution efficiency, processing time is long. To improve processing efficiency, the server automatically optimizes the generated initial data processing functions using code optimization techniques. First, the server performs performance analysis on the execution process of the initial data processing functions to identify performance bottlenecks. Then, based on the performance analysis results, the server employs a series of optimization techniques, such as algorithm optimization, memory optimization, and parallel processing, to automatically optimize the initial data processing functions, resulting in the optimized initial data processing functions. The data processing functions (data processing method code) show a significant improvement in execution efficiency, and processing time is greatly reduced. In this way, the server can ensure the efficient execution of data processing functions, improving the overall efficiency of data processing.

[0080] Furthermore, the data processing functions are tested and verified. After the server generates the initial data processing functions, automated unit testing is performed to ensure their correctness and reliability. The server writes a set of test cases for each generated initial data processing function, covering various input scenarios and expected outputs. Then, the server uses automated testing tools to perform unit tests on the initial data processing functions, verifying whether their output results under various input conditions meet expectations. If the test results do not meet expectations, the server will immediately issue a warning and prompt developers to investigate and fix the issue. In this way, the server can ensure that each generated data processing function has undergone rigorous testing and verification, improving the quality and reliability of the data processing method code.

[0081] This application generates initial data processing functions based on a function template library, improving the flexibility of these functions. By optimizing these initial data processing functions, the accuracy of the data processing functions is also improved.

[0082] Based on the above embodiments, there are multiple prompts. After generating the prompt for the input command according to the type and complexity of the user's input command, the following steps are also included: Provide the user with confirmation information regarding the receipt of the input command; Based on the priority and generation order of the prompts, sort multiple prompts to obtain a prompt sequence; Within the prompt sequence, the large language model matching the sorted prompts is called sequentially to generate sorted, callable prompt data.

[0083] For some input commands (or prompts), the computational load is extremely high. For example, the task corresponding to an input command might require de-identifying millions of user records. If a synchronous processing method is used, it would severely block the main process and affect the execution of other input commands. Therefore, this application adopts an asynchronous processing method to handle such input commands. After the server's front-end generates the input command prompt, it immediately returns a task confirmation message to the user (sender). Then, the server places the prompt into a background prompt sequence for queuing. This prompt sequence is ordered according to the prompt's priority and the order in which it was generated.

[0084] In this way, the sender can continue sending other input instructions (tasks) without being blocked by the current computationally intensive task. The background prompt sequence processes the sorted prompts according to the priority and arrival order of the tasks in the prompts. Once the sorted prompts have been processed, the server returns the processing result to the sender. In this way, the server can efficiently handle a large number of time-consuming tasks while ensuring the smooth operation of the main process.

[0085] This application ensures smooth reception of input commands by first returning to the task to receive confirmation information and sorting the prompts, without affecting the processing of prompts.

[0086] Based on the above embodiments, after executing the data processing function to de-identify the target sensitive fields in the data to be processed, the following steps are also included: Store input instructions, obtain historical input instructions, and store the pending data of historical input instructions; When the input command obtained in the next time period matches the historical input command, the pending data of the input command in the next time period is obtained based on the pending data of the historical input command.

[0087] For tasks that frequently require executing repetitive or similar input commands, such as daily anonymization of user data to generate reports, servers employ a result caching strategy to improve processing efficiency. When the server receives an input command for the first time, it executes the command and saves the processing result to the cache (retrieving historical input commands).

[0088] When an input command received in the next time period matches a historical input command, the server retrieves the pending data for the input command in the next time period based on the pending data of the historical input command. When the same or similar input command is received again in the next time period, the server first checks if the cache already contains the processing result (pending data) of a matching historical input command. If the cache contains the processing result of the matching historical input command, the server directly returns the cached processing result (calling the processing result of the matching historical input command as the pending data for the input command in the next time period) without re-executing the input command. This avoids unnecessary computational overhead and improves processing efficiency for repetitive or similar input commands. Furthermore, to ensure the validity of cached results, the server periodically cleans up expired cached results or sets a reasonable cache expiration time based on the data update frequency.

[0089] This application improves the processing efficiency of the same or similar input instructions by storing historical input instructions and their pending data.

[0090] The sensitive data processing method presented in this application allows for more flexible sensitive data governance and improves the effective utilization of data. Traditional sensitive data governance solutions typically target specific sensitive field types for processing. More flexible systems may provide configuration entry points to modify selected sensitive field types and de-identification methods, but these have a certain configuration barrier and cannot handle unstructured data. This application lowers the barrier to entry for sensitive data governance, broadens the types of sensitive fields and processing methods (de-identification methods), and has irreplaceable value in the sensitive data governance of unstructured data.

[0091] The sensitive data processing apparatus provided in this application is described below. The sensitive data processing apparatus described below can be referred to in correspondence with the sensitive data processing method described above.

[0092] like Figure 3 As shown, a sensitive data processing apparatus includes: The prompt generation module 301 is used to generate prompts for input commands based on the type and complexity of the user's input command. The prompts include sensitive field types. Callable data generation module 302 is used to call the large language model matching the prompt to generate callable data for the prompt; The function generation module 303 is used to generate a data processing function for the prompt message based on the function template library and callable data. The desensitization module 304 is used to execute data processing functions to desensitize the target sensitive fields in the data to be processed. The target sensitive fields belong to the sensitive field type.

[0093] The sensitive data processing apparatus provided in this application generates a prompt message for the user's input command based on the type and complexity of the input command. The prompt message includes sensitive field types. It then calls a matching large language model to generate callable data for the prompt message. Based on a function template library and the callable data, it generates a data processing function for the prompt message. Finally, it executes the data processing function to de-identify target sensitive fields in the data to be processed, where the target sensitive fields belong to the sensitive field type. This application determines the sensitive field type of the prompt message based on the input command, enabling flexible determination of sensitive field types according to user needs, thus improving the flexibility of sensitive data processing. By calling a large language model to generate callable data, this application achieves accurate semantic understanding of the prompt message, ensuring the accuracy of the generated callable data, and consequently ensuring the accuracy of the generated data processing function. Finally, by generating a data processing function based on a function template library and the callable data, this application can flexibly generate data processing functions that match user needs, further improving the flexibility of sensitive data processing.

[0094] In one embodiment, the prompt generation module 301 is used to: match a prompt template for the input instruction in the prompt template library based on the type and complexity of the input instruction; perform semantic analysis on the input instruction to obtain the sensitive field type and desensitization method; and perform word recombination on the sensitive field type and desensitization method based on the prompt template to obtain the prompt.

[0095] In one embodiment, the prompt also includes feature extraction instructions, and the callable data generation module 302 is used to: perform semantic understanding on the prompt, obtain at least one sensitive field type, at least one desensitization method, and feature extraction instructions; based on the feature extraction instructions, generate at least one standardized information group with each sensitive field type as the main body and each desensitization method as the processing method; obtain explanations for each standardized information group; and obtain callable data based on each standardized information group and each explanation.

[0096] In one embodiment, a standardized information group consists of a subject and a processing method. The function generation module 303 is used to: match a processing function template in the function template library based on the processing method of the callable data; generate an initial data processing function based on the processing function template, the subject, the processing method, and the explanation; and optimize the initial data processing function to obtain a data processing function.

[0097] In one embodiment, there are multiple prompts, and the callable data generation module 302 is further configured to: provide the user with confirmation information for receiving input instructions; sort the multiple prompts according to their priority and generation order to obtain a prompt sequence; and sequentially call the matching large language model of the sorted prompts in the prompt sequence to generate callable data for the sorted prompts.

[0098] In one embodiment, the desensitization module 304 is further configured to: store input instructions, obtain historical input instructions, and store the pending data of historical input instructions; when the input instruction obtained in the next time period matches the historical input instructions, obtain the pending data of the input instructions in the next time period based on the pending data of the historical input instructions.

[0099] Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4 As shown, the electronic device may include a processor 410, a communications interface 420, a memory 430, and a communication bus 440. The processor 410, communications interface 420, and memory 430 communicate with each other via the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute a sensitive data processing method. This method includes: generating a prompt for the input instruction based on its type and complexity, the prompt including a sensitive field type; calling a matching large language model of the prompt to generate callable data for the prompt; generating a data processing function for the prompt based on a function template library and the callable data; and executing the data processing function to desensitize the target sensitive field in the data to be processed, the target sensitive field belonging to a sensitive field type.

[0100] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0101] On the other hand, this application also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the sensitive data processing method provided by the above methods. The method includes: generating a prompt for the input instruction based on the type and complexity of the user's input instruction, the prompt including a sensitive field type; calling a matching large language model of the prompt to generate callable data for the prompt; generating a data processing function for the prompt based on a function template library and the callable data; and executing the data processing function to desensitize a target sensitive field in the data to be processed, the target sensitive field belonging to a sensitive field type.

[0102] In another aspect, this application also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the sensitive data processing methods provided by the above methods. The method includes: generating a prompt for the input instruction based on the type and complexity of the user's input instruction, the prompt including a sensitive field type; calling a matching large language model of the prompt to generate callable data for the prompt; generating a data processing function for the prompt based on a function template library and the callable data; and executing the data processing function to desensitize a target sensitive field in the data to be processed, the target sensitive field belonging to a sensitive field type.

[0103] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0104] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0105] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for processing sensitive data, characterized in that, include: Based on the type and complexity of the user's input command, a prompt message is generated for the input command, and the prompt message includes sensitive field types; The large language model matching the prompt is invoked to generate callable data for the prompt; The data processing function for generating the prompt message is generated based on the function template library and the callable data. The data processing function is executed to de-identify the target sensitive fields in the data to be processed, where the target sensitive fields belong to the sensitive field type.

2. The sensitive data processing method according to claim 1, characterized in that, The step of generating prompts for user input commands based on their type and complexity includes: Based on the type and complexity of the input instruction, a prompt template for the input instruction is matched in the prompt template library; Perform semantic analysis on the input command to obtain the sensitive field type and desensitization method; Based on the aforementioned prompt template, the sensitive field type and the desensitization method are used to recombine words to obtain the prompt.

3. The sensitive data processing method according to claim 1, characterized in that, The prompt also includes feature extraction instructions. The step of calling the matching large language model of the prompt to generate callable data for the prompt includes: Perform semantic understanding on the prompt to obtain at least one of the sensitive field types, at least one desensitization method, and the feature extraction description; Based on the feature extraction description, taking each of the aforementioned sensitive field types as the main body and each of the aforementioned desensitization methods as the processing method, at least one standardized information group is generated. Obtain explanations of each of the standardized information groups; The callable data is obtained based on each of the standardized information groups and each of the explanations.

4. The sensitive data processing method according to claim 3, characterized in that, A standardized information group consists of a subject and a processing method, wherein the data processing function that generates the prompt based on the function template library and the callable data includes: Based on the method for processing the callable data, a processing function template is matched from the function template library; Based on the processing function template, the main body, the processing method, and the explanation, an initial data processing function is generated; The initial data processing function is optimized to obtain the data processing function.

5. The sensitive data processing method according to claim 1, characterized in that, There are multiple prompts. After generating the prompt for the input command based on its type and complexity, the process further includes: The system should send a confirmation message to the user confirming receipt of the input command. Based on the priority and generation order of the prompts, the multiple prompts are sorted to obtain a prompt sequence; In the sequence of prompts, the matching large language model of the sorted prompts is called sequentially to generate callable data of the sorted prompts.

6. The sensitive data processing method according to claim 1, characterized in that, After executing the data processing function to de-identify the target sensitive fields in the data to be processed, the process further includes: Store the input instruction to obtain historical input instructions, and store the unprocessed data of the historical input instructions; When the input command for the next time period matches the historical input command, the data to be processed for the input command in the next time period is obtained based on the data to be processed for the historical input command.

7. A sensitive data processing device, characterized in that, include: The prompt generation module is used to generate a prompt for the user's input command based on the type and complexity of the input command, and the prompt includes sensitive field types; A callable data generation module is used to call the matching large language model of the prompt to generate callable data for the prompt; The function generation module is used to generate the data processing function for the prompt message based on the function template library and the callable data; The desensitization module is used to execute the data processing function to desensitize the target sensitive fields in the data to be processed, wherein the target sensitive fields belong to the sensitive field type.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the sensitive data processing method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the sensitive data processing method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the sensitive data processing method as described in any one of claims 1 to 6.

Images