Secure input decision-making methods, systems, and devices for large language model interaction

Abstract

This invention relates to a secure input decision-making method, system, and device for large language model interaction. The method includes monitoring user behavior data in a session window and acquiring text data input by the user in the session window; performing sensitivity and semantic recognition on the text data, comparing the behavior data with the user's historical behavior baseline to determine the degree of behavior abnormality; generating an input decision based on the text sensitivity level, potential illegal intent, and behavior abnormality; and executing an output action based on the input decision. The output action includes at least refusing to output text data to the large language model or de-identifying the text data to obtain de-identified text data and outputting the de-identified text data to the large language model. This method, by integrating a three-dimensional protection system at the content, semantic, and behavioral levels, achieves full-scenario coverage from surface violations to covert attacks, effectively solving the problem of missed detections in single-dimensional detection, and improving the civility and friendliness of the network environment.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and natural language processing, and in particular to a secure input decision-making method, system, and device for large language model interaction. Background Technology

[0002] With the rapid development of internet information technology, major websites and social media platforms have become places for people to discuss issues. However, due to the low cost of breaking the law, many people often post sensitive information on social media. This sensitive information involves sensitive topics, and if not blocked, these topics may cause controversy. In addition, these topics may also contain uncivilized or illegal content. If not blocked, these unfriendly remarks may harm users, infringe on their legitimate rights, and damage the civilized and friendly nature of the online environment.

[0003] Therefore, input risk control technologies have emerged on the market to intercept sensitive information. Among them, the most commonly used is sensitive word filtering technology, which identifies whether the input text contains sensitive keywords and blocks or filters them if they are found. This type of input risk control technology has the disadvantage of a single identification dimension, lacks multi-dimensional semantic understanding capabilities, and is difficult to identify complex scenarios such as prompt injection and role inducement. Summary of the Invention

[0004] This invention provides a secure input decision method, system, and device for large language model interaction, in order to overcome at least one of the above-mentioned technical problems existing in the prior art.

[0005] To achieve the above objectives, the embodiments of the present invention adopt the following technical solutions: In a first aspect, the present invention provides a secure input decision method for large language model interaction, comprising: Monitor user behavior data in the session window and obtain the text data entered by the user in the session window; The text data is subjected to sensitivity identification to determine the text sensitivity level; Semantic recognition is performed on the text data to determine whether there is any potential illegal intent; The behavioral data is compared with the user's historical behavioral baseline to determine the degree of behavioral abnormality of the user; Input decisions are generated based on the text sensitivity level, the potential violation intent, and the degree of behavioral anomaly. Based on the input decision, an output action is executed, the output action including at least refusing to output the text data to the large language model or performing de-identification processing on the text data to obtain de-identified text data, and outputting the de-identified text data to the large language model.

[0006] In one possible implementation of the first aspect, prior to performing sensitivity identification and potential conflict identification on the text data, the method further includes: The encoding of the text data is detected and converted into a unified encoding. The length of the text data is validated, and text data exceeding a preset length is discarded; The text data is formatted in a basic way to obtain a standard text data package.

[0007] In one possible implementation of the first aspect, the step of performing sensitivity identification on the text data and determining the text sensitivity level includes: The standard text data packet is identified by using the AC automaton multi-pattern matching algorithm to obtain the sensitive words in the standard text data packet. The identified sensitive words are matched with preset sensitive words in a preset sensitive word classification database to obtain the sensitivity level of the sensitive words, and the sensitivity level of the sensitive words is used as the sensitivity level of the text. The sensitive word classification database includes at least L1, L2 and L3 levels. L1 level represents serious violations, L2 level represents general violations, and L3 level is a custom level.

[0008] In one possible implementation of the first aspect, the potential violation intent includes potential intent and malicious speech, and the step of performing semantic recognition on the text data to determine whether a potential violation intent exists includes: The standard text data packet is subjected to feature extraction to obtain a semantic feature vector; The semantic feature vector is input into a pre-trained intention recognition model to identify potential intentions and output the potential intention probability. The semantic feature vector is input into a pre-trained speech recognition model to identify malicious speech and output the probability of malicious speech.

[0009] In one possible implementation of the first aspect, the step of extracting features from the standard text data packet to obtain a semantic feature vector includes: The standard text data packet is input into the pre-trained Llama-Prompt-Guard model, which outputs the semantic feature vector.

[0010] In one possible implementation of the first aspect, the potential violation intent further includes potential emotion, and the step of performing semantic recognition on the text data to determine whether a potential violation intent exists further includes: The semantic feature vector is input into a pre-trained emotion recognition model to perform emotion recognition and output the probability of potential emotions.

[0011] In one possible implementation of the first aspect, comparing the behavioral data with the user's historical behavioral baseline to determine the user's behavioral anomaly includes: Extract behavioral features from the behavioral data and construct a behavioral feature vector, wherein the behavioral features include request frequency features, request content features, and behavioral sequence features; The behavioral feature vector is compared with the user's historical behavioral baseline to determine the degree of behavioral abnormality.

[0012] In one possible implementation of the first aspect, the process of de-identifying the text data to obtain de-identified text data includes: Based on the category of sensitive words, the sensitive words are desensitized using preset desensitization rules to obtain desensitized text data; The desensitization rules include: Replace named entities with generic tags; Partial masking is applied to the string of numbers.

[0013] Compared with the prior art, the present invention has at least the following beneficial effects: The present invention provides a secure input decision method for large language model interaction. By integrating a three-dimensional protection system at the content level, semantic level and behavior level, it achieves full-scenario coverage from surface violations to covert attacks, effectively solves the problem of missed detection in single-dimensional detection, and can improve the civility and friendliness of the network environment.

[0014] Secondly, the present invention provides a secure input decision system for large language model interaction, comprising: The data acquisition module is used to monitor user behavior data in the session window and acquire the text data entered by the user in the session window; The sensitive word recognition module is used to identify the sensitivity of the text data and determine the sensitivity level of the text; The semantic analysis module is used to perform semantic recognition on the text data to determine whether there is any potential illegal intent; The behavior analysis module is used to compare the behavior data with the user's historical behavior baseline to determine the degree of abnormality of the user's behavior; The input decision module is used to generate input decisions based on the text sensitivity level, the potential violation intent, and the abnormality of the behavior; The decision execution module is used to execute output actions based on the input decision, and the output actions include at least refusing to output the text data or outputting the text data after desensitization.

[0015] Thirdly, the present invention provides an electronic device comprising: at least one processor and at least one memory, wherein the memory stores computer-readable instructions; the computer-readable instructions are executed by one or more of the processors, causing the electronic device to implement a secure input decision method for interaction with large language models as described in any implementation of the first aspect.

[0016] Fourthly, the present invention provides a storage medium having a computer-executable program stored thereon, the computer-executable program being used to cause a computer to execute a secure input decision method for interaction with large language models as in any implementation of the first aspect.

[0017] Understandably, the beneficial effects of the system of the second aspect, the electronic device of the third aspect, and the storage medium of the fourth aspect provided above can be referred to in light of the beneficial effects of the first aspect and any of its possible design embodiments, which will not be repeated here. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention; Figure 2 A flowchart illustrating a secure input decision-making method for large language model interaction provided in this embodiment of the invention; Figure 3 This is a structural block diagram of a secure input decision system for large language model interaction provided in an embodiment of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be described below with reference to the accompanying drawings. In the description of the present invention, unless otherwise stated, " / " indicates that the objects before and after are in an "or" relationship. For example, A / B can represent A or B. The "or" in the present invention is merely a description of the relationship between the related objects, indicating that three relationships can exist. For example, A or B can represent: A alone, A and B simultaneously, and B alone. A and B can be singular or plural. Furthermore, in the description of the present invention, unless otherwise stated, "multiple" refers to two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items.

[0021] Furthermore, in the embodiments of the present invention, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of the present invention should not be construed as superior or more advantageous than other embodiments or designs. Rather, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner to facilitate understanding.

[0022] With the rapid development of internet information technology, major websites and social media platforms have become places for people to discuss issues. However, due to the low cost of breaking the law, many people often post sensitive information on social media. This sensitive information involves sensitive topics, and if not blocked, these topics may cause controversy. In addition, these topics may also contain uncivilized or illegal content. If not blocked, these unfriendly remarks may harm users, infringe on their legitimate rights, and damage the civilized and friendly nature of the online environment.

[0023] Therefore, input risk control technologies have emerged on the market to intercept sensitive information. Among them, the most commonly used is sensitive word filtering technology, which identifies whether the input text contains sensitive keywords and blocks or filters them if they are found. This type of input risk control technology has the disadvantage of a single identification dimension, lacks multi-dimensional semantic understanding capabilities, and is difficult to identify complex scenarios such as prompt injection and role inducement.

[0024] In view of this, on the one hand, embodiments of the present invention provide a secure input decision method for interaction with a large language model, comprising: monitoring user behavior data in a conversation window and acquiring text data input by the user in the conversation window; performing sensitivity identification on the text data to determine the text sensitivity level; performing semantic identification on the text data to determine whether there is a potential illegal intent; comparing the behavior data with the user's historical behavior baseline to determine the user's behavior abnormality; generating an input decision based on the text sensitivity level, the potential illegal intent, and the behavior abnormality; and executing an output action based on the input decision, wherein the output action includes at least refusing to output the text data to the large language model or performing de-identification processing on the text data to obtain de-identified text data, and outputting the de-identified text data to the large language model.

[0025] The present invention provides a secure input decision-making method for large language model interaction. By integrating a three-dimensional protection system at the content, semantic, and behavioral levels, it achieves full-scenario coverage from surface violations to covert attacks, effectively solving the problem of missed detections in single-dimensional detection, and improving the civility and friendliness of the network environment.

[0026] In some embodiments, the secure input decision method for large language model interaction provided by the present invention can be executed by any electronic device 20 with data processing capabilities, such as a general-purpose computer, personal computer, laptop computer, switch, or tablet computer, etc. The specific implementation of the electronic device 20 is not limited here.

[0027] Figure 1 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention is shown. The electronic device 20 includes a processor 210, a memory 220, and a communication interface 230.

[0028] Processor 210 may include one or more processing cores. Processor 210 connects to various parts within electronic device 200 using various interfaces and lines, and performs various functions and processes data of electronic device 200 by running or executing instructions, programs, code sets, or instruction sets stored in memory 220, and by calling data stored in memory 220. Optionally, processor 210 may be implemented using at least one of the following hardware forms: Central Processing Unit (CPU), Graphics Processing Unit (GPU), Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA).

[0029] The memory 220 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 220 may include a non-transitory computer-readable storage medium. The memory 220 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 220 may include a stored program area. The stored program area may store instructions for implementing an operating system, instructions for implementing at least one function, instructions for implementing the various method embodiments described above, etc.

[0030] Communication interface 230 is used to communicate with other devices, equipment or communication networks, such as data storage devices, image processing devices or Ethernet, wireless access network (RAN), wireless local area network (WLAN), etc.

[0031] In terms of physical implementation, the aforementioned devices (such as processor 210, memory 220, and communication interface 230) can each be devices within the same device (such as a laptop computer). Alternatively, at least two of these devices can be located within the same device, i.e., as different devices within the same device, similar to the deployment of devices or components in a distributed system.

[0032] It is understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device 20. In other embodiments of the present invention, the electronic device 20 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0033] The following description, in conjunction with the accompanying drawings, illustrates a secure input decision-making method for large language model interaction provided by an embodiment of the present invention.

[0034] like Figure 2 As shown, embodiments of the present invention provide a secure input decision method for large language model interaction, which may include, but is not limited to: S1: Monitor user behavior data in the session window and obtain the text data entered by the user in the session window.

[0035] In specific implementation, the session window in this embodiment of the invention refers to a session window that communicates with the user and can interact with them. This embodiment of the invention provides two interface protocols: RESTful API (HTTP / HTTPS) and WebSocket, and supports JSON format data packets containing information such as user_id, session_id, timestamp, and text_content.

[0036] In one feasible implementation, after obtaining the text data input by the user in the session window, the present invention can preprocess the text data, and the preprocessing may include, but is not limited to, the following: The encoding of the text data is detected and converted into a unified encoding. The length of the text data is validated, and text data exceeding a preset length is discarded; The text data is formatted in a basic way to obtain a standard text data package.

[0037] In specific implementation processes, the encoding methods of the text data obtained in the embodiments of the present invention may vary, such as GBK, UTF-8 or Unicode. The embodiments of the present invention uniformly convert the encoding of the text data to UTF-8 encoding.

[0038] In the specific implementation process, the text data in this embodiment of the invention is formatted in a basic manner, which may include, but is not limited to, removing invisible characters (such as \x00) and extra spaces from the text, and performing case normalization, etc., which are not limited here.

[0039] This invention converts the acquired text data into a standard text data packet by performing unified encoding, length verification, and basic formatting preprocessing, which facilitates subsequent data processing.

[0040] S2: Perform sensitivity identification on the text data to determine the text sensitivity level.

[0041] In one feasible implementation, the sensitivity identification of the text data and the determination of the text sensitivity level described in this embodiment of the invention may include, but is not limited to: The Aho-Corasick automaton multi-pattern matching algorithm is used to identify sensitive words in the standard text data packet to obtain the sensitive words in the standard text data packet; The identified sensitive words are matched with preset sensitive words in a preset sensitive word classification database to obtain the sensitivity level of the sensitive words, and the sensitivity level of the sensitive words is used as the sensitivity level of the text. The sensitive word classification database includes at least L1, L2 and L3 levels. L1 level represents serious violations, L2 level represents general violations, and L3 level is a custom level.

[0042] In its implementation, the Aho-Corasick automaton multi-pattern matching algorithm constructs a finite state machine, enabling it to simultaneously match all predefined keywords in O(n) time complexity.

[0043] It should be noted that, in addition to the AC automata multi-pattern matching algorithm, other algorithms such as KMP and Rabin-Karp can also be used for sensitive word identification in the embodiments of the present invention, and no limitation is made here.

[0044] In specific implementation, the sensitive word classification database in this embodiment of the invention is classified according to the sensitivity of keywords. For example, keywords involving extreme speech belong to L1 level (serious violation), while keywords such as profanity, demeaning ability, and physical attacks belong to L2 level (general violation). Furthermore, this embodiment of the invention sets a customizable L3 level, allowing users to set sensitive words and sensitivity levels according to their actual needs. For example, research and development companies can define confidential research and development sensitive words in L3 level and then set the sensitivity level of those words, thereby expanding its applicability.

[0045] It should be noted that the L1 and L2 level thesauruses in this embodiment of the invention support hot updates, such as receiving incremental update packages (diff packages) from the thesaurus management server through a publish-subscribe model, etc., which are not limited here. This embodiment of the invention uses hot updates to update the L1 and L2 level thesauruses, enabling rapid response and identification of emerging sensitive content, and improving its applicability.

[0046] S3: Perform semantic recognition on the text data to determine whether there is any potential illegal intent.

[0047] In one feasible implementation, the potential illegal intent in this embodiment of the invention may include, but is not limited to, potential intentions and malicious statements. The step of performing semantic recognition on the text data to determine whether a potential illegal intent exists includes: The standard text data packet is subjected to feature extraction to obtain a semantic feature vector; The semantic feature vector is input into a pre-trained intention recognition model to identify potential intentions and output the potential intention probability. The semantic feature vector is input into a pre-trained speech recognition model to identify malicious speech and output the probability of malicious speech.

[0048] In one feasible implementation, the feature extraction of the standard text data packet to obtain a semantic feature vector in this embodiment of the invention includes: The standard text data packet is input into the pre-trained Llama-Prompt-Guard model, which outputs the semantic feature vector.

[0049] The Llama-Prompt-Guard model can understand the nuances and context of language, reduce false positives, cope with ever-changing and complex language bypass techniques, and, after being trained on large-scale data, can identify previously unseen harmful expression variants.

[0050] In the specific implementation process, when training the Llama-Prompt-Guard model, this embodiment of the invention collected a large number of prompt injection data samples to train it, so that it has a stronger ability to represent malicious prompts.

[0051] In the specific implementation process, the intention recognition model in the embodiments of the present invention adopts a binary classification model based on bert-base-chinese fine-tuning, and collects a large number of prompt injection data samples, such as tens of thousands of positive (normal questions) and negative (various prompt injection attacks) samples to train it, and outputs the probabilities of three types of intentions: "normal", "suspicious" and "malicious".

[0052] In specific implementation, the speech recognition model in this embodiment of the invention adopts an Erlangshen-Roberta fine-tuned multi-label classification model, setting multiple dimensions including insult and threat, and outputting a multi-dimensional probability vector to represent the probability of malicious speech. If any dimension exceeds a preset threshold, it is determined to be malicious speech.

[0053] In one feasible implementation, the potential violation intent in this embodiment of the invention further includes potential emotions, and the step of performing semantic recognition on the text data to determine whether a potential violation intent exists further includes: The semantic feature vector is input into a pre-trained emotion recognition model to perform emotion recognition and output the probability of potential emotions.

[0054] In specific implementation, the emotion recognition model of this invention can share core features with the speech recognition model and output the probability of three types of emotions: "positive", "neutral" and "negative".

[0055] In specific implementation, extreme emotions in the embodiments of the present invention can be used as auxiliary features for behavioral analysis, and are not limited here.

[0056] S4: Compare the behavioral data with the user's historical behavioral baseline to determine the degree of behavioral abnormality of the user.

[0057] In one feasible implementation, the step of comparing the behavioral data with the user's historical behavioral baseline to determine the user's behavioral abnormality in this embodiment of the invention may include, but is not limited to: Extract behavioral features from the behavioral data and construct a behavioral feature vector, wherein the behavioral features include request frequency features, request content features, and behavioral sequence features; The behavioral feature vector is compared with the user's historical behavioral baseline to determine the degree of behavioral abnormality.

[0058] In specific implementation, the request frequency feature in this embodiment of the invention can be the total number of requests within a specified time period, such as the number of requests in the last 1 minute / 5 minutes / 7 minutes, etc., and is not limited here. The request content feature in this embodiment of the invention can include the number of times sensitive words are triggered, the probability of potential intent, and the probability of malicious speech. The behavioral sequence feature in this embodiment of the invention can be a transition matrix for calculating request types, such as the sequence probability of "query -> modify -> system instruction".

[0059] In specific implementation, in order to further identify user behavior and obtain deeper potential anomalies, embodiments of the present invention may, but are not limited to, construct the behavioral sequence features into a graph structure, where nodes represent requests and codes represent the order of requests, and then use graph embedding learning algorithms (such as GraphSAGE) to automatically learn node representations, thereby capturing deep dependencies and potential anomalies in the behavioral sequence.

[0060] In specific implementation, this embodiment of the invention maintains a dynamic behavioral baseline for each user. This baseline adaptively adjusts as user behavior changes. Specifically, it continuously updates the mean and standard deviation of user behavioral characteristics using an exponentially weighted moving average (EWMA) algorithm to ensure that the baseline reflects recent behavioral trends. During the anomaly detection phase, an improved standardized deviation (Z-Score) algorithm is used to calculate a comprehensive anomaly score for the user's current behavior. This score measures the degree of deviation between the current behavioral characteristics and the dynamic baseline, i.e., the degree of behavioral anomaly.

[0061] S5: Generate an input decision based on the text sensitivity level, the potential violation intent, and the abnormality of the behavior.

[0062] In specific implementation, embodiments of the present invention may, but are not limited to, adopt a hybrid decision-making mechanism that combines the Drools rule engine with a lightweight XGBoost model, collect historical decision data and result feedback, train the XGBoost model to fine-tune and optimize the results of the rule engine, so as to handle gray areas that are difficult to cover by rules.

[0063] In specific implementation, the Drools rules in this embodiment of the invention may include, but are not limited to: IF Keyword Match Level = L1 THEN Decision = "Reject"; If the probability of potential intent is greater than 0.9 and the degree of behavioral abnormality is greater than 2.5, then the decision is to "reject and block the user". ELSE IF Probability of malicious speech > 0.7 THEN Decision="Respond after replacing content", etc., without any restrictions.

[0064] S6: Execute an output action based on the input decision. The output action includes at least refusing to output the text data to the large language model or performing desensitization processing on the text data to obtain desensitized text data, and then outputting the desensitized text data to the large language model.

[0065] In one feasible implementation, the text data desensitization processing described in this embodiment of the invention to obtain desensitized text data may include, but is not limited to: Based on the category of sensitive words, the sensitive words are desensitized using preset desensitization rules to obtain desensitized text data; The desensitization rules include: Replace named entities with generic tags; Partial masking is applied to the string of numbers.

[0066] In specific implementation, embodiments of the present invention may, but are not limited to, using the LAC (Lexical Analysis of Chinese) lexical analysis tool to perform named entity recognition (NER) to identify personal names (PER), place names (LOC), organization names (ORG), etc. in the text, without limitation.

[0067] In specific implementation, the replacement of named entities with generic tags in the embodiments of the present invention may include, but is not limited to: The named entities are differentiated according to entity type and risk level. For example: High-risk entities (such as specific person names) should be directly replaced with [REDACTED]. For general entities, replace them with [PERSON] or [LOCATION] depending on their category; no specific restrictions are imposed here.

[0068] In the specific implementation process, for number strings with fixed patterns such as telephone numbers, ID card numbers, and bank card numbers, the embodiments of the present invention can use a regular expression engine (PCRE2) for recognition and partial masking (such as 138****1234) formatting.

[0069] In specific implementation, in addition to refusing to output the text data to the large language model and outputting desensitized text data, the embodiments of the present invention may also perform delayed response (such as introducing random delay (such as 500-2000ms) to interfere with automated attack scripts) or upgraded verification (such as triggering secondary verification (such as slider verification code)), etc., which are not limited here.

[0070] In specific implementation, after outputting the desensitized text data to the large language model, the embodiments of the present invention can also perform a secondary security scan on the response information of the large language model to ensure the security and compliance of the response information itself.

[0071] In specific implementation, embodiments of the present invention may use a lightweight classifier (such as a small language model or text classification model finely tuned based on BERT or RoBERTa architecture) to perform a secondary security scan on the response information of the large language model, without limitation.

[0072] In specific implementation, after performing the output action, the embodiments of the present invention may include, but are not limited to: Record the data of the generation and execution process of the input decision (including original input, intermediate results, and final decision) and synchronize it to the audit database to establish a closed-loop feedback mechanism. Mark misjudgment cases for regular iterative updates to all models and rule bases.

[0073] The secure input decision-making method for large language model interaction provided in this invention achieves full-scenario coverage from surface violations to covert attacks by integrating a three-dimensional protection system at the content, semantic, and behavioral levels. It effectively solves the problem of missed detections in single-dimensional detection and can improve the civility and friendliness of the network environment.

[0074] Based on the aforementioned secure input decision-making method for large language model interaction provided in the first aspect, embodiments of the present invention provide a secure input decision-making system for large language model interaction, such as... Figure 3 As shown, the secure input decision system for large language model interaction includes: The data acquisition module 110 is used to monitor the user's behavior data in the session window and acquire the text data entered by the user in the session window; The sensitive word recognition module 120 is used to perform sensitivity recognition on the text data and determine the sensitivity level of the text; The semantic analysis module 130 is used to perform semantic recognition on the text data to determine whether there is any potential illegal intent; The behavior analysis module 140 is used to compare the behavior data with the user's historical behavior baseline to determine the degree of abnormality of the user's behavior; The input decision module 150 is used to generate input decisions based on the text sensitivity level, the potential violation intent, and the abnormality of the behavior; The decision execution module 160 is used to execute an output action based on the input decision, wherein the output action includes at least refusing to output the text data or outputting the text data after desensitization.

[0075] Based on the secure input decision-making method for large language model interaction provided in the first aspect, this embodiment of the invention also provides a storage medium storing a computer-executable program. The computer-executable program is used to cause a computer to execute the secure input decision-making method for large language model interaction as described in any implementation of the first aspect. Explanations of the relevant content and descriptions of the beneficial effects of any of the computer-readable storage media provided above can be found in the corresponding embodiments described above, and will not be repeated here.

[0076] Those skilled in the art will understand that the program for implementing all or part of the steps of the above embodiments, which can be executed by a program instructing related hardware, can be stored in a computer-readable storage medium. The storage medium mentioned above can be a read-only memory, a random access memory, etc. The processing unit or processor mentioned above can be a central processing unit, a general-purpose processor, an application-specific integrated circuit (ASIC), a microprocessor (DSP), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof.

[0077] This invention also provides a computer program product containing instructions that, when executed on a computer, cause the computer to perform any of the methods described in the above embodiments. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this invention is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., SSD), etc.

[0078] It should be noted that the devices for storing computer instructions or computer programs provided in the embodiments of the present invention, such as, but not limited to, the aforementioned memory, computer-readable storage medium, and communication chip, are all non-transitory. Those skilled in the art should recognize that the functions described in the embodiments of the present invention in one or more of the above examples can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable storage medium or transmitted as one or more instructions or code on a computer-readable storage medium. Computer-readable storage media include computer storage media and communication media, wherein communication media include any medium that facilitates the transmission of computer programs from one place to another. Storage media can be any available medium accessible to general-purpose or special-purpose computers.

[0079] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A secure input decision-making method for large language model interaction, characterized in that, include: Monitor user behavior data in the session window and obtain the text data entered by the user in the session window; The text data is subjected to sensitivity identification to determine the text sensitivity level; Semantic recognition is performed on the text data to determine whether there is any potential illegal intent; The behavioral data is compared with the user's historical behavioral baseline to determine the degree of behavioral abnormality of the user; Input decisions are generated based on the text sensitivity level, the potential violation intent, and the degree of behavioral anomaly. Based on the input decision, an output action is executed, the output action including at least refusing to output the text data to the large language model or performing de-identification processing on the text data to obtain de-identified text data, and outputting the de-identified text data to the large language model.

2. The secure input decision method for large language model interaction according to claim 1, characterized in that, After obtaining the text data entered by the user in the session window, the method further includes: The encoding of the text data is detected and converted into a unified encoding. The length of the text data is validated, and text data exceeding a preset length is discarded; The text data is formatted in a basic way to obtain a standard text data package.

3. The secure input decision method for large language model interaction according to claim 2, characterized in that, The process of performing sensitivity identification on the text data and determining the text sensitivity level includes: The standard text data packet is identified by using the AC automaton multi-pattern matching algorithm to obtain the sensitive words in the standard text data packet. The identified sensitive words are matched with preset sensitive words in a preset sensitive word classification database to obtain the sensitivity level of the sensitive words, and the sensitivity level of the sensitive words is used as the sensitivity level of the text. The sensitive word classification database includes at least L1, L2 and L3 levels. L1 level represents serious violations, L2 level represents general violations, and L3 level is a custom level.

4. The secure input decision method for large language model interaction according to claim 2, characterized in that, The potential violation intent includes potential intentions and malicious statements. The step of performing semantic recognition on the text data to determine whether a potential violation intent exists includes: The standard text data packet is subjected to feature extraction to obtain a semantic feature vector; The semantic feature vector is input into a pre-trained intention recognition model to identify potential intentions and output the potential intention probability. The semantic feature vector is input into a pre-trained speech recognition model to identify malicious speech and output the probability of malicious speech.

5. The secure input decision method for large language model interaction according to claim 4, characterized in that, The step of extracting features from the standard text data packet to obtain a semantic feature vector includes: The standard text data packet is input into the pre-trained Llama-Prompt-Guard model, which outputs the semantic feature vector.

6. The secure input decision method for large language model interaction according to claim 4, characterized in that, The potential violation intent also includes potential emotions, and the step of performing semantic recognition on the text data to determine whether a potential violation intent exists further includes: The semantic feature vector is input into a pre-trained emotion recognition model to perform emotion recognition and output the probability of potential emotions.

7. The secure input decision method for large language model interaction according to claim 1, characterized in that, The step of comparing the behavioral data with the user's historical behavioral baseline to determine the user's behavioral abnormality includes: Extract behavioral features from the behavioral data and construct a behavioral feature vector, wherein the behavioral features include request frequency features, request content features, and behavioral sequence features; The behavioral feature vector is compared with the user's historical behavioral baseline to determine the degree of behavioral abnormality.

8. The secure input decision method for large language model interaction according to claim 1, characterized in that, The process of de-identifying the text data to obtain de-identified text data includes: Based on the category of sensitive words, the sensitive words are desensitized using preset desensitization rules to obtain desensitized text data; The desensitization rules include: Replace named entities with generic tags; Partial masking is applied to the string of numbers.

9. A secure input decision system for large language model interaction, characterized in that, include: The data acquisition module is used to monitor user behavior data in the session window and acquire the text data entered by the user in the session window; The sensitive word recognition module is used to identify the sensitivity of the text data and determine the sensitivity level of the text; The semantic analysis module is used to perform semantic recognition on the text data to determine whether there is any potential illegal intent; The behavior analysis module is used to compare the behavior data with the user's historical behavior baseline to determine the degree of abnormality of the user's behavior; The input decision module is used to generate input decisions based on the text sensitivity level, the potential violation intent, and the abnormality of the behavior; The decision execution module is used to execute output actions based on the input decision, and the output actions include at least refusing to output the text data or outputting the text data after desensitization.

10. An electronic device, characterized in that, include: A memory, one or more processors; the memory is coupled to the processors; wherein the memory stores computer program code, the computer program code including computer instructions, which, when executed by the processor, cause the electronic device to perform a secure input decision method for interaction with a large language model as described in any one of claims 1 to 8.

Images