Data stealing attack processing method and device, storage medium and electronic equipment

Abstract

The invention discloses a data stealing attack processing method and device, a storage medium and electronic equipment, and the method comprises the steps: obtaining the input request content of a user side for a target large language model, carrying out the data stealing attack detection of the user side, and obtaining a stealing attack detection result, and controlling the target large language model to perform content generation processing based on the input request content to obtain initial output response content, and if the steal attack detection result is a steal attack type, performing content modification processing on the initial output response content to obtain target output response content, and performing request reply processing on the user side based on the target output response content, and if the stealing attack detection result is a normal request type, performing request reply processing on the user side based on the initial output response content.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method, device, storage medium and electronic device for processing data theft attacks. Background Art

[0002] With the rapid development of computer technology, Large Language Models (LLMs) can handle a variety of natural language tasks in daily life, such as text classification, question answering, dialogue, etc., and can provide users with intelligent and personalized services;

[0003] Large language models provide users with a series of functional services, but they also face security threats from data theft attacks. Data theft attacks mainly refer to model theft attacks on large language models. Data theft attacks often attempt to copy or approximately reconstruct a model with similar functions from a trained target large language model through multiple outputs to provide corresponding functional services without obtaining the internal information or training data of the original model. For the service maintenance side and basic user side of large language models, it is very likely to cause data privacy damage and lead to data leakage security threats. Summary of the invention

[0004] This specification provides a data theft attack processing method, device, storage medium and electronic device, the technical solution is as follows:

[0005] In a first aspect, this specification provides a method for processing a data theft attack, the method comprising:

[0006] Obtain the input request content of the user end for the target large language model;

[0007] Performing data theft attack detection on the user terminal to obtain a theft attack detection result, and controlling the target large language model to perform content generation processing based on the input request content to obtain initial output response content;

[0008] If the theft attack detection result is a theft attack type, modifying the initial output response content to obtain target output response content, and performing a request reply process on the user end based on the target output response content;

[0009] If the theft attack detection result is a normal request type, a request reply process is performed on the user terminal based on the initial output response content.

[0010] In a second aspect, this specification provides a data theft attack processing device, the device comprising:

[0011] A content acquisition module is used to obtain the input request content of the user end for the target large language model;

[0012] An attack detection module, used to perform data theft attack detection on the user terminal to obtain a theft attack detection result, and control the target large language model to perform content generation processing based on the input request content to obtain an initial output response content;

[0013] An attack processing module, configured to, if the theft attack detection result is a theft attack type, modify the initial output response content to obtain a target output response content, and perform a request reply process on the user terminal based on the target output response content;

[0014] The attack processing module is further configured to perform a request reply process on the user terminal based on the initial output response content if the theft attack detection result is a normal request type.

[0015] In a third aspect, the present specification provides a computer storage medium, wherein the computer storage medium stores at least one instruction, wherein the instruction is suitable for being loaded by a processor and executing the method steps of one or more embodiments of the present specification.

[0016] In a fourth aspect, the present specification provides a computer program product, wherein the computer program product stores at least one instruction, wherein the instruction is suitable for being loaded by a processor and executing the method steps of one or more embodiments of the present specification.

[0017] In a fifth aspect, the present specification provides an electronic device, which may include: a processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the method steps of one or more embodiments of the present specification.

[0018] The beneficial effects brought by the technical solutions provided by some embodiments of this specification include at least:

[0019] In one or more embodiments of the present specification, the electronic device obtains the input request content of the user terminal for the target large language model, performs data theft attack detection on the user terminal to obtain the theft attack detection result, and controls the target large language model to perform content generation processing based on the input request content to obtain the initial output response content. If the theft attack detection result is a theft attack type, the initial output response content is modified to obtain the target output response content, and the user terminal is requested to reply based on the target output response content. If the theft attack detection result is a normal request type, the user terminal is requested to reply based on the initial output response content. By real-time monitoring of the input request content of the user terminal and performing data theft attack detection on the user terminal, the data security related to the large language model is effectively improved, and the model theft attack can be timely identified and prevented, while minimizing the impact on normal users, and the overall security and user experience of services related to the large language model are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in this specification or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 This is a scenario diagram of a data theft attack processing system provided in this specification;

[0022] Figure 2 This is a flowchart of a method for processing data theft attacks provided in this manual;

[0023] Figure 3 This is a schematic diagram of a data theft attack scenario provided in this manual;

[0024] Figure 4 This is a flow chart of data theft attack detection provided in this manual;

[0025] Figure 5 This is a flow chart of a stealing attack detection model provided in this specification;

[0026] Figure 6 It is a flowchart of a content modification process provided in this specification;

[0027] Figure 7 It is a data theft attack processing device provided in this manual;

[0028] Figure 8 It is a structural schematic diagram of an electronic device provided in this manual. DETAILED DESCRIPTION

[0029] The following will be combined with the drawings in this specification to clearly and completely describe the technical solutions in this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this specification.

[0030] In the description of this specification, it should be understood that the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In the description of this specification, it should be noted that, unless otherwise clearly specified and limited, "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices. For those of ordinary skill in the art, the specific meanings of the above terms in this specification can be understood in specific circumstances. In addition, in the description of this specification, unless otherwise specified, "multiple" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B, which can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are an "or" relationship.

[0031] In the related art, in order to prevent data theft attacks, the access control method of API interface can be restricted: the access rights and frequency of LLMs can be limited to reduce the amount of data that attackers can use. Access control and identity authentication methods can be used to ensure that only verified users can access large models, increasing the difficulty of attacks. However, these methods of preventing data theft attacks cannot completely stop determined attackers, who may find ways to bypass controls, which makes them inadequately respond to complex data theft attack methods. Based on this, some of the related art methods of preventing data theft attacks have certain limitations.

[0032] The present specification is described in detail below with reference to specific embodiments.

[0033] See also Figure 1 , is a scenario diagram of a data theft attack processing system provided in this specification. Figure 1 As shown, the data theft attack processing system may include at least a client cluster and a service platform 100 .

[0034] The client cluster may include at least one client, such as Figure 1 As shown, it specifically includes client 1 corresponding to user 1, client 2 corresponding to user 2, ..., client n corresponding to user n, where n is an integer greater than 0.

[0035] Each client in the client cluster may be an electronic device with communication function, including but not limited to: wearable device, handheld device, personal computer, tablet computer, vehicle-mounted device, smart phone, computing device or other processing device connected to a wireless modem, etc. Electronic devices may be called different names in different networks, such as: user equipment, access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication device, user agent or user device, cellular phone, cordless phone, personal digital assistant (PDA), electronic device in 5G network or future evolution network, etc.

[0036] The service platform 100 can be a separate server device, such as a rack-mounted, blade, tower, or cabinet-mounted server device, or a workstation, mainframe computer, or other hardware device with strong computing power; it can also be a server cluster composed of multiple servers, and the servers in the service cluster can be composed in a symmetrical manner, wherein each server has equivalent functions and status in the transaction link, and each server can provide services to the outside independently, and the independent service can be understood as not requiring the assistance of other servers.

[0037] In one or more embodiments of the present specification, the service platform 100 may establish a communication connection with at least one client in the client cluster, and complete data interaction during the data theft attack processing process, such as online transaction data interaction, based on the communication connection.

[0038] It should be noted that the service platform 100 and at least one client in the client cluster establish a communication connection through a network for interactive communication, wherein the network can be a wireless network or a wired network, the wireless network includes but is not limited to a cellular network, a wireless local area network, an infrared network or a Bluetooth network, and the wired network includes but is not limited to Ethernet, a universal serial bus (USB) or a controller local area network. In one or more embodiments of the specification, technologies and / or formats including Hyper Text Mark-up Language (HTML), Extensible Markup Language (XML), etc. are used to represent data (such as a target compressed package) exchanged through a network. In addition, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPsec), etc. can also be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technologies can also be used to replace or supplement the above data communication technologies.

[0039] The data theft attack processing system embodiment provided in this specification and the data theft attack processing method in one or more embodiments belong to the same concept. The execution subject corresponding to the data theft attack processing method involved in one or more embodiments of the specification can be the above-mentioned service platform 100; the execution subject corresponding to the data theft attack processing method involved in one or more embodiments of the specification can also be the electronic device corresponding to the client, which is specifically determined based on the actual application environment. The embodiment of the data theft attack processing system and its implementation process can be detailed in the following method embodiment, which will not be repeated here.

[0040] based on Figure 1 The scenario diagram is shown, and the data theft attack processing method provided by one or more embodiments of this specification is introduced in detail below.

[0041] See also Figure 2 , is a flow chart of a data theft attack processing method for one or more embodiments of this specification, which can be implemented by a computer program and can be run on a data theft attack processing device based on the von Neumann system. The computer program can be integrated into an application or run as an independent tool application. The data theft attack processing device can be a service platform or a terminal.

[0042] Specifically, the data theft attack processing method includes:

[0043] S102: Obtaining input request content of the user terminal for the target large language model;

[0044] The target large language model can be understood as being obtained based on the basic large language model, such as the ChatGPT model, the Wenxin Yiyan model, the Tongyi Qianwen model, etc. The large language model that provides transaction services to users for data theft attack detection can be directly used as the target large language model. The type of the target large language model is not limited in this specification.

[0045] The input request content can be understood as the description information that the user expects the target large language model to generate content. For example, the image style description, image content description, image specification description and other description information of the image that is expected to be generated all belong to the input request content. The input request content is used by the user end to instruct the target large language model to generate content that meets the user's expectations.

[0046] Specifically, the input request content is a question or statement provided by the user to the target large language model, which is used to guide the target large language model to generate relevant replies or responses. The target large language model usually generates content in response to the input request content and generates the most relevant content or answer.

[0047] In a feasible implementation, in order to prevent data theft attacks, a request monitoring system can be set before the target large language model. The user-side input to the target large language model must first pass through the request monitoring system. The request monitoring system can first receive the input request content from the user side, and the request monitoring system inputs the input request content into the target large language model.

[0048] S104: performing data theft attack detection on the user terminal to obtain a theft attack detection result, and controlling the target large language model to perform content generation processing based on the input request content to obtain initial output response content;

[0049] Indicatively, by analyzing the behavior and pattern of user requests on the user side, it is detected whether there is a possibility of a data theft attack, thereby obtaining a theft attack detection result.

[0050] In a feasible implementation, request behavior features can be extracted from the user end, and the request behavior features include request frequency features, request content diversity features, and time pattern features. By identifying data theft attack behaviors based on the request behaviors, the theft attack detection results can be obtained.

[0051] In a feasible implementation, a normal request behavior pattern of normal request behavior and an abnormal request behavior pattern of abnormal request behavior can be established in advance, and then the current target request behavior pattern is extracted for the current user terminal, and then the target request behavior pattern is compared with the normal request behavior pattern, and the target request behavior pattern is compared with the abnormal request behavior pattern in real time to identify data theft attack behavior.

[0052] Schematically, in order to prevent data theft attacks, a request monitoring system and a risk assessment system for theft attack detection can be set before the target large language model. The risk assessment system performs data theft attack detection on the user terminal to obtain the theft attack detection result to evaluate whether the request is risky. At the same time, the input request content is sent to the target large language model, and the target large language model is controlled to perform content generation processing based on the input request content to obtain the initial output response content. The target large language model generates content in response to the input request content, and generates the most relevant content or answer as the initial output response content. In this specification, the initial output response content is not directly used as the final output response to the user terminal, but the initial output response content is adjusted based on the theft attack detection result to avoid data theft attacks.

[0053] S106: If the theft attack detection result is a theft attack type, modify the initial output response content to obtain target output response content, and perform a request reply process on the user terminal based on the target output response content;

[0054] For example, see Figure 3 , Figure 3 This is a schematic diagram of a data theft attack scenario. Figure 3In the process, (1) the user initiates a model request: the user initiates an input request content to the target large language model through the API interface; (2) the request monitoring system forwards the request to the risk assessment system after receiving it and forwards the request; (3) the risk assessment system evaluates: the user terminal is subjected to data theft attack detection detection to obtain the theft attack detection result, so as to evaluate whether this request has risk, and the input request content is sent to the target large language model; (4) the model runs: the target large language model runs normally, controls the target large language model to generate content based on the input request content to obtain the initial output response content, and returns the initial output response content to the risk assessment system; (5) the risk assessment system detects the type of theft attack detection result. If the theft attack detection result is the theft attack type, the initial output response content of the model is forwarded to the dynamic output modification system. The system can implement strategies to modify the content of the initial output response content based on the theft attack detection results, such as introducing noise, providing fuzzy results or other forms of output modification to ensure that the target large language model does not leak key information or integrity of the model; if the theft attack detection result is a normal request type, the initial output response content is sent to the request monitoring system to process the request reply to the user end. (6) If the theft attack detection result is a theft attack type, after the dynamic output modification system completes the processing, the adjusted result is returned to the risk assessment system. (7) The risk assessment system re-evaluates the modified result to ensure that the modification does not damage the normal user experience, and returns the final "output result-target output response content" to the request monitoring system. (8) The request monitoring system sends this "response-target output response content" that has been protected and adjusted to the attacker. Since the response received by the attacker has undergone dynamic output modification, the data cannot be effectively used to reverse engineer the original model, thereby protecting the data related to the large language model from theft.

[0055] S108: If the theft attack detection result is a normal request type, a request reply process is performed on the user terminal based on the initial output response content.

[0056] Further, after determining that the theft attack detection result is a normal request type, the electronic device can directly send the initial output response content to the user end, and the user end displays it to the user, and the user can perform a content generation dialogue process for the output response content.

[0057] In one or more embodiments of the present specification, the electronic device obtains the input request content of the user terminal for the target large language model, performs data theft attack detection on the user terminal to obtain the theft attack detection result, and controls the target large language model to perform content generation processing based on the input request content to obtain the initial output response content. If the theft attack detection result is a theft attack type, the initial output response content is modified to obtain the target output response content, and the user terminal is requested to reply based on the target output response content. If the theft attack detection result is a normal request type, the user terminal is requested to reply based on the initial output response content. By real-time monitoring of the input request content of the user terminal and performing data theft attack detection on the user terminal, the data security related to the large language model is effectively improved, and the model theft attack can be timely identified and prevented, while minimizing the impact on normal users, and the overall security and user experience of services related to the large language model are improved.

[0058] See also Figure 4 , Figure 4 This is a flow chart of a data theft attack detection process proposed in one or more embodiments of this specification. When performing the data theft attack detection on the user terminal to obtain the theft attack detection result, the following steps can be referred to:

[0059] S202: Obtaining user request behavior data of the user terminal for the target large language model;

[0060] In an illustrative manner, input request content from a user terminal is received, and a request behavior log record for the user terminal is obtained, user request behavior data is determined from the request behavior log record, and data theft attack detection is performed based on the user request behavior data and the input request content to obtain a theft attack detection result.

[0061] Exemplarily, an interface, such as an API endpoint, is first set to receive input request content from a user end. The input request content can be in the form of text, voice input, or other data.

[0062] Exemplarily, log settings are performed in advance, and a function is set to automatically record all request behavior logs entering this interface. The request behavior log record includes the specific content of the request, timestamp, user identification information, etc. Then, after the user-side access interface inputs the request content, the user-side request behavior data is logged. The request behavior log record includes but is not limited to request details, request session information, user identity information, device information, request time information, etc. The request behavior log record is parsed by the user-side request behavior, and the collected request behavior log records are analyzed to identify the user's request behavior pattern data to generate user request behavior data. User request behavior data includes but is not limited to request frequency data, request periodicity data, request diversity data, request time data, session length data, request content complexity data, geographic location and device information data, etc. User request behavior data can characterize the comprehensive portrait of user behavior on the user side to a certain extent, which can facilitate subsequent monitoring, security analysis and improvement of user experience.

[0063] S204: Using a preset risk assessment indicator to perform a theft attack indicator evaluation on the user request behavior data to obtain a theft attack indicator detection result;

[0064] Indicatively, potential theft activities can be detected by setting specific theft attack indicators and indicator thresholds, thereby generating theft attack indicator detection results;

[0065] Optionally, the stealing attack indicators include, but are not limited to, one or more fittings of a request frequency indicator, a request pattern consistency indicator, a session duration indicator, a device address anomaly indicator, a request content complexity indicator, and the like;

[0066] Request frequency metric: A very high request frequency may indicate that an automated script or program is trying to steal model data;

[0067] Request pattern consistency indicator: If a large number of requests are highly consistent in content or structure, it may be an attempt at theft;

[0068] Session duration metrics: Unusually short or long session times may indicate unusual usage patterns;

[0069] Indicators of unusual device addresses: Requests from atypical geographic locations or frequently changing IP addresses may be signs of attempts to hide identity.

[0070] Request content complexity indicator: The request content is very simple or repetitive, which may indicate an attempt to use the model to generate data to train another model.

[0071] Optionally, a threshold is set for each theft attack indicator. When the theft attack indicator value representing the user behavior exceeds the threshold, the system marks the user's request behavior as a potential theft risk from the theft attack indicator dimension and uses it as the result detail data of the theft attack indicator detection result.

[0072] Furthermore, the thresholds are set based on historical data analysis and combined with the security requirements of specific application scenarios.

[0073] Indicatively, the theft attack detection result is generated based on the result details data of all indicator evaluations. For example, the theft attack risk rating can be performed based on the result details data. The theft attack risk rating can determine whether the theft attack detection result is a theft attack type or a normal request type. The generated theft attack detection result includes the theft attack detection type and result details data. These ratings reflect the degree of threat that user behavior may pose to system security.

[0074] S206: Obtaining a theft attack detection result based on the theft attack indicator detection result.

[0075] Indicatively, the theft attack indicator detection result may be regarded as a theft attack detection result.

[0076] S208: performing data theft attack detection using a theft attack detection model based on the user request behavior data to obtain a theft attack model detection result;

[0077] In this specification, the theft attack detection based on the theft attack detection model utilizes machine learning technology. By learning and training a large number of samples of known theft attack threats, a theft attack detection model is constructed to identify new request input security risks.

[0078] The model input of the theft attack detection model is the user request behavior data, and the model output of the theft attack detection model is the theft attack model detection result. The theft attack model detection result may include the theft attack detection type and result detail data. The result detail data may be understood as interpretation data of a type determination process for the theft attack detection type.

[0079] In actual applications, user request behavior data is input into the theft attack detection model to perform data theft attack detection to obtain the theft attack model detection result, and the theft attack model detection result is output;

[0080] In a feasible implementation, the model training process of the theft attack detection model is explained as follows:

[0081] A2: Create an initial stealing attack detection model;

[0082] Model creation: Create an initial theft attack detection model based on the machine learning model;

[0083] Optionally, the machine learning model may be obtained by adopting (basic) large language model (LLMs) technology, and the machine learning model may be implemented by fitting one or more models such as convolutional neural network (CNN) model, deep neural network (DNN) model, recurrent neural network (RNN) model, embedding model, gradient boosting decision tree (GBDT) model, logistic regression (LR) model, etc.

[0084] A4: Obtain user request behavior data samples User request behavior data samples carry the theft attack model detection result label;

[0085] Training data preparation: Based on the platform's historical model content, the database and network are input in advance to obtain a large number of user request behavior data samples, and a theft attack model detection result label is annotated on the user request behavior data sample;

[0086] A6: Input the user request behavior data samples into the initial stealing attack detection model for at least one round of model training;

[0087] A8: During each round of model training, the predicted theft attack model detection result output by the initial theft attack detection model based on the user request behavior data sample is determined, and the model parameters of the initial theft attack detection model are fine-tuned based on the predicted theft attack model detection result and the theft attack model detection result label to obtain a trained theft attack detection model.

[0088] Model training process: In the process of model forward training, user request behavior data samples are input into the initial theft attack detection model for at least one round of model training. The initial theft attack detection model identifies risks of user request behavior data samples and obtains the detection results of the predicted theft attack model. In the process of model reverse training, the model loss is calculated based on the predicted theft attack model detection results and the theft attack model detection result labels. Based on the model loss, the model parameters of the initial theft attack detection model are adjusted until the initial theft attack detection model completes the model end training conditions and obtains a trained theft attack detection model.

[0089] Optionally, the model loss may be obtained by using a model loss calculation formula in related technologies, such as a hinge loss calculation formula, a vector distance loss calculation formula, a cross entropy loss calculation formula, a contrast loss calculation formula, and the like.

[0090] Optionally, the model training end condition of the model may include, for example, the value of the loss function is less than or equal to a preset loss function threshold, the number of iterations reaches a preset number threshold, etc. The specific model training end condition can be determined based on actual conditions and is not specifically limited here.

[0091] See also Figure 5 , Figure 5 This is a flow chart of a stealing attack detection model proposed in one or more embodiments of this specification. To perform data stealing attack detection based on user request behavior data using the stealing attack detection model to obtain the stealing attack model detection result, the following method can be referred to:

[0092] S3002: Extracting request behavior description features based on the user request behavior data;

[0093] The request behavior description feature includes at least one of a request frequency feature, a request behavior pattern feature, a request content meta-information feature, and a request time feature;

[0094] S3004: Generate a request behavior description for the user terminal based on the request behavior description feature;

[0095] Exemplarily, a preset request behavior template may be used, and the request behavior description feature may be input into the preset request behavior template for data filling, so as to automatically generate a request behavior description for the user terminal;

[0096] S3006: Generate a data theft attack detection prompt word for the request behavior description, input the data theft attack detection prompt word and the request behavior description into a theft attack detection large model to perform data theft attack detection, and output the theft attack model detection result;

[0097] The theft attack detection large model is obtained by adapting the theft attack detection scenario using a basic large language model. The basic large language model may be, for example, a ChatGPT model, a Wenxinyiyan model, an LLMs model, etc. The type of the basic large language model is not limited in this specification.

[0098] The data theft attack detection prompt word can be understood as a descriptive text used to instruct the theft attack detection large model to perform theft attack detection processing based on the input data. The data theft attack detection prompt word can be a preset detection prompt word template. In actual applications, the data theft attack detection prompt word is automatically generated based on the detection prompt word template and combined with the request behavior description.

[0099] For example, the data theft attack detection prompt can be "Assuming you are a security expert, please help me conduct an in-depth theft attack analysis based on the request behavior description I entered;

[0100] Further, the data theft attack detection prompt word and the request behavior description are input into the theft attack detection large model to perform data theft attack detection, and the theft attack model detection result is output;

[0101] The following is an example of a model training process for a large stealing attack detection model:

[0102] Model creation: First, the basic large language model is obtained, and then the initial theft attack detection large model is constructed. The initial theft attack detection large model can be composed of a scenario adaptation module for the prompt adjustment scenario and a basic large language module based on the basic large language model.

[0103] Sample data acquisition: Obtain a large amount of sample combination data. The sample combination data consists of data theft attack detection prompt words and sample request behavior descriptions.

[0104] Sample data labeling: Based on the needs of the prompt adjustment scenario, an expert service is introduced to manually label the sample combination data with the corresponding theft attack model detection result label;

[0105] Model training process: The sample combination data is input into the initial theft attack detection large model for at least one round of model training. The initial theft attack detection model identifies the theft attack risk of the sample request behavior description in the sample combination data to obtain the predicted theft attack model detection result; in the model reverse training process, the model loss is calculated based on the predicted theft attack model detection result and the theft attack model detection result label. Based on the model loss, the parameters of the scene adaptation module in the initial theft attack detection large model are adjusted and the model parameters of the basic large language module are controlled unchanged until the model training end conditions are met to obtain the theft attack detection large model.

[0106] Indicatively, the pre-trained basic large language model can quickly be applied to a new theft attack detection scenario without retraining a new model. Only fine-tuning the basic large language model is required.

[0107] S210: Obtaining a theft attack detection result based on the theft attack model detection result;

[0108] Indicatively, the theft attack model detection result can be used as a theft attack detection result;

[0109] In some embodiments, the theft attack detection result may be obtained based on at least one of the theft attack model detection result and the theft attack indicator detection result.

[0110] In one or more embodiments of the present specification, the data security related to the large language model is effectively improved by real-time monitoring of the input request content of the user terminal and detecting data theft attacks on the user terminal. The model theft attacks can be identified and prevented in a timely manner, while minimizing the impact on normal users, thereby improving the security and user experience of services related to the large language model as a whole.

[0111] See also Figure 6 , Figure 6 This is a flow chart of a content modification process proposed in one or more embodiments of this specification. The content modification process of the initial output response content is performed to obtain the target output response content. The following method can be referred to:

[0112] S4002: Determine the theft attack information for the initial output response content based on the theft attack detection result, and modify the initial output response content using a response content modification system based on the theft attack information to obtain reference output response content;

[0113] The theft attack information such as theft attack type, intensity, attack source, etc. can characterize the attribute information of the theft attack, and the theft attack information of the initial output response content can be extracted from the result data of the theft attack detection result;

[0114] Schematically, a response content modification system is pre-set to dynamically adjust the initial output response content according to the theft attack information. The content modification process includes perturbing, simplifying or completely rewriting some parts of the output content to make it less effective for the theft attacker while maintaining its usefulness to normal users as much as possible.

[0115] Content modification processing may include adding noise, using synonyms, changing sentence structure, etc. The purpose of content modification processing is to reduce the direct usefulness of the output content while ensuring its actual value to end users.

[0116] Exemplarily, the step of modifying the initial output response content using a response content modification system based on the theft attack information to obtain reference output response content includes:

[0117] The theft attack information and the initial output response content are input into a response content modification system, and the risk-sensitive content and model kernel mechanism-related content for the initial output response content are determined by the response content modification system. Based on the risk-sensitive content and the model kernel mechanism-related content, the initial output response content is subjected to content modification processing to obtain a reference output response content.

[0118] Specifically, the response content modification system identifies risk-sensitive content on the initial output response content: the system analyzes the initial output response content in combination with the theft attack information, and parses and determines any information contained therein that may be considered sensitive. This information constitutes risk-sensitive content, which includes specific data points, technical details, or other content that can provide attackers with the ability to learn or copy models.

[0119] Specifically, the response content modification system "identifies the model's core mechanism-related content" for the initial output response content. The system will identify the core mechanism content that is directly related to the core algorithm or operating mechanism of the large model. For example, when the model generates a specific type of answer, it uses a unique algorithm logic or data processing method, which will be considered sensitive content. For example, when the model generates a specific type of answer, it reveals the model parameters;

[0120] Specifically, the response content modification system modifies the initial output response content based on the risk-sensitive content and the model core mechanism-related content to obtain the reference output response content, which can be done in the following ways:

[0121] Deleting or replacing sensitive information: Delete or replace information identified as high-risk to avoid direct exposure.

[0122] Abstraction and generalization method: Abstract the specific "risk-sensitive content and model core mechanism-related content" to reduce details while retaining answers that are useful to users.

[0123] Noise addition method: Add information noise to the risk-sensitive content and the model kernel mechanism-related content so that the output is still meaningful to users but not enough to train or accurately replicate the model.

[0124] In a feasible implementation, the response content modification system can be a response content modification model obtained through training based on a machine learning model. The theft attack information and the initial output response content are input into the response content modification model. The risk-sensitive content and model core mechanism-related content for the initial output response content are determined through the response content modification model. Based on the risk-sensitive content and the model core mechanism-related content, the initial output response content is modified to obtain a reference output response content.

[0125] The input of the response content modification model is the theft attack information and the initial output response content, and the output of the response content modification model is the reference output response content;

[0126] Furthermore, the content modification processing of the initial output response content based on the risk-sensitive content and the model kernel mechanism associated content to obtain the target output response content may be performed in the following manner:

[0127] B2: performing content noise processing on the risk-sensitive content and the model core mechanism-related content to obtain fuzzy response content;

[0128] B4: performing content update processing on the initial output response content based on the fuzzy response content to obtain reference output response content.

[0129] Using the above method, the responsive content modification system can effectively identify and process output content that may increase the risk of model theft, while ensuring that users can obtain valuable information, protecting the core technology and intellectual property rights of the model, and maintaining user trust and satisfaction.

[0130] S4004: Perform data theft attack detection on the reference output response content to obtain a theft attack detection result;

[0131] In a feasible implementation, a theft attack rule set can be pre-configured, and rule detection processing based on the theft attack rule set uses a pre-defined rule set to screen and filter potential security risks in the reference output response content. The risk rule information of these risk rule sets is based on known malicious data theft attack problems, malicious data theft attack instructions and other data theft attack features to define the risk element rule composition. By matching the reference output response content with several theft attack rules in the theft attack rule set, it is possible to quickly identify possible theft attack output threats and obtain the theft attack detection results;

[0132] In a feasible implementation, a data theft attack output detection model can be created and trained in advance based on a machine learning model. The data theft attack output detection model builds a data theft attack output detection model to identify new security risks by learning and training a large number of known security threat samples. The data theft attack output detection model can automatically learn the characteristics and patterns of different types of theft attack security threats, and classify and judge them according to the characteristics of the reference output response content.

[0133] Indicatively, the reference output response content may be input into a data theft attack output detection model to output the theft attack detection result;

[0134] S4006: If the theft attack detection result is a normal request type, use the reference request content as the target output response content;

[0135] S4008: If the theft attack detection result is a theft attack type, the reference request content is used as the initial output response content and the step of modifying the initial output response content using a response content modification system based on the theft attack information to obtain a reference output response content is executed.

[0136] In one or more embodiments of the present specification, by real-time monitoring of the input request content of the user terminal and detecting data theft attacks on the user terminal, and modifying and adjusting the response content, the data security related to the large language model is effectively improved, and model theft attacks can be identified and prevented in a timely manner, while minimizing the impact on normal users, thereby improving the security and user experience of large language model related services as a whole.

[0137] The following will be combined Figure 7 , the data theft attack processing device provided in this specification is introduced in detail. It should be noted that, Figure 7 The data stealing attack processing device shown is used to execute this instruction Figure 1 to Figure 6 For the convenience of explanation, only the parts related to this specification are shown. For specific technical details not disclosed, please refer to this specification. Figure 1 to Figure 6 The embodiment shown.

[0138] See also Figure 7 , which shows a schematic diagram of the structure of the data theft attack processing device of the present specification. The data theft attack processing device 1 can be implemented as all or part of the user terminal through software, hardware or a combination of both. According to some embodiments, the data theft attack processing device 1 includes a content acquisition module 11, an attack detection module 12 and an attack processing module 13, which are specifically used to:

[0139] The content acquisition module 11 is used to obtain the input request content of the user terminal for the target large language model;

[0140] An attack detection module 12 is used to perform data theft attack detection on the user terminal to obtain a theft attack detection result, and control the target large language model to perform content generation processing based on the input request content to obtain an initial output response content;

[0141] The attack processing module 13 is used for, if the theft attack detection result is a theft attack type, modifying the initial output response content to obtain a target output response content, and performing a request reply process on the user terminal based on the target output response content;

[0142] The attack processing module 13 is further configured to perform a request reply process on the user terminal based on the initial output response content if the theft attack detection result is a normal request type.

[0143] Optionally, the attack detection module 12 is used to:

[0144] Acquire user request behavior data of the user terminal for the target large language model;

[0145] A data theft attack detection is performed based on the user request behavior data to obtain a theft attack detection result.

[0146] Optionally, the attack detection module 12 is used to:

[0147] Using a preset risk assessment indicator to perform a theft attack indicator evaluation on the user request behavior data to obtain a theft attack indicator detection result;

[0148] A theft attack detection result is obtained based on the theft attack indicator detection result.

[0149] Optionally, the attack detection module 12 is used to:

[0150] Based on the user request behavior data, a theft attack detection model is used to perform data theft attack detection to obtain a theft attack model detection result;

[0151] The theft attack detection result is obtained based on the theft attack model detection result.

[0152] Optionally, the attack detection module 12 is used to:

[0153] Extracting request behavior description features based on the user request behavior data, wherein the request behavior description features include at least one of request frequency features, request behavior pattern features, request content meta-information features, and request time features;

[0154] Generate a request behavior description for the user terminal based on the request behavior description feature;

[0155] Generate a data theft attack detection prompt word for the request behavior description, input the data theft attack detection prompt word and the request behavior description into a theft attack detection large model to perform data theft attack detection, and output the theft attack model detection result;

[0156] The theft attack detection large model is obtained by adapting the theft attack detection scenario using a basic large language model.

[0157] Optionally, the attack processing module 13 is used to:

[0158] Determining the theft attack information for the initial output response content based on the theft attack detection result, and modifying the initial output response content using a response content modification system based on the theft attack information to obtain reference output response content;

[0159] Performing data theft attack detection on the reference output response content to obtain a theft attack detection result;

[0160] If the theft attack detection result is a normal request type, the reference request content is used as the target output response content;

[0161] If the theft attack detection result is a theft attack type, the reference request content is used as the initial output response content and the step of modifying the initial output response content using a response content modification system based on the theft attack information to obtain a reference output response content is performed.

[0162] Optionally, the attack processing module 13 is used to:

[0163] The theft attack information and the initial output response content are input into a response content modification system, and the risk-sensitive content and model kernel mechanism-related content for the initial output response content are determined by the response content modification system. Based on the risk-sensitive content and the model kernel mechanism-related content, the initial output response content is subjected to content modification processing to obtain a reference output response content.

[0164] It should be noted that the data theft attack processing device provided in the above embodiment only uses the division of the above functional modules as an example when executing the data theft attack processing method. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the data theft attack processing device provided in the above embodiment and the data theft attack processing method embodiment belong to the same concept, and the implementation process thereof is detailed in the method embodiment, which will not be repeated here.

[0165] The above serial numbers in this specification are for description only and do not represent the advantages or disadvantages of the embodiments.

[0166] In one or more embodiments of the present specification, the electronic device obtains the input request content of the user terminal for the target large language model, performs data theft attack detection on the user terminal to obtain the theft attack detection result, and controls the target large language model to perform content generation processing based on the input request content to obtain the initial output response content. If the theft attack detection result is a theft attack type, the initial output response content is modified to obtain the target output response content, and the user terminal is requested to reply based on the target output response content. If the theft attack detection result is a normal request type, the user terminal is requested to reply based on the initial output response content. By real-time monitoring of the input request content of the user terminal and performing data theft attack detection on the user terminal, the data security related to the large language model is effectively improved, and the model theft attack can be timely identified and prevented, while minimizing the impact on normal users, and the overall security and user experience of services related to the large language model are improved.

[0167] The present specification also provides a computer storage medium, which can store multiple instructions, which are suitable for being loaded and executed by a processor as described above. Figure 1 to Figure 6 The data theft attack processing method of the embodiment shown in the figure can be found in the specific execution process. Figure 1 to Figure 6 The specific description of the illustrated embodiment will not be repeated here.

[0168] The present specification also provides a computer program product, which stores at least one instruction, and the at least one instruction is loaded and executed by the processor as described above. Figure 1 to Figure 6 The data theft attack processing method of the embodiment shown in the figure can be found in the specific execution process. Figure 1 to Figure 6 The specific description of the illustrated embodiment will not be repeated here.

[0169] Please refer to Figure 8 , is a block diagram of a structure of an electronic device provided in an embodiment of this specification. The electronic device in this specification may include one or more of the following components: a processor 1010, a memory 1020, an input device 1030, an output device 1040, and a bus 1050. The processor 1010, the memory 1020, the input device 1030, and the output device 1040 may be connected via a bus 1050.

[0170] The processor 1010 may include one or more processing cores. The processor 1010 uses various interfaces and lines to connect various parts of the entire terminal, and executes various functions and processes data of the terminal 100 by running or executing instructions, programs, code sets or instruction sets stored in the memory 1020, and calling data stored in the memory 1020. Optionally, the processor 1010 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 1010 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 1010, but may be implemented separately through a communication chip.

[0171] The memory 1020 may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory 1020 includes a non-transitory computer-readable storage medium. The memory 1020 may be used to store instructions, programs, codes, code sets, or instruction sets.

[0172] The input device 1030 is used to receive input instructions or data, and the input device 1030 includes but is not limited to a keyboard, a mouse, a camera, a microphone, or a touch device. The output device 1040 is used to output instructions or data, and the output device 1040 includes but is not limited to a display device and a speaker. In the embodiment of this specification, the input device 1030 can be a temperature sensor for obtaining the operating temperature of the terminal. The output device 1040 can be a speaker for outputting audio signals.

[0173] In addition, those skilled in the art will appreciate that the structure of the terminal shown in the above drawings does not constitute a limitation on the terminal, and the terminal may include more or fewer components than shown, or combine certain components, or arrange the components differently. For example, the terminal also includes a radio frequency circuit, an input unit, a sensor, an audio circuit, a wireless fidelity (WIFI) module, a power supply, a Bluetooth module, and other components, which will not be described in detail here.

[0174] In the embodiments of this specification, the execution subject of each step may be the terminal described above. Optionally, the execution subject of each step is the operating system of the terminal. The operating system may be an Android system, an IOS system, or other operating systems, which is not limited in the embodiments of this specification.

[0175] exist Figure 8 In the electronic device, the processor 1010 can be used to call the program stored in the memory 1020 and perform the following steps, specifically:

[0176] Obtain the input request content of the user end for the target large language model;

[0177] Performing data theft attack detection on the user terminal to obtain a theft attack detection result, and controlling the target large language model to perform content generation processing based on the input request content to obtain initial output response content;

[0178] If the theft attack detection result is a theft attack type, modifying the initial output response content to obtain target output response content, and performing a request reply process on the user end based on the target output response content;

[0179] If the theft attack detection result is a normal request type, a request reply process is performed on the user terminal based on the initial output response content.

[0180] In a feasible implementation manner, the processor 1010 performs the data theft attack detection on the user terminal to obtain the theft attack detection result, and performs the following steps:

[0181] Acquire user request behavior data of the user terminal for the target large language model;

[0182] A data theft attack detection is performed based on the user request behavior data to obtain a theft attack detection result.

[0183] In a feasible implementation manner, the processor 1010 performs the data theft attack detection based on the user request behavior data to obtain the theft attack detection result, including:

[0184] Using a preset risk assessment indicator to perform a theft attack indicator evaluation on the user request behavior data to obtain a theft attack indicator detection result;

[0185] A theft attack detection result is obtained based on the theft attack indicator detection result.

[0186] In a feasible implementation manner, the processor 1010 performs the data theft attack detection based on the user request behavior data to obtain the theft attack detection result, including:

[0187] Based on the user request behavior data, a theft attack detection model is used to perform data theft attack detection to obtain a theft attack model detection result;

[0188] The theft attack detection result is obtained based on the theft attack model detection result.

[0189] In a feasible implementation manner, the processor 1010 performs the data theft attack detection based on the user request behavior data using the theft attack detection model to obtain the theft attack model detection result, including:

[0190] Extracting request behavior description features based on the user request behavior data, wherein the request behavior description features include at least one of request frequency features, request behavior pattern features, request content meta-information features, and request time features;

[0191] Generate a request behavior description for the user terminal based on the request behavior description feature;

[0192] Generate a data theft attack detection prompt word for the request behavior description, input the data theft attack detection prompt word and the request behavior description into a theft attack detection large model to perform data theft attack detection, and output the theft attack model detection result;

[0193] The theft attack detection large model is obtained by adapting the theft attack detection scenario using a basic large language model.

[0194] In a feasible implementation manner, the processor 1010 performs the content modification processing on the initial output response content to obtain the target output response content, including:

[0195] Determining the theft attack information for the initial output response content based on the theft attack detection result, and modifying the initial output response content using a response content modification system based on the theft attack information to obtain reference output response content;

[0196] Performing data theft attack detection on the reference output response content to obtain a theft attack detection result;

[0197] If the theft attack detection result is a normal request type, the reference request content is used as the target output response content;

[0198] If the theft attack detection result is a theft attack type, the reference request content is used as the initial output response content and the step of modifying the initial output response content using a response content modification system based on the theft attack information to obtain a reference output response content is performed.

[0199] In a feasible implementation manner, the processor 1010 executes the step of modifying the initial output response content using the response content modification system based on the theft attack information to obtain the reference output response content, including:

[0200] The theft attack information and the initial output response content are input into a response content modification system, and the risk-sensitive content and model kernel mechanism-related content for the initial output response content are determined by the response content modification system. Based on the risk-sensitive content and the model kernel mechanism-related content, the initial output response content is subjected to content modification processing to obtain a reference output response content.

[0201] In one or more embodiments of the present specification, the electronic device obtains the input request content of the user terminal for the target large language model, performs data theft attack detection on the user terminal to obtain the theft attack detection result, and controls the target large language model to perform content generation processing based on the input request content to obtain the initial output response content. If the theft attack detection result is a theft attack type, the initial output response content is modified to obtain the target output response content, and the user terminal is requested to reply based on the target output response content. If the theft attack detection result is a normal request type, the user terminal is requested to reply based on the initial output response content. By real-time monitoring of the input request content of the user terminal and performing data theft attack detection on the user terminal, the data security related to the large language model is effectively improved, and the model theft attack can be timely identified and prevented, while minimizing the impact on normal users, and the overall security and user experience of services related to the large language model are improved.

[0202] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0203] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in the embodiments of this specification are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions. For example, input request content, user request behavior data, etc. are all obtained with full authorization.

[0204] The above disclosure is only the preferred embodiment of this specification, which certainly cannot be used to limit the scope of rights of this specification. Therefore, equivalent changes made according to the claims of this specification are still within the scope covered by this specification.

Claims

1. A method for processing a data theft attack, the method comprising: Obtain the input request content of the user end for the target large language model; Performing data theft attack detection on the user terminal to obtain a theft attack detection result, and controlling the target large language model to perform content generation processing based on the input request content to obtain initial output response content; If the theft attack detection result is a theft attack type, modifying the initial output response content to obtain target output response content, and performing a request reply process on the user end based on the target output response content; If the theft attack detection result is a normal request type, a request reply process is performed on the user terminal based on the initial output response content.

2. According to the method of claim 1, the step of performing data theft attack detection on the user terminal to obtain the theft attack detection result comprises: Acquire user request behavior data of the user terminal for the target large language model; A data theft attack detection is performed based on the user request behavior data to obtain a theft attack detection result.

3. According to the method of claim 2, the step of performing data theft attack detection based on the user request behavior data to obtain the theft attack detection result comprises: Using a preset risk assessment indicator to perform a theft attack indicator evaluation on the user request behavior data to obtain a theft attack indicator detection result; A theft attack detection result is obtained based on the theft attack indicator detection result.

4. According to the method of claim 2, the step of performing data theft attack detection based on the user request behavior data to obtain the theft attack detection result comprises: Based on the user request behavior data, a theft attack detection model is used to perform data theft attack detection to obtain a theft attack model detection result; The theft attack detection result is obtained based on the theft attack model detection result.

5. According to the method of claim 4, the step of performing data theft attack detection based on the user request behavior data using the theft attack detection model to obtain the theft attack model detection result comprises: Extracting request behavior description features based on the user request behavior data, wherein the request behavior description features include at least one of request frequency features, request behavior pattern features, request content meta-information features, and request time features; Generate a request behavior description for the user terminal based on the request behavior description feature; Generate a data theft attack detection prompt word for the request behavior description, input the data theft attack detection prompt word and the request behavior description into a theft attack detection large model to perform data theft attack detection, and output the theft attack model detection result; The theft attack detection large model is obtained by adapting the theft attack detection scenario using a basic large language model.

6. The method according to claim 1, wherein the step of modifying the initial output response content to obtain the target output response content comprises: Determining the theft attack information for the initial output response content based on the theft attack detection result, and modifying the initial output response content using a response content modification system based on the theft attack information to obtain reference output response content; Performing data theft attack detection on the reference output response content to obtain a theft attack detection result; If the theft attack detection result is a normal request type, the reference request content is used as the target output response content; If the theft attack detection result is a theft attack type, the reference request content is used as the initial output response content and the step of modifying the initial output response content using a response content modification system based on the theft attack information to obtain a reference output response content is performed.

7. The method according to claim 6, wherein the step of modifying the initial output response content using a response content modification system based on the theft attack information to obtain reference output response content comprises: The theft attack information and the initial output response content are input into a response content modification system, and the risk-sensitive content and model kernel mechanism-related content for the initial output response content are determined by the response content modification system. Based on the risk-sensitive content and the model kernel mechanism-related content, the initial output response content is subjected to content modification processing to obtain a reference output response content.

8. A data theft attack processing device, the device comprising: A content acquisition module is used to obtain the input request content of the user end for the target large language model; An attack detection module, used to perform data theft attack detection on the user terminal to obtain a theft attack detection result, and control the target large language model to perform content generation processing based on the input request content to obtain an initial output response content; An attack processing module, configured to, if the theft attack detection result is a theft attack type, modify the initial output response content to obtain a target output response content, and perform a request reply process on the user terminal based on the target output response content; The attack processing module is further configured to perform a request reply process on the user terminal based on the initial output response content if the theft attack detection result is a normal request type.

9. A computer storage medium storing a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executing the method steps according to any one of claims 1 to 7.

10. A computer program product, the computer program product storing at least one instruction, wherein the at least one instruction is loaded by a processor and executes the method steps according to any one of claims 1 to 7.

11. An electronic device, comprising: A processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the method steps as claimed in any one of claims 1 to 7.

Images