Method, device, medium and product for security auditing of user interaction with model content

Abstract

The embodiments of the present disclosure disclose a method and device for security auditing of user interaction with model content, a medium and a product. The method comprises: deploying an API gateway, intercepting and analyzing data packets passing through the API gateway, obtaining the IP address of the user and the request data input by the user, constructing a user portrait, and then obtaining a target audit model set; using the target audit model set to perform security auditing on the request data, when the security auditing of the request data is passed, inputting the request data into a target intelligent model, intercepting the reply data output by the target intelligent model at the API gateway, and performing security auditing on the reply data, when the security auditing of the reply data is passed, feeding back the reply data to the user. The method can realize double security auditing of user request and model reply data by intercepting request data and reply data at the API gateway, avoid intrusive auditing interference, effectively deal with complex evasion means, and protect the content security and compliance of the model service platform.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of API content auditing, and particularly relates to a method and device for security auditing of user and model interaction content, a medium and a product. BACKGROUND

[0002] With the rapid development of artificial intelligence technology, large language models (LLM) and multi-modal large models have been widely applied in many scenarios such as dialogue generation, content creation, intelligent customer service, etc. These models can provide efficient and intelligent services for users, greatly improving the efficiency of production and life. However, the security and compliance of the content in the process of user and model interaction have become increasingly prominent, attracting widespread attention from all sectors of society. In this context, technologies for auditing and filtering the input and output content of large models have emerged.

[0003] However, most auditing systems use intrusive auditing methods, which can interfere with the normal operation of model services and affect service performance and stability. In addition, with the continuous improvement of model capabilities and the continuous evolution of attack methods, content security auditing faces more complex and concealed evasion challenges. Attackers try to bypass conventional content detection and filtering by using context evasion, semantic variants, homophonic words, network slang, and even multi-modal input. These new evasion methods greatly increase the detection difficulty, making it difficult for existing content security systems to effectively identify and block non-compliant content, and unable to fully guarantee the content security and compliance of the model service platform. SUMMARY

[0004] Therefore, the embodiments of the present disclosure provide a method and device for security auditing of user and model interaction content, which can realize double security auditing of user requests and model reply data by intercepting request data and reply data at the API gateway, avoid intrusive auditing interference, effectively deal with complex evasion methods, and guarantee the content security and compliance of the model service platform.

[0005] In a first aspect, the embodiments of the present disclosure provide a method for security auditing of user and model interaction content, which adopts the following technical solution:

[0006] Deploy an API gateway, intercept and parse data packets passing through the API gateway, obtain the IP address of the user and the request data input by the user, and determine the target intelligent model requested by the user;

[0007] Based on the IP address and the request data, construct a user portrait;

[0008] Based on the IP address and the user portrait, obtain a target audit model set;

[0009] performing security auditing on the request data using the target audit model set;

[0010] when the security auditing on the request data passes, inputting the request data into the target intelligent model, and intercepting reply data output by the target intelligent model at the API gateway;

[0011] performing security auditing on the reply data, and feeding back the reply data to the user when the security auditing on the reply data passes.

[0012] Optionally, the data packets passing through the API gateway are intercepted and parsed to obtain the IP address of the user and the request data of the user, and the target intelligent model requested by the user is determined, including:

[0013] mounting eBPF to a preset position, and using eBPF to intercept all data packets passing through the API gateway; wherein the preset position includes but is not limited to a network interface layer, an entry point of a network protocol stack, and a network connection point of an API gateway;

[0014] parsing the IP header of the data packet according to the IP protocol specification to obtain the IP address of the user;

[0015] determining the application layer protocol used by the data packet by parsing the transport layer header of the data packet;

[0016] extracting a complete request body from the data packet based on the application layer protocol;

[0017] extracting the request data input by the user from the complete request body based on the application layer protocol;

[0018] extracting a model identifier from the header of the complete request body, and determining the target intelligent model based on the model identifier.

[0019] Optionally, the user portrait is constructed based on the IP address and the request data, including:

[0020] obtaining the tenant information of the user by matching the IP address with a preset user information library;

[0021] obtaining the historical access behavior data of the user;

[0022] preprocessing the request data, and integrating the tenant information, the historical access behavior data, and the preprocessed request data into a user data set;

[0023] extracting key features from the user data set, and constructing a user portrait based on the key features.

[0024] Optionally, the obtaining of the target audit model set based on the IP address and the user portrait comprises:

[0025] The IP address is resolved using an IP address library to obtain a geographical location of the user;

[0026] The first audit model is determined from an area audit model set based on the geographical location;

[0027] The user occupation is obtained based on the IP address and the user portrait;

[0028] The second audit model is determined from an occupation audit model set based on the user occupation;

[0029] The user age is extracted from the user portrait;

[0030] The third audit model is determined from an age audit model set based on the user age;

[0031] The first audit model, the second audit model and the third audit model are all target audit models, and all target audit models constitute a target audit model set.

[0032] Optionally, the obtaining of the user occupation based on the IP address and the user portrait comprises:

[0033] The IP address is matched with all preset fields in a field database; wherein the field database stores a mapping relationship between a preset field and an occupation type;

[0034] If the IP address contains any preset field, the occupation type corresponding to the preset field is taken as the user occupation;

[0035] If the IP address does not contain any preset field, the user occupation is extracted from the user portrait.

[0036] Optionally, the security audit of the request data using the target audit model set comprises:

[0037] The request data is respectively input into each target audit model to obtain a sensitive category and a corresponding confidence degree output by each target audit model;

[0038] A risk threshold of each sensitive category is obtained;

[0039] The confidence degree of each sensitive category is respectively compared with the corresponding risk threshold;

[0040] If the confidence degree of any sensitive category is greater than or equal to the corresponding risk threshold, it is determined that the security audit of the request data fails;

[0041] If the confidence of all sensitive categories is less than the corresponding risk threshold, it is determined that the security audit of the request data is passed.

[0042] Optionally, the risk threshold of each sensitive category is obtained, comprising:

[0043] extracting a behavior feature label and a security feature label of the user from the user portrait;

[0044] obtaining a weight coefficient based on the behavior feature label and the security feature label;

[0045] multiplying the weight coefficient with a preset basic threshold of each sensitive category to obtain the risk threshold of each sensitive category.

[0046] Optionally, the security audit method of the user and the model interaction content further comprises:

[0047] When the reply data security audit is not passed, the request data and the reply data are stored in the backtracking database as a set of associated data;

[0048] Based on all the associated data in the backtracking database, each target audit model in the target audit model set is supervised and fine-tuned.

[0049] In a second aspect, the embodiments of the present disclosure further provide a security audit system for user and model interaction content, which adopts the following technical scheme:

[0050] A request data acquisition module is configured to deploy an API gateway, intercept and parse data packets passing through the API gateway, acquire an IP address of a user and request data input by the user, and determine a target intelligent model requested by the user;

[0051] A user portrait construction module is configured to construct a user portrait based on the IP address and the request data;

[0052] An audit model acquisition module is configured to acquire a target audit model set based on the IP address and the user portrait;

[0053] A request data audit module is configured to use the target audit model set to perform a security audit on the request data;

[0054] A reply data interception module is configured to input the request data into the target intelligent model when the security audit of the request data is passed, and intercept reply data output by the target intelligent model at the API gateway;

[0055] The reply data auditing module is configured to perform security auditing on the reply data, and feed back the reply data to the user when the security auditing on the reply data is passed.

[0056] In a third aspect, the embodiments of the present disclosure further provide a computer device, which adopts the technical scheme as follows:

[0057] The computer device comprises:

[0058] at least one processor; and

[0059] a memory connected with the at least one processor in communication; wherein

[0060] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the security auditing method of the user and the model interaction content as described in any one of the above.

[0061] In a fourth aspect, the embodiments of the present disclosure further provide a computer readable storage medium, which stores computer instructions for causing a computer to perform the security auditing method of the user and the model interaction content as described in any one of the above.

[0062] In a fifth aspect, the embodiments of the present disclosure further provide a computer program product, which comprises computer programs / instructions, and the computer programs / instructions are executed by a processor to implement the steps of the method as described in any one of the above.

[0063] The method for security auditing of user and model interaction content provided by the embodiments of the present disclosure can intercept and analyze the source of user and intelligent model interaction by deploying an API gateway in front of the large model inference service. The user portrait can comprehensively reflect the characteristics and behavior patterns of the user. By combining IP addresses and request data to build a user portrait, the user's usage habits, preferences, historical request content, and other information can be understood. Different users may have different security risks, different IP addresses may come from different network environments, and the user portrait also reflects the diversified characteristics of the user. According to the IP address and the user portrait, a target audit model set is obtained, which can select the most suitable target audit model for the user, improving the accuracy and pertinence of the audit. The target audit model set contains multiple target audit models, and each model may audit the request data from different angles, greatly improving the accuracy and reliability of detection. The request data is input into the target intelligent model only after passing the security audit, avoiding the interference and potential attacks of non-compliant data on the target intelligent model. The API gateway intercepts the reply data output by the target intelligent model, realizing the whole-process audit of the user and model interaction content. Because even if the request data passes the audit, the target intelligent model may produce non-compliant reply content during processing due to various reasons. Through the audit of the reply data, the security risks and non-compliant information that may exist can be further filtered out, ensuring that the content finally fed back to the user is safe and compliant. Moreover, in the face of complex evasion methods such as context evasion, semantic variants, homophonic words, and network popular words used by attackers, the audit of the reply data increases the ability to identify these hidden risks, effectively responds to new evasion challenges, and promotes the healthy development of the large model inference service platform.

[0064] The above description is only a summary of the technical solutions of the present disclosure. In order to more clearly understand the technical means of the present disclosure, the contents of the specification can be implemented, and in order for the above and other purposes, features and advantages of the present disclosure to be more apparent and easy to understand, the following preferred embodiments are described in detail below, and the accompanying drawings are described as follows. BRIEF DESCRIPTION OF DRAWINGS

[0065] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following will briefly introduce the drawings needed in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and those skilled in the art can obtain other drawings according to these drawings without creating any creative labor.

[0066] Figure 1 The flowchart of the method for security auditing of user and model interaction content provided by the embodiments of the present disclosure;

[0067] Figure 2A flowchart illustrating the data packet interception and parsing method provided in this embodiment of the disclosure;

[0068] Figure 3 A flowchart illustrating the user profile construction method provided in this embodiment of the disclosure;

[0069] Figure 4 A flowchart illustrating the target audit model determination method provided in this embodiment of the disclosure;

[0070] Figure 5 A flowchart illustrating the user occupation acquisition method provided in this embodiment of the disclosure;

[0071] Figure 6 A flowchart illustrating the request data security auditing method provided in this embodiment of the disclosure;

[0072] Figure 7 A schematic diagram of a security auditing system for user-model interaction content provided in this embodiment of the disclosure;

[0073] Figure 8 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present disclosure. Detailed Implementation

[0074] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0075] It should be understood that the following specific examples illustrate the implementation of this disclosure, and those skilled in the art can easily understand other advantages and effects of this disclosure from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. This disclosure can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this disclosure. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. Based on the embodiments in this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0076] It should be noted that various aspects of embodiments within the scope of the appended claims are described below. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this disclosure, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using structures and / or functionalities other than one or more of the aspects set forth herein.

[0077] It should also be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this disclosure. The drawings only show the components related to this disclosure and are not drawn according to the number, shape and size of the components in actual implementation. In actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0078] Furthermore, specific details are provided in the following description to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that the described aspects can be practiced without these specific details.

[0079] Reference Figure 1 This disclosure provides a method for security auditing of user-model interaction content, including the following steps:

[0080] S1: Deploy an API gateway to intercept and parse data packets passing through the API gateway, obtain the user's IP address and the user's input request data, and determine the target intelligent model requested by the user.

[0081] S2: Build user profiles based on IP addresses and request data;

[0082] S3: Obtain the target audit model set based on IP address and user profile;

[0083] S4: Use the target audit model set to perform security audits on the request data;

[0084] S5: When the security audit of the request data passes, input the request data into the target intelligent model and intercept the response data output by the target intelligent model at the API gateway;

[0085] S6: Perform security audit on the response data. When the security audit of the response data passes, the response data will be fed back to the user.

[0086] This disclosure provides a security auditing method for user-model interaction content. An API Gateway is a middleware component that acts as a "gatekeeper" for a system or service. By deploying an API Gateway at the front end of a large model inference service, it is possible to intercept and parse user-intelligent model interactions at the source, ensuring that all user request packets passing through the gateway are processed without omission. This comprehensively covers user-intelligent model interaction scenarios and provides a complete data foundation for subsequent security audits. User profiles comprehensively reflect user characteristics and behavioral patterns. By combining IP addresses and request data to construct user profiles, information such as user habits, preferences, and historical request content can be understood.

[0087] Different users may pose different security risks, and different IP addresses may originate from different network environments. User profiles also reflect the diverse characteristics of users. By obtaining a set of target audit models based on IP addresses and user profiles, the most suitable target audit model can be selected for each user, ensuring that the target audit model matches the user's actual situation and improving the accuracy and relevance of the audit. As user behavior changes and new security threats emerge, the set of target audit models can be dynamically adjusted based on changes in user profiles and IP addresses, ensuring that the audit strategy remains effective at all times.

[0088] The target audit model set contains multiple target audit models, each auditing the request data from different perspectives. This multi-faceted auditing approach can more comprehensively detect security vulnerabilities and non-compliant content in the request data, greatly improving the accuracy and reliability of detection. Request data is only input into the target intelligent model after passing security audits, avoiding interference and potential attacks from non-compliant data. This ensures the normal operation and stability of the target intelligent model and reduces the possibility of malfunctions, incorrect output results, or sensitive information due to malicious data input.

[0089] By intercepting the response data output by the target intelligent model at the API gateway, end-to-end auditing of the user-model interaction is achieved. This approach, which focuses not only on the security of user input but also on auditing the content output by the target intelligent model, ensures that the entire interaction process complies with security and compliance requirements. Even if the request data passes auditing, the target intelligent model may still generate non-compliant response content during processing for various reasons. Auditing the response data further filters out potential security risks and non-compliant information, ensuring that the final content fed back to the user is safe and compliant. Furthermore, in the face of complex evasion techniques employed by attackers, such as context avoidance, semantic variations, homophones, and internet slang, auditing the response data enhances the ability to identify these hidden risks, effectively addressing new evasion challenges and promoting the healthy development of large-scale model inference service platforms.

[0090] In S1, during the interaction between the user and the target intelligent model, the user's request data input to the target intelligent model and the target intelligent model's response data together constitute the interaction content. To ensure the compliance of the interaction content, an API gateway is deployed at the front end of the large model inference service. The API gateway intercepts the user's input request data to facilitate subsequent security auditing, thereby ensuring the security and legality of the data. (Refer to...) Figure 2 The flowchart illustrating the data packet interception and parsing method demonstrates the steps involved in "intercepting and parsing data packets passing through the API gateway, obtaining the user's IP address and request data, and determining the target intelligent model requested by the user":

[0091] S11: Mount eBPF to a preset location and use eBPF to intercept all data packets passing through the API gateway; wherein, the preset location includes, but is not limited to, the network interface layer, the entry point of the network protocol stack, and the network connection point of the API gateway.

[0092] S12: Parse the IP header of the data packet according to the IP protocol specification to obtain the user's IP address;

[0093] S13: Determine the application layer protocol used by the data packet by parsing the transport layer header of the data packet;

[0094] S14: Extract the complete request body from the data packet based on the application layer protocol;

[0095] S15: Extract user-input request data from the complete request body based on application layer protocols;

[0096] S16: Extract the model identifier from the header of the complete request body, and determine the target intelligent model based on the model identifier.

[0097] In S11, eBPF (extended Berkeley Packet Filter) is a mechanism for running user-defined programs within the Linux kernel. It features high performance, zero intrusion, and secure isolation, allowing for precise capture of request data in kernel mode without altering user-space business logic. Using bpftool or other tools, the written eBPF program is loaded into the kernel. The loading process verifies the eBPF program to ensure its security and correctness. Leveraging the key network processing paths within the operating system kernel, the eBPF program captures network request data passing through the API gateway in real time, without requiring any changes to existing business logic or model services. It is zero-intrusive and protocol-independent. These key network processing paths include, but are not limited to, the network interface layer, entry points of the network protocol stack, and network connection points of the API gateway. For the network interface layer, the `tc` (Traffic Control) tool can be used to mount the eBPF program to the network interface's ingress or egress queue. Here, the `socket_filter` mechanism can work in conjunction with the eBPF program. `socket_filter` allows filtering rules to be set at the socket level. When packets enter or leave the network interface, they are initially filtered by `socket_filter` before the eBPF program is triggered for further capture and processing. For the entry point of the network protocol stack, mechanisms such as `kprobe` or `tracepoint` can be used to mount the eBPF program to a kernel function. `tcp_sendmsg` is a crucial function in the Linux kernel network protocol stack for handling TCP data transmission. When the kernel processes network packets and calls the `tcp_sendmsg` function, the eBPF program mounted on this function intercepts the packets and performs corresponding operations, such as parsing IP headers and transport layer headers. Simultaneously, `socket_filter` can also be applied at this location to filter packets entering the protocol stack, ensuring that only packets conforming to the rules are further processed. For the network connection points of the API gateway, depending on the specific implementation of the API gateway, it may be necessary to use specific hooks or extension mechanisms to mount the eBPF program to the appropriate location. During this process, socket_filter can assist the eBPF program in filtering and controlling data packets entering and leaving the API gateway, improving the API gateway's data processing efficiency and security.

[0098] In S12, after the eBPF program captures a data packet, it needs to locate the start of the IP header. In an Ethernet frame, the IP header usually follows the Ethernet header, which is typically 14 bytes long. Therefore, the IP header can be found using an offset. According to the IP protocol specification, the IP header contains multiple fields, such as version number, header length, source IP address, and destination IP address. By reading the corresponding byte positions, the values ​​of these fields can be obtained. For example, the source IP address in the IPv4 header is located in bytes 12-15. The eBPF program can read these four bytes and convert them into IP address format. It is necessary to consider both IPv4 and IPv6. The IPv6 header format is different from IPv4 and is longer. The eBPF program needs to determine whether the data packet is IPv4 or IPv6 based on the version number field in the IP header and use the appropriate parsing method. The parsed source IP address is the user's IP address.

[0099] In S13, after parsing the IP header, the transport layer protocol (such as TCP or UDP) is determined based on the protocol field of the IP header. Then, the start position of the transport layer header is calculated based on the header length field of the IP header. The TCP header includes fields such as source port, destination port, and sequence number; the UDP header includes fields such as source port, destination port, and length. By reading the values ​​of these fields, the source and destination port information can be obtained. Different application layer protocols typically use fixed port numbers. For example, the HTTP protocol uses ports 80 or 443 by default, while certain ports may be used for other application layer protocols such as gRPC. The eBPF program can determine the application layer protocol used by the data packet based on the parsed port number.

[0100] In S14, once the application layer protocol is determined, the eBPF program extracts the request body from the data packet according to the corresponding protocol specifications. For the HTTP protocol, the start position and length of the request body are found based on the HTTP request format, thereby extracting the request body content. For the gRPC protocol, the gRPC request body is extracted from the data packet according to its protocol rules.

[0101] The extracted request body data is stored in a RingBuffer. As a highly efficient circular buffer, the RingBuffer enables fast data transfer from kernel space to user space, avoiding frequent data copying and significantly improving transmission efficiency. In user space, the program reads the extracted request body data from the RingBuffer. Since network packets may be transmitted in multiple fragments, session reassembly is necessary in user space. For the HTTP protocol, the scattered packet fragments are reassembled into a complete HTTP request based on the HTTP request header information (such as the Content-Length field) and the request body data. For the gRPC protocol, session reassembly is performed according to its protocol rules to recover the complete gRPC request, ultimately obtaining the complete request body.

[0102] In S15, within the HTTP protocol, request data may be distributed across different parts of the request body. When users use the model, it's typically a POST request, where the request data is generally within the request body. The appropriate parsing method is used based on the request body's format (e.g., form data, JSON data, etc.). For example, if the request body is in JSON format, a JSON parsing library (such as the `json` module in Python) can be used to parse it into a Python object, thereby extracting the user's request data. For gRPC requests, based on the message structure defined by its protocol, corresponding serialization and deserialization methods are used to extract the request data. gRPC typically uses Protocol Buffers for data serialization; therefore, the Protocol Buffers library is needed to parse the request body and obtain the user's request data.

[0103] In S16, the fields related to the model identifier are searched in the header information of the complete request body. Different systems may use different field names to identify the smart model; common ones include custom request header fields such as X-Target-Model and Model-Name. By reading the value of the corresponding header field, the model identifier is extracted. Based on the extracted model identifier, the corresponding smart model is searched in the model list of the large model inference service. This model is the target smart model requested by the user.

[0104] In S2, refer to Figure 3 The flowchart illustrating the user profile building method shows that "building a user profile based on IP address and request data" includes the following steps:

[0105] S21: Obtain the user's tenant information by matching the IP address with a preset user information database;

[0106] S22: Obtain the user's historical access behavior data;

[0107] S23: Preprocess the request data, and integrate tenant information, historical access behavior data and preprocessed request data into a user dataset;

[0108] S24: Extract multiple key features from the user dataset according to the preset type, and build a user profile based on the key features.

[0109] In S21, a key-value pair user information database is pre-defined. This database stores the mapping relationship between IP addresses and tenant information, where the IP address is the key and the corresponding tenant information (such as tenant ID, tenant name, tenant industry, tenant size, etc.) is the value. When a user's IP address is obtained, it is used as a query condition to search the pre-defined user information database and extract the corresponding tenant information.

[0110] In S22, you can choose to use tracking codes provided by third-party analytics tools such as Google Analytics and Baidu Analytics, or develop your own tracking code based on business needs. The tracking code executes automatically when a user visits a page, collecting user access data such as access time, page URL, dwell time, and page redirection paths. This data is sent to a designated server for storage and analysis. On the server side, by writing code to record relevant access information during user request processing, a logging framework can be used to record information such as the user's IP address, request time, request method, and request URL to a log file. User access information can then be extracted from the log file. The access data collected by the tracking code and the access information extracted from the log file are integrated into the user's historical access behavior data.

[0111] In S23, preprocessing includes data cleaning, formatting, and standardization. Data cleaning refers to removing invalid, duplicate, and noisy data from the requested data. For example, if the requested data contains special characters or garbled text, it can be filtered and replaced. Formatting refers to converting the requested data into a uniform format for subsequent analysis and processing. For example, date and time data can be converted into a standard format, and text data can be encoded. Standardization refers to standardizing the requested data to give it the same scale and range. For example, normalization methods can be used to scale numerical data to between 0 and 1.

[0112] Use database table join operations or data processing tools (such as the Pandas library) to merge tenant information, historical access behavior data, and preprocessed request data into a single data structure to form a user dataset for subsequent feature extraction and analysis.

[0113] In S24, key features include basic attribute features, behavioral features, and security features. Basic attribute features include, but are not limited to, the user's age and occupation. Users of different age groups may have different risk preferences and common security concerns when using the large model. For example, teenagers may be more susceptible to harmful information, requiring stricter content review during audits; while older adults may be more vulnerable to fraudulent information, necessitating attention to fraudulent or misleading responses during audits. Different occupations involve different specific types of information and data. For instance, financial professionals may input sensitive financial information, requiring a focus on data security and compliance during audits; medical personnel may input patients' private information, necessitating assurance that this information is not leaked.

[0114] Behavioral characteristics include, but are not limited to, the frequency of a user's historical requests, the pattern of request timing, and the type of content requested. Frequent requests may indicate abnormal behavior, such as malicious attacks or bulk data acquisition; therefore, requests with excessively high historical request frequencies require focused auditing. The regularity of a user's request timing can also reflect their behavioral patterns. For example, a user who typically uses large models during daytime work hours but suddenly makes frequent requests late at night may be acting abnormally and requires more rigorous auditing. The types of content users frequently request, such as text, images, and videos, as well as specific subject areas such as technology, entertainment, and health, can all provide insights for security audits. Different types of content may pose different security risks; for example, requests involving politically sensitive topics or personal privacy information require special attention.

[0115] Security features include, but are not limited to, the user's historical violation count and device information. If a user has committed multiple violations in the past, such as entering malicious code or spreading false information, these records will become important security audit references. For users with a history of violations, stricter audit strategies are required. Information such as the type of device used by the user and the operating system version is also relevant to security auditing. For example, devices using outdated operating systems may have more security vulnerabilities, requiring more rigorous checks on their requested data.

[0116] To ensure that key features accurately reflect user characteristics and effectively serve subsequent analysis and decision-making, multiple targeted tags are assigned to users by matching key features with preset tagging rules. For example, if a user is 25 years old, falling within the pre-defined 19-35 age range, they are labeled "youth"; if a user initiates more than 100 requests in a day, they are labeled "high-frequency request" according to the rules. Integrating these key features and the corresponding user tags constructs a comprehensive and detailed user profile that depicts user characteristics and behavioral patterns.

[0117] In S3, refer to Figure 4 The flowchart illustrating the method for determining the target audit model, "Obtaining the set of target audit models based on IP address and user profile," includes the following steps:

[0118] S31: Use the IP address database to resolve IP addresses and obtain the user's geographical location;

[0119] S32: Determine the primary audit model from the regional audit model set based on geographic location;

[0120] S33: Obtain user occupation based on IP address and user profile;

[0121] S34: Based on the user's profession, determine the second audit model from the set of professional audit models;

[0122] S35: Extract user age from user profile;

[0123] S36: Based on the user's age, determine the third audit model from the set of age audit models;

[0124] S37: The first audit model, the second audit model, and the third audit model are all target audit models, and all target audit models constitute the target audit model set.

[0125] In S31, IP address libraries such as GeoIP and IPIP.net are selected. Based on the API or tools provided by the selected IP address library, the user's IP address is used as input to call the corresponding resolution function or interface to obtain the user's geographical location information, including country, province, city, region, etc.

[0126] Reference Figure 5 The flowchart illustrating the method for obtaining user occupations shows that "obtaining user occupations based on IP address and user profile" includes the following steps:

[0127] S331: Match the IP address with all preset fields in the field database; if the IP address contains any preset field, execute S332; if the IP address does not contain any preset field, execute S333.

[0128] S332: Use the corresponding occupation type in the preset field as the user's occupation;

[0129] S333: Extract user occupation from user profile.

[0130] The database stores the mapping relationship between preset fields and occupation types, with preset fields serving as keys and corresponding occupation types as values. IP addresses, as readily available key information in network interactions, allow for rapid preliminary determination of a user's occupation by matching IP address-related information with preset fields. This method avoids complex analysis of user profiles and significantly improves data processing efficiency. From a data timeliness perspective, IP addresses reflect a user's current network environment in real time and can be directly linked to the user's company or organization. This makes occupational information determined based on IP addresses more relevant to the user's current situation and more accurate. In contrast, occupational information in user profiles typically comes from past user entries, which may change over time, leading to information lag. Based on these characteristics, when acquiring user occupational information, priority is given to starting with IP address-related information, fully utilizing its timeliness and relevance. Only when it is difficult to determine a user's occupation based on IP address should relevant information be extracted from the user profile. This constructs a more efficient and accurate user occupation determination process. Furthermore, to improve the timeliness of user profiles, they need to be updated promptly when user information changes.

[0131] In content security auditing and filtering scenarios, significant differences in standards arise due to varying laws and regulations across different countries and regions, as well as unique compliance needs among different tenants (enterprises, institutions). Different industries or enterprises also develop their own personalized content compliance guidelines, targeting users of different age groups with varying content access restrictions and audit rigor. Existing content auditing systems, unable to adapt to these multi-dimensional differences, struggle to meet cross-border, cross-regional, cross-industry, and all-age-group compliance requirements, potentially leading to inaccurate audit results or failure to meet the requirements of specific regions and tenants. This solution reflects the user's geographical location as their country and region, selecting a corresponding primary audit model from a regional audit model set. This allows the API gateway to match the applicable audit rules and standards for that region based on the user's country or region. Furthermore, this solution selects a suitable secondary audit model based on the user's occupational characteristics and a corresponding tertiary audit model based on age. Utilizing these audit models to perform security audits on requested data ensures that audit rules are more aligned with the user's actual situation. This allows for more accurate identification and filtering of content that does not comply with local laws and regulations, industry standards, and enterprise-specific requirements, avoiding misjudgments or omissions caused by auditing based on a single standard.

[0132] In S4, refer to Figure 6 The flowchart illustrating the request data security auditing method shows that "using the target audit model set to perform security auditing on request data" includes the following steps:

[0133] S41: Input the request data into each target audit model and obtain the sensitive category and confidence level of each target audit model.

[0134] S42: Obtain the risk threshold for each sensitive category;

[0135] S43: Compare the confidence level of each sensitive category with the corresponding risk threshold; if the confidence level of any sensitive category is greater than or equal to the corresponding risk threshold, then proceed to S44; if the confidence level of all sensitive categories is less than the corresponding risk threshold, then proceed to S45.

[0136] S44: The security audit of the requested data failed.

[0137] S45: The security audit of the requested data has been approved.

[0138] In the steps described above, within the security audit process, each target audit model conducts a detailed analysis of the input request data. The target audit model accurately identifies sensitive information within the request data, clarifies the specific types of this sensitive information, and then outputs the corresponding sensitivity category and its confidence level. The confidence level reflects the reliability of the model's judgment that the data belongs to a specific sensitivity category. These confidence levels for sensitivity categories are compared with pre-set risk thresholds, which are based on the potential security risks associated with different sensitivity categories. This comparison allows for a scientific and accurate determination of the security of the request data. Only when the confidence levels for all sensitivity categories are less than the corresponding risk thresholds is the security audit of the request data considered passed. This approach achieves a comprehensive, detailed, and accurate security review of the request data, effectively identifying potential sensitive information and its risk level.

[0139] Furthermore, behavioral and security feature tags are extracted from the user profile. Based on these tags, weighting coefficients are obtained. These coefficients are then multiplied by a pre-defined base threshold for each sensitive category to derive the risk threshold for each category. The formula for calculating the weighting coefficients is as follows:

[0140]

[0141] In the formula, Indicates the weighting coefficient; Indicates the first The weight score of the behavioral feature label; This represents the total number of behavioral feature labels; Indicates the first Weight scores for various security feature labels; This indicates the total number of security feature tags.

[0142] Among them, the behavioral feature tags include at least one of the following: historical request frequency tags, request time pattern tags, and request content type tags; historical request frequency tags include one of the following: high-frequency request tags, medium-frequency request tags, and low-frequency request tags; request time pattern tags include one of the following: abnormal request time tags and normal request time tags; and request content type tags are identifiers that summarize and classify the key features and attributes of the request content. The content they cover will vary depending on different application scenarios and classification systems. For example, they include text, image, audio, and video categories, and are further subdivided into more specific tags under different categories, such as movie tags, TV series tags, short video tags, and instructional video tags under the video category.

[0143] Different tags are assigned different weight scores. Taking request frequency tags as an example, high-frequency request tags have positive weight scores, while low-frequency request tags have negative weight scores. By associating the acquired user tags with their corresponding weight scores and summing these weight scores, a weight coefficient applicable to the current user can be obtained. Using this weight coefficient to adjust the base threshold yields a more reasonable risk threshold. This method of dynamically adjusting risk thresholds based on user tags ensures the comprehensiveness and accuracy of data security audits, effectively protecting data security. Furthermore, it avoids over-review caused by using uniform, fixed audit standards, significantly improving audit efficiency and enabling the system to operate efficiently and stably while meeting security compliance requirements.

[0144] In S5 and S6, simply auditing the user's request data is insufficient to comprehensively assess the potential security risks throughout the entire interaction process. This is because the request data may only contain surface information, and during the target intelligent model's data processing, the model's own characteristics, limitations of the training data, or potential vulnerabilities may lead to security risks in the output response data, such as sensitive information leakage, misleading, or malicious inducement. Therefore, even after the security audit of the request data passes, it is still necessary to audit the response data output by the target intelligent model to ensure that the information ultimately presented to the user is safe, accurate, and compliant. This prevents inappropriate content in the response data from causing adverse effects on users, organizations, or society, further strengthening the security and reliability of the entire interaction process and ensuring the system operates in a safe and stable state. The security audit of response data can be conducted using the same principles as the request data security audit, employing multiple different types of security audit models, or a unified security audit model can be used to obtain the corresponding sensitivity categories and confidence levels, thereby determining the security audit results of the response data. Only when the response data security audit passes will the response data be intercepted at the API gateway and fed back to the user.

[0145] Furthermore, when the response data security audit fails, it indicates an error occurred in the previous security audit of the request data. The request and response data are then linked together and stored in the backtracking database. Based on all the linked data in the backtracking database, each target audit model in the target audit model set is subjected to supervised fine-tuning. The backtracking database acts as a "problem data warehouse," specifically used to store interactive data found to have security issues during audits. Supervised fine-tuning of the target audit models in this database enables better identification of similar security problems. Over time and with the accumulation of problem data, the target audit models become increasingly intelligent, more effectively ensuring the security of user input data.

[0146] A decision tree or rule chain is constructed. When the security audit of the requested data fails, or the security audit of the requested data passes but the security audit of the response data fails, the confidence level obtained during the security audit is matched with the decision tree or rule chain to determine the corresponding handling strategy, such as terminating the connection, rate limiting response, logging, and alarm reporting. The eBPF Map is a data structure provided by eBPF, similar to a key-value pair store, which can be used to share data between kernel space and user space. Therefore, the policy judgment result can be written back to the eBPF Map channel from user space and passed to kernel space. The deployed eBPF program reads the control signals and performs actual operations such as packet interception, RST response, and rate limiting. Policy control can be executed without leaving the kernel path, with latency as low as microseconds, achieving millisecond-level policy feedback. This solution constructs a real-time response closed loop, avoiding the security failure problem of "slow identification and slow feedback".

[0147] This solution's content recognition and policy control operate independently outside the model service, offering exceptional flexibility and real-time performance. Its value is significant; it's plug-and-play, adaptable to LLM or API services deployed in any language, framework, or platform, and supports local deployment, edge node operation, and cross-platform migration, demonstrating high reusability. Furthermore, it supports hot updates of policy rules and models, allows for the combination of multiple audit output types, and can integrate with cloud platforms such as Kubernetes to achieve multi-tenant policy distribution. It is suitable for AI service platforms or API gateways with high security sensitivity and complex traffic.

[0148] Reference Figure 7 This disclosure provides a security auditing system for user-model interaction content, including:

[0149] The request data acquisition module 101 is used to deploy the API gateway, intercept and parse data packets passing through the API gateway, obtain the user's IP address and the request data entered by the user, and determine the target intelligent model requested by the user.

[0150] User profile building module 102 is used to build user profiles based on IP address and request data;

[0151] The audit model acquisition module 103 is used to acquire a set of target audit models based on IP address and user profile;

[0152] The request data auditing module 104 is used to perform security auditing on the request data using the target auditing model set.

[0153] The response data interception module 105 is used to input the request data into the target intelligent model when the security audit of the request data passes, and to intercept the response data output by the target intelligent model at the API gateway;

[0154] The response data audit module 106 is used to perform security audits on response data. When the response data security audit passes, the response data is fed back to the user.

[0155] The various variations and specific examples of the security auditing method for user-model interaction content provided above are also applicable to the security auditing system for user-model interaction content provided in this disclosure. Through the foregoing detailed description of the security auditing method for user-model interaction content, those skilled in the art can clearly understand the implementation method of the security auditing system for user-model interaction content. For the sake of brevity, it will not be described in detail here.

[0156] A computer device according to embodiments of the present disclosure includes a memory and a processor. The memory is used to store non-transitory computer-readable instructions. Specifically, the memory may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may, for example, include random access memory (RAM) and / or cache memory. The non-volatile memory may, for example, include read-only memory (ROM), hard disk, flash memory, etc.

[0157] The processor may be a central processing unit (CPU) or other processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the computer device to perform desired functions. In one embodiment of this disclosure, the processor is used to execute computer-readable instructions stored in the memory, causing the computer device to perform all or part of the steps of the security auditing method for user-model interaction content described in the foregoing embodiments of this disclosure.

[0158] Those skilled in the art will understand that, in order to solve the technical problem of how to achieve a good user experience, this embodiment may also include well-known structures such as communication buses and interfaces, and these well-known structures should also be included within the protection scope of this disclosure.

[0159] like Figure 8 This is a schematic diagram of a computer device provided for an embodiment of the present disclosure. It illustrates a structural schematic diagram suitable for implementing the computer device in the embodiments of the present disclosure. Figure 8 The computer device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0160] like Figure 8As shown, a computer device may include a processor (such as a central processing unit, graphics processing unit, etc.), which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) or programs loaded from storage devices into random access memory (RAM). The RAM also stores various programs and data required for the operation of the computer device. The processor, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0161] Typically, the following devices can be connected to the I / O interface: input devices, such as sensors or visual information acquisition devices; output devices, such as displays; storage devices, such as magnetic tapes or hard drives; and communication devices. Communication devices allow the computer device to communicate wirelessly or wiredly with other devices (such as edge computing devices) to exchange data. Although Figure 8 A computer apparatus with various devices is shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or included alternatively.

[0162] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processor, all or part of the steps of the security auditing method for user-model interaction content according to embodiments of this disclosure are performed.

[0163] For a detailed description of this embodiment, please refer to the corresponding descriptions in the foregoing embodiments, which will not be repeated here.

[0164] A computer-readable storage medium according to embodiments of the present disclosure stores non-transitory computer-readable instructions. When the non-transitory computer-readable instructions are executed by a processor, all or part of the steps of the security auditing method for user-model interaction content described in the foregoing embodiments of the present disclosure are performed.

[0165] The aforementioned computer-readable storage media include, but are not limited to: optical storage media (e.g., CD-ROM and DVD), magneto-optical storage media (e.g., MO), magnetic storage media (e.g., magnetic tape or portable hard drive), media with built-in rewritable non-volatile memory (e.g., memory card), and media with built-in ROM (e.g., ROM cartridge).

[0166] For a detailed description of this embodiment, please refer to the corresponding descriptions in the foregoing embodiments, which will not be repeated here.

[0167] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0168] In this disclosure, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The block diagrams of devices, apparatuses, devices, and systems involved in this disclosure are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as "comprising," "including," "having," etc., are open-ended terms meaning "including but not limited to," and are used interchangeably with them. The terms "or" and "and" as used herein refer to the terms "and / or," and are used interchangeably with them unless the context clearly indicates otherwise. The term "such as" as used herein refers to the phrase "such as but not limited to," and is used interchangeably with it.

[0169] Additionally, as used herein, the "or" used in a list of items beginning with "at least one" indicates a separate list, such that a list of, for example, "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not imply that the described example is preferred or better than other examples.

[0170] It should also be noted that in the systems and methods of this disclosure, the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered as equivalent solutions to this disclosure.

[0171] Various changes, substitutions, and modifications can be made to the technology described herein without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of this disclosure is not limited to the specific aspects of the processes, machines, manufactures, events, means, methods, and actions described above. Currently existing or later-developed processes, machines, manufactures, events, means, methods, or actions that perform substantially the same function or achieve substantially the same result as the corresponding aspects described herein can be utilized. Therefore, the appended claims include such processes, machines, manufactures, events, means, methods, or actions within their scope.

[0172] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0173] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.

Claims

1. A method for security auditing user-model interaction content, characterized in that, include: Deploy an API gateway to intercept and parse data packets passing through the API gateway, obtain the user's IP address and the user's input request data, and determine the target intelligent model requested by the user. Based on the IP address and the request data, a user profile is constructed; Based on the IP address and the user profile, obtain the target audit model set; The step of obtaining the target audit model set based on the IP address and the user profile includes: The IP address is resolved using an IP address database to obtain the user's geographical location; Based on the aforementioned geographical location, a first audit model is determined from the set of regional audit models; Based on the IP address and the user profile, obtain the user's occupation; Based on the user's profession, a second audit model is determined from the set of profession audit models; Extract the user's age from the user profile; Based on the user's age, a third audit model is determined from the set of age audit models; The first audit model, the second audit model, and the third audit model are all target audit models, and all target audit models constitute a target audit model set. The requested data is security audited using the target audit model set. The step of performing security audits on the request data using the target audit model set includes: The request data is input into each target audit model to obtain the sensitive category and corresponding confidence level output by each target audit model. Obtain the risk threshold for each sensitive category; The confidence level for each sensitive category is compared with the corresponding risk threshold. If the confidence level of any sensitive category is greater than or equal to the corresponding risk threshold, the security audit of the requested data is deemed to have failed. If the confidence level of all sensitive categories is less than the corresponding risk threshold, then the security audit of the requested data is deemed to have passed. When the security audit of the requested data passes, the requested data is input into the target intelligent model, and the response data output by the target intelligent model is intercepted at the API gateway; The response data is subjected to security audit, and when the security audit of the response data passes, the response data is fed back to the user.

2. The security auditing method for user-model interaction content according to claim 1, characterized in that, The process of intercepting and parsing data packets passing through the API gateway, obtaining the user's IP address and request data, and determining the target intelligent model requested by the user includes: eBPF is mounted to a preset location, and eBPF is used to intercept all data packets passing through the API gateway; wherein, the preset location includes, but is not limited to, the network interface layer, the entry point of the network protocol stack, and the network connection point of the API gateway. The IP header of the data packet is parsed according to the IP protocol specification to obtain the user's IP address; By parsing the transport layer header of the data packet, the application layer protocol used by the data packet can be determined; The complete request body is extracted from the data packet based on the application layer protocol. Based on the application layer protocol, the user-input request data is extracted from the complete request body; The model identifier is extracted from the header of the complete request body, and the target intelligent model is determined based on the model identifier.

3. The security auditing method for user-model interaction content according to claim 1, characterized in that, The step of constructing a user profile based on the IP address and the request data includes: The user's tenant information is obtained by matching the IP address with a preset user information database; Obtain users' historical browsing behavior data; The request data is preprocessed to integrate the tenant information, the historical access behavior data, and the preprocessed request data into a user dataset; Key features are extracted from the user dataset, and user profiles are constructed based on the key features.

4. The security auditing method for user-model interaction content according to claim 1, characterized in that, The step of obtaining the user's occupation based on the IP address and the user profile includes: The IP address is matched against all preset fields in the field database; wherein, the field database stores the mapping relationship between preset fields and occupation types; If the IP address contains any preset field, then the occupation type corresponding to the preset field will be used as the user's occupation. If the IP address does not contain any preset fields, then the user's occupation is extracted from the user profile.

5. The security auditing method for user-model interaction content according to claim 1, characterized in that, The process of obtaining the risk threshold for each sensitive category includes: Extract user behavior feature tags and security feature tags from the user profile; Based on the behavioral feature labels and the security feature labels, obtain the weight coefficients; The weighting coefficients are multiplied by the preset base threshold for each sensitive category to obtain the risk threshold for each sensitive category.

6. The security auditing method for user-model interaction content according to claim 1, characterized in that, When the security audit of the response data fails, the request data and the response data are combined into a set of associated data and stored in the backtracking database; Supervised fine-tuning is performed on each target audit model in the target audit model set based on all associated data in the backtracking database.

7. A computer device, characterized in that, The computer device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the security auditing method for user-model interaction content as described in any one of claims 1-6.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to perform the security auditing method for user-model interaction content as described in any one of claims 1-6.

9. A computer program product comprising computer instructions, characterized in that, When executed by a processor, the computer instructions implement the steps of the security auditing method for user-model interaction content as described in any one of claims 1-6.

Images