A method, apparatus, device, and medium for retracting illegal output content of a large model.

Abstract

This application discloses a method, apparatus, device, and medium for retracting illegal output content of a large model, relating to the field of content security technology. The method includes: when a target large model is accessed through a target browser, during page initialization, a preset front-end script is embedded into the front-end code via a firewall server so that the preset front-end script runs when the front-end code is loaded; based on the firewall server and the preset front-end script, the interaction interface path of the large model is identified through a preset bidirectional communication protocol so that the firewall server can listen to the interaction information of the target large model; the firewall server detects the output content of the large model in the interaction information to determine whether illegal output content exists; if so, a unique identifier is generated for the output content of the large model, and a retraction command is sent to the preset front-end script via the preset bidirectional communication protocol so that the preset front-end script can retract the output content of the large model based on the retraction command.

Description

Technical Field

[0001] This invention relates to the field of content security technology, and in particular to a method, apparatus, device, and medium for retracting illegally output content in large-scale models. Background Technology

[0002] With the widespread application of large-scale language models, their potential risks are becoming increasingly prominent. In particular, the issue of manipulating the model's output to contain inappropriate content via specific prompts poses a significant challenge. To improve system security and user experience, it's crucial to promptly retract such content. However, existing large-scale language models typically lack a native retraction mechanism. Implementing content retraction usually requires custom development and integration of an SDK (Software Development Kit) at the application level that interfaces with the large model. However, this necessitates structural modifications to existing applications, leading to high development and implementation costs, significantly increasing software engineering costs and complexity. Furthermore, it lacks broad applicability and offers limited security and response speed for handling sensitive information.

[0003] In summary, optimizing the method for retracting illegal output content in large models to reduce complexity and improve processing security and response speed are urgent technical problems that need to be solved. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a method, apparatus, device, and medium for retracting illegal output content of large models, which can optimize the method for retracting illegal output content of large models to reduce complexity and improve processing security and response speed. The specific solution is as follows:

[0005] Firstly, this application provides a method for retracting illegal output content of a large model, including:

[0006] When the target large model is accessed through the target browser, during the page initialization process of the target browser, a preset front-end script is embedded into the front-end code in the form of a reverse proxy through the firewall server, so that the target browser runs the preset front-end script when loading the front-end code;

[0007] Based on the firewall server and the preset front-end script, the large model interaction interface path of the target large model is identified through a preset bidirectional communication protocol, so that the firewall server can listen to the interaction information of the target large model based on the large model interaction interface path.

[0008] The firewall server detects the output content of the large model in the interactive information to determine whether there is any illegal output content in the output content of the large model, and obtains the corresponding judgment result; the illegal output content is the content in the output content of the large model that meets the preset illegal judgment conditions.

[0009] If the judgment result indicates that there is illegal output content in the output content of the large model, a corresponding unique identifier is generated for the output content of the large model, and a retraction instruction including the unique identifier is sent to the preset front-end script through the preset two-way communication protocol, so that the preset front-end script can perform a retraction operation on the output content of the large model based on the retraction instruction.

[0010] Optionally, during the page initialization process of the target browser, embedding a preset front-end script into the front-end code in the form of a reverse proxy through a firewall server includes:

[0011] After the target browser receives a large model access request through the target front end and sends the large model access request to the browser server of the target browser, the firewall server intercepts the large model access request in the form of a reverse proxy and forwards the large model access request to the browser server, so that the browser server generates corresponding front-end code for the large model access request and feeds the front-end code back to the target browser;

[0012] The firewall server intercepts the front-end code, parses the front-end code to identify the target front-end code tag in the front-end code, and inserts the preset front-end script at the end of the target front-end code tag.

[0013] Optionally, after running the preset front-end script, the process further includes:

[0014] The preset front-end script creates a target storage space in the target browser's memory and continuously monitors the target browser's interface requests so as to save the monitored interface request information to the target storage space and identify the interface request information saved in the target storage space as historical interface request information.

[0015] Optionally, before identifying the large model interaction interface path of the target large model based on the firewall server and the preset front-end script through a preset bidirectional communication protocol, the method further includes:

[0016] After the preset front-end script starts running, a communication connection is established with the firewall server through the preset bidirectional communication protocol, and an initial interface identification instruction is sent to the firewall server. After the firewall server receives the initial interface identification instruction, it jumps to the step of identifying the large model interaction interface path of the target large model based on the firewall server and the preset front-end script through the preset bidirectional communication protocol. The initial interface identification instruction is used to inform the firewall server that the large model interaction interface has not yet been identified.

[0017] Optionally, the step of identifying the large model interaction interface path of the target large model based on the firewall server and the preset front-end script through a preset bidirectional communication protocol, so that the firewall server can listen to the interaction information of the target large model based on the large model interaction interface path, includes:

[0018] After the target browser receives the large model interaction request through the target front-end and sends the large model interaction request to the target large model, the firewall server intercepts the large model interaction request to obtain the large model input content in the large model interaction request, and sends the large model interaction request to the target large model so that the target large model generates the corresponding target large model output content in response to the large model interaction request, and sends the target large model output content to the target browser;

[0019] The firewall server intercepts the output content of the target large model and sends an interface path identification instruction, including the output content of the target large model and the input content of the large model, to the preset front-end script. The preset front-end script then iterates through the historical interface request information based on the interface path identification instruction to determine the interface path whose request parameters are consistent with the input content of the large model and whose response content is consistent with the output content of the target large model. The preset front-end script then identifies the interface path as the large model interaction interface path of the target large model and feeds back the large model interaction interface path to the firewall server.

[0020] The firewall server receives the large model interaction interface path and binds it to the large model interaction interface path so as to listen to the interaction information of the target large model based on the large model interaction interface path.

[0021] Optionally, the step of detecting the large model output content in the interactive information through the firewall server to determine whether there is any illegal output content in the large model output content and obtaining the corresponding judgment result includes:

[0022] The firewall server performs keyword matching on the output content of the large model in the interactive information based on a preset keyword matching method. If the matching result indicates that there is content in the output content of the large model that matches the target keyword, it is determined that there is illegal output content in the output content of the large model, and a corresponding judgment result is obtained. If the matching result indicates that there is no content in the output content of the large model that matches the target keyword, it is determined that there is no illegal output content in the output content of the large model, and a corresponding judgment result is obtained.

[0023] Alternatively, the firewall server can perform semantic analysis on the large model output content in the interactive information. If the analysis result indicates that the large model output content contains content that conforms to preset violation rules, then it is determined that the large model output content contains violation output content, and a corresponding judgment result is obtained. If the matching result indicates that the large model output content does not contain content that conforms to preset violation rules, then it is determined that the large model output content does not contain violation output content, and a corresponding judgment result is obtained.

[0024] Optionally, the step of generating a corresponding unique identifier for the output content of the large model and sending a retraction instruction including the unique identifier to the preset front-end script through the preset two-way communication protocol, so that the preset front-end script performs a retraction operation on the output content of the large model based on the retraction instruction, includes:

[0025] Generate a unique identifier for the output content of the large model, and append the unique identifier to the end of the output content of the large model;

[0026] The retraction instruction, including the unique identifier, is sent to the preset front-end script via the preset two-way communication protocol. The preset front-end script then obtains the unique identifier from the retraction instruction, locates the target parent element in the target browser's document object model based on the unique identifier, and replaces the content of the target parent element with the target prompt information to perform a retraction operation on the large model output content. The target parent element is the parent element including the unique identifier; the target prompt information is a prompt indicating that the large model output content has been detected as violating regulations and has been retracted.

[0027] Secondly, this application provides a device for retracting illegal output content of a large model, comprising:

[0028] The script embedding module is used to embed a preset front-end script into the front-end code in the form of a reverse proxy through a firewall server during the page initialization process of the target browser when the target large model is accessed through the target browser, so that the target browser runs the preset front-end script when loading the front-end code; the target large model is a model that meets the preset large model conditions;

[0029] The path identification module is used to identify the large model interaction interface path of the target large model based on the firewall server and the preset front-end script through a preset bidirectional communication protocol, so that the firewall server can listen to the interaction information of the target large model based on the large model interaction interface path.

[0030] The content detection module is used to detect the output content of the large model in the interactive information through the firewall server, so as to determine whether there is any illegal output content in the output content of the large model and obtain the corresponding judgment result; the illegal output content is the content in the output content of the large model that meets the preset illegal judgment conditions.

[0031] The instruction sending module is used to generate a corresponding unique identifier for the output content of the large model if the judgment result indicates that there is illegal output content in the output content of the large model, and send a withdrawal instruction including the unique identifier to the preset front-end script through the preset two-way communication protocol, so that the preset front-end script can perform a withdrawal operation on the output content of the large model based on the withdrawal instruction.

[0032] Thirdly, this application provides an electronic device, comprising:

[0033] Memory, used to store computer programs;

[0034] A processor is used to execute the computer program to implement the aforementioned method for retracting the illegal output content of the large model.

[0035] Fourthly, this application provides a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned method for retracting illegal output content of a large model.

[0036] In this application, when a target large model is accessed through a target browser, during the page initialization process of the target browser, a preset front-end script is embedded into the front-end code in the form of a reverse proxy through a firewall server, so that the target browser runs the preset front-end script when loading the front-end code; the target large model is a model that meets preset large model conditions; based on the firewall server and the preset front-end script, the large model interaction interface path of the target large model is identified through a preset bidirectional communication protocol, so that the firewall server listens to the interaction information of the target large model based on the large model interaction interface path; the firewall server detects the output content of the large model in the interaction information to determine whether there is any illegal output content in the output content of the large model, and obtains a corresponding judgment result; the illegal output content is the content in the output content of the large model that meets the preset illegal judgment conditions; if the judgment result indicates that there is illegal output content in the output content of the large model, a corresponding unique identifier is generated for the output content of the large model, and a withdrawal instruction including the unique identifier is sent to the preset front-end script through the preset bidirectional communication protocol, so that the preset front-end script performs a withdrawal operation on the output content of the large model based on the withdrawal instruction. As can be seen from the above, in this application, when the target large model is accessed through the target browser and the browser page is initialized, the firewall server embeds a preset front-end script in the form of a reverse proxy into the front-end code. This causes the browser to run the script when loading the front-end code. With the help of the firewall server and the preset front-end script, the interaction interface path of the target large model is identified through a preset bidirectional communication protocol. This allows the firewall server to listen to the interaction information of the target large model based on this path. The firewall server detects the output content of the large model in the interaction information and determines whether there is any illegal output content that meets the preset violation conditions. If so, a unique identifier is generated for the output content, and a retraction instruction containing this identifier is sent to the preset front-end script through the preset bidirectional communication protocol. The preset front-end script then performs the retraction operation. In this way, through the process described above in this application, a pre-defined front-end script is embedded in the form of a reverse proxy, enabling real-time monitoring of large model interactions. The bidirectional communication protocol ensures smooth interface identification and command transmission, while the detection and retraction mechanism for illegal content can filter inappropriate output in a timely manner, effectively standardizing the interactive behavior of large models, ensuring the compliance of content obtained by users, improving the security and reliability of large model applications, and further optimizing the method for retracting illegal output content of large models to reduce complexity and improve the security and response speed of processing. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0038] Figure 1 This application discloses a flowchart of a method for retracting illegal output content of a large model.

[0039] Figure 2 This is a flowchart illustrating a reverse proxy and script injection process disclosed in this application;

[0040] Figure 3 This is a schematic diagram of the process for path recognition of a large model interaction interface disclosed in this application;

[0041] Figure 4 This is a flowchart illustrating the detection and withdrawal process for one type of illegal content disclosed in this application;

[0042] Figure 5 This is a schematic diagram of a device for retracting illegal output content of a large model disclosed in this application;

[0043] Figure 6 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0044] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0045] With the widespread application of large-scale language models, their potential risks are becoming increasingly prominent. In particular, the issue of manipulating the model's output to contain inappropriate content via specific prompts poses a significant challenge. To improve system security and user experience, it's crucial to promptly retract such content. However, existing large-scale language models typically lack a native retraction mechanism. Implementing content retraction usually requires custom development and integration of an SDK (Software Development Kit) at the application level that interfaces with the large model. However, this necessitates structural modifications to existing applications, leading to high development and implementation costs, significantly increasing software engineering costs and complexity. Furthermore, it lacks broad applicability and offers limited security and response speed for handling sensitive information.

[0046] To overcome the aforementioned technical problems, this application provides a method for retracting illegal output content of large models, optimizing the method to reduce complexity and improve processing security and response speed.

[0047] See Figure 1 As shown in the figure, this invention discloses a method for retracting illegal output content of a large model, including:

[0048] Step S11: When the target large model is accessed through the target browser, during the page initialization process of the target browser, a preset front-end script is embedded into the front-end code in the form of a reverse proxy through the firewall server, so that the target browser runs the preset front-end script when loading the front-end code.

[0049] In this embodiment, when the target large model is accessed through the target browser, during the browser page initialization process, a preset front-end script is embedded into the front-end code in the form of a reverse proxy through a firewall server, so that the target browser automatically runs the preset front-end script when loading the front-end code. The target large model refers to a type of generative language model designed based on a deep learning architecture, capable of automatically generating text or responding to queries based on context, and widely used in natural language processing tasks, including but not limited to dialogue systems, text generation, and information retrieval. The firewall server (i.e., MAF, Model Access Firewall) is a security device or software that uses a proxy mechanism to monitor and control the input and output data during the large model access process in real time, effectively intercepting potential malicious requests while ensuring that content conforming to preset rules is allowed to pass. The preset front-end script is a preset custom JavaScript script (a scripting programming language) containing a basic framework for subsequently recording API (Application Programming Interface) requests, establishing connections, and performing content withdrawal operations.

[0050] Specifically, when the target large model is accessed, after the target browser receives the large model access request through the target front-end and sends the large model access request to the browser server of the target browser, the firewall server intercepts the large model access request in the form of a reverse proxy and forwards the large model access request to the browser server, so that the browser server generates corresponding front-end code for the large model access request and feeds the front-end code back to the target browser; the firewall server intercepts the front-end code and parses the front-end code to determine the target front-end code tag in the front-end code, and inserts the preset front-end script at the end of the target front-end code tag. The target front-end code tag can be a tag (head tag) or a tag (body tag). That is, as shown... Figure 2 The diagram illustrates a reverse proxy and script injection process provided in this application. The firewall server acts as an intermediate layer, proxying web page requests from a large-scale application. After the target browser receives the large-scale access request through the target frontend and sends it to its browser server, the firewall server intercepts the request in a reverse proxy manner and forwards it to the browser server. When the browser server generates frontend code and sends it back to the target browser, the firewall server intercepts the frontend code again, parses its HTML (Hypertext Markup Language) document structure to identify the target frontend code tag, and dynamically inserts a pre-defined frontend script at the end of that tag.

[0051] It should be noted that after the preset front-end script runs, its processing flow is as follows: create a target storage space in the memory of the target browser through the preset front-end script, and continuously monitor the interface request situation of the target browser, so as to save the monitored interface request information to the target storage space, and determine the interface request information saved in the target storage space as historical interface request information. Among them, the target storage space is a storage space in the form of an object or an array, which is used to store information such as API request paths, parameters, and return contents; the interface request information includes, but is not limited to, request paths (such as / api / chat), request parameters (such as {"prompt": "Hello"}), response contents (such as the text returned by the large model), etc. That is, the preset front-end script creates an object or an array in the memory of the target browser, continuously monitors the interface request situation of the target browser, saves the monitored interface request information to the target storage space, and determines these saved information as historical interface request information to form a historical request log for subsequent API path identification. In this way, through the form of reverse proxy embedding the script, this embodiment can monitor and manage the interaction of the large model without interfering with the normal access of the large model, which helps to improve the controllability of the large model usage process and ensure the standardization and security of the large model interaction; complete the script embedding without affecting the normal access process, ensuring that the front-end script can be loaded and run by the browser together with the front-end code, which not only ensures the smoothness of the large model access, but also enhances the controllability of the interaction process; create a dedicated storage space in the browser memory and monitor interface requests in real time, which can systematically record and store the historical information of interface requests, ensure the continuity and integrity of information recording, and help to more accurately grasp the interaction dynamics between the large model and the browser for subsequent API path identification.

[0052] Step S12: Based on the firewall server and the preset front-end script, identify the large model interaction interface path of the target large model through a preset two-way communication protocol, so that the firewall server can monitor the interaction information of the target large model based on the large model interaction interface path.

[0053] In this embodiment, with the help of the firewall server and the preset front-end script, the large model interaction interface path of the target large model is identified through a preset two-way communication protocol, so that the firewall server can monitor the interaction information of the target large model based on this path. Among them, the preset two-way communication protocol can be the WebSocket protocol (a protocol for realizing real-time two-way communication between the browser and the server).

[0054] It should be noted that before identifying the large model interaction interface path, a connection needs to be established between the firewall server and the preset front-end script, and it needs to be determined whether the large model interaction interface path can be identified. The process is as follows: After the preset front-end script starts running, it establishes a communication connection with the firewall server through the preset bidirectional communication protocol and sends an initial interface identification command to the firewall server. Upon receiving the initial interface identification command, the firewall server jumps to the step of identifying the large model interaction interface path of the target large model based on the firewall server and the preset front-end script through the preset bidirectional communication protocol. The initial interface identification command is used to inform the firewall server that the large model interaction interface has not yet been identified. That is, after the preset front-end script starts running, it establishes a communication connection with the firewall server through the preset bidirectional communication protocol and simultaneously sends the initial interface identification command `reportLLMApiPath`, the format of which is as follows:

[0055] {

[0056] "command": "reportLLMApiPath",

[0057] "llmApiPath": "unknown"

[0058] };

[0059] This instruction informs the firewall server that it has not yet identified the large model interaction interface, specifically the LLM (Large Language Model) interface (llmApiPath: "unknown"). The initial value of llmApiPath is "unknown," instructing the firewall server to proceed to the step of identifying the target large model interaction interface path upon receiving this instruction. It should be noted that this initial interface identification instruction is initiated when the firewall server first establishes a communication connection with the firewall server via the preset bidirectional communication protocol. Since this is the first connection, it is impossible to know the target large model's interaction interface at this time; therefore, the initial interface identification instruction is needed to identify the large model interaction interface path.

[0060] Specifically, after the target browser receives a large model interaction request through the target front-end and sends the large model interaction request to the target large model, the firewall server intercepts the large model interaction request to obtain the large model input content in the large model interaction request, and then sends the large model interaction request to the target large model, so that the target large model generates corresponding target large model output content in response to the large model interaction request, and sends the target large model output content to the target browser; the firewall server intercepts the target large model output content and sends a message including the target large model output content and... The interface path identification instruction for the large model input content allows the preset front-end script to traverse the historical interface request information based on the interface path identification instruction to determine the interface path whose request parameters match the large model input content and whose response content matches the target large model output content. This interface path is then identified as the target large model's large model interaction interface path, and this large model interaction interface path is fed back to the firewall server. The firewall server receives the large model interaction interface path and binds it to the target large model's interaction interface path to monitor the target large model's interaction information. That is, as... Figure 3 The diagram illustrates a process for identifying the interaction interface path of a large model according to this application. When a user inputs content on a page and triggers a large model interaction request, the target browser receives the large model interaction request through the target front-end and sends it to the target large model. During the transmission process, the firewall server intercepts the request to obtain the large model input content, and then sends the request to the target large model. When the target large model generates its output content and sends it to the target browser, the firewall server intercepts the output content and then sends an interface path identification instruction containing the input and output content to the preset front-end script. The instruction format is as follows:

[0061] {

[0062] "command": "analyzeLLMApiPath",

[0063] "inputContent": "{Content to be input to the LLM}",

[0064] "outputContent": "{LLM output content}"

[0065] };

[0066] Wherein, analyzeLLMApiPath is the interface path identification instruction; inputContent is the input content of the large model; and outputContent is the output content of the target large model. After receiving the interface path identification instruction, the preset front-end script iterates through the historical interface request information based on this instruction to determine the interface path where the request parameters match the input content and the response content matches the output content. This path is then used as the large model interaction interface path of the target large model (e.g., / v1 / chat / completions), and fed back to the firewall server via the interface path feedback instruction. The instruction format of the interface path feedback instruction is as follows:

[0067] {

[0068] "command": "reportLLMApiPath",

[0069] "llmApiPath": " / v1 / chat / completions"

[0070] };

[0071] Wherein, `reportLLMApiPath` is the interface path feedback instruction; `llmApiPath` represents the actual identified interface path. Upon receiving the interface path feedback instruction, the firewall server can confirm that the front-end has located the large model interaction interface path and bind it to it, so as to listen to the interaction information of the target large model based on this path. In this way, this embodiment achieves collaboration between the firewall server and the preset front-end script through a preset bidirectional communication protocol. After accurately identifying the interaction interface path, it ensures that the firewall server can selectively listen to interaction information, guaranteeing the accuracy and effectiveness of the listening and enhancing the monitoring capability of the large model interaction process. Timely triggering of the interface identification process avoids subsequent monitoring failures caused by interface identification delays or omissions, ensuring the smooth start of the entire large model interaction monitoring process. Through the firewall server's interception of interaction requests and output content, combined with the front-end script's traversal and matching of historical interface information, the interaction interface path of the large model is accurately locked, ensuring both the accuracy of interface path identification and targeted monitoring of the large model interaction process, improving the controllability and monitoring efficiency of the large model interaction.

[0072] Step S13: The firewall server detects the output content of the large model in the interaction information to determine whether there is any illegal output content in the output content of the large model, and obtains the corresponding judgment result; the illegal output content is the content in the output content of the large model that meets the preset illegal judgment conditions.

[0073] In this embodiment, the firewall server detects the large model output content in the interactive information and determines whether there is any illegal output content that meets the preset violation judgment conditions, such as sensitive words, trade secrets, dangerous code, etc., and then obtains the corresponding judgment result.

[0074] Specifically, the firewall server performs keyword matching on the large model output content in the interactive information based on a preset keyword matching method. If the matching result indicates that the large model output content contains content that matches the target keyword, then it is determined that the large model output content contains illegal output content, and a corresponding judgment result is obtained. If the matching result indicates that the large model output content does not contain content that matches the target keyword, then it is determined that the large model output content does not contain illegal output content, and a corresponding judgment result is obtained. Alternatively, the firewall server performs semantic analysis on the large model output content in the interactive information. If the analysis result indicates that the large model output content contains content that conforms to preset violation rules, then it is determined that the large model output content contains illegal output content, and a corresponding judgment result is obtained. If the matching result indicates that the large model output content does not contain content that conforms to preset violation rules, then it is determined that the large model output content does not contain illegal output content, and a corresponding judgment result is obtained. That is, as... Figure 4 The diagram illustrates a process for detecting and retracting illegal content provided in this application. The firewall server in this embodiment can detect illegal output content using multiple methods. First, based on a preset keyword matching method, it performs keyword matching on the output content of the large model in the interactive information. If content matching the target keyword exists, it is determined that there is illegal output content; otherwise, it is determined that there is none. Second, it performs semantic analysis on the output content of the large model. If the analysis results show that content conforms to preset violation rules, it is determined that there is illegal output content; otherwise, it is determined that there is none. In this way, this embodiment screens the output content of the large model through clear violation judgment criteria, enabling timely identification of non-compliant information, which helps to standardize the output of the large model and improve the security and reliability of the large model application. By screening the output content of the large model from different dimensions, keyword matching can quickly lock in clear violation information, while semantic analysis can deeply understand the meaning of the content to identify potential violations. This ensures both detection efficiency and improves the accuracy of identifying complex illegal content, further strengthening the supervision of the output content of the large model.

[0075] Step S14: If the judgment result indicates that there is illegal output content in the output content of the large model, then a corresponding unique identifier is generated for the output content of the large model, and a retraction instruction including the unique identifier is sent to the preset front-end script through the preset two-way communication protocol, so that the preset front-end script can perform a retraction operation on the output content of the large model based on the retraction instruction.

[0076] In this embodiment, if it is determined that there is illegal output content, a corresponding unique identifier, i.e., a random string, is generated for the illegal large model output content. Then, a retraction instruction containing the unique identifier is sent to the preset front-end script through the preset two-way communication protocol, so that the preset front-end script performs a retraction operation on the large model output content based on the retraction instruction.

[0077] Specifically, after confirming the existence of the violation output content, a corresponding unique identifier is generated for the large model output content, and this unique identifier is appended to the end of the large model output content. A retraction command including the unique identifier is sent to the preset front-end script via the preset two-way communication protocol. The preset front-end script then obtains the unique identifier from the retraction command, searches for the target parent element in the target browser's document object model based on the unique identifier, and replaces the content of the target parent element with the target prompt information to perform a retraction operation on the large model output content. The target parent element is the parent element including the unique identifier; the target prompt information is a prompt indicating that the large model output content has been detected as violating output content and has been retracted. That is, a corresponding unique identifier is generated for the large model output content, and this identifier is appended to the end of the large model output content. Then, a retraction command containing this unique identifier is sent to the preset front-end script via the preset two-way communication protocol. The command format of the retraction command is as follows:

[0078] {

[0079] "command": "revocation",

[0080] "outputChars": "The text content that has been output",

[0081] "uuid": "eb861a76-3b5a-4ae7-b52b-b424f868aa0f"

[0082] };

[0083] Wherein, `revocation` is the revocation instruction; `uuid` is the unique identifier; and `outputChars` is the currently output content, including the unique identifier. After the preset front-end script receives the revocation instruction, it obtains the unique identifier from the instruction, finds the target parent element containing the identifier in the target browser's Document Object Model (DOM tree), and then replaces the content in the target parent element with a target prompt message indicating that the output content has been revoked due to violation, such as "Violation detected, content has been revoked," thereby completing the revocation operation of the large model's output content. In this way, this embodiment achieves efficient instruction transmission through a two-way communication protocol; by binding the unique identifier to the output content, the front-end script can accurately locate the violation content to be processed, and combine it with DOM operations to achieve content replacement, ensuring the accuracy and targeting of the revocation operation. At the same time, the clear prompt message allows users to clearly understand the content status, improving the transparency of the interaction, quickly eliminating the adverse effects of violation content, and further strengthening the supervision effect of the large model's output.

[0084] As can be seen from the above, in this embodiment of the application, when the target large model is accessed through the target browser and the browser page is initialized, the firewall server embeds a preset front-end script in the form of a reverse proxy into the front-end code, so that the browser runs the script when loading the front-end code. With the help of the firewall server and the preset front-end script, the interaction interface path of the target large model is identified through a preset bidirectional communication protocol, so that the firewall server can listen to the interaction information of the target large model based on the path. The firewall server detects the output content of the large model in the interaction information and determines whether there is any illegal output content that meets the preset violation conditions. If so, a unique identifier is generated for the output content, and a retraction instruction containing this identifier is sent to the preset front-end script through the preset bidirectional communication protocol, and the preset front-end script performs the retraction operation.In this way, through the above-described process of the embodiments of this application, on the one hand, by embedding scripts in the form of reverse proxy, the interaction of the large model can be monitored and managed without interfering with the normal access of the large model, which helps to improve the controllability of the large model usage process and ensure the standardization and security of the large model interaction; on the other hand, the script embedding is completed without affecting the normal access process, ensuring that the front-end script can be loaded and run by the browser along with the front-end code, which not only ensures the smoothness of the large model access but also enhances the controllability of the interaction process; and on the other hand, by creating a dedicated storage space in the browser memory and listening to interface requests in real time, the historical information of interface requests can be systematically recorded and stored, ensuring information recording. The continuity and integrity of this system help to more accurately grasp the dynamic interaction between the large model and the browser, enabling subsequent API path identification. On one hand, a pre-defined bidirectional communication protocol enables collaboration between the firewall server and the pre-defined front-end script. Accurate identification of the interaction interface path ensures that the firewall server can selectively monitor interaction information, guaranteeing the accuracy and effectiveness of the monitoring and enhancing the monitoring capabilities of the large model's interaction process. On the other hand, timely triggering of the interface identification process avoids subsequent monitoring failures due to interface identification delays or omissions, ensuring the smooth start of the entire large model interaction monitoring process. Furthermore, the firewall server's interception of interaction requests and output content, combined with the front-end script... By traversing and matching historical interface information, the interaction interface paths of the large model were accurately located. This ensured the accuracy of interface path identification and enabled targeted supervision of the large model's interaction process, improving the controllability and efficiency of supervision. On the one hand, screening the output content of the large model through clear violation judgment criteria can promptly identify non-compliant information, helping to standardize the output of the large model and improve the security and reliability of its application. On the other hand, screening the output content of the large model from different dimensions—keyword matching can quickly identify clear violation information, while semantic analysis can deeply understand the meaning of the content to identify potential violations—ensures both detection efficiency and improves the handling of complex violations. The improved accuracy of content identification further strengthens the supervision of the output content of the large model. On the one hand, it achieves efficient transmission of instructions through a two-way communication protocol; on the other hand, by binding a unique identifier to the output content, the front-end script can accurately locate the illegal content that needs to be processed. Combined with document object model operations, it realizes content replacement, ensuring the accuracy and targeting of the retraction operation. At the same time, clear prompts allow users to clearly understand the content status, improving the transparency of the interaction and quickly eliminating the adverse effects of illegal content. This further strengthens the supervision of the output of the large model, thereby optimizing the method for retracting illegal output content of the large model to reduce complexity and improve the security and response speed of processing.

[0085] Accordingly, see Figure 5 As shown in the embodiment of this application, a device for retracting illegal output content of a large model is also provided, including:

[0086] The script embedding module 11 is used to embed a preset front-end script into the front-end code in the form of a reverse proxy through a firewall server during the page initialization process of the target browser when the target large model is accessed through the target browser, so that the target browser runs the preset front-end script when loading the front-end code; the target large model is a model that meets the preset large model conditions.

[0087] The path identification module 12 is used to identify the large model interaction interface path of the target large model based on the firewall server and the preset front-end script through a preset bidirectional communication protocol, so that the firewall server can listen to the interaction information of the target large model based on the large model interaction interface path.

[0088] The content detection module 13 is used to detect the output content of the large model in the interactive information through the firewall server, so as to determine whether there is any illegal output content in the output content of the large model and obtain the corresponding judgment result; the illegal output content is the content in the output content of the large model that meets the preset illegal judgment conditions.

[0089] The instruction sending module 14 is used to generate a corresponding unique identifier for the output content of the large model if the judgment result indicates that there is illegal output content in the output content of the large model, and send a withdrawal instruction including the unique identifier to the preset front-end script through the preset two-way communication protocol, so that the preset front-end script can perform a withdrawal operation on the output content of the large model based on the withdrawal instruction.

[0090] As can be seen from the above, in this embodiment of the application, when the target large model is accessed through the target browser and the browser page is initialized, the firewall server embeds a preset front-end script in the form of a reverse proxy into the front-end code, so that the browser runs the script when loading the front-end code. With the help of the firewall server and the preset front-end script, the interaction interface path of the target large model is identified through a preset bidirectional communication protocol, so that the firewall server can listen to the interaction information of the target large model based on the path. The firewall server detects the output content of the large model in the interaction information and determines whether there is any illegal output content that meets the preset violation conditions. If so, a unique identifier is generated for the output content, and a retraction instruction containing this identifier is sent to the preset front-end script through the preset bidirectional communication protocol, and the preset front-end script performs the retraction operation. In this way, through the above process of the embodiments of this application, a preset front-end script is embedded in the form of a reverse proxy, realizing real-time monitoring of large model interactions. The bidirectional communication protocol ensures smooth interface identification and instruction transmission, and the detection and retraction mechanism of illegal content can filter inappropriate output in a timely manner, effectively standardize the interactive behavior of large models, ensure the compliance of content obtained by users, improve the security and reliability of large model applications, and further optimize the method of retracting illegal output content of large models to reduce complexity and improve the security and response speed of processing.

[0091] In some specific embodiments, the script embedding module 11 may specifically include:

[0092] The request forwarding unit is configured to, after the target browser receives a large model access request through the target front-end and sends the large model access request to the browser server of the target browser, intercept the large model access request through the firewall server in the form of a reverse proxy, and forward the large model access request to the browser server, so that the browser server generates corresponding front-end code for the large model access request and feeds back the front-end code to the target browser;

[0093] The code parsing unit is used to intercept the front-end code through the firewall server, parse the front-end code to determine the target front-end code tag in the front-end code, and insert the preset front-end script to the end of the target front-end code tag.

[0094] In some specific implementations, the method for retracting the illegal output content of the large model may further include:

[0095] The situation monitoring unit is used to create a target storage space in the memory of the target browser through the preset front-end script, and continuously monitor the interface request situation of the target browser so as to save the monitored interface request information to the target storage space, and determine the interface request information saved in the target storage space as historical interface request information.

[0096] In some specific implementations, the method for retracting the illegal output content of the large model may further include:

[0097] The first instruction sending unit is configured to establish a communication connection with the firewall server through the preset bidirectional communication protocol after the preset front-end script starts running, and send an initial interface identification instruction to the firewall server, so that after the firewall server receives the initial interface identification instruction, it will jump to the step of identifying the large model interaction interface path of the target large model based on the firewall server and the preset front-end script through the preset bidirectional communication protocol; the initial interface identification instruction is an instruction used to inform the firewall server that the large model interaction interface has not yet been identified.

[0098] In some specific embodiments, the path identification module 12 may specifically include:

[0099] The request interception unit is configured to intercept the large model interaction request through the firewall server after the target browser receives the large model interaction request through the target front end and sends the large model interaction request to the target large model, so as to obtain the large model input content in the large model interaction request and send the large model interaction request to the target large model, so that the target large model generates corresponding target large model output content in response to the large model interaction request and sends the target large model output content to the target browser;

[0100] The second instruction sending unit is used to intercept the output content of the target large model through the firewall server, and send an interface path identification instruction including the output content of the target large model and the input content of the large model to the preset front-end script, so that the preset front-end script can traverse the historical interface request information based on the interface path identification instruction to determine the interface path whose request parameters are consistent with the input content of the large model and whose response content is consistent with the output content of the target large model, and determine the interface path as the large model interaction interface path of the target large model, and feed back the large model interaction interface path to the firewall server;

[0101] The path binding unit is used to receive the large model interaction interface path through the firewall server and bind it to the large model interaction interface path so as to listen to the interaction information of the target large model based on the large model interaction interface path.

[0102] In some specific embodiments, the content detection module 13 may specifically include:

[0103] The first condition determination unit is used to perform keyword matching on the output content of the large model in the interactive information through the firewall server based on a preset keyword matching method. If the obtained matching result indicates that there is content in the output content of the large model that matches the target keyword, then it is determined that there is illegal output content in the output content of the large model, and a corresponding judgment result is obtained; if the matching result indicates that there is no content in the output content of the large model that matches the target keyword, then it is determined that there is no illegal output content in the output content of the large model, and a corresponding judgment result is obtained.

[0104] Alternatively, the second condition determination unit is used to perform semantic analysis on the large model output content in the interactive information through the firewall server. If the analysis result indicates that there is content in the large model output content that conforms to the preset violation rules, then it is determined that there is violation output content in the large model output content, and a corresponding judgment result is obtained; if the matching result indicates that there is no content in the large model output content that conforms to the preset violation rules, then it is determined that there is no violation output content in the large model output content, and a corresponding judgment result is obtained.

[0105] In some specific embodiments, the instruction sending module 14 may specifically include:

[0106] The identifier concatenation unit is used to generate a corresponding unique identifier for the output content of the large model and concatenate the unique identifier to the end of the output content of the large model.

[0107] A withdrawal instruction sending unit is configured to send a withdrawal instruction including the unique identifier to the preset front-end script via the preset bidirectional communication protocol, so that the preset front-end script can obtain the unique identifier in the withdrawal instruction, and based on the unique identifier, find the target parent element in the document object model of the target browser, and replace the content in the target parent element with the target prompt information, so as to perform a withdrawal operation on the output content of the large model; the target parent element is the parent element including the unique identifier; the target prompt information is a prompt information indicating that the output content of the large model has been detected as violating the output content and has been withdrawn.

[0108] Furthermore, embodiments of this application also disclose an electronic device, Figure 6 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the large-model violation output content retraction method disclosed in any of the foregoing embodiments. Furthermore, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0109] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0110] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0111] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the large-scale illegal output content retraction method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.

[0112] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned method for retracting the illegal output content of the large model. The specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0113] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0114] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0115] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0116] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0117] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for retracting illegal output content of a large model, characterized in that, include: When the target large model is accessed through the target browser, during the page initialization process of the target browser, a preset front-end script is embedded into the front-end code in the form of a reverse proxy through the firewall server, so that the target browser runs the preset front-end script when loading the front-end code; Based on the firewall server and the preset front-end script, the large model interaction interface path of the target large model is identified through a preset bidirectional communication protocol, so that the firewall server can listen to the interaction information of the target large model based on the large model interaction interface path. The firewall server detects the output content of the large model in the interactive information to determine whether there is any illegal output content in the output content of the large model, and obtains the corresponding judgment result; the illegal output content is the content in the output content of the large model that meets the preset illegal judgment conditions. If the judgment result indicates that there is illegal output content in the output content of the large model, a corresponding unique identifier is generated for the output content of the large model, and a retraction instruction including the unique identifier is sent to the preset front-end script through the preset two-way communication protocol, so that the preset front-end script can perform a retraction operation on the output content of the large model based on the retraction instruction. Specifically, the step of generating a unique identifier for the output content of the large model and sending a retraction instruction including the unique identifier to the preset front-end script via the preset two-way communication protocol, so that the preset front-end script can perform a retraction operation on the output content of the large model based on the retraction instruction, includes: Generate a unique identifier for the output content of the large model, and append the unique identifier to the end of the output content of the large model; The retraction instruction, including the unique identifier, is sent to the preset front-end script via the preset two-way communication protocol. The preset front-end script then obtains the unique identifier from the retraction instruction, locates the target parent element in the target browser's document object model based on the unique identifier, and replaces the content of the target parent element with the target prompt information to perform a retraction operation on the large model output content. The target parent element is the parent element including the unique identifier; the target prompt information is a prompt indicating that the large model output content has been detected as violating regulations and has been retracted.

2. The method for retracting illegal output content of a large model according to claim 1, characterized in that, During the page initialization process in the target browser, the preset front-end script is embedded into the front-end code via a firewall server in the form of a reverse proxy, including: After the target browser receives a large model access request through the target front end and sends the large model access request to the browser server of the target browser, the firewall server intercepts the large model access request in the form of a reverse proxy and forwards the large model access request to the browser server, so that the browser server generates corresponding front-end code for the large model access request and feeds the front-end code back to the target browser; The firewall server intercepts the front-end code, parses the front-end code to identify the target front-end code tag in the front-end code, and inserts the preset front-end script at the end of the target front-end code tag.

3. The method for retracting illegal output content of a large model according to claim 1, characterized in that, After running the preset front-end script, the process also includes: The preset front-end script creates a target storage space in the target browser's memory and continuously monitors the target browser's interface requests so as to save the monitored interface request information to the target storage space and identify the interface request information saved in the target storage space as historical interface request information.

4. The method for retracting illegal output content of a large model according to claim 1, characterized in that, Before identifying the large model interaction interface path of the target large model based on the firewall server and the preset front-end script through a preset bidirectional communication protocol, the method further includes: After the preset front-end script starts running, a communication connection is established with the firewall server through the preset bidirectional communication protocol, and an initial interface identification instruction is sent to the firewall server. After the firewall server receives the initial interface identification instruction, it jumps to the step of identifying the large model interaction interface path of the target large model based on the firewall server and the preset front-end script through the preset bidirectional communication protocol. The initial interface identification instruction is used to inform the firewall server that the large model interaction interface has not yet been identified.

5. The method for retracting illegal output content of a large model according to claim 3, characterized in that, The step of identifying the large model interaction interface path of the target large model based on the firewall server and the preset front-end script through a preset bidirectional communication protocol, so that the firewall server can listen to the interaction information of the target large model based on the large model interaction interface path, includes: After the target browser receives the large model interaction request through the target front-end and sends the large model interaction request to the target large model, the firewall server intercepts the large model interaction request to obtain the large model input content in the large model interaction request, and sends the large model interaction request to the target large model so that the target large model generates the corresponding target large model output content in response to the large model interaction request, and sends the target large model output content to the target browser; The firewall server intercepts the output content of the target large model and sends an interface path identification instruction, including the output content of the target large model and the input content of the large model, to the preset front-end script. The preset front-end script then iterates through the historical interface request information based on the interface path identification instruction to determine the interface path whose request parameters are consistent with the input content of the large model and whose response content is consistent with the output content of the target large model. The preset front-end script then identifies the interface path as the large model interaction interface path of the target large model and feeds back the large model interaction interface path to the firewall server. The firewall server receives the large model interaction interface path and binds it to the large model interaction interface path so as to listen to the interaction information of the target large model based on the large model interaction interface path.

6. The method for retracting illegal output content of a large model according to claim 1, characterized in that, The step involves the firewall server detecting the output content of the large model in the interactive information to determine whether there is any illegal output content in the output content of the large model, and obtaining the corresponding judgment result, including: The firewall server performs keyword matching on the output content of the large model in the interactive information based on a preset keyword matching method. If the matching result indicates that there is content in the output content of the large model that matches the target keyword, it is determined that there is illegal output content in the output content of the large model, and a corresponding judgment result is obtained. If the matching result indicates that there is no content in the output content of the large model that matches the target keyword, it is determined that there is no illegal output content in the output content of the large model, and a corresponding judgment result is obtained. Alternatively, the firewall server can perform semantic analysis on the large model output content in the interactive information. If the analysis result indicates that the large model output content contains content that conforms to preset violation rules, then it is determined that the large model output content contains violation output content, and a corresponding judgment result is obtained. If the matching result indicates that the large model output content does not contain content that conforms to preset violation rules, then it is determined that the large model output content does not contain violation output content, and a corresponding judgment result is obtained.

7. A device for retracting illegal output content of a large model, characterized in that, include: The script embedding module is used to embed a preset front-end script into the front-end code in the form of a reverse proxy through the firewall server during the page initialization process of the target browser when the target large model is accessed through the target browser, so that the target browser runs the preset front-end script when loading the front-end code; The target large model is a model that meets the preset large model conditions; The path identification module is used to identify the large model interaction interface path of the target large model based on the firewall server and the preset front-end script through a preset bidirectional communication protocol, so that the firewall server can listen to the interaction information of the target large model based on the large model interaction interface path. The content detection module is used to detect the output content of the large model in the interactive information through the firewall server, so as to determine whether there is any illegal output content in the output content of the large model and obtain the corresponding judgment result. The violation output content refers to the content in the output content of the large model that meets the preset violation judgment conditions; The instruction sending module is used to generate a corresponding unique identifier for the output content of the large model if the judgment result indicates that there is illegal output content in the output content of the large model, and send a withdrawal instruction including the unique identifier to the preset front-end script through the preset two-way communication protocol, so that the preset front-end script can perform a withdrawal operation on the output content of the large model based on the withdrawal instruction. The instruction sending module includes: The identifier concatenation unit is used to generate a corresponding unique identifier for the output content of the large model and concatenate the unique identifier to the end of the output content of the large model. A withdrawal instruction sending unit is configured to send a withdrawal instruction including the unique identifier to the preset front-end script via the preset bidirectional communication protocol, so that the preset front-end script can obtain the unique identifier in the withdrawal instruction, and based on the unique identifier, find the target parent element in the document object model of the target browser, and replace the content in the target parent element with the target prompt information, so as to perform a withdrawal operation on the output content of the large model; the target parent element is the parent element including the unique identifier; the target prompt information is a prompt information indicating that the output content of the large model has been detected as violating the output content and has been withdrawn.

8. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the method for retracting illegal output content of a large model as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, Used to store computer programs; wherein, when the computer programs are executed by a processor, they implement the method for retracting illegal output content of a large model as described in any one of claims 1 to 6.

Images