Method and device for detecting attack behavior, storage medium and electronic equipment

Abstract

The embodiment of the invention discloses a method and device for detecting an attack behavior, a storage medium and electronic equipment, response content returned by a target large model based on a question request is received in response to a target application, the response content is analyzed, and the response content comprises code information; in the analysis process, determining whether the response content has an attack behavior or not according to a preset detection rule on the basis of tangency points embedded in one or more target functions related to the sensitive operation; and if it is determined that the response content has an attack behavior, executing a preset response operation for the response content, otherwise, enabling the target function to continue to be executed, and rendering and displaying presentation information corresponding to the response content in the target application.

Description

Technical Field

[0001] The present invention relates to computer technology, and in particular to a method, device, storage medium and electronic device for detecting attack behavior. Background Art

[0002] With the rapid development of technology, large language models have become a hot topic in the field of artificial intelligence. However, with its widespread application, security issues have become increasingly prominent. Recently, OWASP (an open source, non-profit global security organization dedicated to application software security research) released the top ten security threats of LLM. Front-end malicious code attacks against large models are also called Insecure Output Handling. This vulnerability occurs when downstream components blindly accept the output of large language models (LLMs) without proper review.

[0003] In some cases, LLM will generate code information and return it to the user. These codes will cause XSS (Cross Site Scripting) through browser parsing. Attackers can use this vulnerability to steal the current user's cookies, geographic location and other private information accessing the large model, which poses a serious risk of privacy leakage. A common defense solution is to use rules to filter malicious code in the output link of the large model. This rule-based defense method is extremely easy to be bypassed by flexible and changeable malicious codes, and the actual effect is not good; or directly disable the code function on the access end (browser or APP). Although this can completely solve the corresponding security risks, it greatly weakens the display function of the access end and is unlikely to be implemented. Summary of the invention

[0004] The purpose of the embodiments of this specification is to provide a method, device, storage medium and electronic device for detecting attack behavior.

[0005] The embodiment of this specification provides a method for detecting attack behavior. The method proposes a technology for protecting a large model from front-end malicious code attacks based on security aspects. The method mainly includes an aspect program running in an access terminal (browser or APP) and a corresponding detection engine. The aspect program corresponds to each target function related to sensitive operations in the access terminal. The detection engine identifies attack behaviors according to preset detection rules at the above tangent points. When an attack behavior is found, the aspect program executes a preset corresponding operation. If no attack behavior is found, the aspect program continues to execute the behavior after the tangent point. Compared with traditional solutions, it does not rely on rules, has higher security, does not need to modify the infrastructure of the large model service, has better convenience, does not need to disable the code function of the access terminal, has feasibility, and can resist various attacks against detection keywords by directly intercepting the API commonly used for cross-site scripting attacks. It has higher recall rate and robustness. The method includes: In response to the target application receiving the response content returned by the target large model based on the question request, parsing the response content, wherein the response content includes code information; During the parsing process, based on the cut points embedded in one or more target functions related to the sensitive operation, determining whether the response content contains attack behavior according to preset detection rules; If it is determined that the response content contains attack behavior, a preset response operation is performed on the response content; otherwise, the target function continues to be executed, and presentation information corresponding to the response content is rendered and displayed in the target application.

[0006] Furthermore, determining whether the response content contains attack behavior according to a preset detection rule includes: By detecting the function call status of the one or more target functions, it is determined whether the response content contains attack behavior.

[0007] Further, the step of detecting the function call conditions of the one or more target functions to determine whether the response content contains attack behavior includes: If it is detected that at least one of the one or more target functions is called, it is determined whether the response content contains any attack behavior.

[0008] Furthermore, if it is detected that at least one of the one or more target functions is called, determining whether the response content contains attack behavior includes: If it is detected that at least one of the one or more target functions is called, and the degree of association between the at least one target function and the question request is less than or equal to a first preset threshold, it is determined that the response content contains an attack behavior.

[0009] Furthermore, if it is detected that at least one of the one or more target functions is called, determining whether the response content contains attack behavior includes: If it is detected that the first target function among the one or more target functions is called, the execution of the first target function is suspended. If it is continued to be detected that at least one second target function among the one or more target functions that has a preset association relationship with the first target function is called, it is determined that there is an attack behavior in the response content.

[0010] Furthermore, the method further comprises: Detect whether there is code information in the response content; if so, enable the attack behavior detection function of the response content of the target large model at the tangent point.

[0011] Furthermore, the method further comprises: In response to executing a preset response operation for the response content, or in response to rendering and displaying presentation information corresponding to the response content in the target application, the attack behavior detection function for the response content of the target large model is turned off at the tangent point.

[0012] Furthermore, the preset response operation includes at least one of the following: Blocking the parsing process; issuing a warning regarding the content of the response; The response content is written into the log.

[0013] Further, the preset response operation includes issuing a warning for the response content and / or writing the response content into a log; Wherein, the method further comprises: The target function continues to be executed, and the presentation information corresponding to the response content is rendered and displayed in the target application.

[0014] Further, the preset response operation includes blocking the parsing process; Wherein, the method further comprises: The target function is terminated and continued to execute.

[0015] The embodiment of this specification also provides a device for detecting attack behavior, including: A parsing module, configured to parse the response content returned by the target large model based on the question request in response to the target application receiving the response content, wherein the response content includes code information; An attack behavior detection module, used to determine whether the response content contains attack behavior according to preset detection rules based on the cut points embedded in one or more target functions related to sensitive operations during the parsing process; The response module is used to execute a preset response operation on the response content if it is determined that the response content contains attack behavior, otherwise, the target function continues to be executed and the presentation information corresponding to the response content is rendered and displayed in the target application.

[0016] The embodiments of the present specification also provide a computer program product, wherein the computer program product stores at least one instruction, and the at least one instruction is suitable for being loaded by a processor and executing the above method steps.

[0017] The embodiments of the present specification also provide a storage medium, wherein the storage medium stores a computer program, and the computer program is suitable for being loaded by a processor and executing the steps of the above method.

[0018] An embodiment of the present specification also provides an electronic device, comprising: a processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the steps of the above method.

[0019] In the embodiments of the present specification, a technology for protecting a large model from front-end malicious code attacks based on security aspects is proposed, which mainly includes an aspect program running in an access terminal (browser or APP) and a corresponding detection engine. The cut points corresponding to the aspect program are various target functions related to sensitive operations in the access terminal. The detection engine identifies attack behaviors at the above cut points according to preset detection rules. When an attack behavior is found, the aspect program executes the preset corresponding operation. If no attack behavior is found, the aspect program continues to execute the behavior after the cut point. Compared with traditional solutions, it does not rely on rules, has higher security, and does not require modification of the infrastructure of the large model service. It has better convenience, does not require disabling the code function of the access terminal, and is feasible. By directly intercepting the API commonly used for cross-site scripting attacks, it can resist various attacks that bypass detection keywords, and has higher recall and robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A flowchart of a method for detecting attack behavior provided in an embodiment of this specification.

[0021] Figure 2 The present invention is a flowchart of a method for detecting attack behavior according to an example provided in an embodiment of the present specification.

[0022] Figure 3 A schematic diagram of the structure of a device for detecting attack behavior provided in an embodiment of this specification.

[0023] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of this specification. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solutions and advantages of this specification more clear, the technical solutions of this specification will be clearly and completely described below in combination with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of them. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this specification.

[0025] See also Figure 1 , is a flow chart of a method for detecting attack behavior provided in an embodiment of this specification. In an embodiment of this specification, the method for detecting attack behavior is applied to a device for detecting attack behavior (hereinafter referred to as "attack behavior detection device") or an electronic device equipped with an attack behavior detection device. Figure 1 The process shown is described in detail, and the method for detecting attack behavior may specifically include the following steps: S102, in response to the target application receiving the response content returned by the target macro model based on the question request, parsing the response content, wherein the response content includes code information.

[0026] In some embodiments, the target application includes but is not limited to an application (App, Application, an application is defined as a software package that can run independently), a mini-program (a mini-program cannot run independently and needs to run in a host program), a browser (a software application for accessing the Internet that allows users to obtain and display information on a web page by entering a URL (web page address) or clicking a link), etc. It should be noted that the above-mentioned applications are only examples and not limitations. Those skilled in the art should understand that any type of application can be included in the protection scope of this specification, and this specification does not limit this.

[0027] In some embodiments, a large model (also known as a large language model, Large Language Model, or LLM for short) is a deep learning model trained based on massive text data, which can generate natural language text or understand the meaning of language text. Such models can perform a variety of natural language processing tasks, including but not limited to text classification, question-answering, and dialogue. As the model size grows, LLM can generate more accurate and coherent outputs while processing more complex and longer input sequences. In addition, larger models can also cover a wider range of knowledge and language contexts, thereby providing more comprehensive and targeted answers and solutions. Here, this specification does not limit the model type, model structure, and model parameters of the target large model.

[0028] In some embodiments, in response to a trigger operation (e.g., a click operation) performed by a user in a target application, the target application generates a corresponding question request and sends the question request to the target large model. The target large model generates corresponding content based on the question request and returns it to the target application. In some embodiments, the question request includes a prompt, that is, the request content of the question request is a prompt, which refers to input information or instructions provided to a computer program or model. In the large language model, the prompt is a question or statement provided by the user to the large model, which is used to guide the large model to generate relevant replies or responses. After receiving a prompt, the large model will generate the most relevant response content to the prompt based on its internally trained knowledge and algorithms. In some embodiments, the prompt word is generally composed of user input content and instructions. The user input content includes but is not limited to the content entered by the user in the target application, the content selected by the user in the target application, etc. The instruction is determined based on the trigger operation performed by the user. For example, based on the button clicked by the user, the instruction associated with the button is determined. For example, the prompt word is "Please translate the following content into English 'I am the squad leader'", where the user input content is "I am the squad leader" and the instruction is "Please translate the following content into English". It should be noted that the above prompt words are only examples and not limitations. Those skilled in the art should understand that prompt words of any content can be included in the protection scope of this specification, and this specification does not limit this.

[0029] In some embodiments, the response content returned by the target large model includes code information, and the type of the code information includes but is not limited to JavaScript (a directly interpreted scripting language), Markdown (a lightweight markup language), etc. It should be noted that the above-mentioned code information types are only examples and not limitations. Those skilled in the art should understand that any type of code information can be included in the protection scope of this specification, and this specification does not limit this.

[0030] In some embodiments, the response content returned by the target large model also includes presentation information that needs to be presented in the target application. After receiving the response content returned by the target large model, the target application will parse the response content and try to render and display the presentation information in the response content that needs to be presented in the target application in the target application.

[0031] S104, during the parsing process, based on the cut points embedded in one or more target functions related to the sensitive operation, determine whether the response content contains attack behavior according to preset detection rules.

[0032] In some embodiments, the aspect program running in the target application embeds a point of intersection corresponding to the aspect program in one or more target functions related to sensitive operations (i.e., target functions used to perform sensitive operations, sensitive operations include but are not limited to operations of initiating network connections, operations of reading cookies, operations of reading geographic locations, operations of executing codes, operations of accessing or modifying storage, etc., which are not limited in this specification). In the process of parsing the response content, if at least one of the one or more target functions initiates a function call request by the code information in the response content, the aspect program can intercept the function call request at the point of intersection (i.e., before at least one target function is executed), and detect whether there is an attack behavior in the response content through the corresponding detection engine, wherein the target function may refer to an API (Application Programming Interface) provided in the target application. Interface, application programming interface), or it may also refer to an API provided in the operating system of the attack behavior detection device, including but not limited to an API for initiating a network connection, an API for reading cookies, an API for reading geographic location, an API for executing code, an API for accessing or modifying storage, etc. It should be noted that the above-mentioned APIs are only examples and not limitations. Those skilled in the art should understand that any API related to sensitive operations (i.e., APIs for performing sensitive operations) can be included in the protection scope of this specification, and this specification does not impose any restrictions on this.

[0033] In some embodiments, the detection engine detects whether the response content contains attack behavior based on preset detection rules. The preset detection rules refer to rules for indicating how to determine whether the response content contains attack behavior based on the function call status of the one or more target functions and / or the function input parameters. For example, the preset detection rule is that if a preset number threshold of target functions (for example, at least one target function) among the one or more target functions is initiated with a function call request by the code information in the response content, then it is determined that the response content contains attack behavior. For another example, the preset detection rule is that if at least one target function among the one or more target functions that has no association with the question request is initiated with a function call request by the code information in the response content, or at least one target function among the one or more target functions that has no association with the presentation information in the response content that needs to be presented in the target application is initiated with a function call request by the code information in the response content, then it is determined that the response content contains attack behavior. For another example, the preset detection rule is that if multiple target functions among the one or more target functions that have a preset association If a function call request is initiated by the code information in the response content, then it is determined that the response content contains attack behavior. For example, the preset detection rule is that if multiple target functions among the one or more target functions are initiated with function call requests by the code information in the response content and the call order corresponding to the multiple target functions meets the preset order condition, then it is determined that the response content contains attack behavior. For example, the preset detection rule is that if at least one target function among the one or more target functions is initiated with function call requests by the code information in the response content and the parameter value of the input parameter of the at least one target function meets the preset requirement, then it is determined that the response content contains attack behavior. Wherein, whether the parameter value meets the preset requirement can be determined by performing semantic content recognition on the parameter value, or, regular matching can be performed on the parameter value based on one or more preset regular expressions, and whether the parameter value meets the preset requirement can be determined according to the matching result, or, the parameter value can be input into a trained model to obtain the indication information output by the model to indicate whether the parameter value meets the preset requirement. This specification does not limit this. It should be noted here that the above detection rules are only examples and not limitations. Those skilled in the art should understand that detection rules of any form and any content can be included in the protection scope of this specification, and this specification does not impose any restrictions on this.

[0034] S106, if it is determined that the response content contains attack behavior, perform a preset response operation on the response content, otherwise, allow the target function to continue to execute, and render and display presentation information corresponding to the response content in the target application.

[0035] In some embodiments, if the aspect program determines that the response content contains an attack behavior, a preset response operation will be performed for the response content, and the preset response operation includes but is not limited to blocking the parsing process corresponding to the response content, issuing a warning for the response content, writing the response content into a log, etc. At this time, the target function can be continued to execute based on its intercepted function call request, or the target function can be stopped from continuing to execute based on its intercepted function call request, or the user can choose whether to continue to execute the target function based on its intercepted function call request, and this specification does not limit this. It should be noted that the above-mentioned preset response operation is only for example and not for limitation. Those skilled in the art should understand that any type of preset response operation can be included in the protection scope of this specification, and this specification does not limit this. In some embodiments, if the aspect program determines that the response content does not contain an attack behavior, the aspect program will continue to execute based on its intercepted function call request, that is, the aspect program will continue to execute the behavior after the cut point until the parsing process of the response content is completed, and the presentation information in the response content that needs to be presented in the target application is rendered and displayed in the target application.

[0036] In some embodiments, the aspect program determines a preset response operation suitable for the answer content from among multiple preset response operations based on a preset response rule, and executes the preset response operation on the answer content. For example, based on the preset response rule, according to at least one of the question content of the question request, the presentation content of the presentation information in the answer content that needs to be presented in the target application, the question time, the relevant user information of the user who initiated the question request, and the relevant description information of the target macro model, a preset response operation matching the answer content under the response rule is determined from among multiple preset response operations, and the preset response operation is executed on the answer content.

[0037] In the embodiments of the present specification, a technology for protecting a large model from front-end malicious code attacks based on security aspects is proposed, which mainly includes an aspect program running in an access terminal (browser or APP) and a corresponding detection engine. The cut points corresponding to the aspect program are various target functions related to sensitive operations in the access terminal. The detection engine identifies attack behaviors at the above cut points according to preset detection rules. When an attack behavior is found, the aspect program executes the preset corresponding operation. If no attack behavior is found, the aspect program continues to execute the behavior after the cut point. Compared with traditional solutions, it does not rely on rules, has higher security, and does not require modification of the infrastructure of the large model service. It has better convenience, does not require disabling the code functions of the access terminal (such as JavaScript functions, Markdown functions, etc.), has feasibility, and can resist various attacks against detection keywords by directly intercepting APIs commonly used in cross-site scripting attacks, and has higher recall and robustness.

[0038] In some embodiments, the determining whether the response content contains attack behavior according to preset detection rules includes: determining whether the response content contains attack behavior by detecting the function call status of the one or more target functions. In some embodiments, the preset detection rules refer to rules for indicating how to determine whether the response content contains attack behavior according to the function call status of the one or more target functions. In the process of parsing the response content, it is possible to determine whether the response content contains attack behavior based on at least one target function among the one or more target functions that initiates a function call request based on the code information in the response content, and based on the function call status of the at least one target function called by the code information in the response content.

[0039] In some embodiments, the step of determining whether the response content contains an attack behavior by detecting the function call of the one or more target functions includes: if at least one of the one or more target functions is detected to be called, determining whether the response content contains an attack behavior. In some embodiments, if at least one of the one or more target functions is detected to initiate a function call request by the code information in the response content, it can be determined that the response content contains an attack behavior. In some embodiments, if it is detected that the target functions of the one or more target functions that reach a preset number threshold are initiated by the code information in the response content, it can be determined that the response content contains an attack behavior, wherein the preset number threshold can be preset and fixed, or can also be dynamically determined according to the question request, for example, by performing semantic recognition on the question request to determine the question difficulty or question complexity of the question request, and determining the preset number threshold according to the question difficulty or question complexity, the higher the question difficulty or the higher the question complexity, the larger the corresponding preset number threshold, and for example, according to the question type corresponding to the question request, according to the mapping relationship between the preset question type and the number threshold, the number threshold mapped by the question type is used as the preset number threshold.

[0040] In some embodiments, if at least one of the one or more target functions is detected to be called, determining whether the answer content contains an attack behavior includes: if at least one of the one or more target functions is detected to be called, and the degree of association between the at least one target function and the question request is less than or equal to a first preset threshold, determining that the answer content contains an attack behavior. In some embodiments, if at least one of the one or more target functions is detected to initiate a function call request by the code information in the answer content, and the degree of association between the at least one target function and the question request is less than or equal to the first preset threshold, that is, there is no association relationship between the at least one target function and the question request, then it can be determined that the answer content contains an attack behavior. In some embodiments, it is only necessary to satisfy that there is a target function whose degree of association with the question request is less than or equal to the first preset threshold among at least one of the target functions whose function call request is initiated by the code information in the answer content, or it is necessary to satisfy that the average degree of association between at least one of the target functions whose function call request is initiated by the code information in the answer content and the question request is less than or equal to the first preset threshold before it can be determined that the answer content contains an attack behavior.

[0041] In some embodiments, if it is detected that at least one of the one or more target functions is called, determining whether the response content contains attack behavior includes: if it is detected that a first target function of the one or more target functions is called, causing the first target function to suspend execution; if it is continued to be detected that at least one second target function of the one or more target functions that has a preset association relationship with the first target function is called, determining that the response content contains attack behavior. In some embodiments, the preset detection rule includes one or more target function association sets, each association set includes multiple target functions with preset association relationships. For each association set, during the parsing process of the response content, only when all the target functions in the association set are called by the code information in the response content, can it be determined whether the response content involves an attack. If only some of the target functions in the association set are called by the code information in the response content, it is not possible to determine whether the response content involves an attack for the time being, and it is necessary to continue waiting for the remaining target functions in the association set to be called by the code information in the response content. If all the target functions in the association set are called by function, it can be determined that the response content involves an attack. If the timeout has expired or the parsing has not ended until all the target functions in the association set have been called by function, it can be determined that the response content does not involve an attack. In some embodiments, a target function association set in the preset detection rule includes a first target function and at least one second target function. There is a preset association relationship between the first target function and the at least one second target function. If it is detected that the first target function is called by the code information in the response content, it is temporarily impossible to determine whether the response content contains attack behavior. The first target function is first suspended from execution. If it is detected that at least one second target function remaining in the target function association set except the first target function is called by the code information in the response content, it can be determined that the response content contains attack behavior.

[0042] In some embodiments, the method further includes: detecting whether there is code information in the response content; if there is, enabling the attack behavior detection function of the response content of the target large model at the cut-off point. In some embodiments, the aspect program running in the target application can enable or disable the attack behavior detection function of the response content at each cut-off point corresponding to the aspect program. In the process of parsing the response content, if at least one of the one or more target functions initiates a function call request by the code information in the response content, if the attack behavior detection function of the response content has been enabled at each cut-off point, the aspect program will intercept the function call request at the cut-off point, and after interception, detect whether there is an attack behavior in the response content through the corresponding detection engine. If the attack behavior detection function of the response content has been disabled at each cut-off point, the aspect program will not intercept the function call request at the cut-off point, and will not detect whether there is an attack behavior in the response content through the corresponding detection engine after interception. In some embodiments, before parsing the response content, the aspect program will first detect whether there is code information in the response content. Only when there is code information in the response content, will the attack behavior detection function regarding the response content be enabled at each cutting point corresponding to the aspect program, that is, the aspect program will intercept the function call initiated by the code information in the response content to the target function at the cutting point embedded in the one or more target functions, and after the interception, determine whether there is an attack behavior in the response content according to the preset detection rules.

[0043] In some embodiments, the method further includes: in response to executing a preset response operation for the response content, or in response to rendering and displaying the presentation information corresponding to the response content in the target application, closing the attack behavior detection function for the response content of the target large model at the cut-off point. In some embodiments, if it is determined that the response content contains an attack behavior, then after executing a preset response operation for the response content, that is, in response to executing a preset response operation for the response content, the aspect program will close the attack behavior detection function for the response content at each of its corresponding cut-off points, that is, the aspect program will not continue to intercept the function call request at the cut-off point, nor will it continue to detect whether the response content contains an attack behavior through the corresponding detection engine after interception. In some embodiments, if it is determined that the response content does not contain any attack behavior, then after rendering and displaying the presentation information in the response content that needs to be presented in the target application in the target application, that is, in response to rendering and displaying the presentation information corresponding to the response content in the target application, the aspect program will turn off the attack behavior detection function regarding the response content at its corresponding respective aspect points, that is, the aspect program will not continue to intercept the function call request at the aspect point, nor will it continue to detect whether the response content contains any attack behavior through the corresponding detection engine after the interception.

[0044] In some embodiments, the preset response operation includes at least one of the following: blocking the parsing process; issuing a warning for the response content; writing the response content to a log. In some embodiments, the preset response operation may include blocking the parsing process for the response content, stopping the parsing of the response content, so that the target function stops executing based on its intercepted function call request, and the presentation information corresponding to the response content will not be rendered and displayed in the target application. In some embodiments, the preset response operation may also include issuing a warning for the response content, and the warning information corresponding to the response content may be presented to the user in the target application. At this time, the user may choose whether to allow the target function to continue to execute based on its intercepted function call request. In some embodiments, the preset response operation may also include writing the response content to a log, or, further includes writing the question request to a log. At this time, the target function will still continue to execute based on its intercepted function call request.

[0045] In some embodiments, the preset response operation includes issuing a warning for the response content and / or writing the response content to a log; wherein the method further includes: causing the target function to continue to execute, and rendering and displaying presentation information corresponding to the response content in the target application. In some embodiments, if the preset response operation includes at least one of issuing a warning for the response content and writing the response content to a log, then the target function will also continue to execute based on its intercepted function call request, that is, the aspect program will continue to execute the behavior after the cut point.

[0046] In some embodiments, the preset response operation includes blocking the parsing process; wherein the method further includes: causing the target function to stop executing. In some embodiments, if the response operation includes blocking the parsing process of the response content, the target function will also stop executing based on its intercepted function call request, that is, the aspect program will not continue to execute the behavior after the cut point, and will not render and display the presentation information corresponding to the response content in the target application.

[0047] Figure 2 The present invention is a flowchart of a method for detecting attack behavior according to an example provided in an embodiment of the present specification.

[0048] like Figure 2As shown, the user initiates a question request to the big model, and the big model returns the answer content corresponding to the question request. The browser or APP on the user side parses the answer content and tries to render and display the presentation information in the answer content. The aspect program intercepts the JavaScript and Markdown executed in the rendering process based on the cut-off points embedded in one or more target functions related to sensitive operations, and calls the detection engine. When the detection engine finds an attack behavior, it makes necessary responses according to the pre-set rules. If no attack is found in the above methods, the aspect program continues to execute the behavior after the cut-off point and continues to render and display the presentation information in the answer content.

[0049] Figure 3 A schematic diagram of a structure of a device for detecting attack behaviors based on a cut point provided in an embodiment of this specification, the device for detecting attack behaviors (hereinafter referred to as "attack behavior detection device 1") can be implemented as all or part of an electronic device through software, hardware, or a combination of both. According to some embodiments, the attack behavior detection device 1 includes a parsing module 11, an attack behavior detection module 12, and a response module 13.

[0050] The parsing module 11 is used for parsing the response content returned by the target large model based on the question request in response to the target application receiving the response content, wherein the response content includes code information; An attack behavior detection module 12, used to determine whether the response content contains an attack behavior according to preset detection rules based on the cut points embedded in one or more target functions related to the sensitive operation during the parsing process; The response module 13 is used to execute a preset response operation on the response content if it is determined that the response content contains an attack behavior, otherwise, the target function continues to be executed and the presentation information corresponding to the response content is rendered and displayed in the target application.

[0051] In some embodiments, determining whether the response content contains attack behavior according to a preset detection rule includes: By detecting the function call status of the one or more target functions, it is determined whether the response content contains attack behavior.

[0052] In some embodiments, the step of detecting the function call conditions of the one or more target functions to determine whether the response content contains attack behavior includes: If it is detected that at least one of the one or more target functions is called, it is determined whether the response content contains any attack behavior.

[0053] In some embodiments, if it is detected that at least one of the one or more target functions is called, determining whether the response content contains attack behavior includes: If it is detected that at least one of the one or more target functions is called, and the degree of association between the at least one target function and the question request is less than or equal to a first preset threshold, it is determined that the response content contains an attack behavior.

[0054] In some embodiments, if it is detected that at least one of the one or more target functions is called, determining whether the response content contains attack behavior includes: If it is detected that the first target function among the one or more target functions is called, the execution of the first target function is suspended. If it is continued to be detected that at least one second target function among the one or more target functions that has a preset association relationship with the first target function is called, it is determined that there is an attack behavior in the response content.

[0055] In some embodiments, the attack behavior detection device 1 is further used for: Detect whether there is code information in the response content; if so, enable the attack behavior detection function of the response content of the target large model at the tangent point.

[0056] In some embodiments, the attack behavior detection device 1 is further used for: In response to executing a preset response operation for the response content, or in response to rendering and displaying presentation information corresponding to the response content in the target application, the attack behavior detection function for the response content of the target large model is turned off at the tangent point.

[0057] In some embodiments, the preset response operation includes at least one of the following: Blocking the parsing process; issuing a warning regarding the content of the response; The response content is written into the log.

[0058] In some embodiments, the preset response operation includes issuing a warning for the response content and / or writing the response content into a log; The attack behavior detection device 1 is also used for: The target function continues to be executed, and the presentation information corresponding to the response content is rendered and displayed in the target application.

[0059] In some embodiments, the predetermined response operation includes blocking the parsing process; The attack behavior detection device 1 is also used for: The target function is terminated and continued to execute.

[0060] The above device embodiments correspond to the method embodiments. For specific descriptions, please refer to the description of the method embodiments, which will not be repeated here. The device embodiments are obtained based on the corresponding method embodiments and have the same technical effects as the corresponding method embodiments. For specific descriptions, please refer to the corresponding method embodiments.

[0061] The embodiment of the present specification also provides a computer storage medium, which can store multiple instructions, and the instructions are suitable for being loaded by a processor to execute the method of the embodiment of the present specification.

[0062] The embodiments of the present specification also provide a computer program product, which stores at least one instruction, and the at least one instruction is loaded by the processor to execute the method of the embodiments of the present specification.

[0063] The embodiments of this specification also provide Figure 4 The structural diagram of the electronic device shown in FIG. Figure 4 At the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include hardware required for other services. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to implement the above-mentioned voice activity detection method.

[0064] Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the executor of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0065] In the 1990s, it was very clear whether the improvement of a technology was hardware improvement (for example, improvement of the circuit structure of diodes, transistors, switches, etc.) or software improvement (improvement of the method flow). However, with the development of technology, many improvements of the method flow today can be regarded as direct improvements of the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that the improvement of a method flow cannot be implemented with hardware entity modules. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask chip manufacturers to design and make dedicated integrated circuit chips. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one kind of HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0066] The controller may be implemented in any suitable manner, for example, the controller may take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (e.g., software or firmware) executable by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller, examples of which include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320, and the memory controller may also be implemented as part of the control logic of the memory. It is also known to those skilled in the art that, in addition to implementing the controller in a purely computer-readable program code manner, the controller may be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, such a controller may be considered as a hardware component, and the devices for implementing various functions included therein may also be considered as structures within the hardware component. Or even, the devices for implementing various functions may be considered as both software modules for implementing the method and structures within the hardware component.

[0067] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0068] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0069] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0070] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0071] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0072] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0073] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0074] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0075] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0076] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0077] It should be understood by those skilled in the art that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0078] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0079] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0080] The above description is only an embodiment of the present specification and is not intended to limit the present specification. For those skilled in the art, the present specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification shall be included in the scope of the claims of the present specification.

Claims

1. A method for detecting attack behavior, comprising: In response to the target application receiving the response content returned by the target large model based on the question request, parsing the response content, wherein the response content includes code information; During the parsing process, based on the cut points embedded in one or more target functions related to the sensitive operation, determining whether the response content contains attack behavior according to preset detection rules; If it is determined that the response content contains attack behavior, a preset response operation is performed on the response content; otherwise, the target function continues to be executed, and presentation information corresponding to the response content is rendered and displayed in the target application.

2. According to the method of claim 1, determining whether the response content contains attack behavior according to a preset detection rule comprises: By detecting the function call status of the one or more target functions, it is determined whether the response content contains attack behavior.

3. The method according to claim 2, wherein the step of detecting the function call conditions of the one or more target functions to determine whether the response content contains attack behavior comprises: If it is detected that at least one of the one or more target functions is called, it is determined whether the response content contains any attack behavior.

4. The method according to claim 3, wherein if it is detected that at least one of the one or more target functions is called, determining whether the response content contains attack behavior comprises: If it is detected that at least one of the one or more target functions is called, and the degree of association between the at least one target function and the question request is less than or equal to a first preset threshold, it is determined that the response content contains an attack behavior.

5. The method according to claim 3, wherein if it is detected that at least one of the one or more target functions is called, determining whether the response content contains attack behavior comprises: If it is detected that the first target function among the one or more target functions is called, the execution of the first target function is suspended. If it is continued to be detected that at least one second target function among the one or more target functions that has a preset association relationship with the first target function is called, it is determined that there is an attack behavior in the response content.

6. The method according to claim 1, further comprising: Detecting whether there is code information in the response content; If it exists, the attack behavior detection function of the response content of the target large model is enabled at the cut point.

7. The method according to claim 6, further comprising: In response to executing a preset response operation for the response content, or in response to rendering and displaying presentation information corresponding to the response content in the target application, the attack behavior detection function for the response content of the target large model is turned off at the tangent point.

8. The method according to claim 1, wherein the preset response operation comprises at least one of the following: Blocking the parsing process; issuing a warning regarding the content of the response; The response content is written into the log.

9. The method according to claim 8, wherein the preset response operation comprises issuing a warning for the response content and / or writing the response content into a log; in, The method further comprises: The target function continues to be executed, and the presentation information corresponding to the response content is rendered and displayed in the target application.

10. The method according to claim 8, wherein the preset response operation comprises blocking the parsing process; in, The method further comprises: The target function is terminated and continued to execute.

11. A device for detecting attack behavior, comprising: A parsing module, configured to parse the response content returned by the target large model based on the question request in response to the target application receiving the response content, wherein the response content includes code information; An attack behavior detection module, used to determine whether the response content contains attack behavior according to preset detection rules based on the cut points embedded in one or more target functions related to sensitive operations during the parsing process; The response module is used to execute a preset response operation on the response content if it is determined that the response content contains an attack behavior, otherwise, the target function continues to be executed and the presentation information corresponding to the response content is rendered and displayed in the target application.

12. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

13. An electronic device, characterized in that: include: A processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the steps of the method as claimed in any one of claims 1 to 10.

14. A computer program product having at least one instruction stored thereon, characterized in that: When the at least one instruction is executed by the processor, the steps of the method described in any one of claims 1 to 10 are implemented.

Images