Tool determination method and device, equipment, medium and product

Abstract

The embodiment of the invention provides a tool determination method and device, equipment, a medium and a product, and relates to the field of data processing. According to the method, the input risk intention is automatically judged by carrying out multi-dimensional compliance detection on the statement input by the user and combining a probability calculation model, and the input safety does not need to be verified by manually contrasting static rules; meanwhile, the credibility and the risk value of the tool are automatically calculated according to the state data of the target tool, and the reliable and low-risk first tool is screened out, so that the complexity of manually sorting tool rules and intervening in complex scene correction is avoided; in the subsequent process, a final adaptive tool can be determined by automatically calculating the deviation degree between user input and the first tool behavior sequence, automatic processing of all links in the whole tool determination process is achieved, and process delay caused by manual intervention is reduced; the problem that in the prior art, tool determination depends on static rules, the process is tedious, and the efficiency is low due to manual intervention is solved, and the tool determination efficiency is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of data processing, and in particular to a tool determination method and device, equipment, medium and product. BACKGROUND

[0002] Tool determination refers to the process of determining a tool that meets the requirements from a plurality of preset tools when an intelligent agent performs a complex task. Current intelligent agents based on large models are rapidly developing, and the intelligent agents can achieve the goal entrusted by the user by calling various external tools. This calling mechanism enhances the flexibility of the system. How to scientifically and reasonably determine the tool is related to the reliability of the task execution of the intelligent agent, and becomes the key to guarantee the value realization of the intelligent agent system.

[0003] In the prior art, the functions, boundaries and calling permissions of various tools are usually first sorted out, the task types and safety requirements of the business scenarios are combined, the tool calling specification is formulated, the tools suitable for the scene and the minimum calling permissions of the tools are determined, the core requirements of the user task are then analyzed and the scene is judged, the tools that are functionally suitable and permission compliant are screened by comparing the specification, and the tools are determined as callable objects to ensure that the calling is adapted to the task requirements and safety rules.

[0004] However, the prior art has the problem of low tool determination efficiency. In the prior art, tool determination relies on static preset strategies and permission rules, and when determining the tool, the user requirements need to be parsed according to a fixed process, and the rules need to be checked and screened one by one. Artificial intervention is often required to correct complex or edge scenarios. The overall process lacks flexible adaptation capability, resulting in a cumbersome tool determination process and low efficiency. SUMMARY

[0005] The embodiments of the present application provide a tool determination method, device, equipment, medium and product to solve the problem of low tool determination efficiency in the prior art.

[0006] In a first aspect, the embodiments of the present application provide a tool determination method, comprising:

[0007] obtaining a user input sentence and a plurality of state data of a target tool; wherein the user input sentence is used to represent user requirements, and the target tool is any one of a plurality of preset tools;

[0008] performing multi-dimensional compliance detection on the user input sentence, and when the multi-dimensional compliance detection result is compliance detection pass, inputting the user input sentence into a preset probability calculation model to obtain a suspicious probability of the user input sentence; wherein the multi-dimensional compliance detection result includes the compliance detection pass, and the suspicious probability is used to represent the possibility of risk intention of the user input sentence;

[0009] calculate a reputation degree and a risk value of the target tool according to the plurality of state data of the target tool, wherein the reputation degree is used to represent reliability of the target tool, and the risk value is used to represent a possibility of the target tool existing in a risk in a preset invocation scenario;

[0010] screen the plurality of tools to obtain a plurality of first tools, wherein the plurality of first tools are tools in the plurality of tools, whose reputation degrees are greater than or equal to a preset reputation degree threshold value and whose risk values are less than a preset risk threshold value;

[0011] in response to the suspicious probability of the user input statement being less than a preset suspicious threshold value, obtain a behavior sequence of each of the first tools and calculate a deviation degree between the user input statement and the behavior sequence of each of the first tools, wherein the behavior sequence of each of the first tools is used to represent an effect generated by each of the first tools after the invocation scenario is invoked;

[0012] determine the first tool corresponding to the minimum value of the deviation degrees as a final tool, wherein the final tool is a tool meeting the user demand.

[0013] In a second aspect, an embodiment of the present application provides a tool determination apparatus, comprising:

[0014] a first obtaining module, configured to obtain a user input statement and a plurality of state data of a target tool, wherein the user input statement is used to represent a user demand, and the target tool is any one of a plurality of preset tools;

[0015] a compliance detection module, configured to perform multi-dimensional compliance detection on the user input statement, and when a multi-dimensional compliance detection result is a compliance detection pass, input the user input statement into a preset probability calculation model to obtain a suspicious probability of the user input statement, wherein the multi-dimensional compliance detection result comprises the compliance detection pass, and the suspicious probability is used to represent a possibility of the user input statement existing in a risk intention;

[0016] a first calculating module, configured to calculate a reputation degree and a risk value of the target tool according to the plurality of state data of the target tool, wherein the reputation degree is used to represent reliability of the target tool, and the risk value is used to represent a possibility of the target tool existing in a risk in a preset invocation scenario;

[0017] a screening module, configured to screen the plurality of tools to obtain a plurality of first tools, wherein the plurality of first tools are tools in the plurality of tools, whose reputation degrees are greater than or equal to a preset reputation degree threshold value and whose risk values are less than a preset risk threshold value;

[0018] The second acquisition module is used to acquire the behavior sequence of each of the first tools in response to the user input statement having a suspicious probability of less than a preset suspicious threshold, and to calculate the deviation between the user input statement and the behavior sequence of each of the first tools; wherein, the behavior sequence of each of the first tools is used to represent the effect produced by each of the first tools after being invoked in the invocation scenario;

[0019] The determination module is used to determine the first tool corresponding to the minimum deviation as the final tool; wherein the final tool refers to the tool that meets the user's needs.

[0020] Thirdly, this application provides an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0021] The memory stores computer-executed instructions;

[0022] When the processor executes computer execution instructions stored in the memory, it is used to implement the tool determination method as described in any of the first aspects.

[0023] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the tool determination method as described in any of the first aspects.

[0024] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, is used to implement the tool determination method as described in any of the first aspects.

[0025] This application provides a tool determination method, apparatus, device, medium, and product that automatically determines the risky intent of user input by performing multi-dimensional compliance checks and combining them with a probability calculation model, eliminating the need for manual verification of input security against static rules. Simultaneously, it automatically calculates the tool's reputation and risk value based on the target tool's status data, selecting a reliable and low-risk first tool, avoiding the tedious manual review of tool rules and intervention in complex scenarios. Furthermore, it automatically calculates the deviation between the user input and the first tool's behavior sequence to determine the final suitable tool. The entire tool determination process automates each step, reducing process delays caused by manual intervention. This effectively solves the inefficiency problems of existing technologies that rely on static rules, have cumbersome processes, and require manual intervention, thus improving the efficiency of tool determination while ensuring the security of tool invocation. Attached Figure Description

[0026] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0027] Figure 1 This is a schematic diagram illustrating an application scenario of the tool determination method provided in the embodiments of this application;

[0028] Figure 2 One of the flowcharts illustrating the tool determination method provided in the embodiments of this application;

[0029] Figure 3 One of the flowcharts illustrating the tool determination method provided in the embodiments of this application;

[0030] Figure 4 One of the flowcharts illustrating the tool determination method provided in the embodiments of this application;

[0031] Figure 5 This is a schematic diagram of the structure of the tool determining device provided in the embodiments of this application;

[0032] Figure 6 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application.

[0033] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0034] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0035] In the embodiments of this application, the terms "first" and "second" are used to distinguish identical or similar items with substantially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that "first" and "second" do not necessarily imply difference. It should be noted that in the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner. In the embodiments of this application, "at least one" refers to one or more, and "more than one" refers to two or more.

[0036] It should be noted that the phrase "at...time" in the embodiments of this application can refer to the instant at which a certain situation occurs, or to a period of time after the occurrence of a certain situation. The embodiments of this application do not specifically limit this. In addition, the tool determination method, apparatus, device, medium, and product provided in the embodiments of this application are only examples. A tool determination method, apparatus, device, medium, and product may also include more or fewer contents.

[0037] First, let me explain the terms used in this application:

[0038] Intelligent agents are entities with autonomous perception and decision-making capabilities. They can acquire environmental information through perception, combine their own knowledge and goals to reason and make decisions, and then act to influence the environment or tasks. They can also autonomously complete specific tasks or achieve goals through interaction and cooperation with the environment and other intelligent agents. They are widely used in fields such as artificial intelligence. They can interact with the external environment by calling external tools to perform various complex tasks, thereby realizing functional expansion and practical application needs.

[0039] Semantic variants refer to attacks that modify or transform the template in various ways to generate variants with different expressions but the same core meaning. Figure 1 The methods used to enrich the types of risk samples include paraphrasing content, creating spelling variations, encoding, converting to other languages, adjusting single-turn dialogues into multi-turn dialogues with step-by-step guidance, and splitting sensitive commands and splicing them together with context.

[0040] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0041] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of the present invention will now be described with reference to the accompanying drawings.

[0042] To clearly understand the technical solution of this application, we will first provide a detailed introduction to the solutions of existing technologies. Currently, intelligent agents based on large models are developing rapidly. These agents can achieve user-delegated goals by invoking various external tools, and this invocation mechanism enhances the system's flexibility. How to scientifically and rationally determine these tools becomes crucial to ensuring the realization of the value of the intelligent agent system.

[0043] In existing technologies, it is necessary to analyze the core requirements of user tasks and determine the scenario, then filter tools that are functionally compatible and have compliant permissions according to specifications, and determine the callable objects to ensure that the calls are compatible with task requirements and security rules. However, the tool determination in existing technologies largely relies on statically preset strategies and permission rules. The tool determination process requires analyzing user requirements and verifying and filtering them one by one according to the rules, and the overall process lacks flexibility and adaptability, making the tool determination process cumbersome. Therefore, existing technologies suffer from low efficiency in tool determination.

[0044] Therefore, addressing the issue of low efficiency in tool selection in existing technologies, this research found that to solve this problem, automation can be implemented across the entire process: automating the assessment of user input risks, the evaluation and screening of tool reliability and risk, and the matching of needs with tool effectiveness. This automation replaces static rule dependence and manual intervention, improving tool selection efficiency while ensuring safety. ① Intelligent models can be introduced to analyze the core intent and risk level of user needs, and automatically generate adaptation rules based on real-time tool status data, eliminating the need for manual verification or correction of complex scenarios. Dynamic rule generation combined with intelligent decision-making overcomes the limitations of static rules and reduces manual intervention. ② User needs and tools can be decomposed into quantifiable feature dimensions, establishing feature mapping and matching algorithms. Algorithms automatically calculate the similarity or compatibility between need features and tool features, locating suitable tools and achieving rapid and accurate matching between tools and needs. ③ A pre-defined tool pool can be dynamically hierarchically managed, dividing tools into subsets with different priorities or specific scenarios based on dimensions such as tool reliability, risk level, and scenario adaptation frequency. Meanwhile, pre-adaptation verification of tool subsets and common scenarios is completed in advance. When user requests are received, the corresponding tool subset is quickly matched first, and then fine-tuning is performed within the subset, without the need to verify all tools one by one.

[0045] Specifically, an automated decision-making framework based on dynamic risk assessment and intelligent intent matching can be constructed. Abandoning the traditional approach of relying on manual intervention and static rule bases, it uses a data-driven method to calculate the risk probability of user requests and the credibility of tools. Based on this, it precisely matches user intent with the historical behavioral effects of tools at the semantic level, automatically outputting the optimal tool selection. This integrates the originally discrete process requiring multiple manual judgments into a continuous, adaptive intelligent decision-making system, thereby improving the efficiency of tool selection while ensuring safety and compliance.

[0046] This application discloses a tool determination method, apparatus, device, medium, and product. By performing multi-dimensional compliance checks on user input statements and combining them with a probability calculation model, it automatically determines the input risk intent, eliminating the need for manual verification of input security against static rules. Simultaneously, it automatically calculates the tool's reputation and risk value based on the target tool's status data, selecting a reliable and low-risk first tool. This avoids the tedious process of manually reviewing tool rules and intervening in complex scenarios for correction. Furthermore, it automatically calculates the deviation between the user input and the first tool's behavior sequence to determine the final suitable tool. The entire tool determination process is automated, reducing process delays caused by manual intervention. This solves the inefficiency problems of existing technologies that rely on static rules, have cumbersome processes, and require manual intervention, thus improving the efficiency of tool determination while ensuring the security of tool invocation.

[0047] Based on the above-mentioned inventive discovery, the technical solution of this application is proposed.

[0048] The application scenarios of the tool determination method provided in the embodiments of the present invention are described below. Figure 1 This is a schematic diagram illustrating an application scenario for the tool determination method provided in the embodiments of this application. For example... Figure 2 As shown, this application scenario includes a mobile terminal 101 and a server 102. The mobile terminal 101 sends user input statements to the server 102. The server 102 obtains multiple status data of the target tool. The server 102 performs multi-dimensional compliance checks on the user input statements. When the multi-dimensional compliance check result is a pass, the server 102 inputs the user input statements into a preset probability calculation model to obtain the suspicious probability of the user input statements. Based on the multiple status data of the target tool, the server 102 calculates the credibility and risk value of the target tool. The server 102 filters multiple tools to obtain multiple first tools. In response to the suspicious probability of the user input statements being less than a preset suspicious threshold, the server 102 obtains the behavior sequences of each first tool and calculates the deviation between the user input statements and the behavior sequences of each first tool. The server 102 determines the first tool corresponding to the minimum deviation as the final tool.

[0049] The embodiments of the present invention will now be described with reference to the accompanying drawings.

[0050] Figure 2 This is one of the flowcharts illustrating the tool determination method provided in the embodiments of this application. Figure 3 As shown, in this embodiment, the execution entity of this invention is a server. Therefore, the tool determination method provided in this embodiment includes the following steps:

[0051] S201. Obtain user input statements and multiple status data of the target tool; wherein, the user input statements are used to express user needs, and the target tool is any one of a number of preset tools.

[0052] Specifically, user input can be obtained by receiving text directly entered by the user through a pre-defined interactive entry point, or by receiving text information converted from speech. For multiple status data of the target tool, this can be obtained by collecting the tool's past call records, function execution result feedback, and real-time monitoring of the tool's current operating status. The user input obtained in this step forms the basis for subsequent multi-dimensional compliance testing and determining whether the request carries risky intent. The target tool's status data, on the other hand, is the core basis for calculating the tool's credibility and risk value, and for initial tool screening, providing initial data support for the entire tool selection process.

[0053] S202. Perform multi-dimensional compliance checks on the user input statement, and when the multi-dimensional compliance check result is that the compliance check is passed, input the user input statement into a preset probability calculation model to obtain the suspicious probability of the user input statement; wherein, the multi-dimensional compliance check result includes compliance check passing, and the suspicious probability is used to indicate the possibility that the user input statement has a risky intention.

[0054] Specifically, the system checks user input from multiple dimensions, including whether the content contains illegal information, whether the format conforms to the input specifications for tool calls, and whether the request matches the preset tool call scenario. If no non-compliance issues are found after the check, the compliance check is considered passed. The user input that passes the check is then fed into a pre-trained probability calculation model. This model, combining historical risk input data and the underlying intent characteristics of the statement, automatically outputs a probability of suspicion reflecting the possibility of a risky intent in the input statement. This step not only initially eliminates obviously non-compliant input through multi-dimensional checks, avoiding subsequent ineffective tool matching processes, but also further accurately identifies potential risky intents through the probability calculation model, providing a security-level judgment basis for subsequent tool selection and preventing security issues caused by risky requests to the tool.

[0055] S203. Calculate the reputation score and risk value of the target tool based on multiple status data of the target tool; wherein, the reputation score is used to represent the reliability of the target tool, and the risk value is used to represent the possibility that the target tool may have risks under the preset calling scenario.

[0056] Specifically, the process begins by analyzing reliability-related information from the target tool's status data, such as the frequency of successful historical calls, records of failures, and user evaluations of the tool's effectiveness. This information is then weighted according to its importance, and a comprehensive evaluation yields a credibility score that reflects the tool's reliability. Simultaneously, risk-related information is extracted from the status data, such as the number of security issues the tool has encountered in past call scenarios and instances of functional abnormalities leading to risk events. Combined with pre-defined risk assessment criteria, a risk value is quantified to indicate the likelihood of risk in the current call scenario. This step transforms the tool's status data into quantifiable credibility and risk values, providing clear evaluation criteria for subsequent tool selection. It helps quickly identify reliable and low-risk tools, preventing unreliable or high-risk tools from entering the subsequent matching process, thus ensuring the security and effectiveness of subsequent calls from the tool level.

[0057] S204. Filter multiple tools to obtain multiple first tools; where multiple first tools refer to tools whose credit rating is greater than or equal to a preset credit rating threshold and whose risk value is less than a preset risk threshold.

[0058] Specifically, a pre-defined credibility threshold and risk acceptable threshold can be established first. Then, the credibility of each tool is compared with the credibility threshold, and the risk value of each tool is compared with the risk acceptable threshold. Only tools whose credibility reaches or exceeds the credibility threshold and whose risk value is lower than the risk acceptable threshold are retained. These retained tools constitute the multiple first tools. This step can eliminate tools with insufficient reliability or high risk from all preset tools in advance, narrowing down the range of tools for precise matching with user needs. It avoids performing invalid adaptation calculations on tools that do not meet safety and reliability requirements in later stages, reducing the workload of subsequent processes, improving overall efficiency, and ensuring that all tools entering the matching stage have basic reliability and safety attributes.

[0059] S205. In response to the user input statement having a suspicious probability less than a preset suspicious threshold, obtain the behavior sequence of each first tool and calculate the deviation between the user input statement and the behavior sequence of each first tool; wherein, the behavior sequence of each first tool is used to represent the effect produced by each first tool after it is invoked in the calling scenario.

[0060] Specifically, after confirming that the probability of suspicion in the user input statement meets the preset low-risk requirement, information such as the actual effects and completed tasks after being called can be extracted from the past call records of each primary tool to construct the behavior sequence of each primary tool. Then, the core needs expressed by the user input statement are compared with the actual capabilities of each tool as reflected in its behavior sequence. Following a preset comparison logic, the deviation between the user input statement and each primary tool's behavior sequence is calculated. This step, based on the initial risk assessment, ensures that tool adaptation is only performed for low-risk needs. Simultaneously, by quantifying the degree of matching between needs and tool effects, it accurately filters out the range of tools that better fit user needs, providing a core basis for determining the final tools. This avoids invalid adaptation calculations and ensures the accuracy of tool-need adaptation.

[0061] S206. The first tool corresponding to the minimum deviation is determined as the final tool; whereby the final tool refers to the tool that meets the user's needs.

[0062] Specifically, the deviation of each primary tool can be analyzed to identify the smallest deviation, and then the primary tool corresponding to the smallest deviation can be determined as the final tool. This step is based on the control of the user's input risk intent and the screening of tool reputation and risk value in the early stage. By locking in the tool with the smallest deviation from the user's needs, it ensures that the final tool not only maintains the basic attributes of reliability and low risk, but also fits the user's core needs to the greatest extent. This avoids invalid calls caused by inappropriate tool-to-demand adaptation, and finally completes the closed loop from initial tool screening to precise demand matching, ensuring that tool calls are both safe and effectively meet user needs.

[0063] This embodiment provides a tool determination method that automatically judges the risky intent of user input by performing multi-dimensional compliance checks and combining them with a probability calculation model, eliminating the need for manual verification of input security against static rules. Simultaneously, it automatically calculates the tool's reputation and risk value based on the target tool's status data, selecting a reliable and low-risk first tool. This avoids the tedious process of manually reviewing tool rules and intervening in complex scenarios for correction. Furthermore, it automatically calculates the deviation between the user input and the first tool's behavior sequence to determine the final suitable tool. The entire tool determination process is automated, reducing process delays caused by manual intervention. This solves the inefficiency problems of existing technologies that rely on static rules, have cumbersome processes, and require manual intervention, thus improving the efficiency of tool determination while ensuring the security of tool invocation.

[0064] In one possible design, multi-dimensional compliance detection includes sensitive word detection and risk intent detection. S202: Perform multi-dimensional compliance detection on the user input statement. If the multi-dimensional compliance detection result is a pass, input the user input statement into a preset probability calculation model to obtain the suspicious probability of the user input statement, including:

[0065] S2021. Perform sensitive word detection on the user input statement and obtain the sensitive word detection result; wherein, the sensitive word detection result includes sensitive word detection passed.

[0066] Specifically, user input is compared to a pre-defined set of sensitive words, checking each word and sentence for any words belonging to that set. If no sensitive words are found, the sensitive word detection passes; if sensitive words are found, the detection fails, thus generating the sensitive word detection result. This step quickly identifies input containing sensitive information, filtering out obviously non-compliant user requests in advance and preventing such requests from entering the subsequent tool matching process, reducing unnecessary processing steps.

[0067] S2022. In response to the sensitive word detection result being that the sensitive word detection is passed, risk intent detection is performed on the user input statement to obtain the risk intent detection result; wherein, the risk intent detection result includes risk intent detection passed.

[0068] Specifically, after confirming that the user's input statement does not contain sensitive words, the semantics, contextual relationships, and potential appeals of the input statement can be analyzed in conjunction with the characteristics of statements with historically risky intentions. This analysis can determine whether the input statement contains intentions such as prompt injection or unauthorized access manipulation, and output the probability of suspicion. If no such intent is detected, the risk intent detection is considered passed; otherwise, it is considered failed, thus forming the risk intent detection result. This step, as a further verification after sensitive word detection, can identify inputs that do not contain sensitive words but contain hidden risks, further filtering non-compliant requests and adding a layer of security for subsequent tool calls, avoiding potential oversights that may occur with only sensitive word detection.

[0069] S2023. In response to the risk intent detection result being that the risk intent detection is passed, the user input statement is input into the probability calculation model to obtain the suspicious probability of the user input statement.

[0070] Specifically, after confirming that the user's input statement passes the risk intent detection, the statement is passed to a pre-trained probability calculation model. The model combines the characteristics of various historical risk intent statements with the semantic information of the input statement to calculate and output a numerical value reflecting the likelihood of the statement carrying a risk intent—the probability of suspicion. This step provides a quantitative and detailed evaluation of the risk level of the user's input statement, compensating for any minor oversights in the preceding qualitative detection. It offers a precise risk reference for subsequent decisions on whether to continue the tool's screening process, further ensuring the security of tool usage.

[0071] The technical effect of this solution in this embodiment is as follows: by specifying multi-dimensional compliance detection as a progressive detection process of sensitive word detection and risk intent detection, the sensitive word detection quickly filters user input containing obviously illegal content, and performs in-depth risk intent detection on the input that passes the sensitive word detection. Only when both pass the detection and the input passes the detection does the suspicious probability calculation stage begin. This layered detection mechanism not only improves the accuracy of compliance detection and effectively avoids non-compliant input interfering with subsequent risk assessment, but also optimizes the efficiency of the detection process, laying a reliable foundation for accurately judging the risk intent of user input.

[0072] In one possible design, S2021, sensitive word detection is performed on the user input statement to obtain sensitive word detection results; wherein, the sensitive word detection results include sensitive word detection passing, including:

[0073] S20211. Perform character segmentation on the user input statement to obtain the character sequence of the user input statement.

[0074] Specifically, the statement can be broken down into its constituent characters, including letters and punctuation, following the natural order of the user's input. These characters are then arranged in their original sequence to form the character sequence of the user's input. This step decomposes the complete statement into basic units that can be analyzed individually, providing a foundation for extracting sensitive words from the character sequence. This ensures that sensitive word extraction is accurate down to each character, improving the level of detail in the detection.

[0075] S20212. Extract sensitive words from the character sequence of the user input statement to obtain the sensitive word sequence of the user input statement.

[0076] Specifically, consecutive character combinations in a character sequence are compared with a pre-defined sensitive word database to identify matching elements. These matching elements are then arranged in the order they appear in the character sequence to form a sensitive word sequence of the user's input. This step is used to accurately locate sensitive content within the sentence, providing a concrete object for subsequent calculations of the sentence's sensitivity based on the sensitive word sequence, ensuring that sensitive word detection accurately captures sensitive information within the sentence.

[0077] S20213. Calculate the sensitivity of the user input statement based on the sensitive word sequence of the user input statement and the preset sensitivity calculation rules.

[0078] Specifically, the process begins by analyzing the type and occurrence of each sensitive word in the sensitive word sequence. Then, based on pre-defined sensitivity calculation rules, a corresponding sensitivity contribution value is assigned to each sensitive word. Finally, the sensitivity contribution values ​​of all sensitive words are aggregated and calculated according to the rules to determine the overall sensitivity level of the user input statement. This step quantifies the sensitivity level of the statement, providing a specific numerical basis for subsequent judgments on whether sensitive word detection passes.

[0079] S20214. In response to the user's input statement having a sensitivity level less than the preset sensitivity threshold, the sensitive word detection result is set to "sensitive word detection passed".

[0080] Specifically, after confirming that the sensitivity of the user's input statement is below a preset sensitivity threshold, the statement's sensitive word detection result can be marked as passed. This step is used to clarify that the statement meets the requirements regarding sensitive words, providing a basis for judgment for further processing of the statement, ensuring that only statements that comply with the sensitive word specifications proceed to the next stage, and guaranteeing the compliance of the processing process.

[0081] The technical effect of this solution in this embodiment is as follows: by performing step-by-step processing on user input statements, including character segmentation, sensitive word extraction, sensitivity calculation, and comparison with thresholds, it can accurately identify sensitive content contained in the statements, objectively judge the sensitivity compliance of the statements through quantitative methods, effectively filter out statements that comply with sensitive word specifications, ensure the safety and compliance of subsequent processing of user input statements, improve the accuracy and reliability of sensitive word detection, and prevent non-compliant statements from entering subsequent stages.

[0082] In one possible design, multiple state data include location status, data signature status, capability data, and slot data. S203, based on the multiple state data of the target tool, calculate the reputation and risk value of the target tool, including:

[0083] S2031. Determine the digital authentication result of the first tool based on the location status and data signature status; wherein, the location status includes a first status and a second status, the first status indicates that the first tool is in a preset trusted database, the second status indicates that the first tool is not in a trusted database, the data signature status includes a third status and a fourth status, the third status indicates that the data signature of the first tool is valid, the fourth status indicates that the data signature of the first tool is invalid, and the digital authentication result indicates the compliance status of the first tool at the identity level.

[0084] Specifically, the process begins by checking whether the tool's location is within a pre-defined trusted database and whether its data signature is valid. If the tool's location is within a trusted database or its data signature is valid, the tool's digital identity verification is considered successful. If the tool's location is not within a trusted database and its data signature is invalid, the tool's digital identity verification is considered unsuccessful, thus determining the final digital identity verification result. This step verifies the tool's compliance at the identity level, providing a basis for calculating the tool's credibility based on historical call success rates. It also preemptively excludes non-compliant tools, preventing them from entering subsequent screening processes and ensuring the security of subsequent calls from the tool's identity source.

[0085] For example, it can be used to verify whether the tool is in a trusted tool library and whether the data signature is valid. Let the digital authentication result be I. When the tool is in a trusted tool library and the signature is valid, I=1; when either the tool is in a trusted tool library or the signature is valid, I=0.5; when the verification tool is not in a trusted tool library and the data signature is invalid, I=0, and the verification is directly blocked.

[0086] S2032. In response to the first tool's location status being in the first state or its data signature status being in the third state, obtain the historical call success rate of the first tool; wherein, the historical call success rate refers to the call success rate of the first tool within a preset first time period, and the end time of the first time period is earlier than the current time.

[0087] Specifically, after confirming that the location status of the first tool matches the preset trusted database, or that the data signature status is valid, all call records within a preset time period up to the current time are filtered from the tool's past call records. The ratio of successful calls to the total number of calls within that time period is then calculated to obtain the historical call success rate of the first tool. This step is used to obtain key data reflecting the tool's past reliability, providing a core basis for subsequently calculating the tool's credibility in conjunction with digital identity verification results, and thus helping to initially identify tools with stable performance from among many tools.

[0088] S2033. Calculate the credibility of the first tool based on the digital authentication result of the first tool and the historical call success rate of the first tool.

[0089] Specifically, a basic reputation score can be set first based on the digital identity verification results of the first tool. A higher base score is assigned if verification is successful, and a lower base score is assigned if verification fails. Then, the basic reputation score is adjusted based on the historical call success rate. A higher success rate results in a greater increase in the base score, while a lower success rate results in a greater decrease. The overall reputation of the first tool is obtained through a comprehensive calculation of the base score and the adjustment. This step, by combining the digital identity verification results with the historical call success rate, makes the reputation calculation more comprehensive and objective, providing a precise quantitative basis for subsequently selecting reliable tools and avoiding the bias that may result from relying on only a single dimension to evaluate tool reliability.

[0090] For example, the formula for calculating creditworthiness can be expressed as:

[0091]

[0092] in, For reputation, For digital identity verification results, For historical call success rate, and As weight, and The sum of is 1.

[0093] S2034. Calculate the risk value based on the user input statement, capability data, and slot data; wherein, capability data is used to represent the function of the first tool, and slot data is used to represent the multiple parameters required to call the first tool and the format of each parameter.

[0094] Specifically, the process begins by analyzing whether the user's input matches the functionality reflected in the tool's capability data, determining if the requirement exceeds the tool's functional scope or conflicts with it. Next, combining this with slot data, the process checks whether the calling parameters involved in the requirement conform to the tool's required parameter format, and whether there are any anomalies that could potentially cause risks. Based on the degree of deviation in functionality matching and the number of parameter compliance and adaptation issues, the risk value of the first tool is quantified. This step quantifies the tool's calling risk under current user needs from two dimensions: the adaptability of requirements to tool functionality and parameter compliance. This helps avoid calling failures and security risks caused by functionality mismatch or parameter issues, providing crucial evidence for subsequent selection of low-risk tools.

[0095] The technical effect of this solution in this embodiment is as follows: First, by determining the digital identity verification result based on location and data signature at the identity level, tools with compliant identities are initially screened out, preventing non-compliant tools from entering the subsequent process; then, the historical call success rate of compliant tools is obtained, and the credibility is calculated in combination with the identity verification result, so that the reliability evaluation of the tool takes into account both identity compliance and actual usage performance, reducing the bias of single-dimensional evaluation; at the same time, risk values ​​are calculated based on user input statements, capability data, and slot data, quantifying the risks of tool and user needs in terms of functional adaptation and parameter compliance, excluding tools with high risk of not adapting to needs, and improving the accuracy of tool selection.

[0096] In one possible design, S2031, based on the location status and data signature status, determines the digital authentication result of the first tool, including:

[0097] S20311. In response to the location state being in the first state and the data signature state being in the third state, the digital authentication result of the first tool is set to the first value; wherein, the digital authentication result of the first tool being the first value is used to indicate that the first tool is in a trusted database and that the data signature of the first tool is valid.

[0098] Specifically, once both conditions are met—that the tool is located in a pre-defined trusted database and that its data signature is valid—the tool's digital identity verification result can be set as the corresponding identifier. This reflects the tool's status as both located in a trusted database and possessing a valid data signature. This step accurately identifies the tool's compliance at the identity level. By clearly distinguishing different combinations of location and signature status, it provides detailed identity verification criteria for subsequent calculations of tool credibility using other indicators, ensuring the accuracy and discriminativeness of identity evaluation.

[0099] S20312. In response to the location state being in the first state and the data signature state being in the fourth state, the digital authentication result of the first tool is set to the second value; wherein, the digital authentication result of the first tool being set to the second value indicates that the first tool is in a trusted database and that the data signature of the first tool is invalid.

[0100] Specifically, after confirming that the tool is located in a pre-defined trusted database and that its data signature is invalid, the tool's digital identity verification result can be set as the corresponding identifier. This indicates that the tool is in a trusted database but its data signature is invalid. This step is used to accurately distinguish situations where the tool has some compliance flaws at the identity level, providing detailed identity verification basis for subsequent calculation of the tool's credibility using other indicators, and ensuring that the evaluation of the tool's identity compliance can cover different combinations of states.

[0101] S20313. In response to the location state being in the second state and the data signature state being in the third state, the digital authentication result of the first tool is set to the third value; wherein, the digital authentication result of the first tool being set to the third value indicates that the first tool is not in the trusted database and that the data signature of the first tool is valid.

[0102] Specifically, after confirming that the tool's location is not in a pre-defined trusted database and that its data signature is valid, the tool's digital identity verification result can be set as the corresponding identifier. This step is used to accurately identify the tool's special compliance status at the identity level. By clearly distinguishing such status combinations, it provides detailed identity verification basis for subsequent calculation of the tool's credibility using other indicators, ensuring that the evaluation of the tool's identity compliance can cover all possible status combinations, and improving the completeness and accuracy of the evaluation.

[0103] S20314. In response to the location state being in the second state and the data signature state being in the fourth state, the digital authentication result of the first tool is set to the fourth value; wherein, the digital authentication result of the first tool being the fourth value is used to indicate that the first tool is not in the trusted database and that the data signature of the first tool is invalid.

[0104] Specifically, if both conditions are met—that the tool is not located in a pre-defined trusted database and that its data signature is invalid—then the tool's digital identity verification result can be set as the corresponding identifier. This indicates that the tool is neither in a trusted database nor has its data signature. This step is used to accurately identify situations where the tool is completely non-compliant at the identity level. By clearly distinguishing such combinations of states, it provides detailed identity verification criteria for subsequent calculations of tool credibility using other indicators, ensuring that the evaluation of tool identity compliance can fully cover all possible combinations of states, thus improving the comprehensiveness and accuracy of the evaluation.

[0105] The technical effect of this solution in this embodiment is that by setting corresponding digital identity verification results for different combinations of the tool's location status and data signature status, various compliance situations of the tool at the identity level are clearly defined. This classification method provides an accurate basis for subsequent calculation of tool credibility based on digital identity verification results, ensuring the rigor and consistency of the identity verification process. This makes the subsequent evaluation of tool reliability more accurate, effectively improving the reliability of tool identity compliance judgment and providing solid identity verification support for screening safe and trustworthy tools.

[0106] In one possible design, S2034 calculates the risk value based on the user input statement, capability data, and slot data, including:

[0107] S20341. Calculate semantic inconsistency based on user input statements and capability data; wherein, semantic inconsistency is used to represent the degree of deviation between the capabilities of the first tool and user needs at the semantic level.

[0108] Specifically, the process begins by analyzing the core semantic needs expressed in the user's input, clarifying the user's desired goals and required functional directions. Next, the semantics of the tool's functions, as reflected in the capability data, are analyzed to determine the tool's achievable goals and functional scope. By comparing the degree of alignment between the two in terms of core goals and functional directions, the semantic deviations are quantified, thus revealing semantic inconsistencies. This step measures the semantic matching degree between the tool's functions and user needs, providing a basis for subsequent risk value calculations based on parameter structure differences, and preventing invalid calls or risks due to semantic mismatch between tool functions and user needs.

[0109] For example, the formula for calculating semantic inconsistency can be expressed as:

[0110]

[0111] in, This is due to semantic inconsistency. Enter a statement for the user. For capability data, This represents semantic encoding.

[0112] S20342. Perform feature extraction on the user input statement to obtain multiple extracted data; wherein, the multiple extracted data are used to represent the structured operation instructions for the first tool extracted from the user's needs.

[0113] Specifically, semantic parsing can be performed on user input statements to extract key information related to tool invocation, such as specific operation instructions, required parameter content, and logical relationships between parameters. This information is then transformed into a structured form, forming multiple extracted data sets. This step converts user requirements into structured information that can be directly compared with the tool parameter specifications, providing a concrete basis for subsequent calculations of inconsistencies in tool parameter structure and ensuring accurate determination of whether the parameter requirements in the user's needs match the tool parameter specifications.

[0114] S20343. Based on slot data and multiple extracted data, calculate the inconsistency of tool parameter structure; wherein, the inconsistency of tool parameter structure is used to represent the degree of deviation between the structured operation instructions extracted from user requirements and the parameter specifications of the first tool.

[0115] Specifically, the number of parameters, format requirements, and logical relationships between parameters specified in the slot data can be compared with the number of parameters, actual format, and logical relationships between parameters in multiple extracted data sets. Differences in parameter quantity (missing or redundant), format mismatches, and logical relationship conflicts can be statistically analyzed, and the severity of these differences can be quantified to determine the inconsistency in the tool's parameter structure. This step measures the degree of matching between user-defined operational instructions and the tool's parameter specifications at the parameter structure level, providing a basis for subsequent risk value calculations based on semantic inconsistencies and preventing tool call failures or anomalies due to parameter structure mismatches.

[0116] For example, the formula for calculating inconsistencies in tool parameter structures can be expressed as:

[0117]

[0118] in, The inconsistency in the tool parameter structure This represents the number of inconsistent slots verified through operations such as regular expressions and enumeration checks. This indicates the total number of slots for the current tool.

[0119] S20344. Calculate the risk value based on semantic inconsistency and inconsistency in tool parameter structure.

[0120] Specifically, the severity of semantic inconsistency and tool parameter structure inconsistency can be combined, assigning corresponding influence weights to each. The quantitative results of both are then comprehensively calculated according to their weights to obtain a numerical value that reflects the overall matching risk between the tool and user needs at both the semantic and parameter structure levels—that is, the risk value. This step integrates the matching deviations at the semantic and parameter structure levels into a unified risk quantification indicator, comprehensively measuring the invocation risk of the tool under the current user needs. This provides a comprehensive and clear evaluation basis for subsequent screening of low-risk tools, avoiding the selection of high-risk tools due to incomplete risk assessment from a single dimension.

[0121] The technical effect of this solution in this embodiment is as follows: By comprehensively calculating the tool risk value from two key dimensions—semantic adaptation and parameter compliance—it first evaluates the degree of semantic deviation between user input and tool capability data to accurately determine whether the tool's functions match user needs, avoiding invalid calls or security risks caused by functional mismatch. Then, by extracting structured operation instructions from user input and combining them with tool slot data, it checks the deviation between instructions and parameter specifications to prevent call anomalies or security risks caused by non-compliant parameters. This multi-dimensional evaluation method breaks the one-sidedness of single-dimensional judgment, making the calculation of risk values ​​more targeted and comprehensive. It can accurately identify potential risks in tool calls, providing a reliable basis for subsequent screening of low-risk tools.

[0122] In one possible design, S20344 calculates the risk value based on semantic inconsistency and inconsistency in the structure of tool parameters, including:

[0123] S203441. Calculate the overall inconsistency score based on semantic inconsistency and tool parameter structure inconsistency; wherein, the overall inconsistency score is used to represent the degree of overall deviation between the first tool and user needs at both the semantic and parameter structure levels.

[0124] Specifically, the degree of deviation between semantic inconsistency and tool parameter structure inconsistency can be combined. Based on the magnitude of their impact on the matching of tools with user needs, appropriate weights can be assigned. The quantitative results of both can be integrated and calculated according to their weights to obtain a numerical value that comprehensively reflects the overall deviation of the tool from user needs at both the semantic and parameter structure levels—that is, the comprehensive inconsistency score. This step integrates the deviations of the two dimensions into a unified comprehensive indicator, providing a foundation for subsequent calculations of risk values ​​based on factors such as suspicion probability, credibility, and tool sensitivity. This ensures that the risk value calculation comprehensively covers the deviations between the tool and needs in the core matching dimensions.

[0125] For example, the formula for calculating the overall inconsistency score can be expressed as:

[0126]

[0127] in, To incorporate the inconsistency score, This is due to semantic inconsistency. The inconsistency in the tool parameter structure and For weighted hyperparameters, and The sum of is 1, when Intercept when the inconsistency threshold is exceeded.

[0128] S203442. Obtain the first weight, second weight, third weight, fourth weight, and instrument sensitivity of the first instrument; wherein, the instrument sensitivity of the first instrument is used to represent the risk level of the first instrument.

[0129] Specifically, four weights corresponding to the importance of suspicion probability, comprehensive inconsistency score, reputation, and tool sensitivity in risk value calculation can be extracted from pre-defined risk assessment rules. Simultaneously, tool sensitivity is determined based on the functional attributes of the first tool and historical risk call records. This step obtains the key influence coefficients and the tool's own risk level information required for risk value calculation. This provides a reasonable weight allocation basis and fundamental data for subsequent risk value calculations combining suspicion probability, comprehensive inconsistency score, reputation, and other factors, ensuring that the risk value calculation reflects the actual impact of each factor on the risk.

[0130] S203443. Calculate the risk value based on the probability of doubt, the overall inconsistency score, the credibility, the instrument sensitivity, the first weight, the second weight, the third weight, and the fourth weight; wherein, the first weight is used to indicate the importance of the probability of doubt in the risk value calculation, the second weight is used to indicate the importance of the overall inconsistency score in the risk value calculation, the third weight is used to indicate the importance of the credibility in the risk value calculation, and the fourth weight is used to indicate the importance of the instrument sensitivity in the risk value calculation.

[0131] Specifically, the risk value can be calculated by multiplying the probability of suspicion by the first weight, the overall inconsistency score by the second weight, the credibility score by the third weight based on its impact on risk, and the tool sensitivity by the fourth weight. These products are then aggregated according to a pre-defined integration method to obtain the risk value. This step integrates multiple dimensions of factors, including the degree of suspicion of user input, the overall deviation between the tool and the need, the tool's own credibility, and the tool's inherent risk level, into a unified risk quantification indicator through reasonable weight allocation, comprehensively and accurately evaluating the overall risk of tool usage.

[0132] For example, the formula for calculating the risk value can be expressed as:

[0133]

[0134] Where R is the risk value. For function, , , and As weight.

[0135] The technical effect of this solution in this embodiment is as follows: By first integrating semantic inconsistency and tool parameter structure inconsistency into a comprehensive inconsistency score, it intuitively reflects the overall deviation of the tool from user needs in terms of semantics and parameters. Then, it further incorporates the probability of suspicious user input, the tool's credibility, and the tool's sensitivity reflecting its own risk level. Furthermore, by flexibly adjusting the importance of each factor in the risk assessment with different weights, it breaks the one-sidedness of evaluating risk from a single or few dimensions. This multi-dimensional and adjustable risk value calculation method can accurately control the role of each influencing factor according to the actual scenario requirements, making the final risk value more closely reflect the real risk situation, improving the comprehensiveness of the risk assessment, and providing a reliable basis for subsequent selection of safe and suitable tools.

[0136] Figure 2 This is one of the flowcharts illustrating the tool determination method provided in this application embodiment. In this embodiment, in Figure 4 Based on the provided embodiments, the tool determination method is further explained. The tool determination method includes:

[0137] S301. Obtain user input statements and multiple status data of the target tool; wherein, the user input statements are used to express user needs, and the target tool is any one of a number of preset tools.

[0138] S302. Perform multi-dimensional compliance checks on user input statements, and when the multi-dimensional compliance check result is that the compliance check passes, input the user input statement into a preset probability calculation model to obtain the suspicious probability of the user input statement; wherein, the multi-dimensional compliance check result includes compliance check passing, and the suspicious probability is used to represent the possibility that the user input statement has a risky intention.

[0139] S303. Calculate the reputation score and risk value of the target tool based on multiple status data of the target tool; wherein, the reputation score is used to represent the reliability of the target tool, and the risk value is used to represent the possibility that the target tool may have risks under the preset calling scenario.

[0140] S304. Filter multiple tools to obtain multiple first tools; wherein, multiple first tools refer to tools whose credit rating is greater than or equal to a preset credit rating threshold and whose risk value is less than a preset risk threshold.

[0141] S305. In response to the user input statement having a suspicious probability less than a preset suspicious threshold, obtain the behavior sequence of each first tool and calculate the deviation between the user input statement and the behavior sequence of each first tool; wherein, the behavior sequence of each first tool is used to represent the effect produced by each first tool after it is invoked in the calling scenario.

[0142] S306. The first tool corresponding to the minimum deviation is determined as the final tool; where the final tool refers to the tool that meets the user's needs.

[0143] S301-S306 are similar to S201-S206, and will not be described again in this embodiment.

[0144] S307. Invoke the final tool to obtain the behavior sequence of the final tool; wherein, the behavior sequence of the final tool is used to represent the effect produced by the final tool after it is invoked in the invoking scenario.

[0145] Specifically, following the final tool's invocation specifications, the required parameters are provided to trigger its execution. A series of operations and results generated during the tool's operation are recorded, and these operations and results are organized in chronological order to form the final tool's behavior sequence. This step captures the tool's actual performance and effects after being invoked, providing a concrete basis for subsequent calculations of the deviation score between its behavior sequence and user needs. This helps determine whether the tool truly meets user needs and provides a foundation for potential decisions to stop invocation.

[0146] S308. Calculate the deviation score between the behavior sequence of the final tool and the user's needs; wherein, the deviation score is used to indicate the degree of deviation between the behavior sequence of the final tool and the user's needs.

[0147] Specifically, we can first analyze the actual effects and goals achieved by the final tool's behavioral sequence, and then compare them with the expected effects and goals in the user's needs. This identifies differences in core requirements, degree of achievement, and outcome direction. Based on the number, importance, and scope of these differences, we quantify their severity to obtain a deviation score. This step is used to accurately measure the degree of match between the tool's actual performance and the user's needs, providing a concrete basis for determining whether to stop using the tool and preventing the tool from continuing to run without meeting user requirements.

[0148] S309. In response to a deviation score exceeding a preset score threshold, stop calling the final tool.

[0149] Specifically, once the deviation score between the final tool's behavior sequence and user needs exceeds a preset score threshold, the operation to interrupt the final tool's execution process can be triggered, terminating its ongoing activities and preventing the tool from continuing to execute subsequent steps. This step is used to promptly stop the tool from continuing to run due to deviations from user needs, avoiding resource waste caused by the tool's inability to meet user expectations, preventing unsuitable results, ensuring the invocation process can be flexibly adjusted according to actual matching conditions, and improving the effectiveness and resource utilization efficiency of tool invocation. After stopping the final tool invocation, suspicious samples that triggered interception during operation and cases with significant deviations between user input and tool execution results can be collected, manually reviewed, and added to the training dataset. Combining automatically generated attack templates and semantic variant techniques, more training samples are generated to improve model robustness. Tool invocation records are regularly organized, and invocation success rates are recalculated for dynamic maintenance of tool reputation. Security data is manually reviewed and fed back to the multidimensional risk function training set. Online logistic regression is used for updates. , , and Equal weighting ensures that the overall risk value R maintains high discriminative power under new attack models. Through the above mechanisms, the risk intent detection mechanism can be continuously upgraded, the tool reputation score can be dynamically adjusted based on online performance, the overall risk assessment function can be dynamically optimized based on feedback, and the global system can continuously adapt to and defend against new attacks.

[0150] The technical effect of this solution in this embodiment is as follows: by obtaining the behavior sequence of the tool and calculating its deviation score from the user's needs after determining and calling the final tool, a subsequent verification step is added to the tool calling effect, avoiding the problem that the tool execution effect may still not meet the needs after the initial screening; when the deviation score exceeds the preset threshold, the calling is stopped in time, which can effectively prevent unsuitable tools from continuing to run, prevent security risks or task failures caused by execution deviations, realize timely risk mitigation, and improve the fault tolerance and security of the tool calling process.

[0151] Figure 2 This is one of the flowcharts illustrating the tool determination method provided in this application embodiment. In this embodiment, in Figure 5 Based on the provided embodiments, the tool determination method is further explained. The tool determination method includes:

[0152] S401. Obtain multiple security samples and multiple risk samples; wherein, multiple security samples refer to multiple preset user input statements that do not contain sensitive words or risky intentions, and multiple risk samples refer to multiple preset user input statements that contain sensitive words or risky intentions.

[0153] Specifically, safe samples can be obtained by filtering statements from normal user interactions that do not contain sensitive words or risky intent. Alternatively, such statements can be pre-defined as safe samples based on common compliance user needs. Simultaneously, statements containing sensitive words or risky intent can be extracted from known risky user interaction records as risk samples. This process provides initial data for subsequent semantic variation processing of the samples, expanding the sample size and diversity, and ultimately laying the foundation for training a probabilistic calculation model that can accurately identify risks.

[0154] S402. Perform semantic variant processing on multiple security samples to obtain multiple variant security samples.

[0155] Specifically, for each safe sample, multiple variant safe samples can be generated by replacing words with similar meanings, adjusting sentence structure, and changing the order of content, while preserving the core semantics of the original sample and its features of being free of sensitive words and risky intent. This process expands the number and diversity of safe samples, making them richer for subsequent training of the probability calculation model. This helps improve the model's ability to recognize safe sentences with different expressions and enhances the model's generalization performance.

[0156] Semantic variant processing refers to generating new samples that differ from the original sentence in wording, sentence structure, or surface text features, while maintaining the core intent or semantics of the sentence. This is achieved through rewriting, reconstruction, or introducing noise. The aim is to simulate different language forms used by users in the real world to express the same needs, thereby effectively expanding the dataset and enhancing the model's understanding of semantic essence, improving its robustness against evasion attacks through text variants. Semantic variant processing can include synonym substitution, sentence structure transformation, expression normalization, and insertion of irrelevant information.

[0157] S403. Perform semantic variant processing on multiple risk samples to obtain multiple variant risk samples.

[0158] Specifically, for each risk sample, multiple variant risk samples can be generated by replacing words with similar meanings, adjusting sentence structure, and changing the order of content, while preserving the core semantics of the original sample and its features of containing sensitive words or risk intent. This process expands the number and diversity of risk samples, making the risk samples used for training the probability calculation model more abundant. This helps improve the model's ability to recognize risk statements with different expressions and enhances the model's generalization performance.

[0159] S404. Train the model based on multiple safe samples, multiple risk samples, multiple variant safe samples, and multiple variant risk samples to obtain a probability calculation model.

[0160] Specifically, multiple safe samples, multiple risk samples, multiple variant safe samples, and multiple variant risk samples can be integrated into training data. The model to be trained learns the feature differences between safe and risky statements from this data. By continuously adjusting the model parameters to optimize its ability to recognize safe and risky features, the model is optimized until it can stably distinguish between different types of samples. Ultimately, a probabilistic calculation model that can be used to calculate the risk probability of a statement is obtained. This step is used to train a model with accurate recognition capabilities through a rich variety of samples, providing a reliable computational foundation for subsequent risk assessment of user-input statements and ensuring more accurate risk probability calculations.

[0161] For example, binary classification 0-1 training data can be constructed, with safe samples corresponding to label 0 and risky samples corresponding to label 1. Safe samples can be obtained from non-risk requests in real business scenarios that have undergone manual review. Risky samples can be obtained in two ways: one is by automatically generating them using attack templates and semantic variants, and the other is by extracting inducement fragments from interception logs and enhancing them with strategy variants for high-risk tools such as disk writing and database deletion. Strategies include: changing single-turn dialogues to multi-turn step-by-step guidance, splitting sensitive commands and splicing them through context, etc.

[0162] The model uses a lightweight encoder, such as BERT with a binary classification head. In real training data, there may be a situation where the number of safe samples is large and the number of risk samples is small, i.e., a class imbalance problem. Therefore, FocalLoss is used for training loss.

[0163]

[0164] in, To focus on the loss function, The probability of suspicion. y and y represent the true label and the probability that the model predicts the sample as a risky sample, respectively. Used to adjust the weights between categories, for example, when there are few risk samples, risk samples The value can be set to a larger value. This is a focusing factor, targeting difficult-to-classify samples. In cases of extreme sample imbalance, adjustments can be made... and This reduces the dominant role of easily classified samples in the loss, allowing the model to focus more on difficult-to-classify samples.

[0165] Model outputs suspicious probability , Suspicious threshold: when At that time, the user inputs information into the large model for tool selection and parameter extraction; when At that time, intercept directly.

[0166] S405. Obtain user input statements and multiple status data of the target tool; wherein, the user input statements are used to express user needs, and the target tool is any one of a number of preset tools.

[0167] S406. Perform multi-dimensional compliance checks on user input statements, and when the multi-dimensional compliance check result is that the compliance check passes, input the user input statement into a preset probability calculation model to obtain the suspicious probability of the user input statement; wherein, the multi-dimensional compliance check result includes compliance check passing, and the suspicious probability is used to represent the possibility that the user input statement has a risky intention.

[0168] S407. Calculate the reputation score and risk value of the target tool based on multiple status data of the target tool; wherein, the reputation score is used to represent the reliability of the target tool, and the risk value is used to represent the possibility that the target tool may have risks under the preset calling scenario.

[0169] S408. Filter multiple tools to obtain multiple first tools; where multiple first tools refer to tools whose credit rating is greater than or equal to a preset credit rating threshold and whose risk value is less than a preset risk threshold.

[0170] S409. In response to the user input statement having a suspicious probability less than a preset suspicious threshold, obtain the behavior sequence of each first tool and calculate the deviation between the user input statement and the behavior sequence of each first tool; wherein, the behavior sequence of each first tool is used to represent the effect produced by each first tool after it is invoked in the calling scenario.

[0171] S410. The first tool corresponding to the minimum deviation is determined as the final tool; where the final tool refers to the tool that meets the user's needs.

[0172] S405-S410 are similar to S201-S206, and will not be described again in this embodiment.

[0173] The technical effect of this solution in this embodiment is as follows: by acquiring safe and risky samples, and performing semantic variant processing on the two types of samples to enrich the sample forms, and then using these diverse samples to train the probability calculation model, the model can be exposed to more safe and risky inputs in different forms of expression, thereby improving the model's ability to recognize various variant inputs, enhancing the model's generalization and judgment accuracy; the probability calculation model trained in this way can accurately calculate the suspicious probability of user input statements, providing a reliable basis for judging the user's risky input intention in the subsequent tool determination process.

[0174] Figure 5This is a schematic diagram of the structure of the tool determining device provided in the embodiments of this application. Figures 2 to 4 As shown, the tool determining device includes:

[0175] The first acquisition module 501 is used to acquire user input statements and multiple status data of the target tool; wherein, the user input statements are used to express user needs, and the target tool is any one of a number of preset tools.

[0176] The compliance detection module 502 is used to perform multi-dimensional compliance detection on user input statements. When the multi-dimensional compliance detection result is that the compliance detection is passed, the user input statement is input into a preset probability calculation model to obtain the suspicious probability of the user input statement. The multi-dimensional compliance detection result includes compliance detection passing and suspicious probability is used to indicate the possibility that the user input statement has risky intent.

[0177] The first calculation module 503 is used to calculate the reputation and risk value of the target tool based on multiple status data of the target tool; wherein, the reputation represents the reliability of the target tool, and the risk value represents the possibility that the target tool may have risks under a preset calling scenario.

[0178] The filtering module 504 is used to filter multiple tools to obtain multiple first tools; wherein, multiple first tools refer to tools whose reputation is greater than or equal to a preset reputation threshold and whose risk value is less than a preset risk threshold.

[0179] The second acquisition module 505 is used to acquire the behavior sequence of each first tool in response to the user input statement having a suspicious probability of less than a preset suspicious threshold, and to calculate the deviation between the user input statement and the behavior sequence of each first tool; wherein, the behavior sequence of each first tool is used to represent the effect produced by each first tool after it is invoked in the calling scenario.

[0180] The determination module 506 is used to determine the first tool corresponding to the minimum deviation as the final tool; where the final tool refers to the tool that meets the user's needs.

[0181] In one possible design, multidimensional compliance detection includes sensitive word detection and risk intent detection. Compliance detection module 502 includes:

[0182] The first detection unit is used to perform sensitive word detection on the user input statement and obtain sensitive word detection results; wherein, the sensitive word detection results include sensitive word detection passed.

[0183] The second detection unit is used to perform risk intent detection on the user input statement in response to the sensitive word detection result being that the sensitive word detection passed, and to obtain the risk intent detection result; wherein, the risk intent detection result includes risk intent detection passed.

[0184] The probability calculation unit is used to input the user input statement into the probability calculation model in response to the risk intent detection result being that the risk intent detection is passed, so as to obtain the suspicious probability of the user input statement.

[0185] In one possible design, multiple state data include location state, data signature state, capability data, and slot data. The first calculation module 503 includes:

[0186] The first determining unit is used to determine the digital authentication result of the first tool based on the location status and the data signature status; wherein, the location status includes a first status and a second status, the first status is used to indicate that the first tool is in a preset trusted database, the second status is used to indicate that the first tool is not in a trusted database, the data signature status includes a third status and a fourth status, the third status is used to indicate that the data signature of the first tool is valid, the fourth status is used to indicate that the data signature of the first tool is invalid, and the digital authentication result is used to indicate the compliance status of the first tool at the identity level.

[0187] The first acquisition unit is used to acquire the historical call success rate of the first tool in response to the first tool's location status being in the first state or its data signature status being in the third state; wherein, the historical call success rate refers to the call success rate of the first tool within a preset first time period, and the end time of the first time period is earlier than the current time.

[0188] The first calculation unit is used to calculate the credibility of the first tool based on the digital authentication result of the first tool and the historical call success rate of the first tool.

[0189] The second calculation unit is used to calculate the risk value based on the user input statement, capability data, and slot data; wherein, capability data is used to represent the function of the first tool, and slot data is used to represent the multiple parameters required to call the first tool and the format of each parameter.

[0190] In one possible design, the first defined unit includes:

[0191] The first setting component is configured to, in response to the location state being in the first state and the data signature state being in the third state, set the digital authentication result of the first tool to a first value; wherein, the digital authentication result of the first tool being set to the first value indicates that the first tool is in a trusted database and that the data signature of the first tool is valid.

[0192] The second setting component is used to set the digital authentication result of the first tool to a second value in response to the location state being in the first state and the data signature state being in the fourth state; wherein, the digital authentication result of the first tool being set to the second value is used to indicate that the first tool is in a trusted database and that the data signature of the first tool is invalid.

[0193] The third setting component is used to set the digital authentication result of the first tool to a third value in response to the location state being in the second state and the data signature state being in the third state; wherein, the digital authentication result of the first tool being set to the third value is used to indicate that the first tool is not in the trusted database and that the data signature of the first tool is valid.

[0194] The fourth setting component is used to set the digital authentication result of the first tool to a fourth value in response to the location state being in the second state and the data signature state being in the fourth state; wherein, the digital authentication result of the first tool being in the fourth value is used to indicate that the first tool is not in the trusted database and that the data signature of the first tool is invalid.

[0195] In one possible design, the second computing unit includes:

[0196] The first computational component is used to calculate semantic inconsistency based on user input statements and capability data; wherein, semantic inconsistency is used to represent the degree of deviation between the capabilities of the first tool and the user's needs at the semantic level.

[0197] The feature extraction component is used to extract features from user input statements to obtain multiple extracted data; among which, the multiple extracted data are used to represent the structured operation instructions for the first tool extracted from user needs.

[0198] The second calculation component is used to calculate the inconsistency of tool parameter structure based on slot data and multiple extracted data; wherein, the inconsistency of tool parameter structure is used to represent the degree of deviation between the structured operation instructions extracted from user requirements and the parameter specifications of the first tool.

[0199] The third computational component is used to calculate risk values ​​based on semantic inconsistencies and inconsistencies in the structure of tool parameters.

[0200] In one possible design, the third computational component includes:

[0201] The first calculation component is used to calculate a comprehensive inconsistency score based on semantic inconsistency and inconsistency in tool parameter structure; wherein, the comprehensive inconsistency score is used to represent the degree of comprehensive deviation between the first tool and user needs at both the semantic and parameter structure levels.

[0202] The first acquisition component is used to acquire the first weight, the second weight, the third weight, the fourth weight, and the tool sensitivity of the first tool; wherein, the tool sensitivity of the first tool is used to represent the risk level of the first tool.

[0203] The second calculation component is used to calculate the risk value based on the probability of suspicion, the overall inconsistency score, the credibility, the instrument sensitivity, the first weight, the second weight, the third weight, and the fourth weight. The first weight is used to indicate the importance of the probability of suspicion in the calculation of the risk value, the second weight is used to indicate the importance of the overall inconsistency score in the calculation of the risk value, the third weight is used to indicate the importance of the credibility in the calculation of the risk value, and the fourth weight is used to indicate the importance of the instrument sensitivity in the calculation of the risk value.

[0204] In one possible design, the tool determining device further includes:

[0205] The calling module is used to invoke the final tool and obtain the behavior sequence of the final tool; the behavior sequence of the final tool is used to represent the effect produced by the final tool after it is invoked in the calling scenario.

[0206] The scoring module is used to calculate the deviation score between the final tool's behavior sequence and user needs; the deviation score is used to represent the degree of deviation between the final tool's behavior sequence and user needs.

[0207] The "Stop Calling" module is used to stop calling the final tool in response to a deviation score exceeding a preset score threshold.

[0208] In one possible design, the tool determining device further includes:

[0209] The third acquisition module is used to acquire multiple security samples and multiple risk samples. Among them, multiple security samples refer to multiple preset user input statements that do not contain sensitive words or risky intentions, and multiple risk samples refer to multiple preset user input statements that contain sensitive words or risky intentions.

[0210] The first variant processing module is used to perform semantic variant processing on multiple security samples to obtain multiple variant security samples.

[0211] The second variant processing module is used to perform semantic variant processing on multiple risk samples to obtain multiple variant risk samples.

[0212] The model training module is used to train the model based on multiple safe samples, multiple risk samples, multiple variant safe samples, and multiple variant risk samples to obtain a probability calculation model.

[0213] The tool determination device provided in this embodiment can perform... Figures 2 to 4The technical solution of the tool determination method embodiment shown herein, its implementation principle and technical effects are similar to Figure 6 The implementation of the tool determination method shown is similar and will not be described in detail here.

[0214] Figure 6 This is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of this application. ​ As shown, the electronic device 60 includes at least one processor 601 and a memory 602. The electronic device 60 also includes a communication component 603. The processor 601, memory 602, and communication component 603 are connected via a bus 604.

[0215] In a specific implementation, at least one processor 601 executes computer execution instructions stored in memory 602, causing at least one processor 601 to implement a tool determination method of the above embodiment.

[0216] The specific implementation process of processor 601 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0217] In the above embodiments, it should be understood that the processor 601 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0218] The memory 602 may include high-speed RAM memory, and may also include non-volatile memory (NVM), such as at least one disk storage.

[0219] Bus 604 can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Bus 604 can be divided into address bus, data bus, control bus, etc. For ease of illustration, the bus 604 in the accompanying drawings of this application is not limited to only one bus or one type of bus.

[0220] The above description of the functions implemented by electronic devices and main control devices has introduced the solutions provided by the embodiments of the present invention. It is understood that, in order to implement the above functions, the electronic device or main control device includes hardware structures and / or software modules corresponding to the execution of each function. By combining the units and algorithm steps of the various examples described in the embodiments of the present invention, the embodiments of the present invention can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the technical solutions of the embodiments of the present invention.

[0221] This application also provides a computer-readable storage medium storing computer-executable instructions. When executed by a processor, these instructions are used to implement a tool determination method according to the above embodiments. In a specific implementation of the aforementioned tool determination method, each module can be implemented as a processor.

[0222] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0223] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in application-specific integrated circuits (ASICs). Alternatively, the processor and the readable storage medium can exist as discrete components in an electronic device or a host device.

[0224] This application also provides a computer program product, including a computer program, which, when executed by a processor, is used to implement a tool determination method described in the above embodiments.

[0225] The computer program is stored in a readable storage medium, and at least one processor can read the computer program from the readable storage medium and execute the computer program to perform the scheme provided in any of the above embodiments.

[0226] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disk, or optical disk.

[0227] The technical solutions of this application have been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it is readily understood by those skilled in the art that the scope of protection of this application is obviously not limited to these specific embodiments. The above embodiments are only used to illustrate the technical solutions of this application and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A method for determining a tool, characterized in that, include: Acquire user input statements and multiple status data of the target tool; wherein, the user input statements are used to represent user needs, and the target tool is any one of a set of preset tools; The user input statement undergoes a multi-dimensional compliance check. When the multi-dimensional compliance check result is a pass, the user input statement is input into a preset probability calculation model to obtain the suspicious probability of the user input statement. The multi-dimensional compliance check result includes the pass of the compliance check, and the suspicious probability is used to indicate the possibility that the user input statement has a risky intent. Based on multiple status data of the target tool, the reputation score and risk value of the target tool are calculated; wherein, the reputation score is used to represent the reliability of the target tool, and the risk value is used to represent the probability that the target tool has risks under a preset calling scenario; The plurality of tools are filtered to obtain a plurality of first tools; wherein, the plurality of first tools refer to the tools among the plurality of tools whose reputation is greater than or equal to a preset reputation threshold, and whose risk value is less than a preset risk threshold; In response to the user input statement having a suspicious probability less than a preset suspicious threshold, the behavior sequence of each of the first tools is obtained, and the deviation between the user input statement and the behavior sequence of each of the first tools is calculated; wherein, the behavior sequence of each of the first tools is used to represent the effect produced by each of the first tools after being invoked in the invocation scenario; The first tool corresponding to the minimum deviation is determined as the final tool; wherein, the final tool refers to the tool that meets the user's needs.

2. The tool determination method according to claim 1, characterized in that, The multi-dimensional compliance detection includes sensitive word detection and risk intent detection. The user input statement undergoes multi-dimensional compliance detection, and when the multi-dimensional compliance detection result indicates that the compliance detection has passed, the user input statement is input into a preset probability calculation model to obtain the suspicious probability of the user input statement, including: Sensitive word detection is performed on the user input statement to obtain sensitive word detection results; wherein, the sensitive word detection results include sensitive word detection passing; In response to the sensitive word detection result being that the sensitive word detection passed, risk intent detection is performed on the user input statement to obtain a risk intent detection result; wherein, the risk intent detection result includes risk intent detection passing; In response to the risk intent detection result being that the risk intent detection is passed, the user input statement is input into the probability calculation model to obtain the suspicious probability of the user input statement.

3. The tool determination method according to claim 1, characterized in that, The multiple status data include location status, data signature status, capability data, and slot data. The calculation of the target tool's reputation and risk value based on these multiple status data includes: Based on the location status and the data signature status, the digital authentication result of the first tool is determined; wherein, the location status includes a first status and a second status, the first status indicates that the first tool is in a preset trusted database, the second status indicates that the first tool is not in the trusted database, the data signature status includes a third status and a fourth status, the third status indicates that the data signature of the first tool is valid, the fourth status indicates that the data signature of the first tool is invalid, and the digital authentication result indicates the compliance status of the first tool at the identity level; In response to the first tool's location state being the first state, or the data signature state being the third state, the historical call success rate of the first tool is obtained; wherein, the historical call success rate refers to the call success rate of the first tool within a preset first time period, and the end time of the first time period is earlier than the current time; The credibility of the first tool is calculated based on the digital authentication result of the first tool and the historical call success rate of the first tool; The risk value is calculated based on the user input statement, the capability data, and the slot data; wherein the capability data represents the function of the first tool, and the slot data represents multiple parameters required to invoke the first tool and the format of each parameter.

4. The tool determination method according to claim 3, characterized in that, Determining the digital authentication result of the first tool based on the location status and the data signature status includes: In response to the location state being the first state and the data signature state being the third state, the digital authentication result of the first tool is set to a first value; wherein, the digital authentication result of the first tool being the first value indicates that the first tool is in the trusted database and that the data signature of the first tool is valid. In response to the location state being the first state and the data signature state being the fourth state, the digital authentication result of the first tool is set to a second value; wherein, the digital authentication result of the first tool being set to the second value indicates that the first tool is in the trusted database and the data signature of the first tool is invalid. In response to the location state being the second state and the data signature state being the third state, the digital authentication result of the first tool is set to the third value; wherein, the digital authentication result of the first tool being the third value indicates that the first tool is not in the trusted database, and the data signature of the first tool is valid; In response to the location state being the second state and the data signature state being the fourth state, the digital authentication result of the first tool is set to the fourth value; wherein, the digital authentication result of the first tool being the fourth value indicates that the first tool is not in the trusted database and that the data signature of the first tool is invalid.

5. The tool determination method according to claim 3, characterized in that, The step of calculating the risk value based on the user input statement, the capability data, and the slot data includes: Based on the user input statement and the capability data, semantic inconsistency is calculated; wherein, the semantic inconsistency is used to represent the degree of deviation between the capabilities of the first tool and the user's needs at the semantic level; Feature extraction is performed on the user input statement to obtain multiple extracted data; wherein, the multiple extracted data are used to represent the structured operation instructions for the first tool extracted from the user's needs; Based on the slot data and the multiple extracted data, the inconsistency of the tool parameter structure is calculated; wherein, the inconsistency of the tool parameter structure is used to represent the degree of deviation between the structured operation instructions extracted from the user requirements and the parameter specifications of the first tool; The risk value is calculated based on the semantic inconsistency and the inconsistency in the tool parameter structure.

6. The tool determination method according to claim 5, characterized in that, The step of calculating the risk value based on the semantic inconsistency and the inconsistency in the tool parameter structure includes: Based on the semantic inconsistency and the inconsistency of the tool parameter structure, a comprehensive inconsistency score is calculated; wherein, the comprehensive inconsistency score is used to represent the degree of overall deviation between the first tool and the user's needs at both the semantic and parameter structure levels; Obtain the first weight, the second weight, the third weight, the fourth weight, and the tool sensitivity of the first tool; wherein, the tool sensitivity of the first tool is used to represent the risk level of the first tool; The risk value is calculated based on the suspicion probability, the overall inconsistency score, the credibility, the instrument sensitivity, the first weight, the second weight, the third weight, and the fourth weight; wherein the first weight represents the importance of the suspicion probability in the risk value calculation, the second weight represents the importance of the overall inconsistency score in the risk value calculation, the third weight represents the importance of the credibility in the risk value calculation, and the fourth weight represents the importance of the instrument sensitivity in the risk value calculation.

7. The tool determination method according to claim 1, characterized in that, After determining the first tool corresponding to the minimum deviation as the final tool, the method further includes: The final tool is invoked to obtain a sequence of behaviors of the final tool; wherein, the sequence of behaviors of the final tool is used to represent the effect produced by the final tool after it is invoked in the invocation scenario; Calculate a deviation score between the behavior sequence of the final tool and the user's needs; wherein the deviation score is used to represent the degree of deviation between the behavior sequence of the final tool and the user's needs; In response to the deviation score exceeding a preset score threshold, the invocation of the final tool is stopped.

8. The tool determination method according to claim 1, characterized in that, Before acquiring the user input statement and multiple state data of the target tool, the process also includes: Acquire multiple security samples and multiple risk samples; wherein, the multiple security samples refer to multiple preset user input statements that do not contain sensitive words or risky intentions, and the multiple risk samples refer to multiple preset user input statements that contain sensitive words or risky intentions. Semantic variant processing is performed on the multiple security samples to obtain multiple variant security samples; The multiple risk samples are semantically variant processed to obtain multiple variant risk samples; The probability calculation model is obtained by training the model based on the multiple safe samples, the multiple risk samples, the multiple variant safe samples, and the multiple variant risk samples.

9. A tool determining device, characterized in that, include: The first acquisition module is used to acquire user input statements and multiple status data of the target tool; wherein, the user input statements are used to express user needs, and the target tool is any one of a set of preset tools; The compliance detection module is used to perform multi-dimensional compliance detection on the user input statement, and when the multi-dimensional compliance detection result is that the compliance detection is passed, the user input statement is input into a preset probability calculation model to obtain the suspicious probability of the user input statement; wherein, the multi-dimensional compliance detection result includes the compliance detection being passed, and the suspicious probability is used to indicate the possibility that the user input statement has a risky intention; The first calculation module is used to calculate the reputation and risk value of the target tool based on multiple status data of the target tool; wherein the reputation represents the reliability of the target tool, and the risk value represents the probability that the target tool is at risk in a preset calling scenario; The filtering module is used to filter the plurality of tools to obtain a plurality of first tools; wherein, the plurality of first tools refers to tools among the plurality of tools whose reputation is greater than or equal to a preset reputation threshold and whose risk value is less than a preset risk threshold; The second acquisition module is used to acquire the behavior sequence of each of the first tools in response to the user input statement having a suspicious probability of less than a preset suspicious threshold, and to calculate the deviation between the user input statement and the behavior sequence of each of the first tools; wherein, the behavior sequence of each of the first tools is used to represent the effect produced by each of the first tools after being invoked in the invocation scenario; The determination module is used to determine the first tool corresponding to the minimum deviation as the final tool; wherein the final tool refers to the tool that meets the user's needs.

10. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; When the processor executes the computer execution instructions stored in the memory, it is used to implement the tool determination method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the tool determination method as described in any one of claims 1 to 8.

12. A computer program product, characterized in that, Includes a computer program, which, when executed by a processor, is used to implement the tool determination method as described in any one of claims 1 to 8.

Images