Large model tool calling method and system based on risk assessment

Abstract

The invention discloses a large model tool calling method and system based on risk assessment, and the method comprises the following steps: receiving a natural language instruction of a user, and obtaining a calling request of the user based on a large model; generating a risk score based on the user call request; generating a confirmation strategy based on the risk score, the user preference adjustment factor and the emergency degree adjustment factor; executing a corresponding confirmation process based on the confirmation strategy; based on the confirmation process, the controller executes tool calling; by the adoption of the technical scheme, the problem of balance between safety and user experience in tool calling is solved, a safer and more intelligent interaction mechanism is provided for voice control and large language model driven tool calling, a risk self-adaptive confirmation process and a multi-stage confirmation mechanism provide sufficient safety guarantee, and meanwhile the safety of a user is improved. And the user interaction experience is improved to the greatest extent.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of human-computer interaction and computer security, in particular to a large model tool calling method based on risk assessment and a system thereof. BACKGROUND

[0002] With the continuous maturity of large language model (hereinafter referred to as large model, LLM) technology, the use of natural language to realize the control of various tools and system functions has fallen from a technical concept to a real application, which has profoundly changed the mode of human-computer interaction. However, the current large model tool calling method still has obvious deficiencies in the operation confirmation mechanism, especially in the differentiation of security confirmation, dynamic risk assessment, user individualization adaptation and the comprehensiveness of parameter confirmation, which need to be improved to meet the growing demand for intelligent calling security and user experience.

[0003] Firstly, the existing tool calling method generally adopts a single security confirmation mechanism, which fails to differentiate according to the actual risk level of the operation. Whether it is a low-risk query operation or a high-risk financial transaction, the same confirmation process is often used, resulting in redundant confirmation for users in low-risk scenarios, which reduces the experience, while high-risk operations may pose a security risk due to insufficient confirmation. Secondly, the existing tool calling method lacks dynamic assessment capability for operation risk, relying only on predefined static rules, which cannot comprehensively consider key factors such as parameter content (such as the size of the transfer amount), user historical behavior patterns and environmental context (such as operation time and location), making it difficult to implement a confirmation strategy that matches real-time risks. In addition, the existing mechanism is deficient in user individualization, failing to fully consider the differences in confirmation frequency tolerance, operation familiarity and real-time context (such as time urgency) among different users, resulting in a lack of adaptability in the confirmation strategy, which affects user experience. Finally, at the parameter level, the system often only confirms the operation as a whole, ignoring the separate prompting and verification of key parameters, especially high-sensitive parameters. This defect not only makes users possibly ignore the setting of important parameters, but also leads to the need to re-initiate the entire process for error modification, reducing interaction efficiency and security, and lacking a key confirmation mechanism for high-risk parameters. SUMMARY

[0004] In view of the above problems, the present application provides a large model tool calling method based on risk assessment, which solves the problem of insufficient risk assessment capability of the existing tool calling method.

[0005] To achieve the above purpose, the inventors provide a large model tool calling method based on risk assessment, which includes the following steps:

[0006] Receiving a user natural language instruction, and obtaining a user calling request based on a large model;

[0007] generating a risk score based on the user invocation request;

[0008] generating a confirmation strategy based on the risk score, a user preference adjustment factor and an urgency adjustment factor;

[0009] executing a corresponding confirmation process based on the confirmation strategy;

[0010] based on the confirmation process, the controller executes a tool invocation.

[0011] Further, the user invocation request comprises a tool identifier, an operation intention, parameters, and user information.

[0012] Further, the step of generating a risk score based on the user invocation request comprises the following steps:

[0013] obtaining a tool inherent risk level, a parameter risk score, a user permission factor, a historical behavior analysis, and an environmental factor based on the user invocation request;

[0014] adopting a multi-factor comprehensive scoring model to score the risk based on the tool inherent risk level, the parameter risk score, the user permission factor, the historical behavior analysis, and the environmental factor;

[0015] The calculation formula of the risk score R of the multi-factor comprehensive scoring model is:

[0016] R = min([(R b · w b + R p · w p )· F u · F h + F e ], 10)

[0017] wherein R b represents the tool inherent risk level; R p represents the parameter risk score; F u represents the user permission factor; F h represents the historical behavior analysis; and F e represents the environmental factor.

[0018] Further, the calculation formula of the confirmation strategy L is as follows:

[0019]

[0020] wherein R represents the risk score; P a represents the user preference adjustment factor; U f represents the urgency adjustment factor. represents a floor function.

[0021] Further, the confirmation level based on the confirmation strategy executes a corresponding confirmation process, and in the step of executing the tool call, the executing the corresponding confirmation process further includes canceling operation, performing operation, and parameter correction according to the user response.

[0022] Further, the confirmation process includes a direct execution process, an implicit confirmation process, an explicit confirmation process, or a multi-stage confirmation process.

[0023] Further, the parameter correction is based on the large model extracting parameter modification intention from the user's unstructured natural language.

[0024] Further, the implementation of the execution controller is as follows:

[0025] P i = φ(R, T i , H, C);

[0026] Wherein, P i ∈ [0, 1] represents the participation weight of the large model in the decision link i; R represents the risk score; T i represents the type of decision link; H represents the historical performance matrix, recording the historical accuracy rate of the model in different links; C represents the current context feature vector.

[0027] Further, the controller execution tool adopts a "prediction-observation-adjustment" cycle mode for monitoring, continuously comparing the state change in the execution process with the expected intention, and when detecting that the state change in the execution process deviates from the expected intention, performing operations including pausing execution, requesting user confirmation, or automatically adjusting the execution path.

[0028] A tool calling system based on risk assessment, comprising:

[0029] A risk assessment module: for receiving user call requests and comprehensively analyzing tool types, parameter values, user permissions, historical use records, and current environmental factors to generate a risk score;

[0030] A confirmation strategy generator for generating a confirmation strategy based on the risk score, user preference adjustment factor, and urgency adjustment factor, the confirmation strategy including confirmation level, confirmation method, and confirmation content;

[0031] A multi-level confirmation process manager: for executing the corresponding confirmation process of the confirmation level, the confirmation process including a direct execution process, an implicit confirmation process, an explicit confirmation process, or a multi-stage confirmation process;

[0032] A user feedback analyzer: for situational response understanding and intention recognition, and analyzing user responses to the confirmation process, judging the effectiveness of the confirmation process and processing parameter correction in the confirmation process;

[0033] Execution controller: used to control the final execution of the tool, including execution permission control and execution process monitoring.

[0034] Unlike the prior art, the above technical solution solves the balance between security and user experience in tool invocation through risk assessment and dynamic confirmation strategy generation, providing a safer and smarter interaction mechanism for voice control and large language model driven tool invocation. The risk adaptive confirmation process and multi-level confirmation mechanism provide sufficient security protection while maximizing user interaction experience.

[0035] The above invention content is only a summary of the technical solution of the present application. In order to enable those skilled in the art to more clearly understand the technical solution of the present application, and then can be implemented according to the content of the description and the drawings, and in order to let the above purpose and other purposes, characteristics and advantages of the present application can be more easily understood, the following is described in conjunction with the specific embodiments of the present application and the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0036] The accompanying drawings are only used to illustrate the principles, implementation methods, applications, characteristics and effects of the specific embodiments and other related contents of the present application, and cannot be considered as a limitation of the present application.

[0037] In the drawings of the specification:

[0038] Figure 1 Flowchart of the method for invoking a large model tool based on risk assessment according to the specific embodiments;

[0039] Figure 2 Flowchart of the direct execution process according to the specific embodiments;

[0040] Figure 3 Flowchart of the implicit confirmation process according to the specific embodiments;

[0041] Figure 4 Flowchart of the explicit confirmation process according to the specific embodiments;

[0042] Figure 5 Flowchart of the multi-stage confirmation process according to the specific embodiments. DETAILED DESCRIPTION

[0043] In order to explain the possible application scenarios, technical principles, specific schemes that can be implemented, and achieve the purpose and effect, the following will be described in conjunction with the specific examples listed and the accompanying drawings. The examples described in this paper are only used to more clearly illustrate the technical solution of the present application, therefore only as an example, and cannot be used to limit the protection scope of the present application.

[0044] The term "embodiment" is mentioned herein means that the specific features, structures, or characteristics described in connection with the embodiment can be included in at least one embodiment of the present application. The term "embodiment" appearing in various places in the specification does not necessarily refer to the same embodiment, nor does it particularly limit the independence or association between other embodiments. In principle, in the present application, as long as there is no technical contradiction or conflict, each technical feature mentioned in each embodiment can be combined in any way to form a corresponding implementable technical solution.

[0045] Unless otherwise defined, the meanings of the technical terms used herein are the same as those commonly understood by those skilled in the art to which the present application belongs; the use of related terms herein is only for the purpose of describing specific embodiments, and is not intended to limit the present application.

[0046] In the description of the present application, the phrase "and / or" is a description of the logical relationship between the objects, which means that there can be three relationships, for example, A and / or B, which means that there are three cases: A exists, B exists, and A and B exist at the same time. In addition, the character " / " herein generally represents that the associated objects before and after are a "or" logical relationship.

[0047] In the present application, terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual quantity, primary and secondary, or order relationship between the entities or operations.

[0048] In the present application, without more limitation, the "includes", "contains", "has" or other similar open expressions used in the sentence are intended to cover non-exclusive inclusion, and these expressions do not exclude the presence of other elements in the process, method or product including the described elements, so that the process, method or product including a series of elements can not only include those limited elements, but also include other elements not explicitly listed, or also include the elements inherent to such process, method or product.

[0049] As the same understanding as in the "Guidelines for Examination", in the present application, the expressions "greater than", "less than", "exceed" and the like are understood as not including the number; the expressions "above", "below", "within" and the like are understood as including the number. In addition, in the description of the embodiments of the present application, the meaning of "multiple" is more than two (including two), and similar expressions related to "multiple" are also understood in this way, for example, "multiple groups", "multiple times" and the like, unless otherwise explicitly limited.

[0050] In the description of the embodiments of the present application, the spatial relative expressions used, such as "central", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "vertical", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential", and the like, indicate the orientation or positional relationship shown in the specific embodiments or the drawings, and are only used for the convenience of describing the specific embodiments of the present application or for the convenience of the reader to understand, and do not indicate or imply that the indicated device or component must have a particular position, a particular orientation, or be constructed or operated in a particular orientation, and therefore cannot be understood as a limitation on the embodiments of the present application.

[0051] The processor described in the embodiments of the present application can be implemented by hardware, firmware, software or a combination thereof, and can use at least one of circuit, single or multiple application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), central processing units (CPU), controllers, microcontrollers, microprocessors, and other physical, biological or chemical structures that can realize the same or equivalent functions as the above-mentioned processors, such as biological neurons, quantum computing units, DNA computing units, etc., so that the processor can execute part or all of the steps or any combination of the steps mentioned in the computer programs or methods of the embodiments of the present application.

[0052] The computer program involved in the embodiments can be stored in a computer device readable storage medium, including but not limited to magnetic disk, magnetic tape, magnetic card, floppy disk, flash memory, optical disc, optical card, read-only memory (ROM), random access memory (RAM), erasable programmable ROM (EPROM) and electrically erasable programmable ROM (EEPROM) and the like, and also includes other biological, physical or chemical structures that can realize similar or equivalent functions to the above-mentioned storage media, such as DNA, RNA, protein and the like units with information storage ability. In specific embodiments, the storage medium can be one of the above-mentioned medium types, or a combination of the above-mentioned medium types. In different embodiments, the computer program involved in the embodiments can be centrally stored in a single medium, or distributedly stored in multiple media. The storage medium containing the computer device readable storage medium can be a non-volatile memory or a random access memory. These computer device readable storage media can be built-in in the device, or connected with the device as an external device or part of the external device. In some embodiments, the storage medium with the computer device readable storage medium is deployed locally; in other embodiments, the storage medium can also be deployed remotely from the processor, such as network-attached storage accessed via RF circuit or external port and communication network, wherein the communication network can be the Internet, one or more intranets, local area network (LAN), wide area network (WAN), storage area network (SAN) and the like, or a suitable combination thereof, as long as the computer device can access the storage medium. In addition, the computer program involved in the embodiments can be stored in plaintext / ciphertext, or can be designed as training data, and integrated and reorganized by model training to be implicitly saved in the parameter state of the deep neural network or other machine learning model.

[0053] A large model tool calling method based on risk assessment is generated through risk assessment and dynamic confirmation strategy, which solves the balance problem of security and user experience in tool calling, provides a safer and smarter interaction mechanism for voice control and large language model driven tool calling, and a risk adaptive confirmation process and multi-level confirmation mechanism maximizes the user interaction experience while providing sufficient security protection.

[0054] The following Figure 1 , provides an embodiment of a large model tool calling method based on risk assessment, which includes the following steps:

[0055] S10, receiving a user natural language instruction, and obtaining a user calling request based on a large model;

[0056] S20, generating a risk score based on the user calling request;

[0057] S30, generating a confirmation strategy based on the risk score, the user preference adjustment factor, and the urgency adjustment factor;

[0058] S40, executing a corresponding confirmation process based on the confirmation strategy;

[0059] S50, based on the confirmation process, the controller executes tool invocation.

[0060] The user invocation request of step S10 refers to a user using natural language or structured instructions to obtain a user invocation request based on a large model to command the tool to perform a specific task or obtain a certain resource. The user invocation request usually includes tool identification, operation intent (action type), parameters (determine the specific content and scope of the operation), user information (including user identification, user permission, IP address, device, etc.), etc. The above-mentioned large model (large language model, LLM) is an artificial intelligence (AI) based, trained by massive text data, and can understand, generate and process natural language prediction model, such as GPT series, Gemini series, Claude, etc.

[0061] In step S20, the step of generating a risk score based on the user invocation request includes the following steps:

[0062] Based on the user invocation request, obtain the tool inherent risk level, parameter risk score, user permission factor, historical behavior analysis, and environmental factors;

[0063] Based on the tool inherent risk level, parameter risk score, user permission factor, historical behavior analysis, and environmental factors, a multi-factor comprehensive scoring model is used to score the risk;

[0064] The calculation formula of the risk score R of the multi-factor comprehensive scoring model is:

[0065] R = min([(R b ·w b +R p ·w p )·F u ·F h +F e ], 10)

[0066] Wherein, R b represents the tool inherent risk level; w b represents the inherent risk weight; R p represents the parameter risk score; w p represents the parameter risk weight; F u is the user permission factor; F h represents the historical behavior analysis; F e is the environmental factor.

[0067] The tool inherent risk level refers to the potential risk level that the tool itself may bring after being invoked for execution. This is a risk level preset based on a comprehensive assessment of its operation capabilities (including deletion, writing, modification, execution, reading, etc.), data impact range (including global impact, local impact, or no impact), and compliance requirements (involving personal privacy, financial data, health information, etc.). Through the risk level preset by the tool, the tool inherent risk level is quickly queried and determined by means of the tool identifier in the user invocation request. In some embodiments, the tool inherent risk level can include low risk, medium-low risk, medium risk, medium-high risk, and high risk, represented by 0-10 points. For example:

[0068] Query tool: low risk (0-2 points);

[0069] Content creation tool: medium-low risk (2-4 points);

[0070] System configuration tool: medium risk (4-6 points);

[0071] Communication sending tool: medium-high risk (6-8 points);

[0072] Financial transaction tool: high risk (8-10 points).

[0073] The parameter risk score is obtained by analyzing the parameters in the user invocation request, analyzing the parameter type and value range, and calculating the parameter risk score. The risk of the tool itself may not be high, but the parameters passed in may increase its risk level. Therefore, the overall risk score of this user invocation request can be dynamically adjusted according to the actual content of the parameters. The analysis of the parameter content can include the analysis of the parameter type and value range, and of course the analysis of the parameters in combination with the tool and operation intent.

[0074] Among them, the parameter type and value range are analyzed, and the parameter risk score is calculated. The specific formula is as follows:

[0075]

[0076] Among them, S i represents the sensitivity coefficient of parameter i; V i represents the abnormality degree of the value of parameter i; I i represents the irreversibility coefficient of parameter i; w i , w v , and w I represent the weight coefficients.

[0077] The user permission factor is based on the user information in the user initiating the call request to obtain the user permission level, so as to adjust the risk score according to the user permission level. In some embodiments, the user can be set as an administrator user, a normal user, a visitor user, a new user, etc. Different user types set different risk coefficients, such as the user permission factor of the administrator user is 0.7, the user permission factor of the normal user is 1.0, the user permission factor of the visitor user is 1.3, and the user permission factor of the new user is 1.5.

[0078] The historical behavior analysis is based on the user information in the user initiating the call request to obtain the user past behavior pattern, and analyzes the risk of the current operation based on the historical behavior; the behavior reference threshold corresponding to the user can be established through the historical behavior, and the risk score is triggered to increase based on the behavior deviating from the reference threshold in the current operation. The specific formula is as follows:

[0079]

[0080] Wherein: C represents the consistency score of the current operation and the user historical pattern; C0 represents the consistency reference threshold; k represents the sensitivity parameter.

[0081] The environment factor represents the independent risk brought by the context environment in which the user initiates the call request. The time, location, device and other environment factors of the user initiating the call request can be considered. Specifically, it can be judged whether the time of initiating the call request is in the regular time, if it is in the regular operation time, the risk value is unchanged, if it is in the irregular operation time, the risk value is increased; it can be judged whether the location of initiating the call request is in the regular geographical position, if it is in the regular geographical position, the risk value is unchanged, if it is in the irregular geographical position, the risk value is increased; it can be judged whether the device of initiating the call request is in the regular device, if it is the regular device, the risk value is unchanged, if it is the irregular device, the risk value is increased. In some embodiments, the risk value of the irregular operation time is 1.5; the risk value of the irregular geographical position is 2; the risk value of the irregular device is 1.8.

[0082] In step S30, the calculation formula of the confirmation strategy L in the step of generating the confirmation strategy based on the risk score, the user preference adjustment factor and the urgency adjustment factor is as follows:

[0083]

[0084] Wherein, R represents the risk score; P a represents the user preference adjustment factor; U f represents the urgency adjustment factor; represents the floor function.

[0085] The aforementioned user preference adjustment factor is used to record users' reactions and preferences regarding confirmation methods. The user preference adjustment factor is calculated as follows:

[0086]

[0087] Among them, R i It is the user's response rating to the i-th confirmation method; w i It represents the time decay weight, with recent reactions receiving a higher weight; m is the number of historical records.

[0088] The aforementioned urgency adjustment factor determines whether a situation is urgent by assessing the current environment, context, and time. The specific calculation method is as follows:

[0089] U f =α·T c +β·S u +γ·C f

[0090] Among them, T c S represents the time constraint factor; u This represents the urgency signal extracted from speech features; C f α represents the context urgency factor; α, β, and γ represent the weighting coefficients.

[0091] Step S40, the confirmation process based on the confirmation strategy and the corresponding confirmation level, including the execution of the tool call, further includes canceling, executing, and correcting parameters based on user response. The confirmation process prompts the user with the content of the call request to confirm whether to execute the operation and whether the parameters are correct. It enables collaborative work based on real-time user feedback, providing interactive and correctable intelligent decision-making capabilities.

[0092] The confirmation process includes direct execution, implicit confirmation, explicit confirmation, or multi-stage confirmation. Wherein:

[0093] See Figure 2 As shown, the above direct execution process is suitable for low-risk operations, requiring no confirmation before execution, and can optionally provide post-execution notification.

[0094] See Figure 3 As shown, the implicit decision-making process described above is suitable for low- to medium-risk operations. It generates a restatement of the operation intent, confirms it through the user's default acceptance or corrective feedback, analyzes the user's response, and decides whether to execute. This includes the following steps:

[0095] Generate a restatement of the operation intent based on the user's request and display it to the user;

[0096] Receiving user response, the response includes cancel operation, execute operation, parameter modification;

[0097] If the user response is parameter modification, repeat the above steps.

[0098] Referring to Figure 4 The above explicit confirmation process is applicable to medium-risk operations, generates an explicit confirmation request, requires the user to provide an explicit positive response, and supports parameter-level modification response processing. It includes the following steps:

[0099] Generating an explicit confirmation request according to the user call request;

[0100] Requesting user confirmation;

[0101] Receiving user response, the response includes cancel operation, execute operation, parameter modification;

[0102] If the user response is parameter modification, repeat the above steps.

[0103] Referring to Figure 5 The above multi-stage confirmation process is applicable to high-risk operations, and the multi-stage confirmation process can be used for different security concerns at each stage, supporting parameter modification and risk reevaluation. It includes the following steps:

[0104] Generating an intent confirmation according to the user call request;

[0105] User response to intent confirmation; if yes, proceed to parameter confirmation, otherwise cancel operation;

[0106] User response to parameter confirmation; if yes, proceed to security verification, otherwise modify or cancel parameters;

[0107] User response to security verification; if passed, execute operation, otherwise verification failed, cancel operation.

[0108] In step S50, the execution controller undertakes the important security gatekeeping and execution management responsibilities. Considering that large models may produce "hallucinations" (generate false information), the execution controller implements a strict hallucination control mechanism, and for critical decisions, relies on verification data rather than model speculation. Specifically, a model participation quantification framework is established to dynamically and finely control the "speaking rights" of large models in the decision-making process. It is not simply handing over the task completely to the model or completely to the rules, but intelligently allocating the weights of the two according to the risk, historical performance and current scene. The implementation of the execution controller is as follows:

[0109] P i = φ (R, T i , H, C) ;

[0110] Where P i∈ [0, 1] represents the participation weight of the large model in the decision link i; R represents the risk score; T i is the decision link type; H represents the historical performance matrix, which records the historical accuracy of the model in different links; C represents the current context feature vector.

[0111] Even if all the previous links have passed, the execution process itself may have unexpected accidents. This mechanism ensures that the execution does not deviate from the track and can respond to emergencies. To this end, the controller execution tool adopts a "prediction-observation-adjustment" cycle mode for monitoring, continuously comparing the state changes in the execution process with the expected intention, and when detecting deviation of the state changes in the execution process from the expected intention, performing operations including pausing execution, requesting user confirmation, or automatically adjusting the execution path.

[0112] The present application also provides a tool calling system based on risk assessment, which comprises:

[0113] a risk assessment module: for receiving user calling requests and comprehensively analyzing tool types, parameter values, user permissions, historical use records and current environmental factors to generate a risk score;

[0114] a confirmation strategy generator for generating a confirmation strategy based on the risk score, user preference adjustment factor and urgency adjustment factor, the confirmation strategy including confirmation level, confirmation method and confirmation content;

[0115] a multi-level confirmation process manager: for executing the confirmation process corresponding to the confirmation level, the confirmation process including direct execution process, implicit confirmation process, explicit confirmation process or multi-stage confirmation process;

[0116] a user feedback analyzer: for situational response understanding and intention recognition and analyzing user responses to the confirmation process, judging the effectiveness of the confirmation process and processing parameter corrections in the confirmation process;

[0117] an execution controller: for controlling the final execution of the tool, including execution permission control and execution process monitoring.

[0118] The user feedback analyzer uses the natural language understanding mechanism of the large model to accurately understand and process the natural language responses of the user to the confirmation request. The implementation of this module mainly includes the following links:

[0119] (1) Situational response understanding and intention recognition

[0120] User feedback analysis makes full use of the natural language understanding mechanism of large models, combined with multi-model feedback fusion (in addition to text responses, the system can also fuse multi-modal signals such as voice emotion features (such as hesitation, firmness), facial expressions (such as nodding, shaking) and other multi-modal signals), and the response is analyzed in the complete interaction context. In addition, based on the dialogue history, confirmation request content, user preference model, etc., a complete context representation is constructed to more accurately determine the user's true intention.

[0121] (ii) Parameter modification intention extraction

[0122] In the tool invocation scenario, users often want to modify parameters in the confirmation stage. Parameter modification intention extraction in the large model environment mainly includes:

[0123] Semantic slot filling enhancement: The system converts predefined tool parameters into semantic slots, and uses the semantic understanding ability of the large model to accurately extract parameter modification intentions from user unstructured natural language. For example, from "not 26 degrees, change to 24 degrees" to extract the modified value of the temperature parameter.

[0124] Implicit parameter inference: The large model can understand the context and common sense, and infer the user's implicit parameter modification that is not explicitly expressed. For example, when the user says "the temperature is too high", the system can infer a reasonable temperature reduction based on the current setting value and the user's historical preferences.

[0125] Fuzzy expression quantization: The large model can convert the user's fuzzy quantitative expression into precise parameter values. For example, converting "a little cooler" to a specific temperature reduction value, or converting "a little earlier" to a specific time adjustment.

[0126] Finally, the confidence evaluation of parameter extraction uses a multi-dimensional scoring mechanism, considering factors such as semantic matching degree, context consistency, common sense rationality, etc. For low-confidence extraction results, the system will take appropriate risk mitigation strategies, such as explicit confirmation or providing a default safe value.

[0127] In order to further understand the large model tool invocation method based on risk assessment described in the present application, the following provides a plurality of embodiments.

[0128] Example 1: Smart home control scenario

[0129] Use case: User controls smart home devices through the system

[0130] Implementation process:

[0131] User voice command: "Set the temperature in the living room to 26 degrees"

[0132] Risk assessment:

[0133] Tool type: Environment control (basic risk: 3.5 points)

[0134] Parameter Assessment: Temperature parameter within normal range (Parameter Risk: 1.5 points)

[0135] User Type: Family Member (User Factor: 1.0)

[0136] Historical Behavior: Aligns with user's historical usage patterns (Historical Factor: 0.9)

[0137] Environmental Factors: Normal usage period and location (Environmental Risk: 0 points)

[0138] Calculated Risk Score: 4.5 points (Moderate Risk)

[0139] Confirm Strategy Generation:

[0140] Select Explicit Confirmation Strategy based on Risk Score

[0141] User Preference Analysis indicates user prefers concise confirmation

[0142] No current emergency

[0143] Execute Confirmation Process:

[0144] System generates confirmation prompt: "You want to set the living room temperature to 26 degrees. Confirm?" User responds: "Yes"

[0145] System executes temperature adjustment command

[0146] Post-execution feedback: "Living room temperature has been set to 26 degrees."

[0147] Example Two: Financial Transfer Scenario

[0148] Use Case: User initiates bank transfer through voice assistant

[0149] Implementation Process:

[0150] User voice command: "Transfer 5000 yuan to Zhang San"

[0151] Risk Assessment:

[0152] Tool Type: Financial Transfer (Base Risk: 9 points)

[0153] Parameter Assessment: Large transfer amount (Parameter Risk: 4 points)

[0154] User Type: Verified user (User Factor: 1.0)

[0155] Historical Behavior: Unconventional transfer object (Historical Factor: 1.3)

[0156] Environmental Factors: Regular location (Environmental Risk: 0 points)

[0157] Risk score calculated: 8.7 (High risk)

[0158] Confirmation strategy generation:

[0159] Based on the risk score, a multi-stage confirmation strategy is selected, and a three-stage confirmation process is designed

[0160] Execution of the confirmation process:

[0161] Stage 1: Intention confirmation

[0162] System: "Do you want to transfer 5000 yuan to Zhang San?"

[0163] User: "Yes"

[0164] Stage 2: Key parameter confirmation

[0165] System: "Please confirm that the payee is Zhang San (phone tail number 1234)"

[0166] User: "Confirm"

[0167] System: "The transfer amount is 5000 yuan, please confirm"

[0168] User: "That's right"

[0169] (3) Stage 3: Security verification

[0170] System: "For account security, please provide the SMS verification code"

[0171] User provides verification code

[0172] System verification passed

[0173] Perform the transfer operation

[0174] Provide detailed execution results: "Successfully transferred 5000 yuan to Zhang San (phone tail number 1234), transaction number TX123456".

[0175] Finally, it should be noted that although the above embodiments have been described in the specification and drawings of the present application, it does not limit the patent protection scope of the present application. Any equivalent structure or equivalent process substitution or modification based on the essential concept of the present application, using the content described in the specification and drawings, and directly or indirectly implementing the technical solutions of the above embodiments in other related technical fields, etc. are included in the patent protection scope of the present application.

Claims

1. A method for invoking a large model tool based on risk assessment, characterized in that, Includes the following steps: It receives natural language instructions from users and obtains user call requests based on a large model; Risk scores are generated based on user requests. A confirmation strategy is generated based on risk scores, user preference adjustment factors, and urgency adjustment factors. Execute the corresponding confirmation process based on the confirmation strategy; Based on the confirmation process, the controller executes the tool call.

2. A method for invoking a large model tool based on risk assessment, characterized in that, The user request includes tool identifier, operation intent, parameters, and user information.

3. The method for invoking large model tools based on risk assessment according to claim 1, characterized in that, The step of generating a risk score based on a user's request includes the following steps: Based on user request data, we obtain the tool's inherent risk level, parameter risk score, user permission factors, historical behavior analysis, and environmental factors. Risk scoring is performed using a multi-factor comprehensive scoring model based on the tool's inherent risk level, parameter risk score, user permission factors, historical behavior analysis, and environmental factors. The formula for calculating the risk score R of the multi-factor comprehensive scoring model is as follows: R=min([(R b ·w b +R p ·w p )·F u ·F h +F e ],10) Among them, R b Indicates the inherent risk level of the instrument; R p F represents the parameter risk score; u Indicates user permission factors; F h Indicates historical behavior analysis; F e Indicates environmental factors.

4. The method for invoking large model tools based on risk assessment according to claim 1, characterized in that, The calculation formula for the confirmation strategy L is as follows: Where R represents the risk score; P a Indicates the user preference adjustment factor; U f Indicates the urgency adjustment factor; This represents the floor function.

5. The method for invoking large model tools based on risk assessment according to claim 1, characterized in that, The confirmation process based on the confirmation strategy is executed according to the corresponding confirmation level. In the step of calling the execution tool, the execution of the corresponding confirmation process also includes canceling, executing, and correcting parameters based on the user's response.

6. The method for invoking large model tools based on risk assessment according to claim 1, characterized in that, The confirmation process includes a direct execution process, an implicit confirmation process, an explicit confirmation process, or a multi-stage confirmation process.

7. The method for invoking large model tools based on risk assessment according to claim 1, characterized in that, The parameter correction is based on extracting the parameter modification intent from the user's unstructured natural language using a large model.

8. The method for invoking large model tools based on risk assessment according to claim 1, characterized in that, The implementation method of the execution controller is as follows: P i =φ(R,T i ,H,C); Among them, P i ∈[0,1] represents the participation weight of the large model in decision-making stage i; R represents the risk score; T i H represents the decision-making stage type; H represents the historical performance matrix, which records the model's historical accuracy at different stages; C represents the current context feature vector.

9. The method for invoking large model tools based on risk assessment according to claim 1, characterized in that, The controller execution tool uses a "predict-observe-adjust" loop mode for monitoring, continuously comparing the state changes during the execution process with the expected intent. When a deviation between the state changes during the execution process and the expected intent is detected, actions such as pausing execution, requesting user confirmation, or automatically adjusting the execution path are taken.

10. A tool invocation system based on risk assessment, characterized in that, include: Risk assessment module: Used to receive user requests and generate risk scores by comprehensively analyzing tool type, parameter values, user permissions, historical usage records and current environmental factors; A confirmation strategy generator is used to generate confirmation strategies based on risk scores, user preference adjustment factors, and urgency adjustment factors. The confirmation strategy includes confirmation level, confirmation method, and confirmation content. Multi-level confirmation process manager: used to execute confirmation processes corresponding to the confirmation level, including direct execution processes, implicit confirmation processes, explicit confirmation processes, or multi-stage confirmation processes; User feedback analyzer: used for contextual response understanding and intent recognition, as well as parsing user responses to the confirmation process, determining the validity of the confirmation process, and handling parameter corrections in the confirmation process; Execution controller: Used to control the final execution of the tool, including execution permission control and execution process monitoring.

Images