Interaction intention driven tool scheduling method, device and equipment and medium

Abstract

The invention relates to the technical field of artificial intelligence, can be applied to business scenes such as financial science and technology and medical health, and discloses an interaction intention-driven tool scheduling method, device and equipment and a medium. The method comprises the steps of obtaining to-be-processed interaction data of visual data or text data, executing cross-modal semantic analysis to generate a unified interaction intention, and sending the unified interaction intention to a server; detecting intention association strength between the matching tool and the context state data, selecting a context association mode based on the strength, determining a target dialogue node, scheduling the matching tool to perform sandbox execution to obtain a tool execution result, generating response content, and sending the response content to the server; and selectively updating the to-be-processed interaction data, the unified interaction intention, the response content, the tool execution result, the target dialogue node and the extracted service entity data to context state data. According to the method, the interaction accuracy is improved through cross-modal analysis and a switchable context association mechanism, tool decoupling is achieved in a sandbox execution mode, and the system adaptability and stability are enhanced.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a tool scheduling method, apparatus, device and storage medium driven by interactive intent. Background Technology

[0002] In the evolution of AI interactive systems, the form of dialogue and interaction between users and systems has gradually shifted from static menu-based to dynamic semantic-driven. However, existing technologies still have significant shortcomings in multimodal understanding, context management, and tool scheduling, making it difficult to meet the comprehensive requirements for intelligent response efficiency and scalability in high-concurrency business environments.

[0003] In the fintech business sector, existing conversational systems still largely rely on text input for intent recognition and business guidance, lacking the ability to process unstructured visual data such as images, screenshots of invoices, and scanned copies of contracts. This forces users to supplement information through lengthy text descriptions or by redirecting to external upload portals, increasing the interaction burden. Furthermore, traditional interaction processes are primarily form-based or flowchart-driven, with fixed interaction paths requiring users to click through each step, which is particularly inefficient in complex business scenarios (such as insurance claims or credit intention assessments). In addition, traditional dialogue engines rely on fixed state machine structures to establish contextual dependencies, lacking the flexibility to respond to intent shifts or topic jumps. Especially when user input contains complex intents or semantic breaks, the system attempts to maintain the old context, leading to misleading responses and impacting user experience and business accuracy.

[0004] In the healthcare field, intelligent consultation systems and health assessment platforms often rely on dialogue to complete tasks such as describing symptoms, assisting in diagnosis, or recommending services. However, current systems are mostly based on single-turn dialogues, lacking the ability to maintain information across turns and dynamically adjust the dialogue state. For example, when a patient uploads medical images or screenshots of examination reports, the system cannot combine image information to help generate an accurate intent, still relying on the patient's self-description of key information, which can easily lead to misunderstandings. Furthermore, due to the rigidity of the state management mechanism, even after the patient adds new information, the system continues to reason along the old path, resulting in incorrect recommendations or suggestions, which may seriously affect diagnostic judgment. Experimental data shows that in more than five consecutive rounds of interaction, the probability of intent deviation increases significantly without a dynamic state control mechanism. Summary of the Invention

[0005] The main objective of this invention is to provide an interactive intent-driven tool scheduling method, apparatus, device, and storage medium, aiming to solve the technical problems of the lack of a unified interactive process that can simultaneously support visual and textual multimodal semantic parsing, dynamic context association management, and scalable tool scheduling mechanism, resulting in inaccurate understanding of user intent, inflexible switching of dialogue states, high coupling of tool calls, and poor scalability.

[0006] To achieve the above objectives, the present invention provides an interactive intent-driven tool scheduling method, comprising:

[0007] Acquire interactive data to be processed, which includes visual or textual data;

[0008] Cross-modal semantic parsing is performed on the interaction data to be processed to generate a unified interaction intent;

[0009] Detect the strength of intent association between the unified interaction intent and the context state data;

[0010] Based on the intent association strength, the context association mode is dynamically selected, and the target dialogue node is determined based on the context association mode and the unified interaction intent.

[0011] Based on the tool identifier of the target dialogue node, the executable tools in the scheduling tool registry that match the tool identifier are executed in a sandbox to obtain the tool execution result;

[0012] Based on the tool execution result, a response content is generated, and according to the context association mode, the pending interaction data of the current interaction, the unified interaction intent, the response content, the tool execution result, the target dialogue node, and the business entity data extracted during the tool call are selectively updated to the context state data.

[0013] Furthermore, to achieve the above objectives, the present invention provides an interactive intent-driven tool scheduling device, comprising:

[0014] The multimodal input processing module is used to acquire interactive data to be processed, which includes visual or text data.

[0015] The cross-modal semantic parsing module is used to perform cross-modal semantic parsing on the interaction data to be processed, and generate a unified interaction intent;

[0016] The intent association analysis module is used to detect the intent association strength between the unified interaction intent and the context state data;

[0017] The dialogue decision engine module is used to dynamically select the context association mode based on the intent association strength, and determine the target dialogue node based on the context association mode and the unified interaction intent.

[0018] The tool scheduling and execution module is used to schedule executable tools in the tool registry that match the tool identifier of the target dialogue node for sandboxed execution, and obtain the tool execution result.

[0019] The context state management module is used to generate response content based on the tool execution result, and selectively update the context state data with the pending interaction data of the current interaction, the unified interaction intent, the response content, the tool execution result, the target dialogue node, and the business entity data extracted during the tool call, according to the context association mode.

[0020] Furthermore, to achieve the above objectives, the present invention also provides a computer device, the computer device including a memory, a processor, and an interactive intent-driven tool scheduler stored in the memory and executable on the processor, wherein when the interactive intent-driven tool scheduler is executed by the processor, it implements the steps of the interactive intent-driven tool scheduling method as described above.

[0021] Furthermore, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing an interactive intent-driven tool scheduler, wherein the interactive intent-driven tool scheduler, when executed by a processor, implements the steps of the interactive intent-driven tool scheduling method as described above.

[0022] Beneficial Effects: This invention relates to the field of artificial intelligence technology and can be applied to business scenarios such as fintech and healthcare. It discloses an interactive intent-driven tool scheduling method, apparatus, device, and medium, comprising: acquiring interactive data to be processed containing visual or textual data; performing cross-modal semantic parsing to generate a unified interactive intent; detecting the intent association strength between the interactive intent and context state data; dynamically selecting a context association mode based on the association strength; determining a target dialogue node by combining the mode and the interactive intent; scheduling a matching executable tool to perform sandboxed operations to generate a tool execution result; generating response content based on the execution result; and selectively updating the interactive data to be processed, the unified interactive intent, the response content, the tool execution result, the target dialogue node, and the business entity data extracted during the tool invocation process to the context state data. This invention, by integrating multimodal semantic parsing and dynamic context association mechanisms, can achieve unified intent recognition of visual and textual information and flexibly switch context modes by combining real-time state judgment, thereby ensuring context relevance and interruptibility during the dialogue process; simultaneously, by improving the decoupling and scalability of tool invocation through a sandboxed tool scheduling method based on tool identifiers, it achieves rapid response and dialogue-driven process adaptation in complex business scenarios. Attached Figure Description

[0023] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings:

[0024] Figure 1 This is a schematic diagram of an application environment for an interactive intent-driven tool scheduling method according to an embodiment of the present invention;

[0025] Figure 2 This is a flowchart illustrating an embodiment of the interactive intent-driven tool scheduling method of the present invention;

[0026] Figure 3 This is a schematic diagram of the functional modules of a preferred embodiment of the interactive intent-driven tool scheduling device of the present invention;

[0027] Figure 4 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention;

[0028] Figure 5 This is another structural schematic diagram of a computer device according to one embodiment of the present invention. Detailed Implementation

[0029] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0030] The interaction intent-driven tool scheduling method provided in this embodiment of the invention can be applied to, for example... Figure 1In this application environment, the user terminal communicates with the server via a network. The server can obtain interactive data to be processed, including visual or textual data, from the user terminal, perform cross-modal semantic parsing to generate a unified interactive intent, detect the intent association strength between the interactive intent and the context state data, dynamically select a context association mode based on the association strength, determine the target dialogue node by combining the mode with the interactive intent, schedule a matching executable tool to perform sandboxed operations to generate tool execution results, generate response content based on the execution results, and selectively update the context state data with the interactive data to be processed, the unified interactive intent, the response content, the tool execution results, the target dialogue node, and the business entity data extracted during the tool invocation process. This invention, by integrating multimodal semantic parsing and dynamic context association mechanisms, can achieve unified intent recognition of visual and textual information, and flexibly switch context modes by combining real-time state judgment, thereby ensuring context relevance and interruptibility during the dialogue process; at the same time, the sandboxed tool scheduling method based on tool identifiers improves the decoupling and scalability of tool invocation, realizing rapid response and dialogue-driven process adaptation in complex business scenarios. The user terminal can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers. The invention will now be described in detail through specific embodiments.

[0031] Please see Figure 2 , Figure 2 This is a flowchart illustrating an embodiment of the interactive intent-driven tool scheduling method provided by the present invention. It should be noted that although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0032] like Figure 2 As shown, the interactive intent-driven tool scheduling method proposed in this invention includes the following steps:

[0033] S10, acquire interactive data to be processed, which includes visual data or text data;

[0034] In this embodiment, during the process of receiving user-side interactive input, it is first necessary to collect raw data from the user terminal through an input channel that supports multimodal data structures. This raw data may include image information, text input, or other types of expressive signals. Visual data is usually represented as a pixel-level matrix structure, and its sources may include camera capture results, uploaded static images, image frames, etc. Its essence is two-dimensional array data with a spatial coordinate system, containing color channel information such as RGB or YUV. Text data includes directly typed string information, natural language text transcribed by a speech recognition module, etc., and its structure is usually a character sequence in UTF-8 or GBK encoding format. To facilitate subsequent unified semantic analysis, the raw data must be standardized. For visual data, image decoding is first required to restore compressed formats such as JPEG and PNG to the original pixel matrix. This process is implemented using visual libraries such as OpenCV and Pillow. Subsequently, resolution standardization is performed to ensure that the image size participates in the analysis in a unified dimension, generally using a fixed scaling on the long side to avoid content distortion. After standardization, color space conversion is performed, transforming RGB into Lab, HSV, and other color space formats that enhance contrast perception, facilitating structural recognition in subsequent image semantic extraction modules. For text data, the input encoding format must first be detected by checking the byte order of the first character or using tools such as chardet to determine the encoding, and then uniformly converting it to UTF-8 encoding format. Subsequently, sensitive personal information, illegal keywords, or noisy segments in the plaintext need to be removed or replaced. This process can be completed based on regular expression matching or entity recognition algorithms, ultimately retaining high-quality semantic text as the unified input format. Whether it is visual data or text data, after standardization, it will be encapsulated into a unified structure format. This structure contains original source information, data type identifiers, timestamps, and sequence identifiers, facilitating cross-module scheduling and use.

[0035] User-end data acquisition and management can be achieved by integrating multi-channel input devices. Visual data channels include camera interfaces, image upload components, or multimedia data parsing interfaces; text data channels include keyboard input boxes, speech recognition backend output modules, etc. A distributed image enhancement engine can also be introduced during standardization processing to perform image decoding, color correction, and resolution reconstruction. Entity recognition models can be configured to enhance the efficiency of identifying sensitive items in text and support asynchronous filtering or annotation. Edge-side data downsampling strategies or compression channels can be configured to ensure real-time data accessibility, taking into account the uplink and downlink bandwidth of different input sources. When encapsulating interactive data to be processed, data source fields or confidence score fields can be introduced for priority ranking or anomaly identification in subsequent processing stages.

[0036] Example: In healthcare services, medical image files uploaded by users (such as X-rays and CT images) can be decoded and standardized, and then combined with text descriptions of symptoms to form structured interactive input, providing mixed text and image support for intelligent consultation systems.

[0037] In the financial sector, the ID photos and identity verification text information uploaded by users through mobile banking are uniformly packaged and processed by the system to support subsequent risk control verification, compliance review and other tasks, ensuring an accurate and reliable processing foundation in anti-fraud, anti-money laundering and other scenarios.

[0038] This embodiment effectively integrates multi-source inputs such as images and text by constructing a unified format of interactive data to be processed, enhancing cross-modal semantic alignment capabilities and improving the accuracy and adaptability of semantic parsing. Simultaneously, the standardized processing mechanism reduces the probability of parsing errors caused by differences in input formats, ensuring the generalization stability of the subsequent semantic reasoning module across multiple scenarios. The unified encapsulation of the data structure also provides a highly available and consistent input interface for the tool scheduling module.

[0039] S20, perform cross-modal semantic parsing on the interaction data to be processed to generate a unified interaction intent;

[0040] In this embodiment, before semantic recognition, information types of different modalities must be parsed from the interaction data to be processed. This process involves the initial deconstruction of the input data. For visual data, the original image matrix needs to be extracted first, preserving its structural features, including edge contours, texture variations, and target contours. Convolutional feature encoding networks can be used to compress image signals into semantic representation vectors during image input data extraction. Text input data extraction relies on a language annotation module, which constructs a language feature space using methods such as word segmentation, part-of-speech tagging, and named entity recognition to extract semantic entities such as keywords, action descriptions, and subject-object relationships.

[0041] Text semantic vector generation typically employs pre-trained language models for feature encoding, such as BERT, RoBERTa, and ERNIE. The text input is segmented into a token sequence, mapped to a high-dimensional semantic vector through an embedding layer, and then passed through a multi-layer Transformer encoder structure to generate a context-aware representation. Image semantic vector generation can utilize deep neural networks such as ResNet and Vision Transformer to extract semantic components from multi-scale regions and perform global encoding.

[0042] To align the semantic spaces of images and text, a cross-modal alignment module is needed, often employing joint embedding space mapping or attention-based fusion strategies. Through alignment, semantic vectors from two different modalities are projected into a shared space, making them semantically comparable and fusionable. The semantic features generated by further fusion in the shared space can be used to construct a semantic label distribution, i.e., a probability prediction on a predefined label set. This semantic label distribution represents the system's range of inferences about the current interaction intent.

[0043] Finally, a unified interaction intent is determined based on the label with the highest confidence in the label distribution. This interaction intent is the result of cross-modal information synthesis, possessing scene adaptability, semantic coherence, and contextual consistency, providing input for subsequent dialogue guidance and tool scheduling.

[0044] A dual-tower model based on the Transformer architecture can be employed, extracting features separately for the image and text branches and completing semantic alignment in a shared embedding space. Alternatively, a cross-fusion network based on attention weight allocation can be used to create stronger coupling between text and images during the interaction phase. Furthermore, the semantic label set acquired during training can be expanded through label enhancement mechanisms, enabling the system to have stronger generalization capabilities across different business contexts such as medical consultations or financial customer service. When constructing the semantic label distribution, temperature scaling or confidence normalization mechanisms can be introduced to ensure the stability and interpretability of label selection.

[0045] Example Explanation: In healthcare applications, patients upload skin images and add text input describing "redness, swelling, and itching." The system identifies the color features and shape contours of the lesion area through the image channel and the keywords "redness" and "itching" through the text channel. After merging, a unified interactive intent of "skin allergy" is formed, which is then processed by the diagnostic engine.

[0046] In the financial sector, when a user uploads a photo of their bank card and enters "replace new card", the image path identifies the card number and expiration date area, and the text path identifies the intent keyword "replace", ultimately forming an interactive intent of "bank card replacement request" and triggering subsequent business processes.

[0047] This embodiment constructs a semantic parsing mechanism oriented towards multimodal input, enabling collaborative understanding of information such as images and text, thereby improving the accuracy and robustness of semantic parsing. The generation of unified interaction intents provides clear semantic anchors for subsequent context management and tool scheduling, effectively mitigating the ambiguity of expression and misjudgment of intent caused by heterogeneous information input, and enhancing the overall intelligence level of the interaction process.

[0048] S30, Detect the intensity of intent association between the unified interaction intent and the context state data;

[0049] In this embodiment, the correlation between the current interaction intent and the historical context is identified to support subsequent context mode switching and dialogue node selection. First, an information structure representing the historical interaction trajectory needs to be extracted from the context state data. The dialogue history stack is responsible for storing previous intents and response data. This structure is typically arranged in timestamp order and can contain data units of multiple dimensions, such as semantic tags, tool call records, and entity slot filling information.

[0050] Historical intent sequences are sets of interaction intent trajectories extracted from the dialogue history stack, representing the evolution path of a user's behavioral goals in consecutive rounds. To compare the similarity between the current unified interaction intent and the historical intent sequence, a semantic similarity evaluation module needs to be constructed. This module can quantify the semantic closeness between the current intent and the historical intent using methods such as vector cosine similarity, Euclidean distance, BERT-based embedding matching, and SimCSE embedding.

[0051] To differentiate the influence of older information, a time-order-based programmable decay function is introduced. This function applies time weights (such as a decreasing exponential function or a learned decay curve) to exert a lower influence on intention information from earlier rounds, ensuring that historical intentions closer to the current interaction time point have a higher contribution. The introduction of historical decay weights enhances the scheduling capabilities of the dialogue's short-term memory.

[0052] Furthermore, to prevent false associations, the system needs to detect whether the user has entered an explicit isolation instruction, such as semantic anchors like "change the question" or "start again." Explicit isolation instructions can trigger context chain breaking logic; upon detecting such instructions, the intent association strength is immediately set to zero to avoid incorrect referencing of past dialogues. If no explicit isolation is detected, the system uses the initial result, calculated by combining semantic similarity with historical decay weights, as the intent association strength to guide subsequent context strategy selection and dialogue node judgment.

[0053] A semantic similarity function can be modeled using multi-layered neural networks. The current intent vector is input into the matching model along with historical intent sequences, and an attention mechanism is used to focus on important historical trajectories. Alternatively, a trainable decay function can be introduced to dynamically adjust the historical contribution weights within different time windows. For isolated instruction detection, key phrase matching combined with language model confidence can determine the presence of a reset intent. A dialogue emotion recognition mechanism can also be introduced to assist in link break detection. All of the above structures can be flexibly configured at the parameter level to adapt to different interaction frequencies and industry needs.

[0054] Example Explanation: In healthcare, if a patient enters "Try another doctor" after five consecutive consultations, the system recognizes this as an explicit isolation instruction, reduces the intent association strength to zero, and restarts a new consultation dialogue context. If the input is "What should I do if the medication recommended last time didn't work?", then semantic similarity matching and time decay weighting are used to identify that this question is highly relevant to the historical treatment goals, maintaining the continuity of the dialogue and ensuring the accuracy of the recommendation.

[0055] In the financial sector, if a user submits messages such as "I want to check my account" or "How much was charged to my credit card last month?" consecutively, the system can extract their intent to check the bill and enhance historical conversation references. If a user suddenly enters "Complain to customer service", the system will automatically identify the disconnect and switch the conversation direction to the appeal process.

[0056] This embodiment introduces semantic similarity and historical decay mechanisms to dynamically adjust the correlation between the current intent and historical information, effectively solving the problems of excessive historical interference or delayed response due to broken links in long-term dialogues. Simultaneously, by combining isolation instruction detection, it enhances the system's sensitivity to changes in user intent, making multi-turn interactions more flexible and coherent, and effectively improving the intelligent scheduling capabilities of context management.

[0057] S40, dynamically select the context association mode according to the intent association strength, and determine the target dialogue node based on the context association mode and the unified interaction intent;

[0058] In this embodiment, the intent association strength obtained from the preceding analysis is transformed into the selection criterion for the context association strategy, thereby determining how the dialogue establishes a connection between the historical path and the current interaction. Context association patterns are a discriminative mechanism used to guide the system on whether to continue the current context state, and typically include at least two types: continuous association pattern and single-turn isolation pattern. The continuous association pattern is used to express that the current interaction semantically continues the historical context content, which is suitable for multi-turn continuous business; the single-turn isolation pattern indicates that the current interaction does not depend on existing historical information, and is suitable for new starting topics or context jumps.

[0059] The system first sets a threshold for the strength of association to distinguish whether to maintain contextual association. When the intent association strength is greater than or equal to the threshold, it indicates that the current intent has semantic consistency with the historical context, and the system sets the context association mode to continuous association mode; otherwise, it sets it to single-round isolation mode, thereby avoiding irrelevant historical content from interfering with the current judgment.

[0060] In continuous association mode, the system not only considers the current unified interaction intent itself, but also needs to combine the dialogue history stack in the context state data to determine the target dialogue node. The dialogue history stack provides topic trajectory and entity evolution information, which can be used to help determine the target business path that the user truly wants to reach. Conversely, in single-turn isolation mode, to avoid semantic pollution and contextual misleading, the system only performs rapid location of the target dialogue node based on the current unified interaction intent, skipping historical trajectory analysis.

[0061] A target dialogue node is a structural unit within the system used to receive user input and trigger tool calls. It typically includes fields such as node type, tool identifier, expected input slot, and subsequent path. This structure needs to be determined by both the context mode and the current intent to ensure that the response path conforms to the current user goal without causing state pollution.

[0062] Whether to enter a continuous association mode can be determined by a preset numerical threshold (e.g., 0.65), or a dynamic learning strategy can be used to adjust the threshold, such as updating the association threshold in real time based on the intent deviation statistics in a sliding window. In selecting the target dialogue node, a graph-based dialogue flowchart can be constructed, and the most suitable node can be determined jointly by a graph neural network or an attention-based dialogue router based on contextual history and current semantic matching. In isolation mode, a pure intent classification model can be constructed to directly map a unified interaction intent to the most matching target node in the node set.

[0063] Example Explanation: In healthcare services, if a user repeatedly submits questions like "How do I interpret yesterday's medical check-up report?" or "Which indicators are abnormal?", the system will maintain a continuous association mode based on a high intent-based association strength, and determine the current target node as a report interpretation service module based on historical dialogue nodes. If the user suddenly sends "I want to book an appointment with an ophthalmologist," the intent-based association strength drops sharply, and the system switches to a single-round isolation mode, directly locating the appointment booking node based on the new intent, preventing the ophthalmology appointment from being mistakenly associated with the medical check-up report process.

[0064] In the financial sector, after a user checks the risk level of a wealth management product, they can send a message "Recommend products suitable for conservative investment." The system will retain the user's risk preferences and the information of the products they have queried in the continuous association mode to assist in decision-making and recommendations. However, if a user enters "How do I apply for a credit card limit increase?", it will be considered a semantic jump, enter the isolation mode, and switch to the relevant credit card process nodes.

[0065] This embodiment introduces an intent-related strength-driven context mode switching mechanism to achieve flexible control of the dialogue context, effectively adapting to both continuous topic progression and non-continuous transition scenarios. By leveraging dynamic association modes and intent-driven dialogue node selection logic, the system maintains contextual coherence while also possessing reset capabilities, improving the accuracy of the interaction path and the user's intent reception, and reducing false responses and path drift issues.

[0066] S50, based on the tool identifier of the target dialogue node, the executable tool in the tool registry center that matches the tool identifier is scheduled for sandboxed execution to obtain the tool execution result;

[0067] In this embodiment, the tool identifier of the target dialogue node is a structured field, typically a unique string identifier or a named tag with a version number, used to explicitly point to a specific tool or interface entity in the business logic. The tool registry is a service module that centrally manages all tool metadata, recording information such as the identifier, execution path, input / output mode description, security label, and dependency version of each tool. This center can be implemented using a distributed registration service, such as one based on Zookeeper, Etcd, or a custom configuration center.

[0068] The system first sends a query request to the tool registry based on the tool identifier carried in the target dialogue node to obtain executable tool instances that match the identifier. The matching strategy not only requires that the tool identifiers be literally identical, but may also include a judgment on version compatibility to ensure consistency of the runtime environment. In actual implementation, the tool identifier usually also needs to carry environment tags (such as prod, test) in order to load tool entities under different deployment paths.

[0069] After obtaining the tool instance, the system also needs to retrieve the tool's input mode description, which specifies the acceptable input field types, format requirements, required field markers, and other parameter constraints. Subsequently, the system extracts the business entity slot content written in previous interactions from the context state data, and extracts structured input values ​​that meet the parameter requirements based on the input mode description, constructing a complete tool call parameter set.

[0070] To ensure execution security and isolation, the system uses a sandboxed environment to execute executable tools during runtime. This sandbox environment can be built using containers (such as Docker) or lightweight function runtime platforms (such as Serverless Runtime), restricting system permissions, memory usage, and network access capabilities to prevent tool execution from interfering with the main system. After completing its execution in the sandbox, the tool returns the raw execution result, which may be in JSON, XML, or binary data format.

[0071] To enhance unified processing capabilities, the system will perform multi-protocol format conversion upon receiving the raw execution results. The conversion process requires parsing and standardizing the raw results into a unified data structure, such as a standardized JSON structure, based on the target dialogue node or the result protocol type definition preset by the tool registry, for use as the tool execution result output in subsequent steps.

[0072] Information from the tool registry can be dynamically loaded via configuration files or API interfaces. The tool registry's data structure can use a key-value mapping, where the key is the tool identifier and the value is the tool's execution path and input / output rules. Sandbox execution can utilize user-space kernels like gVisor to build secure execution containers, or it can achieve platform-independent tool isolation based on WebAssemblyruntime. Format conversion can introduce schema mapping templates, supporting conversion from various heterogeneous protocols (such as SOAP and gRPC) to JSON or Graph structures for easy unified integration.

[0073] Example Explanation: In a healthcare business scenario, when a user requests "generate a chart analyzing a recent physical examination report," the tool identifier corresponding to the target dialogue node points to a chart generation service tool. The system loads the tool's execution path and parameter mode from the tool registry, extracts the physical examination indicator data from the context as input, runs the tool in a sandbox, generates a visual chart JSON result, performs format conversion, and returns it to the front end for display.

[0074] In the financial sector, when a user requests to "calculate monthly loan payments," the system identifies the corresponding tool identifier, locates the loan calculation tool in the registry center, and extracts the loan amount, interest rate, and period from the context as parameters. It then executes the calculation in a sandbox environment and returns a detailed monthly payment structure. Through this mechanism, the system can dynamically adapt to service tools with complex parameter dependencies in different fields such as healthcare and finance, achieving secure, flexible, and rapid response and task scheduling in high-concurrency scenarios.

[0075] This embodiment introduces a dynamic tool scheduling mechanism based on tool identifiers to decouple tool calls and flexibly bind service components during interaction. The sandboxed execution environment ensures call security and provides fault isolation capabilities. Combined with multi-protocol format conversion processing, it enhances the system's ability to adapt to heterogeneous tools and its standardized output capabilities. Overall, it improves the stability, scalability, and platform compatibility of tool scheduling, reducing the risk of service failures and the cost of interface reconstruction.

[0076] S60, generate response content based on the tool execution result, and selectively update the context state data with the pending interaction data of the current interaction, the unified interaction intent, the response content, the tool execution result, the target dialogue node, and the business entity data extracted during the tool call, according to the context association mode.

[0077] In this embodiment, the tool execution result refers to the structured output data returned after the executable tool runs and completes parameter processing in the sandbox environment. This result is typically presented in JSON or a similar parsable format and may include fields such as numerical values, charts, suggested statements, or status codes. Generating response content refers to the system constructing a user-readable, user-friendly, and semantically complete structured response data structure based on the execution result for subsequent rendering and display or task distribution. The construction process requires embedding the tool's return value into the response framework according to the response template or preset layout logic defined by the target dialogue node. Simultaneously, necessary content is processed through entity name replacement, field format conversion, and tag wrapping to ensure that the user receives standardized response information.

[0078] The context association mode is a dynamically determined operating mode, which can be a continuous association mode or a single-round isolation mode, determining the current interaction's strategy for writing context state. After identifying the current association mode, the system updates several elements related to this round of interaction to the context state data as needed. The interaction data to be processed is the standardized input received at the start of this round of interaction. The unified interaction intent is a tag semantic vector or structured intent tag generated through semantic fusion. The response content is the aforementioned generated output text or result component. The tool execution result is the standard output structure after sandbox operation.

[0079] The target dialogue node is the business node selected by the system in the current interaction. Its metadata includes the node name, role settings, tool pointing identifier, and response template structure, which should be included in the history for easy dialogue recovery. Business entity data is information obtained or supplemented from the context state during tool execution or input parameter extraction, such as user identity, time range, amount, or disease label. These fields have significant entity attributes and are usually reused or enhanced in multiple interaction rounds.

[0080] When the current context association mode is persistent association mode, the system encapsulates the above multiple field structures into an interaction event record and adds it to the dialogue history stack in the context state to achieve multi-turn tracing and continuous response. In single-turn isolation mode, only the business entity data extracted during the tool call is updated to the business entity slot, avoiding unnecessary interference to the global context and ensuring state cleanliness and task independence.

[0081] The logic for filling in response content can be implemented using template engines such as Handlebars and Freemarker. The tool execution results can be embedded into the response component using predefined structures and placeholders. Context state data can adopt a key-value state management structure, such as maintaining multiple stacked dialogue records based on a Redis hash structure. Each record contains interaction data, intent tags, response summaries, and target node information. Update strategies are controlled by a strategy selector module, dynamically loading the write path and field selection set based on the context association mode. In continuous association mode, the dialogue history stack is constructed using a timestamp-ordered structure, supporting backtracking and precise location. In single-round isolation mode, only overwriting or supplementing business entity slots is performed, maintaining a short-term, lightweight operation.

[0082] Example Explanation: In a healthcare scenario, when a user requests a diagnostic suggestion from the system by uploading pathological images and symptom descriptions, the system obtains the tool execution results as a diagnostic suggestion and risk score. Based on these results, a response content structure is generated and displayed, while the user-uploaded data, the identified diagnostic intent, the system suggestion, and image tags are written into the context state. In continuous association mode, this state can be further supplemented with medical history and medication-related questions in subsequent inquiries.

[0083] In financial transactions, users submit loan parameters to calculate monthly payments. The tool then outputs the monthly repayment amount and total interest. The system pushes the response to the front end and selectively stores the user input and calculation results in business entity slots for future repayment simulations or credit limit adjustments. By differentiating modes to control content updates, the system achieves flexible organization of intelligent dialogue processes and effective state memory.

[0084] This embodiment decouples and dynamically configures the unified response generation mechanism and context state writing strategy, enabling the structured output generated in multimodal interactions to be flexibly organized and evolve according to dialogue needs. In continuous dialogue scenarios, it preserves the complete dialogue trajectory, enhancing multi-turn reasoning and personalized recommendation capabilities; in short-time task scenarios, it avoids redundant writing and historical pollution, optimizing execution efficiency and context purity. This processing mechanism improves the system's interactive memory management capabilities, state tracking capabilities, and multi-turn intelligent planning capabilities.

[0085] This invention relates to the field of artificial intelligence technology and can be applied to business scenarios such as fintech and healthcare. It discloses an interactive intent-driven tool scheduling method, apparatus, device, and medium, comprising: acquiring interactive data to be processed containing visual or textual data; performing cross-modal semantic parsing to generate a unified interactive intent; detecting the intent association strength between the interactive intent and context state data; dynamically selecting a context association mode based on the association strength; determining a target dialogue node by combining the mode and the interactive intent; scheduling a matching executable tool to perform sandboxed operations to generate a tool execution result; generating response content based on the execution result; and selectively updating the interactive data to be processed, the unified interactive intent, the response content, the tool execution result, the target dialogue node, and business entity data extracted during the tool invocation process to the context state data. This invention, by integrating multimodal semantic parsing and dynamic context association mechanisms, can achieve unified intent recognition of visual and textual information and flexibly switch context modes by combining real-time state judgment, thereby ensuring context relevance and interruptibility during the dialogue process; simultaneously, by improving the decoupling and scalability of tool invocation through a sandboxed tool scheduling method based on tool identifiers, it achieves rapid response and dialogue-driven process adaptation in complex business scenarios.

[0086] In one embodiment, step S10 above includes:

[0087] S101, receives raw interactive data submitted by the user terminal through the multi-channel input interface;

[0088] S102, the data type of the original interaction data is identified through the metadata analysis engine;

[0089] S103, when the data type is a visual data type, decode the original interactive data to obtain the original pixel matrix, perform resolution standardization processing on the original pixel matrix to obtain a standard resolution matrix, and perform color space conversion on the standard resolution matrix to generate standardized visual data.

[0090] S104, when the data type is a text data type, detect the character encoding format of the original interactive data, convert the character encoding format into a unified encoding format to obtain intermediate text data, and perform sensitive information filtering processing on the intermediate text data to generate standardized text data.

[0091] S105, the standardized visual data or standardized text data is encapsulated into interactive data to be processed.

[0092] In this embodiment, the acquisition of raw interactive data is accomplished through a multi-channel input interface. This interface includes not only traditional text input channels, such as keyboard input and mobile speech-to-text, but also visual channels such as image upload, video frame capture, and file import, enabling parallel access to multiple input paths. In the design and implementation, each input channel needs to register a unique identifier and has a corresponding data capture module and data monitoring mechanism to support asynchronous submission and front-end event-driven upload requests. After receiving the interactive data, the system uses a metadata analysis engine to quickly parse and extract the structure of the data stream. The metadata analysis engine can perform multi-dimensional discrimination on the input data based on indicators such as MIME type, file header binary signature, content distribution characteristics, character set identifier, or image metadata (such as Exif), determining whether it is an image, text, or other type, and executing different processing paths based on these criteria.

[0093] When the data type is visual, the visual parsing module is first invoked to decode the image or video frame data, obtaining the raw pixel matrix. This pixel matrix may have multiple encoding formats (such as JPEG, PNG, BMP), and the appropriate decoder needs to be automatically selected based on the metadata. The obtained raw pixel matrix may have significant differences in size, resolution, and number of channels, so resolution normalization processing needs to be performed. Typically, bilinear interpolation or region resampling methods are used to adjust the image to the standard input size supported by the system (such as 224×224 or 512×512) to ensure the consistency of subsequent model inputs and computational stability. To adapt to the color input format requirements of different depth models, the normalized image also needs to be converted from the original color space (such as RGB, YUV) to a unified color space, generally to linear RGB or grayscale image space, to improve the semantic model's sensitivity to edges, textures, and brightness in the visual input. Finally, standardized visual data with a regular structure is obtained. This data can be encapsulated as a tensor object or matrix array for subsequent feature extraction or cross-modal fusion processing.

[0094] When the data type is text, the character encoding format needs to be identified first. This step addresses the issue of mixed character sets from multiple languages ​​in user input data, such as UTF-8, GB2312, and ISO-8859-1. Encoding identification can be achieved through character distribution probability, BOM header detection, and encoded byte length analysis. A unified encoding conversion operation is then performed, typically setting the target format to UTF-8 to ensure cross-platform compatibility. The converted intermediate text data may still contain redundant characters, special symbols, or user privacy data; therefore, a sensitive information filtering module is needed for structural purification. This filtering module integrates a rule-based matcher (such as regular expressions) with a context-based neural model to jointly identify sensitive fields such as phone numbers, ID numbers, financial terms from a keyword database, and medical information, and performs masking or deletion to ensure semantic integrity while also considering data security and compliance. The final output is standardized text data with a unified structure, clean semantics, and suitable for further semantic modeling.

[0095] The standardized visual and text data generated by the above processing paths need to be uniformly encapsulated. The encapsulation structure needs to define data type tags, original data sources, timestamp information, and reference indexes for the encapsulated content to support metadata scheduling and modality selection across modal processing stages. The encapsulation method can use protocol buffers (such as Protobuf) or multimodal object structures (such as JSON-LD) for format unification and tag alignment, ensuring that subsequent processing modules can directly call the data without additional decoding, while also supporting traceability analysis and context evolution management of task source data.

[0096] The entire process constitutes a standardized input data flow path for multimodal input preprocessing and standardization, which not only improves the usability, structure, and semantic cleanliness of the raw data, but also builds a reusable and highly consistent input foundation for cross-modal semantic modeling and dialogue intent recognition.

[0097] This embodiment constructs a multi-channel input system and integrates a metadata-driven data type recognition mechanism to achieve dynamic parsing and differentiated processing of input data from different modalities. This allows the system to maintain input flexibility while achieving a structured and unified representation. In the visual path, standard resolution adjustment and color space standardization enhance the model adaptability and feature expressiveness of image content. In the text path, character encoding standardization and sensitive information filtering improve the universality and semantic security of text data. The encapsulated standardized data provides a unified data structure, format compatibility, and semantic clarity foundation for subsequent semantic modeling, intent recognition, and context evolution, effectively improving the preprocessing efficiency and information understanding capabilities of the multimodal interaction system.

[0098] In one embodiment, step S20 above includes:

[0099] S201, extract text input data and image input data from the interaction data to be processed respectively;

[0100] S202, Input the text input data into the text intent recognition model to generate a text semantic vector;

[0101] S203, input the image input data into the image semantic recognition model to generate an image semantic vector;

[0102] S204, The text semantic vector and image semantic vector are fused through a cross-modal alignment network to generate fused semantic features;

[0103] S205, Generate a semantic label distribution based on the fused semantic features;

[0104] S206, determine a unified interaction intent based on the semantic tag distribution.

[0105] In this embodiment, the information in the interactive data to be processed often includes both visual and textual elements, such as a medical record image with explanatory text or a customer complaint text with an attached image screenshot. To extract the potential semantic information contained in both modalities, modal decoupling processing of the data is required first. In implementation, the system traverses and filters the encapsulated interactive data structure, identifying the text input data and image input data fields contained within it through data tags or metadata headers, and loading them into the corresponding processing channels.

[0106] For text input data, the processing flow includes preprocessing operations such as pre-segmentation, stop word removal, dictionary mapping, and encoding format standardization before inputting it into a text intent recognition model. This model typically employs a deep neural network based on the Transformer architecture, such as BERT, RoBERTa, or the lightweight DistilBERT model. It models cross-sentence and cross-structural dependencies in long texts through a self-attention mechanism, thereby generating text semantic vectors with rich semantic representation capabilities. In healthcare scenarios, word vector models can be fine-tuned using clinical corpora or professional terminology databases to enhance the recognition of professional entities such as disease terms and treatment processes. In the financial sector, regulatory clauses and product terminology corpora can be introduced as training sets to improve the accuracy of recognizing asset-related terms and risk control intent.

[0107] Image input data needs to be processed by an image semantic recognition model, which typically includes a convolutional neural network (CNN) or a more advanced image coding structure (such as CLIP's Vision Encoder). Before input, the image undergoes uniform size cropping, color standardization, and normalization to fit the model's training parameters and ensure that semantic regions are not distorted. The model extracts mid-to-high-order semantic features such as texture, edges, and shapes from the image through multi-layer convolutional operations, ultimately generating a stable set of image semantic vectors. In medical scenarios, the model can learn to recognize lesion shapes and organ regions by training on medical image datasets (such as X-ray and CT images); in the financial field, the model can recognize key information areas such as table structures, seals, or handwritten signatures in contract documents.

[0108] Both types of modal vectors are input into the cross-modal alignment network, which aims to establish a semantic mapping relationship between the visual and textual modalities. Common structures include two-stream encoders, shared embedding space mapping layers, and contrastive learning mechanisms (such as InfoNCE). During training, the system establishes a unified semantic space by aligning the semantic relationships between text and image pairs, making the similarity between text and images measurable within this space. In actual operation, the network receives text and image semantic vectors and performs feature fusion operations, such as vector concatenation, weighted averaging, attention fusion, and gating mechanisms, to generate fused semantic features. These features possess cross-modal consistency and sufficient semantic representation, and can be directly used by subsequent downstream classification or sequence modeling tasks.

[0109] The generated fused semantic features are input into the label distribution generation module, which performs multi-class predictions based on the intent label set defined in the historical corpus or training set. The system uses the softmax activation function or the sigmoid multi-label strategy to calculate the probability distribution of the fused semantic features on each intent label, forming a semantic label distribution. This label distribution not only contains the most likely main intent label, but can also be used for intent uncertainty analysis and multi-intent candidate ranking, providing parameter support for subsequent dynamic dialogue strategies.

[0110] Ultimately, the system selects the label with the highest confidence from the semantic label distribution as the unified interaction intent output, or generates multiple candidate interaction intents based on a threshold strategy and records their confidence intervals. The unified interaction intent is not merely a subjective understanding extracted from a single modality, but rather an abstract dialogue goal that integrates two or more modalities, accurately reflecting the user's true intent and behavioral expectations. This generation method improves the accuracy of the dialogue system in discerning input intents in complex scenarios and significantly reduces the risk of misjudgment caused by modal ambiguity in intent recognition.

[0111] This embodiment constructs a dual-path processing mechanism combining text and image semantic recognition models. Building upon this, it introduces a cross-modal alignment network to fuse semantic representations. The system can simultaneously capture the implicit semantic information in both text and images, uniformly modeling it as a consistent semantic label distribution. This process effectively solves the semantic fragmentation problem inherent in traditional dialogue systems when handling mixed text and image inputs, significantly improving the accuracy and robustness of intent recognition when facing multimodal inputs. The generated unified interactive intent not only possesses higher semantic expressiveness but also adapts to complex cross-modal expressions and semantically dense interaction tasks in healthcare and financial scenarios, enhancing the system's ability to understand and respond to the user's true intent.

[0112] In one embodiment, step S30 above includes:

[0113] S301, Extract the dialogue history stack from the context state data;

[0114] S302, Obtain the historical intent sequence in the dialogue history stack;

[0115] S303, determine the semantic similarity between the unified interaction intent and the historical intent sequence;

[0116] S304, determines historical attenuation weights based on a programmable attenuation function;

[0117] S305, determine the initial association strength based on the semantic similarity and the historical decay weight;

[0118] S306, Detect the presence of explicit isolation commands;

[0119] S307, when an explicit isolation instruction exists, sets the intent association strength to zero;

[0120] S308, when there is no explicit isolation instruction, the initial association strength is used as the intended association strength.

[0121] In this embodiment, context state data is a structured collection of information used in a dialogue system to track and record historical interaction states. It typically includes a dialogue history stack, context variables, tool call results, and user state labels. To determine whether the unified interaction intent input by the current user is related to existing semantic targets in historical dialogues, a mechanism for measuring the strength of this association needs to be introduced as a key indicator for state switching and associative response strategies. The core of detecting the strength of the intent association between the unified interaction intent and the context state data lies in constructing a dynamic semantic matching system and a decay perception mechanism.

[0122] The system first extracts the dialogue history stack from the context state data. The dialogue history stack is a collection of historical dialogue rounds organized chronologically. Each record stores structured data such as the historical intent identified in that round of interaction, trigger nodes, and tool invocation identifiers. In implementation, the history stack is typically organized as a timestamp-sorted data queue, supporting fast indexing and backtracking operations. The system traverses all entries in the stack structure and extracts the historical intent from each dialogue record, constructing a historical intent sequence. This sequence is a multi-round semantic trajectory that reflects the evolution path of the user's intent throughout the entire interaction process.

[0123] A unified interaction intent needs to be compared semantically with the sequence of historical intents. To this end, the system inputs the unified interaction intent and each historical intent into a semantic encoder, converting them into high-dimensional vector representations. Common methods include vectorization models based on Sentence-BERT and dense semantic embedding networks. Then, measures such as cosine similarity or Euclidean distance are used to calculate the semantic similarity score between the current interaction intent and each historical intent. Higher semantic similarity indicates that the current user intent is close to the semantic goals of this round of historical interactions, suggesting strong contextual connectivity.

[0124] Considering that semantic similarity cannot fully reflect the semantic weight of time decay, the system introduces a programmable decay function to model the changing trend of the importance of historical rounds in the overall semantic association. This decay function is a customizable parameterized function structure; the input is the relative position index of the historical round, and the output is the decay weight value of that round. Common forms of decay functions include exponential decay functions, piecewise linear decay functions, and jump quantile functions. By configuring decay function parameters, such as the basic decay factor, time span adjustment coefficient, and isolation flag adjustment parameters, the system assigns different weight intensities to each historical intent.

[0125] The semantic similarity of each historical intent is multiplied by its corresponding historical decay weight to obtain the single-round contribution value of that round to the current intent. The system then sums up the contribution values ​​of all rounds to obtain the initial association strength between the current unified interaction intent and the entire historical intent sequence. This process not only integrates semantic proximity but also reflects the time decay characteristics of historical states, thereby achieving association modeling that is more consistent with cognitive logic.

[0126] To support users' need to actively reset the context, the system employs an explicit isolation instruction detection mechanism. Explicit isolation instructions are expressions in user input that imply semantic breaks, such as "change the topic," "start over," or "don't use the previous one." These expressions are parsed by the semantic recognition model, which generates corresponding isolation flags. Upon detecting an explicit isolation instruction, the system directly sets the intent association strength to zero to disrupt the inheritance chain of context states. Otherwise, the system retains the initial association strength value as a semantic connectivity indicator during the current context inference process.

[0127] This association strength is not only used for subsequent context association mode selection, but also serves as an input basis for intent redirection, context clearing, and topic shift judgment, providing support for the flexibility of multi-turn dialogue flow.

[0128] This embodiment constructs a dynamic association strength calculation mechanism based on the coupling of semantic similarity and historical decay weights. The system can not only accurately capture the potential connection between the current interaction intent and historical semantics, but also dynamically interrupt contextual inheritance relationships through explicit isolation instruction detection. This effectively balances intent coherence and topic switching flexibility in multi-turn interactions. This mechanism significantly improves the dialogue system's ability to understand contextual information in complex tasks, avoiding the problem of inflexible dialogue state transitions under a fixed decay strategy. It is particularly suitable for dialogue processes with long task chains and frequent semantic jumps in healthcare and financial scenarios.

[0129] In one embodiment, step S40 above includes:

[0130] S401, compare the intent association strength with a preset association threshold;

[0131] S402, when the intent association strength is greater than or equal to the preset association threshold, the context association mode is set to the continuous association mode;

[0132] S403, when the intent association strength is less than the preset association threshold, the context association mode is set to single-round isolation mode;

[0133] S404, in the continuous association mode, the target dialogue node is determined based on the dialogue history stack in the unified interaction intent and context state data;

[0134] S405, in the single-round isolation mode, the target dialogue node is determined based on the unified interaction intent.

[0135] In this embodiment, after calculating the correlation strength between the unified interaction intent and the historical context, the system needs to deduce whether the current dialogue continues the historical context. To this end, a context association pattern is introduced as the divergence logic for dialogue state management. The context association pattern is a control strategy flag that identifies whether the current dialogue depends on historical semantic links, used to indicate whether the system considers historical state factors during the selection of target dialogue nodes.

[0136] First, the system compares the currently calculated intent association strength with a preset association threshold. This threshold is used to determine whether the current interaction is within a significant range of historical semantic continuation. This threshold can be obtained through statistical analysis of the distribution of historical session data and is typically set as a floating value between 0.3 and 0.7. To enhance the system's adaptability to different user behavior patterns, this threshold can also be dynamically adjusted, such as by weighting parameters like user activity level, topic switching frequency, and task path length to form an adaptive threshold.

[0137] When the current intent association strength is greater than or equal to the preset threshold, the system sets the current context association mode to "continuous association mode". In this mode, the system believes that the current user input intent still has a valid association with the historical semantic path, and should continue the original context semantic chain, and participate in the reasoning and selection process of the target dialogue node in combination with the historical interaction path.

[0138] In the persistent association mode, the system uses both the current unified interaction intent and the dialogue history stack information in the context state data to determine the target dialogue node. Specifically, the system first identifies path branches where the topic may continue by using the most recently triggered nodes marked in the dialogue history stack and their response content; then, it calculates the semantic similarity between the current unified interaction intent and the candidate path branches; finally, it selects the optimal dialogue node as the receiving point for the current round of response based on the path score ranking. During this process, the transition graph structure between dialogue nodes needs to have explicit edge weight definitions to support the determination of different path jump costs and response priorities.

[0139] Conversely, when the intent association strength is below a preset threshold, it indicates a significant semantic disconnect between the current user input and the historical dialogue state. In this case, the context association mode is set to "single-turn isolation mode." In this mode, the system infers solely based on the current unified interaction intent, completely disregarding historical semantic trajectories. This strategy is suitable for scenarios involving strong topic switching, temporary requests, or tasks restarted with explicit instructions.

[0140] In single-turn isolation mode, the system directly inputs the unified interaction intent into the task classifier or the multi-turn dialogue node selection network. The network matches the preset dialogue node index based on the current semantic distribution and determines the most likely trigger node through attention mechanism, semantic classification boundary or task mapping table, avoiding interference from historical state and ensuring independent processing of new intent.

[0141] The determination of the target dialogue node will directly affect the logical path of subsequent tool calls, business execution, and response generation. Therefore, the reasonable division of the context association mode is the key to ensuring that the entire multi-turn interaction process is both coherent and flexible in transitions.

[0142] This embodiment constructs a context association mode switching mechanism based on intent association strength, enabling flexible judgment between context inheritance and semantic breakage in multi-turn interactions. This allows the system to adaptively decide whether to continue the historical semantic link for dialogue node selection, improving the responsiveness of the dialogue system in long-link tasks and temporary switching needs. The continuous association mode supports task tracking and multi-turn completion, while the single-turn isolation mode ensures topic jumps and task independence. The combination of the two can significantly reduce semantic drift caused by erroneous inheritance, enhancing the system's state robustness and context adaptability in complex business processes.

[0143] In one embodiment, step S50 above includes:

[0144] S501, Based on the tool identifier, obtain a matching executable tool from the tool registry center;

[0145] S502, Obtain the input mode description corresponding to the tool identifier from the tool registration center;

[0146] S503, extract business entity slots from context state data;

[0147] S504, Based on the input mode description, obtain tool input parameters from the business entity slot;

[0148] S505, verify the compatibility between the tool input parameters and the input mode description;

[0149] S506, When the verification passes, execute the executable tool in the sandbox environment to generate the original execution result;

[0150] S507, perform multi-protocol format conversion on the original execution result, and use the converted data as the tool execution result.

[0151] In this embodiment, after the system determines the target dialogue node based on a unified interaction intent and context association pattern, it further utilizes the tool identifier carried by the target dialogue node as a retrieval basis to search for executable tools matching the identifier in the tool registry center. The tool identifier is usually a unique logical marker used to point to a specific business processing logic or microservice unit. Its naming strategy can adopt naming rules based on business type, functional module, version information, etc., to ensure discriminability and uniqueness during registration and retrieval.

[0152] After obtaining the executable tool entity, the system also needs to synchronously obtain the corresponding input pattern description. The input pattern description is a structured metadata definition that clarifies the type, quantity, constraints, and data dependencies of the parameters required for tool execution. Common formats include input template specifications based on JSON Schema, Protocol Buffers, or custom DSLs. This input pattern not only provides static definitions of input parameters but can also include dynamic validation expressions and default value mapping logic.

[0153] To assemble the tool's invocation parameters, the system needs to extract business entity slots generated during the current interaction from the context state data. Business entity slots refer to structured semantic fragments identified and populated during multiple rounds of interaction. They are typically stored as key-value pairs, containing information such as field name, field value, source node, and confidence level. For example, in financial transactions, slots might include "Account Type = Enterprise," "Transaction Amount = 50,000," and "Application Purpose = Operating Funds"; in healthcare, they might include "Department = Endocrinology," "Symptoms = Fatigue and Weakness," and "Imaging Data ID = IMG23592," etc.

[0154] The system searches for matching fields in the current business entity slots based on the parameter dependency paths defined in the input pattern description and extracts them as tool input parameters. If issues such as missing fields, mismatched types, or inconsistent formats exist, the system will trigger a parameter validation mechanism. This mechanism compares and verifies each extracted input parameter against the input pattern description, including field name consistency, data type matching, compliance with range constraints, and validation of format requirements (such as date format, numerical precision, and enumeration value limits). The system will only proceed with the tool call if all parameters pass validation.

[0155] The tool's execution takes place in a sandbox environment. A sandbox environment is an execution container isolated from the main system, used to control the tool's resource boundaries, security permissions, network access scope, and so on. This environment can be built based on container technologies (such as Docker), lightweight virtualization technologies (such as gVisor, Firecracker), or interpreted runtime sandboxes (such as Node.js VM, Python Restricted Execution). The purpose of sandboxing is to prevent the tool from damaging the system's main process or leaking information during runtime, which is especially important when the tool comes from third-party components or is uploaded and registered automatically.

[0156] After the tool completes execution, it returns the original execution result. The original result may be in JSON, XML, CSV, or binary format, or even a custom protocol response with multiple nested structures. To ensure the uniformity and operability of subsequent system processing modules, a multi-protocol format conversion needs to be performed on the original execution result. This conversion process is completed by the protocol adapter module, which performs format unification processing according to the current system's standard intermediate representation (such as a unified JSON structure or internal data middleware protocol) to ensure that the result has structural consistency, field traceability, and retains metadata such as status indicators, exception information, and processing time from the original response.

[0157] Ultimately, the converted data is encapsulated as the tool's execution result and synchronized to subsequent processes for response content generation and context updates. This execution path demonstrates the decoupling between business logic and tool capabilities, and also enables unified access and secure invocation of multiple source tools.

[0158] This embodiment obtains tools and their input mode descriptions from the tool registry based on tool identifiers, and completes parameter extraction and verification by combining business entity slots in context state data. Then, it performs secure isolation execution and format standardization in a sandbox environment, achieving a highly reliable, loosely coupled, and securely controllable dynamic invocation mechanism for business tools. This mechanism not only improves compatibility and reusability with different tools, but also enhances the system's ability to accept heterogeneous tool response results through format conversion and parameter mode adaptation, effectively supporting flexible expansion and decoupled deployment of functional modules in business processes.

[0159] In one embodiment, step S60 above includes:

[0160] S601, Construct a response data structure based on the execution result of the tool;

[0161] S602, Generate response content containing the aforementioned response data structure;

[0162] S603, Obtain business entity data extracted during tool invocation;

[0163] S604, identify the current context association pattern;

[0164] S605, when the current context association mode is the continuous association mode, add the interaction data to be processed, the unified interaction intent, the response content, the tool execution result and the target dialogue node to the dialogue history stack, and write the business entity data into the business entity slot;

[0165] S606, when the current context association mode is single-round isolation mode, write the business entity data into the business entity slot;

[0166] S607, Update the dialogue history stack and business entity slots to the context state data.

[0167] In this embodiment, after the system obtains the execution result of the executable tool, it constructs a structured response output based on the result to generate user-perceptible response content. The tool execution result is typically in a semi-structured or structured data format, including status codes, main business fields, auxiliary prompt fields, and debugging information. The system uses a response data structure construction module to map the fields returned by the tool to a unified response format according to a predefined response template. This mapping process relies on the binding rules between the response fields and the system's internal semantics, and supports operations such as field renaming, type conversion, unit standardization, and status information encapsulation. For example, in a medical scenario, if the tool returns "blood_glucose=6.8", the system can convert it to "fasting blood glucose value is 6.8mmol / L, result is normal" using template rules; in a financial scenario, if the tool returns "risk_score=0.72", it can generate a structured language prompt such as "current risk score is 0.72, manual review is recommended".

[0168] Subsequently, the system encapsulates the constructed response data structure into response content. This response content can be formatted and generated using a Natural Language Generation (NLG) system, or it can be generated as a JSON UI component template based on component-based rendering to adapt to different front-end interface protocols and display requirements. The response content generation module integrates system context, such as user preferences, dialogue tone, and historical dialogue style, to ensure the semantic coherence and presentation consistency of the generated content.

[0169] During response generation, the system also needs to synchronously extract business entity data identified during tool invocation. This data originates from the input parameter extraction stage before tool execution and may include information such as entity category, field name, populated value, data source node, and context mapping path. Business entity data is the core unit constituting the interaction context state, and its structure determines the depth of the system's reasoning capabilities in subsequent rounds of interaction.

[0170] The system then identifies the current context association pattern. This pattern is typically stored as an identifier in the interaction state cache, indicating whether the current dialogue continues the historical semantic link. If the current pattern is a continuous association pattern, the system writes all semantic states and execution data from the current interaction process into the context. Specifically, this includes appending elements such as the interaction data to be processed (i.e., the original input of this round), the unified interaction intent (intent labels generated after cross-modal alignment), the response content (structured natural language or templated rendered content), the tool execution result (standard response after protocol conversion), and the target dialogue node (location node in the semantic guidance tree) to the dialogue history stack, and updating the extracted business entity data to the business entity slots. The dialogue history stack is a time-series structure that records the important inputs and outputs of each round of dialogue, supporting historical tracing and cross-round semantic retrieval; the business entity slots are dynamically updated key-value mapping tables that maintain the real-time state of structured fields, supporting subsequent slot filling and template-driven logic.

[0171] If the current context association mode is single-turn isolation mode, the system will only retain the business entity data identified in this turn and update it to the business entity slot, while the dialogue history stack will remain unchanged. This mechanism is used to prevent multi-turn semantic pollution or offset and is suitable for lightweight scenarios with one-question-one-answer interactions, such as query-based, verification-based, and conditional intent inputs. In this mode, the context state after the response is generated will only reflect the current operation and will not inherit the historical semantic trajectory.

[0172] Finally, the dialogue history stack and business entity slots are synchronously written to the context state data store. This storage module can be implemented based on a key-value database (such as Redis, LevelDB), a graph database (such as Neo4j), or a state management middleware (such as Recoil, Zustand). The update strategy must support atomic writes, version recording, and conflict resolution mechanisms to ensure context consistency and system stability.

[0173] Example Description: In a healthcare business scenario, patients initiate interactive operations through smart terminal devices (such as medical institution clients, health mini-programs, or voice assistant devices) to express their purpose for seeking medical treatment or their health needs. The system first receives the data submitted by the user, which may include voice commands, uploaded images of physical examination reports, or real-time photos of skin lesions. After receiving this raw input, the system transmits it to the data access channel through the input interface. The metadata analysis engine performs type determination on the input; for example, image files (JPEG format skin images) are recognized as visual data, while the text content transcribed from the user's speech is recognized as text data.

[0174] If the input is visual data, the system decodes the image to obtain the raw pixel matrix, and then performs resolution normalization and color space conversion to obtain normalized image data. If the input is text data, such as "I've had a slight fever recently, could you please give me a preliminary analysis?", the system first performs character encoding normalization and sensitive word filtering on the text, ultimately forming normalized text data. After this stage, all inputs will be encapsulated into interactive data to be processed.

[0175] Subsequently, the system performs cross-modal semantic parsing. Visual data is fed into a medical image semantic recognition model for processing, such as identifying suspected eczema or urticaria areas in skin lesion images; text data enters the semantic recognition module to identify keywords such as "low-grade fever" and "preliminary analysis." Next, a cross-modal alignment network fuses these two types of semantic vectors to form a unified fused semantic feature. Based on this, the system generates a semantic label distribution, such as "minor infection" and "skin discomfort," and determines the final interaction intent label "request preliminary health self-check."

[0176] The system then assesses the relevance of the interaction intent to the user's previous contextual states in the conversation. By extracting the dialogue history stack, it finds that the user inquired about "red spots on the skin" and "mild itching" in the first three rounds of dialogue, and forms a sequence of historical intents based on this. The system calculates the semantic similarity between the current intent and these historical intents, applies a time weight through a programmable decay function, and combines this with the presence of explicit isolation instructions such as "new question" or "start again" to generate an intent association strength value. If the current input does not explicitly change the topic, the intent association strength is relatively high.

[0177] Based on the comparison between the intent relevance strength and a preset threshold, the system sets the context mode to "continuous relevance mode." In this mode, the system searches for connectable business nodes on the existing semantic link and identifies an optimal match between the current issue and the "health self-check service" node. Therefore, the system decides to jump to this target dialogue node.

[0178] The target dialogue node contains the tool identifier "infection_risk_assessment_tool", which is a local infection risk assessment tool. The system searches for the executable version of this tool and its input parameter description document through the tool registry. Combining the entity data extracted from the user's previous dialogues: "body temperature = 37.6℃", "lesion site = arm", "duration = 3 days", and "accompanying symptom = itching", the system maps these fields to the input slots required by the tool. After the tool input parameters pass compatibility checks, the assessment tool is executed in a sandbox environment.

[0179] After the tool finishes execution, it returns the result "Infection Risk Level = Low, Observation Recommended". The system formats and encapsulates this result and calls the natural language generation module to generate the response content: "Based on your input, the current preliminary assessment is a mild infection risk. It is recommended to pay attention to personal hygiene and continue to observe. If symptoms worsen, it is recommended to seek medical attention promptly." The response content is displayed in both structured data and text data formats for user interface display and subsequent status tracking, respectively.

[0180] The system identifies the current interaction as a continuous association pattern, and therefore appends the interaction data (images and text), interaction intent (preliminary health self-check), tool results, target nodes, and business entities (such as body temperature and symptoms) to the dialogue history stack and business entity slots. This update action allows the system to directly utilize existing semantic and entity fields from the historical context in subsequent conversations, such as "Can I take cold medicine now?", improving response accuracy and contextual consistency.

[0181] Ultimately, all updates are synchronously written into the context state data to support multi-turn continuous interaction and deep context fusion. The entire process forms a complete closed loop, from visual and text data fusion and understanding, semantic recognition, tool scheduling, response generation to context writing back, providing a highly robust, context-sensitive, and flexible tool access mechanism for intelligent dialogue systems in the healthcare field.

[0182] In a fintech scenario, a user initiates a loan inquiry through the intelligent customer service interface of a financial service platform. At the start of the interaction, the platform receives raw data submitted by the user, including a text input box stating "I'd like to see if I can borrow 50,000 yuan," as well as uploaded photos of their ID card and screenshots of their employment certificate. The input interface collects this data into a set of raw interaction data, which is then passed into the data access process.

[0183] The metadata analysis module determines the data type of the input. The platform identifies text information as natural language text and images as visual data. Visual data is decoded by the image processing unit to extract the original pixel matrix, and then further normalized in resolution and transformed in color space to form standardized image data. Text data is processed by the character encoding unification and financial sensitive word (such as fraud, cash-out, etc.) filtering module to generate standardized text data. These two types of standardized data are jointly packaged into interactive data to be processed and submitted to the subsequent processing chain.

[0184] Next, the system extracts text input data and image input data respectively, and feeds them into pre-trained text semantic recognition models and image semantic recognition models. The text model identifies user intent containing keywords such as "loan" and "amount of 50,000" and generates corresponding semantic vectors; the image model identifies uploaded materials as identity authentication documents and employment certificates, and extracts image-level semantic features. A cross-modal alignment network is used to align and fuse the two types of semantic vectors, outputting a unified semantic feature tensor. Based on this tensor, a semantic label distribution is generated, ultimately determining the unified interaction intent as "conduct loan eligibility pre-screening".

[0185] The platform obtains the current user's contextual state data and extracts historical dialogue records. The detection results show that the user previously inquired about topics such as "I earn 10,000 yuan a month, what recommendations do you have?" The system extracts intent sequences from these historical dialogues and performs semantic similarity calculations with the current interaction intent. An initial association strength is generated by adding a time dimension weight to a programmatically defined historical decay function. After confirming that the current dialogue does not contain instructions such as "restart," the system retains this initial association strength as the current intent association strength.

[0186] After comparing the intensity of the intent with a preset threshold, the system determines that the user's intent is highly correlated with the historical context, and therefore the context mode is set to "continuous association mode". Based on the current intent and the historical dialogue stack, the platform determines the current optimal target dialogue node as the "loan pre-approval tool node".

[0187] The associated tool for this node is identified as "credit_risk_eval_tool_v2". The system searches the tool registry and loads the executable version of the tool and the required input mode description. Next, it extracts necessary fields from the business entity slots, including monthly income, years of employment, occupation type, and credit score, as input parameters for the loan risk assessment tool. The system performs parameter validation to ensure compatibility with the tool's input mode, and after successful validation, starts the tool in an isolated sandbox environment.

[0188] After the tool completes its execution, it outputs a structured assessment result: a credit risk level of moderate to low, a recommended loan amount of 30,000, and a recommended loan term of 12 months. The platform standardizes the result according to the protocol format and calls the response content generation engine to construct a natural language feedback: "Based on your income and submitted materials, the system assesses your current loanable amount to be approximately 30,000, and recommends a loan term of 12 months. Do you need to apply further?" The generated structured result and natural language text synchronously form the response content.

[0189] Because the context mode is continuously associated, the platform fully appends the current interaction data, unified interaction intent, response content, tool results, and target dialogue node to the user's dialogue history stack, and writes the business entity data extracted from this round of interaction (such as occupation = civil servant, annual income = 120,000 yuan, credit rating = B+) into the business entity slot. Finally, the dialogue history and entity slots are updated together in the context state data.

[0190] This complete process achieves a closed-loop chain, from the fusion and understanding of multimodal input (text and images), to dynamic context-based switching, loosely coupled tool invocation and response generation, and finally to the structured updating of state data. While maintaining high response accuracy, the system supports multi-turn continuous dialogue with users and ensures dynamic scalability of business processes. It is particularly suitable for demanding, context-sensitive intelligent interaction tasks in the fintech field, such as credit assessment, intelligent wealth management, risk alerts, and process review.

[0191] This embodiment achieves semantic mapping and format unification of multimodal tool execution results output to the user by constructing a standard response data structure and dynamically generating response content. Based on this, the system flexibly controls the context state update path according to the context association pattern, enabling the system to possess dynamic state evolution capabilities for multi-turn dialogues and lightweight semantic avoidance capabilities in isolation mode. This mechanism ensures dialogue continuity while also improving the system's fault tolerance for short-term tasks and the clarity of task boundaries, achieving bidirectional synchronization between the semantic state flow and the tool call trajectory, effectively supporting the scalability and robustness of human-computer interaction in complex business scenarios.

[0192] In one embodiment, an interaction intent-driven tool scheduling device is provided, which corresponds one-to-one with the interaction intent-driven tool scheduling method described in the above embodiments. (Refer to...) Figure 3 , Figure 3 This is a schematic diagram of the functional modules of a preferred embodiment of the interactive intent-driven tool scheduling device of the present invention. The modules include a multimodal input processing module 10, a cross-modal semantic parsing module 20, an intent association analysis module 30, a dialogue decision engine module 40, a tool scheduling execution module 50, and a context state management module 60. Detailed descriptions of each functional module are as follows:

[0193] The multimodal input processing module 10 is used to acquire interactive data to be processed, which includes visual data or text data;

[0194] The cross-modal semantic parsing module 20 is used to perform cross-modal semantic parsing on the interaction data to be processed and generate a unified interaction intent;

[0195] The intent association analysis module 30 is used to detect the intent association strength between the unified interaction intent and the context state data;

[0196] The dialogue decision engine module 40 is used to dynamically select the context association mode according to the intent association strength, and determine the target dialogue node based on the context association mode and the unified interaction intent.

[0197] The tool scheduling and execution module 50 is used to schedule executable tools in the tool registry that match the tool identifier of the target dialogue node for sandboxed execution, so as to obtain the tool execution result.

[0198] The context state management module 60 is used to generate response content based on the tool execution result, and selectively update the context state data with the pending interaction data of the current interaction, the unified interaction intent, the response content, the tool execution result, the target dialogue node, and the business entity data extracted during the tool call, according to the context association mode.

[0199] In one embodiment, the multimodal input processing module 10 is specifically used for:

[0200] Receive raw interactive data submitted by the user through the multi-channel input interface;

[0201] The data type of the original interaction data is identified through the metadata analysis engine;

[0202] When the data type is a visual data type, the original interactive data is decoded to obtain the original pixel matrix, the original pixel matrix is ​​subjected to resolution standardization processing to obtain a standard resolution matrix, and the standard resolution matrix is ​​subjected to color space conversion to generate standardized visual data.

[0203] When the data type is a text data type, the character encoding format of the original interactive data is detected, the character encoding format is converted into a unified encoding format to obtain intermediate text data, and sensitive information filtering processing is performed on the intermediate text data to generate standardized text data.

[0204] The standardized visual data or standardized text data is encapsulated into interactive data to be processed.

[0205] In one embodiment, the cross-modal semantic parsing module 20 is specifically used for:

[0206] Extract text input data and image input data from the interaction data to be processed, respectively;

[0207] The text input data is fed into a text intent recognition model to generate a text semantic vector;

[0208] The image input data is input into the image semantic recognition model to generate an image semantic vector;

[0209] By fusing the text semantic vectors and image semantic vectors through a cross-modal alignment network, fused semantic features are generated;

[0210] A semantic label distribution is generated based on the fused semantic features;

[0211] A unified interaction intent is determined based on the distribution of the semantic tags.

[0212] In one embodiment, the intent association analysis module 30 is specifically used for:

[0213] Extract the dialogue history stack from the context state data;

[0214] Obtain the historical intent sequence from the dialogue history stack;

[0215] Determine the semantic similarity between the unified interaction intent and the historical intent sequence;

[0216] Historical attenuation weights are determined based on a programmable attenuation function.

[0217] The initial association strength is determined based on the semantic similarity and the historical decay weight;

[0218] Detect the presence of explicit isolation commands;

[0219] When an explicit isolation directive exists, set the intent association strength to zero;

[0220] When no explicit isolation instruction is available, the initial association strength is used as the intended association strength.

[0221] In one embodiment, the dialogue decision engine module 40 is specifically used for:

[0222] Compare the intent association strength with a preset association threshold;

[0223] When the intent association strength is greater than or equal to the preset association threshold, the context association mode is set to the continuous association mode;

[0224] When the intent association strength is less than the preset association threshold, the context association mode is set to single-round isolation mode;

[0225] In the continuous association mode, the target dialogue node is determined based on the dialogue history stack in the unified interaction intent and context state data;

[0226] In the single-round isolation mode, the target dialogue node is determined based on the unified interaction intent.

[0227] In one embodiment, the tool scheduling and execution module 50 is specifically used for:

[0228] Based on the tool identifier, obtain the matching executable tool from the tool registry center;

[0229] Obtain the input mode description corresponding to the tool identifier from the tool registry center;

[0230] Extract business entity slots from context state data;

[0231] Based on the input pattern description, tool input parameters are obtained from the business entity slot;

[0232] Verify the compatibility between the tool input parameters and the input mode description;

[0233] When the verification passes, the executable tool is executed in the sandbox environment to generate the original execution results;

[0234] The original execution result is converted to a multi-protocol format, and the converted data is used as the tool execution result.

[0235] In one embodiment, the context state management module 60 is specifically used for:

[0236] Construct a response data structure based on the execution results of the aforementioned tool;

[0237] Generate response content containing the aforementioned response data structure;

[0238] Retrieve business entity data extracted during tool invocation;

[0239] Identify the current context association pattern;

[0240] When the current context association mode is the continuous association mode, the interaction data to be processed, the unified interaction intent, the response content, the tool execution result and the target dialogue node are added to the dialogue history stack, and the business entity data is written to the business entity slot.

[0241] When the current context association mode is single-round isolation mode, the business entity data is written to the business entity slot;

[0242] Update the dialogue history stack and business entity slots to the context state data.

[0243] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 4As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides determination and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external user terminals via a network connection. When the computer program is executed by the processor, it implements the functions or steps of an interactive intent-driven tool scheduling method on the server side.

[0244] In one embodiment, a computer device is provided, which may be a user terminal, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides determination and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements user-side functions or steps of an interactive intent-driven tool scheduling method.

[0245] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps:

[0246] Acquire interactive data to be processed, which includes visual or textual data;

[0247] Cross-modal semantic parsing is performed on the interaction data to be processed to generate a unified interaction intent;

[0248] Detect the strength of intent association between the unified interaction intent and the context state data;

[0249] Based on the intent association strength, the context association mode is dynamically selected, and the target dialogue node is determined based on the context association mode and the unified interaction intent.

[0250] Based on the tool identifier of the target dialogue node, the executable tools in the scheduling tool registry that match the tool identifier are executed in a sandbox to obtain the tool execution result;

[0251] Based on the tool execution result, a response content is generated, and according to the context association mode, the pending interaction data of the current interaction, the unified interaction intent, the response content, the tool execution result, the target dialogue node, and the business entity data extracted during the tool call are selectively updated to the context state data.

[0252] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0253] Acquire interactive data to be processed, which includes visual or textual data;

[0254] Cross-modal semantic parsing is performed on the interaction data to be processed to generate a unified interaction intent;

[0255] Detect the strength of intent association between the unified interaction intent and the context state data;

[0256] Based on the intent association strength, the context association mode is dynamically selected, and the target dialogue node is determined based on the context association mode and the unified interaction intent.

[0257] Based on the tool identifier of the target dialogue node, the executable tools in the scheduling tool registry that match the tool identifier are executed in a sandbox to obtain the tool execution result;

[0258] Based on the tool execution result, a response content is generated, and according to the context association mode, the pending interaction data of the current interaction, the unified interaction intent, the response content, the tool execution result, the target dialogue node, and the business entity data extracted during the tool call are selectively updated to the context state data.

[0259] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and user side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0260] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0261] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0262] It should be noted that if any software tools or components not belonging to this company appear in the embodiments of this application, they are merely illustrative examples and do not represent actual use. The embodiments described above are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A tool scheduling method driven by interactive intent, characterized in that, Includes the following steps: Acquire interactive data to be processed, which includes visual or textual data; Cross-modal semantic parsing is performed on the interaction data to be processed to generate a unified interaction intent; Detect the strength of intent association between the unified interaction intent and the context state data; Based on the intent association strength, the context association mode is dynamically selected, and the target dialogue node is determined based on the context association mode and the unified interaction intent. Based on the tool identifier of the target dialogue node, the executable tools in the scheduling tool registry that match the tool identifier are executed in a sandbox to obtain the tool execution result; Based on the tool execution result, a response content is generated, and according to the context association mode, the pending interaction data of the current interaction, the unified interaction intent, the response content, the tool execution result, the target dialogue node, and the business entity data extracted during the tool call are selectively updated to the context state data.

2. The tool scheduling method driven by interactive intent as described in claim 1, characterized in that, Acquire interactive data to be processed, which includes visual or textual data, including: Receive raw interactive data submitted by the user through the multi-channel input interface; The data type of the original interaction data is identified through the metadata analysis engine; When the data type is a visual data type, the original interactive data is decoded to obtain the original pixel matrix, the original pixel matrix is ​​subjected to resolution standardization processing to obtain a standard resolution matrix, and the standard resolution matrix is ​​subjected to color space conversion to generate standardized visual data. When the data type is a text data type, the character encoding format of the original interactive data is detected, the character encoding format is converted into a unified encoding format to obtain intermediate text data, and sensitive information filtering processing is performed on the intermediate text data to generate standardized text data. The standardized visual data or standardized text data is encapsulated into interactive data to be processed.

3. The tool scheduling method driven by interactive intent as described in claim 1, characterized in that, Perform cross-modal semantic parsing on the interaction data to be processed to generate a unified interaction intent, including: Extract text input data and image input data from the interaction data to be processed, respectively; The text input data is fed into a text intent recognition model to generate a text semantic vector; The image input data is input into the image semantic recognition model to generate an image semantic vector; By fusing the text semantic vectors and image semantic vectors through a cross-modal alignment network, fused semantic features are generated; A semantic label distribution is generated based on the fused semantic features; A unified interaction intent is determined based on the distribution of the semantic tags.

4. The tool scheduling method driven by interactive intent as described in claim 1, characterized in that, Detecting the intensity of intent association between the unified interaction intent and the context state data includes: Extract the dialogue history stack from the context state data; Obtain the historical intent sequence from the dialogue history stack; Determine the semantic similarity between the unified interaction intent and the historical intent sequence; Historical attenuation weights are determined based on a programmable attenuation function. The initial association strength is determined based on the semantic similarity and the historical decay weight; Detect the presence of explicit isolation commands; When an explicit isolation directive exists, set the intent association strength to zero; When no explicit isolation instruction is present, the initial association strength is used as the intended association strength.

5. The tool scheduling method driven by interactive intent as described in claim 1, characterized in that, Based on the intent association strength, a context association mode is dynamically selected, and the target dialogue node is determined based on the context association mode and the unified interaction intent, including: Compare the intent association strength with a preset association threshold; When the intent association strength is greater than or equal to the preset association threshold, the context association mode is set to the continuous association mode; When the intent association strength is less than the preset association threshold, the context association mode is set to single-round isolation mode; In the continuous association mode, the target dialogue node is determined based on the dialogue history stack in the unified interaction intent and context state data; In the single-round isolation mode, the target dialogue node is determined based on the unified interaction intent.

6. The tool scheduling method driven by interactive intent as described in claim 1, characterized in that, Based on the tool identifier of the target dialogue node, the executable tools in the scheduling tool registry that match the tool identifier are executed in a sandbox to obtain the tool execution results, including: Based on the tool identifier, obtain the matching executable tool from the tool registry center; Obtain the input mode description corresponding to the tool identifier from the tool registry center; Extract business entity slots from context state data; Based on the input pattern description, tool input parameters are obtained from the business entity slot; Verify the compatibility between the tool input parameters and the input mode description; When the verification passes, the executable tool is executed in the sandbox environment to generate the original execution results; The original execution result is converted to a multi-protocol format, and the converted data is used as the tool execution result.

7. The tool scheduling method driven by interactive intent as described in claim 1, characterized in that, Based on the tool execution result, a response content is generated. Then, according to the context association pattern, the pending interaction data of the current interaction, the unified interaction intent, the response content, the tool execution result, the target dialogue node, and the business entity data extracted during the tool invocation process are selectively updated to the context state data, including: Construct a response data structure based on the execution results of the aforementioned tool; Generate response content containing the aforementioned response data structure; Retrieve business entity data extracted during tool invocation; Identify the current context association pattern; When the current context association mode is the continuous association mode, the interaction data to be processed, the unified interaction intent, the response content, the tool execution result and the target dialogue node are added to the dialogue history stack, and the business entity data is written to the business entity slot. When the current context association mode is single-round isolation mode, the business entity data is written to the business entity slot; Update the dialogue history stack and business entity slots to the context state data.

8. A tool scheduling device driven by interactive intent, characterized in that, The interaction intent-driven tool scheduling device includes: The multimodal input processing module is used to acquire interactive data to be processed, which includes visual or text data. The cross-modal semantic parsing module is used to perform cross-modal semantic parsing on the interaction data to be processed, and generate a unified interaction intent; The intent association analysis module is used to detect the intent association strength between the unified interaction intent and the context state data; The dialogue decision engine module is used to dynamically select the context association mode based on the intent association strength, and determine the target dialogue node based on the context association mode and the unified interaction intent. The tool scheduling and execution module is used to schedule executable tools in the tool registry that match the tool identifier of the target dialogue node for sandboxed execution, and obtain the tool execution result. The context state management module is used to generate response content based on the tool execution result, and selectively update the context state data with the pending interaction data of the current interaction, the unified interaction intent, the response content, the tool execution result, the target dialogue node, and the business entity data extracted during the tool call, according to the context association mode.

9. A computer device, characterized in that, The computer device includes a memory, a processor, and an interactive intent-driven tool scheduler stored in the memory and executable on the processor. When executed by the processor, the interactive intent-driven tool scheduler implements the steps of the interactive intent-driven tool scheduling method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The storage medium stores an interactive intent-driven tool scheduler, which, when executed by a processor, implements the steps of the interactive intent-driven tool scheduling method as described in any one of claims 1-7.

Images