User intention accurate matching method oriented to massive skills, electronic equipment and storage medium

Abstract

The invention discloses a massive skill-oriented accurate user intention matching method, electronic equipment and a storage medium. The method comprises the following steps: acquiring demand information input by a user through an interactive interface of a target AI agent; performing hierarchical filtering on all skills capable of being called by the target AI intelligent agent by utilizing a preset multi-level screening mechanism based on the demand information to obtain a candidate skill set; calculating the confidence of each skill in the candidate skill set relative to the demand information based on a pre-trained large language model, and determining a target matching skill according to a confidence calculation result; and executing a corresponding skill calling strategy based on a confidence coefficient calculation result, and completing a user demand response. Through the combination of a multi-level screening mechanism, a large language model multi-level intention recognition mechanism and a multi-level skill calling strategy, high-precision, low-delay and easy-to-expand intention matching under massive skills is realized, and the user experience and the system robustness are remarkably improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, specifically to a method for accurately matching user intent with a vast amount of skills, an electronic device, and a storage medium. Background Technology

[0002] With the rapid development of Large Language Model (LLM) technology, AI agent platforms have widely integrated various practical skills, covering multiple high-frequency scenarios such as office work (e.g., leave applications, expense reimbursement submissions), life services (e.g., food delivery ordering, traffic inquiries), creative design (e.g., AI drawing, copywriting generation), and information inquiries (e.g., weather, meal card balance). However, traditional LLM typically uses a method of "directly parsing user queries and matching skills" in the process of intent recognition and skill invocation. When the platform integrates a large number of skills (e.g., more than 10), the following problems are prone to occur: 1. Severe intent confusion: Multiple skills have similar semantics, making it difficult for the model to distinguish them; 2. Lengthy matching path: It requires traversing all skills for comparison, resulting in high computational overhead and slow response time; 3. Lack of structured management: There is no classification system for skills, and adding or modifying skills requires adjusting the underlying algorithm, resulting in high maintenance costs; 4. Lack of reliability assessment: The matching results lack quantitative confidence assessment, often requiring multiple interactions for confirmation, resulting in a poor user experience.

[0003] Therefore, when the scale of skills reaches the level of "thousands and tens of thousands", traditional methods can no longer meet the requirements of accurate, efficient and low-latency intent matching. There is an urgent need for a method that can achieve accurate and efficient matching of user intents under massive skills. Summary of the Invention

[0004] This application provides a method, electronic device, and storage medium for accurate user intent matching for a large number of skills, in order to solve the problems of intent confusion, low efficiency, and maintenance difficulties in traditional LLM when the number of skills is large. It achieves high-precision, low-latency, and easily scalable intent matching for a large number of skills, significantly improving user experience and system robustness.

[0005] The primary objective of this application is to provide a method for accurately matching user intent across a vast array of skills.

[0006] The aforementioned objective of this application is achieved through the following technical solution: A method for accurate matching of user intent across a vast array of skills, the method comprising: Obtain the user's request information input through the interactive interface of the target AI agent; Based on the aforementioned demand information, a preset multi-level filtering mechanism is used to perform hierarchical filtering on all skills that the target AI agent can invoke, thereby obtaining a candidate skill set. The confidence level of each skill in the candidate skill set relative to the demand information is calculated based on the pre-trained large language model, and the target matching skill is determined based on the confidence level calculation results. Based on the confidence score calculation result, the corresponding skill invocation strategy is executed to complete the user's request response.

[0007] Preferably, the step of using a preset multi-level filtering mechanism based on the demand information to perform hierarchical filtering on all skills that the target AI agent can invoke, resulting in a candidate skill set, includes: Based on the aforementioned demand information, a keyword matching mechanism is used to perform an initial screening of all skills that the target AI agent can invoke, resulting in a preliminary skill set. Based on the aforementioned demand information, a speech similarity matching mechanism is used to perform intent-based fine screening on the initial skill set to obtain the candidate skill set.

[0008] Preferably, the initial screening of all skills that the target AI agent can invoke based on the demand information using a keyword matching mechanism yields the following skills set after initial screening: The requirement information is matched with a keyword matching list of all skills that the target AI agent can invoke to obtain the initial skill set, wherein... Each skill that the target AI agent can invoke is individually configured with a dedicated keyword matching list. Each keyword matching list stores multiple keywords that are strongly related to the corresponding skill. At least one keyword in the keyword matching list corresponding to the skills in the initial screening skill set matches the requirement information.

[0009] Preferably, the step of using a dialogue similarity matching mechanism based on the demand information to perform intent-based fine-tuning on the initially screened skill set to obtain the candidate skill set includes: The similarity between the demand information and the script list of all skills in the initially screened skill set is calculated, and the candidate skill set is determined based on the similarity calculation results. Each skill that the target AI agent can invoke is individually configured with a dedicated script list. Each script list stores multiple skill scripts that are strongly related to the corresponding skill. If any of the dialogue lists in the skills of the initial skill set contain a dialogue list with a similarity ≥ a preset similarity threshold to the requirement information, then the skills corresponding to all dialogue lists with a similarity ≥ the preset similarity threshold are selected as the candidate skill set. If there is no script list in the script list of each skill in the initial screening skill set that has a similarity to the requirement information greater than or equal to a preset similarity threshold, the initial screening skill set will be used as the candidate skill set.

[0010] Preferably, the step of calculating the similarity between the demand information and the script list of all skills in the initial screening skill set includes: The semantic understanding model is used to calculate the contextual semantic similarity between the demand information and the skill statements in the statement list of all skills in the initial screening skill set; The similarity of the keyword features between the demand information and the skill phrases of all skills in the initial skill set is calculated using an attention mechanism model. The weights of the calculated results of contextual semantic similarity and keyword feature similarity are dynamically adjusted by an adaptive gating mechanism to obtain the final similarity calculation result.

[0011] Preferably, the step of calculating the confidence level of each skill in the candidate skill set relative to the demand information based on a pre-trained large language model, and determining the target matching skill based on the confidence level calculation result, includes: When there is a list of dialogues in the dialogue list of each skill in the initial screening skill set that has a similarity to the requirement information greater than or equal to a preset similarity threshold, the confidence of each skill in the candidate skill set relative to the requirement information is calculated using the pre-trained large language model, and the unique candidate skill with the highest confidence is selected as the target matching skill. When there is no dialogue list in the dialogue list of each skill in the initial screening skill set that has a similarity to the demand information greater than or equal to a preset similarity threshold, the demand information is first matched with multiple preset domains using the pre-trained large language model to determine a unique target domain. Then, under the target domain, the demand information is matched with multiple subcategories corresponding to the target domain using the pre-trained large language model to determine a unique target subcategory. Finally, under the target subcategory, the confidence of multiple skills corresponding to the target subcategory relative to the demand information is calculated using the pre-trained large language model, and the unique candidate skill with the highest confidence is selected as the target matching skill.

[0012] Preferably, the confidence level is calculated by weighting multiple dimensions of parameters, and the calculation formula is as follows: Confidence=α·S+β·D+γ·H-λ·U Where Confidence is the confidence level, S is the semantic matching degree, D is the domain weight, H is the historical accuracy, and U is the uncertainty penalty; α, β, and γ are positive weighting coefficients, λ is the penalty coefficient, and α+β+γ=1.

[0013] Preferably, the step of executing the corresponding skill invocation strategy based on the confidence score calculation result includes: When the confidence score calculation result is greater than or equal to the preset confidence threshold A, the target matching skill is directly invoked without user confirmation; When the preset confidence threshold B < confidence calculation result < preset confidence threshold A, a confirmation request is sent to the user, and the target matching skill is invoked or the target matching skill is re-filtered based on the user's feedback; When the confidence score calculation result is less than or equal to the preset confidence score threshold B, the user is prompted to re-enter the required information. Among them, the preset reliability threshold A is greater than the preset reliability threshold B.

[0014] The second objective of this application is to provide an electronic device.

[0015] The second objective of this application is achieved through the following technical solution: An electronic device, comprising: The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method for precise matching of user intent for a large number of skills as described in the first objective of this application.

[0016] A third objective of this application is to provide a computer-readable storage medium.

[0017] The aforementioned objective three of this application is achieved through the following technical solution: A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method for precise matching of user intent for a large number of skills as described in the first objective of this application.

[0018] Compared with the prior art, this application has the following beneficial effects: 1. Through a three-level matching mechanism of initial screening, fine screening and confidence matching, it supports the identification of more than 1,000 skills without confusion, completely solving the problem of efficiency drop in traditional LLM after more than 10 skills; 2. Based on a domain-subclass hierarchical system, adding, deleting, or modifying skills only requires maintaining the corresponding document, making maintenance more convenient and efficient; 3. Utilize confidence-based metrics to drive skill retrieval strategies, balancing accuracy and interaction efficiency; 4. Users can invoke any skill with just one sentence using only natural language input, without needing to learn command syntax; 5. Modular design allows for independent optimization of each component, adapting to different AI platform architectures. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart of the main process of a method for accurately matching user intent with a large number of skills in an embodiment of this application; Figure 2 This is a detailed flowchart of a method for accurately matching user intent with a large number of skills, as described in an embodiment of this application. Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0021] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0022] In the embodiments provided in this application, it should be understood that the disclosed methods and systems can be implemented in other ways. The system embodiments described below are merely illustrative. For example, the division of units and modules is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or modules can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, and can be electrical, mechanical, or other forms.

[0023] In addition, each functional unit in the various embodiments of this application can be integrated into a single processor, or each unit can be a separate device, or two or more units can be integrated into a single device; each functional unit in the various embodiments of this application can be implemented in hardware or in the form of hardware plus software functional units.

[0024] Those skilled in the art will understand that all or part of the steps of the following method embodiments can be implemented by program instructions and related hardware. The aforementioned program instructions can be stored in a computer-readable storage medium. When the program instructions are executed, they perform the steps of the following method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0025] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "a plurality of" or "several" means two or more, unless otherwise explicitly specified.

[0026] This application provides a method for accurately matching user intent across a vast array of skills, such as... Figure 1-2 As shown, the method may include the following steps: S1, Obtain the user's requirement information input through the interactive interface of the target AI agent; Users input their needs through natural language into the front-end interactive interface of the target AI agent (such as a chat window, voice assistant, or APP input box), such as "Draw me a picture of a sunset at the beach" or "Please take a day off tomorrow." The target AI agent captures the original text in real time as the input source for subsequent intent recognition.

[0027] Specifically, in this embodiment, the interactive interface supports multimodal input, including at least one of text input, voice input, and image input; when the input is in non-text form, it is first converted into text format requirement information by the conversion module before entering the subsequent filtering process.

[0028] S2, based on the demand information, uses a preset multi-level filtering mechanism to classify and filter all the skills that the target AI agent can call, and obtains a candidate skill set; After obtaining the user's input requirements, the filtering process begins. First, based on the requirements, a pre-set multi-level filtering mechanism is used to categorize and filter all skills that the target AI agent can access, resulting in a candidate skill set.

[0029] By using a multi-level screening mechanism to filter all skills, the data processing burden of subsequent large language models for intent recognition is effectively reduced.

[0030] S3. Calculate the confidence of each skill in the candidate skill set relative to the demand information based on the pre-trained large language model, and determine the target matching skill based on the confidence calculation results. After obtaining a candidate skill set by classifying and filtering all skills through a multi-level screening mechanism, the unique target matching skill is further identified and matched through a large language model. Specifically, the confidence of each skill in the candidate skill set relative to the demand information is calculated based on the pre-trained large language model, and the target matching skill is determined based on the confidence calculation results.

[0031] S4 executes the corresponding skill invocation strategy based on the confidence calculation result to complete the user's request response.

[0032] Finally, based on the calculated confidence level, an appropriate skill invocation strategy is adopted to complete the response to user needs.

[0033] In this embodiment, different confidence calculation results correspond to different skill invocation strategies, which can not only effectively meet the diverse needs of users, but also improve the intelligence level and operating efficiency of the system while ensuring service quality.

[0034] In summary, this embodiment provides a method for accurately matching user intent across a vast array of skills. First, it acquires the user's demand information input through the target AI agent's interactive interface. Then, based on the demand information, it uses a pre-defined multi-level filtering mechanism to perform hierarchical filtering of all skills that the target AI agent can invoke, obtaining a candidate skill set. Next, it calculates the confidence level of each skill in the candidate skill set relative to the demand information based on a pre-trained large language model, and determines the target matching skill based on the confidence level calculation results. Finally, it executes the corresponding skill invocation strategy based on the confidence level calculation results to complete the user demand response.

[0035] This embodiment of the user intent accurate matching method for massive skills effectively solves the four core challenges of accuracy, efficiency, maintainability, and user experience in intent recognition under massive skill scenarios through an innovative combination of structured filtering, quantitative decision-making, and intelligent execution. It achieves high-precision, low-latency, and easily scalable intent matching under massive skills, significantly improving user experience and system robustness. It is applicable to a wide range of scenarios such as enterprise-level AI platforms, intelligent customer service, and personal assistants, and has strong technological advancement and industrial application value.

[0036] In one embodiment, in step S2, the step of using a preset multi-level filtering mechanism based on demand information to perform hierarchical filtering on all skills that the target AI agent can invoke, resulting in a candidate skill set including: Based on demand information, a keyword matching mechanism is used to perform an initial screening of all skills that the target AI agent can invoke, resulting in a preliminary skill set. When using a multi-level screening mechanism to filter all skills, the first step is to use a keyword matching mechanism based on the demand information to perform an initial screening of all skills that the target AI agent can call. This step quickly eliminates most irrelevant skills, reduces the burden of subsequent processing, significantly reduces the number of skills that need further analysis, and improves the speed and efficiency of the initial screening.

[0037] Based on demand information, a dialogue similarity matching mechanism is used to further refine the intent of the initial skill set, resulting in a candidate skill set.

[0038] For the skill set after initial screening, the intent of the skill set is further screened by the speech similarity matching mechanism, thereby filtering out skills whose speech does not match the demand information, further narrowing down the possible skill range and improving the quality of the final match.

[0039] In one embodiment, based on demand information, a keyword matching mechanism is used to perform an initial intent screening of all skills that the target AI agent can invoke, resulting in a pre-screened skill set including: The demand information is matched with a keyword matching list of all skills that the target AI agent can access, resulting in a preliminary skill set. Each skill that the target AI agent can invoke is individually configured with its own keyword matching list. Each keyword matching list stores multiple keywords that are strongly related to the corresponding skill. After initial screening, at least one keyword in the keyword matching list corresponding to the skills in the skill set matches the requirement information.

[0040] In this embodiment, during the initial skill screening, the user's input request information is compared with a keyword matching list of all available skills. For example, for a request to "reserve a restaurant seat," the system checks if it contains keywords such as "reservation" or "restaurant." If at least one keyword matches, the skill is retained for the next round of screening. This simple string matching quickly eliminates most irrelevant skills, reducing the burden of subsequent processing and significantly decreasing the number of skills requiring further analysis, thus improving the speed and efficiency of the initial screening.

[0041] In one embodiment, based on demand information, a speech similarity matching mechanism is used to further refine the intent screening of the initially screened skill set, resulting in a candidate skill set including: The similarity between the demand information and the script list of all skills in the initial skill set is calculated, and the candidate skill set is determined based on the similarity calculation results. Each skill that the target AI agent can invoke is individually configured with a dedicated script list. Each script list stores multiple skill scripts that are strongly related to the corresponding skill. If, in the initial skill set, there are skill lists in the script list of each skill with a similarity to the requirement information greater than or equal to a preset similarity threshold, then the skills corresponding to all script lists with a similarity greater than or equal to the preset similarity threshold will be considered as candidate skill sets. If there is no script list in the script list of each skill in the initial screening skill set that has a similarity to the requirement information greater than or equal to the preset similarity threshold, the initial screening skill set will be used as the candidate skill set.

[0042] Each skill that can be invoked by the AI ​​agent (such as "leave application", "AI drawing", "meal card balance inquiry") is configured with a unique list of scripts. This list is manually or semi-automatically constructed by operations personnel or algorithms based on historical user logs, high-frequency expressions, business scenarios, etc., and contains several typical user expressions that are strongly related to the skill (i.e., "skill scripts").

[0043] Example: 1. Skill Name: Leave Request Script List: ["I have something to do tomorrow, please take 1 day off", "I need to take sick leave", "I'm renovating my house next Wednesday, please take 2 days off", "I'm requesting 1 day of personal leave"] 2. Skill Name: AI Drawing Phrases: ["Draw me a landscape painting", "Generate a starry sky map", "Create an illustration of a sunset at the beach", "Draw a cartoon cat"] This embodiment of intent screening, based on the initial keyword screening which significantly reduces the range of skills, further filters out candidate skills that highly match the user's true intent through refined semantic comparison, while retaining the ability to tolerate vague or simplified expressions.

[0044] Based on the similarity calculation results, this embodiment determines the final candidate skill set using the following method: Scenario 1: High similarity in dialogue exists (≥ preset threshold, such as 80%) If, after initial screening, at least one skill in the skill set has a similarity ≥ a threshold (e.g., 0.8) with the user's input. Only these highly similar skills are retained as the candidate skill set, while the remaining skills from the initial screening are eliminated.

[0045] This method for determining the candidate skill set significantly reduces the LLM decision space and improves the efficiency and accuracy of subsequent matching.

[0046] Scenario 2: No highly similar statements (all < threshold) If the similarity of the wording for all skills is below the threshold (e.g., the user input is a vague expression such as "get a picture" or "check it out") If no skills are eliminated, the entire initial skill set will be used as a candidate skill set and submitted to the subsequent LLM for in-depth intent analysis (such as initiating a three-level reduction mechanism of "domain → subcategory → skill").

[0047] This method of determining the candidate skill set avoids misfiltering correct skills due to users' brief or non-standard expressions, thus ensuring system robustness.

[0048] In one embodiment, calculating the similarity between the demand information and the script list of all skills in the initial skillset includes: The semantic understanding model is used to calculate the contextual semantic similarity between the demand information and the skill statements in the statement list of all skills in the initial skill set; The similarity of keyword features between demand information and the skill statements of all skills in the initial skill set is calculated using an attention mechanism model. The weights of the calculated results of contextual semantic similarity and keyword feature similarity are dynamically adjusted by an adaptive gating mechanism to obtain the final similarity calculation result.

[0049] Based on the initial screening of the skill set, further determination is made as to which skills truly match the user's current intent. Due to the diversity of user expressions (such as colloquialisms, omissions, and synonym substitutions), relying solely on keywords or a single semantic model is prone to misjudgment. Therefore, this embodiment designs a dual-channel fusion similarity calculation architecture that balances overall semantic consistency (contextual understanding) and key intent word matching (keyword sensitivity), and achieves a dynamic balance of their weights through an adaptive gating mechanism to adapt to different types of user input.

[0050] Specifically, the similarity calculation process is as follows: Step 1: Calculate contextual semantic similarity using a semantic understanding model This example uses BERT-Whitening or a similar pre-trained semantic coding model.

[0051] Processing flow: Input user needs information (such as "I need to take a day off tomorrow") and skill-based phrases (such as "I need to ask for leave") into the semantic encoder respectively; Obtain its corresponding sentence vector representation; Whitening is performed on sentence vectors to eliminate redundant dimensions in the semantic space and improve the isotropy of vector distribution; Calculate the cosine similarity between the two to obtain the contextual semantic similarity, which is between [0,1].

[0052] The contextual semantic similarity calculation method in this embodiment can identify expressions that are semantically similar but use different words (such as "take a vacation" and "ask for leave", "generate an image" and "draw a picture"). Be sensitive to the overall structure and context of sentences, and avoid taking things out of context.

[0053] Example: User input: "Help me check how much money is left on my meal card" Skill response: "Cafe balance inquiry" Although there are no completely identical words, the semantic model can recognize that both refer to "checking meal card balance" and output high similarity.

[0054] Step 2: Calculate keyword feature similarity using an attention mechanism model. This embodiment uses an Attention-GRU (Gated Recurrent Unit with Attention) network.

[0055] Processing flow: User input and skill-related phrases are segmented and embedded into words respectively; Capture word order information by encoding sequences using GRU; An attention mechanism is introduced to automatically learn and weight keywords that play a decisive role in intent judgment (such as "ask for leave", "draw", "query", "reimburse" etc.). The similarity is calculated based on the weighted keyword vectors to obtain the keyword feature similarity, which is between [0,1].

[0056] The keyword feature similarity enhancement in this embodiment strengthens the contribution of core verbs / nouns and suppresses interference from irrelevant modifiers. It is particularly effective for short, imperative expressions (such as "draw a picture" or "check the balance") and can identify scenarios where there are few keywords but the intent is clear.

[0057] Example: User input: "Make a picture" skill description: "Generate a star map" Although the semantic model may score low due to information sparsity, the attention mechanism can focus on the keyword "picture" to improve the matching confidence.

[0058] Step 3: Fuse the dual-channel results using an adaptive gating mechanism This embodiment's gated network design: Construct a lightweight neural network (such as a single-layer MLP) with the user's original text or its statistical features (such as length, keyword density, and syntactic complexity) as input; Output a dynamic weight w∈[0,1] to adjust the contribution ratio of the two channels.

[0059] Example of adaptive logic: If the user input is long and semantically complete (e.g., "I would like to ask for a day off next Wednesday because I want to go house hunting"), then w is close to 1, emphasizing the contextual semantics; If the user input is short and highlights key words (such as "draw the sun" or "check meal card"), then w is close to 0, emphasizing keyword matching.

[0060] This embodiment provides a refined and adaptive semantic similarity calculation method to accurately measure the semantic relevance between user input (demand information) and preset dialogue for each skill during the dialogue matching stage. This scheme integrates deep contextual understanding capabilities and keyword sensitivity, and through a dynamic weight adjustment mechanism, enables similarity calculation to capture both the overall semantic intent and highlight key operational words, thereby significantly improving the accuracy and robustness of intent screening. This mechanism allows the system to automatically switch matching strategies based on input characteristics, combining flexibility and stability.

[0061] In one embodiment, the confidence level of each skill in the candidate skill set relative to the demand information is calculated based on a pre-trained large language model, and the target matching skill is determined based on the confidence level calculation results, including: When there is a list of dialogues in the dialogue list of each skill in the initial screening skill set that has a similarity to the requirement information greater than or equal to a preset similarity threshold, the confidence of each skill in the candidate skill set relative to the requirement information is calculated using a pre-trained large language model, and the unique candidate skill with the highest confidence is selected as the target matching skill. If, in the initial screening of the skill set, there is no list of dialogues in the skill set that has a similarity to the requirement information greater than or equal to a preset similarity threshold, the requirement information is first matched with multiple preset domains using a pre-trained large language model to determine a unique target domain. Then, within the target domain, the requirement information is matched with multiple subcategories corresponding to that target domain using the pre-trained large language model to determine a unique target subcategory. Finally, within the target subcategory, the confidence of multiple skills corresponding to that target subcategory relative to the requirement information is calculated using the pre-trained large language model, and the unique candidate skill with the highest confidence is selected as the target matching skill.

[0062] The target matching skill determination method in this embodiment is based on the scenario-based, structured confidence calculation and skill decision-making mechanism of the pre-trained large language model (LLM). For user input with different semantic clarity, it dynamically selects the most suitable intent recognition path, thereby achieving high-precision and high-robust target skill matching in a massive skill library.

[0063] This implementation intelligently switches between two LLM inference strategies—"direct matching" and "three-level progressive domain reduction"—based on the results of the speech similarity screening. The following is a detailed description of this embodiment: Traditional LLM typically uses a "full skill traversal + cue word injection" approach for intent recognition. When the number of skills is large, it is prone to problems such as intent confusion (multiple skill descriptions are similar), lengthy inference paths, and unstable output.

[0064] This embodiment divides user input into two typical scenarios through pre-emptive speech similarity judgment, and customizes the optimal LLM invocation strategy for each scenario: In scenarios with high semantic clarity: if the similarity of the statements is greater than or equal to a preset similarity threshold (e.g., 80%), the best candidates are selected directly from the candidate skill set. In scenarios with low semantic clarity: if the similarity of all statements is less than the preset similarity threshold, initiate a three-level progressive matching process: "Domain → Subcategory → Skill".

[0065] This divide-and-conquer design improves response speed in high-confidence scenarios while ensuring recognition accuracy under ambiguous representations.

[0066] The determination strategies for the two different scenarios are explained in detail below: Scenario 1: Highly similar statements exist; direct confidence ranking is used. Triggering condition: During the script screening stage, at least one skill's script has a similarity to the user's needs information that is greater than or equal to a preset threshold (e.g., 0.8).

[0067] Processing flow: 1. Use the candidate skill set (usually 1-5 items) selected by high similarity screening as the input range of LLM; 2. Construct structured prompts, including: Original user needs; Meta-information of candidate skills, including functional descriptions, applicable scenarios, and input parameters; 3. Call a pre-trained large language model (such as Qwen, Llama, etc.) and ask it to output the matching confidence score for each skill; 4. Select the unique skill with the highest confidence level as the target matching skill.

[0068] Example The user entered: "Please draw me an illustration of a sunset at the beach." Script matching: The script has a similarity of 0.92 to a specific script used by the "AI drawing" skill. Candidate skill set: {AI drawing, PPT enhancement, photo restoration} LLM evaluation output confidence scores: AI drawing (9.4) > PPT enhancement (6.1) > Photo restoration (5.3) Final choice: AI drawing The direct confidence ranking strategy in this scenario has the following advantages: 1. LLM only needs to make decisions within a very small candidate set, resulting in a fast response time; 2. Highly similar statements provide strong semantic anchors, making it difficult for LLM to deviate; 3. Avoid unnecessary domain classification costs.

[0069] Scenario 2: No highly similar dialogue, three-level progressive domain narrowing Triggering condition: The similarity between the wording of all the skills screened and the user input is less than the preset threshold, indicating that the user's expression is vague, brief or non-standard (such as "get a picture", "check it out", "help me with something").

[0070] Processing flow (three-level progression) Level 1: Domain-Level Matching Several high-level business areas are pre-defined (as described in the disclosure document): Office administration tasks (leave requests, expense reimbursements, approvals) Information inquiry services (weather, meal card, express delivery) Creative design (drawing, copywriting, video) Lifestyle services (food delivery, ride-hailing, ticket booking) Tools and aids (translation, OCR, calculation) LLM matches user needs with the definitions of various domains and outputs the most likely target domain (unique).

[0071] Level 2: Subcategory-Level Matching Within the defined target domain, further matching is performed to specific subcategories: For example, "Office Affairs" → {Leave Application, Expense Reimbursement, Meeting Room Booking} "Creative Design Category" → {Image Generation, Copywriting, PPT Creation} LLM determines a unique target subclass based on user input and subclass description.

[0072] Level 3: Skill-level confidence calculation Only within the skill set of this target subclass (usually ≤10 items), call LLM to calculate the confidence of each skill relative to the user's needs; The single skill with the highest confidence level will be selected as the final match result.

[0073] Example The user typed: "Check how much money is left in my card." The similarity of the wording was all <0.8 (meaning the expression was vague). Level 1: LLM determines it belongs to the "information query" category; Level 2: Matches the "Account Balance Inquiry" subcategory within this domain; Level 3: Among the skills such as "meal card balance", "bank card balance" and "campus card", "meal card balance inquiry" has the highest confidence level (8.7) when combined with the context (such as the user's frequently used locations and historical behavior). Final choice: Meal card balance inquiry The three-level progressive domain reduction strategy in this scenario has the following advantages: 1. By using the hierarchical constraints of domain → subclass → skill, the search space of LLM is significantly reduced, preventing unrestrained development; 2. Even if the user's expression is unclear, the business knowledge system can provide a backup plan; 3. Each step has a clear classification basis, which facilitates debugging and auditing.

[0074] This embodiment cleverly combines the capabilities of a large language model with structured business knowledge through a speech similarity-driven dual-mode LLM decision mechanism, effectively controlling computational complexity and interaction costs while ensuring the accuracy of intent recognition.

[0075] In one embodiment, the confidence level in step S3 is calculated by weighting multiple dimensions of parameters, and the calculation formula is as follows: Confidence=α·S+β·D+γ·H-λ·U Where Confidence is the confidence level, S is the semantic matching degree, D is the domain weight, H is the historical accuracy, and U is the uncertainty penalty; α, β, and γ are positive weighting coefficients, λ is the penalty coefficient, and α+β+γ=1.

[0076] This embodiment provides a multi-dimensional, quantifiable, and interpretable confidence calculation method for scientifically evaluating the matching reliability between candidate skills and user needs. This method integrates four key dimensions—semantic understanding, domain knowledge, historical behavior, and uncertainty risk—and uses a weighted linear combination to generate the final confidence score, providing a reliable basis for subsequent skill invocation strategies (such as direct execution, user confirmation, or re-entry). The meaning and value range of each parameter in the formula are explained below: Semantic matching degree S: means the degree of semantic similarity between user input and skill-based dialogue, with a value range of [0.0, 1.0]. Domain weight D: This represents the degree of fit between the business domain to which the skill belongs and the user's intent domain, with a value range of [0.0, 1.0] (high confidence domains can be enhanced to ≤1.2). Historical accuracy H: This represents the success rate of this skill in past executions (with time decay), and its value ranges from [0.0, 1.0]. Uncertainty penalty U: This measures the ambiguity or risk level of the current matching result, and its value ranges from [0.0, 1.0]. Positive weighting coefficients α, β, γ: These control the contribution ratio of each positive factor, satisfying α + β + γ = 1. They are usually set to α = 0.6, β = 0.25, and γ = 0.15. The penalty coefficient λ represents the strength of the suppression of confidence by the control of uncertainty, and is usually set to λ=0.3.

[0077] The final confidence value is normalized or thresholded and then used for decision-making (e.g., >9, 6~9, ≤6, etc.).

[0078] The confidence calculation scheme proposed in this embodiment is an intelligent evaluation system that integrates semantics, knowledge, experience and risk awareness. It effectively solves the problem of traditional LLM being overconfident or lacking evidence in intent matching, and provides core technical support for safe, efficient and reliable AI services in massive skill scenarios. It has significant technological advancement and industrial application value.

[0079] In one embodiment, step S4, executing the corresponding skill invocation strategy based on the confidence calculation result, includes: When the confidence score calculation result is greater than or equal to the preset confidence threshold A, the target matching skill is directly invoked without user confirmation; When the preset confidence threshold B < confidence calculation result < preset confidence threshold A, a confirmation request is sent to the user, and the target matching skill is invoked or the target matching skill is re-filtered based on the user's feedback; When the confidence score calculation result is less than or equal to the preset confidence score threshold B, the user is prompted to re-enter the required information. Among them, the preset reliability threshold A is greater than the preset reliability threshold B.

[0080] This embodiment provides a tiered skill invocation strategy mechanism based on quantified confidence levels. After user intent matching is completed, the mechanism intelligently determines subsequent interactive behaviors based on the system's confidence level in the matching results. This mechanism divides the confidence level into three intervals by setting two preset thresholds (A and B, where A>B), and configures differentiated execution strategies for each interval, thereby achieving a dynamic balance between automation efficiency, operational security, and user experience.

[0081] The following is a detailed description of this embodiment: I. Core Design Concept Traditional AI systems typically employ extreme strategies such as "total trust" or "total inquiry": Quanxin: LLM output is executed immediately → prone to errors due to mismatch (such as submitting expense reports incorrectly); Full confirmation: Requires user confirmation every time → cumbersome interaction, reduces efficiency.

[0082] This embodiment abandons the black-and-white approach mentioned above and instead introduces a confidence-driven three-level response mechanism, enabling the system to have the intelligent judgment ability to "know when to execute automatically, when to carefully confirm, and when to refuse to execute".

[0083] II. Specific Strategy Explanation 1. High confidence interval: Confidence ≥ threshold A (e.g., A=9) Execution strategy: directly call the target matching skill without user confirmation.

[0084] Applicable scenarios: The user's expression is clear and standardized; the semantic matching degree is high; the field of the skill is clear; the historical execution success rate is high; the uncertainty is low.

[0085] Typical example: User input: "Apply for one day's personal leave tomorrow" The system matches the "Leave Application" skill with a confidence level of 9.3 The system automatically fills in the form and submits the OA process.

[0086] Advantages of this skill call strategy: Achieve "zero interaction" efficient service; improve the user experience of high-frequency and low-risk operations.

[0087] 2. Medium confidence interval: Threshold B < Confidence < Threshold A (e.g., B = 6, A = 9) Execution strategy: Initiate a confirmation request to the user, for example: "Do you want to execute the [Reimbursement Form Submission] skill?" If the user replies "Yes" or clicks to confirm → execute the target skill; If the user denies (e.g., "No", "Cancel") → return to the process starting point and re-obtain or parse the user's requirements.

[0088] Applicable scenarios: The user's expression is ambiguous or may have multiple interpretations; multiple skills have similar semantics (e.g., "Drawing" and "PPT Beautification"); sensitive operations are involved (e.g., financial, approval-related skills); the historical accuracy rate fluctuates greatly.

[0089] Typical example: User input: "Help me process this invoice" System candidate skills: {Invoice Recognition, Reimbursement Submission, Image Enhancement} Final confidence level = 7.8 The system asks: "Do you want to use the [Reimbursement Form Submission] function?" Advantages of this skill call strategy: Avoid incorrect execution caused by misjudgment of medium confidence; Retain user control and enhance trust; Support negative feedback closed-loop and can be used to optimize the model.

[0090] 3. Low confidence interval: Confidence ≤ Threshold B (e.g., B = 6) Execution strategy: Do not execute any skills and prompt the user to re-enter the requirement information, for example: "I'm sorry, I didn't quite understand your needs. Please describe them more clearly, such as 'I want to ask for leave' or 'Please check my meal card balance.'" Applicable scenarios: When user input is too brief (e.g., "get something"); when the expression is vague or unclear; No valid keywords or wording match; high domain classification entropy (the system cannot determine which category it belongs to).

[0091] Typical example: User typed: "Hey, could you do this for me?" All skill confidence levels are ≤5.2 The system guides users to use more explicit commands.

[0092] Advantages of this skill activation strategy: To prevent the negative experience caused by guesswork-based execution; Proactively guide users to express themselves correctly and improve the quality of long-term interactions; Reduce the pressure on the backend system from invalid calls.

[0093] III. Threshold Setting and Business Adaptation Threshold A (high confidence threshold): usually set to 9.0, corresponding to the "highly reliable" scenario, requiring semantic matching degree > 0.95, domain weight > 0.85, and historical accuracy > 0.9.

[0094] Threshold B (low confidence threshold): Usually set to 6.0, as the dividing line between "unreliable" and "worth trying".

[0095] Configurability: A and B can be dynamically adjusted according to the business security level. For high-risk scenarios such as finance / medical: increase A (e.g., 9.5) and decrease B (e.g., 7.0); For low-risk scenarios such as entertainment / query: reduce A (e.g., 8.5) to improve fault tolerance.

[0096] The specific method of executing the corresponding skill invocation strategy based on the confidence score calculation result in this embodiment has the following advantages: 1. High-risk operations need to be confirmed, and low-confidence operations are blocked to avoid accidental execution; 2. High-confidence operation is fully automated, reducing unnecessary interactions; 3. The interaction intensity is dynamically adjusted based on the system's "confidence level," making it more natural and intelligent; 4. The source of confidence is transparent, which facilitates operational analysis and strategy optimization; 5. It has the ability to defend against ambiguous, erroneous, and malicious input.

[0097] In summary, this embodiment constructs an adaptive, reliable, and user-friendly skill invocation decision-making mechanism by dividing confidence thresholds and implementing a hierarchical execution strategy. This solution effectively addresses the core contradiction of "fast, accurate, and secure" in scenarios with a large number of skills.

[0098] like Figure 3 As shown, this application provides an electronic device 3, which includes a memory 301, a processor 302, and a computer program 303 stored in the memory 301 and executable on the processor 302. The memory 301 and the processor 302 communicate with each other via a bus 304. When the processor 302 executes the computer program 303, it implements the steps of the user intent precise matching method for massive skills as described in any of the above method embodiments of this application.

[0099] Specifically, the electronic device 3 can be an industrial control computer, PC, smart mobile terminal, or other intelligent device with memory and processor, or it can be a computer component with memory and processor such as a CPU or GPU. In this embodiment, the electronic device 3 is a smart mobile terminal, such as a smartphone.

[0100] The electronic device in this embodiment has the same technical effect as the user intent precise matching method for massive skills in the above embodiments, and will not be described again here.

[0101] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the user intent precise matching method for massive skills as described in any of the above method embodiments of this application.

[0102] The computer-readable storage medium in this embodiment has the same technical effect as the user intent precise matching method for massive skills in the above embodiments, and will not be described again here.

[0103] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0104] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0105] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0106] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for precise matching of user intent across a vast array of skills, characterized in that, The method includes: Obtain the user's request information input through the interactive interface of the target AI agent; Based on the aforementioned demand information, a preset multi-level filtering mechanism is used to perform hierarchical filtering on all skills that the target AI agent can invoke, thereby obtaining a candidate skill set. The confidence level of each skill in the candidate skill set relative to the demand information is calculated based on the pre-trained large language model, and the target matching skill is determined based on the confidence level calculation results. Based on the confidence score calculation result, the corresponding skill invocation strategy is executed to complete the user's request response.

2. The method for precise matching of user intent based on massive skill sets according to claim 1, characterized in that, The process of using a preset multi-level filtering mechanism to perform hierarchical filtering on all skills that the target AI agent can invoke based on the demand information, resulting in a candidate skill set including: Based on the aforementioned demand information, a keyword matching mechanism is used to perform an initial screening of all skills that the target AI agent can invoke, resulting in a preliminary skill set. Based on the aforementioned demand information, a speech similarity matching mechanism is used to perform intent-based fine screening on the initial skill set to obtain the candidate skill set.

3. The method for precise matching of user intent based on massive skill sets as described in claim 2, characterized in that, The initial intent screening of all skills that the target AI agent can invoke is performed using a keyword matching mechanism based on the demand information, resulting in a preliminary skill set including: The requirement information is matched with a keyword matching list of all skills that the target AI agent can invoke to obtain the initial skill set, wherein... Each skill that the target AI agent can invoke is individually configured with a dedicated keyword matching list. Each keyword matching list stores multiple keywords that are strongly related to the corresponding skill. At least one keyword in the keyword matching list corresponding to the skills in the initial screening skill set matches the requirement information.

4. The method for precise matching of user intent based on massive skill sets according to claim 2, characterized in that, The process of using a dialogue similarity matching mechanism to perform intent-based fine-tuning on the initially screened skill set based on the demand information, resulting in the candidate skill set, includes: The similarity between the demand information and the script list of all skills in the initially screened skill set is calculated, and the candidate skill set is determined based on the similarity calculation results. Each skill that the target AI agent can invoke is individually configured with a dedicated script list. Each script list stores multiple skill scripts that are strongly related to the corresponding skill. If any of the dialogue lists in the skills of the initial skill set contain a dialogue list with a similarity ≥ a preset similarity threshold to the requirement information, then the skills corresponding to all dialogue lists with a similarity ≥ the preset similarity threshold are selected as the candidate skill set. If there is no script list in the script list of each skill in the initial screening skill set that has a similarity to the requirement information greater than or equal to a preset similarity threshold, the initial screening skill set will be used as the candidate skill set.

5. The method for precise matching of user intent based on massive skill sets according to claim 4, characterized in that, The step of calculating the similarity between the demand information and the speech list of all skills in the initial screening skill set includes: The semantic understanding model is used to calculate the contextual semantic similarity between the demand information and the skill statements in the statement list of all skills in the initial screening skill set; The similarity of the keyword features between the demand information and the skill phrases of all skills in the initial skill set is calculated using an attention mechanism model. The weights of the calculated results of contextual semantic similarity and keyword feature similarity are dynamically adjusted by an adaptive gating mechanism to obtain the final similarity calculation result.

6. The method for precise matching of user intent based on massive skill sets according to claim 4, characterized in that, The step of calculating the confidence level of each skill in the candidate skill set relative to the demand information based on a pre-trained large language model, and determining the target matching skill based on the confidence level calculation results, includes: When there is a list of dialogues in the dialogue list of each skill in the initial screening skill set that has a similarity to the requirement information greater than or equal to a preset similarity threshold, the confidence of each skill in the candidate skill set relative to the requirement information is calculated using the pre-trained large language model, and the unique candidate skill with the highest confidence is selected as the target matching skill. When there is no dialogue list in the dialogue list of each skill in the initial screening skill set that has a similarity to the demand information greater than or equal to a preset similarity threshold, the demand information is first matched with multiple preset domains using the pre-trained large language model to determine a unique target domain. Then, under the target domain, the demand information is matched with multiple subcategories corresponding to the target domain using the pre-trained large language model to determine a unique target subcategory. Finally, under the target subcategory, the confidence of multiple skills corresponding to the target subcategory relative to the demand information is calculated using the pre-trained large language model, and the unique candidate skill with the highest confidence is selected as the target matching skill.

7. The method for precise matching of user intent based on massive skill sets according to claim 6, characterized in that, The confidence level is calculated by weighting multiple dimensions of parameters, and the calculation formula is as follows: Confidence=α·S+β·D+γ·H-λ·U Where Confidence is the confidence level, S is the semantic matching degree, D is the domain weight, H is the historical accuracy, and U is the uncertainty penalty; α, β, and γ are positive weighting coefficients, λ is the penalty coefficient, and α+β+γ=1.

8. The method for precise matching of user intent based on massive skill sets according to any one of claims 1-7, characterized in that, The execution of the corresponding skill invocation strategy based on the confidence level calculation result includes: When the confidence score calculation result is greater than or equal to the preset confidence threshold A, the target matching skill is directly invoked without user confirmation; When the preset confidence threshold B < confidence calculation result < preset confidence threshold A, a confirmation request is sent to the user, and the target matching skill is invoked or the target matching skill is re-filtered based on the user's feedback; When the confidence score calculation result is less than or equal to the preset confidence score threshold B, the user is prompted to re-enter the required information. Among them, the preset reliability threshold A is greater than the preset reliability threshold B.

9. An electronic device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the user intent precise matching method for massive skills as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method for precise matching of user intents for a massive number of skills as described in any one of claims 1-8.

Images