Application ability cognition and interaction method based on large language model

Abstract

The invention discloses an application ability cognition and interaction method based on a large language model. The invention discloses an application ability cognition and interaction method based on a large language model, which realizes more intelligent and efficient application cognition and interaction experience by utilizing strong language understanding and generation ability of the large language model. According to the method disclosed by the invention, the cognitive ability of the user in various application scenes can be improved, a more personalized and natural interaction mode can be provided, and more convenient and intelligent application experience is brought to the user.

Description

Technical Field

[0001] The present invention relates to an application capability recognition and interaction method, and in particular to an application capability recognition and interaction method based on a large language model. Background Art

[0002] Artificial Intelligence (AI) is an important driving force for a new round of scientific and technological revolution and industrial transformation. It is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. AI is an important part of the intelligent discipline. It attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. AI is a very broad science, including robots, speech recognition, image recognition, natural language processing, expert systems, machine learning, computer vision, etc.

[0003] Large Language Model (LLM) refers to a deep learning model trained with a large amount of text data, which can generate natural language text or understand the meaning of language text. Large language models can handle a variety of natural language tasks, such as text classification, question answering, and dialogue, and are an important path to artificial intelligence. Currently, large language models use a Transformer architecture and pre-training objectives (such as Language Modeling) similar to small models. The difference from small models is the increase in model size, training data, and computing resources.

[0004] With the continuous development of artificial intelligence technology, large language models have shown great application potential in various fields. However, how to effectively use large language models to improve application cognition and interactive experience is one of the current challenges. Summary of the invention

[0005] The technical problem to be solved by the present invention is: how to effectively use a large language model to improve application capability cognition and interactive experience.

[0006] In order to solve the above technical problems, the technical solution of the present invention is to disclose an application capability recognition and interaction method based on a large language model, which is characterized by comprising the following steps:

[0007] Step 1: Obtain the user's dialogue input data;

[0008] Step 2: Obtain page data, wherein the page data includes data in the main page currently opened by the user and data of all sub-pages contained in the main page;

[0009] Step 3: Generate a page description based on the page data obtained in step 2 and a predefined page description template;

[0010] Step 4: Generate an intent recognition instruction based on the page description generated in step 3, the user's dialogue input data obtained in step 1, and a predefined intent recognition instruction template;

[0011] Step 5: Based on the generated intent recognition instruction, the intent recognition module performs intent recognition and generates a page operation instruction and a text answer, which specifically includes the following steps:

[0012] Step 5.1, the intention recognition module calls the large model, and according to the function description of each main page in the page description in the intention recognition instruction obtained in step 4 and the user's dialogue input data, a list of main pages related to the user's dialogue input data is screened out from all the main pages;

[0013] Step 5.2: The intention recognition module calls the large model to calculate the confidence for each main page in the list of main pages obtained in step 5.1;

[0014] Step 5.3, the intention recognition module selects the main page with the highest confidence and a confidence value greater than a preset value from the list of main pages obtained in step 5.1;

[0015] Step 5.4, the intention recognition module calls the large model, and based on the main page selected in step 5.3, the functional descriptions of each sub-page in the page description in the intention recognition instruction obtained in step 4, and the user's dialogue input data, a list of sub-pages related to the user's dialogue input data is screened out from all the sub-pages included in the main page selected in step 5.3;

[0016] Step 5.5: The intention recognition module calls the large model to calculate the confidence for each sub-page in the list of sub-pages obtained in step 5.4;

[0017] Step 5.6, the intention recognition module selects the sub-page with the highest confidence and a confidence value greater than a preset value from the list of sub-pages obtained in step 5.4;

[0018] Step 5.7, the intention recognition module calls the large model, and based on the sub-page selected in step 5.6, the description of the operable actions of each sub-page in the page description in the intention recognition instruction obtained in step 4, and the user's dialogue input data, a list of operable actions related to the user's dialogue input data is screened out from all the operable actions included in the sub-page selected in step 5.6;

[0019] Step 5.8: The intention recognition module calls the large model to calculate the confidence for each operable action in the list of operable actions obtained in step 5.7;

[0020] Step 5.9, the intention recognition module selects the operable action with the highest confidence and a confidence value greater than a preset value from the list of operable actions obtained in step 5.7;

[0021] Step 5.10: The intention recognition module merges the selected main page obtained in step 5.3, the selected sub-page obtained in step 5.6, and the selected operable action obtained in step 5.9 to obtain a complete page operation instruction in JSON format;

[0022] Step 5.11, the intention recognition module calls the large model, and based on the sub-page selected in step 5.6, the definition and value of the data in each sub-page in the page description in the intention recognition instruction obtained in step 4, and the user's dialogue input data, all data contents related to the user's dialogue input data are filtered out from all the data contained in the sub-page selected in step 5.6;

[0023] Step 5.12: The intention recognition module calls the large model to summarize the page operation instructions obtained in step 5.10 and the data content obtained in step 5.11, and generates a text description of the page operation instructions and data content as a text answer to the user's dialogue input data;

[0024] Step 6: Based on the page operation instructions generated in step 5.10 and the text answers generated in step 5.12, perform corresponding operations on the page and display the text answers.

[0025] Preferably, in step 3, the content of the page specification template includes a function description of each main page, a function description of each sub-page, a description of the operable actions of each sub-page, a definition and a default value of the data of each main page, and a definition and a default value of the data in each sub-page, and step 3 includes the following steps:

[0026] Step 3.1, from the page specification template, copy a description of the content, wherein the description of the content includes the function description of each main page in the page specification template, the function description of each sub-page, and the description of the operable actions of each sub-page;

[0027] Step 3.2, based on the main page currently opened by the user, copy a copy of the data class content, wherein the data class content includes the definition and default value of the data of the main page currently opened by the user in the page specification template, and the definition and default value of the data of all sub-pages contained in the main page currently opened by the user;

[0028] Step 3.3, replace the default values ​​of all data in the content of the data class obtained in step 3.2 with the actual values ​​of the page data obtained in step 2, and obtain the content of the data class replaced with the actual values ​​of the page data;

[0029] Step 3.4: Merge the contents of the description class obtained in step 3.1 and the contents of the data class obtained in step 3.3 which have been replaced with the actual values ​​of the page data to obtain a page description.

[0030] Preferably, in step 4, the intention recognition instruction template includes a user input placeholder, a page description placeholder, a main page selection intention recognition prompt word, a sub-page selection intention recognition prompt word, and a sub-page operation action selection prompt word, then step 4 specifically includes the following steps:

[0031] Step 4.1, replacing the user input placeholder in the predefined intention recognition instruction template with the user's dialogue input data obtained in step 1, to obtain the intention recognition instruction template replaced with the actual user's dialogue input data;

[0032] Step 4.2: Based on the intention recognition instruction template obtained in step 4.1 and replaced with the actual user's dialogue input data, the page description placeholder therein is replaced with the page description generated in step 3 to obtain the intention recognition instruction.

[0033] Preferably, the large model used in step 5 is a model obtained after fine-tuning the Tongyi Qianwen 2.5 large language model using a self-made training data set, wherein the self-made training data set contains the intent recognition instructions and corresponding outputs that the large model needs to learn, and the output is a page operation instruction in json format.

[0034] Preferably, in step 6, the operation performed on the page specifically includes switching to the main page selected in the page operation instruction, displaying the sub-page selected in the page operation instruction, and executing the operation action in the sub-page selected in the page operation instruction.

[0035] The present invention discloses an application capability recognition and interaction method based on a large language model, which realizes a more intelligent and efficient application recognition and interaction experience by utilizing the powerful language understanding and generation capabilities of the large language model. The method disclosed in the present invention can not only enhance the user's cognitive ability in various application scenarios, but also provide a more personalized and natural interaction method, bringing a more convenient and intelligent application experience to users. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 A flowchart of an application capability recognition and interaction method according to an embodiment of the present invention;

[0037] Figure 2 Schematic diagram of an application page according to an embodiment of the present invention. DETAILED DESCRIPTION

[0038] The present invention will be further described below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms fall within the scope limited by the appended claims of the application equally.

[0039] The main purpose of the present invention is to provide a method for calling the functions of the application system based on the data and interface elements displayed by the application through the intention recognition of the large language model, so as to realize flexible function calling and data display, which can be widely used in various applications. At the same time, the present invention can dynamically adapt to various applications that often need to adjust the application functions according to changes in business needs.

[0040] like Figure 1 As shown, to achieve the above purpose, according to an embodiment of the present invention, a method for application capability recognition and interaction is proposed, comprising the following steps:

[0041] Step 1: Obtain the user's dialogue input data, where the dialogue input data is in the form of text and voice.

[0042] Step 2: Obtain page data, wherein the page data includes data in the main page currently opened by the user and data of all sub-pages contained in the main page. Figure 2 FIG. 1 is a schematic diagram of an application page of an embodiment of the present invention. Figure 2 As shown, the application includes 2 main pages, each of which includes 3 sub-pages. The main pages and sub-pages contain some data, and the sub-pages also contain some operable actions, such as clicking a button, selecting a drop-down box, etc. A user can open at most one main page at a time.

[0043] Step 3: Generate a page specification based on the page data obtained in step 2 and a predefined page specification template. In the embodiment of the present invention, the content of the page specification template includes a function description of each main page, a function description of each subpage, a description of the operable actions of each subpage, a definition and default value of the data of each main page, and a definition and default value of the data in each subpage. In the embodiment of the present invention, each subpage refers to a collection of subpages contained in all main pages.

[0044] In the embodiment of the present invention, step 3 specifically includes the following steps:

[0045] Step 3.1, copy a description content from the page description template, wherein the description content includes the function description of each main page in the page description template, the function description of each sub-page, and the description of the operable actions of each sub-page.

[0046] Step 3.2: Based on the main page currently opened by the user, a copy of the data class content is obtained, wherein the data class content includes the definition and default values ​​of the data of the main page currently opened by the user in the page specification template, and the definition and default values ​​of the data of all sub-pages contained in the main page currently opened by the user.

[0047] Step 3.3: Based on the content of the data class obtained in step 3.2 and the page data obtained in step 2, replace the default values, that is, replace the default values ​​of all data in the content of the data class obtained in step 3.2 with the actual values ​​of the page data obtained in step 2, and obtain the content of the data class that has been replaced with the actual values ​​of the page data.

[0048] Step 3.4: Merge the contents of the description class obtained in step 3.1 and the contents of the data class obtained in step 3.3 which have been replaced with the actual values ​​of the page data to obtain a page description.

[0049] Step 4: Generate an intent recognition instruction based on the page description generated in step 3, the user's dialogue input data obtained in step 1, and the predefined intent recognition instruction template. In the embodiment of the present invention, the intent recognition instruction template includes 5 parts: user input placeholder, page description placeholder, main page selection intent recognition prompt word, sub-page selection intent recognition prompt word, and sub-page operation action selection prompt word.

[0050] In the embodiment of the present invention, step 4 specifically includes the following steps:

[0051] Step 4.1, replace the user input placeholder in the pre-defined intention recognition instruction template with the user's dialogue input data obtained in step 1, and obtain the intention recognition instruction template that has been replaced with the actual user's dialogue input data.

[0052] Step 4.2: Based on the intention recognition instruction template obtained in step 4.1 and replaced with the actual user's dialogue input data, the page description placeholder therein is replaced with the page description generated in step 3 to obtain the intention recognition instruction.

[0053] Step 5. Based on the generated intent recognition instruction, the intent recognition module performs intent recognition and generates page operation instructions and text answers. In an embodiment of the present invention, the page operation instruction specifically includes three parts: main page selection instruction, sub-page selection instruction, and sub-page operation action selection instruction. The intention recognition module will call the big model for reasoning calculation in some sub-steps during the process of intent recognition. See the description of the sub-steps below for details. In an embodiment of the present invention, the big model called is a big model obtained by the inventor after pre-improvement based on the existing big model. The existing big model used specifically is the Tongyi Qianwen 2.5 big language model, and the specific improvement made is to use a self-made training data set to fine-tune the instruction training. The training data set contains the intent recognition instructions and corresponding outputs that the big model needs to learn, and the output is the page operation instructions in json format.

[0054] In the embodiment of the present invention, step 5 specifically includes the following steps:

[0055] Step 5.1, the intention recognition module calls the large model, and based on the function description of each main page in the page description in the intention recognition instruction obtained in step 4 and the user's dialogue input data, a list of main pages related to the user's dialogue input data is screened out from all main pages.

[0056] Step 5.2: The intention recognition module calls the large model to calculate the confidence for each main page in the list of main pages obtained in step 5.1. The higher the confidence of a main page, the higher the semantic relevance between the function description of the main page and the user's dialogue input data.

[0057] Step 5.3: The intention recognition module selects the main page with the highest confidence and a confidence value greater than 0.9 from the list of main pages obtained in step 5.1. If there is no main page that meets the conditions, the selected main page is empty.

[0058] Step 5.4: The intention recognition module calls the large model, and based on the main page selected in step 5.3, the functional description of each sub-page in the page description in the intention recognition instruction obtained in step 4, and the user's dialogue input data, a list of sub-pages related to the user's dialogue input data is filtered out from all the sub-pages contained in the main page selected in step 5.3.

[0059] Step 5.5: The intention recognition module calls the large model to calculate the confidence for each sub-page in the list of sub-pages obtained in step 5.4. The higher the confidence of a sub-page, the higher the semantic relevance between the functional description of the sub-page and the user's dialogue input data.

[0060] Step 5.6: The intention recognition module selects the sub-page with the highest confidence and a confidence value greater than 0.9 from the list of sub-pages obtained in step 5.4. If there is no sub-page that meets the conditions, the selected sub-page is empty.

[0061] Step 5.7: The intention recognition module calls the large model, and based on the sub-page selected in step 5.6, the description of the operable actions of each sub-page in the page description in the intention recognition instruction obtained in step 4, and the user's dialogue input data, a list of operable actions related to the user's dialogue input data is filtered out from all the operable actions contained in the sub-page selected in step 5.6.

[0062] Step 5.8: The intention recognition module calls the large model to calculate the confidence for each operable action in the list of operable actions obtained in step 5.7. The higher the confidence of an operable action, the higher the semantic relevance between the description of the operable action and the user's dialogue input data.

[0063] Step 5.9: The intention recognition module selects the operable action with the highest confidence and a confidence value greater than 0.9 from the list of operable actions obtained in step 5.7. If there is no operable action that meets the conditions, the selected operable action is empty.

[0064] Step 5.10: The intention recognition module merges the selected main page obtained in step 5.3, the selected sub-page obtained in step 5.6, and the selected operable action obtained in step 5.9 to obtain a complete page operation instruction in JSON format.

[0065] Step 5.11, the intention recognition module calls the large model, and based on the sub-page selected in step 5.6, the definition and value of the data in each sub-page in the page description in the intention recognition instruction obtained in step 4, and the user's dialogue input data, all data content related to the user's dialogue input data is filtered out from all the data contained in the sub-page selected in step 5.6.

[0066] Step 5.12: The intention recognition module calls the large model to summarize the page operation instructions obtained in step 5.10 and the data content obtained in step 5.11, and generates a text description of the page operation instructions and data content as a text response to the user's dialogue input data.

[0067] Step 6: Based on the page operation instruction generated in step 5.10 and the text answer generated in step 5.12, the page is operated accordingly and the text answer is displayed. The operation performed on the page specifically includes switching to the main page selected in the page operation instruction, displaying the sub-page selected in the page operation instruction, and executing the operation action in the sub-page selected in the page operation instruction.

Claims

1. A method for application capability recognition and interaction based on a large language model, characterized in that: The following steps are involved: Step 1: Obtain the user's dialogue input data; Step 2: Obtain page data, wherein the page data includes data in the main page currently opened by the user and data of all sub-pages contained in the main page; Step 3: Generate a page description based on the page data obtained in step 2 and a predefined page description template; Step 4: Generate an intent recognition instruction based on the page description generated in step 3, the user's dialogue input data obtained in step 1, and a predefined intent recognition instruction template; Step 5: Based on the generated intent recognition instruction, the intent recognition module performs intent recognition and generates a page operation instruction and a text answer, which specifically includes the following steps: Step 5.1, the intention recognition module calls the large model, and according to the function description of each main page in the page description in the intention recognition instruction obtained in step 4 and the user's dialogue input data, a list of main pages related to the user's dialogue input data is screened out from all the main pages; Step 5.2: The intention recognition module calls the large model to calculate the confidence for each main page in the list of main pages obtained in step 5.1; Step 5.3, the intention recognition module selects the main page with the highest confidence and a confidence value greater than a preset value from the list of main pages obtained in step 5.1; Step 5.4, the intention recognition module calls the large model, and based on the main page selected in step 5.3, the functional descriptions of each sub-page in the page description in the intention recognition instruction obtained in step 4, and the user's dialogue input data, a list of sub-pages related to the user's dialogue input data is screened out from all the sub-pages included in the main page selected in step 5.3; Step 5.5: The intention recognition module calls the large model to calculate the confidence for each sub-page in the list of sub-pages obtained in step 5.4; Step 5.6, the intention recognition module selects the sub-page with the highest confidence and a confidence value greater than a preset value from the list of sub-pages obtained in step 5.4; Step 5.7, the intention recognition module calls the large model, and based on the sub-page selected in step 5.6, the description of the operable actions of each sub-page in the page description in the intention recognition instruction obtained in step 4, and the user's dialogue input data, a list of operable actions related to the user's dialogue input data is screened out from all the operable actions included in the sub-page selected in step 5.6; Step 5.8: The intention recognition module calls the large model to calculate the confidence for each operable action in the list of operable actions obtained in step 5.7; Step 5.9, the intention recognition module selects the operable action with the highest confidence and a confidence value greater than a preset value from the list of operable actions obtained in step 5.7; Step 5.10: The intention recognition module merges the selected main page obtained in step 5.3, the selected sub-page obtained in step 5.6, and the selected operable action obtained in step 5.9 to obtain a complete page operation instruction in JSON format; Step 5.11, the intention recognition module calls the large model, and based on the sub-page selected in step 5.6, the definition and value of the data in each sub-page in the page description in the intention recognition instruction obtained in step 4, and the user's dialogue input data, all data contents related to the user's dialogue input data are filtered out from all the data contained in the sub-page selected in step 5.6; Step 5.12: The intention recognition module calls the large model to summarize the page operation instructions obtained in step 5.10 and the data content obtained in step 5.11, and generates a text description of the page operation instructions and data content as a text answer to the user's dialogue input data; Step 6: Based on the page operation instructions generated in step 5.10 and the text answers generated in step 5.12, perform corresponding operations on the page and display the text answers.

2. The method for application capability recognition and interaction based on a large language model as claimed in claim 1, characterized in that: In step 3, the content of the page specification template includes the function description of each main page, the function description of each sub-page, the description of the operable actions of each sub-page, the definition and default value of the data of each main page, and the definition and default value of the data in each sub-page, and step 3 includes the following steps: Step 3.1, from the page specification template, copy a description of the content, wherein the description of the content includes the function description of each main page in the page specification template, the function description of each sub-page, and the description of the operable actions of each sub-page; Step 3.2, based on the main page currently opened by the user, copy a copy of the data class content, wherein the data class content includes the definition and default value of the data of the main page currently opened by the user in the page specification template, and the definition and default value of the data of all sub-pages contained in the main page currently opened by the user; Step 3.3, replace the default values ​​of all data in the content of the data class obtained in step 3.2 with the actual values ​​of the page data obtained in step 2, and obtain the content of the data class replaced with the actual values ​​of the page data; Step 3.4: Merge the contents of the description class obtained in step 3.1 and the contents of the data class obtained in step 3.3 which have been replaced with the actual values ​​of the page data to obtain a page description.

3. The method for application capability recognition and interaction based on a large language model as claimed in claim 1, characterized in that: In step 4, the intention recognition instruction template includes a user input placeholder, a page description placeholder, a main page selection intention recognition prompt word, a sub-page selection intention recognition prompt word, and a sub-page operation action selection prompt word, then step 4 specifically includes the following steps: Step 4.1, replacing the user input placeholder in the predefined intention recognition instruction template with the user's dialogue input data obtained in step 1, to obtain the intention recognition instruction template replaced with the actual user's dialogue input data; Step 4.2: Based on the intention recognition instruction template obtained in step 4.1 and replaced with the actual user's dialogue input data, the page description placeholder therein is replaced with the page description generated in step 3 to obtain the intention recognition instruction.

4. The method for application capability recognition and interaction based on a large language model as claimed in claim 1, characterized in that: The large model used in step 5 is a model obtained by fine-tuning the Tongyi Qianwen 2.5 large language model using a self-made training data set, wherein the self-made training data set contains the intent recognition instructions and corresponding outputs that the large model needs to learn, and the output is a page operation instruction in json format.

5. The method for application capability recognition and interaction based on a large language model as claimed in claim 1, characterized in that: In step 6, the operation performed on the page specifically includes switching to the main page selected in the page operation instruction, displaying the sub-page selected in the page operation instruction, and executing the operation action in the sub-page selected in the page operation instruction.

Images