Question and answer method and device, terminal, storage medium and program product

Abstract

The embodiment of the invention discloses a question and answer method and device, a terminal, a storage medium and a program product, and relates to the field of artificial intelligence. The method is applied to a terminal and comprises the steps that under the condition that a query statement is received, the query type of the query statement is determined, the query type comprises a first query type and a second query type, and the query statement belonging to the first query type depends on searching for local data resources of the terminal; the query statements belonging to the second query type do not depend on searching local data resources; under the condition that the query statement belongs to the first query type, searching local data resources based on the query statement to obtain a resource search result; based on the resource search results and the query statements, query answers corresponding to the query statements are generated through a large language model; and under the condition that the query statement belongs to the second type of query statements, generating a query answer corresponding to the query statement through a large language model based on the query statement.

Description

Technical Field

[0001] Embodiments of the present application relate to the field of artificial intelligence, and in particular, to a question-and-answer method, apparatus, terminal, storage medium, and program product. Background Art

[0002] With the development of artificial intelligence technology, question-and-answer systems based on large language models have been widely used in many fields such as intelligent assistants and customer services. Among them, Retrieval-Augmented Generation (RAG) is a technology that combines local knowledge retrieval and generation models. By retrieving local data resources, the input of the large model is enriched, thereby generating more accurate and useful answers.

[0003] However, local knowledge retrieval on terminals faces a series of challenges. For example, the query statements input by users to the terminal may involve retrieving local data resources of different resource types, and the expression methods of the query statements are flexible and diverse, and the accuracy of local knowledge retrieval may not be sufficient. For another example, there are relatively large limitations in aspects such as the computing power, storage capacity, and battery life of the terminal, which pose higher requirements for the real-time performance and resource consumption of local knowledge retrieval. Summary of the Invention

[0004] Embodiments of the present application provide a question-and-answer method, apparatus, terminal, storage medium, and program product. The technical solutions are as follows:

[0005] On the one hand, embodiments of the present application provide a question-and-answer method, the method includes:

[0006] When a query statement is received, determining the query type of the query statement, the query type including a first query type and a second query type, wherein, query statements belonging to the first query type rely on searching the local data resources of the terminal, and query statements belonging to the second query type do not rely on searching the local data resources;

[0007] When the query statement belongs to the first query type, searching the local data resources based on the query statement to obtain a resource search result; based on the resource search result and the query statement, generating a query answer corresponding to the query statement through a large language model;

[0008] When the query statement belongs to the second type of query statement, generating the query answer corresponding to the query statement through the large language model based on the query statement.

[0009] On the other hand, embodiments of the present application provide a question-and-answer apparatus, the apparatus includes:

[0010] A classification module, configured to determine the query type of the query statement when receiving the query statement, where the query type includes a first query type and a second query type. Among them, the query statement belonging to the first query type depends on searching the local data resources of the terminal, and the query statement belonging to the second query type does not depend on searching the local data resources.

[0011] A search module, configured to search the local data resources based on the query statement when the query statement belongs to the first query type, and obtain a resource search result.

[0012] A generation module, configured to generate a query answer corresponding to the query statement through a large language model based on the resource search result and the query statement.

[0013] The generation module is further configured to generate the query answer corresponding to the query statement through the large language model based on the query statement when the query statement belongs to the second type of query statement.

[0014] On the other hand, an embodiment of the present application provides a terminal, where the terminal includes a processor and a memory, and at least one computer instruction is stored in the memory. The at least one computer instruction is loaded and executed by the processor to implement the method described in the above aspect.

[0015] On the other hand, an embodiment of the present application provides a computer-readable storage medium, in which at least one computer instruction is stored. The computer instruction is loaded and executed by a processor to implement the method described in the above aspect.

[0016] On the other hand, an embodiment of the present application provides a computer program product, which includes computer instructions stored in a computer-readable storage medium. The processor of the terminal reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the terminal executes the method provided in various optional implementation manners of the above aspect.

[0017] In the embodiments of the present application, when a query statement is received, the query type of the query statement is determined. When the query statement belongs to the first query type, it indicates that the query statement depends on searching the local data resources of the terminal. Therefore, the terminal inputs the resource search results obtained from the search of the local resource data and the query statement into the large language model, which can enable the large language model to generate a query answer in combination with the resource search results, thereby improving the accuracy of the query answer. When the query statement belongs to the second query type, it indicates that the query statement does not depend on searching the local data resources of the terminal. Therefore, the terminal directly inputs the query statement into the large language model to obtain a query answer, and no longer needs to search the local data resources, thereby greatly improving the generation efficiency of the query answer and reducing the resource consumption of local knowledge retrieval. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0019] Figure 1 is a logical schematic diagram of retrieval-augmented generation in the related art;

[0020] Figure 2 is a flowchart of a question-answering method provided by an exemplary embodiment of the present application;

[0021] Figure 3 is a schematic diagram of generating a query answer when the first query type includes a machine resource query type, a multimodal query type, and a document resource query type, and the second query type includes a chatting query type in an exemplary embodiment of the present application;

[0022] Figure 4 is a schematic diagram of determining the dependence degree of a query statement on the local data resources of each resource type in an exemplary embodiment of the present application;

[0023] Figure 5 is a schematic diagram of determining a candidate resource search sequence corresponding to the machine resource type in an exemplary embodiment of the present application;

[0024] Figure 6 is a schematic diagram of determining a candidate resource search sequence corresponding to the document resource type in an exemplary embodiment of the present application;

[0025] Figure 7 is a schematic diagram of determining a candidate resource search sequence corresponding to the multimodal resource type in an exemplary embodiment of the present application;

[0026] Figure 8 is a structural block diagram of a question-and-answer device provided by an exemplary embodiment of the present application;

[0027] Figure 9 is a structural block diagram of a terminal provided by an exemplary embodiment of the present application. Detailed implementation manners

[0028] To make the objectives, technical solutions, and advantages of the present application clearer, the following will further describe the embodiments of the present application in detail with reference to the accompanying drawings.

[0029] Retrieval-Augmented Generation (RAG) is a technology that combines local knowledge retrieval and a generation model. By retrieving local data resources, it enriches the input of the large model, thereby generating more accurate and useful answers.

[0030] See Figure 1 , Figure 1 is a logical schematic diagram of retrieval-augmented generation in the related art.

[0031] As Figure 1 shown, in the process of performing retrieval-augmented generation in the related art, when the user inputs a query statement "What are the main points worthy of attention in this year's financial report document?", the terminal retrieves in the local files of the terminal based on the query statement, and assembles the retrieved results (such as the full text of the financial report document) and the query statement with the prompt words, and then inputs the assembled prompt words into the large language model to obtain the answer generated by the large language model.

[0032] In the process of retrieval-augmented generation, the terminal needs to retrieve local files according to the query statement input by the user. However, the user may need to retrieve local data resources of different resource types, and the expression of the query statement is extremely flexible, which easily leads to poor accuracy of local knowledge retrieval. At the same time, the terminal has great limitations in computing power, storage capacity, and battery life, which pose more stringent requirements on the real-time performance and resource consumption of local knowledge retrieval.

[0033] Based on this, the present application proposes a question-and-answer method, which can enhance the accuracy of local knowledge retrieval, reduce the resource consumption of local knowledge retrieval, and improve the efficiency of retrieval-augmented generation.

[0034] See Figure 2 , Figure 2 is a flowchart of the question-and-answer method provided by an exemplary embodiment of the present application. In some embodiments, this method is executed by a terminal. The method includes the following steps.

[0035] Step 210, when a query statement is received, determine the query type of the query statement. The query type includes a first query type and a second query type. Among them, a query statement belonging to the first query type depends on searching the local data resources of the terminal, and a query statement belonging to the second query type does not depend on searching the local data resources.

[0036] Regarding the input method of the query statement, optionally, the user can input the query statement in any possible way such as typing input, voice input, or handwriting input, and there is no restriction on this.

[0037] Optionally, the local data resources include but are not limited to file resources on the terminal local (such as word files, pdf files, etc.), picture resources, video resources, audio resources, usage resources (including personalized setting data of the operating system and various application programs, which helps to adapt to the specific needs of users), database resources (such as various types of structured data stored locally), cache resources (such as common data segments temporarily stored by the terminal during operation), log resources, and data resources created by the user locally (such as reading notes, sticky notes, etc.).

[0038] A query statement belonging to the first query type depends on searching the local data resources of the terminal. Only for example, when the query statement is "Find the summary content in last year's financial report", generating the query answer corresponding to this query statement depends on the terminal retrieving the financial report file of last year from the local data resources. Another example, when the query statement is "Which page of the book was the last reading note recorded on", generating the query answer corresponding to this query statement depends on the terminal retrieving the reading note document and the corresponding book document from the local data resources.

[0039] Optionally, the first query type can include a usage resource query type. Only for example, when the query statements are "How to connect a Bluetooth headset", "The setting method of automatically adjusting the screen brightness according to the current time", "How to install a new application program", "The terminal's battery drains too fast. What methods can be used to optimize the battery life", etc., the query statements belong to the usage resource type in the first query type, and these query statements depend on retrieving the local data resources of the usage resource type of the terminal (such as usage setting entries).

[0040] Optionally, the first query type may include a multimodal resource query type. By way of example only, in cases where the query statement is "What color is the clothes I'm wearing in my ID photo", "What is the shooting date of the cruise ship photo taken in Mexico", "At what minute in the chemistry lecture recording I listened to yesterday was the preparation method of XX material mentioned", etc., the query statement belongs to the multimodal resource type in the first query type, and these query statements rely on retrieving the local data resources of the multimodal resource type of the terminal (such as pictures, audio, and video).

[0041] Optionally, the first query type may include a document resource query type. By way of example only, in cases where the query statement is "What does this article talk about", "What is worthy of note in that financial report", "Summarize the central idea of Chapter 3 of this book", the query statement belongs to the document resource type in the first query type, and these query statements rely on retrieving the local data resources of the document resource type of the terminal (such as word files, pdf files, EPUB files).

[0042] Query statements belonging to the second query type do not rely on searching the local data resources of the terminal.

[0043] Optionally, the second query type may include a chitchat query type. By way of example only, in cases where the query statement is "What's the weather like today", "What delicious food is nearby", "What do I need to prepare for a trip to Chengdu", etc., the query statement does not rely on searching the local data resources of the terminal.

[0044] Regarding the specific manner in which the terminal determines the query type of the query statement when receiving the query statement, in one possible scenario, the terminal deploys a query type recognition model, and the terminal inputs the query statement into the query type recognition model to obtain the query type output by the query type recognition model; alternatively, the terminal can also determine the query type of the query statement through natural language processing methods such as extracting keywords from the query statement and performing semantic analysis on the query statement, and there is no limitation on this.

[0045] Step 221, when the query statement belongs to the first query type, search the local data resources based on the query statement to obtain a resource search result; based on the resource search result and the query statement, generate a query answer corresponding to the query statement through a large language model.

[0046] When the query statement belongs to the first query type, since the query statement relies on searching the local data resources of the terminal, the terminal searches the local data resources based on the query statement.

[0047] Optionally, the search method includes at least one of an exact search method (such as an exact search for keywords), a fuzzy search method (such as a like search for keywords), and a vector semantic search method.

[0048] Optionally, the terminal can search all or part of the local data resources based on the query statement.

[0049] In some embodiments, the terminal can also determine the degree of dependence of the query statement on the local data resources of different resource types, and search the local data resources of different resource types according to the degree of dependence of the query statement on the local data resources of different resource types to obtain a resource search result. For more content on the terminal searching local data resources based on the query statement, see the following embodiments and their related descriptions, which will not be elaborated here.

[0050] Optionally, the resource search result can include the local data resources of one or more terminals.

[0051] Optionally, the resource search result can be displayed in the form of a list, where the list can include one or more local data resources.

[0052] Merely by way of example, when the query statement is "What are the notable points in last year's financial report", searching the local data resources of the terminal based on the query statement, the obtained resource search result can be displayed in the form of a list, and this list contains 3 documents, namely the financial report file of last year, the financial report file of this year, and the financial report analysis guide.

[0053] Optionally, the number of resources of the local data resources included in the resource search result can be determined by the terminal setting (for example, it is 6).

[0054] In some embodiments, the terminal can search the local data resources based on the query statement to determine the search scores of each local data resource for the query statement, and sort them in descending order to obtain a resource search result.

[0055] Merely by way of example, when the terminal sets the number of resources of the local data resources included in the resource search result to 6, the terminal determines the top 6 local data resources sorted in descending order as the resource search result.

[0056] In one possible implementation, the terminal inputs the resource search result and the query statement into a large language model respectively to obtain a query answer generated by the large language model corresponding to the query statement.

[0057] In another possible implementation, the terminal inputs the resource search result, the query statement, and a prompt after splicing them together into a large language model to obtain a query answer generated by the large language model.

[0058] Optionally, the large language model includes, but is not limited to, GPT series models such as GPT-3, GPT-3.5, GPT-4 and their descendant models, or Deep Seek series models, or any other possible pre-trained large language models, without limitation in this regard.

[0059] Step 222, when the query statement belongs to the second type of query statement, based on the query statement, through the large language model, generate a query answer corresponding to the query statement.

[0060] In some embodiments, when the query statement belongs to the second type of query statement, the query statement does not depend on the terminal to search the local data resources.

[0061] In one possible implementation, the terminal directly inputs the query statement into the large language model to generate a query answer corresponding to the query statement; in another possible implementation, the terminal inputs the query statement and the prompt words together into the large language model to generate a query answer corresponding to the query statement.

[0062] In some other embodiments, the terminal can also input the terminal device information and the query statement together into the large language model, and the large language model combines the terminal device information to generate a query answer that better meets the actual needs of the user.

[0063] Optionally, the terminal device information includes the current time, the current location, terminal device parameters (such as screen brightness, remaining battery power), etc.

[0064] For example only, when the query statement input by the user is "Recommend some delicious foods", the terminal can input the current location, the current time and the query statement together into the large language model, so that the query answer generated by the large language model can recommend dining restaurants near the current location and that are currently in business.

[0065] In summary, when a query statement is received, determine the query type of the query statement. When the query statement belongs to the first query type, it indicates that the query statement depends on searching the local data resources of the terminal. Therefore, the terminal inputs the resource search result obtained by searching the local resource data and the query statement into the large language model, which can enable the large language model to generate a query answer in combination with the resource search result, thereby improving the accuracy of the query answer; when the query statement belongs to the second query type, it indicates that the query statement does not depend on searching the local data resources of the terminal. Therefore, the terminal directly inputs the query statement into the large language model to obtain a query answer, and no longer needs to search the local data resources, thereby greatly improving the generation efficiency of the query answer and reducing the resource consumption of local knowledge retrieval.

[0066] Optionally, the first query type may include a machine resource query type, a multimodal query type, and a document resource query type, and the second query type may include a chat query type.

[0067] See Figure 3 , Figure 3 FIG. is a schematic diagram of generating a query answer provided by an exemplary embodiment of the present application when the first query type includes a machine resource query type, a multimodal query type, and a document resource query type, and the second query type includes a chat query type.

[0068] In some embodiments, the terminal determines the query type of the query statement.

[0069] Optionally, the terminal inputs the query statement into a query type recognition model to obtain the query type.

[0070] Among them, the query type recognition model is trained based on sample query statements and query type true values.

[0071] By way of example only, the query type true values corresponding to the sample query statements include 1 (representing the first query type) and 0 (representing the second query type). At this time, the query type recognition model is a binary classification model.

[0072] By way of example only, the query type true values corresponding to the sample query statements include 1 (representing the machine resource query type in the first query type), 2 (representing the multimodal resource query type in the first query type), 3 (representing the document resource query type in the first query type), and 0 (representing the chat query type in the second query type). At this time, the query type recognition model is a multi-classification model.

[0073] Optionally, the query type recognition model is a model obtained by fine-tuning the BERT model (Bidirectional Encoder Representations from Transformers) based on sample query statements and query type true values.

[0074] In some embodiments, when the query statement belongs to the first query type, the terminal determines the degree of dependence of the query statement on local data resources of different resource types.

[0075] Figure 3 Taking the different resource types including machine resource types, multimodal resource types, and document resource types as an example for illustration, but not constituting any limitation on the resource types, those skilled in the art can set the resource types according to actual needs.

[0076] In some embodiments, the terminal determines the degree of dependence of the query statement on local data resources of different resource types.

[0077] See Figure 4 , Figure 4 which is a schematic diagram for determining the dependency degree of a query statement on local data resources of each resource type provided by an exemplary embodiment of the present application.

[0078] As Figure 4 shown, the terminal can extract keywords from the query statement by using the word segmentation technology in natural language processing; use the part-of-speech recognition technology to identify the part of speech to which the keywords belong; and use the entity extraction method to determine the modification relationship between different keywords.

[0079] In some embodiments, the terminal can input the keywords in the query statement, the part of speech to which the keywords belong, and the modification relationship between different keywords into the resource type dependency degree prediction model to obtain the dependency degree of the query statement on local data resources of each resource type.

[0080] Optionally, the resource type dependency degree prediction model is a Logistic Regression Model.

[0081] In some embodiments, the resource type dependency degree prediction model outputs the dependency degree of the query statement on local data resources of each resource type. By way of example only, the dependency degree of query statement a on the device resource is 0.2, the dependency degree on the multimodal resource is 0.3, and the dependency degree on the document resource is 0.5.

[0082] In some embodiments, according to the dependency degree of the query statement on local data resources of different resource types, local data resources of different resource types are searched to obtain a resource search result.

[0083] In some embodiments, the terminal searches local data resources of each resource type according to the query statement to obtain a candidate resource search sequence corresponding to each resource type respectively.

[0084] Wherein, the candidate resource search sequence is a sequence obtained by sorting the candidate resource search results.

[0085] In a possible implementation manner, the terminal searches local data resources of a resource type by using different search paths for any one of the resource types according to the query statement to obtain a path resource search sequence corresponding to each different search path respectively.

[0086] Optionally, different search paths include at least one of different search methods, different objects to be searched, and different search objects.

[0087] Optionally, the search methods include at least one of an exact search method, a fuzzy search method, and a vector semantic search method.

[0088] For example only, the exact search method may be to perform an exact search on the keywords in the query statement, the fuzzy search method may be to perform a like search on the keywords in the query statement, or to search for synonyms of the keywords in the query statement, and the vector semantic search method may be a search method that matches the query semantic vector obtained by extracting from the query statement and the semantic vector obtained by extracting from the local data resource.

[0089] Optionally, the object to be searched includes at least one of the query statement, the keywords in the query statement, the synonyms of the keywords, and the words representing time and / or location in the query statement.

[0090] For example only, the terminal can directly use the query statement to search the local data resource, or can use only the keywords extracted from the query statement to search the local data resource, or can also use the synonyms of the keywords to search the local data resource, or can use the words representing time (such as time words representing holidays, absolute time words, recent time words, etc.) and / or location in the query statement to search the local data resource.

[0091] Optionally, the search object includes at least one level of local data resources, and different levels of local data resources have different data volumes.

[0092] For example only, when the search object is a document resource, the terminal can regard the entire document resource as the local data resource of the first level (or called the document resource at the doc level), and regard the multiple text chunks obtained by splitting the document resource as the local data resource of the second level (or called the document resource at the chunk level). Among them, the data volume of the local data resource of the second level is smaller than that of the local data resource of the first level. When performing a search, the terminal can choose to determine the local data resource of the first level as the search object, or determine the local data resource of the second level as the search object, or determine the local data resources of the first level and the second level together as the search object, and there is no restriction on this.

[0093] As Figure 3 shown, for different resource types of machine resources, multimodal resources, and document resources, the terminal respectively adopts different search paths to search the local data resources corresponding to the machine resources, multimodal resources, and document resources, and different search paths correspond to different path resource search sequences.

[0094] For the same resource type, local data resources of this resource type are searched through different search paths, and the path resource search sequences corresponding to each obtained search path are integrated into a candidate resource sequence. A more comprehensive candidate resource search sequence can be obtained for this resource type from different search paths, avoiding omissions in the search and improving the accuracy of the search for local data source resources of this resource type.

[0095] In some embodiments, the terminal determines a candidate resource search sequence corresponding to the resource type based on the path resource search sequences respectively corresponding to different search paths.

[0096] In a possible implementation, the terminal determines the candidate resource search sequence based on the quantity limit of the search results included in the candidate resource search sequence and the search scores of the search results in each path resource search sequence.

[0097] By way of example only, the terminal searches for usage resources using search path 1, search path 2, and search path 3, and obtains path resource search sequences a, b, and c respectively. Among them, the search scores corresponding to the 5 search results in path resource search sequence a are 90 points, 80 points, 70 points, 60 points, and 50 points respectively; the search scores corresponding to the 3 search results in path resource search sequence b are 99 points, 89 points, and 79 points respectively; the search scores corresponding to the 2 search results in path resource search sequence c are 100 points and 60 points respectively, and the quantity limit of the search results included in the candidate resource search sequence is 5. Then the terminal determines the 5 search results with search scores of 100 points, 99 points, 90 points, 89 points, and 79 points as the candidate resource search sequence corresponding to the usage resources.

[0098] After determining the candidate resource search sequences corresponding to each resource type, the terminal can integrate the candidate resource search sequences corresponding to each resource type according to the dependence degree of the query statement on the local data resources of different resource types to obtain a resource search result.

[0099] In a possible implementation, the terminal determines the allocation ratio corresponding to different resource types according to the dependence degree of the query statement on the local data resources of different resource types.

[0100] Among them, the allocation ratio is positively correlated with the dependence degree.

[0101] By way of example only, if the dependence degree of query statement a on usage resources is 0.2, the dependence degree on multimodal resources is 0.3, and the dependence degree on document resources is 0.5, then the terminal can determine that the allocation ratios of usage resources, multimodal resources, and document resources are 20%, 30%, and 50% respectively.

[0102] In some embodiments, the terminal filters out resource search results from the search results included in the candidate resource search sequences corresponding to different resource types according to the allocation ratios corresponding to the different resource types.

[0103] By way of example only, when the number of resource search results included in the resource search results is 10, the terminal filters out the top 2 search results from the candidate resource search sequence corresponding to the usage resources, the top 3 search results from the candidate resource search sequence corresponding to the multimodal resources, and the top 5 search results from the candidate resource search sequence corresponding to the document resources as the resource search results.

[0104] By querying the degree of dependence of the local data resources of different resource types, the candidate resource search sequences corresponding to each resource type are integrated to obtain the resource search results, which not only ensures the comprehensiveness of the search results in the resource search results, but also can focus on the resource types with higher dependence degrees, thereby improving the accuracy of the resource search results and providing high-quality input content for the subsequent large language model to generate query answers with reference to the resource search results.

[0105] To improve data security and compliance and protect the privacy and security of users, in a possible scenario, when the query statement belongs to the first query type, the terminal performs content security detection on the resource search results and the text content of the query statement; when the query statement belongs to the second query type, the terminal performs content security detection on the text content of the query statement.

[0106] Regarding the specific implementation method of content security detection, optionally, the terminal can pre-set a sensitive word list and perform sensitive word detection on the text content of the resource search results and / or the query statement based on the sensitive word list to ensure that the text content input to the large language model meets the requirements of security detection.

[0107] By way of example only, taking the resource search results as an example, the content security detection includes the following steps: (a) The terminal tokenizes each search result in the resource search results to obtain a vocabulary list. (b) Traverse the vocabulary list and check whether each word in the vocabulary list is in the sensitive word list. (c) If the search result contains a sensitive word, remove the search result from the resource search results.

[0108] In some embodiments, the terminal inputs the resource search results and / or the query statement after content security detection into the large language model to obtain a query answer.

[0109] Through content security detection, it is possible to control the resource search results at the content level, prevent inappropriate content from having a negative impact on the large model, and at the same time avoid the leakage of user privacy. Through sensitive word filtering, the security and reliability of the system can be effectively improved.

[0110] In some embodiments, the resource types include at least one of machine usage resource types, multimodal resource types, and document resource types.

[0111] The following will introduce one by one the specific determination methods of the candidate resource search sequences corresponding to the three resource types of machine usage resource types, multimodal resource types, and document resource types.

[0112] In some embodiments, when the resource type is a machine usage resource type, the data of the machine usage resource type in the local data resource is the machine usage data resource. The machine usage data resource includes a first-level machine usage data resource and a second-level machine usage data resource.

[0113] Among them, the first-level machine usage data resource includes the name and address of the setting entry, and the second-level machine usage data resource includes the attributes of the setting entry.

[0114] By way of example only, the first-level machine usage data resource can be the name and address of the Bluetooth setting entry, the name and address of the screen brightness setting entry, the name and address of the font size setting entry, or the name and address of the setting entry of any other possible machine usage data, and there is no limitation thereto. Optionally, the first-level machine usage data resource is also referred to as the machine usage data resource at the doc level.

[0115] By way of example only, the second-level machine usage data resource can be "Bluetooth is turned on", "The screen display mode is vivid", "The font size is 24", etc., and there is no limitation thereto. Optionally, the second-level machine usage data resource is also referred to as the machine usage data resource at the chunk level.

[0116] See Figure 5 , Figure 5 which is a schematic diagram of determining the candidate resource search sequence corresponding to the machine usage resource type provided by an exemplary embodiment of the present application.

[0117] Optionally, the terminal can extract keywords from the query statement through the word segmentation technology in natural language processing, and perform word segmentation on the first-level machine usage data resource to obtain the keywords in the query statement.

[0118] In some embodiments, the terminal searches the first-level machine usage data resource in an exact search manner according to the keywords in the query statement to obtain the first-path machine usage resource search sequence.

[0119] Optionally, the terminal can sort the multiple machine usage setting entries in the first-level machine usage data resource in the manner of dmp original inverted sorting to obtain the first-path machine usage resource search sequence.

[0120] Optionally, the terminal may filter the usage setting entries in the first-level usage data resources that do not meet the keyword matching degree condition by means of keyword matching degree filtering, so as to obtain the first-path usage resource search sequence.

[0121] Optionally, the terminal may determine the arrangement order of the usage setting entries in the first-path usage resource search sequence according to the keyword matching degree of the usage setting entries, and the higher the keyword matching degree, the more forward the arrangement order.

[0122] In some embodiments, the terminal searches the first-level usage data resources in a fuzzy search manner according to the query statement, so as to obtain the second-path usage resource search sequence.

[0123] Optionally, the terminal may first use the dmp conjunction processing technology to process the conjunctions (such as "because", "so", "and", etc.) in the query statement, and use the processed text for searching.

[0124] Optionally, the terminal may sort the multiple usage setting entries in the first-level usage data resources in the manner of dmp original setting like sorting, so as to obtain the second-path usage resource search sequence.

[0125] Optionally, the terminal may filter the usage setting entries in the first-level usage data resources that do not meet the keyword matching degree condition by means of like search matching degree filtering, so as to obtain the second-path usage resource search sequence.

[0126] Optionally, the terminal may determine the arrangement order of the usage setting entries in the second-path usage resource search sequence according to the like search matching degree of the usage setting entries, and the higher the keyword matching degree, the more forward the arrangement order.

[0127] In some embodiments, the terminal determines the fourth-path usage resource search sequence according to the N search results with the highest search scores in the first-path usage resource search sequence and the second-path usage resource search sequence.

[0128] Optionally, the search score of the search results in the first-path usage resource search sequence is determined based on the keyword matching degree, and the higher the keyword matching degree, the higher the search score; the search score of the search results in the second-path usage resource search sequence is determined based on the like search matching degree, and the higher the like search matching degree, the higher the search score.

[0129] Optionally, the terminal may also perform deduplication and sorting on the search results in the first-path usage resource search sequence and the second-path usage resource search sequence, and then determine the fourth-path usage resource search sequence based on the N search results with the highest search scores.

[0130] Among them, each search result in the fourth-path machine resource search sequence is the docID corresponding to the machine setting entry. For example only, the fourth-path machine resource search sequence includes 6 search results, which are docID1 corresponding to the Bluetooth setting entry, docID2 corresponding to the screen brightness setting entry, docID3 corresponding to the font size setting entry, docID4 corresponding to the playback volume setting entry, docID5 corresponding to the scheduled shutdown setting entry, and docID6 corresponding to the network connection setting entry.

[0131] In some embodiments, the terminal performs a search on the setting semantic vectors of the second-level machine data resources by using a vector semantic search method based on the query semantic vectors extracted from the query statement, and converts the searched second-level machine data resources into corresponding first-level machine data resources to obtain a third-path machine resource search sequence.

[0132] Optionally, the terminal uses an embedding layer to embed the query statement to obtain a query semantic vector, or the terminal can also use any other possible embedding model, which is not limited herein.

[0133] Optionally, the terminal pre-embeds the second-level machine data resources (i.e., the machine data resources at the chunk level) through an embedding layer to obtain setting semantic vectors, and stores the setting semantic vectors.

[0134] Optionally, the terminal can calculate the distance between the query semantic vector and the setting semantic vector according to the Euclidean distance, the Chebyshev distance, or any other possible metric method, and determine the vector semantic matching degree between the query semantic vector and the setting semantic vector according to the distance, and the distance is negatively correlated with the vector semantic matching degree.

[0135] Since the second-level machine data resources are machine data resources at the chunk level, the result obtained by using the vector semantic search method is the chunkID. Therefore, the terminal can also map the chunkID to the corresponding docID, that is, the first-level machine data resources corresponding to the second-level machine data resources.

[0136] Optionally, the terminal can select a specific number (such as 6) of search results from the first-level machine data resources corresponding to the second-level machine data resources as the third-path machine resource search sequence according to the vector scoring and sorting method.

[0137] In some embodiments, the terminal fuses the fourth-path machine resource search sequence and the third-path machine resource search sequence based on the sorting positions of the search results in the fourth-path machine resource search sequence and the third-path machine resource search sequence, and the fusion weight to obtain a candidate resource search sequence.

[0138] Regarding the specific methods of the machine resource search sequence for the fourth path and the machine resource search sequence for the third path, in some embodiments, the terminal may adopt a weight allocation strategy to fuse the search results of the machine resource search sequence for the fourth path and the search results in the machine resource search sequence for the third path to obtain a candidate resource search sequence.

[0139] By way of example only, if the weight allocation strategy indicates that the fusion weight corresponding to the search result of the machine resource search sequence for the fourth path is w1, and the fusion weight corresponding to the search result of the machine resource search sequence for the third path is w2, then the terminal may re - sort each search result according to the product of the search score of each search result in the machine resource search sequence for the fourth path and the fusion weight w1, and the product of the search score of each search result in the machine resource search sequence for the third path and the fusion weight w2, and select the top M search results as the candidate resource search sequence in the descending order after re - sorting.

[0140] Optionally, the search score of the search result may be determined according to the ranking order of the search result in its own sequence, and the higher the ranking order, the higher the search score.

[0141] By way of example only, the product of the search score of the search result and the fusion weight is the mixed - sorting score rrf_score, which is expressed by the following formula.

[0142] rrf_score = weight * (1 / (rank + c));

[0143] Wherein, weight is the fusion weight, (1 / (rank + c)) is the search score, rank is the ranking position of the search result in its own sequence, and c is a constant (for example, 60).

[0144] Regarding the determination method of the weight allocation strategy, in some embodiments, when the highest search score of the search results in the machine resource search sequence for the fourth path is greater than the score threshold, the terminal determines the first weight allocation strategy; when the highest search score is less than or equal to the score threshold, the terminal determines the second weight allocation strategy.

[0145] Among them, the fusion weight of the search results in the machine resource search sequence for the fourth path indicated by the second weight allocation strategy is lower than the fusion weight of the search results in the machine resource search sequence for the fourth path indicated by the first weight allocation strategy.

[0146] Optionally, the score threshold is a value preset by the terminal, such as 60 points.

[0147] For example, when the highest search score of the search results in the fourth path machine resource search sequence is 80 points, the terminal determines that in the first weight allocation strategy, the fusion weight w1 corresponding to the search results of the fourth path machine resource search sequence is 0.5, and the fusion weight w2 corresponding to the search results of the third path machine resource search sequence is also 0.5.

[0148] For example, when the highest search score of the search results in the fourth path machine resource search sequence is 50 points, the terminal determines that in the second weight allocation strategy, the fusion weight w1 corresponding to the search results of the fourth path machine resource search sequence is 0.3, and the fusion weight w2 corresponding to the search results of the third path machine resource search sequence is 0.7.

[0149] In this embodiment, whether the highest search score of the search results in the fourth path machine resource search sequence is greater than the score threshold can reflect the quality of the search results obtained by using the exact search method and the fuzzy search method. When the highest search score is greater than the score threshold, it indicates that the quality of the search results is relatively high, and the fusion weights of the search results in the fourth path machine resource search sequence obtained by using the exact search method and the fuzzy search method can be relatively increased; when the highest search score is less than or equal to the score threshold, it indicates that the quality of the search results is relatively low, and the fusion weights of the search results in the third path machine resource search sequence obtained by using the vector semantic search method can be relatively increased, so as to improve the accuracy of the obtained candidate resource search sequence, and further improve the accuracy of the subsequent determined resource search results.

[0150] In some embodiments, the resource type includes a document resource type, and the document data resources in the local data resources include first-level document data resources and second-level document data resources.

[0151] Among them, the first-level document data resources include document titles. Optionally, the first-level document data resources can also be referred to as document data resources at the doc level.

[0152] Among them, the second-level document data resources include document contents. Optionally, the second-level document data resources are content blocks obtained by dividing the document contents into blocks. Optionally, the second-level document data source resources can also be referred to as document data resources at the chunk level.

[0153] See Figure 6 , Figure 6 is a schematic diagram of determining a candidate resource search sequence corresponding to the document resource type provided by an exemplary embodiment of the present application.

[0154] The terminal can first perform searches through different search paths to obtain a first-path document resource search sequence and a second-path document resource search sequence.

[0155] Regarding the method for determining the first-path document resource search sequence, in some embodiments, the terminal can perform an exact search on the second-level document data resources according to the keywords, synonyms, and words representing time and / or location in the query statement, and, according to the query semantic vector extracted from the query statement, perform a vector semantic search on the second document semantic vector of the second-level document data resources to obtain the first-path document resource search sequence.

[0156] Among them, the search results in the first-path document resource search sequence are second-level document data resources (chunk-level document data resources).

[0157] Such as Figure 6 shown, the terminal performs an exact search on the second-level document data resources according to the keywords in the query statement to obtain the document resource search sequence A; performs an exact search on the second-level document data resources according to the synonyms of the keywords in the query statement to obtain the document resource search sequence B; performs an exact search on the second-level document data resources according to the holidays and absolute time words to obtain the document resource search sequence C.

[0158] For example only, the keyword in the query statement is "financial report", its synonym is "financial statement", the holiday word can be "Dragon Boat Festival", and the absolute time word can be "last year" or "six o'clock in the evening".

[0159] Optionally, the synonym is a word with a similar meaning generated by the terminal according to the keyword, or a word determined by the terminal according to the keyword and a pre-configured synonym table.

[0160] In some embodiments, the terminal can integrate the search results in the document resource search sequences A to C according to the search scores into sequence 1.

[0161] In some embodiments, the terminal arranges the search results in the path resource search sequences corresponding to different search paths in the candidate resource search sequence corresponding to the resource type according to the sorting priority.

[0162] Among them, in the case where the object to be searched in the search path is a word representing time and / or location, the search results in the path resource search sequence corresponding to the search path correspond to the fourth sorting priority (or called time word boosting).

[0163] When the object to be searched in the search path is a keyword, the search results in the search result corresponding path resource search sequence corresponding to the search path correspond to the fifth sorting priority.

[0164] When the object to be searched in the search path is a synonym of the keyword, the search results in the search result corresponding path resource search sequence corresponding to the search path correspond to the sixth sorting priority (or called synonym weight reduction).

[0165] Among them, the fourth sorting priority is higher than the fifth sorting priority, and the fifth sorting priority is higher than the sixth sorting priority.

[0166] For example, during the integration process, it is possible to control the fusion weight of the search results in document resource search sequence B to be lower than the fusion weight of the search results in document resource search sequence A (referred to as synonym weight reduction), and control the fusion weight of the search results in document resource search sequence C to be higher than the fusion weight of the search results in document resource search sequence A (referred to as time word weight increase), so that the obtained sequence 1 can contain more search results related to the time words in the query statement, and compared with the search results related to the keywords in the query statement, there are fewer search results related to the synonyms of the keywords, thus ensuring the rationality and accuracy of the search results in the sequence.

[0167] Optionally, the terminal controls the fusion weight of the search results in document resource search sequence B to be 0.3, the fusion weight of the search results in document resource search sequence A to be 1, and the fusion weight of the search results in document resource search sequence C to be 2. The terminal sorts the search results in document resource search sequences A - C in descending order according to the product of the search scores and the fusion weights of the search results, and obtains sequence 1.

[0168] As Figure 6 shown, the terminal performs a search on the second document semantic vector of the second-level document data resource (chunk-level document data resource) using the vector semantic search method according to the query semantic vector extracted from the query statement, and obtains document resource search sequence G.

[0169] Optionally, the query semantic vector is extracted by the terminal according to the query statement by the embedding layer, and the second document semantic vector is extracted by the terminal according to the embedding layer from the second-level document data resource.

[0170] In some embodiments, the terminal can sort the search scores of the search results in sequence 1 and document resource search sequence G from high to low, and retain the TOP6 results as the mixed arrangement result at the chunk level, that is, the first path document resource search sequence. Optionally, the terminal can also sort the search scores of the search results in sequence 1 and document resource search sequence G after weighting from high to low, and the weighting weight is determined by the terminal setting, and there is no limitation on this.

[0171] In some embodiments, the terminal searches the first-level document data resources in an exact search manner according to keywords, searches the first-level document data resources in a fuzzy search manner according to a query statement, a word representing time, and / or a location, and searches the first document semantic vector of the first-level document data resources in a vector semantic search manner according to a query semantic vector extracted from the query statement, to obtain a second-path document resource search sequence.

[0172] Among them, the search results in the second-path document resource sequence are the first-level document data resources (i.e., the document data resources at the doc level).

[0173] Such as Figure 6 in, the terminal searches the first-level document data resources in an exact search manner according to the keywords in the query statement, and obtains a document resource search sequence D by using the dmp original inverted index sorting method.

[0174] In addition, the terminal searches the first-level document data resources in a fuzzy search manner according to the query statement and a time word, and obtains a document resource search sequence E by using the dmp original setting like sorting method.

[0175] In some embodiments, the terminal fuses the search results in the document resource search sequence D and the document resource search sequence E according to the product of the search score and the fusion weight, to obtain sequence 2, where the fusion weight of the search results including the time word in the query statement is higher than the fusion weight of the search results not including the time word in the query statement.

[0176] Such as Figure 6 in, the terminal searches the first document semantic vector of the first-level document data resources (document data resources at the doc level) in a vector semantic search manner according to the query semantic vector extracted from the query statement, to obtain a document resource search sequence F.

[0177] In some embodiments, the terminal sorts the search results in sequence 2 and the document resource search sequence F in descending order according to the product of the search score and the fusion weight, to obtain a second-path document resource search sequence. Among them, the fusion weight of the search results including the time word in the query statement is higher than the fusion weight of the search results not including the time word in the query statement.

[0178] In the first-path document resource search sequence, each search result is a chunk-level document data resource. In the second-path document resource search sequence, each search result is a doc-level document data resource. The integration rules for the first-path document resource search sequence and the second-path document resource search sequence include the following three cases.

[0179] (1) When the first-level document data resource corresponding to the second-level document data resource in the first path document resource search sequence also belongs to the search results in the second path document resource search sequence, the second-level document data resource is arranged in the candidate resource search sequence with the first sorting priority.

[0180] For example, the terminal inversely looks up the docID corresponding to the chunkID of the search results in the first path document resource search sequence. If this docID is also hit by the search results in the second path document resource search sequence, then this docID is ranked with the highest priority.

[0181] (2) When the second-level document data resource corresponding to the first-level document data resource in the second path document resource search sequence does not belong to the search results in the first path document resource search sequence, the first second-level document data resource corresponding to the first-level document data resource is arranged in the candidate resource search sequence with the second sorting priority. Among them, the first sorting priority is higher than the second sorting priority.

[0182] If only the docID in the second path document resource search sequence is hit, but the chunkID corresponding to this docID in the first path document resource search sequence is not hit, then the terminal can arrange the first second-level document data resource corresponding to the first-level document data resource in the candidate resource search sequence with the second-highest priority.

[0183] For example, when the file corresponding to this docID can read the file content, the character content of the first chunk is filled as the content of this chunk.

[0184] (3) When the first-level document data resource corresponding to the second-level document data resource in the first path document resource search sequence does not belong to the search results in the second path document resource search sequence, the second-level document data resource is arranged in the candidate resource search sequence with the third sorting priority. Among them, the second sorting priority is higher than the third sorting priority.

[0185] For example, in the case where only the chunkID in the first path document resource search sequence is hit, but the docID corresponding to this chunkID in the second path document resource search sequence is not hit, the terminal arranges the chunkID with the lowest priority.

[0186] In some embodiments, the terminal can also search the local data resources of the resource type according to the conditional search terms in the query statement, and obtain the candidate resource search sequences corresponding to the resource types respectively.

[0187] Among them, the conditional retrieval terms include at least one of the resource modification time, resource download time, resource format, and resource data volume.

[0188] For example, in the case where the query statement is "Help me find financial pdf documents", the terminal searches for pdf documents in the local files and obtains Sequence 3 according to the search results.

[0189] Such as Figure 6 In, when the query statement hits the conditional retrieval terms, the terminal retrieves the search results that meet the conditional retrieval terms from the document data resources as Sequence 3, and determines the TOP K results obtained through the integration rule and the TOP P-K results in Sequence 3 as the candidate resource search sequence. Wherein, P is a positive integer, and K is a positive integer less than P.

[0190] In some embodiments, the query statement may also include recent time terms (such as the most recent 3 years, the most recent 2 months, etc.).

[0191] In some embodiments, when the query statement hits the recent time term, the terminal can also reorder each search result in the candidate resource search sequence according to the latest modification time to meet the user's needs.

[0192] For example, in the case where the query statement is "Analyze the financial reports of the most recent 3 years", the terminal takes the financial reports of this year, last year, and the previous ones as the top 3 search results in the candidate resource search sequence.

[0193] In some embodiments, the resource type also includes multimodal resource types, and the local data resources include multimodal data resources.

[0194] Optionally, the multimodal data resources include pictures, audio, video, or any other possible rich media resources, and there is no limitation on this.

[0195] Taking the multimodal data resource as a picture as an example, see Figure 7 , Figure 7 It is a schematic diagram for determining the candidate resource search sequence corresponding to the multimodal resource type provided by an exemplary embodiment of the present application.

[0196] Such as Figure 7 As shown, determining the candidate resource search sequence corresponding to the multimodal resource type includes at least one of the following methods.

[0197] (1) OCR recall.

[0198] In some embodiments, the terminal searches for the text extracted from the multimodal data resources according to the query statement using the first search path, and obtains the path resource search sequence corresponding to the first search path.

[0199] Among them, the search method of the first search path includes at least one of an exact search method and a vector semantic search method.

[0200] For example, for image resources, first use OCR technology to extract the text information in the image. Then, perform keyword exact matching and vector semantic similarity calculation on the extracted text information to obtain images related to the query. Keyword exact matching can quickly find images containing the query keywords, while vector semantic similarity calculation can find semantically related images.

[0201] (2) TAG recall.

[0202] In some embodiments, the terminal searches a knowledge graph composed of tag information (TAG) of different multimodal data resources according to the query statement by using a second search path, and obtains a path resource search sequence corresponding to the second search path.

[0203] Images are usually annotated with some tags (Tag), such as shooting time, location, scene, theme, etc. These tags can form a knowledge graph representing the semantic relationships between images. By searching for tags related to the query in the knowledge graph, corresponding images can be found. In addition, the time and location tags of the images can also be used for filtering and sorting in the time and space range.

[0204] (3) Text description recall.

[0205] In some embodiments, the terminal searches the text description of the multimodal data resources according to the query statement by using a third search path, and obtains a path resource search sequence corresponding to the third search path.

[0206] For example, Image Caption is a text description of the content of an image. By calculating the text vector similarity of the Image Caption, images semantically related to the query can be found.

[0207] (4) CLIP vector recall.

[0208] In some embodiments, the terminal searches the multimodal semantic vectors of the multimodal data resources according to the query semantic vector extracted from the query statement by using a fourth search path, and obtains a path resource search sequence corresponding to the fourth search path.

[0209] CLIP (Contrastive Language-Image Pre-training) is a pre-training model that maps images and text to the same semantic space. By encoding the query and the image into CLIP vectors respectively and calculating the similarity between them, images semantically related to the query can be found.

[0210] In this embodiment, the images obtained by searching in the above various ways can be integrated to obtain a candidate resource search sequence corresponding to the image resources.

[0211] See Figure 8 , Figure 8 which is a structural block diagram of a question-and-answer device provided by an exemplary embodiment of the present application. The device includes:

[0212] A classification module 801, configured to determine the query type of the query statement when receiving the query statement, where the query type includes a first query type and a second query type. Among them, the query statement belonging to the first query type depends on searching the local data resources of the terminal, and the query statement belonging to the second query type does not depend on searching the local data resources.

[0213] A search module 802, configured to search the local data resources based on the query statement when the query statement belongs to the first query type to obtain a resource search result.

[0214] A generation module 803, configured to generate a query answer corresponding to the query statement through a large language model based on the resource search result and the query statement.

[0215] The generation module 803 is further configured to generate the query answer corresponding to the query statement through the large language model based on the query statement when the query statement belongs to the second type of query statement.

[0216] Optionally, the search module 802 is configured to:

[0217] When the query statement belongs to the first query type, determine the degree of dependence of the query statement on the local data resources of different resource types.

[0218] Search the local data resources of different resource types according to the degree of dependence of the query statement on the local data resources of different resource types to obtain the resource search result.

[0219] Optionally, the search module 802 is configured to:

[0220] Search the local data resources of each resource type according to the query statement to obtain a candidate resource search sequence corresponding to each resource type, where the candidate resource search sequence is a sequence obtained by sorting the candidate resource search results.

[0221] Integrate the candidate resource search sequences corresponding to each of the resource types according to the dependence degrees of the local data resources of different resource types on the query statement to obtain the resource search result.

[0222] Optionally, the search module 802 is configured to:

[0223] According to the query statement, for any one of the resource types in each of the resource types, search the local data resources of the resource type using different search paths to obtain path resource search sequences corresponding to different search paths;

[0224] Based on the path resource search sequences corresponding to different search paths, determine the candidate resource search sequence corresponding to the resource type.

[0225] Optionally, different search paths include at least one of different search methods, different objects to be searched, and different search objects;

[0226] The search method includes at least one of an exact search method, a fuzzy search method, and a vector semantic search method;

[0227] The object to be searched includes at least one of the query statement, keywords in the query statement, synonyms of the keywords, and words representing time and / or location in the query statement;

[0228] The search object includes at least one level of the local data resources, and different levels of the local data resources have different data volumes.

[0229] Optionally, the resource type includes a machine usage resource type, the machine usage data resources in the local data resources include first-level machine usage data resources and second-level machine usage data resources, the first-level machine usage data resources include the names and addresses of setting entries, and the second-level machine usage data resources include the attributes of the setting entries; the search module 802 is configured to:

[0230] According to the keywords in the query statement, search the first-level machine usage data resources using the exact search method to obtain a first-path machine usage resource search sequence;

[0231] According to the query statement, search the first-level machine usage data resources using the fuzzy search method to obtain a second-path machine usage resource search sequence;

[0232] Based on the query semantic vector extracted from the query statement, perform a search on the set semantic vector of the second-level machine data resources using the vector semantic search method, and convert the second-level machine data resources obtained from the search into the corresponding first-level machine data resources to obtain the third-path machine resource search sequence.

[0233] Optionally, the search module 802 is used for:

[0234] Determine the fourth-path machine resource search sequence according to the N search results with the highest search scores in the first-path machine resource search sequence and the second-path machine resource search sequence;

[0235] Based on the sorting positions of the search results in the fourth-path machine resource search sequence and the third-path machine resource search sequence, and the fusion weights, fuse the fourth-path machine resource search sequence and the third-path machine resource search sequence to obtain the candidate resource search sequence.

[0236] Optionally, the search module 802 is used for:

[0237] When the highest search score of the search results in the fourth-path machine resource search sequence is greater than the score threshold, determine the first weight allocation strategy;

[0238] When the highest search score is less than or equal to the score threshold, determine the second weight allocation strategy, where the fusion weights of the search results in the fourth-path machine resource search sequence indicated by the second weight allocation strategy are lower than the fusion weights of the search results in the fourth-path machine resource search sequence indicated by the first weight allocation strategy.

[0239] Optionally, the resource type includes the document resource type, the document data resources in the local data resources include the first-level document data resources and the second-level document data resources, the first-level document data resources include document titles, and the second-level document data resources include document contents; the search module 802 is used for:

[0240] Perform an exact search on the second-level document data resources according to the keywords, synonyms, and words representing time and / or location in the query statement, and perform a vector semantic search on the second document semantic vector of the second-level document data resources using the query semantic vector extracted from the query statement to obtain the first-path document resource search sequence, and the search results in the first-path document resource search sequence are the second-level document data resources;

[0241] Search the first-level document data resources using the exact search method according to the keywords, and search the first-level document data resources using the fuzzy search method according to the query statement, the words representing time and / or location, and search the first-level document data resources using the vector semantic search method according to the query semantic vector extracted from the query statement to obtain a second-path document resource search sequence, and the search results in the second-path document resource sequence are the first-level document data resources.

[0242] Optionally, the search module 802 is used for:

[0243] When the first-level document data resources corresponding to the second-level document data resources in the first-path document resource search sequence also belong to the search results in the second-path document resource search sequence, arrange the second-level document data resources in the candidate resource search sequence with the first sorting priority;

[0244] When the second-level document data resources corresponding to the first-level document data resources in the second-path document resource search sequence do not belong to the search results in the first-path document resource search sequence, arrange the first second-level document data resource corresponding to the first-level document data resources in the candidate resource search sequence with the second sorting priority;

[0245] When the first-level document data resources corresponding to the second-level document data resources in the first-path document resource search sequence do not belong to the search results in the second-path document resource search sequence, arrange the second-level document data resources in the candidate resource search sequence with the third sorting priority;

[0246] Wherein, the first sorting priority is higher than the second sorting priority, and the second sorting priority is higher than the third sorting priority.

[0247] Optionally, the search module 802 is used for:

[0248] Arrange the search results in the path resource search sequences corresponding to different search paths in the candidate resource search sequence corresponding to the resource type according to the sorting priority;

[0249] Wherein, when the object to be searched in the search path is the word representing time and / or location, the search results in the path resource search sequence corresponding to the search path correspond to the fourth sorting priority;

[0250] When the object to be searched in the search path is the keyword, the search result in the path resource search sequence corresponding to the search path corresponds to the fifth sorting priority;

[0251] When the object to be searched in the search path is the near synonym of the keyword, the search result in the path resource search sequence corresponding to the search path corresponds to the sixth sorting priority;

[0252] Among them, the fourth sorting priority is higher than the fifth sorting priority, and the fifth sorting priority is higher than the sixth sorting priority.

[0253] Optionally, the resource type includes a multimodal resource type, and the local data resource includes a multimodal data resource; The search module 802 is used for at least one of the following:

[0254] According to the query statement, search the text extracted from the multimodal data resource by using the first search path to obtain the path resource search sequence corresponding to the first search path, and the search method of the first search path includes at least one of the exact search method and the vector semantic search method;

[0255] According to the query statement, search the knowledge graph composed of the label information of different multimodal data resources by using the second search path to obtain the path resource search sequence corresponding to the second search path;

[0256] According to the query statement, search the text description of the multimodal data resource by using the third search path to obtain the path resource search sequence corresponding to the third search path;

[0257] According to the query semantic vector extracted from the query statement, search the multimodal semantic vector of the multimodal data resource by using the fourth search path to obtain the path resource search sequence corresponding to the fourth search path.

[0258] Optionally, the search module 802 is used for:

[0259] According to the conditional retrieval term in the query statement, search the local data resource of the resource type to obtain the candidate resource search sequence corresponding to the resource type respectively;

[0260] Among them, the conditional retrieval term includes at least one of the resource modification time, resource download time, resource format, and resource data volume.

[0261] Optionally, the search module 802 is used for:

[0262] Determine the allocation ratio corresponding to each of the resource types according to the degree of dependence of the local data resources of different resource types on the query statement, where the allocation ratio is positively correlated with the degree of dependence;

[0263] Filter out the resource search results from the search results included in the candidate resource search sequences corresponding to each of the resource types according to the allocation ratio corresponding to each of the resource types.

[0264] Optionally, the search module 802 is configured to:

[0265] Input the keywords in the query statement, the part of speech to which the keywords belong, and the modification relationship between different keywords into a resource type dependence degree prediction model to obtain the degree of dependence of the query statement on the local data resources of each of the resource types.

[0266] Optionally, the classification module 801 is configured to:

[0267] Input the query statement into a query type recognition model to obtain the query type, where the query type recognition model is trained based on sample query statements and query type true values.

[0268] See Figure 9 , Figure 9 is a structural block diagram of a terminal provided by an exemplary embodiment of the present application.

[0269] The terminal can execute the question-and-answer method of the above embodiment. The terminal can be an electronic device that supports running a reading application, such as a smart phone, a laptop computer, a tablet computer, a navigation device, a smart TV, etc. The terminal may also be referred to by other names such as a user device, a portable terminal, etc. The terminal may include one or more of the following components: a processor 910 and a memory 920.

[0270] Optionally, the processor 910 is connected to various parts within the entire electronic device through various interfaces and circuits. By running or executing instructions, programs, code sets, or instruction sets stored in the memory 920, and by calling data stored in the memory 920, it performs various functions of the electronic device and processes data. Optionally, the processor 910 may be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), or programmable logic array (PLA). The processor 910 may integrate one or a combination of several of a central processing unit (CPU), a graphics processing unit (GPU), a neural-network processing unit (NPU), and a baseband chip, etc. Among them, the CPU mainly processes the operating system, user interface, application programs, etc.; the GPU is responsible for rendering and drawing the content to be displayed on the touch display screen; the NPU is used to implement artificial intelligence (AI) functions; the baseband chip is used to process wireless communications. It can be understood that the above baseband chip may not be integrated into the processor 910 and may be implemented separately by a single chip.

[0271] The memory 920 may include random access memory (RAM) and may also include read-only memory (ROM). Optionally, the memory 920 includes a non-transitory computer-readable storage medium. The memory 920 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 920 may include a program storage area and a data storage area. Among them, the program storage area may store instructions for implementing the operating system, instructions for at least one function (such as touch control function, sound playback function, image playback function, etc.), instructions for implementing the following various method embodiments, etc.; the data storage area may store data created according to the use of the electronic device (such as audio data, phone book, etc.).

[0272] In addition, those skilled in the art can understand that the structure of the terminal shown in the above drawings does not limit the terminal. The terminal may include more or fewer components than shown in the drawings, or combine certain components, or have different component arrangements.

[0273] An embodiment of the present application further provides a computer-readable storage medium, in which at least one instruction is stored, and the at least one instruction is loaded and executed by a processor to implement the method described in the above embodiment. Optionally, the computer-readable storage medium may include: ROM, RAM, solid state drives (SSDs), optical discs, etc. Among them, RAM may include resistive random access memory (ReRAM) and dynamic random access memory (DRAM).

[0274] An embodiment of the present application further provides a computer program product or a computer program. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method provided in various optional implementation manners of the above aspects.

[0275] The foregoing are only optional embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall be included within the protection scope of the present application.

Claims

1. A question-answering method, characterized in that, The method is applied to a terminal, and the method includes: When a query statement is received, determining the query type of the query statement, where the query type includes a first query type and a second query type. Among them, a query statement belonging to the first query type depends on searching the local data resources of the terminal, and a query statement belonging to the second query type does not depend on searching the local data resources; When the query statement belongs to the first query type, searching the local data resources based on the query statement to obtain a resource search result; based on the resource search result and the query statement, generating a query answer corresponding to the query statement through a large language model; When the query statement belongs to the second type of query statement, generating the query answer corresponding to the query statement through the large language model based on the query statement.

2. The method according to claim 1, characterized in that, The step of, when the query statement belongs to the first query type, searching the local data resources based on the query statement to obtain a resource search result includes: When the query statement belongs to the first query type, determining the degree of dependence of the query statement on the local data resources of different resource types; According to the degree of dependence of the query statement on the local data resources of different resource types, searching the local data resources of different resource types to obtain the resource search result.

3. The method according to claim 2, wherein The step of, according to the degree of dependence of the query statement on the local data resources of different resource types, searching the local data resources of different resource types to obtain the resource search result includes: According to the query statement, searching the local data resources of each resource type to obtain a candidate resource search sequence corresponding to each resource type respectively, where the candidate resource search sequence is a sequence obtained by sorting candidate resource search results; According to the degree of dependence of the query statement on the local data resources of different resource types, integrating the candidate resource search sequences corresponding to each resource type respectively to obtain the resource search result.

4. The method according to claim 3, wherein The step of, according to the query statement, searching the local data resources of each resource type to obtain a candidate resource search sequence corresponding to each resource type respectively includes: According to the query statement, for any one of the resource types among each resource type, using different search paths to search the local data resources of the resource type to obtain a path resource search sequence corresponding to each different search path; Based on the path resource search sequences corresponding to different search paths respectively, determining the candidate resource search sequence corresponding to the resource type.

5. The method according to claim 4, characterized in that, The different search paths include at least one of different search methods, different objects to be searched, and different search objects; The search method includes at least one of an exact search method, a fuzzy search method, and a vector semantic search method; The object to be searched includes at least one of the query statement, the keywords in the query statement, the synonyms of the keywords, and the words representing time and / or location in the query statement; The search object includes at least one level of the local data resources, and different levels of the local data resources have different data volumes.

6. The method according to claim 5, wherein The resource type includes machine usage resource types. The machine usage data resources in the local data resources include first-level machine usage data resources and second-level machine usage data resources. The first-level machine usage data resources include the names and addresses of the set items, and the second-level machine usage data resources include the attributes of the set items; According to the query statement, for any one of the resource types among the various resource types, different search paths are used to search the local data resources of the resource type, and path resource search sequences corresponding to the different search paths are obtained, including: According to the keywords in the query statement, the first-level machine usage data resources are searched by using the exact search method to obtain a first-path machine usage resource search sequence; According to the query statement, the first-level machine usage data resources are searched by using the fuzzy search method to obtain a second-path machine usage resource search sequence; According to the query semantic vector extracted from the query statement, for the set semantic vector of the second-level machine usage data resources, the vector semantic search method is used for searching, and the second-level machine usage data resources obtained by the search are converted into the corresponding first-level machine usage data resources to obtain a third-path machine usage resource search sequence.

7. The method according to claim 6, wherein Based on the path resource search sequences corresponding to different search paths, determining the candidate resource search sequence corresponding to the resource type includes: According to the N search results with the highest search scores in the first-path machine usage resource search sequence and the second-path machine usage resource search sequence, determining a fourth-path machine usage resource search sequence; Based on the sorting positions of the search results in the fourth-path machine usage resource search sequence and the third-path machine usage resource search sequence, and the fusion weights, the fourth-path machine usage resource search sequence and the third-path machine usage resource search sequence are fused to obtain the candidate resource search sequence.

8. The method according to claim 7, wherein The method further includes: When the highest search score of the search results in the fourth-path machine usage resource search sequence is greater than the score threshold, determining a first weight allocation strategy; When the highest search score is less than or equal to the score threshold, determining a second weight allocation strategy, where the fusion weight of the search results in the fourth-path machine usage resource search sequence indicated by the second weight allocation strategy is lower than the fusion weight of the search results in the fourth-path machine usage resource search sequence indicated by the first weight allocation strategy.

9. The method according to claim 5, wherein The resource types include document resource types. The document data resources in the local data resources include first-level document data resources and second-level document data resources. The first-level document data resources include document titles, and the second-level document data resources include document contents; According to the query statement, for any one of the resource types in each of the resource types, different search paths are used to search the local data resources of the resource type, and path resource search sequences corresponding to the different search paths are obtained, including: According to the keywords, synonyms, and words representing time and / or location in the query statement, the second-level document data resources are searched using the exact search method, and according to the query semantic vector extracted from the query statement, the second document semantic vector of the second-level document data resources is searched using the vector semantic search method to obtain a first-path document resource search sequence, and the search results in the first-path document resource search sequence are the second-level document data resources; According to the keywords, the first-level document data resources are searched using the exact search method, and according to the query statement and the words representing time and / or location, the first-level document data resources are searched using the fuzzy search method, and according to the query semantic vector extracted from the query statement, the first document semantic vector of the first-level document data resources is searched using the vector semantic search method to obtain a second-path document resource search sequence, and the search results in the second-path document resource sequence are the first-level document data resources.

10. The method according to claim 9, wherein Based on the path resource search sequences corresponding to the different search paths, determining the candidate resource search sequence corresponding to the resource type includes: In the case where the first-level document data resources corresponding to the second-level document data resources in the first-path document resource search sequence also belong to the search results in the second-path document resource search sequence, the second-level document data resources are arranged in the candidate resource search sequence with the first sorting priority; In the case where the second-level document data resources corresponding to the first-level document data resources in the second-path document resource search sequence do not belong to the search results in the first-path document resource search sequence, the first second-level document data resource corresponding to the first-level document data resources is arranged in the candidate resource search sequence with the second sorting priority; In the case where the first-level document data resources corresponding to the second-level document data resources in the first-path document resource search sequence do not belong to the search results in the second-path document resource search sequence, the second-level document data resources are arranged in the candidate resource search sequence with the third sorting priority; Among them, the first sorting priority is higher than the second sorting priority, and the second sorting priority is higher than the third sorting priority.

11. The method according to claim 9, wherein, Determining the candidate resource search sequence corresponding to the resource type based on the path resource search sequences respectively corresponding to different said search paths includes: Arranging the search results in the path resource search sequences respectively corresponding to different said search paths in the candidate resource search sequence corresponding to the resource type according to the sorting priority; Among them, when the object to be searched in the search path is a word representing time and / or location, the search results in the path resource search sequence corresponding to the search path correspond to the fourth sorting priority; When the object to be searched in the search path is the keyword, the search results in the path resource search sequence corresponding to the search path correspond to the fifth sorting priority; When the object to be searched in the search path is a synonym of the keyword, the search results in the path resource search sequence corresponding to the search path correspond to the sixth sorting priority; Among them, the fourth sorting priority is higher than the fifth sorting priority, and the fifth sorting priority is higher than the sixth sorting priority.

12. The method according to claim 5, wherein The resource type includes multimodal resource types, and the local data resources include multimodal data resources; According to the query statement, for any one of the resource types in each of the resource types, using different search paths to search the local data resources of the resource type to obtain path resource search sequences respectively corresponding to different said search paths includes at least one of the following: According to the query statement, using a first search path to search the text extracted from the multimodal data resources to obtain the path resource search sequence corresponding to the first search path, and the search method of the first search path includes at least one of the exact search method and the vector semantic search method; According to the query statement, using a second search path to search the knowledge graph composed of the label information of different multimodal data resources to obtain the path resource search sequence corresponding to the second search path; According to the query statement, using a third search path to search the text description of the multimodal data resources to obtain the path resource search sequence corresponding to the third search path; According to the query semantic vector extracted from the query statement, using a fourth search path to search the multimodal semantic vectors of the multimodal data resources to obtain the path resource search sequence corresponding to the fourth search path.

13. The method according to claim 3, characterized in that, Searching the local data resources of each of the resource types according to the query statement to obtain candidate resource search sequences respectively corresponding to each of the resource types includes; Searching the local data resources of the resource type according to the conditional retrieval words in the query statement to obtain the candidate resource search sequences respectively corresponding to the resource type; Among them, the conditional retrieval words include at least one of resource modification time, resource download time, resource format, and resource data volume.

14. The method according to claim 3, characterized in that, Integrating the candidate resource search sequences corresponding to each of the resource types according to the degree of dependence of the local data resources of different resource types on the query statement to obtain the resource search result, including: Determining the allocation ratio corresponding to different resource types according to the degree of dependence of the local data resources of different resource types on the query statement, where the allocation ratio is positively correlated with the degree of dependence; Filtering out the resource search result from the search results included in the candidate resource search sequences corresponding to each of the resource types according to the allocation ratio corresponding to different resource types.

15. The method according to any one of claims 2 to 14, characterized in that, The method further includes: Inputting the keywords in the query statement, the part of speech to which the keywords belong, and the modification relationship between different keywords into a resource type dependence degree prediction model to obtain the degree of dependence of the query statement on the local data resources of each of the resource types.

16. The method according to any one of claims 1 to 15, characterized in that, The method further includes: Inputting the query statement into a query type recognition model to obtain the query type, where the query type recognition model is trained based on sample query statements and query type true values.

17. A question-and-answer device, characterized in that, The device includes: A classification module, configured to determine the query type of the query statement when receiving the query statement, where the query type includes a first query type and a second query type. Among them, the query statement belonging to the first query type depends on searching the local data resources of the terminal, and the query statement belonging to the second query type does not depend on searching the local data resources; A search module, configured to search the local data resources based on the query statement when the query statement belongs to the first query type to obtain a resource search result; A generation module, configured to generate a query answer corresponding to the query statement through a large language model based on the resource search result and the query statement; The generation module is further configured to generate the query answer corresponding to the query statement through the large language model based on the query statement when the query statement belongs to the second type of query statement.

18. A terminal, characterized in that, The terminal includes a processor and a memory; the memory stores at least one computer instruction, and the at least one computer instruction is used to be executed by the processor to implement the question-answering method according to any one of claims 1 to 16.

19. A computer-readable storage medium, characterized in that, At least one computer instruction is stored in the computer-readable storage medium, and the computer instruction is loaded and executed by the processor to implement the question-answering method according to any one of claims 1 to 16.

20. A computer program product, characterized in that, The computer program product includes computer instructions, and the computer instructions are stored in a computer-readable storage medium; the processor of the terminal reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the terminal executes the question-answering method according to any one of claims 1 to 16.

Images