Information acquisition method and device, electronic equipment and computer readable medium

Abstract

The embodiment of the invention discloses an information acquisition method and device, electronic equipment and a computer readable medium. The method comprises the steps that first information input by a user is preprocessed, and key information in the first information is obtained; matching the key information with an intention library, and determining a target intention of the user; searching knowledge items associated with the target intention from a preset knowledge base; generating second information based on the target intention and the knowledge item; and inputting the second information into the large model to obtain response information of the large model to the second information. According to the embodiment, the information acquisition accuracy is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and specifically to information acquisition methods, devices, electronic devices, and computer-readable media. Background Art

[0002] With the rapid development of artificial intelligence technology, information processing and intent understanding are becoming increasingly important in many scenarios, especially in the fields of intelligent question answering and customer service. Large models are widely used to process complex language tasks due to their powerful language understanding and generation capabilities.

[0003] In existing technologies, user input is typically fed directly into a large model, which then generates a response. However, large models have high requirements for input quality, which can lead to inaccurate understanding of user intent in some scenarios, and consequently, inaccurate output responses. Summary of the Invention

[0004] The embodiments of the present application provide an information acquisition method, device, electronic device, and computer-readable medium, which can improve the accuracy of information acquisition.

[0005] In the first aspect, an embodiment of the present application provides an information acquisition method, which includes: preprocessing first information input by a user to obtain key information in the first information; matching the key information with an intent library to determine the target intent of the user; retrieving knowledge items associated with the target intent from a preset knowledge base; generating second information based on the target intent and the knowledge items; inputting the second information into a large model to obtain response information of the large model to the second information.

[0006] In the second aspect, an embodiment of the present application provides an information acquisition device, which includes: a preprocessing unit for preprocessing the first information input by the user to obtain key information in the first information; a determination unit for matching the key information with the intent library to determine the target intent of the user; a retrieval unit for retrieving knowledge items associated with the target intent from a preset knowledge base; a generation unit for generating second information based on the target intent and the knowledge items; an input unit for inputting the second information into a large model to obtain response information of the large model to the second information.

[0007] In a third aspect, an embodiment of the present application provides an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any embodiment of the first aspect.

[0008] In a fourth aspect, an embodiment of the present application provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any embodiment of the first aspect.

[0009] The information acquisition method, device, electronic device and computer-readable medium provided in the embodiments of the present application first pre-process the first information input by the user to obtain key information in the first information, which can reduce noise interference and provide an accurate data basis for subsequent intent recognition; then, by matching the key information with the intent library, the user's target intent is determined, which can fully consider the semantic similarity and correlation and improve the accuracy of identifying user needs; then, knowledge items associated with the target intent are retrieved from the preset knowledge base, which can provide targeted background knowledge for the large model and ensure that the generated content is consistent with user needs; then, based on the target intent and knowledge items, the second information is generated, which can make the information input into the large model more accurate and complete; finally, the second information is input into the large model to obtain the response information of the large model to the second information, so that the large model can generate more targeted and valuable response information based on more complete and accurate information, thereby improving the accuracy of information acquisition. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Other features, objects and advantages of the present application will become more apparent upon reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0011] Figure 1 is a flowchart of an embodiment of the information acquisition method of the present application;

[0012] Figure 2 is a flowchart of another embodiment of the information acquisition method of the present application;

[0013] Figure 3 It is a structural diagram of an embodiment of the information acquisition device of the present application;

[0014] Figure 4 It is a structural diagram of an electronic device used to implement an embodiment of the present application. DETAILED DESCRIPTION

[0015] All actions of acquiring signals, information or data in this application are carried out in compliance with the relevant data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.

[0016] The present application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are intended only to illustrate the relevant invention and are not intended to limit the invention. It should also be noted that, for ease of description, only portions relevant to the relevant invention are shown in the accompanying drawings.

[0017] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0018] Please refer to Figure 1 , which illustrates a process 100 according to one embodiment of the information acquisition method of the present application. This information acquisition method can be applied to various electronic devices with data processing capabilities. For example, such electronic devices may include, but are not limited to, smartphones, tablet computers, laptop computers, car computers, PDAs, desktop computers, set-top boxes, smart TVs, cameras, wearable devices, and the like.

[0019] The information acquisition method includes the following steps:

[0020] Step 101: pre-process the first information input by the user to obtain key information in the first information.

[0021] In this embodiment, the first information is the unprocessed information initially input by the user. The first information may be accurately expressed information or may contain redundant content, noise, or ambiguous expressions. The first information may be text or voice, without specific limitation.

[0022] Key information is the content extracted from the first information input by the user that can reflect the core needs of the user. It can be filtered keywords, phrases, or expressions with key semantics.

[0023] Preprocessing involves the initial cleaning, organization, and conversion of raw user input to extract key content, remove irrelevant information, and convert it into a form suitable for subsequent processing. Specifically, preprocessing may include, but is not limited to, at least one of the following: word segmentation, stop word removal, punctuation cleaning, low-frequency word removal, keyword extraction, semantic filtering, and speech-to-text conversion, among other steps, though these are not specifically defined here.

[0024] For example, if a user enters "I want to learn about the latest developments and applications of artificial intelligence in the medical field," removing stop words and low-frequency words yields "I want to learn about the latest developments and applications of artificial intelligence in the medical field." Keyword extraction yields "Learn about artificial intelligence, medical field, new developments, applications." Semantic filtering retains key information, yielding "artificial intelligence, medical field, developments, applications."

[0025] In some optional implementations of this embodiment, step 101 may include the following sub-steps:

[0026] Sub-step 1011: segment the first information input by the user to obtain a segmentation result.

[0027] Word segmentation refers to the process of splitting continuous natural language text into word units with independent semantic and syntactic attributes. Word segmentation methods such as Jieba can be used to perform word segmentation on the first information to obtain a word segmentation result. The word segmentation result can be a word sequence.

[0028] As an example, when the user inputs "I want to know about the development of artificial intelligence in the medical field", after performing word segmentation using Jieba, the word segmentation result is "I want / to know / about / artificial intelligence / in / the medical / field / of / development / situation".

[0029] Sub-step 1012, remove stop words and low-frequency words from the word segmentation result to obtain an initial word set.

[0030] Stop words refer to words that appear frequently in the text but contribute little to the semantics, such as meaningless words like "of", "了", "in", "and". In natural language processing, it is usually necessary to remove stop words to reduce the data dimension, improve processing efficiency, and the accuracy of semantic expression. The stop word list can be customized according to specific application scenarios.

[0031] Low-frequency words refer to words that appear infrequently in the text corpus and are usually considered to be words with particularity or low generality. Removing low-frequency words can reduce noise and highlight high-frequency and important words. The word frequency threshold can be set according to the corpus size and application requirements. For example, words with a word frequency less than 2 are set as low-frequency words.

[0032] Specifically, a stop word list can be established first, and the stop word list contains common stop words. Then, remove the stop words from the word segmentation result. Then, count the word frequency of each word in the word segmentation result in the text corpus, and determine the low-frequency words in the word segmentation result according to the set word frequency threshold. Finally, remove the low-frequency words from the word segmentation result, and the remaining words can be summarized into an initial word set.

[0033] Continuing with the above word segmentation result as an example, using the stop word list, stop words "I want", "about", "in", "of" can be removed. Setting the word frequency threshold to 2, after counting the word frequency, it is found that the word frequencies of "to know" and "situation" are low and they are removed, obtaining the initial word set "artificial intelligence, medical, field, development".

[0034] Sub-step 1013, extract keywords from the initial word set to obtain a keyword set.

[0035] Specifically, the TF-IDF (Term Frequency-Inverse Document Frequency) algorithm can be used to calculate the weight of each word in the initial word set. TF (Term Frequency) represents the frequency of a word in a text, and IDF (Inverse Document Frequency) is used to measure the importance of a word in the entire corpus. The calculation formula for TF-IDF is:

[0036] TF-IDF(t i ,d j )=TF(t i ,d j )×IDF(t i )

[0037] Among them, TF(t i ,d j ) is the word t in the initial word set i In the document d j The number of occurrences in document d j The ratio of the total number of words in IDF(t i )=log(N / contains words t i N is the total number of documents. In practice, a corpus containing 10,000 documents can be used to calculate the IDF value.

[0038] According to the TF-IDF value, the words with a value higher than the threshold δ can be retained as keywords to obtain a keyword set, which is recorded as K. That is, K = {t i |TF-IDF(t i ,d j )≥δ}. i takes the value of 1, 2, ..., N in sequence. For example, δ may take the value of 0.1.

[0039] Sub-step 1014: extracting the word vector of each keyword in the keyword set based on the pre-trained word embedding model.

[0040] Word embedding models, such as Word2Vec and BERT (Bidirectional Encoder Representations from Transformer), are pre-trained on large-scale corpora and are used to map words into a low-dimensional vector space. These models capture the semantic information and context of words, placing words with similar meanings closer together in the vector space. This provides effective semantic representation for natural language processing tasks.

[0041] A word vector is a low-dimensional vector obtained by converting words through a word embedding model. Each dimension represents the value of a word in different semantic features. The word vector can be used to calculate the similarity between words, perform semantic reasoning, etc.

[0042] Specifically, the pre-trained word embedding model can be used to map the words in the keyword set K to the vector space to obtain the word vector set in, For the word t i Word embedding models usually store words and their corresponding vectors in the form of key-value pairs, and quickly obtain word vectors through word indexing or hash lookup.

[0043] Sub-step 1015 , filtering the keyword set based on the first similarity between word vectors to obtain key information.

[0044] Specifically, we can first calculate the cosine similarity between the word vectors of each keyword in the keyword set and use it as the first similarity. Then, we obtain a pre-set similarity threshold, such as 0.6, and remove keywords whose first similarity with other keywords is lower than the threshold. The formula for calculating cosine similarity is:

[0045]

[0046] Among them, “·” represents the dot product of vectors, and “‖·‖” represents the modulus of the vectors. For the word t i word vectors. For the word t j The word vector of . Sim() represents the cosine similarity calculation function. j takes the values ​​1, 2, ..., N in sequence.

[0047] Through word segmentation, the first information can be initially semantically segmented and lexicalized, providing a basic unit for subsequent processing; by removing stop words and low-frequency words, irrelevant or low-value information can be effectively filtered out, highlighting the important content in the first information; through keyword extraction, the importance of words can be quantified and the keywords of the text can be screened out to achieve semantic concentration and topic focus; through the extraction of word vectors by pre-trained word embedding models, keywords can be mapped to a low-dimensional vector space to achieve keyword vectorization, facilitating subsequent semantic filtering; through semantic filtering based on word vector similarity, key information can be further optimized to ensure that the extracted key information has a high degree of semantic consistency and relevance.

[0048] By performing a series of operations such as word segmentation, removing stop words and low-frequency words, extracting keywords, extracting word vectors, and filtering based on word vector similarity, the key information in the first information input by the user can be gradually extracted, effectively improving the accuracy and semantic relevance of key information extraction, providing high-quality input for subsequent intent recognition and knowledge retrieval, and thus improving the performance and effectiveness of the entire information acquisition method.

[0049] Step 102: Match the key information with the intent library to determine the user's target intent.

[0050] In this embodiment, the intent library stores a collection of various user intents. Each intent can be a structured question text, which is defined and organized to correspond to a specific question pattern or requirement. The target intent is the content that the user wishes to express or achieve, determined after analysis and matching. It is an understanding and summary of the user's needs and can be expressed in the form of structured question text.

[0051] Specifically, the key information obtained through preprocessing can first be converted into a semantic vector. For example, a pre-trained deep learning model, such as the BERT model, can be used to encode the key information and obtain its semantic vector representation. Each intent in the intent library can also be converted into a semantic vector using the same model in advance or in real time. Similarity calculation methods such as cosine similarity and Euclidean distance can then be used to calculate the similarity between the semantic vector of the key information and the semantic vectors of each intent in the intent library. Finally, based on the similarity calculation results, the user's target intent can be determined.

[0052] For example, the intent in the intent library with the highest similarity to the target information can be determined based on the similarity calculation results. If the similarity exceeds a set threshold, the intent can be determined as the user's target intent. As another example, a specified number of intents can be selected from the intent library in descending order of similarity and displayed, allowing the user to manually select the target intent.

[0053] In some optional implementations of this embodiment, step 102 may include the following sub-steps:

[0054] Sub-step 1021 : determining a second similarity between the key information and each intent in the intent library.

[0055] Specifically, the cosine similarity calculation method can be used to calculate the similarity between the semantic vector corresponding to the key information and the semantic vector of each intent in the intent library as the second similarity. For details, please refer to the above description and will not be repeated here.

[0056] Sub-step 1022: If the maximum value of the second similarity is less than the first threshold, a first number of first candidate intents are selected from the intent library in descending order of the second similarity.

[0057] The first threshold is the similarity threshold used to determine the degree of intent matching. Reaching or exceeding this threshold means that the key information highly matches the corresponding intent. The first threshold can be determined based on validation set testing. For example, by testing the similarity distribution of the model on the validation set and determining that when the similarity score is greater than 0.85, the intent recognition accuracy reaches 95%, the first threshold can be set to 0.85 to ensure that highly matching intents can be accurately identified.

[0058] Specifically, it is first determined whether the maximum value of the calculated second similarity is less than a first threshold. If the maximum value of the second similarity is less than the first threshold, it means that there is no intent in the intent library that highly matches the key information. At this time, a first number of first candidate intents can be selected from the intent library in descending order of the second similarity. The first number can be set as needed. For example, the first five intents can be selected as the first candidate intents.

[0059] It should be noted that if the maximum value of the second similarity is greater than or equal to the first threshold, the first information entered by the user can be determined as the user's target intent. It can be understood that the setting of the first threshold sets a high standard for intent matching. When the maximum value of the second similarity reaches or exceeds this threshold, it means that the user's first information input has a clear intent. In this case, the first information is directly determined as the user's target intent, without further screening and user selection, reducing processing steps and improving response speed.

[0060] Sub-step 1023, extracting a second candidate intention from the selected first candidate intention, the second candidate intention having a similarity with the key information greater than a second threshold.

[0061] The second threshold is lower than the first threshold. For example, the second threshold can be set to 0.6. If the second similarity between an intent in the intent library and the key information is lower than the second threshold, it means that the intent is poorly correlated with the key information and can be considered not the user's true intention and ignored. The second candidate intent is an intent in the first candidate intent whose second similarity with the key information is between the first and second thresholds.

[0062] It should be noted that if, among the selected first candidate intentions, there is no second candidate intention whose similarity with the key information is greater than the second threshold, a prompt message may be displayed, which may be used to prompt the user to re-enter the question. It is understandable that, among the selected first candidate intentions, there is no second candidate intention whose similarity with the key information is greater than the second threshold, that is, the second similarity between each first candidate intention and the key information is less than the second threshold, which means that there is no intention in the intent library that can accurately match the user's needs. Displaying the prompt message at this time can guide the user to re-enter a clearer question, so that the system can regain a clearer user intention and avoid subsequent processing of invalid or inaccurate intentions.

[0063] Sub-step 1024 , displaying the extracted second candidate intent.

[0064] Specifically, the extracted second candidate intents can be displayed in a list or other form in the user interface for the user to view and select. In practice, each intent can be described concisely to help the user quickly understand it.

[0065] Sub-step 1025, determining the user's target intention based on the user's selection operation on the displayed second candidate intention.

[0066] Specifically, after the extracted second candidate intents are displayed, the user's operation in the user interface can be detected in real time. When a user's selection operation on any displayed second candidate intent is detected, the second candidate intent can be determined as the user's target intent.

[0067] By setting the first threshold and selecting the first candidate intent, we can further focus on the most likely set of intents while ensuring a certain matching quality, thereby improving the accuracy and efficiency of matching; by setting the second threshold and extracting the second candidate intent, we can filter the first candidate intent again to ensure that only intents with high relevance to key information can enter the user's field of view; by displaying the second candidate intent and determining the target intent based on the user's selected operation, we can ensure that the final target intent matches the user's real needs, thereby achieving accurate intent recognition and personalized services.

[0068] Step 103: Retrieve knowledge items associated with the target intent from the knowledge base.

[0069] In this embodiment, the knowledge base is a pre-built collection of knowledge items, which contains a large number of knowledge items. These knowledge items cover various topics, concepts, and information, and are used to provide the knowledge needed to answer user questions or meet user needs. A knowledge item can be a semantic unit extracted from a knowledge document, specifically a knowledge point, a paragraph of text, a data record, or information in a semantically encoded form. As an example, the knowledge document can first be segmented according to predetermined rules to extract multiple knowledge items. For example, the knowledge document can be segmented into N = 1000 knowledge items.

[0070] In this embodiment, knowledge items associated with the target intent can be retrieved from the knowledge base. Specifically, the similarity between each knowledge item in the knowledge base and the knowledge item associated with the target intent can be determined. If the similarity is greater than a threshold, the knowledge item can be determined to be associated with the target intent.

[0071] In some optional implementations of this embodiment, step 103 may include the following sub-steps:

[0072] Sub-step 1031 , determining the third similarity between the target intention and each knowledge item in the knowledge base.

[0073] Specifically, we can first use a pre-trained deep learning model, such as BERT, GPT, etc., to vectorize the target intent and obtain the semantic vector of the user intent, which is recorded as V u Furthermore, the same method can be used to convert each knowledge item in the knowledge base into a corresponding semantic vector in advance or in real time. Let the i-th knowledge item be K i , whose semantic vector can be expressed as k i Then, the cosine similarity calculation method can be used to calculate the semantic vector V of the target intention u and the semantic vector k of the knowledge item in the knowledge vector library i The third similarity is obtained.

[0074] Sub-step 1032: Select, from the knowledge base, candidate knowledge items whose third similarity with the target intent is greater than a third threshold.

[0075] Specifically, the knowledge items can be sorted in descending order based on the similarity calculation results, and the knowledge items whose third similarity is higher than a preset threshold θ are selected as candidate knowledge items. The threshold θ can be set by testing on a validation set and can be a value that balances recall and precision, such as θ = 0.7. The specific value is not limited here.

[0076] Sub-step 1033, if the number of candidate knowledge items is greater than or equal to the second number, then selecting the second number of candidate knowledge items from the selected candidate knowledge items in descending order of the third similarity as knowledge items associated with the target intent.

[0077] Specifically, because large models have input length limitations, input data length control is required. When the total length of a knowledge item exceeds the limit, some knowledge items with high similarity scores can be retained, while the remaining knowledge items can be deleted. Therefore, a second number, such as 5, can be set, and five candidate knowledge items can be selected in descending order of the third similarity scores as the knowledge items associated with the target intent.

[0078] Sub-step 1034 : If the number of candidate knowledge items is less than the second number, each candidate knowledge item is determined as a knowledge item associated with the target intent.

[0079] By setting a third threshold and a selection mechanism for candidate knowledge items, we can initially screen out a set of knowledge items highly relevant to the target intent while ensuring the quality of the knowledge items, providing high-quality basic data for subsequent screening. By setting a second number and a selection mechanism for knowledge items associated with the target intent, we ensure that the number of knowledge items input into the large model is moderate and highly relevant, providing sufficient context while avoiding information overload.

[0080] Step 104: Generate second information based on the target intention and the knowledge item.

[0081] In this embodiment, the target intent and retrieved knowledge items can be aggregated according to a preset format or template to generate the second information. Because the second information not only includes the user's target intent determined based on the first information input by the user, but also includes the knowledge items associated with the target intent, it can provide the large model with richer knowledge and more precise questions, allowing the large model to output more accurate response information.

[0082] In some optional implementations of this embodiment, step 104 may include the following sub-steps:

[0083] Sub-step 1041 , integrating the target intention and the knowledge items to obtain structured information.

[0084] Specifically, special tags "[User Question]" and "[Related Knowledge]" can be used to distinguish target intent and knowledge items. Let the target intent be Q, and the set of retrieved knowledge items be Ktop_n = {K1, K2, ..., Kn}, then the second information I can be expressed as:

[0085] I = format function (Q, Ktop_n)

[0086] The formatting function is used to combine user intent and knowledge items in a predetermined format, for example:

[0087] I = "User question:" Q + "Related knowledge:" K1 + K2 +, ..., Kn (Q, Ktop_n)

[0088] Sub-step 1042: generating second information based on the second prompt word template and the structured information.

[0089] Here, a second prompt word template can be pre-set. The second prompt word template is used to generate a prefabricated text frame for the second question information, which contains specific prompt statements and placeholders. The placeholders are used to insert content in the structured information. In this way, the large model can be effectively guided to generate expected question information. For example, the second prompt word template can be: Please answer the user question based on the following relevant knowledge: "User question: "Q+" related knowledge: "K1+K2+,...,Kn(Q, Ktop_n)". After filling the target intention Q and the retrieved knowledge item Ktop_n into the second prompt word template, the second information can be obtained.

[0090] By integrating target intent and knowledge items to generate structured information, scattered information resources can be organized into a complete and logically connected information set, providing clear and accurate input for the subsequent generation of secondary question information. By integrating this structured information, the large model can clearly identify the user's question and the relevant knowledge available for reference, avoiding information confusion and omissions.

[0091] Step 105: input the second information into the large model to obtain the response information of the large model to the second information.

[0092] In this embodiment, the large model (LLM) refers to a pre-trained model with large-scale parameters and powerful language generation capabilities. It can generate corresponding answers based on the input question information. It is usually trained with a large amount of text data and has certain semantic understanding and language generation capabilities.

[0093] Response information is generated by the large model based on the input second information to answer user questions or meet user needs. It is the final output provided to the user. The large model can perform natural language processing on the second information to understand its semantics and, based on the knowledge items contained therein, respond to the user's target intent in the second information. It can generate response information tailored to this target intent and ultimately output it to the user.

[0094] The method provided by the above-mentioned embodiment of the present application first pre-processes the first information input by the user to obtain the key information in the first information, which can reduce noise interference and provide an accurate data basis for subsequent intent recognition; then, by matching the key information with the intent library, the user's target intent is determined, which can fully consider the semantic similarity and correlation and improve the accuracy of identifying user needs; then, the knowledge items associated with the target intent are retrieved from the preset knowledge base, which can provide targeted background knowledge for the large model and ensure that the generated content is consistent with user needs; then, based on the target intent and the knowledge items, the second information is generated, which can make the information input into the large model more accurate and complete; finally, the second information is input into the large model to obtain the response information of the large model to the second information, so that the large model can generate more targeted and valuable response information based on more complete and accurate information, thereby improving the accuracy of information acquisition.

[0095] In some optional embodiments, see Figure 2 , the intent library can be generated by the following steps:

[0096] Step 201: Segment the knowledge document into multiple knowledge segments.

[0097] Specifically, the knowledge document can be structurally analyzed and divided into chapters, paragraphs or semantic units to obtain a set of knowledge fragments, which is recorded as D = {d1, d2, ..., d L}. Where d is the knowledge fragment and L is the total number of knowledge fragments.

[0098] For example, a document on the application of artificial intelligence technology can be divided into chapters based on technical areas, application scenarios, etc. Each chapter can be further divided into several paragraphs, and each paragraph serves as a knowledge fragment. Ensure that each knowledge fragment is semantically complete and relatively independent to facilitate subsequent processing.

[0099] Step 202: Determine the information content score of each knowledge segment in the plurality of knowledge segments based on the information entropy method.

[0100] Specifically, the information entropy method can be used to analyze the knowledge fragment d i Calculate the information score I i :

[0101]

[0102] Among them, D i is the knowledge fragment d i The word set in p i,t For word t in knowledge segment d i The probability of word t in knowledge segment d i The number of occurrences in the knowledge fragment d iThe ratio of the total number of words in .

[0103] Step 203 : Based on the information score of each knowledge fragment and the preset granularity parameter, the number of intentions corresponding to each knowledge fragment is generated.

[0104] Specifically, the granularity parameter can be recorded as p. For the knowledge fragment d i , whose number of intentions is m i , m i =p×I i For example, if the granularity parameter is p=1, and the information score of a certain knowledge fragment is 4.5, then the number of intentions corresponding to the knowledge fragment is 4.5. After rounding, the number of intentions corresponding to the knowledge fragment can be set to 5.

[0105] Step 204 , for each knowledge fragment among the multiple knowledge fragments, generate a prompt word based on the knowledge fragment, the number of intentions corresponding to the knowledge fragment and the first prompt word template, and input the prompt word into the big model to obtain the intention corresponding to the knowledge fragment.

[0106] Specifically, for the knowledge segment d i , we can first i and its intention number m i Fill in the first prompt word template to obtain the prompt word; then input the prompt word into the large model to obtain the m corresponding to the knowledge fragment i For example, the first prompt word template is "Please generate {number of intents} possible questions that users may ask based on the following content: {content of the knowledge fragment}".

[0107] Step 205: Summarize the intents corresponding to the knowledge fragments to generate an intent library.

[0108] By segmenting knowledge documents, huge knowledge documents can be decomposed into multiple semantically complete and relatively independent knowledge fragments, which facilitates subsequent targeted processing and knowledge extraction, and improves the efficiency and accuracy of knowledge processing; the information content score of the knowledge fragment is determined based on the information entropy method, which can quantitatively evaluate the information richness and importance of each knowledge fragment, providing an objective basis for the subsequent determination of the number of intents, and ensuring the quality and coverage of the intent library; the number of intents is controlled by granularity parameters, which can flexibly adjust the granularity of the intents to adapt to different application scenarios and needs, so that the intent library is detailed but not too redundant; by summarizing the intentions of each knowledge fragment to generate an intent library, it can comprehensively cover all fields and aspects of the knowledge document, providing rich and comprehensive resources for subsequent intent identification and matching, and improving the ability to understand user intentions and the accuracy of response information generated by large models.

[0109] Further references Figure 3As an implementation of the methods shown in the above figures, the present application provides an embodiment of an information acquisition device, which is similar to Figure 1 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0110] like Figure 3 As shown, the information acquisition device 300 of this embodiment includes: a preprocessing unit 301, used to preprocess the first information input by the user to obtain key information in the above-mentioned first information; a determination unit 302, used to match the above-mentioned key information with the intention library to determine the target intention of the above-mentioned user; a retrieval unit 303, used to retrieve knowledge items associated with the above-mentioned target intention from a preset knowledge base; a generation unit 304, used to generate second information based on the above-mentioned target intention and the above-mentioned knowledge items; an input unit 305, used to input the above-mentioned second information into the large model to obtain response information of the above-mentioned large model to the above-mentioned second information.

[0111] In some optional implementations of this embodiment, the above-mentioned intention library is generated through the following steps: dividing the knowledge document into multiple knowledge fragments; determining the information content score of each knowledge fragment in the above-mentioned multiple knowledge fragments based on the information entropy method; generating the number of intentions corresponding to each knowledge fragment based on the information content score of each knowledge fragment and the preset granularity parameters; for each knowledge fragment in the above-mentioned multiple knowledge fragments, generating a prompt word based on the knowledge fragment, the number of intentions corresponding to the knowledge fragment and the first prompt word template, and inputting the above-mentioned prompt word into the big model to obtain the intention corresponding to the knowledge fragment; summarizing the intentions corresponding to each knowledge fragment to generate the above-mentioned intention library.

[0112] In some optional implementations of this embodiment, the above-mentioned preprocessing unit 301 is also used to: perform word segmentation on the first information input by the user to obtain a word segmentation result; remove stop words and low-frequency words in the above-mentioned word segmentation result to obtain an initial word set; extract keywords from the above-mentioned initial word set to obtain a keyword set; extract the word vector of each keyword in the above-mentioned keyword set based on a pre-trained word embedding model; filter the above-mentioned keyword set based on the first similarity between the above-mentioned word vectors to obtain key information.

[0113] In some optional implementations of this embodiment, the above-mentioned determination unit 302 is also used to: determine the second similarity between the above-mentioned key information and each intention in the above-mentioned intention library; if the maximum value of the second similarity is less than the first threshold, then select a first number of first candidate intentions from the above-mentioned intention library in order of the second similarity from high to low; extract the second candidate intentions whose similarity with the above-mentioned key information is greater than the second threshold from the selected first candidate intentions; display the extracted second candidate intentions; and determine the target intention of the above-mentioned user based on the user's selection operation on the displayed second candidate intention.

[0114] In some optional implementations of this embodiment, the above-mentioned determination unit 302 is also used to: if there is no second candidate intention among the selected first candidate intentions whose similarity with the above-mentioned key information is greater than the above-mentioned second threshold, then a prompt message is displayed, and the above-mentioned prompt message is used to prompt the user to re-enter the question; if the maximum value of the above-mentioned second similarity is greater than or equal to the above-mentioned first threshold, then the first information is determined as the target intention of the above-mentioned user.

[0115] In some optional implementations of this embodiment, the above-mentioned retrieval unit 303 is also used to: determine the third similarity between the above-mentioned target intent and each knowledge item in the knowledge base; select candidate knowledge items from the above-mentioned knowledge base whose third similarity with the above-mentioned target intent is greater than a third threshold; if the number of the above-mentioned candidate knowledge items is greater than or equal to the second number, then select a second number of candidate knowledge items from the selected candidate knowledge items in order of the third similarity from high to low, as knowledge items associated with the above-mentioned target intent; if the number of the above-mentioned candidate knowledge items is less than the above-mentioned second number, then determine each candidate knowledge item as a knowledge item associated with the above-mentioned target intent.

[0116] In some optional implementations of this embodiment, the generating unit 304 is further configured to: integrate the target intention and the knowledge item to obtain structured information; and generate second information based on the second prompt word template and the structured information.

[0117] The device provided by the above-mentioned embodiment of the present application first pre-processes the first information input by the user to obtain key information in the first information, which can reduce noise interference and provide an accurate data basis for subsequent intent recognition; then, by matching the key information with the intent library, the user's target intent is determined, which can fully consider the semantic similarity and correlation and improve the accuracy of identifying user needs; then, the knowledge items associated with the target intent are retrieved from the preset knowledge base, which can provide targeted background knowledge for the large model and ensure that the generated content is consistent with user needs; then, based on the target intent and knowledge items, the second information is generated, which can make the information input into the large model more accurate and complete; finally, the second information is input into the large model to obtain the response information of the large model to the second information, so that the large model can generate more targeted and valuable response information based on more complete and accurate information, thereby improving the accuracy of information acquisition.

[0118] Reference below Figure 4 , which shows a structural schematic diagram of an electronic device for implementing some embodiments of the present application. Figure 4 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0119] like Figure 4 As shown, the electronic device 400 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 402 or a program loaded from a storage device 408 into a random access memory (RAM) 403. Various programs and data required for the operation of the electronic device 400 are also stored in the RAM 403. The processing device 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0120] Typically, the following devices may be connected to the I / O interface 405: an input device 406 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 407 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 408 including, for example, a magnetic disk, a hard disk, etc.; and a communication device 409. The communication device 409 may allow the electronic device 400 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 4 The electronic device 400 is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead. Figure 4 Each block shown in the figure may represent one device, or may represent multiple devices as needed.

[0121] In particular, according to some embodiments of the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from a network via the communication device 409, or installed from the storage device 408, or installed from the ROM 402. When the computer program is executed by the processing device 401, the above-mentioned functions defined in the method of some embodiments of the present application are performed.

[0122] It should be noted that the computer-readable medium described in some embodiments of the present application may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present application, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device, or device. In some embodiments of the present application, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0123] In some embodiments, the client and server can communicate using any currently known or future developed network protocol, such as HTTP (Hypertext Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0124] The computer-readable medium may be included in the electronic device, or may exist independently without being installed in the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: pre-processes the first information input by the user to obtain key information in the first information; matches the key information with the intent library to determine the user's target intent; retrieves knowledge items associated with the target intent from a preset knowledge library; generates second information based on the target intent and the knowledge items; and inputs the second information into the large model to obtain the large model's response information to the second information. This embodiment improves the accuracy of information acquisition.

[0125] Computer program code for performing the operations of some embodiments of the present application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++; and also conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user's computer through any type of network, or can be connected to an external computer (for example, through the Internet using an Internet service provider). The above network includes a local area network (LAN) or a wide area network (WAN).

[0126] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0127] The units described in some embodiments of this application may be implemented in software or hardware. The units described may also be provided in a processor. For example, a processor may be described as comprising a first determination unit, a second determination unit, a selection unit, and a third determination unit. The names of these units do not, in some cases, limit the units themselves.

[0128] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0129] The above description is only an illustration of some preferred embodiments of the present application and the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present application is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features and the technical features with similar functions disclosed in the embodiments of the present application (but not limited to) are replaced with each other to form a technical solution.

Claims

1. An information acquisition method, characterized in that: The method comprises: Preprocessing the first information input by the user to obtain key information in the first information; Matching the key information with the intent library to determine the user's target intent; Retrieving knowledge items associated with the target intent from a knowledge base; generating second information based on the target intention and the knowledge item; The second information is input into the large model to obtain response information of the large model to the second information.

2. The method according to claim 1, characterized in that The intent library is generated by the following steps: Split the knowledge document into multiple knowledge fragments; Determining an information content score for each of the plurality of knowledge fragments based on an information entropy method; Based on the information score of each knowledge fragment and the preset granularity parameter, generating the number of intentions corresponding to each knowledge fragment; For each of the plurality of knowledge fragments, generating a prompt word based on the knowledge fragment, the number of intents corresponding to the knowledge fragment, and the first prompt word template, and inputting the prompt word into the large model to obtain the intent corresponding to the knowledge fragment; The intentions corresponding to the various knowledge fragments are summarized to generate the intention library.

3. The method according to claim 1, characterized in that The preprocessing of the first information input by the user to obtain key information in the first information includes: Performing word segmentation on the first information input by the user to obtain a word segmentation result; Removing stop words and low-frequency words from the word segmentation results to obtain an initial word set; Extract keywords from the initial word set to obtain a keyword set; Extracting word vectors for each keyword in the keyword set based on a pre-trained word embedding model; Based on the first similarity between the word vectors, the keyword set is filtered to obtain key information.

4. The method according to claim 1, wherein The matching of the key information with the intent library to determine the user's target intent includes: Determining a second similarity between the key information and each intent in the intent library; If the maximum value of the second similarity is less than the first threshold, selecting a first number of first candidate intents from the intent library in descending order of the second similarity; Extracting, from the selected first candidate intents, a second candidate intent having a similarity with the key information greater than a second threshold; displaying the extracted second candidate intent; Based on the user's selection operation on the displayed second candidate intent, the user's target intent is determined.

5. The method according to claim 4, characterized in that The step of matching the key information with the intent library to determine the user's target intent further includes: If there is no second candidate intent among the selected first candidate intents whose similarity with the key information is greater than the second threshold, displaying a prompt message, wherein the prompt message is used to prompt the user to re-enter the question; If the maximum value of the second similarity is greater than or equal to the first threshold, the first information is determined as the target intention of the user.

6. The method according to claim 1, characterized in that The retrieving, from a knowledge base, knowledge items associated with the target intent includes: Determining a third similarity between the target intent and each knowledge item in the knowledge base; Selecting, from the knowledge base, candidate knowledge items whose third similarity to the target intent is greater than a third threshold; If the number of the candidate knowledge items is greater than or equal to the second number, selecting a second number of candidate knowledge items from the selected candidate knowledge items in descending order of the third similarity as knowledge items associated with the target intent; If the number of the candidate knowledge items is less than the second number, each candidate knowledge item is determined as a knowledge item associated with the target intent.

7. The method according to claim 1, characterized in that The generating of second information based on the target intention and the knowledge item includes: Integrating the target intention and the knowledge item to obtain structured information; Second information is generated based on the second prompt word template and the structured information.

8. An information acquisition device, characterized in that: The device comprises: a preprocessing unit, configured to preprocess the first information input by the user to obtain key information in the first information; a determination unit, configured to match the key information with an intent library to determine the target intent of the user; A retrieval unit, configured to retrieve knowledge items associated with the target intention from a preset knowledge base; a generating unit, configured to generate second information based on the target intention and the knowledge item; The input unit is used to input the second information into the large model to obtain response information of the large model to the second information.

9. An electronic device, characterized in that: include: one or more processors; a storage device having one or more programs stored thereon, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer-readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images