Information retrieval method and device and electronic equipment
Through the combination of transformation model and associated information, the problem of improving information retrieval accuracy and efficiency in the intelligent question-and-answer system is solved, and more accurate search results are achieved, which are suitable for diverse application scenarios.
Patent Information
- Application Number
- CN202411042042.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-30
- Publication Date
- 2024-11-08
Smart Images

Figure CN118916443A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and more specifically, to an information retrieval method, device and electronic device. Background Art
[0002] Information retrieval is the main way for users to query and obtain information. It is a method and means of finding information. For example, users can input questions into the question-answering system by voice, and then the question-answering system can retrieve information and feedback the retrieval results to the user by voice. However, the accuracy of relevant information retrieval needs to be improved. Summary of the invention
[0003] The present application proposes an information retrieval method, device and electronic device to improve the above-mentioned problems.
[0004] In a first aspect, the present application provides an information retrieval method, comprising: converting input audio into corresponding target text through a conversion model; identifying the user's retrieval intention through the target text to obtain retrieval intention information; obtaining retrieval input information through the retrieval intention information and corresponding associated information; and inputting the retrieval input information into a large language model to obtain retrieval results returned by the large language model.
[0005] In the second aspect, the present application provides an information retrieval device, which includes: a speech conversion unit, which is used to convert input audio into corresponding target text through a conversion model; an intention recognition unit, which is used to recognize the user's retrieval intention through the target text to obtain retrieval intention information; a retrieval enhancement unit, which is used to obtain retrieval input information through the retrieval intention information and the corresponding associated information; and a retrieval execution unit, which is used to input the retrieval input information into a large language model to obtain the retrieval results returned by the large language model.
[0006] In a third aspect, the present application provides an electronic device comprising one or more processors and a memory; one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the above-mentioned method.
[0007] In a fourth aspect, the present application provides a computer-readable storage medium storing a program code executable by a processor, wherein the computer-readable storage medium includes the stored program code, wherein the above method is executed when the program code is run.
[0008] In a fifth aspect, the present application provides a computer program product, including a computer program, which implements the above method when executed by a processor.
[0009] The present application provides an information retrieval method, device and electronic device, in which the input audio can be first converted into a corresponding target text through a conversion model, so as to identify the user's search intention through the target text to obtain search intention information; and then the search input information can be obtained through the search intention information and the associated information, so as to input the search input information into the large language model to obtain the search results returned by the large language model. Thus, through the above-mentioned method, the conversion of input audio to text can be completed through the conversion model, and the search intention information can be enhanced through the associated information to obtain the search input information, so that by combining the use of the conversion model and the enhancement of the search intention information, the search results finally fed back by the large language model can be more accurate. BRIEF DESCRIPTION OF THE DRAWINGS
[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.
[0011] Figure 1 A schematic diagram showing an application scenario of the information retrieval method proposed in an embodiment of the present application;
[0012] Figure 2 A schematic diagram showing another application scenario of the information retrieval method proposed in an embodiment of the present application;
[0013] Figure 3 A flowchart of an information retrieval method proposed in an embodiment of the present application is shown;
[0014] Figure 4 A schematic diagram showing the display of search results in an embodiment of the present application is shown;
[0015] Figure 5 A flowchart of an information retrieval method proposed in another embodiment of the present application is shown;
[0016] Figure 6 A flowchart of an information retrieval method proposed in another embodiment of the present application is shown;
[0017] Figure 7 A schematic diagram showing a system architecture involved in the information retrieval method of an embodiment of the present application;
[0018] Figure 8 A structural block diagram of a content recommendation device proposed in an embodiment of the present application is shown;
[0019] Fig. 9 A structural block diagram of an electronic device for executing an information retrieval method according to an embodiment of the present application is shown;
[0020] Fig.10 It is a storage unit of an embodiment of the present application for storing or carrying a program code for implementing the information retrieval method according to an embodiment of the present application. DETAILED DESCRIPTION
[0021] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0022] As the main way for users to query and obtain information, information retrieval is becoming increasingly important in the modern information society. It is not only a basic method and means to find information, but also has a profound impact on the efficiency and quality of people's acquisition of knowledge, problem solving and decision-making. With the rapid development of information technology, the forms and application scenarios of information retrieval are becoming increasingly diverse. For example, in an intelligent question-and-answer system, users can input questions by voice, and the intelligent question-and-answer system can use speech recognition technology to convert speech into text, and then retrieve and process information. This process requires not only the system to have efficient text retrieval capabilities, but also to be able to accurately understand the user's intentions and quickly locate relevant information from massive data. After the retrieval is completed, the system will feedback the results to the user in the form of voice, realizing barrier-free information exchange throughout the process.
[0023] However, despite the continuous advancement of information retrieval technology, the relevant accuracy and efficiency still need to be improved. Therefore, the inventors have proposed an information retrieval method, device and electronic device in this application. In this method, the input audio can be first converted into a corresponding target text through a conversion model, so as to identify the user's search intent through the target text to obtain search intent information; and then the search input information can be obtained through the search intent information and the associated information, so as to input the search input information into the large language model to obtain the search results returned by the large language model. Therefore, through the above-mentioned method, the conversion of input audio to text can be completed through the conversion model, and the search intent information can be enhanced through the associated information to obtain the search input information, so that the search results finally fed back by the large language model can be more accurate.
[0024] The following first introduces the application scenarios involved in the embodiments of the present application.
[0025] In the embodiment of the present application, the information retrieval method provided can be executed by an electronic device. In this way, all steps in the information retrieval method provided in the embodiment of the present application can be executed by the electronic device. For example, Figure 1 As shown, in the case where all steps in the information retrieval method provided in the embodiment of the present application can be executed by an electronic device, all steps can be executed by the processor of the electronic device 100.
[0026] Furthermore, the information retrieval method provided in the embodiment of the present application can also be executed by a server. Correspondingly, in this mode executed by the server, the server can start executing the steps in the information retrieval method provided in the embodiment of the present application in response to a trigger instruction. Among them, the trigger instruction can be sent by an electronic device used by the user, or it can be triggered locally by the server in response to some automation events. For example, the server 200 can start executing the steps in the embodiment of the present application in response to the input audio sent by the electronic device 100.
[0027] In addition, the information retrieval method provided in the embodiment of the present application can also be executed by the electronic device and the server in collaboration. In this way, some steps of the information retrieval method provided in the embodiment of the present application are executed by the electronic device, while other steps are executed by the server. For example, Figure 2 As shown, the electronic device 100 can perform the information retrieval method including: converting the input audio into the corresponding target text through the conversion model, identifying the user's retrieval intention through the target text to obtain the retrieval intention information, and then transmitting the retrieval intention information to the server 200, and then the server 200 performs the subsequent steps. Optionally, after obtaining the retrieval result, the server 200 can return the retrieval result to the electronic device 100, and the electronic device 100 will display it to the user. Among them, the retrieval result can be in text form or language form. When the retrieval result is in text form, the electronic device can display the retrieval result in text form through the screen. When the retrieval result is in audio form, the electronic device can play the retrieval result in audio form through a speaker.
[0028] It should be noted that in this method of collaborative execution by the electronic device and the server, the steps respectively executed by the electronic device and the server are not limited to the methods introduced in the above examples. In actual applications, the steps respectively executed by the electronic device and the server can be dynamically adjusted according to actual conditions.
[0029] It should be noted that the electronic device 100 is Figure 1 and Figure 2In addition to the smartphone shown in , it can also be a tablet computer, a smart watch, an intelligent voice assistant and other devices. Server 200 can be an independent physical server, or it can be a server cluster or a distributed system composed of multiple physical servers. Among them, in the case where the information retrieval method provided in the embodiment of the present application is executed by a server cluster or a distributed system composed of multiple physical servers, different steps in the information retrieval method can be executed by different physical servers respectively, or can be executed in a distributed manner by a server built based on a distributed system.
[0030] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.
[0031] See also Figure 3 , the present application provides an information retrieval method, the method comprising:
[0032] S110: Convert the input audio into the corresponding target text through the conversion model.
[0033] The input audio is input by the user. The content of the input audio may be the content that the user desires to retrieve. For example, if the user desires to obtain the attached restaurant, the content of the input audio may be "recommend me a nearby Sichuan restaurant". For example, if the user desires to learn about the movies that will be released soon, the content of the input audio may be "recommend me some movie information".
[0034] In an embodiment of the present application, the user can input audio in a variety of ways. Optionally, the user can input audio through a voice assistant in a smartphone. Optionally, the user can input audio through a smart speaker. Optionally, the user can input audio through a Bluetooth headset. In an embodiment of the present application, subsequent operation steps can be triggered by a wake-up word. For example, when it is detected that the collected audio includes a preset wake-up word, the user's input audio can be collected. Alternatively, audio collection can be performed when sound is detected, so that the collected audio is used as input audio.
[0035] In an embodiment of the present application, after obtaining the input audio, the input audio can be processed by a conversion model to convert the input audio into a corresponding target text. Wherein, the conversion model is obtained by pre-training the network model to be trained with training data. The trained conversion model can output the corresponding text according to the input audio. Wherein, the network model to be trained can be a recurrent neural network (RNNs), a long short-term memory network (LSTMs), a gated recurrent unit (GRUs) or a Transformer model. The training data may include audio files and corresponding text labels.
[0036] As a way, FunASR can be used as a conversion model to convert input audio into target text.
[0037] S120: Identify the user's search intention through the target text to obtain search intention information.
[0038] After obtaining the target text converted from the input audio, the user's search intention can be identified based on the target text.
[0039] It should be noted that when users express their search needs through audio, what they say may not be a direct expression of their search needs. For example, the user may say "I'm so hungry now". In this case, the user's original intention may be to recommend a restaurant, but the user did not directly tell the system that a restaurant recommendation is needed. If the user's words are directly input into the big data model, the operation performed by the big data model may not be a search for a restaurant. For example, the big data model may subsequently perform a search on how to deal with hunger. Therefore, by performing intent recognition on the target text, it is beneficial for the subsequent big data model to feedback the content required by the user. In an embodiment of the present application, the user's input (input audio) can be matched with a series of predefined intent categories to achieve intent recognition. Among them, the identified intent can cover various user needs from simple information retrieval to performing complex tasks.
[0040] In an embodiment of the present application, query rewriting and keyword extraction may be performed on the target text, and then the user's search intention may be identified based on the rewriting result of the query rewriting and the extracted keywords.
[0041] Among them, query rewriting is an important technology in the field of information retrieval and natural language processing. It refers to a series of operations on the user's original query (for example, the aforementioned target text) to generate one or more new query statements (rewriting results). These new query statements are semantically similar or equivalent to the original query, but may have better retrieval effects or richer information.
[0042] The functions of query rewriting may include:
[0043] Improve retrieval accuracy: By removing noise words in the target text, replacing synonyms or near-synonyms, adjusting the query structure, etc., the rewritten results can more accurately reflect the user's true intention, thereby improving the relevance of the retrieval results.
[0044] Expand search scope: Sometimes users may not know the exact query words or phrases. Rewording can expand the search scope by adding relevant words, phrases or contextual information, helping users find more relevant results.
[0045] Optimize query performance: Rewriting can simplify query statements, reduce calculations, and improve query efficiency based on the specific requirements of search engines or database systems.
[0046] In the embodiments of the present application, there are many methods for query rewriting, including but not limited to:
[0047] Synonym replacement: replace words in the target text with words with similar or identical meanings.
[0048] Query expansion: Add relevant words or phrases to the target text to expand the coverage of the query.
[0049] Query reduction: Remove noise words or irrelevant words in the target text to make the query more refined.
[0050] Query reconstruction: adjust the structure or grammar of the target text to better conform to the search engine's parsing rules or the user's expression habits.
[0051] Context-based rewriting: Use information such as the user's historical query records, current session context, or external knowledge base to rewrite the target text to more accurately reflect the user's intention.
[0052] In the embodiments of the present application, there are multiple ways to perform keyword extraction. As one way, keyword extraction can be based on statistics. For example, keywords can be acquired based on the TF-IDF algorithm. In the TF-IDF algorithm, the word frequency and inverse document frequency of each word in the target text can be calculated, and words with higher TF-IDF values can be selected as keywords. As another way, pre-trained language models such as BERT and RoBERTa can be used to extract keywords from the target text.
[0053] As a method, after obtaining the rewriting results and keywords, the rewriting results and extracted keywords can be used as key features, combined with the original form, length, type, etc. of the target text, to construct a multidimensional feature vector. The constructed feature vector is input into the trained model to output the classification result of the user's search intent, and then the search intent information is obtained based on the classification result.
[0054] For example, the target text obtained by converting the user's input audio may be "I'm a little hungry", and the final search intent information may be "Please recommend nearby food". For example, the target text obtained by converting the user's input audio may be "I feel so bored and don't know what to do", and the final search intent information may be "Please recommend a nearby movie theater or library".
[0055] S130: Obtain search input information by searching the intention information and the associated information.
[0056] In an embodiment of the present application, related information can be understood as information that is content-related to the search intent information. The search intent information can be further enriched through the related information, so that the big data model can more accurately determine the user's search intent.
[0057] As a method, data can be collected from multiple data sources (e.g., applications, databases, or document repositories) in advance, and then the collected data can be stored. In this case, after obtaining the retrieval intention information, the associated information associated with the retrieval intention information can be found from the stored data.
[0058] As one approach, the obtained associated information may be personalized information about the target user.
[0059] In the embodiment of the present application, personalized information can be understood as data representing the habits or preferences of the target user. Alternatively, it can also be understood as data representing the user portrait of the target user. Among them, the target user can be understood as the user referred to when obtaining personalized information, so that the obtained target personalized information corresponds to the target user. Among them, the target personalized information corresponds to the target user, and it can be understood that the obtained target personalized information represents the habits or preferences of the target user.
[0060] In the embodiment of the present application, the target user can be understood as the user of the electronic device (the device executing the information retrieval method), or the user currently using the electronic device. The user of the electronic device can be understood as the user corresponding to the identification information recorded in the electronic device. The identification information can include a user account or a current phone number.
[0061] As a method, personalized information of the user to whom the electronic device belongs can be stored in the electronic device. Among them, the personalized information stored locally in the electronic device can be personalized information about multiple aspects. For example, it can include personalized information about diet, personalized information about travel, personalized information about sports, and personalized information about health. When a designated event is triggered, the personalized information corresponding to the designated event can be obtained as the target personalized information. For example, if the designated event is related to diet, personalized information related to diet can be obtained locally as the target personalized information.
[0062] As another way, the identification information of the user can be stored locally on the electronic device. In this case, the user represented by the identification information can be used as the target user. Optionally, the identification information can be an account number or a phone number. Optionally, the personalized information of all users can be centrally stored in the cloud. In this way, when a specified event is triggered, the target personalized information can be obtained from the cloud based on the stored identification information and the triggered specified event.
[0063] As another way, the current user of the electronic device may not be the original user of the electronic device. For example, in some cases, the original user of the electronic device may lend the electronic device to other users for use. In this case, if the target personalized information is directly obtained based on the original user of the electronic device, the acquired target personalized information may not meet the needs of the current user. For example, the original user of the electronic device is user U1, and user U1 lends the electronic device to user U2 for use. If user U2 inputs audio during the use of the electronic device, but the electronic device still uses user U1's personalized information as associated information, it will not only cause the privacy of user U1 to be leaked, but also prevent user U2 from obtaining the required feedback content. In this way, when it is necessary to obtain associated information, the electronic device can first perform user identification to use the identified user as the target user.
[0064] Among them, the method for performing user identification is not specifically limited in the embodiments of the present application. For example, as a user identification method, the electronic device can collect identity features through a functional device when it needs to obtain associated information, and then identify the user's identity by matching the identity features. Optionally, the functional device can be a camera or a structured light module. For example, the electronic device can perform image acquisition through a front camera, and then perform face recognition on the acquired image to use the identified user as the target user. Based on the foregoing example, if user U1 lends the electronic device to user U2 for use, the electronic device identifies the current user as user U2 through user identification, and the electronic device can obtain the personalized information of user U2 as associated information.
[0065] It should be noted that after the electronic device obtains the identity feature, in order to facilitate the electronic device to successfully complete the user identification, the electronic device may store multiple user identity features. The identity feature may include a face feature, a fingerprint feature, or other identifiable features. For example, as shown in the following table:
[0066] ID Identity User ID 1 Feature C1 User 1 2 Feature C2 User 2 3 Feature C3 User 3
[0067] As shown in the above table, the electronic device stores features C1, C2, and C3, wherein the user ID corresponding to feature C1 is user 1, the user ID corresponding to feature C2 is user 2, and the user ID corresponding to feature C3 is user 3. If the identity feature collected by the electronic device through the functional device is feature C2, it can be identified that user 2 is currently using the electronic device.
[0068] It should be noted that, in the case where there are multiple ways to determine the target user, as one way, the electronic device can determine which way to use to determine the target user based on the current operating mode. For example, if the electronic device is currently in the guest mode, the target user can be determined by collecting identity features through functional devices. For another example, if the electronic device is currently in the master mode, the target user can be determined by obtaining the identification information recorded in the electronic device.
[0069] S140: Input the search input information into the large language model to obtain the search results returned by the large language model.
[0070] Large Language Models (LLMs) are a class of deep learning models with a large number of parameters that demonstrate powerful capabilities in the field of natural language processing. Large language models can learn language patterns, grammar, and semantics by processing large amounts of text data, so that they can understand and generate human language. Among them, large language models usually have billions or even tens of billions of parameters, which enables them to learn rich language features and patterns. Among them, the working principle of large language models can be summarized as receiving input text (for example, the aforementioned retrieval input information), encoding the input text, and then decoding it to generate output predictions. Specifically, the input text is tokenized and converted into word embedding vectors, and then processed through multiple encoder layers, each of which contains a self-attention mechanism and a feedforward neural network. Finally, the internal representation is processed through multiple decoder layers to generate predicted output text (for example, retrieval results).
[0071] In an embodiment of the present application, the large language model can be deployed locally on the electronic device or in the cloud.
[0072] In the embodiment of the present application, the search results returned by the large language model may be in text form. In this case, the electronic device may directly display the search results in text form. Figure 4 As shown in FIG. 1 , if the audio input by the user is “Please recommend some news”, the electronic device can display the text-based search results (eg, retrieved sports news) in the form of cards on the screen. Figure 4 As shown, the retrieved sports news is displayed in the card 10. Alternatively, in some cases, the retrieval results in text form can be converted into retrieval results in audio form, and the retrieval results in audio form can be played, so that in this way, the user can answer questions by voice.
[0073] There are multiple ways to determine how the electronic device ultimately feeds back the search results.
[0074] As a method, it can be configured in advance by the user. In this method, the user can choose the feedback method he likes, and the electronic device can store the feedback method selected by the user. Correspondingly, when the electronic device obtains the textual search results returned by the big data model, it can first detect the feedback method configured by the user. If the feedback method configured by the user is textual feedback, the electronic device can directly display the textual search results on the screen. If the feedback method configured by the user is voice feedback, the electronic device can convert the textual search results into audio search results, and then play the audio search results.
[0075] As a method, the electronic device may make a determination based on actual conditions.
[0076] Optionally, the electronic device can detect the current mode. If the electronic device is currently in silent mode, even if the search results are obtained in audio form, voice playback cannot be performed, or the played voice cannot actually be heard by the user. In this case, you can choose to display only the search results in text form. Optionally, the electronic device can detect the noise of the environment in which it is located. If the noise of the environment in which the electronic device is currently located is large, even if the electronic device performs voice broadcast, the user may not be able to hear the broadcast content well. Furthermore, in this case, the electronic device can also only display the search results in text form.
[0077] As a way, the text-based search results can be converted into audio-based search results through the speech generation model ChatTTS. ChatTTS is a Python-based natural language processing open source project that can be used to build intelligent chatbots and provide high-quality text-to-speech (TTS) functions.
[0078] An information retrieval method provided by this embodiment enables the conversion of input audio to text to be completed through a conversion model, and the retrieval intent information to be enhanced through associated information to obtain retrieval input information, thereby combining the use of the conversion model and the enhancement of the retrieval intent information, so that the retrieval results ultimately fed back by the large language model can be more accurate.
[0079] See also Figure 5 , the present application provides an information retrieval method, the method comprising:
[0080] S210: In response to receiving input audio, determining a target network model from a plurality of network models, wherein the plurality of network models have respective conversion accuracies and / or conversion efficiencies that are different.
[0081] In an embodiment of the present application, multiple network models can be pre-configured, and the multiple network models can all realize the conversion of input audio into corresponding target text. However, the conversion accuracy and / or conversion efficiency of each of the multiple network models are different. Among them, the conversion accuracy represents the accuracy of the converted text when the network model converts the speech into the corresponding text. The conversion efficiency represents the time taken by the network model to convert the speech into the corresponding text.
[0082] S220: Convert the input audio into corresponding target text through a conversion model.
[0083] In multiple network models, the structures included in different network models may be different. As a method, if the determined conversion model is the first network model, the input audio is converted into the corresponding target text through the acoustic model included in the first network model. If the determined conversion model is the second network model, the input audio is converted into the corresponding target text through the acoustic model and language model included in the second network model, wherein the speech model is used to correct the content output by the acoustic model.
[0084] The acoustic model can be understood as the modeling of the sound, which can convert the voice input into the input of the acoustic representation, or it can be understood as giving the probability that the voice belongs to a certain acoustic symbol. The role of the language model can be simply understood as eliminating the problem of polyphones to correct the content output by the acoustic model.
[0085] It should be noted that when the second network model is configured with more language models than the first network model, the conversion accuracy of the second network model will be higher than that of the first network model, and correspondingly, the conversion efficiency of the second network model will be lower than that of the first network model. Under what circumstances, the first network model can be more suitable for real-time conversion scenarios (for example, voice assistants). In this case, if the input audio meets the preset real-time conditions, the determined conversion model is the first network model, and if the input audio does not meet the preset real-time conditions, the determined conversion model is the second network model. Among them, the preset real-time conditions are conditions for determining whether the input audio is in a real-time conversion scenario.
[0086] In the embodiment of the present application, there are multiple ways to determine whether the input audio meets the real-time condition.
[0087] As a method, it is possible to determine whether the preset real-time conditions are met based on the length of the input audio. It should be noted that when the input audio is short, it is relatively easier to accurately convert from speech to text. Correspondingly, when the input audio is long, it is relatively more difficult to accurately convert from speech to text. Therefore, when the input audio is short, selecting the first network model for conversion can achieve accurate conversion while having a higher conversion efficiency. Correspondingly, when the input audio is long, in order to ensure the accuracy of the conversion, the second network model can be selected for conversion.
[0088] Optionally, if the length of the input audio is less than a preset length, it is determined that the input audio meets the preset real-time condition; if the length of the input audio is greater than or equal to the preset length, it is determined that the input audio does not meet the preset real-time condition.
[0089] The preset length may be configured in advance by the developer. Alternatively, the preset length may be determined according to actual conditions. For example, the preset length may be determined according to the device that executes the information retrieval method provided in the embodiment of the present application (for example, the aforementioned Figure 1 as well as Figure 2 electronic device in the Figure 2 It should be noted that the more powerful the processing power of the device, the more data can be processed per unit time. Therefore, for devices with stronger processing power, even if a relatively large network model is run (the larger the network model, the higher the conversion accuracy can be), higher efficiency can be achieved.
[0090] Then as a way, if the processing power of the device is stronger, the preset length can be smaller. Among them, the smaller the preset length, the higher the probability of determining that the input video does not meet the preset real-time conditions when judging the input video, and then selecting the second network model with higher processing accuracy to perform the conversion from language to text, and the processing power of the device itself is strong, so that both higher conversion accuracy and conversion efficiency can be achieved. Correspondingly, if the processing power of the device is weaker, the preset length can be larger. Among them, the larger the preset length, the higher the probability of determining that the input video meets the preset real-time conditions when judging the input video, and then selecting the first network model with lower processing accuracy to perform the conversion from language to text, and then ensuring the conversion efficiency when the processing power of the device is relatively weak.
[0091] In the embodiments of the present application, there are multiple ways to determine the processing capability of a device.
[0092] As a method, the processing capacity of a device can be determined based on the performance of the device's processor. For example, the processing capacity of a device can be determined by the main frequency and core number of the device's processor. The higher the main frequency, the more instructions the processor can complete per unit time, and the faster the processing speed; the more cores there are, the more tasks can be processed simultaneously, and the higher the parallel processing capability is supported.
[0093] As a way, the processing power of a device can be determined based on the cache capacity. The cache is a high-speed channel between the processor and the memory. The larger the cache capacity, the faster the processor can access data and the stronger the data processing capability.
[0094] Alternatively, if the information retrieval task corresponding to the input audio is a preset task, it is determined that the input audio meets the preset real-time condition. It should be noted that in an embodiment of the present application, it may be triggered to obtain the input audio in a task scenario, and subsequent steps may be performed on the input audio. The task scenario can be understood as the scenario corresponding to the information retrieval task corresponding to the input audio. For example, the task scenario may be a voice assistant scenario, in which case the voice assistant may call the microphone of the electronic device to collect the input audio. For another example, the task scenario may be an audio conference scenario.
[0095] Optionally, the preset task can be understood as a task that the user expects to return search results quickly. The preset task can be configured by the user. In this case, the user can configure the task that the user expects to return search results quickly as the preset task.
[0096] S230: Identify the user's search intention through the target text to obtain search intention information.
[0097] S240: Acquire related information corresponding to the search intention information.
[0098] S250: Obtain search input information by searching the intention information and the associated information.
[0099] S260: Input the search input information into the large language model to obtain the search results returned by the large language model.
[0100] This embodiment provides an information retrieval method, so that the conversion of input audio to text can be completed through the conversion model, and the retrieval intention information can be enhanced through the associated information to obtain the retrieval input information, so that the retrieval results finally fed back by the large language model can be more accurate by combining the use of the conversion model and the enhancement of the retrieval intention information. In addition, in this embodiment, a variety of network models for converting speech to text can be pre-configured, so that the conversion model can be flexibly determined according to the current actual situation, which is conducive to improving the flexibility and adaptability of audio processing.
[0101] See also Figure 6 , the present application provides an information retrieval method, the method comprising:
[0102] S310: Convert the input audio into corresponding target text through a conversion model.
[0103] S320: Identify the user's search intention through the target text to obtain search intention information.
[0104] S330: Convert the retrieval intention information into a corresponding first vector representation.
[0105] Among them, the first vector representation can be understood as the vector representation corresponding to the retrieval intention information.
[0106] Among them, the retrieval intention information can be understood as a kind of text, and in the embodiment of the present application, there can be multiple ways to convert the text into a corresponding vector representation.
[0107] As a method, text can be converted into corresponding vector representations based on word embedding. Word embedding is a method of mapping words to a low-dimensional vector space. By learning the embedding vector of each word, the semantic information of the word is encoded into the vector. Among them, the available word embedding models include Word2Vec, GloVe, and FastText.
[0108] As a method, the text can be converted into a corresponding vector representation based on the Bag of Words (BOW) model. The Bag of Words model regards the text as a set of vocabulary items, ignoring the order and grammatical structure between vocabulary items. The vector representation of the text is constructed by counting the number of occurrences or frequencies of each vocabulary item in the text. The commonly used representation method is to use the dimension of the vector to represent the size of the vocabulary, and each dimension of the vector represents the number of occurrences or frequencies of the corresponding vocabulary item.
[0109] As a method, the text can be converted into a corresponding vector representation based on the term frequency-inverse document frequency (TF-IDF). Among them, TF-IDF is a text vectorization method that is improved on the basis of the bag-of-words model. TF (TermFrequency) represents term frequency, that is, the frequency of a word appearing in a specific document; IDF (Inverse DocumentFrequency) represents inverse document frequency, that is, the inverse of the frequency of a word appearing in the entire text collection. TF-IDF calculates the frequency of a vocabulary item in the text and the inverse document frequency in the entire text collection to obtain a weight value, which is used to indicate the importance of the vocabulary item.
[0110] S340: Based on the first vector representation and the content in the vector library, obtain the associated information corresponding to the retrieval intention information, wherein the content in the vector library includes the second vector representation of the pre-collected information.
[0111] The second vector representation of the pre-collected information can be understood as the vector representation of the pre-collected information, and the sources of the content in the vector library can be various. After the collected information is obtained, the collected information can be converted into the second vector representation in the above-mentioned manner.
[0112] After obtaining the aforementioned first vector representation, the associated information corresponding to the search intent information can be obtained from the content in the vector library by calculating the vector similarity. Optionally, when there are multiple pieces of content stored in the vector library, the similarity between the second vector representation of each piece of content and the first vector representation can be calculated, and then the obtained similarities can be sorted, and the content with the highest sorting can be used as the associated information.
[0113] For example, if the multiple contents in the vector library include: content N1, content N2, content N3 and content N4, where the similarity corresponding to content N1 is X1, the similarity corresponding to content N2 is X2, the similarity corresponding to content N3 is X3, and the similarity corresponding to content N4 is X4, if X1 is greater than X2, and X2 is greater than X3, and X3 is greater than X4, and the current rule is to select the content with the highest similarity as the associated information, then content N1 is used as the associated content. Alternatively, the similarity of each content can be compared with the similarity threshold, and then the corresponding content with a similarity greater than the similarity threshold is used as the associated information.
[0114] S350: Obtain search input information by searching the intention information and the associated information.
[0115] In the embodiment of the present application, there may be multiple ways to obtain the retrieval input information based on the associated information and the retrieval intent information.
[0116] As a method, obtaining the search input information through the search intent information and the associated information may include: combining the search intent information and the associated information to obtain the search input information. Combining the search intent information and the associated information may be understood as using preset characters to splice the search intent information and the associated information to obtain the search input information. In this case, the obtained search input information may include the original content of the search intent information and the associated information, and the search intent information and the associated information are connected by the preset characters.
[0117] Exemplarily, the search intent information obtained may be "recommended restaurants in the attached area", and the associated information obtained may include "small noodles", "Cantonese cuisine" and "Hunan cuisine". In this case, if the preset character is "+", the search input information obtained may be "recommended restaurants in the attached area + small noodles + Cantonese cuisine + Hunan cuisine".
[0118] As a method, obtaining the search input information through the search intention information and the associated information may include: updating the search intention information through the associated information to obtain the search input information. In this way, the search input information obtained may only include the original meaning of the associated information, but not all the original contents of the associated information. Exemplarily, the search intention information obtained may be "recommended restaurants in the attached area", and the associated information obtained may include "Cantonese cuisine" and "Hunan cuisine", then the search input information obtained may be "recommended square cuisine restaurants and Hunan cuisine restaurants with better attached areas".
[0119] S360: Input the search input information into the large language model to obtain the search results returned by the large language model.
[0120] In an embodiment of the present application, when displaying the retrieval results returned by the large language model, the display method can be determined according to the specific retrieval results. It should be noted that the retrieval results returned by the large language model may include image content in addition to text content. The image content may include pictures and / or videos. In the case where the retrieval results returned by the large language model include image content, the electronic device can detect whether it has the ability to display the content. If it has the ability to display the content, the electronic device can directly display the image content. Correspondingly, if it is detected that the electronic device itself does not have the ability to display the content, the image content can be transmitted to other devices for display.
[0121] For example, if the electronic device is a smart phone, it will detect that it has the ability to display content, and the electronic device can directly display the image content. For another example, if the electronic device is a smart speaker, it may detect that it does not have the ability to display content, and the image content can be transferred to other devices for display.
[0122] In an embodiment of the present application, an electronic device can detect whether it has the ability to display content in a variety of ways. As one way, the electronic device can automatically detect its own display hardware capabilities through its operating system or driver. It should be noted that electronic devices usually display image content through a screen, and in order for the screen to work properly, a driver for controlling the screen will be installed in the electronic device. In this case, if the electronic device detects that it has a driver for controlling the screen installed, it determines that it has the ability to display content. Otherwise, it is determined that it does not have the ability to display content.
[0123] As another way, a function support file may be stored in the electronic device, and the functions supported by the electronic device are stored in the function support file. Exemplarily, the content of the function support file may be as shown in the following table:
[0124] ID Function Name Support 1 Content Display yes 2 Audio Playback yes 3 Infrared function yes ......
[0125] Based on the above table, the electronic device can determine whether the electronic device is capable of displaying image content by querying the function support file.
[0126] Exemplarily, the input audio obtained by the electronic device may be "Please introduce Shenzhen", then the search results returned by the final big data model may include not only text introduction content about Shenzhen, but also promotional videos. In this case, the electronic device may first detect whether it has the ability to display content. If it is detected that it does not have the ability to display content (for example, when the electronic device is a smart speaker that is not equipped with a screen), the text introduction content may be converted into audio form so that the audio content can be played through the speaker. And the electronic device can transmit the promotional video to a nearby smartphone, TV or tablet computer for playback. For example, the electronic device can first play the audio content locally, and then transfer the promotional video to a nearby smartphone, TV or tablet computer for playback after the playback is completed. If it is detected that it has the ability to display content, the electronic device can introduce the text introduction content and the promotional video locally.
[0127] The present embodiment provides an information retrieval method, so that the conversion of input audio to text can be completed through the conversion model, and the retrieval intention information can be enhanced through the associated information to obtain the retrieval input information, so that by combining the use of the conversion model and the enhancement of the retrieval intention information, the retrieval results finally fed back by the large language model can be more accurate. In addition, in the present embodiment, information can be collected in advance through various channels, and the collected information can be processed to obtain the content in the vector library, so that after the retrieval intention information is obtained, the associated information can be obtained from the vector library by vector comparison, which is conducive to improving the accuracy of the associated information obtained, and is also conducive to the subsequent large language model to perform more accurate information retrieval.
[0128] Next, pass Figure 7 The information retrieval method involved in the embodiment of the present application is introduced below with reference to the contents shown in .
[0129] like Figure 7 As shown, the system architecture involved in the information retrieval method includes a voice command parsing layer, a command retrieval enhancement layer and a command response layer.
[0130] In the voice command parsing layer, after the user speaks the voice content, the voice collector can collect the voice content spoken by the user. Among them, the voice collector can be called through APP, H5 interface or SDK. After the voice collector collects the audio data stream (input audio), the audio data stream can be transmitted to the voice receiver. Among them, the audio data stream received by the voice receiver can be in FFmpeg, PCM (Pulse Code Modulation), FLAC (Free Lossless Audio Codec) or MP3 (MPEG-1 Audio Layer III) format. Among them, the voice collector can be running in the device on the user side, and the voice receiver can be running in the server. After the voice collector receives the audio data stream, the audio data stream can be stored in the message queue, and then converted into the corresponding target text by real-time dictation or offline transcription.
[0131] Among them, in the voice command parsing layer, the conversion is completed by the automatic speech recognition module (Automatic Speech Recognition). Figure 7 As shown, there are two corresponding conversion paths in the automatic speech recognition module. In one conversion path, after endpoint detection, the target text is obtained by successively passing through the acoustic model, punctuation prediction, and inverse text regularization. In the other conversion path, after endpoint detection, the target text is obtained by successively passing through the acoustic model, language model, punctuation prediction, and inverse text regularization.
[0132] Among them, Inverse Text Normalization (ITN) can play a key role in the text post-processing module of speech recognition results. It realizes the text conversion from the spoken domain (oral expression) to the written domain (written expression). In the speech recognition system, since the speech input may contain non-standard, colloquial expressions, the role of inverse text normalization is to convert these non-standard texts into a more standard form that conforms to written reading habits, thereby making the converted target text clearer and easier to understand.
[0133] The target text output by the voice command parsing layer will be stored in the message queue between the voice command parsing layer and the command retrieval enhancement layer.
[0134] In the instruction retrieval enhancement layer, the retrieval enhancement generator can be used to generate
[0135] Generator) is used to perform search enhancement. Specifically, after obtaining the target text, the search enhancement generator can perform query rewriting and keyword extraction operations on the target text, thereby realizing subsequent intent recognition (i.e., obtaining search intent information). After completing intent recognition, search optimization and prompt word processing can be performed in sequence to obtain case search input information. Furthermore, the search input information can be input into the large language model to obtain the search results fed back by the large language model. For the search results, they can be first stored in the message queue between the instruction search enhancement layer and the instruction response layer.
[0136] Among them, prompt word processing can be understood as: after receiving the intended content of the query and retrieval (retrieval input information), the related prompts other than the retrieval content can be supplemented and expanded around the large language model, such as: specifying the role of the model (professional weather forecaster), listing the steps required to complete the task (querying the weather conditions of the day and reporting the content in the way of CCTV weather forecast), etc.
[0137] After the speech synthesizer (Text-to-Speech) of the command response layer obtains the search results, it can perform text analysis on the search results, and then obtain the search results in audio form through processing steps such as acoustic model and waveform synthesis.
[0138] See also Figure 8 The present application provides an information retrieval device 400, the device 400 comprising:
[0139] The speech conversion unit 410 is used to convert the input audio into the corresponding target text through the conversion model.
[0140] The intention recognition unit 420 is used to recognize the user's search intention through the target text to obtain search intention information.
[0141] The search enhancement unit 430 is used to obtain search input information through search intention information and corresponding associated information.
[0142] The search execution unit 440 is used to input the search input information into the large language model to obtain the search results returned by the large language model.
[0143] As a method, the speech conversion unit 410 is specifically configured to determine a target network model from a plurality of network models in response to receiving input audio, wherein the plurality of network models have different conversion accuracies and / or conversion efficiencies.
[0144] As a method, the speech conversion unit 410 is specifically used to convert the input audio into the corresponding target text through the acoustic model included in the first network model if the determined conversion model is the first network model; if the determined conversion model is the second network model, the input audio is converted into the corresponding target text through the acoustic model and language model included in the second network model, wherein the speech model is used to correct the content output by the acoustic model.
[0145] As a method, the speech conversion unit 410 further specifically determines that if the input audio meets the preset real-time condition, the conversion model is the first network model; if the input audio does not meet the preset real-time condition, the conversion model is the second network model.
[0146] Optionally, the speech conversion unit 410 further specifically determines that the input audio meets a preset real-time condition if the length of the input audio is less than a preset length; or, if the information retrieval task corresponding to the input audio is a preset task, determines that the input audio meets the preset real-time condition.
[0147] As a method, the retrieval enhancement unit 430 is specifically used to convert the retrieval intention information into a corresponding first vector representation; based on the first vector representation and the content in the vector library, obtain the associated information corresponding to the retrieval intention information, wherein the content in the vector library includes a second vector representation of pre-collected information.
[0148] As a method, the search enhancement unit 430 is specifically used to combine the search intention information and the associated information to obtain the search input information; or, to update the search intention information through the associated information to obtain the search input information.
[0149] As a method, the search execution unit 440 is specifically used to input the search input information into the large language model to obtain the search results in text form returned by the large language model, convert the search results in text form into search results in audio form, and play the search results in audio form.
[0150] The present application provides an information retrieval device, through which the above-mentioned device can complete the conversion of input audio into text through a conversion model, and can enhance the retrieval intention information through associated information to obtain retrieval input information, so that by combining the use of the conversion model and the enhancement of the retrieval intention information, the retrieval results ultimately fed back by the large language model can be more accurate.
[0151] It should be noted that the device embodiments in the present application correspond to the aforementioned method embodiments, and the specific implementation principles of each unit in the device embodiments are similar to the principles in the aforementioned method embodiments. The specific contents in the device embodiments can be found in the method embodiments, and will not be repeated in the device embodiments.
[0152] The following will be combined Fig. 9 An electronic device provided by the present application is described.
[0153] See also Fig. 9 Based on the above information retrieval method and device, the embodiment of the present application also provides an electronic device 1000 that can execute the above information retrieval method. The electronic device 1000 includes one or more (only one is shown in the figure) processors 105, a memory 104, an audio playback module 106 and an audio acquisition device 108 that are coupled to each other. Among them, the memory 104 stores a program that can execute the content of the above embodiment, and the processor 105 can execute the program stored in the memory 104.
[0154] Among them, the processor 105 may include one or more processing cores. The processor 105 uses various interfaces and lines to connect various parts of the entire electronic device 1000, and executes various functions and processes data of the electronic device 1000 by running or executing instructions, programs, code sets or instruction sets stored in the memory 104, and calling data stored in the memory 104. Optionally, the processor 105 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 105 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 105, but may be implemented separately through a communication chip.
[0155] The memory 104 may include a random access memory (RAM) or a read-only memory (ROM). The memory 104 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 104 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc.
[0156] Furthermore, in addition to the aforementioned components, the electronic device 1000 may further include a network module 110 and a sensor module 112 .
[0157] The network module 110 is used to realize information interaction between the electronic device 1000 and other devices. For example, a connection can be established with other audio playback devices or other electronic devices, and information interaction is performed based on the established connection. As a way, the network module 110 of the electronic device 1000 is a radio frequency module, which is used to receive and send electromagnetic waves, realize the mutual conversion of electromagnetic waves and electrical signals, and thus communicate with a communication network or other devices. The radio frequency module may include various existing circuit elements for performing these functions, such as antennas, radio frequency transceivers, digital signal processors, encryption / decryption chips, user identity modules (SIM) cards, memories, etc. For example, the radio frequency module can interact with external devices through electromagnetic waves sent or received.
[0158] The sensor module 112 may include at least one sensor. Specifically, the sensor module 112 may include but is not limited to: a pressure sensor, a motion sensor, an acceleration sensor, and other sensors.
[0159] Among them, the pressure sensor may be a sensor that detects the pressure generated by pressing on the electronic device 1000. That is, the pressure sensor detects the pressure generated by contact or pressing between the user and the electronic device 1000, for example, the pressure generated by contact or pressing between the user's ear and the electronic device 1000. Therefore, the pressure sensor can be used to determine whether contact or pressing occurs between the user and the electronic device 1000, and the magnitude of the pressure.
[0160] Among them, the acceleration sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary, and can be used for applications that recognize the posture of the electronic device 1000 (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc. In addition, the electronic device 1000 can also be equipped with other sensors such as gyroscopes, barometers, hygrometers, thermometers, etc., which will not be repeated here.
[0161] The audio collection device 108 is used to collect audio signals. Optionally, the audio collection device 110 includes a plurality of audio collection devices, which may be microphones.
[0162] Please refer to Fig.10 , which shows a structural block diagram of a computer-readable storage medium provided in an embodiment of the present application. The computer-readable medium 1100 stores program codes, which can be called by a processor to execute the method described in the above method embodiment.
[0163] The computer readable storage medium 1100 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk, or a ROM. Optionally, the computer readable storage medium 1100 includes a non-transitory computer-readable storage medium. The computer readable storage medium 1100 has storage space for program code 1110 that executes any method step in the above method. These program codes can be read from or written to one or more computer program products. The program code 1110 can be compressed, for example, in an appropriate form.
[0164] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.
[0165] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.
[0166] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.
[0167] In summary, the present application provides an information retrieval method, device, and electronic device, in which the input audio can be first converted into a corresponding target text through a conversion model, so as to identify the user's search intent through the target text to obtain search intent information; and then the search input information can be obtained through the search intent information and the associated information, so as to input the search input information into the large language model to obtain the search results returned by the large language model. Thus, through the above-mentioned method, the conversion of input audio to text can be completed through the conversion model, and the search intent information can be enhanced through the associated information to obtain the search input information, so that by combining the use of the conversion model and the enhancement of the search intent information, the search results finally fed back by the large language model can be more accurate.
[0168] In the embodiments of the present application, by introducing emerging technologies such as speech recognition model (the aforementioned conversion model), general large model, external training data set, semantic recognition model, speech synthesis model, etc., the robustness and accuracy of speech understanding, speech conversion, content retrieval and feedback in the speech interaction link are enhanced. Compared with traditional solutions, human-computer interaction methods such as speech interaction can achieve higher information transmission efficiency, faster acoustic information conversion, and more accurate information retrieval in cross-space scenarios such as furniture, car, education, travel, and medical care.
[0169] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical feature diagrams therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims
1. An information retrieval method, characterized in that: The method comprises: Convert the input audio into the corresponding target text through the conversion model; Identifying the user's search intent through the target text to obtain search intent information; Obtaining search input information through the search intention information and the corresponding associated information; The search input information is input into a large language model to obtain a search result returned by the large language model.
2. The method according to claim 1, characterized in that The step of converting the input audio into the corresponding target text by using the conversion model includes: If the determined conversion model is the first network model, the input audio is converted into the corresponding target text by using the acoustic model included in the first network model; If the determined conversion model is the second network model, the input audio is converted into the corresponding target text through the acoustic model and the language model included in the second network model, wherein the speech model is used to correct the content output by the acoustic model.
3. The method according to claim 2, characterized in that The method further comprises: If the input audio satisfies the preset real-time condition, the determined conversion model is the first network model; If the input audio does not meet the preset real-time condition, the determined conversion model is the second network model.
4. The method according to claim 3, characterized in that The method further comprises: If the length of the input audio is less than the preset length, determining that the input audio meets the preset real-time condition; or, If the information retrieval task corresponding to the input audio is a preset task, it is determined that the input audio meets the preset real-time condition.
5. The method according to claim 1, characterized in that The method further comprises: Converting the search intent information into a corresponding first vector representation; Based on the first vector representation and the content in the vector library, the associated information corresponding to the retrieval intention information is obtained, wherein the content in the vector library includes the second vector representation of the pre-collected information.
6. The method according to claim 1, characterized in that The obtaining of the search input information through the search intention information and the associated information includes: Combining the search intention information and the associated information to obtain search input information; or, The search intention information is updated through the associated information to obtain search input information.
7. The method according to claim 1, characterized in that Inputting the search input information into the large language model to obtain the search results returned by the large language model includes: Inputting the search input information into a large language model to obtain a search result in text form returned by the large language model; Converting the text-based search results into audio-based search results; Play the search results in audio format.
8. An information retrieval device, characterized in that: The device comprises: A speech conversion unit, used to convert the input audio into the corresponding target text through the conversion model; An intention recognition unit, used to recognize the user's search intention through the target text to obtain search intention information; A search enhancement unit, used to obtain search input information through the search intention information and the corresponding associated information; The retrieval execution unit is used to input the retrieval input information into the large language model to obtain the retrieval result returned by the large language model.
9. An electronic device, characterized in that: including one or more processors and memory; One or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the method according to any one of claims 1 to 7.
10. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 7 when being executed by a processor.
Citation Information
Cited By
Interaction method and device based on intelligent agent, storage medium and computer equipment
CN119476502A
Visualization method driven by SQL generation based on large model preference learning
CN120030036A
Task retrieval method, electronic equipment and medium
CN120448531A
Interaction method and device based on multi-model cooperation, intelligent agent and electronic equipment
CN120805926A
File transmission method and device and related equipment
CN121173804A
