Electronic device and method for operating virtual assistant thereof
Patent Information
- Authority / Receiving Office
- TW · TW
- Patent Type
- Applications
- Current Assignee / Owner
- ASUSTEK COMPUTER INC
- Filing Date
- 2025-01-22
- Publication Date
- 2026-08-01
AI Technical Summary
Existing virtual assistants have limited understanding of diverse user commands and rely on cloud services, which can compromise privacy and security due to network latency and data upload risks.
Implementing a local semantic search using a language embedding model to generate a target prompt instruction for a generative natural language model, enabling the virtual assistant to understand complex commands without cloud reliance.
Enhances the virtual assistant's understanding of diverse commands, improves response accuracy, and ensures privacy and security by processing data locally.
Smart Images
Figure TWG2TA001069725_001 
Figure TWG2TA001069725_002 
Figure TWG2TA001069725_003
Abstract
Description
[Technical Field]
[0001] This disclosure relates to an electronic device and a method of operating its virtual assistant. [Previous Technology]
[0002] With the advancement of technology, virtual assistants with permanent background access have been widely used in devices such as mobile phones, computers, and smart speakers. Current virtual assistants can assist with some simple tasks; however, existing virtual assistants on devices typically only understand a limited number of preset commands, making it difficult to meet the diverse needs of users and thus affecting the user experience. On the other hand, existing virtual assistants can also use large-scale models or language analysis systems in the cloud to interpret user requests. Although cloud services have a deeper understanding of natural language, the potential network latency and the risk of user privacy being uploaded to the cloud often limit the user experience and even raise concerns about privacy and security. [Summary of the Invention]
[0003] This disclosure provides a virtual assistant operation method applicable to an electronic device including an input device and an output device. The method includes the following steps: Receiving a user command via the input device through the virtual assistant; Obtaining a semantic search result by performing a semantic search on a database containing multiple functional description texts based on the user command using a Language Embedding Model; Generating a target prompt instruction, including a target text or the user command from the multiple functional description texts, based on the semantic search result; Inputting the target prompt instruction into a generative natural language model, and outputting the model response text of the generative natural language model through the output device.
[0004] This disclosure provides an electronic device including an output device, an input device, a storage device, and a processor. The storage device records a plurality of instructions. The processor is coupled to the output device, the input device, and the storage device, and is configured to execute the aforementioned instructions to perform the following operations: Receiving a user command via a virtual assistant through the input device; Obtaining a semantic search result by performing a semantic search in a database containing a plurality of functional description texts based on the user command using a language embedding model; Generating a target prompt instruction including a target text or the user command from the plurality of functional description texts based on the semantic search result; Inputting the target prompt instruction into a generative natural language model, and outputting the model response text of the generative natural language model through the output device.
[0005] Based on the above, in this embodiment of the invention, a semantic search can be performed on a database containing multiple functional description texts according to the user's command, so as to generate a target prompt instruction for input into the generative natural language model based on the semantic search results. The target prompt instruction can then be input into the generative natural language model to generate model response text, enabling the virtual assistant to provide the model response text to the user through an output device. Therefore, the virtual assistant can better understand diverse user commands, and the correctness and accuracy of the virtual assistant's responses can be improved based on the database's recorded data.
[0006] To make this disclosure more obvious and understandable, specific embodiments are provided below, and detailed descriptions are given in conjunction with the accompanying drawings.
Implementation Method
[0007] Some embodiments of the present invention will now be described in detail with reference to the accompanying drawings. Component symbols used in the following description are considered the same or similar components when they appear in different drawings. These embodiments are only a part of the present invention and do not disclose all possible implementations of the invention. More precisely, these embodiments are merely examples of the apparatus and methods within the scope of the present invention's patent application.
[0008] Referring to Figure 1, in this embodiment, the electronic device 100 may include an input device 110, an output device 120, a storage device 130, and a processor 140. The electronic device 100 may be, for example, a smartphone, a laptop, a tablet computer, a desktop computer, or a smart wearable device, etc., an electronic device with virtual assistant services, and this embodiment is not limited thereto.
[0009] The input device 110 is used to receive user input information, such as a touch input device, keyboard, mouse, or microphone, etc., and this invention is not limited to this. In the embodiments of this invention, the input device 110 can be used to receive user commands input by the user.
[0010] The storage device 130 is used to store data and software modules (such as operating systems, applications, drivers) accessible to the processor 140. It can be, for example, any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory, hard disk, or a combination thereof.
[0011] The output device 120 is used to output information, such as a speaker or a display, etc., and this invention is not limited to this. The display may be a liquid crystal display (LCD), a light-emitting diode (LED) display, an organic light-emitting diode (OLED) display, or other types of displays, and this invention is not limited to this. In this embodiment, the display may show the user interface of the virtual assistant service.
[0012] Processor 140 is coupled to input device 110, output device 120, and storage device 130. Processor 140 is, for example, a central processing unit (CPU), application processor (AP), or other programmable general-purpose or special-purpose microprocessor, digital signal processor (DSP), image signal processor (ISP), graphics processing unit (GPU), or other similar devices, integrated circuits, or combinations thereof. Processor 140 can access and execute software modules recorded in storage device 130 to implement the virtual assistant operation method in the embodiments of the present invention. The above-mentioned software modules can be broadly interpreted as instructions, instruction sets, code, program code, program, application, software suite, thread, process, function, etc., regardless of whether it is called software, firmware, middleware, microcode, hardware description language, or others.
[0013] Please refer to Figures 1 and 2 simultaneously. The method of this embodiment is applicable to the electronic device 100 described above. The following describes the detailed steps of the virtual assistant operation method of this embodiment in conjunction with the various components of the electronic device 100. To clearly explain the possible implementation of this invention, Figure 3 will be used as a supplementary explanation below. Please refer to Figure 3 as well.
[0014] In step S210, the processor 140 receives a user command UC1 via an input device 110 through a virtual assistant. In some embodiments, the virtual assistant may be an embedded digital assistant embedded in an operating system (OS). The virtual assistant running on the processor 140 may receive the user command UC1 via the input device 110. In addition, the virtual assistant may provide voice output or visual output corresponding to the user command UC1 via the output device 120 of the electronic device 100.
[0015] In different embodiments, the user command UC1 may be a voice input command or a text input command. For example, the user may use the input device 110 to input the user command UC1 in the input field of the user interface displayed on the screen. Alternatively, the user may speak the user command UC1, and the processor 140 may receive the voice input through the input device 110. The user command UC1 is an unformatted natural language command.
[0016] In some embodiments, the user can wake up the virtual assistant by speaking a wake-up keyword, performing a specific wake-up gesture, or pressing a wake-up hotkey. During the wake-up of the virtual assistant, the processor 140 can receive user command UC1 via the virtual assistant.
[0017] In step S220, the processor 140 uses a Language Embedding Model M31 to perform a semantic search in a database that records multiple functional description texts FT_1 to FT_n according to the user command UC1, thereby obtaining a semantic search result SR1.
[0018] The language embedding model M31 can be used to convert input text into semantic feature vectors in a multi-dimensional feature space. In some embodiments, the language embedding model M31 can capture the semantic and contextual information of the text. As shown in Figure 3, the language embedding model M31 can be used to convert the user command UC1 into the corresponding semantic feature vector SF1. In different embodiments, the language embedding model M31 is, for example, a BERT (Bidirectional Encoder Representations from Transformers) model, a GPT (Generative Pre-trained Transformer) model, a Bag-of-Words Model, or a USE (Universal Sentence Encoder) model, etc., and this application is not limited to this.
[0019] In some embodiments, when the processor 140 performs a semantic search, the processor 140 searches for target text with similar semantics from the text database db1 based on the semantic feature vector SF1 of the user command UC1. More specifically, the text database db1 records multiple functional description texts FT_1 to FT_n. The functional description texts FT_1 to FT_n may include multiple device knowledge texts associated with the electronic device 100 (e.g., operation suggestions, operation instructions, and basic device information, etc.). Alternatively, the functional description texts FT_1 to FT_n may include multiple application auxiliary texts (e.g., the name of the third-party application and the services it provides, etc.) that the virtual assistant can invoke.
[0020] As shown in Figure 3, the functional description texts FT_1 to FT_n in the text database db1 can be converted into corresponding semantic feature vectors FF_1 to FF_n using the language embedding model M31. The semantic feature vectors FF_1 to FF_n of each of the functional description texts FT_1 to FT_n can be recorded in the vector database db2. Specifically, the semantic feature vectors FF_1 to FF_n can be respectively bound to the functional description texts FT_1 to FT_n and stored in the vector database db2. The text database db1 and the vector database db2 can be stored in the storage device 130.
[0021] When the processor 140 performs a semantic search, the processor 140 can calculate the semantic similarity between the semantic feature vector SF1 of the user command UC1 and the semantic feature vectors FF_1 to FF_n of each function description text FT_1 to FT_n. For example, the processor 140 can calculate the cosine similarity, Euclidean distance, or Manhattan distance between two semantic feature vectors to obtain the semantic similarity between the semantic feature vector SF1 and each semantic feature vector FF_1 to FF_n. In some embodiments, when the semantic similarity between two semantic feature vectors is higher than a threshold value, the processor 140 can determine that the two semantic feature vectors are similar to each other. When the semantic similarity between two semantic feature vectors is not higher than the threshold value, the processor 140 can determine that the two semantic feature vectors are not similar to each other.
[0022] Based on the binding relationship between multiple semantic feature vectors FF_1 to FF_n and functional description texts FT_1 to FT_n, the processor 140 can determine whether the functional description texts FT_1 to FT_n include target texts semantically similar to the user command UC1 based on the semantic similarity between each semantic feature vector FF_1 to FF_n and the semantic feature vector SF1. For example, when the semantic similarity between semantic feature vector FF_1 and semantic feature vector SF1 meets the search conditions, the processor 140 can determine that the semantic search result SR1 includes the target text matching the user command UC1, and this target text is the functional description text FT_1. When the semantic similarity between all semantic feature vectors FF_1 to FF_n and semantic feature vector SF1 does not meet the search conditions, the processor 140 can determine that the semantic search result SR1 does not include the target text matching the user command UC1.
[0023] In step S230, the processor 140 generates a target prompt instruction TP1, which includes a target text or a user command UC1, from multiple function description texts FT_1 to FT_n, based on the semantic search result SR1.
[0024] In some embodiments, when the semantic search result SR1 includes target text matching the user command UC1, the processor 140 can generate a target prompt instruction TP1 that includes the target text from a plurality of functional description texts FT_1 to FT_n. Further, when the semantic search result SR1 includes target text matching the user command UC1, the target prompt instruction TP1 including the target text can be used to request the generative natural language model M32 to generate model output based on the target text. On the other hand, when the semantic search result SR1 does not include target text matching the user command UC1, the processor 140 can generate a target prompt instruction TP1 including the user command UC1. Further, when the semantic search result SR1 does not include target text matching the user command UC1, the processor 140 can infer that the user command UC1 is general chat-oriented, and the target prompt instruction TP1 including the user command UC1 can be used to request the generative natural language model M32 to directly generate model output based on the user command UC1.
[0025] In step S240, the processor 140 inputs the target prompt instruction TP1 into a generative natural language model M32, and outputs the model response text MR1 of the generative natural language model M32 through the output device 120. The generative natural language model M32 can output natural language text based on the input text. In different embodiments, the generative natural language model M32 may be, for example, a GPT (Generative Pre-trained Transformer) model, a T5 model, or an LLaMA (Large Language Model Meta AI) model, etc., and this application is not limited to this. That is, the generative natural language model M32 can respond to receiving the target prompt instruction TP1 and output the model response text MR1. Afterwards, the virtual assistant can provide the model response text MR1 to the user through the output device 120.
[0026] In some embodiments, the model parameters of the generative natural language model M32 are recorded in the storage device 130, and the generative natural language model M32 can run on the local electronic device 100. Furthermore, in this embodiment, the virtual assistant of the electronic device 100 can more accurately understand the user's complex requests through local databases and semantic searches. In summary, the electronic device 100 does not need to rely on cloud services to understand the user's command UC1 and generate the virtual assistant's response, thereby enhancing privacy protection, data security, and avoiding response delays.
[0027] Please refer to Figures 1, 4A, and 4B simultaneously. The method of this embodiment is applicable to the electronic device 100 described above. The following describes the detailed steps of the virtual assistant operation method of this embodiment in conjunction with the various components of the electronic device 100.
[0028] In step S410, the processor 140 receives a user command through the input device 110 via a virtual assistant.
[0029] In step S420, the processor 140 performs a semantic search on a database containing multiple functional description texts based on a user command using a language embedding model, thereby obtaining a semantic search result. In some embodiments, the processor 140 may perform a semantic search on multiple text databases, which respectively record texts of different text categories. In some embodiments, step S420 may be implemented as steps S421 to S427.
[0030] In step S421, the processor 140 uses a language embedding model to obtain the semantic feature vector of the user command. In step S422, the processor 140 determines whether the semantic feature vector of the user command is similar to multiple semantic feature vectors of multiple functional description texts. The operating principles of steps S421 to S422 have been described above and will not be repeated here.
[0031] In step S423, when the semantic feature vector of the user command is similar to the semantic feature vector of the target text in multiple functional description texts, the processor 140 can obtain semantic search results including the target text matching the user command. The target text is one of multiple functional description texts. When the semantic feature vector of the user command is similar to the semantic feature vector of the target text in multiple functional description texts, it means that the processor 140 can search for the target text matching the semantic feature vector of the user command from multiple functional description texts in the database. Based on this, in step S423, the processor 140 can obtain semantic search results including the target text matching the user command.
[0032] On the other hand, in step S424, when the semantic feature vector of the user command is not similar to the semantic feature vectors of multiple functional description texts, the processor 140 can generate an extended query string based on the user command. That is, when the processor 140 cannot retrieve any functional description text matching the semantic feature vector of the user command from the database, the processor 140 can generate an extended query string based on the user command. In other words, the processor 140 can expand the user command into a variety of strings to search for text content that may meet the user's needs from the database.
[0033] In some embodiments, the processor 140 may generate an expanded query string based on a query expansion suggestion and a user command using a generative natural language model. Specifically, the query expansion suggestion requests the generative natural language model to generate a string with similar semantics or word meaning. Therefore, when the generative natural language model receives the query expansion suggestion and the user command, the generative natural language model may output an expanded query string with similar semantics or word meaning to the user command. Alternatively, in some embodiments, the expanded query string may be generated based on the principle of word association. Furthermore, this invention does not limit the number of expanded query strings; it can be set according to the actual application.
[0034] In step S425, processor 140 uses a language embedding model to obtain the semantic feature vector of the extended query string. In step S426, processor 140 determines whether the semantic feature vector of the extended query string is similar to multiple semantic feature vectors of multiple functional description texts. The operations of obtaining semantic feature vectors and determining semantic similarity in steps S425 to S426 have been explained above and will not be repeated here.
[0035] In step S423, when the semantic feature vector of the extended query string is similar to the semantic feature vector of the target text in multiple functional description texts, the processor 140 can obtain semantic search results including the target text matching the user command. Furthermore, when the semantic feature vector of the extended query string is similar to the semantic feature vector of the target text in multiple functional description texts, the processor 140 can determine that the semantic search results include the target text.
[0036] In step S427, when the semantic feature vector of the extended query string is not similar to the multiple semantic feature vectors of multiple functional description texts, the processor 140 can obtain semantic search results that do not include the target text that matches the user command.
[0037] It should also be noted that, in some embodiments, the processor 140 may repeatedly execute steps S424 to S426 to compare whether the semantic feature vectors of multiple extended query strings are similar to the multiple semantic feature vectors of multiple functional description texts. When the number of extended query strings reaches a certain number and the semantic feature vectors of these extended query strings are not similar to the multiple semantic feature vectors of multiple functional description texts, the processor 140 may decide not to include semantic search results that match the target text of the user command.
[0038] Next, referring to FIG4B, in step S430, the processor 140 generates a target prompt instruction that includes a target text or user command from a plurality of functional description texts based on the semantic search results. In some embodiments, step S430 may be implemented as steps S431 to S432.
[0039] In step S431, when the semantic search result includes target text that matches the user command, the processor 140 can generate a target prompt instruction that includes the target text according to a preset prompt format. That is, when the processor 140 finds target text with similar semantics in the text database according to the user command or its extended query string, the processor 140 can generate a target prompt instruction that includes the target text according to a preset prompt format.
[0040] In some embodiments, when the target text is device knowledge text, the processor 140 can generate a target prompt instruction including the target text according to a first preset format. For example, assuming the target text is an operation suggestion recorded in a text database explaining how to solve device power consumption, the target prompt instruction generated by the processor 140 may be "Please reply to the user command according to the 'target text'". That is, the target text may include device knowledge text, and the target prompt instruction can be used to request a generative natural language model to answer the user command according to the device knowledge text.
[0041] For example, suppose the user's command is "What to do about the phone's battery draining too quickly?" The processor 140 can search for one or more target texts in the device knowledge database based on the semantic feature vector of "What to do about the phone's battery draining too quickly?". These target texts are "Power-saving tips for mobile phones 1", "Power-saving tips for mobile phones 2", and "Power-saving tips for mobile phones 3". Therefore, the target prompt instruction generated by the processor 140 may include, for example, "Please answer 'What to do about the phone's battery draining too quickly' based on the following documents" and "Power-saving tips for mobile phones 1", "Power-saving tips for mobile phones 2", and "Power-saving tips for mobile phones 3". Based on this, the generative natural language model will respond to the user's question based on the "Power-saving tips for mobile phones 1", "Power-saving tips for mobile phones 2", and "Power-saving tips for mobile phones 3" recorded in the text database, thereby improving the accuracy and correctness of the response.
[0042] In some embodiments, when the target text is application auxiliary text, the processor 140 can generate a target prompt instruction including the target text according to a second preset format. For example, assuming the target text is the application name and corresponding application function of a third-party application, the target prompt instruction generated by the processor 140 may be "Please generate a program execution instruction to control the 'application name of the third-party application' to execute the 'application function' according to the 'user command'". The aforementioned program execution instruction may be an API instruction provided to the third-party application. That is, the target text may include application auxiliary text, and the target prompt instruction is used to request the generative natural language model to generate a program execution instruction based on the application auxiliary text and the user command.
[0043] In step S432, when the semantic search results do not include target text matching the user command, the processor 140 may generate a target prompt instruction that includes the user command. That is, when the processor 140 is unable to find target text with similar semantics from the text database based on the user command or its extended query string, the processor 140 may directly use the user command as the target prompt instruction.
[0044] In step S440, the processor 140 inputs the target prompt instruction into a generative natural language model and outputs the model response text of the generative natural language model through the output device 120. In some embodiments, step S440 may be implemented as steps S441 to S448.
[0045] In step S441, when the target text is application auxiliary text, the processor 140 can use a generative natural language model to generate a program execution instruction based on the application auxiliary text and the user command. Next, in step S443, the processor 140 can provide the program execution instruction generated by the generative natural language model based on the target prompt instruction to a third-party application. In step S444, the processor 140 can provide the program output data of the third-party application to the generative natural language model, so that the generative natural language model can output model response text based on the program's generated data.
[0046] For example, suppose the user command is "Control the living room air conditioner to turn on at 6 PM". The processor 140 can search for one or more target texts in the device knowledge database based on the semantic feature vector of "Control the home air conditioner to turn on at 6 PM", and these target texts are "Smart home appliance control application sets device turn-on time". Then, the processor 140 can generate API instructions (i.e., program execution instructions) for the smart home appliance control application (i.e., third-party application) based on the target text "Smart home appliance control application sets home appliance turn-on time" and the user command "Control the home air conditioner to turn on at 6 PM". The program execution instructions may include the time parameter "6 PM" in the user command, the control target parameter "living room air conditioner", and the program action "turn on device". Then, the smart home appliance control application can execute subsequent application functions after obtaining the program execution instructions and reply with an application function execution result (i.e., program output data). Then, the generative natural language model can output model reply text based on the application function execution result. For example, if the application function's execution result is "Setup complete", the generative natural language model's response text could be "The living room air conditioner will be turned on at 6 PM". If the application function's execution result is "Setup failed", the generative natural language model's response text could be "The living room air conditioner setup failed".
[0047] Furthermore, in step S445, when the target text is device knowledge text, the processor 140 can use a generative natural language model to answer the user's command based on the device knowledge text and generate model response text. That is, when the processor 140 searches for semantically similar device knowledge text in the device knowledge database according to the user's instruction, the processor 140 can use a generative natural language model to answer the user's command based on the device knowledge text and generate model response text.
[0048] Alternatively, in step S446, when the target prompt instruction does not include the target text, the processor 140 may use a generative natural language model to generate model response text based on the user command. That is, in some embodiments, when the processor 140 does not find any semantically similar text from the local database based on the user command, the processor 140 may directly input the user command into the generative natural language model.
[0049] In step S447, the processor 140 can determine whether the model response text is qualified. In some embodiments, the processor 140 can use a natural language classifier or, more specifically, a generative natural language model to determine whether the model response text is qualified. The input of the natural language classifier is text, and its output is the probability of being qualified. If the output of the natural language classifier is lower than a threshold, the processor 140 determines that the input model response text is unqualified. When the processor 140 uses a generative natural language model to determine this, the natural language model can output which categories the input model response text might belong to, such as violence, hatred, harm, or sexually related content. Conversely, the processor 140 can determine that the model response text is qualified. In step S448, when the model response text is qualified, the processor 140 can output the model response text of the generative natural language model through the output device 120. In this way, the user can obtain the model response text generated by the virtual assistant using the generative natural language model.
[0050] Please refer to Figure 5, which is a schematic diagram of a virtual assistant according to an embodiment of this case. User U1 can provide user command UC1 to the virtual assistant. The semantic search module 510 of the virtual assistant can use a language embedding model to perform a semantic search on the device knowledge database db51 and the third-party application database db52 according to the user command UC1. The device knowledge database db51 can record multiple device knowledge texts, which may include basic device information, device operation suggestions, or device operation instructions, etc. The third-party application database db52 can record multiple application auxiliary texts, which may include multiple third-party applications and their executable application functions.
[0051] When the semantic search module 510 cannot find device knowledge text or application auxiliary text matching the user command UC1 based on the semantic feature vector of the user command UC1, the virtual assistant's query expansion module 520 can generate one or more extended query strings based on the user command UC1 using the generative natural language model M51. The semantic search module 510 can use a language embedding model to perform a semantic search on the device knowledge database db51 and the third-party application database db52 based on one or more extended query strings.
[0052] Next, the semantic search module 510 can provide the semantic search results to the prompt generation module 530 of the virtual assistant. The prompt generation module 530 can generate a target prompt instruction based on the semantic search results and input the target prompt instruction into the generative natural language model M51. When the target prompt instruction includes a device knowledge text in the device knowledge database db51, the target prompt instruction can include the device knowledge text and the user command UC1, causing the generative natural language model M51 to output model response text based on the device knowledge text and the user command UC1. When the semantic search results do not include any device knowledge text or application assistance text matching the user command UC1, the target prompt instruction is the user command UC1, and the generative natural language model M51 can output model response text based on the user command UC1.
[0053] When the target prompt instruction includes an application auxiliary text in the device knowledge database db52, the target prompt instruction may include the application auxiliary text and the user command UC1, causing the generative natural language model M51 to generate program execution instructions based on the application auxiliary text and the user command UC1 and output the program execution instructions to the instruction executor 540. The instruction executor 540 may provide the program execution instructions to a third-party application and input the program response data of the third-party application into the generative natural language model M51. The generative natural language model M51 may output model response text based on the program response data of the third-party application. Thus, the virtual assistant's response filter 550 may determine whether the model response text is qualified in order to decide whether to provide the model response text to the user U1.
[0054] In summary, in this embodiment of the invention, a semantic search can be performed on a database containing multiple functional description texts based on user commands, and a target prompt instruction can be generated based on the semantic search results to be input into a generative natural language model. The target prompt instruction can then be input into the generative natural language model to generate model response text, allowing the virtual assistant to provide the model response text to the user via an output device. Based on this, the virtual assistant can better understand diverse user commands, and the correctness and accuracy of the virtual assistant's responses can be improved based on the database's recorded data. Furthermore, through the local database, semantic search, and generative natural language model, the virtual assistant does not need to rely on cloud services to understand user commands and generate response content, thereby enhancing privacy protection, data security, and avoiding response delays.
[0055] Although this disclosure has been disclosed above with reference to embodiments, it is not intended to limit this disclosure. Anyone skilled in the art may make some modifications and refinements without departing from the spirit and scope of this disclosure. Therefore, the scope of protection of this disclosure shall be determined by the appended claims and their equivalents. [Simplified Explanation of the Diagram]
[0056] FIG1 is a block diagram of an electronic device according to an embodiment of the present invention. FIG2 is a flowchart of a virtual assistant operation method according to an embodiment of the present invention. FIG3 is a schematic diagram of a virtual assistant operation method according to an embodiment of the present invention. FIG4A and FIG4B are flowcharts of a virtual assistant operation method according to an embodiment of the present invention. FIG5 is a schematic diagram of a virtual assistant according to an embodiment of the present invention.
Claims
1. A virtual assistant operation method, applicable to an electronic device including an input device and an output device, the method comprising: A user command is received via the input device through a virtual assistant; A semantic search result is obtained by using a language embedding model to perform a semantic search on a database containing multiple functional description texts based on the user command. Based on the semantic search result, a target text or a target prompt instruction including the user command is generated, wherein when the semantic search result includes the target text matching the user command, the target prompt instruction including the target text is generated according to a preset prompt format. The target prompt instruction is then input into a generative natural language model, and the model response text of the generative natural language model is output through the output device.
2. The virtual assistant operation method as described in claim 1, wherein the step of obtaining the semantic search result by performing a semantic search in the database recording the plurality of function description texts based on the user command using the language embedding model includes: The semantic feature vector of the user command is obtained using the language embedding model. Determine whether the semantic feature vector of the user command is similar to the semantic feature vectors of the multiple function description texts; And when the semantic feature vector of the user command is similar to the semantic feature vector of the target text in the plurality of functional description texts, obtain the semantic search results including the target text that matches the user command.
3. The virtual assistant operation method as described in claim 2, wherein the step of obtaining the semantic search result by performing a semantic search in the database recording the plurality of function description texts based on the user command using the language embedding model further includes: When the semantic feature vector of the user command is not similar to the semantic feature vector of the multiple functional description texts, an extended query string is generated according to the user command. The semantic feature vector of the extended query string is obtained using the language embedding model; it is determined whether the semantic feature vector of the extended query string is similar to the semantic feature vectors of the multiple functional description texts; and when the semantic feature vector of the extended query string is similar to the semantic feature vector of the target text in the multiple functional description texts, the semantic search result including the target text matching the user command is obtained.
4. The virtual assistant operation method as described in claim 3, wherein the step of generating the at least one extended query string based on the user command includes: Based on a query expansion suggestion and the user command, the generative natural language model is used to generate the expanded query string.
5. The virtual assistant operation method described in claim 3, wherein the step of obtaining the semantic search result by performing a semantic search in the database recording the plurality of function description texts based on the user command using the language embedding model further includes: When the semantic feature vector of the extended query string is not similar to the semantic feature vectors of the multiple functional description texts, the semantic search results excluding the target text that matches the user command are obtained.
6. The virtual assistant operation method as described in claim 1, wherein the step of generating the target prompt instruction, which includes the target text or the user command from the plurality of function description texts, based on the semantic search result, comprises: If the semantic search results do not include the target text that matches the user command, a target prompt instruction that includes the user command is generated.
7. The virtual assistant operation method as described in claim 1, wherein the target text includes a device knowledge text, and the target prompt instruction is used to request the generative natural language model to respond to the user command based on the device knowledge text.
8. The virtual assistant operation method as described in claim 1, wherein the target text includes an application auxiliary text, and the target prompt instruction is used to request the generative natural language model to generate a program execution instruction based on the application auxiliary text and the user command.
9. The virtual assistant operation method as described in claim 8, wherein the steps of inputting the target prompt instruction into the generative natural language model and outputting the model response text of the generative natural language model through the output device include: The generative natural language model generates program execution instructions based on the target prompt instructions and provides them to a third-party application. The third-party application's program output data is provided to the generative natural language model so that the generative natural language model can output the model's response text based on the program's generated data.
10. The virtual assistant operation method as described in claim 1, wherein the steps of inputting the target prompt instruction into the generative natural language model and outputting the model response text of the generative natural language model through the output device include: Determine whether the model's response text is acceptable; And when the model response text is valid, the output device outputs the model response text of the generative natural language model.
11. An electronic device comprising: Output device; Input device; Storage device that records multiple instructions; The system includes a processing device connected to the output device, the input device, and the storage device, configured to execute the instructions to: receive a user command via the input device through a virtual assistant; obtain a semantic search result by performing a semantic search on a database containing multiple functional description texts based on the user command using a language embedding model; generate a target prompt instruction that includes a target text from the multiple functional description texts or the user command based on the semantic search result, wherein when the semantic search result includes the target text matching the user command, the target prompt instruction that includes the target text is generated according to a preset prompt format; and input the target prompt instruction into a generative natural language model and output the model response text of the generative natural language model through the output device.