Human-computer interaction method and apparatus, vehicle, terminal, medium, and program product

Abstract

The present disclosure relates to a human-computer interaction method and apparatus, a vehicle, a terminal, a medium, and a program product. The method comprises: acquiring question information input by a user; determining a task type corresponding to the question information, and generating task information corresponding to the task type, wherein the task information comprises a task execution object and task parameters, and the task execution object has a corresponding task execution mode; on the basis of the task parameters, executing the task information in the corresponding task execution mode of the task execution object, and obtaining a task execution result corresponding to the task information, wherein the task execution mode comprises one or more of an interface calling mode, a task instruction generation mode, and an answer generation mode; and performing reasoning on the task execution result to output reply information of the question information.

Description

Human-computer interaction method, device, vehicle, terminal, medium and program product

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This disclosure claims priority to Chinese patent application No. 2023117489851 filed on December 19, 2023, with applicant Beijing Rockwell Technology Co., Ltd., and application name “Human-computer interaction method, device, vehicle, terminal and medium”, the full text of which is incorporated by reference into this disclosure. Technical Field

[0003] The present disclosure relates to the field of natural language processing technology, and in particular to a human-computer interaction method, device, vehicle, terminal, medium, and program product. Background Art

[0004] Currently, in-car voice interaction systems can handle a variety of interactive scenarios, including task-based conversations and knowledge quizzes. Task-based conversations account for over 70% of in-car voice interaction scenarios. Task-based conversations involve users issuing control commands to the vehicle computer system, such as "open the window" or "play a song." Knowledge quizzes (including small talk) primarily rely on search queries, such as "What is the capital of China?" or "Who is the richest person in the world?"

[0005] In related technologies, in task-based conversations, the vehicle-mounted system receives control commands from the user and notifies the corresponding controller to execute them. Meanwhile, knowledge quizzes and casual chats provide users with answers through retrieval. Specifically, after receiving a user's question, the vehicle-mounted system sends it to a server. The server searches a database for the closest question and answer, and returns the answer to the vehicle-mounted system, which then provides the user with the answer. However, existing human-vehicle interaction is limited by the rigid interaction process within the in-vehicle voice interaction system, and cannot meet the demand for result reasoning in various interaction scenarios. Summary of the Invention

[0006] In order to solve the above technical problems, the present disclosure provides a human-computer interaction method, device, vehicle, terminal, medium and program product.

[0007] The present disclosure first provides a human-computer interaction method, comprising:

[0008] Get the question information entered by the user;

[0009] Determine a task type corresponding to the question information and generate task information corresponding to the task type; the task information includes a task execution object and task parameters; the task execution object has a corresponding task execution method; the task types include a retrieval-based question and answer type, a logic-based question and answer type, a command control type, a generative question and answer type, and a function request type;

[0010] Based on the task parameters, execute the task information according to the task execution method corresponding to the task execution object, and obtain a task execution result corresponding to the task information; the task execution method includes one or more of an interface call method, a task instruction generation method, and an answer generation method;

[0011] Reasoning is performed on the task execution result, and reply information of the question information is output.

[0012] The present disclosure also provides a human-computer interaction device, including:

[0013] An acquisition module is configured to acquire question information input by a user;

[0014] a planning module configured to determine a task type corresponding to the question information and generate task information corresponding to the task type; the task information includes a task execution object and task parameters; the task execution object has a corresponding task execution method; the task types include a search-based question-answering type, a logic-based question-answering type, a command-control type, a generative question-answering type, and a function request type;

[0015] a generation module configured to execute the task information according to a task execution method corresponding to the task execution object based on the task parameters, and obtain a task execution result corresponding to the task information; the task execution method includes one or more of an interface call method, a task instruction generation method, and an answer generation method;

[0016] The reasoning module is configured to reason about the task execution result and output reply information of the question information.

[0017] An embodiment of the present disclosure further provides a vehicle-mounted terminal, comprising: a memory and a processor; the memory stores a computer program, and the processor implements the human-computer interaction method as described above when executing the computer program.

[0018] The embodiment of the present disclosure further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the human-computer interaction method as described above is implemented.

[0019] An embodiment of the present disclosure further provides a vehicle, comprising: the human-computer interaction device as described above.

[0020] The embodiment of the present disclosure further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the human-computer interaction method as described above is implemented.

[0021] An embodiment of the present disclosure further provides a computer program, which includes computer-readable code. When the computer-readable code runs in an electronic device, the processor of the electronic device executes the computer program to implement the human-computer interaction method as described above.

[0022] An embodiment of the present disclosure also provides a computer program product, which includes a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device implements the human-computer interaction method as described above when executing it.

[0023] The technical solution provided by the embodiments of the present disclosure has the following advantages over the prior art:

[0024] Obtain the question information input by the user, determine the task type corresponding to the question information, and generate task information corresponding to the task type, wherein the task information includes a task execution object and task parameters, and the task execution object has a corresponding task execution method; task types include: retrieval-based question and answer type, logical question and answer type, instruction control type, generative question and answer type, and function request type; based on the task parameters, execute the task information according to the task execution method corresponding to the task execution object, and obtain the task execution result corresponding to the task information, the execution method of the task information includes: one or more of: interface call method, task instruction generation method, and answer generation method, infer the task execution result, and output reply information of the question information. By performing task planning on the problem information, the task type corresponding to the problem information is obtained, and the task execution object and task parameters corresponding to the task type are generated. Since different interaction scenarios correspond to different task execution objects and task parameters, and different task objects have corresponding task execution methods, for different interaction scenarios, based on the task parameters, the corresponding task information is executed according to the execution method corresponding to the task execution object, and the task execution result corresponding to the task information is obtained. Then, the task execution result is inferred to obtain the reply information of the problem information, thereby realizing result reasoning in multiple interaction scenarios, overcoming the defect that the result reasoning in multiple interaction scenarios cannot be realized due to the solidified interaction process of the in-vehicle voice interaction system.

[0025] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, rather than limiting the present disclosure. Other features and aspects of the present disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0027] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0028] FIG1 is a schematic flow chart of a human-computer interaction method provided by an embodiment of the present disclosure;

[0029] FIG2 is a schematic structural diagram of a human-computer interaction device provided by an embodiment of the present disclosure;

[0030] FIG3 is a schematic structural diagram of a vehicle-mounted terminal provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0031] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features therein can be combined with each other in the absence of conflict.

[0032] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0033] Relational terms such as “first” and “second” in the description and claims of this disclosure are merely used to distinguish one entity or operation from another entity or operation, but do not necessarily require or imply any actual relationship or order between these entities or operations.

[0034] In the embodiments of the present disclosure, words such as "exemplarily" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of the present disclosure should not be interpreted as being more preferred or advantageous than other embodiments or designs. Specifically, the use of words such as "exemplarily" or "for example" is intended to present related concepts in a concrete manner. In addition, in the description of the embodiments of the present disclosure, unless otherwise specified, the meaning of "a plurality of" refers to two or more.

[0035] Glossary:

[0036] Large Language Model (LLM): An AI model designed to understand and generate human language. They are trained on large amounts of text data and can perform a wide range of tasks, including text summarization, translation, sentiment analysis, and more.

[0037] Application Programming Interface (API): Defined as the interaction between multiple software intermediaries, as well as the types of calls or requests that can be made, how to make calls or requests, the data format that should be used, the conventions that should be followed, etc.

[0038] Natural Language Generation (NLG): is a part of natural language processing that generates natural language from machine representation systems such as knowledge bases or logical forms.

[0039] The embodiment of the present disclosure provides a human-computer interaction method; the method can be implemented by a preset large language model. Among them, the preset large language model has various capabilities including dialogue generation, language understanding, knowledge question and answer, logical reasoning, etc. The preset large language model can be used as a large model controller to call external tools, continuously broadening the capability coverage of the large model. At the same time, the preset large language model has its own memory network, and users can choose to let the car system remember personalized preferences and habits based on historical conversations, understand the user's recent status, and provide users with high-quality services. For example, in the embodiment of the present disclosure, the preset large language model can act as a controller. For different scenarios, the preset large language model can independently decide whether to directly generate answers, call external APIs (such as encyclopedias, information, and other special model interfaces), generate terminal-side instructions, etc. According to the external API results and the execution results of the terminal-side instructions, the preset large language model completes the final result reasoning and outputs the reply information of the question information, thereby realizing the unified scheduling of in-vehicle scenes based on the large model.

[0040] In some embodiments, as shown in FIG1 , a human-computer interaction method is provided, including the following steps S11-S14:

[0041] S11. Obtain question information input by the user.

[0042] The question information input by the user may be different types of conversations in different scenarios. In addition, the question information input by the user may include input information of the user's current conversation and / or input information of historical conversations.

[0043] Specifically, users can input question information through various input methods such as voice input and / or text; when users input through voice, the vehicle system can recognize the input voice information and obtain voice recognition text as the question information input by the user; when users input through text, the question information input by the user can be directly input into the vehicle system.

[0044] Exemplarily, the question information input by the user can be small talk, inquiry, or control instructions to the vehicle system, etc.; for example, when the question information input by the user is small talk, it can be "Please help me draw a picture of a child running on the grass", "Please paint the child's clothes red", etc., and when the question information input by the user is an inquiry, it can be "Where is the capital of China?", "How many people are there?", "Is there a concert this month?", and "Where exactly will the concert be held?", etc.; when the question information input by the user is to send a control instruction to the vehicle system, it can be "Please close the car window", "Turn on the air conditioner", and "Start cooling", etc.

[0045] S12. Determine the task type corresponding to the problem information, and generate task information corresponding to the task type.

[0046] Among them, task information includes task execution objects and task parameters; task execution objects have corresponding task execution methods; task types include retrieval-based question and answer types, logical question and answer types, instruction control types, generative question and answer types, and function request types.

[0047] In some embodiments, step S12 may be implemented as follows:

[0048] The question information is input into a preset large language model so that the large language model performs task planning on the question information, determines the task type corresponding to the question information, and generates task information corresponding to the task type.

[0049] Exemplarily, the preset large language model can extract keywords from the question information, determine the corresponding task type according to the keywords of the question information, and then generate corresponding task information.

[0050] By processing problem information with a preset large language model, the efficiency and accuracy of task planning can be improved, thereby improving the accuracy and comprehensiveness of task types and task information.

[0051] Exemplarily, inputting question information into a preset large language model so that the large language model performs task planning on the question information, determines the task type corresponding to the question information, and generates task information corresponding to the task type can be achieved by the following steps:

[0052] a. Input the question information into a preset large language model so that the preset large language model can identify the sentence type of the question information and determine the task type corresponding to the question information.

[0053] For example, search-based question-answering types may include but are not limited to: "What are the works of Li Bai?", "What are the representative works of Zhou xx?", etc.; search-based question-answering types may be targeted at question-answering scenarios such as encyclopedia questions and answers, entertainment information questions and answers, stock price increase questions and answers, and mathematical calculation inquiries.

[0054] For example, logical question-answering types may include, but are not limited to: "What is 25 times 18?", "What is 100 to the power of 5?", etc.

[0055] For example, the command control type may include but is not limited to: "Please close the window", "Turn on the air conditioner", etc. The command control type may be for command control scenarios such as issuing vehicle control commands and querying vehicle status commands.

[0056] For example, generative question-answering types may include, but are not limited to, "Please help me write a letter," "How many glove boxes does the Ideal L9 have?", "Who is the founder of Ideal Auto?", etc. Functional request types may include, but are not limited to, "Please help me draw a picture of the Ideal L9 running on the highway," "Please help me draw a picture of a child running on the grass," etc.

[0057] Optionally, the sentence type identification method may include but is not limited to performing keyword identification on the question information to obtain multiple keywords corresponding to the question information; and matching the corresponding task type for the question information based on the multiple keywords.

[0058] Specifically, the vehicle system receives the question information input by the user and sends the question information to the preset large language model in the server. In response to the user's request, the preset large language model extracts keywords from the question information and determines the corresponding task type based on the keywords in the question information.

[0059] For example, when the question information input by the user is "What are the works of Li x?", "What are the representative works of Zhou xx?", etc., the extracted keywords are "Li x", "works"; "Zhou xx", "representative works". Since the answers to such questions need to rely on external applications (for example, encyclopedia API, search API, information API, etc.), the task type matched to such question information is the retrieval question and answer type; and when the question information input by the user is "What is 25 times 18?", "What is the 5th power of 100?", etc., the extracted keywords are "25", "multiplied", "18"; "100", "5th power". Since the answers to such questions need to rely on internal applications (for example, calculator, etc.), the task type matched to such question information is the logic question and answer type; when the question information input by the user is "Please close the car windows", "Turn on the air conditioner", etc., the extracted keywords are "close", "car windows"; "turn on", "air conditioner". Such questions involve issuing control instructions to the vehicle system, so the task type matched to such question information is the command control type; when the question information input by the user is "How many glove boxes does Ideal L9 have?", "Who is the founder of Ideal Auto?", etc., the extracted keywords are "Ideal L9", "glove box"; "Ideal Auto", "founder". Since the answers to such questions are internal corporate information, the preset large language model can directly output the results, so the task type matched to such question information is the generative question answering type; when the question information input by the user is "Help me draw a picture of Ideal L9 running on the highway", "Please help me draw a picture of a child running on the grass", etc., the extracted keywords are "drawing", "Ideal L9", "high speed", "running", "picture"; "drawing", "child", "grass", "running", "picture". Since the answers to such questions need to rely on applications with professional functions, the task type matched to such question information is the function request type.

[0060] b. Based on the preset correspondence, determine the task execution object corresponding to the task type.

[0061] The preset corresponding relationship includes the corresponding relationship between each task type and each task execution object.

[0062] In some embodiments, the correspondence between task types and task execution objects includes: the task execution object corresponding to the retrieval-based question and answer type is an external application; the task execution object corresponding to the logic-based question and answer type is an internal application; the task execution object corresponding to the instruction control type is a vehicle controller; the task execution object corresponding to the generative question and answer type is a question and answer model; and the task execution object corresponding to the function request type is a function model program.

[0063] Among them, external applications may include but are not limited to encyclopedia, search, information, stock and other related applications; internal applications may include but are not limited to calculator applications, local voice playback applications, local video playback applications, etc.; vehicle controllers may include but are not limited to window controllers, air conditioning controllers, door controllers, etc.; the question and answer model can be a preset large language model, which has various capabilities including dialogue generation, language understanding, knowledge question and answer, logical reasoning, etc.; functional model programs can include professional functional model programs such as the Painting Master application.

[0064] Optionally, when the task type is one or more of a search-based question and answer, a logic-based question and answer, and a function request type, the task execution object corresponding to the task type is determined based on a preset correspondence relationship, which can be achieved in the following manner:

[0065] Based on the preset correspondence, the target calling program required by the task execution object is determined.

[0066] The target calling program is one or more of an external application program, an internal application program, or a functional model program.

[0067] c. Match the corresponding task parameters for the task execution object, and use the task execution object and task parameters as the task information corresponding to the task type.

[0068] Among them, task execution parameters may include content information to be generated, API parameters, function model parameters, etc. For example, when the task execution object is an external application or an internal application, the task execution parameters may include: relevant parameters of the external API and relevant parameters of the internal API; when the task execution object is a function model program, the task execution parameters may include function model parameters; when the task execution object is a vehicle controller, the task execution parameters may include specific execution units and execution actions; when the task execution object is a question-and-answer model, the task execution parameters may include specific content information to be generated.

[0069] For example, matching the corresponding task parameters for the task execution object can be achieved in the following ways:

[0070] Based on the preset data management table, the task parameters of the target calling program required by the task execution object are obtained.

[0071] The preset data management table includes a first data management table and a second data management table.

[0072] In some embodiments, the first data management table is used to manage the relevant parameters of various external applications included in the external program interface calling method and the relevant parameters of various internal applications included in the internal program interface calling method. The relevant parameters include: the names of various applications, interface parameters of various applications, and description information and status information of various applications.

[0073] Accordingly, based on the preset data management table, obtaining the task parameters of the target calling program required by the task execution object can be achieved in the following ways:

[0074] In the first data management table, the target calling program required by the matching task execution object is an external application program or an internal application program, and relevant parameters corresponding to the target calling program are obtained.

[0075] Specifically, in the first data management table, the target calling program required by the task execution object is matched to determine whether it is an external application or an internal application, and the relevant parameters corresponding to the target calling program are obtained. Further, a match is performed based on the name of the target calling program to determine whether the target calling program's status information indicates that it is callable and whether the target calling program has been sent to the vehicle system for execution.

[0076] For example, referring to Table 1, Table 1 is the first data management table. The first data management table is used to uniformly manage external and internal application programming interfaces (APIs). It primarily includes registration, address management, names, and input parameters for various API services. For example, the API description for "QASearch" indicates an encyclopedia knowledge engine. This application can call this API without being executed on the vehicle system.

[0077] Table 1

[0078] For example, when the question information input by the user is "What are Zhou xx's representative works?", the task type is determined to be a search-based question-answering type, and based on a preset corresponding relationship, the task execution object is determined to be an external application. The relevant parameters of the external application interface are obtained through the relevant parameters of multiple applications stored in the first data management table, such as the search API, and the API name, input parameters, type description information, etc. corresponding to the search API. Based on the API name, input parameters, type description information, etc. stored in the first data management table, the search API is called to obtain the interface call result of the search API, such as "Seven-year-old Fragrance, Blue and White Porcelain, Nunchaku, Simple Love, Rice Fragrance, East Wind Breaks, Tornado, All the Way North, Sunny Day, Secret That Cannot Be Told, etc."; when the text information input by the user is "What is 25 times 18?", the task type is determined to be a logical question-and-answer type, and based on the preset corresponding relationship, the task execution object is determined to be an internal application; the relevant parameters of the internal application interface are obtained through the relevant parameters of multiple applications stored in the first data management table, such as the calculation API, and the API name, input parameters, type description information, etc. corresponding to the calculation API. Based on the API name, input parameters, type description information, etc. stored in the first data management table, the calculation API is called to obtain the interface call result of the calculation API, such as "450".

[0079] In some embodiments, the second data management table is used to manage relevant parameters of various functional model programs included in the model program interface calling method. The relevant parameters of the functional model programs include: the names of various functional model programs, interface parameters of various functional model programs, and description information and status information of various functional model programs.

[0080] Accordingly, based on the preset data management table, obtaining the task parameters of the target calling program required by the task execution object can also be achieved in the following ways:

[0081] In the second data management table, the target function model program required by the task execution object is matched, and relevant parameters corresponding to the target function model program are obtained.

[0082] For example, referring to Table 2, Table 2 is a second data management table. The second data management table is used to uniformly manage external professional artificial intelligence model interfaces, mainly including description information, status information, application interface call information, etc. of available artificial intelligence models.

[0083] Table 2

[0084] For example, when the question information input by the user is "Please help me draw a picture of a child running on the grass", the task execution method generated by the preset large language model is the model interface calling method. Since the preset large language model itself does not have a drawing function, it is necessary to call an external model interface to complete the drawing. At this time, the extracted keywords may include "drawing", "child", "grass", "running", and "picture", and further determine the target function model interface corresponding to the execution method, such as the image generation AI model interface, the function of which is to generate images based on text, and the interface parameters corresponding to the AI ​​model interface; based on the interface parameters stored in the second data management table, the AI ​​model interface is called to obtain the corresponding target call result, that is, the drawn picture of a child running on the grass.

[0085] In addition, it should be noted that the first data management table and the second data management table need to be synchronized to the preset large language model in real time.

[0086] In some embodiments, when the first data management table and / or the second data management table are updated, relevant parameters in the first data management table and / or the second data management table are updated.

[0087] Furthermore, after updating the relevant parameters in the first data management table and / or the second data management table, matching the corresponding task parameters for the task execution object can be achieved in the following manner:

[0088] Based on the updated first data management table and / or the second data management table, task parameters of the target calling program required by the task execution object are obtained.

[0089] Specifically, when certain information in the external application, internal application, or functional model program is added, deleted, or modified in the first data management table and / or the second data management table, the relevant parameters of the first data management table and / or the second data management table are added, deleted, or modified accordingly, and based on the updated first data management table and / or the second data management table, the task parameters of the target calling program required for the task execution object are obtained.

[0090] In some embodiments, after matching the task execution object with the corresponding task parameters, the following steps may also be performed:

[0091] Based on the status information in the preset data management table, determine whether the interface of the target calling program is in a callable state; if the interface of the target calling program is in a callable state, output the interface parameters and description information of the target calling program; if the interface of the target calling program is in a non-callable state, output a prompt message.

[0092] The preset data management table includes a first data management table and a second data management table; the prompt information is used to prompt the user that the interface of the target calling program is unavailable.

[0093] Specifically, after matching the corresponding task parameters for the task execution object, based on the status information in the preset data management table, it is determined whether the interface of the target calling program is in a callable state. If the interface of the target calling program is in a callable state, the interface parameters and description information of the target calling program are output, so that the preset large model can make an API call based on the interface parameters and description information of the target calling program in accordance with the task execution method corresponding to the task execution object in the next step, thereby obtaining the task execution result corresponding to the task information. If the interface of the target calling program is not callable, a prompt message is output to inform the user that the interface of the target calling program is unavailable, so that the user can know that the corresponding answer cannot be obtained for the question information currently entered.

[0094] S13. Based on the task parameters, execute the task information in accordance with the task execution method corresponding to the task execution object, and obtain the task execution result corresponding to the task information.

[0095] Among them, the task execution method includes: one or more of: interface calling method, task instruction generation method, and answer generation method; the types of interface calling methods include: external program interface calling method, internal program interface calling method, and model program interface calling method.

[0096] For example, an external API might be an encyclopedia API, a search API, an information API, a stock API, etc. An internal API might be a calculator API, a local voice playback API, a local video playback API, etc. A model interface might be a drawing master API, for example, calling the drawing master API can generate an image based on text.

[0097] For example, based on the task parameters, executing the task information according to the task execution method corresponding to the task execution object, and obtaining the task execution result corresponding to the task information can be achieved in the following ways:

[0098] By presetting a large language model, based on task parameters, the task information is executed in accordance with the task execution method corresponding to the task execution object, and the task execution result corresponding to the task information is obtained.

[0099] Through the above operations, we can fully utilize the efficiency and high precision of the large language model in the data processing dimension, thereby improving the efficiency of obtaining task execution results and improving the accuracy of task execution results.

[0100] S14. Reasoning about the task execution results and outputting response information to the question information.

[0101] Among them, the task execution result includes: the target answer corresponding to the interface call method, the target instruction execution result corresponding to the task instruction generation method, and the target answer corresponding to the answer generation method.

[0102] Specifically, after obtaining the task execution result corresponding to the task execution mode, the task execution result is inferred and response information of the question information is output.

[0103] For example, when the question information input by the user is "What are Zhou xx's representative works?", the task results obtained are "Seven-year-old Fragrance, Blue and White Porcelain, Nunchakus, Simple Love, Fragrance of Rice, East Wind Breaks, Tornado, All the Way North, Sunny Day, Secret that Cannot be Told, etc.", and the preset large language model infers the task results and outputs the reply information: "Zhou xx's representative works include Seven-year-old Fragrance, Blue and White Porcelain, Nunchakus, Simple Love, Fragrance of Rice, East Wind Breaks, Tornado, All the Way North, Sunny Day, Secret that Cannot be Told, etc.".

[0104] For example, when the user inputs the question "Please close the car window", the task result obtained is "Execution successful". After the preset large language model infers the task result, the response message output is: "The car window is opened for you."

[0105] For example, when the question information entered by the user is "Please help me draw a picture of a child running on the grass", the task result obtained is "Drawn for you, please refer to it", and the generated reply information is: "The picture of a child running on the grass has been drawn for you, please refer to the display interface."

[0106] For example, reasoning about the task execution results and outputting the response information of the question information can be achieved in the following ways:

[0107] The task execution results are inferred by a preset large language model, and the response information of the question information is output.

[0108] Through the above processing, we can fully leverage the advantages of the preset large language model in the data reasoning dimension to improve the matching and accuracy of reply information.

[0109] The human-computer interaction method provided by the present disclosure obtains question information input by a user, determines the task type corresponding to the question information, and generates task information corresponding to the task type, wherein the task information includes a task execution object and task parameters, and the task execution object has a corresponding task execution method; task types include: retrieval-based question and answer type, logical question and answer type, instruction control type, generative question and answer type, and function request type; based on the task parameters, the task information is executed according to the task execution method corresponding to the task execution object, and the task execution result corresponding to the task information is obtained, and the execution method of the task information includes: one or more of an interface call method, a task instruction generation method, and an answer generation method, the task execution result is inferred, and reply information of the question information is output. Since different interaction scenarios correspond to different task execution objects and task parameters, and different task objects have corresponding task execution methods, for different interaction scenarios, based on the task parameters, the corresponding task information is executed in accordance with the execution method corresponding to the task execution object, and the task execution result corresponding to the task information is obtained. Then, the task execution result is inferred to obtain the reply information of the question information, thereby realizing result reasoning in multiple interaction scenarios, overcoming the defect that the in-vehicle voice interaction system in related technologies is limited by its solidified data processing process and cannot realize result reasoning in multiple interaction scenarios.

[0110] In some embodiments, when the task execution mode is a task instruction generation mode, executing the task information according to the task execution mode corresponding to the task execution object based on the task parameters and obtaining the task execution result corresponding to the task information can be achieved as follows:

[0111] Generate a task execution instruction based on the task parameters; send the task execution instruction to the corresponding vehicle controller so that the vehicle controller executes the task execution instruction and obtains the execution result corresponding to the task execution instruction.

[0112] The execution results include: successful execution or failed execution.

[0113] Specifically, when the task execution mode is task instruction generation, a task execution instruction is generated based on the task parameters. The task execution instruction is sent to the corresponding vehicle controller of the vehicle system. After the vehicle controller executes the target instruction, it returns the corresponding target instruction execution result. The execution result may be execution success or execution failure.

[0114] For example, when the question information input by the user is "Please close the car window", the corresponding task parameters include "close" and "car window", then the task execution instruction is generated in combination with the task parameters and the long short-term memory network, that is, the "close the car window" instruction, and the "close the car window" instruction is sent to the vehicle system. The control unit of the vehicle system executes the "close the car window" instruction and returns the execution result to the preset large language model; when the control unit of the vehicle system successfully executes the "close the car window" instruction, it returns the result of successful execution; when the control unit of the vehicle system fails to execute the "close the car window" instruction, it returns the result of failed execution.

[0115] By providing feedback on execution results, users can clearly know whether the commands they issued have been executed successfully, thereby improving the user's human-computer interaction experience in the car.

[0116] In some embodiments, as shown in FIG. 2 , a human-computer interaction device 200 is provided, including:

[0117] The acquisition module 210 is configured to acquire question information input by the user;

[0118] Planning module 220 is configured to determine the task type corresponding to the question information and generate task information corresponding to the task type; the task information includes a task execution object and task parameters; the task execution object has a corresponding task execution method; the task types include search-based question and answer type, logical question and answer type, command control type, generative question and answer type, and function request type;

[0119] The generation module 230 is configured to execute the task information according to the task execution method corresponding to the task execution object based on the task parameters, and obtain the task execution result corresponding to the task information; the task execution method includes one or more of an interface call method, a task instruction generation method, and an answer generation method;

[0120] The reasoning module 240 is configured to reason about the task execution result and output reply information of the question information.

[0121] In some embodiments, the planning module is configured to input the question information into a preset large language model so that the preset large language model performs task planning on the question information, determines the task type corresponding to the question information, and generates task information corresponding to the task type.

[0122] As an optional implementation of the embodiment of the present disclosure, the planning module includes:

[0123] The recognition unit is configured to input the question information into a preset large language model, so that the preset large language model recognizes the sentence type of the question information and determines the task type corresponding to the question information;

[0124] A determining unit is configured to determine a task execution object corresponding to a task type based on a preset corresponding relationship; the preset corresponding relationship includes a corresponding relationship between each task type and each task execution object;

[0125] The matching unit is configured to match the corresponding task parameters for the task execution object, and use the task execution object and the task parameters as task information corresponding to the task type.

[0126] As an optional implementation of the embodiment of the present disclosure, the correspondence between task types and task execution objects includes: the task execution object corresponding to the retrieval question and answer type is an external application; the task execution object corresponding to the logic question and answer type is an internal application; the task execution object corresponding to the instruction control type is a vehicle controller; the task execution object corresponding to the generative question and answer type is a question and answer model; and the task execution object corresponding to the function request type is a function model program.

[0127] As an optional implementation of the embodiment of the present disclosure, when the task type is one or more of a search-based question-answering type, a logic-based question-answering type, and a function request type, the determination unit included in the planning module is configured to determine, based on a preset correspondence, a target calling program required for the task execution object; the target calling program is one or more of an external application, an internal application, or a function model program;

[0128] The matching unit included in the planning module is configured to obtain the task parameters of the target calling program required by the task execution object based on a preset data management table; the preset data management table includes a first data management table and a second data management table.

[0129] As an optional implementation of the embodiment of the present disclosure, the first data management table is used to manage the relevant parameters of multiple external applications included in the external program interface calling method and the relevant parameters of multiple internal applications included in the internal program interface calling method, the relevant parameters include: the names of various applications, the interface parameters of various applications, and the description information and status information of various applications; the types of interface calling methods include: external program interface calling method, internal program interface calling method, and model program interface calling method, the matching unit is also configured to match the target calling program required for the task execution object as an external application or an internal application in the first data management table, and obtain the relevant parameters corresponding to the target calling program.

[0130] As an optional implementation of the embodiment of the present disclosure, the second data management table is used to manage relevant parameters of multiple functional model programs included in the model program interface calling method, and the relevant parameters of the functional model programs include: the names of various functional model programs, interface parameters of various functional model programs, and description information and status information of various functional model programs; the matching unit is configured to match the target functional model program required by the task execution object in the second data management table, and obtain relevant parameters corresponding to the target functional model program.

[0131] In some embodiments, after matching the corresponding task parameters for the task execution object, the identification unit is configured to determine whether the interface of the target calling program is in a callable state based on the status information in the preset data management table; if the interface of the target calling program is in a callable state, the interface parameters and description information of the target calling program are output; if the interface of the target calling program is in an uncallable state, a prompt message is output, and the prompt message is used to prompt the user that the interface of the target calling program is unavailable; the preset data management table includes a first data management table and a second data management table.

[0132] As an optional implementation of the embodiment of the present disclosure, when the task execution mode is a task instruction generation mode, the generation module is configured to generate a task execution instruction based on the task parameters; send the task execution instruction to the corresponding vehicle controller so that the vehicle controller executes the task execution instruction and obtains the execution result corresponding to the task execution instruction; the execution result includes: execution success, execution failure.

[0133] The human-computer interaction device disclosed herein obtains question information input by a user, determines the task type corresponding to the question information, and generates task information corresponding to the task type. The task information includes a task execution object and task parameters, and the task execution object has a corresponding task execution method. Task types include: retrieval-based question-answering type, logical question-answering type, command control type, generative question-answering type, and function request type. Based on the task parameters, the task information is executed according to the task execution method corresponding to the task execution object, and a task execution result corresponding to the task information is obtained. The task execution method includes: an interface call method, a task instruction generation method, and an answer generation method. The task execution result is inferred and a response to the question information is output. Since different interaction scenarios correspond to different task execution objects and task parameters, and different task objects have corresponding task execution methods, for different interaction scenarios, the task information is executed according to the execution method corresponding to the task execution object based on the task parameters, and a task execution result corresponding to the task information is obtained. The task execution result is then inferred to obtain a response to the question information. This enables result inference in various interaction scenarios, overcoming the drawback of existing in-vehicle voice interaction systems that are limited by fixed data processing capabilities and cannot achieve result inference in various interaction scenarios.

[0134] For the specific definition of the human-computer interaction device, please refer to the definition of the human-computer interaction method above, and will not be repeated here. Each module in the above-mentioned human-computer interaction device can be implemented in whole or in part by software, hardware, or a combination thereof. The above-mentioned modules can be embedded in or independent of the processor of the electronic device in the form of hardware, or can be stored in the processor of the electronic device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0135] The present disclosure also provides an in-vehicle terminal. Figure 3 is a schematic diagram of the structure of the in-vehicle terminal provided by the present disclosure. As shown in Figure 3, the in-vehicle terminal provided by the present disclosure includes a memory 31 and a processor 32. The memory 31 is used to store computer programs; the processor 32 is used to execute the human-computer interaction method provided by the above-mentioned method embodiment when the computer program is invoked. The in-vehicle terminal includes a processor, memory, a communication interface, a display screen, and an input device, all connected via a system bus. The processor of the in-vehicle terminal provides computing and control capabilities. The memory of the in-vehicle terminal includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operating system and computer programs stored in the non-volatile storage medium. When the computer program is executed by the processor, a human-computer interaction method is implemented. The display screen of the in-vehicle terminal can be a liquid crystal display or an electronic ink display. The input device of the in-vehicle terminal can be a touch screen covering the display screen, buttons, a trackball, or a touchpad provided on the computer device housing, or an external keyboard, touchpad, or mouse.

[0136] Those skilled in the art will understand that the structure shown in FIG3 is merely a block diagram of a portion of the structure related to the disclosed solution, and does not constitute a limitation on the computer device to which the disclosed solution is applied. The specific vehicle-mounted terminal may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0137] In some embodiments, the human-computer interaction device provided by the present disclosure can be implemented in the form of a computer, and a computer program can be run on the electronic device shown in Figure 3. The memory of the electronic device can store the various program modules that make up the human-computer interaction device of the electronic device. The computer program composed of each program module causes the processor to execute the steps of the human-computer interaction method of the electronic device in various embodiments of the present disclosure described in this specification.

[0138] An embodiment of the present disclosure further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the human-computer interaction method provided by the above method embodiment is implemented.

[0139] The embodiments of the present disclosure also provide a vehicle, which may include the human-computer interaction device provided in any of the previous embodiments.

[0140] An embodiment of the present disclosure further provides a computer program, which includes computer-readable code. When the computer-readable code runs in an electronic device, the processor of the electronic device executes the computer program to implement the human-computer interaction method as described in any of the above items.

[0141] An embodiment of the present disclosure also provides a computer program product, which includes a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device implements the human-computer interaction method as described in any of the preceding items when executing the computer-readable code.

[0142] Those skilled in the art will appreciate that embodiments of the present disclosure may be provided as methods, systems, or computer program products. Thus, the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.

[0143] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0144] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0145] Computer-readable media includes both permanent and non-permanent, removable and non-removable storage media. Storage media can implement any method or technology for storing information, which can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change RAM (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media, such as modulated data signals and carrier waves.

[0146] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0147] The foregoing description is intended only to provide specific embodiments of the present disclosure, intended to enable those skilled in the art to understand and implement the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the embodiments described herein, but rather to be construed in the broadest manner consistent with the principles and novel features disclosed herein. Industrial Applicability

[0148] The present disclosure relates to a human-computer interaction method, device, vehicle, terminal, medium and program product; the method includes: obtaining question information input by a user; determining the task type corresponding to the question information, and generating task information corresponding to the task type; the task information includes a task execution object and task parameters; the task execution object has a corresponding task execution method; based on the task parameters, the task information is executed according to the task execution method corresponding to the task execution object, and a task execution result corresponding to the task information is obtained; the task execution method includes: one or more of an interface calling method, a task instruction generating method, and an answer generating method; reasoning about the task execution result, and outputting reply information to the question information.

Claims

1. A human-computer interaction method, the method comprising: Get the question information entered by the user; Determine the task type corresponding to the problem information, and generate task information corresponding to the task type; The task information includes task execution objects and task parameters; The task execution object has a corresponding task execution mode; the task types include retrieval question and answer type, logic question and answer type, instruction control type, generation question and answer type, and function request type; Based on the task parameters, the task information is executed according to the task execution method corresponding to the task execution object, and a task execution result corresponding to the task information is obtained; the task execution method includes one or more of an interface calling method, a task instruction generating method, and an answer generating method; Reasoning is performed on the task execution result, and reply information of the question information is output.

2. The method according to claim 1, wherein: The determining the task type corresponding to the problem information and generating task information corresponding to the task type includes: The question information is input into a preset large language model, so that the preset large language model performs task planning on the question information, determines the task type corresponding to the question information, and generates task information corresponding to the task type.

3. The method according to claim 2, wherein: The step of inputting the question information into a preset large language model so that the preset large language model performs task planning on the question information, determines the task type corresponding to the question information, and generates task information corresponding to the task type, includes: Inputting the question information into the preset large language model so that the preset large language model performs sentence type recognition on the question information and determines the task type corresponding to the question information; Based on a preset corresponding relationship, determining a task execution object corresponding to the task type; the preset corresponding relationship includes a corresponding relationship between each task type and each task execution object; The task execution object is matched with a corresponding task parameter, and the task execution object and the task parameter are used as task information corresponding to the task type.

4. The method according to any one of claims 1 to 3, wherein: The correspondence between the task type and the task execution object includes: the task execution object corresponding to the retrieval-based question and answer type is an external application, the task execution object corresponding to the logic-based question and answer type is an internal application, the task execution object corresponding to the instruction control type is a vehicle controller, the task execution object corresponding to the generative question and answer type is a question and answer model, and the task execution object corresponding to the function request type is a function model program.

5. The method according to claim 4, wherein: When the task type is one or more of the search-based question-and-answer type, the logic-based question-and-answer type, and the function request type, determining the task execution object corresponding to the task type based on the preset corresponding relationship includes: Based on the preset corresponding relationship, determining the target calling program required by the task execution object; the target calling program is one or more of an external application program, an internal application program, and a functional model program; The matching of the task execution object with the corresponding task parameters includes: Based on a preset data management table, task parameters of a target calling program required by the task execution object are obtained; the preset data management table includes a first data management table and a second data management table.

6. The method according to claim 5, wherein: The types of the interface calling modes include external program interface calling modes, internal program interface calling modes, and model program interface calling modes; the first data management table is used to manage the relevant parameters of the various external application programs included in the external program interface calling modes and the relevant parameters of the various internal application programs included in the internal program interface calling modes; the relevant parameters include the names of various application programs, the interface parameters of various application programs, and the description information and status information of various application programs; The step of obtaining the task parameters of the target calling program required by the task execution object based on the preset data management table includes: In the first data management table, the target calling program required for matching the task execution object is an external application or an internal application, and relevant parameters corresponding to the target calling program are obtained.

7. The method according to claim 5 or 6, wherein: The second data management table is used to manage the relevant parameters of the various functional model programs included in the model program interface calling method, and the relevant parameters of the functional model programs include: the names of various functional model programs, the interface parameters of various functional model programs, and the description information and status information of various functional model programs; the task parameters of the target calling program required by the task execution object are obtained based on the preset data management table, and also include: In the second data management table, the target function model program required by the task execution object is matched, and relevant parameters corresponding to the target function model program are obtained.

8. The method according to any one of claims 5 to 7, wherein: After matching the corresponding task parameters for the task execution object, the method further includes: Based on the status information in the preset data management table, determining whether the interface of the target calling program is in a callable state; wherein the preset data management table includes a first data management table and a second data management table; If the interface of the target calling program is in a callable state, outputting the interface parameters and description information of the target calling program; If the interface of the target calling program is in an unavailable state, a prompt message is output, where the prompt message is used to prompt the user that the interface of the target calling program is unavailable.

9. The method according to any one of claims 1 to 4, wherein: When the task type is an instruction control type, executing the task information according to the task execution mode corresponding to the task execution object based on the task parameters, and obtaining the task execution result corresponding to the task information includes: generating a task execution instruction based on the task parameters; The task execution instruction is sent to the corresponding vehicle controller so that the vehicle controller executes the task execution instruction and obtains the execution result corresponding to the task execution instruction; the execution result includes: execution success or execution failure.

10. The method according to any one of claims 1 to 9, wherein: The executing the task information according to the task execution mode corresponding to the task execution object based on the task parameter, and obtaining the task execution result corresponding to the task information, includes: By presetting a large language model, based on the task parameters, executing the task information in a task execution mode corresponding to the task execution object, and obtaining a task execution result corresponding to the task information; The reasoning on the task execution result and outputting the reply information of the question information includes: The task execution result is inferred by the preset large language model, and reply information of the question information is output.

11. A human-computer interaction device, comprising: An acquisition module is configured to acquire question information input by a user; A planning module, configured to determine a task type corresponding to the problem information and generate task information corresponding to the task type; The task information includes task execution objects and task parameters; The task execution object has a corresponding task execution mode; the task types include retrieval question and answer type, logic question and answer type, instruction control type, generation question and answer type, and function request type; A generating module is configured to execute the task information according to the task execution mode corresponding to the task execution object based on the task parameters, and obtain a task execution result corresponding to the task information; the task execution mode includes one or more of an interface calling mode, a task instruction generating mode, and an answer generating mode; The reasoning module is configured to reason about the task execution result and output reply information of the question information.

12. The human-computer interaction device according to claim 11, wherein: The planning module is configured to input the question information into a preset large language model so that the preset large language model performs task planning on the question information, determines the task type corresponding to the question information, and generates task information corresponding to the task type.

13. The human-computer interaction device according to claim 12, wherein: The planning module comprises: an identification unit configured to input the question information into the preset large language model so that the preset large language model performs sentence type identification on the question information and determines a task type corresponding to the question information; A determination unit is configured to determine the task execution object corresponding to the task type based on a preset corresponding relationship; the preset corresponding relationship includes a corresponding relationship between each task type and each task execution object; The matching unit is configured to match the corresponding task parameters for the task execution object, and use the task execution object and the task parameters as the task information corresponding to the task type.

14. The human-computer interaction device according to any one of claims 11 to 13, wherein: The correspondence between the task type and the task execution object includes: the task execution object corresponding to the retrieval-based question and answer type is an external application, the task execution object corresponding to the logic-based question and answer type is an internal application, the task execution object corresponding to the instruction control type is a vehicle controller, the task execution object corresponding to the generative question and answer type is a question and answer model, and the task execution object corresponding to the function request type is a function model program.

15. The human-computer interaction device according to claim 14, wherein: When the task type is one or more of a search-based question-answering type, a logic-based question-answering type, and a function request type, The planning module includes a determining unit configured to determine a target calling program required by the task execution object based on the preset corresponding relationship; the target calling program is one or more of an external application program, an internal application program, and a functional model program; The matching unit included in the planning module is configured to obtain the task parameters of the target calling program required by the task execution object based on a preset data management table; the preset data management table includes a first data management table and a second data management table.

16. The human-computer interaction device according to claim 15, wherein: The types of interface calling methods include external program interface calling methods, internal program interface calling methods, and model program interface calling methods; the first data management table is used to manage the relevant parameters of the various external applications included in the external program interface calling method and the relevant parameters of the various internal applications included in the internal program interface calling method, and the relevant parameters include the names of various applications, interface parameters of various applications, and description information and status information of various applications; the matching unit is configured to match the target calling program required by the task execution object as an external application or an internal application in the first data management table, and obtain the relevant parameters corresponding to the target calling program.

17. The human-computer interaction device according to claim 15 or 16, wherein: The second data management table is used to manage relevant parameters of multiple functional model programs included in the model program interface calling method, and the relevant parameters of the functional model programs include: the names of various functional model programs, interface parameters of various functional model programs, and description information and status information of various functional model programs; the matching unit is also configured to match the target functional model program required by the task execution object in the second data management table, and obtain the relevant parameters corresponding to the target functional model program.

18. The human-computer interaction device according to any one of claims 15 to 17, wherein: After matching the corresponding task parameters for the task execution object, the identification unit is configured to determine whether the interface of the target calling program is in a callable state based on the status information in the preset data management table; if the interface of the target calling program is in a callable state, the interface parameters and description information of the target calling program are output; if the interface of the target calling program is in an uncallable state, a prompt message is output, and the prompt message is used to prompt the user that the interface of the target calling program is unavailable; the preset data management table includes a first data management table and a second data management table.

19. The human-computer interaction device according to any one of claims 11 to 14, wherein: When the task execution method is the task instruction generation method, the generation module is configured to generate a task execution instruction based on the task parameters; send the task execution instruction to the corresponding vehicle controller so that the vehicle controller executes the task execution instruction and obtains the execution result corresponding to the task execution instruction; the execution result includes: execution success or execution failure.

20. The human-computer interaction device according to any one of claims 11 to 19, wherein: The generation module is configured to execute the task information in a task execution mode corresponding to the task execution object through a preset large language model based on the task parameters, and obtain a task execution result corresponding to the task information; infer the task execution result through the preset large language model, and output reply information of the question information.

21. A vehicle, comprising the human-computer interaction device according to any one of claims 11 to 20.

22. An in-vehicle terminal, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the human-computer interaction method according to any one of claims 1 to 10 when executing the computer program.

23. A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the human-computer interaction method according to any one of claims 1 to 10 is implemented.

24. A computer program, comprising a computer-readable code, wherein when the computer-readable code is run in an electronic device, the processor of the electronic device executes the computer-computer interaction method according to any one of claims 1 to 10.

25. A computer program product, comprising a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code, wherein when the computer-readable code runs in a processor of an electronic device, the processor in the electronic device implements the human-computer interaction method as described in any one of claims 1 to 10 when executing.

Images