Human-computer interaction method and electronic device

Abstract

A human-computer interaction method and an electronic device, relating to the field of AI. The method is applied to an electronic device, and comprises: receiving a user instruction, and in a scenario in which the user instruction represents a certain scenario intent, acquiring multiple target APIs according to the scenario intent, the multiple target APIs corresponding to the scenario intent. Then, a result of calling the multiple target APIs is outputted, and the result of calling the multiple APIs can be used to implement a user requirement corresponding to the scenario intent. Thus, an electronic device, on the basis of a scenario intent of a user, can directly output a result of calling an API which meets user requirements under the scenario intent, and the API is directly exposed by an application function. A result of calling multiple APIs is directly outputted, so that a richer, more comprehensive, and more convenient response result is achieved. In addition, user requirements can be met, and a human-computer conversation effect is improved.

Description

Human-computer interaction method and electronic equipment

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on July 19, 2023, with application number 202310892843.6 and application name “Human-computer interaction method and electronic device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of artificial intelligence (AI), and in particular to human-computer interaction methods and electronic devices. Background Art

[0003] At present, with the rapid development of AI technology, in human-computer interaction scenarios, electronic devices can identify user intentions based on user input instructions (such as questions or questions), and respond to user instructions based on AI technology and the identified user intentions. For example, when an electronic device receives a question input by a user, the electronic device will directly display corresponding text, cards, etc. based on the semantics of the user intention identified based on the question as an answer to the user's question. This response method is too simple and cannot meet user needs, resulting in poor human-computer dialogue effects.

[0004] Summary of the Invention

[0005] The present application provides a human-computer interaction method and electronic device that can provide richer, more comprehensive and convenient response results based on the user's scenario intentions, meet user needs, and improve the effect of human-computer dialogue.

[0006] To achieve the above objectives, this application adopts the following technical solutions:

[0007] In a first aspect, a human-computer interaction method is provided, which is applied to an electronic device, and the method includes: receiving a user instruction; when the user instruction represents a first scenario intention, obtaining multiple target application programming interfaces API according to the first scenario intention, and the multiple target APIs correspond to the aforementioned first scenario intention; and finally outputting the results of calling the multiple target APIs, wherein the results of calling the multiple target APIs are used to realize user needs corresponding to the first scenario intention.

[0008] Based on the above technical solution, when a user instruction representing a scenario intention is received, multiple corresponding APIs are obtained according to the user's scenario intention, and the results of calling these multiple APIs are output. The results of calling these multiple APIs can be used to realize the user needs corresponding to the user's scenario intention. In this way, in response to the user's scenario intention, the results of calling the API are directly output to the user, and the API is a direct exposure of the application function. The user can start a certain function of the application by directly operating the result of the API call, making the response information more convenient. In addition, outputting the results of calling multiple APIs makes the response information richer and more comprehensive, which can meet user needs and improve the effect of human-computer dialogue.

[0009] In one possible design, multiple target APIs are obtained according to the first scenario intention, including: obtaining multiple target APIs according to the user portrait and / or the environmental scene information of the electronic device, as well as the first scenario intention, wherein the environmental scene information of the electronic device can be used to characterize at least one of the environmental conditions around the electronic device and the state of the electronic device. Based on this design, multiple APIs under the scenario intention are obtained by combining various information such as the scenario intention, the user portrait, and the environmental scene information of the electronic device, so that the obtained APIs correspond to the user portrait, the environmental scene information of the electronic device, etc. The user portrait, the environmental scene information, etc. can reflect the user's preferences, habits, etc., and thus the obtained APIs can be more in line with the user's preferences and habits. The response result, that is, the output result of calling these multiple APIs, can be more personalized and targeted while meeting the user's needs under the scenario intention, so as to meet the user's actual preferences.

[0010] In one possible design, multiple target APIs are obtained based on the user portrait and / or the environmental scene information of the electronic device, and the first scene intent, including: sending the user portrait and / or the environmental scene information of the electronic device, and the first scene intent to the server; receiving multiple target APIs from the server. Based on this design, the server determines the multiple APIs under the scene intent based on information such as the user portrait, the environmental scene information of the electronic device, and the scene intent, and the computing power of the server is relatively high. In this way, a more accurate API corresponding to the scene intent can be obtained, and the response result is also more accurate.

[0011] One possible design for outputting the results of calling multiple target APIs includes: outputting the results of calling multiple target APIs in a multimodal display format, where the multimodal display format uses a variety of API display methods, including text, images, audio, and video. Based on this design, using a multimodal display format to output the results of calling multiple APIs can make the response more rich and intuitive.

[0012] In one possible design, before outputting the results of calling multiple target APIs in a multimodal display manner, the method further includes: receiving the multimodal display manner of the results of calling multiple target APIs from a server. Based on this design, the electronic device can directly receive the multimodal display manner of the results of calling the APIs from the server and then display the results of the called APIs in accordance with the multimodal display manner. This eliminates the need to perform additional operations to determine the display manner of the results of calling the APIs, thereby saving power consumption of the electronic device.

[0013] In one possible design, the multimodal display method corresponds to the environmental scene information of the electronic device. The environmental scene information of the electronic device characterizes the environmental conditions of the electronic device, the status of the electronic device, etc. Under different environmental conditions, the user may want to display the results of calling the API in different ways. For example: if the electronic device is in a noisy environment, the user may not be able to hear or focus on the video or audio content, so the user may be more inclined to read text or view pictures. If the electronic device is in a quiet environment, the user may be more inclined to watch videos or listen to audio content. In this way, the multimodal display method corresponds to the environmental scene information of the electronic device. The display method of the results of calling the API can be adapted to the current environment, the status of the electronic device, etc. It can meet the preferences of users in different environments under the same scene intention, achieve a "thousand-face" display effect, and better meet the actual needs of users.

[0014] In one possible design, if the environmental scene information of the electronic device indicates that the electronic device is in a first environmental condition, the multimodal display method is the first display method; if the environmental scene information of the electronic device indicates that the electronic device is in a second environmental condition, which is different from the first environmental condition, the multimodal display method is the second display method, which is different from the first display method. Based on this design, for the same scene intent, different multimodal display methods are used under different environmental conditions to display the results of calling multiple APIs obtained under the scene intent, so that different display effects can be achieved under different environmental conditions, satisfying the user's preferences under different environmental conditions.

[0015] In one possible design, the first environmental condition is a quiet environment, and the first display method is to use video and / or audio to display the results of calling multiple target APIs; the second environmental condition is a noisy environment, and the second display method is to use text and / or pictures to display the results of calling multiple target APIs.

[0016] In one possible design, multiple target APIs correspond to user profiles. Based on this design, since user profiles can reflect user preferences and habits, the APIs corresponding to the scenario intent are compatible with the user profile. This allows the APIs under that scenario intent to also be compatible with the user's preferences and habits, better meeting the user's actual needs.

[0017] In one possible design, if the user portrait is a first user portrait, the multiple target APIs are multiple first APIs; if the user portrait is a second user portrait, and the second user portrait is different from the first user portrait, the multiple target APIs are multiple second APIs, and the multiple second APIs and the multiple first APIs include different APIs. Based on this design, for the same scenario intention, different users may correspond to different user portraits, and then the corresponding response information is output according to the different user portraits. In other words, different response information can be recommended for different users, which can flexibly respond to the needs of various users and avoid the homogeneity of the generated response information.

[0018] In one possible design, before receiving the user instruction, the method further includes: generating a user portrait based on a plurality of the second scenario intent, the API corresponding to the second scenario intent, and the user's operation records of the result of calling the API corresponding to the second scenario intent, where the second scenario intent is the scenario intent acquired before receiving the user instruction. Based on this design, the user portrait is generated based on the API corresponding to the scenario intent and the user's operation records of the result of calling the API corresponding to the historical scenario intent. The generated user portrait can reflect the user's true preferences or needs under one or more scenario intentions, and then the user's response result under a certain scenario intention is determined based on the user portrait that can reflect the user's true preferences or needs under one or more scenario intentions. This can make the response result more satisfy the user's needs under the scenario intention, and improve the accuracy and pertinence of the response result.

[0019] In one possible design, before acquiring multiple target APIs based on the first scenario intent, the method further includes: determining the first scenario intent based on a preset scenario intent set and a user instruction, where the preset scenario intent set includes multiple scenario intents. Based on this design, the terminal device determines the first scenario intent based on the preset scenario intent set and the user instruction, which can prevent the user's sensitive data from being uploaded to the cloud and protect the user's privacy.

[0020] In one possible design, the user portrait and / or the environmental scene information of the electronic device, and the first scene intention are included in the first information. The first information also includes at least one of a security verification code and the type of the electronic device. The security verification code is used to verify the first information, and the type of the electronic device is used to determine multiple target APIs. Based on this design, the information sent by the electronic device to the server contains a security verification code, which can ensure the security of the information sent by the electronic device to the server and prevent the information from being tampered with. Since multiple different electronic devices may have the same account (such as a Huawei account), for example, for the same user's mobile phone, tablet, watch, etc., the corresponding user account may be the same. Therefore, some data of these different electronic devices may be shared. In order to prevent the subsequent server from determining the same response results for these different electronic devices, that is, in order to output different response results on different electronic devices with the same account, the electronic device can send its own device type to the server to further improve the output effect of the response result.

[0021] In a second aspect, a human-computer interaction method is provided, which is applied to a server, and the method includes: receiving first information from an electronic device, the first information including a first scenario intention; generating second information based on the first information and an API corresponding to the first scenario intention, the second information including multiple target APIs, the multiple target APIs corresponding to the first scenario intention, and the multiple target APIs are used to implement user needs corresponding to the first scenario intention; and sending the second information to the electronic device.

[0022] In one possible design, the first information also includes a user portrait and / or environmental scene information of the electronic device, wherein the environmental scene information of the electronic device is used to characterize at least one of the environmental conditions around the electronic device and the status of the electronic device.

[0023] In one possible design, the second information also includes a multimodal display method of the results of calling multiple target APIs, and the multimodal display method is a method of using multiple display APIs in text, pictures, audio, and video.

[0024] In one possible design, the multimodal display method corresponds to the environmental scene information of the electronic device.

[0025] In one possible design, if the environmental scene information of the electronic device represents that the electronic device is in a first environmental condition, the multimodal display mode is the first display mode; if the environmental scene information of the electronic device represents that the electronic device is in a second environmental condition, and the second environmental condition is different from the first environmental condition, the multimodal display mode is the second display mode, and the second display mode is different from the first display mode.

[0026] In one possible design, the first environmental condition is a quiet environment, and the first display method is to use video and / or audio to display the results of calling multiple target APIs; the second environmental condition is a noisy environment, and the second display method is to use text and / or pictures to display the results of calling multiple target APIs.

[0027] In one possible design, multiple target APIs correspond to user profiles.

[0028] In one possible design, if the user profile is a first user profile, the multiple target APIs are multiple first APIs; if the user profile is a second user profile, and the second user profile is different from the first user profile, the multiple target APIs are multiple second APIs, and the multiple second APIs and the multiple first APIs include different APIs.

[0029] In one possible design, before generating the second information based on the first information and the API corresponding to the first scenario intent, the method also includes: determining the API corresponding to the first scenario intent based on a preset scenario intent set, the API corresponding to the preset scenario intent set, and the first scenario intent, wherein the preset scenario intent set includes multiple scenario intents, and the API corresponding to the preset scenario intent set is the API corresponding to each scenario intent in the multiple scenario intents.

[0030] In one possible design, generating the second information based on the first information and the API corresponding to the first scenario intent includes: inputting the first information and the API corresponding to the first scenario intent into a preset model, and outputting the second information through the preset model. In this way, the server inputs the user portrait, the electronic device's environmental scenario information, the scenario intent, the API corresponding to the scenario intent, etc. into the model, and outputs the response result through the model, which can obtain a more accurate and rich response result.

[0031] In one possible design, the first information also includes at least one of a security verification code and a type of electronic device. The security verification code is used to verify the first information, and the type of electronic device is used to determine multiple target APIs.

[0032] It should be noted that the technical effects brought about by other designs in the above-mentioned second aspect can be referred to the technical effects brought about by the corresponding designs in the first aspect, and will not be repeated here.

[0033] In a third aspect, an electronic device is provided, which has the function of implementing the method described in the first aspect and any design thereof. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions. In a possible example, the electronic device includes a processing unit (or processing module); the processing unit is used to: receive user instructions; when the user instructions represent the first scene intention, obtain multiple target APIs according to the first scene intention, and the multiple target APIs correspond to the first scene intention; output the results of calling multiple target APIs, and the results of calling multiple target APIs are used to realize user needs corresponding to the first scene intention.

[0034] In one possible design, the processing unit is specifically configured to obtain multiple target APIs based on a user profile and / or environmental scene information of the electronic device, and a first scene intent. The environmental scene information of the electronic device is used to represent at least one of the environmental conditions surrounding the electronic device and the state of the electronic device.

[0035] In one possible design, the electronic device also includes a communication unit; the communication unit is used to: send a user portrait and / or environmental scene information of the electronic device, and a first scene intention to a server; and receive multiple target APIs from the server.

[0036] In one possible design, the processing unit is specifically used to output the results of calling multiple target APIs in a multimodal display manner of the results of calling multiple target APIs. The multimodal display manner is a method of using multiple display APIs in text, pictures, audio, and video.

[0037] In one possible design, the communication unit is also used to receive a multimodal display of the results of calling multiple target APIs from the server.

[0038] In one possible design, the multimodal display method corresponds to the environmental scene information of the electronic device.

[0039] In one possible design, if the environmental scene information of the electronic device represents that the electronic device is in a first environmental condition, the multimodal display mode is the first display mode; if the environmental scene information of the electronic device represents that the electronic device is in a second environmental condition, and the second environmental condition is different from the first environmental condition, the multimodal display mode is the second display mode, and the second display mode is different from the first display mode.

[0040] In one possible design, the first environmental condition is a quiet environment, and the first display method is to use video and / or audio to display the results of calling multiple target APIs; the second environmental condition is a noisy environment, and the second display method is to use text and / or pictures to display the results of calling multiple target APIs.

[0041] In one possible design, multiple target APIs correspond to user profiles.

[0042] In one possible design, if the user profile is a first user profile, the multiple target APIs are multiple first APIs; if the user profile is a second user profile, and the second user profile is different from the first user profile, the multiple target APIs are multiple second APIs, and the multiple second APIs and the multiple first APIs include different APIs.

[0043] In one possible design, the processing unit is also used to generate a user portrait based on the second scenario intention, the API corresponding to the second scenario intention, and multiple operation records of the user's results of calling the API corresponding to the second scenario intention, where the second scenario intention is the scenario intention obtained before receiving the user's instructions.

[0044] In one possible design, the processing unit is also used to determine the first scene intention based on a preset scene intention set and user instructions, where the preset scene intention set includes multiple scene intentions.

[0045] In one possible design, the user portrait and / or environmental scene information of the electronic device, and the first scene intention are included in the first information. The first information also includes at least one of a security verification code and the type of the electronic device. The security verification code is used to verify the first information, and the type of the electronic device is used to determine multiple target APIs.

[0046] In a fourth aspect, a server is provided, which has the function of implementing the method described in the second aspect and any design thereof. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions. In a possible example, the server includes a communication unit (or communication module) and a processing unit (or processing module); the communication unit is used to receive first information from an electronic device, and the first information includes a first scenario intention; the processing unit is used to generate second information according to the first information and an API corresponding to the first scenario intention, and the second information includes multiple target APIs, and the multiple target APIs correspond to the first scenario intention, and the multiple target APIs are used to implement user needs corresponding to the first scenario intention; the communication unit is also used to send the second information to the electronic device.

[0047] In one possible design, the first information also includes a user portrait and / or environmental scene information of the electronic device, wherein the environmental scene information of the electronic device is used to characterize at least one of the environmental conditions around the electronic device and the status of the electronic device.

[0048] In one possible design, the second information also includes a multimodal display method of the results of calling multiple target APIs, and the multimodal display method is a method of using multiple display APIs in text, pictures, audio, and video.

[0049] In one possible design, the multimodal display method corresponds to the environmental scene information of the electronic device.

[0050] In one possible design, if the environmental scene information of the electronic device represents that the electronic device is in a first environmental condition, the multimodal display mode is the first display mode; if the environmental scene information of the electronic device represents that the electronic device is in a second environmental condition, and the second environmental condition is different from the first environmental condition, the multimodal display mode is the second display mode, and the second display mode is different from the first display mode.

[0051] In one possible design, the first environmental condition is a quiet environment, and the first display method is to use video and / or audio to display the results of calling multiple target APIs; the second environmental condition is a noisy environment, and the second display method is to use text and / or pictures to display the results of calling multiple target APIs.

[0052] In one possible design, multiple target APIs correspond to user profiles.

[0053] In one possible design, if the user profile is a first user profile, the multiple target APIs are multiple first APIs; if the user profile is a second user profile, and the second user profile is different from the first user profile, the multiple target APIs are multiple second APIs, and the multiple second APIs and the multiple first APIs include different APIs.

[0054] In one possible design, the processing unit is also used to determine the API corresponding to the first scene intent based on a preset scene intent set, the API corresponding to the preset scene intent set, and the first scene intent, wherein the preset scene intent set includes multiple scene intents, and the API corresponding to the preset scene intent set is the API corresponding to each scene intent in the multiple scene intents.

[0055] In one possible design, the processing unit is specifically used to input the first information and the API corresponding to the first scenario intention into a preset model, and output the second information through the preset model.

[0056] In one possible design, the first information also includes at least one of a security verification code and a type of electronic device. The security verification code is used to verify the first information, and the type of electronic device is used to determine multiple target APIs.

[0057] In a fifth aspect, an electronic device is provided, comprising a processor and a memory, wherein the memory is coupled to the processor, the memory being configured to store computer program code, the computer program code comprising computer instructions, and the processor reading the computer instructions from the memory to cause the electronic device to execute the method described in the first aspect and any design thereof. Optionally, the memory may be coupled to the processor or independent of the memory.

[0058] In one possible design, the electronic device further includes a communication interface that can be used for the electronic device to communicate with other devices. Exemplarily, the communication interface can be a transceiver, an input / output interface, an interface circuit, an output circuit, an input circuit, a pin, or related circuits.

[0059] In one possible design, the electronic device may further include an output device, which may include but is not limited to a display screen and an audio module. The display screen may be used to output image information. The audio module may be used to output audio information.

[0060] In a sixth aspect, a server is provided, comprising a processor, a memory, and a communication interface. The memory and the communication interface are coupled to the processor. The communication interface is configured to communicate with other devices. The memory is configured to store computer program code, wherein the computer program code includes computer instructions. The processor reads the computer instructions from the memory to cause the server to execute the method described in the second aspect and any design thereof. Optionally, the memory may be coupled to the processor or may be independent of the memory.

[0061] In the seventh aspect, a computer-readable storage medium is provided, which includes a computer program. When the computer program is run on an electronic device, the electronic device executes the method described in the first aspect and any one of the designs therein; or, when the computer program is run on a server, the server executes the method described in the second aspect and any one of the designs therein.

[0062] In an eighth aspect, a computer program product is provided, which includes: a computer program or instructions, which, when the computer program or instructions are run on a computer, enables the computer to execute the method described in the first aspect and any one of the designs thereof, or enables the computer to execute the method described in the second aspect and any one of the designs thereof.

[0063] In a ninth aspect, a circuit system is provided, the circuit system including a processing circuit, the processing circuit being configured to execute the method described in any one of the first or second aspects above.

[0064] In the tenth aspect, a chip system is provided, comprising at least one processor and at least one interface circuit, wherein the at least one interface circuit is used to perform transceiver functions and send instructions to the at least one processor. When the at least one processor executes the instructions, the at least one processor executes the method described in any one of the designs in the first or second aspect above.

[0065] In an eleventh aspect, a communication system is provided, comprising the electronic device as described in the third aspect and the server as described in the fourth aspect.

[0066] It should be noted that the technical effects brought about by any design in the above-mentioned third to eleventh aspects can be referred to the technical effects brought about by the corresponding designs in the first and second aspects, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] FIG1 is a schematic diagram of an existing human-computer interaction interface;

[0068] FIG2 is a schematic diagram of a process of an existing human-computer interaction method;

[0069] FIG3 is a schematic diagram of another existing human-computer interaction method;

[0070] FIG4 is a schematic diagram of the architecture of a communication system provided in an embodiment of the present application;

[0071] FIG5 is a schematic diagram of the structure of a server provided in an embodiment of the present application;

[0072] FIG6 is a schematic structural diagram of an electronic device provided in an embodiment of the present application;

[0073] FIG7 is a schematic structural diagram of another electronic device and a server provided in an embodiment of the present application;

[0074] FIG8 is a flow chart of a human-computer interaction method provided in an embodiment of the present application;

[0075] FIG9 is a first schematic diagram of an interface provided in an embodiment of the present application;

[0076] FIG10 is a second schematic diagram of an interface provided in an embodiment of the present application;

[0077] FIG11 is a third schematic diagram of an interface provided in an embodiment of the present application;

[0078] FIG12 is a schematic diagram of an API corresponding to a scenario intent provided in an embodiment of the present application;

[0079] FIG13 is a fourth schematic diagram of an interface provided in an embodiment of the present application;

[0080] FIG14 is a fifth schematic diagram of an interface provided in an embodiment of the present application;

[0081] FIG15 is a flow chart of another human-computer interaction method provided in an embodiment of the present application;

[0082] FIG16 is a schematic diagram of a user portrait provided in an embodiment of the present application;

[0083] FIG17 is a schematic diagram of another user portrait provided in an embodiment of the present application;

[0084] FIG18 is a schematic diagram of an API-associated user tag provided in an embodiment of the present application;

[0085] FIG19 is a schematic diagram of response information corresponding to different user portraits provided in an embodiment of the present application;

[0086] FIG20 is a schematic diagram of response information corresponding to different environmental information provided in an embodiment of the present application;

[0087] FIG21 is a sixth schematic diagram of an interface provided in an embodiment of the present application;

[0088] FIG22 is a seventh schematic diagram of an interface provided in an embodiment of the present application;

[0089] FIG23 is a schematic diagram of the structure of a device provided in an embodiment of the present application;

[0090] FIG24 is a schematic structural diagram of a chip system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0091] The technical solutions provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0092] The terms "including," "having," and any variations thereof mentioned in the description of this application are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or units is not limited to the listed steps or units, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to the process, method, product, or apparatus.

[0093] It should be noted that in the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be interpreted as being more preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0094] In this application, unless otherwise specified, "plurality" means two or more. "And / or" in this document is simply a description of an association between related objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, or B exists alone.

[0095] It is understood that in the embodiments of the present application, scenario intent can refer to the actual or potential needs of users in different scenarios. An application programming interface (API) is a set of definitions, procedures, and protocols that enable communication between computer software. For example, an API defines the types of calls and requests that can be made between different computer software, how to make calls and send requests, and the data format to be used.

[0096] Currently, electronic devices (such as mobile phones) can respond to user input of scene intent based on AI technology. For example, users can input scene intent to the electronic device through various means such as voice. After receiving the scene intent input by the user, the electronic device will invoke a single function in response to the scene intent, such as directly displaying text or cards.

[0097] For example, taking the electronic device as a mobile phone, when the user has a travel demand, the user can input a voice instruction such as "I am going to travel to Beijing at 8:30 tomorrow morning. Please give me some travel advice" into the mobile phone. In response to the voice instruction input by the user, the mobile phone can respond in the form of text or cards. In some examples, the mobile phone may not be able to recognize the user's scene intention based on the voice instruction input by the user. At this time, the mobile phone may display the content of the card 101 shown in Figure 1 (1), such as displaying a text of failed response such as "I don't quite understand". In other examples, after the mobile phone recognizes the user's scene intention based on the voice instruction input by the user, if it is determined that the user's scene intention is a travel intention, it may display the content of the card 102 shown in Figure 1 (2), such as recommending some simple information such as attractions and food culture to the user in the form of text.

[0098] The above response methods are limited to the semantic logic of scene intentions, the response results are too simple, cannot meet user needs, and the effect of human-computer dialogue is poor.

[0099] In an existing related solution, as shown in Figure 2, in a human-computer interaction scenario, a mobile phone receives voice information input by the user. The phone then performs semantic recognition on the voice information based on multimodal information (such as information about the surrounding conversation environment, the user's voice, and (facial or body) expressions) to understand the user's words and output a corresponding voice response. In this solution, the response to the user's inquiry is through voice output, and the response result is relatively simple. It cannot provide a rich, comprehensive, and convenient response based on the user's different scenario intentions, fails to meet user needs, and the human-computer interaction effect is poor.

[0100] In another existing related solution, as shown in Figure 3, during the voice interaction process, user input is received, a feedback statement is generated based on the voice information, and an emotional score is calculated for the feedback statement according to a predetermined emotional scoring mechanism. The emotional type is then determined based on the emotional score, and the feedback statement is played according to the emotional type. In this solution, only the corresponding feedback statement is played based on the user's input voice information, resulting in a relatively simple response. It is unable to provide a comprehensive and convenient response based on the user's different scenario intentions, fails to meet user needs, and the human-computer dialogue effect is poor.

[0101] Based on this, the embodiment of the present application provides a human-computer interaction method that can solve the problem of a single response result to user inquiries, and can provide richer, more comprehensive and convenient response results based on the user's scenario intentions, meet user needs, and improve the effect of human-computer dialogue.

[0102] For example, Figure 4 shows a schematic diagram of the architecture of a communication system for applying a human-computer interaction method provided in an embodiment of the present application. As shown in Figure 4 , the communication system 400 includes an electronic device 401 and a server 402 .

[0103] The electronic device 401 and the server 402 may be connected via wired communication technology and / or wireless communication technology. The wireless communication technology includes, but is not limited to, at least one of the following: near field communication (NFC), Bluetooth (BT) (e.g., traditional Bluetooth or Bluetooth low energy (BLE)), wireless local area networks (WLAN) (e.g., wireless fidelity (Wi-Fi) networks), Zigbee, frequency modulation (FM), infrared (IR), etc.

[0104] Optionally, the electronic device 401 may be, for example, a mobile phone, a personal computer (PC), a tablet computer (Pad), a laptop computer, a desktop computer, a laptop computer with a transceiver function, a wearable device, a vehicle-mounted device, an artificial intelligence (AI) device, or other terminal device. The operating system installed on the electronic device 401 includes but is not limited to Or other operating systems. In some embodiments, the electronic device 401 can be a fixed device or a portable device. This application does not limit the specific type of the electronic device 401 or the installed operating system.

[0105] Optionally, server 402 may be a device or server with computing capabilities, such as a cloud server or a network server. Server 402 may be a single server, a server cluster consisting of multiple servers, or a cloud computing service center. For example, a server cluster may be a server cluster consisting of one or more servers in one or more regions, or servers in one or more computer rooms. Optionally, the operating system of server 402 includes but is not limited to Or other operating systems. This application does not impose any restrictions on the specific type of server 402 and the installed operating system.

[0106] In some embodiments, the electronic device 401 may receive an instruction input by a user and determine the user's scenario intention based on the instruction. Optionally, the electronic device 401 may send the scenario intention to the server 402. Accordingly, after receiving the scenario intention, the server 402 may generate response information based on the scenario intention. Optionally, the server 402 may send the response information to the electronic device 401. Accordingly, after receiving the response information, the electronic device 401 may output the response information, such as displaying the response information on a display screen, broadcasting the response information by voice, etc.

[0107] It should be understood that FIG4 is only a simplified schematic diagram for ease of understanding, and the communication system may also include other devices, which are not shown in FIG4 .

[0108] Optionally, the server 402 in the embodiment of the present application can be implemented using different devices. For example, the server 402 in the embodiment of the present application can be implemented using the device shown in Figure 5. Figure 5 shows a schematic diagram of the hardware structure of the server 402 provided in the embodiment of the present application. The server 402 includes at least one processor 501, a communication line 502, a memory 503, and at least one communication interface 504.

[0109] The processor 501 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the present application.

[0110] Communication link 502 may include a pathway for transmitting information between the aforementioned components.

[0111] The communication interface 504 is used to communicate with other devices (the electronic device 401 shown in Figure 4). In an embodiment of the present application, the communication interface can be a module, a circuit, a bus, an interface, a transceiver or other device that can realize the communication function, for communicating with other devices. Optionally, when the communication interface is a transceiver, the transceiver can be an independently arranged transmitter, which can be used to send information to other devices, and the transceiver can also be an independently arranged receiver for receiving information from other devices. The transceiver can also be a component that integrates the functions of sending and receiving information. The embodiment of the present application does not limit the specific implementation of the transceiver. In some embodiments of the present application, the communication interface 504 can be used to receive one or more of the user's scene intention, the electronic device's environmental scene information, and the user portrait from the electronic device. In other embodiments of the present application, the communication interface 504 can be used to send response information obtained based on one or more of the user's scene intention, the electronic device's environmental scene information, and the user portrait to the electronic device.

[0112] The memory 503 may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, an optical disc storage (including a compact disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 503 may exist independently and be connected to the processor 501 via the communication line 502. The memory 503 may also be integrated with the processor 501.

[0113] Memory 503 is used to store computer-executable instructions for implementing the solution of the present application, and is controlled by processor 501 for execution. Processor 501 is used to execute the computer-executable instructions stored in memory 503, thereby implementing the methods provided in the following embodiments of the present application. For example, processor 501 determines corresponding response information based on one or more of the received user's scenario intention, the electronic device's environmental scenario information, and the user's profile.

[0114] Optionally, the computer-executable instructions in the embodiments of the present application may also be referred to as application code, instructions, computer programs or other names, which are not specifically limited in the embodiments of the present application.

[0115] In some embodiments of the present application, the memory 503 may also be used to store one or more of a scene intent set, an application programming interface (API) corresponding to the scene intent, an AI model, an intent framework system, etc. For a detailed introduction to the scene intent set, the API corresponding to the scene intent, the AI ​​model, the intent framework system, etc., please refer to the following description.

[0116] In a specific implementation, as an embodiment, the processor 501 may include one or more CPUs, such as CPU0 and CPU1 in FIG. 5 .

[0117] In a specific implementation, as an example, server 402 may include multiple processors, such as processor 501 and processor 505 in FIG5 . Each of these processors may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. A processor herein may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0118] The server 402 may be a general-purpose device or a dedicated device, and the embodiment of the present application does not limit the type of the server 402. The server 402 may be a device having a structure similar to that shown in FIG5.

[0119] It should be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on server 402. In other embodiments of the present application, server 402 may include more or fewer components than shown, or may combine or separate certain components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0120] For example, taking the electronic device 401 as a mobile phone as an example, FIG6 shows a schematic diagram of the hardware structure of the electronic device 401 provided in an embodiment of the present application.

[0121] The electronic device 401 includes a processor 610, an external memory interface 620, an internal memory 621, a universal serial bus (USB) interface 630, a charging management module 640, a power management module 641, a battery 642, an antenna 1, an antenna 2, a mobile communication module 650, a wireless communication module 660, an audio module 670, a sensor module 680, a button 690, a motor 691, an indicator 692, a camera 693, a display screen 694, and a subscriber identification module (SIM) card interface 695, etc.

[0122] The processor 610 may include one or more processing units, for example, an application processor (AP), a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0123] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0124] Processor 610 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 610 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 610. If processor 610 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 610 latency, and thus improves system efficiency.

[0125] In some embodiments, processor 610 may include one or more interfaces.

[0126] The charging management module 640 is used to receive charging input from a charger, which can be a wireless charger or a wired charger.

[0127] The power management module 641 is used to connect the battery 642, the charging management module 640 and the processor 610. The power management module 641 receives input from the battery 642 and / or the charging management module 640 to provide power to the processor 610, the internal memory 621, the display 694, the camera 693, and the wireless communication module 660.

[0128] The wireless communication function of the electronic device 401 can be implemented through antenna 1, antenna 2, mobile communication module 650, wireless communication module 660, etc.

[0129] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 401 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization.

[0130] The mobile communication module 650 can provide wireless communication solutions, including 2G / 3G / 4G / 5G, for electronic device 401. The mobile communication module 650 may include at least one filter, a switch, a power amplifier, a low-noise amplifier (LNA), and the like. The mobile communication module 650 can receive electromagnetic waves from antenna 1, filter and amplify the received electromagnetic waves, and transmit them to the modem processor for demodulation. The mobile communication module 650 can also amplify the signals modulated by the modem processor and convert them into electromagnetic waves for radiation via antenna 1.

[0131] The wireless communication module 660 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 401. The wireless communication module 660 can be one or more devices that integrate at least one communication processing module. The wireless communication module 660 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 610. The wireless communication module 660 can also receive the signal to be sent from the processor 610, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0132] In some embodiments, the antenna 1 of the electronic device 401 is coupled to the mobile communication module 650, and the antenna 2 is coupled to the wireless communication module 660, so that the electronic device 401 can communicate with the network and other devices (such as the server 402 shown in Figure 4) through wired communication technology or wireless communication technology.

[0133] In some embodiments of the present application, the electronic device 401 may also perceive the environmental scene information of the electronic device 401 through the antenna 1, the mobile communication module 650, the antenna 2, the wireless communication module 660, and the like.

[0134] Electronic device 401 implements display functionality through a GPU, display screen 694, and an application processor. A GPU is a microprocessor for image processing that connects display screen 694 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 610 may include one or more GPUs that execute program instructions to generate or modify display information.

[0135] The display screen 694 is used to display images, videos, etc. The display screen 694 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLed, a Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 401 may include one or N display screens 694, where N is a positive integer greater than 1. In some embodiments of the present application, the display screen 694 can be used to receive instructions input by a user. In other embodiments of the present application, the display screen 694 can also be used to display response information obtained based on the instructions input by the user.

[0136] The electronic device 401 can realize the shooting function through the ISP, camera 693, video codec, GPU, display screen 694 and application processor.

[0137] The camera 693 is used to capture still images or videos. In some embodiments, the electronic device 401 may include one or N cameras 693, where N is a positive integer greater than 1. In some embodiments of the present application, the camera 693 may also be used to capture images around the electronic device 401 to obtain environmental scene information of the electronic device 401.

[0138] The NPU is a neural-network (NN) computing processor that rapidly processes input information by drawing on the structure of biological neural networks, such as the transmission pattern between neurons in the human brain, and can also continuously self-learn. The NPU can realize applications such as intelligent cognition of electronic device 401, such as image recognition, face recognition, speech recognition, and text understanding. In some embodiments of the present application, the NPU can be used to perform speech recognition on voice commands input by the user.

[0139] The external memory interface 620 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 401. The external memory card communicates with the processor 610 via the external memory interface 620 to implement data storage functions. For example, files such as music and videos can be stored on the external memory card.

[0140] The internal memory 621 can be used to store computer executable program codes, which include instructions. The processor 610 executes various functional applications and data processing of the electronic device 401 by running the instructions stored in the internal memory 621. The internal memory 621 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area can store data created during the use of the electronic device 401 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 621 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0141] In some embodiments of the present application, one or more of a scene intent set, a user portrait, and an intent framework system may be stored in the internal memory 621. For an introduction to the scene intent set, user portrait, and intent framework system, please refer to the following description.

[0142] The electronic device 401 can implement audio functions such as music playback and recording through the audio module 670 and the application processor.

[0143] The audio module 670 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 670 can also be used to encode and decode audio signals. In some embodiments, the audio module 670 can be provided in the processor 610, or some functional modules of the audio module 670 can be provided in the processor 610.

[0144] Optionally, the audio module 670 may include at least one of a speaker, a receiver, a microphone, and an earphone interface.

[0145] In some embodiments of the present application, the audio module 670 may be used to receive instructions input by the user. In other embodiments of the present application, the audio module 670 may also be used to output response information obtained based on the instructions input by the user.

[0146] Optionally, the sensor module 680 may include a pressure sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, etc.

[0147] The buttons 690 include a power button, a volume button, etc. The buttons 690 may be mechanical buttons or touch buttons.

[0148] Motor 691 can generate vibration prompts.

[0149] Indicator 692 can be an indicator light, which can be used to indicate charging status, power changes, messages, notifications, etc.

[0150] The SIM card interface 695 is used to connect a SIM card.

[0151] The above description uses a mobile phone as an example to illustrate the structure of electronic device 401 in the embodiments of the present application, but does not limit the structure or form of electronic device 401. The embodiments of the present application do not limit the structure or form of electronic device 401. For example, the structure of electronic device 401 may also take the form of server 402 as shown in Figure 5, but the present application is not limited to this.

[0152] In other embodiments of the present application, the electronic device 401 may include more or fewer components than those shown in FIG6 , or may combine certain components, separate certain components, replace certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0153] In some embodiments, the electronic device 401 and the server 402 may be divided into functional modules. For example, each functional module may be divided into different functional modules corresponding to each function, or two or more functions may be integrated into one processing unit. The above-mentioned integrated unit may be implemented in the form of hardware or software functional modules.

[0154] As a possible division method, FIG7 shows a schematic diagram of the software structure of an electronic device 401 and a server 402 provided in an embodiment of the present application.

[0155] As shown in FIG. 7 , the electronic device 401 may include a scene intent recognition module 701 and an intent framework system 702 .

[0156] Among them, the scene intent recognition module 701 can be used to identify whether a user instruction is a scene intent, and specifically which scene intent it is. The user instruction can be a user instruction received by the electronic device 401. For specific implementation of how to identify whether a user instruction is a scene intent, and specifically which scene intent it is, please refer to the following description.

[0157] The intent framework system 702 may be a system for implementing communication between the electronic device 401 and the server 402. In some embodiments, the intent framework system 702 may be configured to send the scene intent identified by the scene intent identification module 701 to the server 402. In another embodiment, the intent framework system 702 may be configured to receive response information from the server 402.

[0158] Optionally, the electronic device 401 may further include at least one of a user portrait generation module 703 and an environmental scene information acquisition module 704 .

[0159] The user portrait generation module 703 may be used to generate a user portrait. For more information about user portraits, please refer to the following text. In some embodiments, the user portrait may also be used to generate response information. The intent framework system 702 may also be used to send the user portrait generated by the user portrait generation module 703 to the server 402.

[0160] Environmental scene information acquisition module 704 can be used to acquire environmental scene information. Environmental scene information can be used to represent at least one of the environmental conditions surrounding electronic device 401 and the status of electronic device 401. In some embodiments, this environmental scene information can also be used to generate response information. Intent framework system 702 can also be used to send the environmental scene information acquired by environmental scene information acquisition module 704 to server 402.

[0161] The server 402 may include an intent framework system 711 and a response information generation module 712 .

[0162] Among them, the intent framework system 711 can be a system for implementing communication between the electronic device 401 and the server 402. In some embodiments, the intent framework system 711 can be used to receive one or more of the following from the electronic device 401: scenario intent, the API corresponding to the scenario intent, the user profile, the environmental scenario information, etc. For an introduction to the API corresponding to the scenario intent, please refer to the following description. In other embodiments, the intent framework system 711 can also be used to send response information to the electronic device 401.

[0163] In some other embodiments, the intent framework system 711 may also be used to send one or more of the scenario intent, the API corresponding to the scenario intent, the user profile, the environmental scenario information, etc. to the response information generation module 712. In some other embodiments, the intent framework system 711 may also be used to receive response information from the response information generation module 712.

[0164] In some other embodiments, the intent framework system 711 may also be used to receive notification messages from the electronic device 401, user operational behavior data for response information, and the like. Optionally, the notification messages from the electronic device 401, user operational behavior data for response information, and the like may be reported by system applications and / or third-party applications in the electronic device 401. Exemplarily, the notification messages from the electronic device 401 may include, but are not limited to, various notification messages such as weather notifications, flight delay notifications, train delay notifications, product discount notifications, and traffic congestion notifications. In some embodiments, the intent framework system 711 may also be used to send notification messages from the electronic device 401, user operational behavior data for response information, and the like to the response information generation module 712.

[0165] The response information generation module 712 can be used to generate response information corresponding to the user instruction. In some embodiments, the response information generation module 712 can receive one or more of the following from the intent framework system 711: the scenario intent, the API corresponding to the scenario intent, the user portrait, the environmental scenario information, the notification message of the electronic device 401, the user's operational behavior data for the response information, and the like, and then generate the response information based on one or more of the following: the scenario intent, the API corresponding to the scenario intent, the user portrait, the environmental scenario information, the notification message of the electronic device 401, the user's operational behavior data for the response information, and the like. In some embodiments, the response information generation module 712 can also be used to send the response information to the intent framework system 711.

[0166] In a specific implementation, an AI model (or big model, or AI big model, etc.) can be set in the response information generation module 712. The response information generation module 712 can input one or more of the scene intent, the API corresponding to the scene intent, the user portrait, the environmental scene information, the notification message of the electronic device 401, the user's operation behavior data on the response information, etc. into the AI ​​model, and output the response information through the AI ​​model.

[0167] It can be understood that the AI ​​model can be various models with the ability to understand and generate text. For example, the AI ​​model can be a machine learning model or a deep learning model obtained through training with a large number of parameters and a complex structure. The embodiments of the present application do not impose specific restrictions on the training method of the AI ​​model.

[0168] It can be understood that the software modules included in the electronic device 401 and / or server 402 shown in Figure 7 are only used as a possible division method. In actual applications, the electronic device 401 and / or server 402 may include more or fewer modules, or there may be other module division methods, which are not limited in the embodiments of the present application.

[0169] The technical solutions involved in the following embodiments can all be implemented in devices having the structures shown in Figures 5, 6, and 7, and in a system having the architecture shown in Figure 4.

[0170] It can be understood that the various interfaces of the electronic device shown in the embodiments of the present application are given by taking the electronic device as a mobile phone as an example.

[0171] It is also understood that in the embodiments of the present application, the electronic device and / or server may perform some or all of the steps in the embodiments of the present application. These steps or operations are merely examples, and the embodiments of the present application may also perform other operations or variations of various operations. In addition, the various steps may be performed in a different order than those presented in the embodiments of the present application, and it is possible that not all operations in the embodiments of the present application need to be performed.

[0172] In some embodiments, response information can be generated based on the scene intent and the API corresponding to the scene intent. In this embodiment, Figure 8 shows a flow chart of a human-computer interaction method provided by an embodiment of the present application. As shown in Figure 8, the method includes the following steps:

[0173] S801: The electronic device receives a user instruction.

[0174] Exemplarily, user instructions (or user operations) may include but are not limited to various types of instructions executed by the user, such as voice instructions, key instructions, gesture instructions, etc.

[0175] In some examples, taking the user instruction as a voice instruction as an example, FIG9 shows a schematic diagram of a scenario in which an electronic device receives a voice instruction provided by an embodiment of the present application. As shown in (1) in FIG9 , the electronic device can display a voice instruction receiving interface 900. Optionally, a voice mark 901 can be included in the voice instruction receiving interface 900, and the voice mark 901 facilitates the user to know that the voice instruction receiving function of the electronic device is turned on. Optionally, one or more shortcut tags, such as "selfie", "turn on the flashlight", etc., can also be included in the voice instruction receiving interface 900. The user can also input instructions to the electronic device by performing operations such as clicking on the shortcut tags.

[0176] For example, when the electronic device displays the voice instruction receiving interface 900 shown in FIG9 (1), that is, when the voice instruction receiving function is turned on, if the user says a voice instruction such as "I am going to travel to Beijing at 8:30 tomorrow morning, please give me some travel advice", the electronic device can receive the voice instruction. Optionally, after receiving the voice instruction, the electronic device can also output the recognized voice instruction, such as outputting the recognized voice instruction in the form of text or the like. For example, the electronic device can display the instruction output interface 910 shown in FIG9 (2) to facilitate the user to confirm whether the instruction recognized by the electronic device is consistent with the instruction input by the user.

[0177] In other examples, taking the user instruction as a key instruction as an example, FIG10 shows a schematic diagram of a scenario in which an electronic device receives a key instruction provided by an embodiment of the present application. As shown in FIG10 (1), the electronic device can display a key instruction receiving interface 1000, wherein the key instruction receiving interface 1000 includes a keyboard icon 1001, and the user can input the instruction after calling the input method application through the keyboard icon 1001. Optionally, the key instruction receiving interface 1000 can also include one or more shortcut tags. For an introduction to these shortcut tags, please refer to the introduction to the shortcut tags shown in FIG9 (1). In some examples, the user can perform an operation such as clicking on the voice mark 901 shown in FIG9 (1). In response to the operation, the electronic device can display the key instruction receiving interface 1000 shown in FIG10 (1). Of course, the user can also call up the key instruction receiving interface 1000 in other ways, and the embodiment of the present application does not impose any restrictions on this.

[0178] If the electronic device detects a user clicking on the keyboard icon 1001 shown in (1) of FIG10 , in response to the operation, the electronic device may call an input method application and display a keyboard 1010 as shown in (2) of FIG10 , and the user may input instructions through the keyboard 1010. As shown in (2) of FIG10 , after the user inputs the key instruction "I am going to travel to Beijing at 8:30 tomorrow morning, please give me some travel advice" in the input box through the keyboard 1010, the electronic device may receive the key instruction. Optionally, as shown in (3) of FIG10 , the electronic device may also output the received key instruction so that the user knows that the electronic device has successfully received the input instruction.

[0179] In some embodiments, before the electronic device executes step S801, the electronic device may first turn on the function of receiving user instructions. As a possible implementation, the function can be turned on in real time, or in other words, the function is turned on by default after the electronic device is turned on, so that the user can directly input user instructions to the electronic device. As another possible implementation, the function of the electronic device may be in a closed state. At this time, the user can also start the function of the electronic device before inputting user instructions to the electronic device. For example: the user can perform a turn-on operation. Exemplarily, the turn-on operation may include but is not limited to voice operation (such as voice input wake-up word), button operation (such as long pressing the power off button), gesture operation, etc. In response to the turn-on operation, the electronic device starts the function of receiving user instructions.

[0180] For example, FIG11 shows a schematic diagram of a scenario for enabling the function of receiving user commands of an electronic device provided by an embodiment of the present application. As shown in FIG11 (1), the user inputs the wake-up word "Hi, Xiaox" into the electronic device by voice, and in response to the wake-up word, the electronic device enables the function of receiving user commands. As shown in FIG11 (2), the user long presses the power button of the electronic device, and in response to this operation, the electronic device can also enable the function of receiving user commands.

[0181] Optionally, after the electronic device turns on the function of receiving user commands, the electronic device may display a voice command receiving interface 900 as shown in FIG9 (1), or may directly display a key command receiving interface 1000 as shown in FIG10 (1). Optionally, the electronic device may also output a voice response to the user so that the user knows that the function of receiving user commands of the electronic device has been turned on.

[0182] Optionally, when receiving a user instruction, the electronic device may be in a black screen state or a bright screen state. When in the bright screen state, it may be in a locked screen state or an unlocked state. When in the unlocked state, the running interface of any application may be displayed. This embodiment of the application does not impose any restrictions on this.

[0183] In some scenarios, the function of receiving user commands of an electronic device can be integrated into a smart voice assistant. It is understood that a smart voice assistant can be intelligent software with voice recognition capabilities and can respond to user commands through various technical means.

[0184] S802: The electronic device determines the scene intention according to the user instruction.

[0185] In some scenarios, for a user instruction, the user instruction may correspond to a certain scenario intention (that is, the user instruction is an instruction used to implement the scenario intention), or may not correspond to any scenario intention (that is, the user instruction is not an instruction used to implement the scenario intention).

[0186] For example, if the user command is "Play a song," "Answer the phone for me," "Call my mom," "Set an alarm for 6:00 AM," or "Tell me about the climate in northern China," then the user command does not correspond to a scenario intent. If the user command is "I'm going to Beijing for a trip at 8:30 AM tomorrow. Please give me some travel advice," "What suitable jobs are there currently? Please recommend me," "I want to buy a refrigerator. Please give me some advice," or "I plan to go to Shanghai for medical treatment tomorrow. Please give me some advice," then the user command corresponds to a scenario intent.

[0187] In some embodiments, the electronic device can identify whether a user instruction corresponds to a scene intent based on the scene intent set, and, if the user instruction corresponds to a scene intent, which scene intent it specifically corresponds to. For example, the electronic device can use the scene intent recognition module 701 shown in Figure 7 to identify whether a user instruction corresponds to a scene intent based on the scene intent set, and which scene intent it specifically corresponds to.

[0188] It can be understood that in the embodiment of the present application, the scenario intent set can be a set of user intents. The user intent included in the scenario intent set is a scenario intent, and the user intent not included in the scenario intent set is a non-scenario intent, that is, not a scenario intent. In other words, if a user intent is in the scenario intent set, it means that the user intent is a scenario intent, and if a user intent is not in the scenario intent set, it means that the user intent is not a scenario intent.

[0189] In the embodiment of the present application, the user intent included in the scenario intent set is referred to as scenario intent. In some embodiments, the scenario intent set may include multiple scenario intents, and different scenario intents can represent the user's needs in different scenarios.

[0190] For example, the scenario intent set may include, but is not limited to, one or more scenario intents selected from the group consisting of travel intent, medical treatment intent, learning and education intent, health and fitness intent, shopping and consumption intent, work and career intent, etc. It will be understood that the scenario intents described in the embodiments of this application are merely exemplary illustrations made to facilitate understanding of the embodiments of this application. In actual applications, more or fewer scenario intents, or other types of scenario intents, may be included.

[0191] In some implementations, the scene intent set may be sent by a server to the electronic device. Optionally, the electronic device may receive the scene intent set sent by the server before executing step S802. Optionally, the electronic device may also receive the scene intent set sent by the server before executing step S801, or after step S801 and before step S802. In other implementations, the scene intent set may also be obtained by the electronic device through other means, and the embodiments of the present application do not limit this.

[0192] In this embodiment, as a specific implementation, the electronic device can determine whether a user instruction corresponds to a scene intent based on the keywords corresponding to the scene intent contained in the scene intent set. If a user instruction contains a keyword corresponding to a scene intent in the scene intent set, then the user instruction corresponds to the scene intent. If the user instruction does not contain a keyword corresponding to any scene intent in the scene intent set, then the user instruction does not correspond to the scene intent.

[0193] For example, if the scenario intent set includes travel intent, and the keywords corresponding to travel intent include "travel," "tourism," and "travel out," then if a user command contains any of these keywords, it means that the user command corresponds to travel intent. For example, if the user command is "I'm traveling to Beijing at 8:30 tomorrow morning. Please give me some travel advice," and the user command contains the keywords "travel" and "travel out," it means that the user command corresponds to travel intent.

[0194] For example, if the scenario intent set includes shopping and consumption intent, and the keywords corresponding to these shopping and consumption intents include "buy," "buy," and "shopping," then if a user command contains any of these keywords, it indicates that the user command corresponds to shopping and consumption intent. For example, if the user command "I want to buy a refrigerator, please give me some advice" contains the keyword "buy," it indicates that the user command corresponds to shopping and consumption intent.

[0195] For example, if the scenario intent includes work and career intent, and the keywords corresponding to these intents include "work," "career," and "employment," then if a user command contains any of these keywords, it indicates that the user command corresponds to work and career intent. For example, if the user command is "What suitable jobs are there currently? Please recommend me some?" and the keyword "work" is included in the command, it indicates that the user command corresponds to work and career intent.

[0196] If the electronic device determines that the received user instruction corresponds to a certain scene intention, the electronic device may execute step S803 and subsequent steps. If the electronic device determines that the received user instruction corresponds to a non-scene intention, the electronic device may not execute step S803 and subsequent steps. In some embodiments, the electronic device may not respond. In other embodiments, the electronic device may output response information corresponding to a non-scene intention, etc., and this application does not limit this.

[0197] S803: The electronic device sends the scene intent to the server. Correspondingly, the server receives the scene intent from the electronic device.

[0198] For example, the electronic device may use the intent framework system 702 shown in Figure 7 to send the scenario intent to the server. Correspondingly, the server may also use the intent framework system 711 shown in Figure 7 to receive the scenario intent from the electronic device.

[0199] In some embodiments, the electronic device may first determine the scenario intent corresponding to the user instruction and then send the corresponding scenario intent to the server. For example, if the user instruction is "I am going to travel to Beijing at 8:30 tomorrow morning. Please give me some travel suggestions", the electronic device recognizes that the user instruction is a travel intent and may send the "travel intent" to the server.

[0200] Optionally, in this embodiment, the electronic device may also send some key information corresponding to the identified scenario intention to the server. For example: taking the travel intention as an example, the key information corresponding to the travel intention may include but is not limited to one or more of the travel location, number of travelers, departure time, etc. For another example: taking the medical treatment intention as an example, the attribute information of the medical treatment intention may include but is not limited to one or more of the medical treatment location, departure time, etc. For another example: taking shopping and consumption intentions as an example, the attribute information of shopping and consumption intentions may include but is not limited to the types of purchased items, etc. Optionally, the key information corresponding to the scenario intention may be obtained according to user instructions.

[0201] In some other embodiments, the electronic device may also send the user instruction and the scene intent identified according to the user instruction to the server. Optionally, in this embodiment, the electronic device may no longer send the key information corresponding to the scene intent.

[0202] It is understood that the operation of determining the scene intent based on the user instruction can be performed by the electronic device or by the server. For example, the server can determine the scene intent based on the received user instruction. Optionally, the server can also use the same method as the electronic device, such as identifying the user instruction through the scene intent set to determine the scene intent.

[0203] Of course, the electronic device can also send the user instruction to other devices besides the server, and the other devices will determine the scene intent based on the user instruction, and then send the scene intent to the server directly or indirectly. Optionally, other devices can also use the same method as the electronic device, such as identifying the user instruction through the scene intent set to determine the scene intent. Optionally, if the scene intent set is not stored in the other device, the other device can also obtain the scene intent set from the electronic device and / or the server.

[0204] S804: The server generates response information according to the scenario intent.

[0205] In some embodiments, the server may generate response information based on the scenario intent and the API corresponding to the scenario intent. For example, the server may generate response information based on the scenario intent and the API corresponding to the scenario intent through the response information generation module 712 described in FIG. 7 .

[0206] In some embodiments, a server stores a scenario intent set and an API corresponding to each scenario intent contained in the scenario intent set. Optionally, for each scenario intent, one or more APIs may correspond, and some or all of the one or more APIs may be used to output the results of calling the one or more APIs to the user after receiving the corresponding scenario intent. By calling different APIs, the same or different functions of one or more applications may be implemented, and the functions may be used to meet the user's possible needs under the scenario intent.

[0207] In some embodiments, the API corresponding to each scenario intent included in the scenario intent set can be set according to the user's possible needs under the corresponding scenario intent. In some embodiments, the API corresponding to each scenario intent included in the scenario intent set can be an API supported by the electronic device, that is, the electronic device can successfully call the API corresponding to each scenario intent included in the scenario intent set.

[0208] For example, Figure 12 shows examples of APIs corresponding to some scenario intents provided by embodiments of the present application. For example, taking the travel intent as an example, the APIs corresponding to the travel intent may include an alarm setting API, a weather check API, a flight booking API, and an accommodation booking API. The alarm setting API may include, but is not limited to, APIs for various applications and / or mini-programs that implement alarm setting functions, such as the Clock application API. The weather check API may include, but is not limited to, APIs for various applications and / or mini-programs that implement weather viewing functions, such as the Huawei Weather application API, the GaoGao Weather application API, and the MoMo Weather application API. The flight booking API may include, but is not limited to, APIs for various applications and / or mini-programs that implement flight booking functions, such as the FeiFei application flight query API, the XieXi application flight query API, and the KongKong application flight query API. The accommodation booking API may include, but is not limited to, APIs for various applications and / or mini-programs that implement accommodation booking functions, such as the FeiFei application hotel query API, the XieXi application hotel query API, and the HuaHua application hotel query API. For an introduction to APIs corresponding to other scenario intents, please refer to the introduction to APIs corresponding to the travel intent.

[0209] It can be understood that the API corresponding to the scenario intent described in the embodiments of the present application is only an exemplary illustration made to facilitate the embodiments of the present application. In actual applications, more or fewer APIs, or other types of APIs may be included.

[0210] In a specific embodiment, the server can first determine the API corresponding to the received scenario intent based on the API corresponding to each scenario intent contained in the stored scenario intent set and the scenario intent received from the electronic device, and then input the API corresponding to the received scenario intent and the received scenario intent into the AI ​​model, and output the response information through the AI ​​model. For an introduction to the AI ​​model, please refer to the above. In this way, by outputting the response information through the AI ​​model, more accurate and rich response information can be obtained.

[0211] In some specific implementations, the server may use the intent framework system 711 shown in Figure 7 to input the received scene intent corresponding to the API and the received scene intent into the AI ​​model. The server may also send the response information output by the AI ​​model to the intent framework system 711 shown in Figure 7.

[0212] Of course, in other embodiments, the server may not perform the operation of determining the API corresponding to the received scenario intent based on the API corresponding to each scenario intent contained in the stored scenario intent set and the scenario intent received from the electronic device, but directly input the API corresponding to each scenario intent contained in the stored scenario intent set and the received scenario intent into the AI ​​model, and output response information through the AI ​​model.

[0213] In some embodiments, in conjunction with step S803, when the scenario intent received by the server is a scenario intent identified based on a user instruction, and the server also receives the user instruction or key information corresponding to the scenario intent, the server may also input the user instruction or key information corresponding to the scenario intent into the AI ​​model, and output response information through the AI ​​model. In this way, obtaining response information in combination with the user instruction or key information of the scenario intent can improve the accuracy of the obtained response information.

[0214] In some embodiments, the response information may include but is not limited to one or more APIs, and calling these one or more APIs can implement one or more functions, and these one or more functions can meet the user's possible needs under the scene intent. Optionally, the API included in the response information can be part or all of the APIs corresponding to the scene intent stored in the server. In this way, the response information may include multiple APIs, and calling these APIs can implement multiple functions, and these multiple functions can meet the needs under the scene intent input by the user, providing the user with richer and more comprehensive response information, which can improve the effect of human-computer dialogue.

[0215] Optionally, the response information may also include a multimodal display of the results of calling each API. It is understood that a multimodal display may refer to a method of displaying the results of calling one or more APIs using a combination of multiple media types (such as, but not limited to, text, images, video, audio, etc.). Displaying the results of calling multiple APIs in a multimodal display can make the response information richer and more intuitive.

[0216] For example, if the scenario intent is travel, the generated response information may include one or more APIs such as the alarm setting API (such as the Alarm Clock App API), flight booking API (such as the Feifei App Flight Query API), and accommodation booking API (such as the Xie Xie App Hotel Query API, the Feifei App Hotel Query API, and the Huahua App Hotel Query API). The results of calling these APIs are displayed multimodally through text, images, and videos. In addition, videos can also support voice command playback.

[0217] For example, if the scenario intent is shopping and consumption, the generated response information may include one or more APIs such as shopping APIs (such as Taotao application API, Dongdong application API, and Pinxixi application API), coupon purchase APIs (such as official account coupon query API and Tuantuan application coupon query API), price comparison APIs (such as Taotao application price query API and Dongdong application price query API), and product review APIs (such as Xiaolvshu application product review display API). The multimodal display method of calling these APIs is through text, images, and voice.

[0218] For example, if the scenario intent is work and career, the generated response information may include one or more APIs such as recruitment APIs (such as direct recruitment application APIs and job search website application APIs), career development information viewing APIs (such as browser application career development information search APIs), and industry news playback APIs (such as Huawei Video application news playback APIs). The multimodal display method of calling these APIs is through text, images, and videos.

[0219] Optionally, the response information may also include some fixed-format text, preset images, etc. The fixed-format text can serve as a response statement to the user's instruction, making the response information more complete. For example, the fixed-format text may be such as "OK", "The following are xx planning suggestions", etc.

[0220] S805: The server sends a response message to the electronic device. Correspondingly, the electronic device receives the response message from the server.

[0221] For example, the server may send response information to the electronic device through the intent framework system 711 shown in Figure 7. Correspondingly, the electronic device may receive response information from the electronic device through the intent framework system 702 shown in Figure 7.

[0222] S806: The electronic device outputs response information.

[0223] For example, the electronic device may display the response information on a display screen, or may announce the response information through a microphone, earpiece, or headphones connected to the electronic device. It is understood that in the embodiments of the present application, the electronic device outputting the response information may refer to the electronic device outputting corresponding information in response to the user's instruction based on the content of the response information, and does not mean that the information output by the electronic device must be exactly the same as the content of the response information. For example, if the response information contains an API, the information output by the electronic device may be the result of calling the API.

[0224] In some embodiments, the display effect of the result of the API in the call response information output by the electronic device can be determined by the server. As described above, the response information can include a multimodal display method of the result of calling the API, and the electronic device can output the result of the API contained in the call response information according to the multimodal display method included in the response information. In this way, the display method of the result of calling the API is determined by the server, and the electronic device can directly perform the corresponding display according to the display method determined by the server, without the need to perform additional operations to determine the display method of the result of calling the API, which can save the power consumption of the electronic device. In addition, the computing power of the server is higher than the computing power of the electronic device, so that a better display method can be obtained, so that the display method of the result of calling the API is in line with the current scene intention, which can further improve the display effect of the result of calling the API.

[0225] In other embodiments, the display effect of the result of the API in the call response information output by the electronic device may be determined by the electronic device. For example, after receiving the API included in the response information, the electronic device may also set a multimodal display method for the result of calling one or more APIs, and then output the result of calling the one or more APIs according to the set multimodal display method. Optionally, in this embodiment, the response information received by the electronic device may include the multimodal display method determined by the server, or may not include the multimodal display method determined by the server.

[0226] As a possible implementation, the electronic device can randomly set the multimodal display mode. As another possible implementation, the electronic device can set the multimodal display mode according to the environmental scene information of the electronic device. Among them, the environmental scene information can be used to characterize at least one of the environmental conditions around the electronic device and the state of the electronic device. Based on this design, the electronic device sets the multimodal display mode of the API based on the environmental scene information, which can make the display result of the set API correspond to the current environmental scene information. Different display effects can be achieved under different environmental scene information to better meet user needs.

[0227] For example, the environmental scene information may include, but is not limited to, one or more of the quietness of the environment in which the electronic device is located, the location and / or scene in which the electronic device is located, the communication method used by the electronic device, whether the Bluetooth of the electronic device is turned on, the battery level of the electronic device, etc. For example, the quietness of the environment in which the electronic device is located may include quiet or noisy. The scene in which the electronic device is located may include, but is not limited to, home scenes, car scenes, shopping mall scenes, bookstore scenes, subway station scenes, company scenes, restaurant scenes, etc. The communication method used by the electronic device may include, but is not limited to, Wi-Fi networks, 5G networks, 4G networks, etc.

[0228] For example, if an electronic device is in a noisy environment, the user may be unable to hear or focus on the video or audio content, so the user may prefer to read text or view images. In this case, the electronic device can use a multimodal display method that combines text and images. That is, the electronic device can use a combination of text and images to display the results of the API call response information.

[0229] For example, if the electronic device is in a quiet environment, the user may prefer to watch videos or listen to audio content. In this case, the electronic device can use a multimodal display method that combines video and audio. That is, the electronic device can use a combination of video and audio to display the results of the API call response information.

[0230] Optionally, the electronic device's environmental scene information can be collected by the electronic device. For example, the electronic device can obtain the electronic device's environmental scene information by collecting various information such as audio information, positioning information, environmental image information, and geo-fence information. Alternatively, the environmental scene information of the electronic device can be determined by other devices and then sent to the electronic device. Of course, the electronic device can also cooperate with other devices to obtain the electronic device's environmental scene information, and this application does not impose any restrictions on this.

[0231] In some implementations, the electronic device may employ artificial intelligence generated content (AIGC) technology to output the response information in a multimodal display manner. Of course, the electronic device may also employ other technologies to output the response information in a multimodal display manner, and the embodiments of the present application are not limited thereto.

[0232] For example, in conjunction with the user instructions shown in Figures 9 and 10, Figure 13 shows a schematic diagram of an interface for response information output by an electronic device provided in an embodiment of the present application. As shown in Figure 13, the electronic device can display a response interface 1300, wherein the response interface 1300 includes a card 1302, which can be used to display the result of calling the API included in the response information. Optionally, the response interface 1300 can also include a user instruction 1301.

[0233] For example, card 1302 includes the result 1303 of calling the alarm setting API, the result 1304 of calling the flight booking API, and the result 1305 of calling the accommodation booking API. It is understood that the alarm setting API may be an API for implementing the alarm setting function of an application, the flight booking API may be an API for implementing the flight booking function of an application, and the accommodation booking API may be an API for implementing the accommodation booking function of an application.

[0234] The function of setting an alarm can be achieved by calling the result 1303 of the alarm setting API. In some examples, when the electronic device displays the result 1303 of calling the alarm setting API, it can automatically set the alarm for the user, such as when the function button 1303a is turned on. In this way, the electronic device has pre-set the alarm for the user at 7 o'clock in the morning. In other examples, when the electronic device displays the result of calling the alarm setting API, the electronic device has not pre-set the alarm for the user, and the user can make a reservation on his own. For example, the user can directly turn on the function button 1303a, so that the user can reserve an alarm for 7 o'clock in the morning. Alternatively, the user can also perform an operation such as clicking on the result 1303 of calling the alarm setting API. In response to the operation, as shown in (1) in Figure 14, the electronic device can display the alarm interface 1400, and the user can reserve a suitable alarm on the alarm interface 1400.

[0235] The function of booking air tickets can be realized by calling the result 1304 of the ticket booking API. In some examples, the electronic device can display the information of one or more flights by calling the result 1304 of the ticket booking API, and the user can select the appropriate flight based on the information of one or more flights displayed. For example, if the electronic device detects that the user has selected the information of one of the flights, such as the user performs an operation such as clicking on the information of any flight included in the result 1304 of calling the ticket booking API. In response to the operation, as shown in (2) in Figure 14, the electronic device can display the ordering interface 1410 of the flight. The user can complete the ordering of the corresponding flight through the ordering interface 1410. Optionally, the ordering interface 1410 can be a settlement interface for the flight, or it can be other interfaces for the flight, such as an economy class or first class selection interface (Figure 14 (2) is used as an example), and the embodiment of the present application does not impose specific restrictions on this.

[0236] In other examples, the user can also view more flight information by calling the result 1304 of the flight booking API. For example, the user can perform an operation such as clicking on the result 1304 of the flight booking API. In response to this operation, as shown in (3) in Figure 14, the electronic device can display a flight display interface 1420. The user can select a suitable flight to book in the flight display interface 1420.

[0237] The function of accommodation reservation can be realized by calling the result 1305 of the accommodation reservation API. In some examples, the electronic device can display information of one or more bookable hotels by calling the accommodation reservation API, including but not limited to hotel pictures (as shown in Figure 13 as an example), hotel videos, hotel locations, etc. The user can view more hotels by calling the result 1305 of the accommodation reservation API. In some examples, the user can perform operations such as clicking on the result 1305 of the accommodation reservation API. In response to this operation, as shown in (4) of Figure 14, the electronic device can display a hotel ordering interface 1430. The user can select a suitable hotel to order through the hotel ordering interface 1430.

[0238] In other examples, the user may also select a suitable hotel by displaying information about one or more available hotels displayed as a result of calling an accommodation reservation API. For example, if the electronic device detects an action, such as a user clicking on a particular hotel, the electronic device may display a booking interface for that hotel in response to the action, allowing the user to select a suitable room for reservation.

[0239] Optionally, card 1302 may also include texts such as "Because the weather in Beijing is hot and it may rain, it is recommended that you bring summer clothes and rain gear to cope with different weather conditions. When going out, pay attention to sun protection and hydration to avoid heatstroke." to remind users of weather conditions, and / or fixed-format texts such as "OK" and "The following are travel planning suggestions."

[0240] Optionally, card 1302 may further include a button for implementing the functions of one or more API call results included in card 1302, such as a one-click travel button 1306. The electronic device detects that the user performs an operation such as clicking on the one-click travel button 1306. In response to the operation, the electronic device may book the alarm (e.g., a 7 o'clock alarm) displayed in the result 1303 of calling the alarm setting API, and / or the electronic device may choose to book a flight displayed in the result 1304 of calling the flight booking API, and / or the electronic device may choose to book a hotel displayed in the result 1305 of calling the accommodation booking API, etc. Optionally, the electronic device may also display a settlement page for air tickets, hotels, etc., or may directly display a successful booking page, etc.

[0241] Based on the above technical solution, after receiving a user instruction, the electronic device can obtain response information based on the scene intent identified by the user instruction. The response information includes multiple APIs. Calling these multiple APIs can realize multiple functions. These multiple functions can be used to meet the user's needs under the scene intent. Based on the scene intent, the user is provided with the results of calling multiple APIs to realize different functions. The results of calling the APIs are displayed in a multimodal manner, realizing the recommendation of richer, more comprehensive, convenient and intuitive response information, which can meet user needs and improve the effect of human-computer dialogue.

[0242] In some other embodiments, response information can also be generated based on at least one of the user portrait, the environmental scene information of the electronic device, the scene intent, and the API corresponding to the scene intent. In this embodiment, Figure 15 shows a flow chart of another human-computer interaction method provided by an embodiment of the present application. As shown in Figure 15, the method includes the following steps:

[0243] S1501: The electronic device receives a user instruction.

[0244] S1502: The electronic device determines the scene intention according to the user instruction.

[0245] For the introduction of step S1501 and step S1502, please refer to the relevant introduction of step S801 and step S802 shown in Figure 8 respectively.

[0246] S1503: The electronic device sends at least one of the user portrait and the environmental scene information, and the scene intent to the server. Correspondingly, the server receives at least one of the user portrait and the environmental scene information, and the scene intent from the electronic device.

[0247] It is understood that a user profile can refer to a digital portrait of an actual user abstracted from their behavioral data, which can include one or more user tags. A user tag can be understood as an abstract classification and summary of certain behavioral characteristics of a user.

[0248] For example, if a user frequently uses their electronic device to watch anime after midnight, then "late sleeper" and "anime" could be used as user tags. Another example is if a user frequently uses their electronic device to take a taxi home from work at 10 PM, then "workaholic" could be used as a user tag. Another example is if a user frequently purchases legal books from shopping apps, their major is law, and they frequently read legal papers, then their electronic device could use "law" as a user tag. All of the aforementioned user tags can be used to construct a user profile.

[0249] In some scenarios, when the user does not input any user instructions corresponding to the scenario intent into the electronic device, the electronic device or other device does not have the corresponding historical scenario intent of the user, the response information corresponding to the historical scenario intent, and the user's operation behavior on the response information. It is understood that in the embodiments of the present application, the historical scenario intent can refer to the scenario intent recognized by the electronic device based on the user instructions before generating the user profile.

[0250] In this scenario, in some embodiments, one or more user tags can be generated based on user behavior data. This user behavior data can be the behavior data generated by the electronic device before receiving the user instruction corresponding to the scenario intent. For example, the user behavior data can be generated based on the user's operation record on the electronic device. This user behavior data can be stored in the electronic device or in other devices.

[0251] Optionally, in this embodiment, different user portraits may correspond to different users.

[0252] In some examples, the electronic device may generate one or more user tags based on the user's behavior data. In other examples, other devices may generate one or more user tags based on the user's behavior data, and the electronic device may obtain the one or more user tags from the other devices.

[0253] Furthermore, in some examples, an electronic device may generate a user portrait based on some or all of the above one or more user tags. In other examples, other devices (such as a server) may generate a user portrait based on some or all of the above one or more user tags.

[0254] For example, Figure 16 shows a schematic diagram of a user profile provided in an embodiment of the present application. This user profile includes user tags such as: female, undergraduate, Gemini, sub-healthy, unmarried, lawyer, spicy food enthusiast, online literature, fitness, pet owner, weather-conscious, anime, anime, hot pot, and more. It is understood that the user tags shown in Figure 16 are merely examples, and the user profile may also include other user tags.

[0255] In other embodiments, the user profile may also be a user profile generated by statistically analyzing behavioral data of different users. For example, the electronic device and / or other device may also generate one or more user tags based on the behavioral data of different users. Furthermore, the electronic device and / or other device may also generate a user profile based on some or all of the one or more user tags.

[0256] For example, if 80% of users like to take the high-speed rail, 10% like to take the train, and 10% like to take flights, then "likes to take the high-speed rail" will be used as the user tag. Another example: if 90% of users like to stay in hotels and 20% like to stay in B&Bs, then "likes to stay in hotels" will be used as the user tag. Another example: if 70% of users frequently travel with groups and 30% frequently travel alone, then "likes to travel alone" will be used as the user tag. These user tags can be used to build user profiles.

[0257] Optionally, in this embodiment, different users may correspond to the same user portrait.

[0258] In other scenarios, after a user inputs one or more user commands corresponding to scenario intents into an electronic device, the electronic device or other device may store one or more of the following: historical scenario intents, response information corresponding to the historical scenario intents, and user operations on the response information. It is understood that the user's operations on the response information may include, but are not limited to, records of the user's operations on the results of calling the API contained in the response information.

[0259] Optionally, the historical scenario intention, the response information corresponding to the historical scenario intention, the user's operational behavior on the response information, etc. can be stored in the intent framework system 702 and / or the intent framework system 711 as shown in Figure 7. For example: after the intent framework system 702 sends the scenario intention to the server, and after receiving the response information of the scenario intention from the server, these scenario intentions and the response information corresponding to the scenario intention can be stored. These stored scenario intentions are the historical scenario intentions, and the response information is the response information corresponding to the historical scenario intention. Optionally, the user's operational behavior on the response information can be reported to the intent framework system 702 and / or the intent framework system 711 by the system application and / or the third-party application. For example: after the user operates on the result of calling the API included in the response information, the application corresponding to the API (such as the system application, the third-party application, etc.) can collect the user's operational behavior and report these operational behaviors to the intent framework system 702 and / or the intent framework system 711.

[0260] In this scenario, in some embodiments, the electronic device and / or other device (such as but not limited to a server, etc.) can generate a user portrait based on one or more of the historical scenario intent, the response information corresponding to the historical scenario intent, the user's operational behavior on the response information, etc. For example, the electronic device and / or other device can determine one or more user tags based on one or more of the historical scenario intent, the response information corresponding to the historical scenario intent, the user's operational behavior on the response information, etc., and then generate a user portrait based on part or all of the one or more user tags. In this way, by combining the historical scenario intent, the response information corresponding to the historical scenario intent, the user's operational behavior on the response information (such as the user's operation on the result of calling the API included in the response information, such as: setting the result of the called API, etc.), etc., the generated user portrait can reflect the user's true preferences or needs under one or more scenario intents, and then determine the user's response information under a certain scenario intent based on the user portrait that can reflect the user's true preferences or needs under one or more scenario intents, so that the response information can better meet the user's needs under the scenario intent, and improve the accuracy and pertinence of the response information.

[0261] For example, for travel intent, if a user frequently sets an alarm for after 7:00 AM, frequently chooses first class, frequently flies on xx airline, frequently books B&Bs, frequently travels as a couple, and most frequently books sea view rooms, then user tags might include "dislikes waking up before 7:00 AM," "likes comfortable seats," "likes flying on xx airline," "likes B&Bs," "likes traveling as a couple," and "likes sea view rooms." Another example: for shopping and consumption intent, if a user frequently buys jade, frequently watches videos of pet dogs, spends around 1,000 yuan per purchase, and frequently visits xx mall, then user tags might include "likes jade," "likes pet dogs," "spends around 1,000 yuan per purchase," and "likes visiting xx mall." Another example: for work and career intent, if a user frequently searches for programming information, frequently applies for front-end development positions, and their job applications are all located in Beijing, and frequently purchases computer-related books, then user tags might include "good at programming," "likes front-end development positions," and "intended work location is Beijing." These user tags can also be used to construct user profiles.

[0262] For example, Figure 17 shows a schematic diagram of another user profile provided by an embodiment of the present application. The user tags included in this user profile include: undergraduate, good at programming, like homestay, etc., which are not introduced one by one here. It is understandable that the user tags shown in Figure 17 may only be some examples, and the user profile may also include other user tags.

[0263] Optionally, in this embodiment, the electronic device and / or other devices may also determine more user tags in combination with other user behavior data to further determine the user profile, which may make the obtained user profile more comprehensive and accurate.

[0264] Optionally, in this embodiment, different users can correspond to different user profiles. Thus, for the same scenario intent, different users will correspond to different user profiles, and the response information output based on these different user profiles will also be different. In other words, different response information recommendations can be made for different users, flexibly addressing the needs of various users and avoiding the homogeneity of generated response information.

[0265] Optionally, since the user's behavior may be different at different times, that is, the user may have different preferences at different times, the above-mentioned user tags can be updated regularly or dynamically. Correspondingly, the user profile composed of user tags can also be updated regularly or dynamically. This application does not limit the specific update timing and update method. In this way, by updating the user profile according to the user's current preferences, a more accurate user profile can be obtained, and then more accurate response information can be obtained when determining the response information based on the user profile.

[0266] In some embodiments, when a user portrait such as that shown in FIG. 17 does not currently exist, the user portrait described in step S1503 may be a user portrait such as that shown in FIG. 16 . When a user portrait such as that shown in FIG. 17 currently exists, the user portrait described in step S1503 may be a user portrait such as that shown in FIG. 17 . Of course, when user portraits such as those shown in FIG. 16 and FIG. 17 currently exist at the same time, a user portrait such as that shown in FIG. 17 may also be used. Since a user portrait such as that shown in FIG. 17 can better represent the user's real needs, using this user portrait can make the recommended response information more accurate.

[0267] For the introduction of the environmental scene information described in step S1503, please refer to the introduction of the environmental scene information described in step S806 shown in Figure 8.

[0268] For example, the electronic device may also use the intent framework system 702 shown in FIG7 to send at least one of the user portrait and environmental scene information, as well as the scene intent, to the server. Correspondingly, the server may also use the intent framework system 711 shown in FIG7 to receive at least one of the user portrait and environmental scene information, as well as the scene intent, from the electronic device.

[0269] In a specific embodiment, the intent framework system 702 may use a preset data structure to send at least one of a user portrait and environmental scene information, as well as a scene intent to the server. That is, the intent framework system 702 may send a first message containing at least one of a user portrait and environmental scene information, as well as a scene intent to the server, and the data structure of the first information may use a preset data structure. Exemplarily, the preset data structure may include the following multiple information: EnvIntents: representing scene intent; EnvParams: representing environmental scene information; UserPortrait: representing user portrait information (such as user tags contained in the user portrait, etc.).

[0270] Optionally, the preset data structure may further include a CheckCode representing a security check code. The security check code may be used by the server to verify messages from the electronic device, including at least one of a user profile and environmental scenario information, as well as the scenario intent. Upon verification, for example, if the security check code is correct, the server executes step S1504 and subsequent steps. In this way, the inclusion of a security check code in the data structure ensures the security of information sent from the electronic device to the server and prevents tampering with the information.

[0271] Optionally, the data structure may also include DeviceType: characterizing the type of electronic device. Exemplarily, the type of electronic device may include but is not limited to various types such as mobile phones, tablets, watches, etc. Optionally, the type of electronic device may also be combined with the information described in the various embodiments of this application to jointly determine the response information, such as including but not limited to the API included in the response information, the multimodal display method of the result of calling the API, etc. It can be understood that for multiple different electronic devices, they may have the same account (such as a Huawei account). For example: for the same user's mobile phone, tablet, watch, etc., the corresponding user account may be the same. Therefore, some data of these different electronic devices may be shared. In order to prevent the subsequent server from determining the same response information for these different electronic devices, that is, in order to output different response information on different electronic devices with the same account, the electronic device can send its own device type to the server to further improve the output effect of the response information.

[0272] Combined with the data structure described above, taking the user instruction "I am going to travel to Beijing at 8:30 tomorrow morning, please give me some travel suggestions" as an example, the information sent by the intent framework system to the server may include the following: CheckCode: 123321; EnvIntents: travel intent, "I am going to travel to Beijing at 8 o'clock tomorrow morning, please help me arrange it"; EnvParams: Environment: Quiet, Network: Wi-Fi, Bluetooth: On...; UserPortrait: "Don't like to get up before 7:00", "Like comfortable seats", "Like jade", "Like pet dogs"...DeviceType: mobile phone.

[0273] It is understood that in step S1503, the example is that the electronic device sends the user portrait and environmental scene information to the server. In other embodiments, the server may also obtain the user portrait and environmental scene information from devices other than the electronic device, or the server may determine the user portrait and environmental scene information itself, and this embodiment of the application does not impose any restrictions on this.

[0274] For further description of step S1503 , please refer to the description of step S803 shown in FIG. 8 .

[0275] S1504. The server generates response information based on at least one of the user portrait and environmental scene information, as well as the scene intention.

[0276] In some embodiments, the server may generate response information based on at least one of the user portrait and environmental scene information, as well as the scene intent and the API corresponding to the scene intent. For example, the server may also generate response information based on at least one of the user portrait and environmental scene information, as well as the scene intent and the API corresponding to the scene intent through the response information generation module 712 described in FIG7 . For an introduction to the API corresponding to the scene intent, please refer to the relevant introduction of step S804 described in FIG8 .

[0277] In a specific embodiment, the server may also input at least one of the user profile and environmental scenario information, as well as the scenario intent and the API corresponding to the scenario intent, into the AI ​​model, and output response information through the AI ​​model. In this way, by combining the user profile and at least one of the environmental scenario information, as well as the scenario intent and the API corresponding to the scenario intent to determine the response information, a richer and more comprehensive response information can be obtained, which can meet user needs and improve the effect of human-computer dialogue.

[0278] In some specific implementations, the server may use an intent framework system 711 such as that shown in FIG7 to input at least one of the received user profile and environmental scenario information, as well as the scenario intent and the API corresponding to the scenario intent, into the AI ​​model. The server may also send the response information output by the AI ​​model to the intent framework system 711 shown in FIG7.

[0279] Optionally, in this specific implementation, before the intent framework system inputs the user portrait, scenario intent, and the API corresponding to the scenario into the AI ​​model, the intent framework system can match the user tags contained in the user portrait with the API corresponding to the scenario intent, that is, determine which user tags are associated with which APIs. Of course, this association operation can also be performed by the AI ​​model, and the embodiments of the present application are not limited to this. In this way, when displaying a certain API, the specific display effect of the API can be determined based on the user tag associated with the API, so that the response information can be matched with the user portrait, that is, it corresponds to the user's preferences, and the response information can better meet the user's needs.

[0280] For example, as shown in Figure 18, taking the travel scenario intent as an example, the user tags associated with the alarm setting API may include "don't like to get up before 7 o'clock", etc., the user tags associated with the flight booking API may include "like comfortable seats", "often choose first class", "like to take flights of xx airline", etc., and the user tags associated with the accommodation booking API may include "like homestays", "like to travel for two people", "like sea view rooms", etc.

[0281] In some embodiments, the response information corresponds to the user profile. For different user profiles, the response information determined for the same scenario intent may be different, including, but not limited to, different APIs included in the response information. For example, assuming a scenario intent of travel, such as "I'm traveling to Beijing at 8:30 tomorrow morning. Please give me some travel advice," as shown in Figure 19, for user profile A, the response information determined includes APIs for booking high-speed rail tickets, booking star-rated hotels, and setting an alarm for 6:00 AM. Travel reminders include reminders related to bringing keys. For user profile B, the response information determined includes APIs for booking plane tickets, booking a homestay, and setting an alarm for 7:00 AM. Travel reminders include health-related reminders. Thus, for the same scenario intent, different users may correspond to different user profiles, and corresponding response information can be output based on these different user profiles. In other words, different response information can be recommended for different users, flexibly addressing the needs of various users and preventing the generation of homogenized response information.

[0282] In some embodiments, the response information also corresponds to the environmental scenario information. For different environmental scenario information, the response information determined under the same scenario intent may be different, such as, but not limited to, different APIs included in the response information, different multimodal presentation methods of the APIs included in the response information, etc. For example, assuming the scenario intent is a travel intent, the travel intent may be, for example, "I'm traveling to Beijing at 8:30 tomorrow morning. Please give me some travel advice." As shown in FIG20 , for environmental scenario information A, for example, if environmental scenario information A indicates that the electronic device is in a noisy environment, the determined response information includes a hotel picture API, a hotel text description API, an alarm setting API, a weather card description API, etc. For environmental scenario information B, for example, if environmental scenario information B indicates that the electronic device is in a quiet environment, the determined response information may include a hotel video API, a travel blogger video blog API, an alarm setting API, a weather voice broadcast API, etc.

[0283] In this way, under different environmental scene information, the user's preferences may be different, such as: including but not limited to the volume and interface parameters during API display, and then based on different environmental scene information, the appropriate response information and the display method of the response information are recommended, so that the output response information corresponds to the current environmental scene information, which can meet the user's preferences in different environments under the same scene intention, and achieve a "thousand faces" display effect.

[0284] For further description of step S1504 , please refer to the description of step S804 shown in FIG. 8 .

[0285] S1505: The server sends a response message to the electronic device. Correspondingly, the electronic device receives the response message from the server.

[0286] For example, the server may send response information to the electronic device through the intent framework system 711 shown in Figure 7. Correspondingly, the electronic device may receive response information from the electronic device through the intent framework system 702 shown in Figure 7.

[0287] In some embodiments, the intent framework system 711 may also use a preset data structure to send response information to the electronic device. For example, the preset data structure may include the following information: Answers: response information representing the scenario intent; APIGroup: representing the API to be displayed; PlayMethod: representing the multimodal display method.

[0288] Optionally, the preset data structure may also include a CheckCode representing a security check code. This security check code can be used by the electronic device to verify the response information from the server. If verification is successful, for example, if the security check code is correct, the electronic device proceeds to step S1506. In this way, the inclusion of the security check code in the data structure ensures the security of the information sent from the server to the electronic device and prevents tampering with the information.

[0289] Optionally, to avoid the server failing to update the content of the API interface on the electronic device side, the electronic device can also verify the validity of the API contained in the response information after receiving the response information. After determining that it is a valid API, the electronic device can successfully call the API and then output the result of calling the API, so that subsequent users can perform some operations on the results of calling the API, etc., to ensure user experience.

[0290] S1506. The electronic device outputs response information.

[0291] In some embodiments, the electronic device may output the result of calling the API included in the response information according to the multimodal presentation method included in the response information.

[0292] Exemplarily, in combination with the user portrait shown in Figure 19, Figure 21 shows an interface schematic diagram of a response information output by an electronic device provided in an embodiment of the present application. Exemplarily, (1) in Figure 21 is an interface schematic diagram of outputting response information generated based on user portrait A. As shown in (1) in Figure 21, the electronic device can display a response interface 2100, wherein the response interface 2100 includes a card 2102, and the card 2102 can be used to display the result of calling the API included in the response information. Optionally, the response interface 2100 can also include a user instruction 2101.

[0293] Card 2101 includes the result 2103 of calling the alarm setting API, the result 2104 of calling the high-speed rail ticket booking API, and the result 2105 of calling the accommodation booking API. It is understood that the alarm setting API may be an API for implementing the alarm setting function of an application, the high-speed rail ticket booking API may be an API for implementing the high-speed rail ticket booking function of an application, and the accommodation booking API may be an API for implementing the accommodation booking function of an application.

[0294] The function of setting an alarm can be realized by calling the result 2103 of the setting alarm API. The function of booking a high-speed rail ticket can be realized by calling the result 2104 of the booking high-speed rail ticket API. The function of booking an accommodation can be realized by calling the result 2105 of the booking accommodation API.

[0295] Optionally, card 2102 may also include texts such as "Because the weather in Beijing is hot and it may rain, it is recommended that you bring summer clothes and rain gear to cope with different weather conditions. When going out, remember to bring your keys." to remind the user of the weather conditions and to bring messages, and / or fixed-format texts such as "OK" and "The following are travel planning suggestions."

[0296] FIG21 (2) is a schematic diagram of an interface for outputting response information generated based on user profile B. As shown in FIG21 (2), the electronic device may display a response interface 2110, wherein the response interface 2110 includes a card 2112, which may be used to display the result of calling the API included in the response information. Optionally, the response interface 2110 may also include a user instruction 2111.

[0297] For example, card 2111 includes the result of calling the alarm setting API 2113, the result of calling the flight booking API 2114, and the result of calling the accommodation booking API 2115. Please refer to the above description for the introduction of the alarm setting API, flight booking API, and accommodation booking API.

[0298] The function of setting an alarm can be realized by calling the result 2113 of the setting alarm API. The function of booking a flight ticket can be realized by calling the result 2114 of the booking flight ticket API. The function of booking an accommodation can be realized by calling the result 2115 of the booking accommodation API.

[0299] For example, if the user tags included in user profile A reflect that the user does not like to get up later than 6 o'clock, prefers to take the high-speed rail, likes to stay in a homestay, etc., then as shown in (1) in Figure 21, the result 2103 of calling the setting alarm API included in the response information recommends an alarm at 6 o'clock for the user, the result 2104 of calling the high-speed rail ticket API is recommended to the user, and the result 2105 of calling the booking accommodation API recommends a homestay for the user, etc.

[0300] If the tags included in user profile B reflect that the user does not like to get up earlier than 7 o'clock, prefers to take a plane, likes comfortable hotels, etc., then as shown in (2) in Figure 21, the result 2113 of calling the setting alarm API included in the response information recommends a 7 o'clock alarm to the user, the result 2114 of calling the booking flight API is recommended to the user, and the result 2115 of calling the booking accommodation API recommends a star-rated hotel to the user.

[0301] Optionally, under different user portraits, the user's display habits for the results of calling the API, such as but not limited to habits for volume adjustment, may also be different. At this time, the response information may also include indication information, which can be used to indicate the volume of the electronic device. The volume of the electronic device contained in the indication information may also be different. For example: when the electronic device displays the response interface shown in (1) in Figure 21, the volume of the electronic device can be 70%. When the electronic device displays the response information shown in (2) in Figure 21, the volume of the electronic device can be 30%. In this way, the electronic device adjusts the volume when the electronic device outputs the response information according to the user portrait, so that the volume of the electronic device can meet the actual habits of the user.

[0302] Of course, under the same user profile, the user's display habits for the results of calling the API, such as but not limited to volume adjustment, may also vary for different environmental scenario information. In this case, under the same user profile but different environmental scenario information, the volume level of the electronic device included in the indication information in the response information may also be different.

[0303] In this way, generating response information based on user portraits can make the recommendation of response information more meet the actual needs of users, and can implement targeted recommendation of response information for different user portraits, making the response information richer.

[0304] Exemplarily, FIG22 shows a schematic diagram of an interface of response information output by another electronic device provided in an embodiment of the present application. Exemplarily, when the environmental scene information of the electronic device indicates that the electronic device is in a noisy environment, the response information output by the electronic device may be as shown in (1) in FIG22. The electronic device may display a response interface 2200, wherein the response interface 2200 includes a card 2202, and the card 2202 may be used to display the result of calling the API included in the response information. Optionally, the response interface 2200 may also include a user instruction 2201.

[0305] For example, in card 2201, there are the results 2203 of calling the alarm setting API, the results 2204 of calling the flight booking API, and the results 2205 of calling the accommodation booking API. Due to the noisy environment, the user may not be able to hear or focus on the video or audio content, and the user may prefer to read text or view pictures. In this case, as shown in (1) of Figure 22, the results 2203 of calling the alarm setting API, the results 2204 of calling the flight booking API, and the results 2205 of calling the accommodation booking API are displayed using text and pictures.

[0306] For example, when the environmental scene information of the electronic device indicates that the electronic device is in a quiet environment, the response information output by the electronic device may be as shown in (2) in Figure 22. The electronic device may display a response interface 2210, wherein the response interface 2210 includes a card 2212, and the card 2212 may be used to display the result of calling the API included in the response information. Optionally, the response interface 2210 may also include a user instruction 2211.

[0307] For example, in card 2212, the result 2203 of calling the alarm setting API, the result 2204 of calling the flight booking API, and the result 2205 of calling the accommodation booking API are included. Due to the quiet environment, the user may be more inclined to watch videos or listen to audio, etc. At this time, as shown in (1) in Figure 22, the result 2213 of calling the alarm setting API, the result 2214 of calling the flight booking API, and the result 2215 of calling the accommodation booking API are displayed using audio and video. For example, the electronic device can voice broadcast the content of the result 2213 of calling the alarm setting API, "The alarm has been set for you at 7 o'clock in the morning", and / or voice broadcast the flight information included in the result 2214 of calling the flight booking API, and / or display videos related to homestays in the result 2215 of calling the accommodation booking API, such as but not limited to homestay recommendation videos, homestay booking strategies, etc.

[0308] For the introduction of other contents in Figures 21 and 22, please refer to the introduction of the corresponding contents shown in Figure 13.

[0309] In this way, response information is generated according to the environmental scene information, that is, the output response information can be adjusted according to the environmental scene information, which can realize the recommendation of response information suitable for the current environmental scene information, so that the response information can meet the user's preferences under the current environmental scene information.

[0310] Based on the above technical solution, response information is recommended by combining various information such as scene intent, API corresponding to scene intent, user portrait, environmental scene information, etc., so that the response information can correspond to the user portrait and environmental scene information, and can make the response information more personalized and targeted while meeting the user's needs under the input scene intent, so as to meet the user's actual preferences.

[0311] In other embodiments, the server may also combine various notification messages, user operations on the response information, etc. to assist in determining the response information. For example, the server may input various notification messages, user operations on the response information, and the information described in the above embodiments into an AI model, and output the response information through the AI ​​model. In this way, the server also generates response information based on various notification messages, user operations on the response information, etc., which can make the response information richer and more comprehensive, further improving the recommendation results of the response information.

[0312] For example, notification messages may include, but are not limited to, weather notifications, flight delay notifications, train delay notifications, product discount notifications, traffic congestion notifications, and other notification messages. These notification messages may be reported by third-party applications or system applications to the intent framework system 702 and / or intent framework system 711 shown in FIG7 . Of course, to ensure user privacy, the various notification messages obtained by the server, the user's operational behavior in response to the information, and other information may be desensitized data.

[0313] It can be understood that the various operations performed by the server in the embodiment of the present application can also be performed electronically, and the embodiment of the present application does not limit this.

[0314] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of method. It is understandable that, in order to realize the above functions, the device includes a hardware structure and / or software module corresponding to the execution of each function. In combination with the units and algorithm steps of each example described in the embodiment disclosed in this application, the embodiment of the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer-driven hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to realize the described functions for each specific application, but such implementation should not be considered to exceed the scope of the technical solution of the embodiment of the present application.

[0315] As another possible division method, FIG23 is a schematic diagram of the structure of a device 2300 provided in an embodiment of the present application, which can be used to implement the methods described in the above method embodiments. Exemplarily, the device 2300 may include: a processing unit 2301 and a communication unit 2302.

[0316] In one possible example, taking device 2300 as an electronic device, processing unit 2301 is configured to support device 2300 in executing steps S801, S802, and S806 in FIG8 . And / or, processing unit 2301 is configured to support device 2300 in executing steps S1501, S1502, and S1506 in FIG15 . And / or, processing unit 2301 is further configured to support device 2300 in executing other steps performed by the electronic device in the embodiments of the present application.

[0317] Communication unit 2302 is configured to support device 2300 in executing steps S803 and S805 in FIG. 8 . And / or, communication unit 2302 is configured to support device 2300 in executing steps S1503 and S1505 in FIG. 15 . And / or, communication unit 2302 is further configured to support device 2300 in executing other steps performed by the electronic device in the embodiments of the present application.

[0318] In another possible example, taking the device 2300 as a server, the processing unit 2301 is configured to support the device 2300 in executing step S804 in FIG. 8 . And / or, the processing unit 2301 is configured to support the device 2300 in executing step S1504 in FIG. 15 . And / or, the processing unit 2301 is further configured to support the device 2300 in executing other steps performed by the server in the embodiments of the present application.

[0319] Communication unit 2302 is configured to support device 2300 in executing steps S803 and S805 in FIG. 8 . And / or, communication unit 2302 is configured to support device 2300 in executing steps S1503 and S1505 in FIG. 15 . And / or, communication unit 2302 is further configured to support device 2300 in executing other steps performed by the server in the embodiments of the present application.

[0320] Optionally, the device 2300 shown in Figure 23 may further include a storage unit 2303 storing a program or instruction. When the processing unit 2301 executes the program or instruction, the device 2300 shown in Figure 23 may execute the method executed by the electronic device or server in the above embodiment.

[0321] Optionally, the device 2300 shown in Figure 23 may further include an output unit (not shown). The output unit may be used to support the device 2300 to output various information. Optionally, the output unit may be specifically implemented as a display unit, an audio output unit, etc.

[0322] The technical effects of the device 2300 shown in FIG23 can be referenced to the technical effects of the methods described in the above embodiments and will not be further described here. The processing unit 2301 involved in the device 2300 shown in FIG23 can be implemented by a processor or processor-related circuit components, such as a processor or processing module. The communication unit 2302 can be implemented by a transceiver or transceiver-related circuit components, such as a transceiver or transceiver module. The output unit can be implemented by related components such as a display screen and an audio module.

[0323] An embodiment of the present application also provides a chip system, as shown in Figure 24, which includes at least one processor 2401 and at least one interface circuit 2402. The processor 2401 and the interface circuit 2402 can be interconnected via lines. For example, the interface circuit 2402 can be used to receive signals from other devices. For another example, the interface circuit 2402 can be used to send signals to other devices (such as the processor 2401). Exemplarily, the interface circuit 2402 can read instructions stored in the memory and send the instructions to the processor 2401. When the instructions are executed by the processor 2401, the device can execute the various steps performed by the electronic device or server in the above embodiments. Of course, the chip system can also include other discrete devices, which is not specifically limited in the embodiment of the present application.

[0324] Optionally, there may be one or more processors in the chip system. The processor may be implemented in hardware or software. When implemented in hardware, the processor may be a logic circuit, an integrated circuit, etc. When implemented in software, the processor may be a general-purpose processor implemented by reading software code stored in a memory.

[0325] Optionally, the memory in the chip system may be one or more memories. The memory may be integrated with the processor or may be provided separately from the processor, which is not limited in this application. For example, the memory may be a non-transient processor, such as a read-only memory (ROM), which may be integrated with the processor on the same chip or provided on different chips. This application does not specifically limit the type of memory or the configuration of the memory and the processor.

[0326] Exemplarily, the chip system can be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD) or other integrated chips.

[0327] It should be understood that each step in the above method embodiment can be completed by hardware integrated logic circuits in a processor or by software instructions. The method steps disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware processor, or by a combination of hardware and software modules in a processor.

[0328] An embodiment of the present application further provides a computer storage medium, in which computer instructions are stored. When the computer instructions are executed on a device, the device executes the method described in the above method embodiment.

[0329] An embodiment of the present application provides a computer program product, which includes: a computer program or instructions, which, when executed on a computer, causes the computer to execute the method described in the above method embodiment.

[0330] In addition, an embodiment of the present application also provides a device, which can specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor can execute the computer-executable instructions stored in the memory to enable the device to execute the methods in the above-mentioned method embodiments.

[0331] An embodiment of the present application also provides a communication system, including an electronic device and a server, wherein the electronic device is used to execute the method executed by the electronic device in the above method embodiment, and the server is used to execute the method executed by the server in the above method embodiment.

[0332] Among them, the equipment, computer storage medium, communication system, computer program product or chip provided in this embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0333] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0334] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The various embodiments can be combined with each other or referenced to each other without conflict. The device embodiments described above are merely schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0335] Units described as separate components may or may not be physically separate, and components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0336] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0337] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0338] The above content is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A human-computer interaction method, characterized in that: Applied to electronic equipment, the method comprises: Receive user instructions; In a case where the user instruction represents a first scenario intention, acquiring a plurality of target application programming interfaces APIs according to the first scenario intention, the plurality of target APIs corresponding to the first scenario intention; Output the results of calling the multiple target APIs, and the results of calling the multiple target APIs are used to realize the user needs corresponding to the first scenario intention.

2. The method according to claim 1, characterized in that The acquiring the multiple target APIs according to the first scenario intent includes: The multiple target APIs are acquired according to the user portrait and / or the environmental scene information of the electronic device, and the first scene intention, wherein the environmental scene information of the electronic device is used to characterize the environmental conditions around the electronic device and at least one of the states of the electronic device.

3. The method according to claim 2, characterized in that The acquiring the multiple target APIs according to the user portrait and / or the environmental scene information of the electronic device and the first scene intention includes: Sending the user portrait and / or the environmental scene information of the electronic device, and the first scene intention to a server; The plurality of target APIs are received from the server.

4. The method according to any one of claims 1 to 3, characterized in that: The output results of calling the multiple target APIs include: The results of calling the multiple target APIs are output in a multimodal display manner, wherein the multimodal display manner is a method of using multiple display APIs in text, picture, audio, and video.

5. The method according to claim 4, characterized in that Before outputting the results of calling the multiple target APIs in a multimodal display manner of the results of calling the multiple target APIs, the method further includes: A multimodal presentation of results of calling the plurality of target APIs is received from the server.

6. The method according to claim 4 or 5, characterized in that: The multimodal display method corresponds to the environmental scene information of the electronic device.

7. The method according to claim 6, characterized in that If the environmental scene information of the electronic device indicates that the electronic device is in a first environmental state, the multimodal display mode is the first display mode; If the environmental scene information of the electronic device indicates that the electronic device is in a second environmental state, and the second environmental state is different from the first environmental state, then the multimodal display mode is a second display mode, and the second display mode is different from the first display mode.

8. The method according to claim 7, characterized in that The first environmental condition is a quiet environment, and the first display method is to use video and / or audio to display the results of calling the multiple target APIs; the second environmental condition is a noisy environment, and the second display method is to use text and / or pictures to display the results of calling the multiple target APIs.

9. The method according to any one of claims 2 to 8, characterized in that: The multiple target APIs correspond to the user portraits.

10. The method according to claim 9, characterized in that If the user portrait is a first user portrait, the multiple target APIs are multiple first APIs; If the user portrait is a second user portrait, and the second user portrait is different from the first user portrait, then the multiple target APIs are multiple second APIs, and the multiple second APIs and the multiple first APIs include different APIs.

11. The method according to any one of claims 2 to 10, characterized in that: Before receiving the user instruction, the method further includes: According to the second scenario intention, the API corresponding to the second scenario intention, and the user portrait are generated by multiple operation records of the user on the results of calling the API corresponding to the second scenario intention, where the second scenario intention is the scenario intention obtained before receiving the user instruction.

12. The method according to any one of claims 1 to 11, characterized in that: Before acquiring multiple target APIs according to the first scenario intent, the method further includes: The first scene intention is determined according to a preset scene intention set and the user instruction, and the preset scene intention set includes multiple scene intentions.

13. The method according to claim 3, characterized in that The user portrait and / or the environmental scene information of the electronic device, and the first scene intention are included in the first information. The first information also includes a security verification code and at least one of the type of the electronic device. The security verification code is used to verify the first information, and the type of the electronic device is used to determine the multiple target APIs.

14. A human-computer interaction method, characterized in that: Applied to a server, the method comprises: Receiving first information from an electronic device, wherein the first information includes a first scenario intention; Generate second information according to the first information and the API corresponding to the first scenario intent, wherein the second information includes a plurality of target APIs, the plurality of target APIs correspond to the first scenario intent, and the plurality of target APIs are used to implement user needs corresponding to the first scenario intent; The second information is sent to the electronic device.

15. The method according to claim 14, characterized in that The first information also includes a user portrait and / or environmental scene information of the electronic device, wherein the environmental scene information of the electronic device is used to characterize at least one of the environmental conditions around the electronic device and the state of the electronic device.

16. The method according to claim 15, characterized in that The second information also includes a multimodal display method of the results of calling the multiple target APIs, and the multimodal display method is a method of using multiple display APIs in text, pictures, audio, and video.

17. The method according to claim 16, characterized in that The multimodal display method corresponds to the environmental scene information of the electronic device.

18. The method according to claim 17, characterized in that If the environmental scene information of the electronic device indicates that the electronic device is in a first environmental state, the multimodal display mode is the first display mode; If the environmental scene information of the electronic device indicates that the electronic device is in a second environmental state, and the second environmental state is different from the first environmental state, then the multimodal display mode is a second display mode, and the second display mode is different from the first display mode.

19. The method according to claim 18, characterized in that The first environmental condition is a quiet environment, and the first display method is to use video and / or audio to display the results of calling the multiple target APIs; the second environmental condition is a noisy environment, and the second display method is to use text and / or pictures to display the results of calling the multiple target APIs.

20. The method according to any one of claims 15 to 19, characterized in that The multiple target APIs correspond to the user profiles.

21. The method according to claim 20, characterized in that If the user portrait is a first user portrait, the multiple target APIs are multiple first APIs; If the user portrait is a second user portrait, and the second user portrait is different from the first user portrait, then the multiple target APIs are multiple second APIs, and the multiple second APIs and the multiple first APIs include different APIs.

22. The method according to any one of claims 14 to 21, characterized in that Before generating second information according to the first information and the API corresponding to the first scenario intention, the method further includes: According to a preset scene intent set, the API corresponding to the preset scene intent set, the first scene intent determines the API corresponding to the first scene intent, wherein the preset scene intent set includes multiple scene intents, and the API corresponding to the preset scene intent set is the API corresponding to each scene intent in the multiple scene intents.

23. The method according to any one of claims 14 to 22, characterized in that The generating second information according to the first information and an API corresponding to the first scene intention includes: The first information and the API corresponding to the first scene intention are input into a preset model, and the second information is output through the preset model.

24. The method according to any one of claims 14 to 23, characterized in that The first information also includes at least one of a security verification code and a type of the electronic device, the security verification code is used to verify the first information, and the type of the electronic device is used to determine the multiple target APIs.

25. An electronic device, characterized in that: The electronic device comprises a processor and a memory, wherein the memory is coupled to the processor, the memory is used to store computer program code, the computer program code comprises computer instructions, and the processor reads the computer instructions from the memory so that the electronic device executes the method as described in any one of claims 1 to 13.

26. A server, characterized in that: It includes a processor, a memory and a communication interface, the memory and the communication interface are coupled to the processor, the communication interface is used to communicate with other devices, the memory is used to store computer program code, the computer program code includes computer instructions, and the processor reads the computer instructions from the memory so that the server executes the method as described in any one of claims 14-24.

27. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a computer program, which, when executed on an electronic device, enables the electronic device to execute the method as described in any one of claims 1 to 13; or, when executed on a server, enables the server to execute the method as described in any one of claims 14 to 24.

28. A computer program product, characterized in that The computer program product comprises: a computer program or instructions, which, when executed on a computer, enables the computer to execute the method according to any one of claims 1 to 13, or enables the computer to execute the method according to any one of claims 14 to 24.

29. A communication system, characterized in that: It comprises an electronic device and a server, wherein the electronic device is used to execute the method as claimed in any one of claims 1 to 13, and the server is used to execute the method as claimed in any one of claims 14 to 24.

Images