Human-computer interaction method and electronic equipment

Abstract

The invention discloses a man-machine interaction method and electronic equipment, and relates to the field of AI. The method is applied to the electronic equipment and comprises the steps that a user instruction is received, multiple target APIs are obtained according to a scene intention in a scene where the user instruction represents the scene intention, and the multiple target APIs correspond to the scene intention. And outputting a result of calling the plurality of target APIs, wherein the result of calling the plurality of APIs can be used for realizing a user demand corresponding to the scene intention. Thus, the electronic equipment can directly output the API calling result meeting the user requirement under the scene intention based on the scene intention of the user, the API is the direct exposure of the application function, the more abundant, comprehensive and convenient response result is achieved by directly outputting the result of calling the multiple APIs, the user requirement can be met, and the man-machine conversation effect is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence (AI), and in particular to a human-computer interaction method and an electronic device. Background Art

[0002] At present, with the rapid development of AI technology, in human-computer interaction scenarios, electronic devices can identify user intentions based on user input instructions (such as questions, or questions), and respond to user instructions based on AI technology and the identified user intentions. For example, when an electronic device receives a question input by a user, the electronic device will directly display corresponding text, cards, etc. based on the semantics of the user intention identified based on the question as an answer to the user's question. This response method is too single and cannot meet user needs, resulting in poor results in human-computer dialogue. Summary of the invention

[0003] The present application provides a human-computer interaction method and electronic device, which can provide richer, more comprehensive and convenient response results based on the user's scenario intention, meet user needs and improve the effect of human-computer dialogue.

[0004] In order to achieve the above objectives, this application adopts the following technical solutions:

[0005] In a first aspect, a human-computer interaction method is provided, which is applied to an electronic device, and the method comprises: receiving a user instruction; when the user instruction represents a first scenario intention, obtaining multiple target application programming interfaces API according to the first scenario intention, and the multiple target APIs correspond to the aforementioned first scenario intention; and finally outputting the results of calling the multiple target APIs, wherein the results of calling the multiple target APIs are used to realize user needs corresponding to the first scenario intention.

[0006] Based on the above technical solution, when a user instruction representing a scenario intention is received, multiple corresponding APIs are obtained according to the user's scenario intention, and the results of calling these multiple APIs are output. The results of calling these multiple APIs can be used to realize the user needs corresponding to the user's scenario intention. In this way, in response to the user's scenario intention, the result of calling the API is directly output to the user, and the API is a direct exposure of the application function. The user can start a certain function of the application by directly operating the result of calling the API, making the response information more convenient. In addition, outputting the results of calling multiple APIs makes the response information richer and more comprehensive, which can meet user needs and improve the effect of human-computer dialogue.

[0007] In a possible design, multiple target APIs are obtained according to the first scene intent, including: obtaining multiple target APIs according to the user portrait and / or the environmental scene information of the electronic device, and the first scene intent, wherein the environmental scene information of the electronic device can be used to characterize at least one of the environmental conditions around the electronic device and the state of the electronic device. Based on this design, multiple APIs under the scene intent are obtained by combining various information such as the scene intent, the user portrait, and the environmental scene information of the electronic device, so that the obtained APIs correspond to the user portrait, the environmental scene information of the electronic device, etc. The user portrait, the environmental scene information, etc. can reflect the user's preferences, habits, etc., and thus the obtained APIs can be more in line with the user's preferences and habits. The response result, that is, the output result of calling these multiple APIs, makes the response result more personalized and targeted while meeting the user's needs under the scene intent, and meets the user's actual preferences.

[0008] In one possible design, multiple target APIs are obtained based on the user portrait and / or the environmental scene information of the electronic device, and the first scene intent, including: sending the user portrait and / or the environmental scene information of the electronic device, and the first scene intent to the server; receiving multiple target APIs from the server. Based on this design, the server determines multiple APIs under the scene intent based on information such as the user portrait, the environmental scene information of the electronic device, and the scene intent, and the computing power of the server is relatively high. In this way, a more accurate API corresponding to the scene intent can be obtained, and the response result is also more accurate.

[0009] In one possible design, the results of calling multiple target APIs are output, including: outputting the results of calling multiple target APIs in a multimodal display manner, where the multimodal display manner is a method of displaying multiple APIs in text, pictures, audio, and video. Based on this design, outputting the results of calling multiple APIs in a multimodal display manner can make the response results richer and more intuitive.

[0010] In one possible design, before outputting the results of calling multiple target APIs in a multimodal display manner, the method further includes: receiving a multimodal display manner of the results of calling multiple target APIs from a server. Based on this design, the electronic device can directly receive a multimodal display manner of the results of calling the APIs from the server, and then display the results of the called APIs in accordance with the multimodal display manner. There is no need to perform an additional operation to determine the display manner of the results of calling the APIs, which can save power consumption of the electronic device.

[0011] In one possible design, the multimodal display method corresponds to the environmental scene information of the electronic device. The environmental scene information of the electronic device characterizes the environmental conditions of the electronic device, the status of the electronic device, etc. Under different environmental conditions, the user may want to display the results of calling the API in different ways. For example: if the electronic device is in a noisy environment, the user may be more inclined to read text or view pictures because the user may not be able to hear or focus on the video or audio content. When the electronic device is in a quiet environment, the user may be more inclined to watch videos or listen to audio content. In this way, the multimodal display method corresponds to the environmental scene information of the electronic device. The display method of the results of calling the API can be adapted to the current environment, the status of the electronic device, etc. It can meet the preferences of users in different environments under the same scene intention, achieve a "thousands of machines, thousands of faces" display effect, and better meet the actual needs of users.

[0012] In one possible design, if the environmental scene information of the electronic device represents that the electronic device is in a first environmental condition, the multimodal display method is the first display method; if the environmental scene information of the electronic device represents that the electronic device is in a second environmental condition, and the second environmental condition is different from the first environmental condition, the multimodal display method is the second display method, and the second display method is different from the first display method. Based on this design, for the same scene intent, different multimodal display methods are used to display the results of calling multiple APIs obtained under the scene intent under different environmental conditions, so that different display effects can be achieved under different environmental conditions, satisfying the user's preferences under different environmental conditions.

[0013] In one possible design, the first environmental condition is a quiet environment, and the first display method is to use video and / or audio to display the results of calling multiple target APIs; the second environmental condition is a noisy environment, and the second display method is to use text and / or pictures to display the results of calling multiple target APIs.

[0014] In one possible design, multiple target APIs correspond to user profiles. Based on this design, since user profiles can reflect user preferences, habits, etc., the API corresponding to the scenario intent is compatible with the user profile, which can make the API under the scenario intent also compatible with the user's preferences, habits, etc., and can better meet the actual needs of users.

[0015] In one possible design, if the user portrait is a first user portrait, the multiple target APIs are multiple first APIs; if the user portrait is a second user portrait, and the second user portrait is different from the first user portrait, the multiple target APIs are multiple second APIs, and the multiple second APIs and the multiple first APIs include different APIs. Based on this design, for the same scenario intention, different users may have different corresponding user portraits, and then the corresponding response information is output according to different user portraits. In other words, different response information can be recommended for different users, which can flexibly respond to the needs of various users and avoid the homogeneity of the generated response information.

[0016] In one possible design, before receiving the user instruction, the method further includes: generating a user portrait based on a second scenario intent, an API corresponding to the second scenario intent, and a plurality of operation records of the user on the result of calling the API corresponding to the second scenario intent, wherein the second scenario intent is the scenario intent acquired before receiving the user instruction. Based on this design, the user portrait is generated based on the API corresponding to the scenario intent and the user's operation records on the result of calling the API corresponding to the historical scenario intent, so that the generated user portrait can reflect the user's real preferences or needs under one or more scenario intents, and then determine the user's response result under a certain scenario intent based on the user portrait that can reflect the user's real preferences or needs under one or more scenario intents, so that the response result can better meet the user's needs under the scenario intent, and improve the accuracy and pertinence of the response result.

[0017] In one possible design, before acquiring multiple target APIs according to the first scenario intent, the method further includes: determining the first scenario intent according to a preset scenario intent set and a user instruction, wherein the preset scenario intent set includes multiple scenario intents. Based on this design, the terminal device determines the first scenario intent according to the preset scenario intent set and the user instruction, which can prevent the user's sensitive data from being uploaded to the cloud and protect the user's privacy.

[0018] In a possible design, the user portrait and / or the environmental scene information of the electronic device, and the first scene intention are included in the first information, and the first information also includes at least one of a security verification code and the type of the electronic device. The security verification code is used to verify the first information, and the type of the electronic device is used to determine multiple target APIs. Based on this design, the information sent by the electronic device to the server contains a security verification code, which can ensure the security of the information sent by the electronic device to the server and prevent the information from being tampered with. Since multiple different electronic devices may have the same account (such as a Huawei account). For example: for the same user's mobile phone, tablet, watch, etc., the corresponding user account may be the same. Therefore, some data of these different electronic devices may be shared. In order to prevent the subsequent server from determining the response results for these different electronic devices to be exactly the same, that is, in order to output different response results on different electronic devices of the same account, the electronic device can send its own device type to the server to further improve the output effect of the response result.

[0019] In a second aspect, a human-computer interaction method is provided, which is applied to a server, and the method includes: receiving first information from an electronic device, the first information including a first scenario intention; generating second information based on the first information and an API corresponding to the first scenario intention, the second information including multiple target APIs, the multiple target APIs corresponding to the first scenario intention, and the multiple target APIs are used to implement user needs corresponding to the first scenario intention; and sending the second information to the electronic device.

[0020] In one possible design, the first information also includes a user portrait and / or environmental scene information of the electronic device, wherein the environmental scene information of the electronic device is used to characterize at least one of the environmental conditions around the electronic device and the state of the electronic device.

[0021] In one possible design, the second information also includes a multimodal display method of the results of calling multiple target APIs, and the multimodal display method is a method of using multiple display APIs in text, pictures, audio, and video.

[0022] In one possible design, the multimodal display method corresponds to the environmental scene information of the electronic device.

[0023] In one possible design, if the environmental scene information of the electronic device represents that the electronic device is in a first environmental condition, the multimodal display mode is the first display mode; if the environmental scene information of the electronic device represents that the electronic device is in a second environmental condition, which is different from the first environmental condition, the multimodal display mode is the second display mode, which is different from the first display mode.

[0024] In one possible design, the first environmental condition is a quiet environment, and the first display method is to use video and / or audio to display the results of calling multiple target APIs; the second environmental condition is a noisy environment, and the second display method is to use text and / or pictures to display the results of calling multiple target APIs.

[0025] In one possible design, multiple target APIs correspond to user profiles.

[0026] In one possible design, if the user portrait is a first user portrait, the multiple target APIs are multiple first APIs; if the user portrait is a second user portrait, and the second user portrait is different from the first user portrait, the multiple target APIs are multiple second APIs, and the multiple second APIs and the multiple first APIs include different APIs.

[0027] In one possible design, before generating the second information based on the first information and the API corresponding to the first scenario intent, the method also includes: determining the API corresponding to the first scenario intent based on a preset scenario intent set and the API corresponding to the preset scenario intent set, wherein the preset scenario intent set includes multiple scenario intents, and the API corresponding to the preset scenario intent set is the API corresponding to each scenario intent in the multiple scenario intents.

[0028] In one possible design, generating the second information according to the first information and the API corresponding to the first scene intent includes: inputting the first information and the API corresponding to the first scene intent into a preset model, and outputting the second information through the preset model. In this way, the server inputs the user portrait, the environmental scene information of the electronic device, the scene intent, the API corresponding to the scene intent, etc. into the model, and outputs the response result through the model, so that a more accurate and rich response result can be obtained.

[0029] In one possible design, the first information also includes at least one of a security verification code and a type of electronic device. The security verification code is used to verify the first information, and the type of electronic device is used to determine multiple target APIs.

[0030] It should be noted that the technical effects brought about by other designs in the above-mentioned second aspect can refer to the technical effects brought about by the corresponding designs in the first aspect, and will not be repeated here.

[0031] In a third aspect, an electronic device is provided, which has the function of implementing the method described in the first aspect and any design thereof. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions. In a possible example, the electronic device includes a processing unit (or processing module); the processing unit is used to: receive user instructions; when the user instructions represent the first scene intention, obtain multiple target APIs according to the first scene intention, and the multiple target APIs correspond to the first scene intention; output the results of calling multiple target APIs, and the results of calling multiple target APIs are used to implement user needs corresponding to the first scene intention.

[0032] In one possible design, the processing unit is specifically used to obtain multiple target APIs based on the user portrait and / or the environmental scene information of the electronic device and the first scene intention. The environmental scene information of the electronic device is used to characterize at least one of the environmental conditions around the electronic device and the state of the electronic device.

[0033] In one possible design, the electronic device also includes a communication unit; the communication unit is used to: send a user portrait and / or environmental scene information of the electronic device, and a first scene intention to a server; and receive multiple target APIs from the server.

[0034] In one possible design, the processing unit is specifically used to output the results of calling multiple target APIs in a multimodal display manner of the results of calling multiple target APIs, and the multimodal display manner is a method of using multiple display APIs in text, pictures, audio, and video.

[0035] In one possible design, the communication unit is also used to receive a multimodal display of the results of calling multiple target APIs from the server.

[0036] In one possible design, the multimodal display method corresponds to the environmental scene information of the electronic device.

[0037] In one possible design, if the environmental scene information of the electronic device represents that the electronic device is in a first environmental condition, the multimodal display mode is the first display mode; if the environmental scene information of the electronic device represents that the electronic device is in a second environmental condition, which is different from the first environmental condition, the multimodal display mode is the second display mode, which is different from the first display mode.

[0038] In one possible design, the first environmental condition is a quiet environment, and the first display method is to use video and / or audio to display the results of calling multiple target APIs; the second environmental condition is a noisy environment, and the second display method is to use text and / or pictures to display the results of calling multiple target APIs.

[0039] In one possible design, multiple target APIs correspond to user profiles.

[0040] In one possible design, if the user portrait is a first user portrait, the multiple target APIs are multiple first APIs; if the user portrait is a second user portrait, and the second user portrait is different from the first user portrait, the multiple target APIs are multiple second APIs, and the multiple second APIs and the multiple first APIs include different APIs.

[0041] In one possible design, the processing unit is also used to generate a user portrait based on a second scenario intent, an API corresponding to the second scenario intent, and multiple operation records of the user's results of calling the API corresponding to the second scenario intent, where the second scenario intent is the scenario intent obtained before receiving the user's instruction.

[0042] In one possible design, the processing unit is also used to determine a first scene intention based on a preset scene intention set and a user instruction, where the preset scene intention set includes multiple scene intentions.

[0043] In one possible design, the user portrait and / or environmental scene information of the electronic device, and the first scene intention are included in the first information. The first information also includes at least one of a security verification code and a type of electronic device. The security verification code is used to verify the first information, and the type of electronic device is used to determine multiple target APIs.

[0044] In a fourth aspect, a server is provided, which has the function of implementing the method described in the second aspect and any design thereof. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions. In a possible example, the server includes a communication unit (or communication module) and a processing unit (or processing module); the communication unit is used to receive first information from an electronic device, and the first information includes a first scene intent; the processing unit is used to generate second information according to the first information and an API corresponding to the first scene intent, and the second information includes multiple target APIs, and the multiple target APIs correspond to the first scene intent, and the multiple target APIs are used to implement user needs corresponding to the first scene intent; the communication unit is also used to send the second information to the electronic device.

[0045] In one possible design, the first information also includes a user portrait and / or environmental scene information of the electronic device, wherein the environmental scene information of the electronic device is used to characterize at least one of the environmental conditions around the electronic device and the state of the electronic device.

[0046] In one possible design, the second information also includes a multimodal display method of the results of calling multiple target APIs, and the multimodal display method is a method of using multiple display APIs in text, pictures, audio, and video.

[0047] In one possible design, the multimodal display method corresponds to the environmental scene information of the electronic device.

[0048] In one possible design, if the environmental scene information of the electronic device represents that the electronic device is in a first environmental condition, the multimodal display mode is the first display mode; if the environmental scene information of the electronic device represents that the electronic device is in a second environmental condition, which is different from the first environmental condition, the multimodal display mode is the second display mode, which is different from the first display mode.

[0049] In one possible design, the first environmental condition is a quiet environment, and the first display method is to use video and / or audio to display the results of calling multiple target APIs; the second environmental condition is a noisy environment, and the second display method is to use text and / or pictures to display the results of calling multiple target APIs.

[0050] In one possible design, multiple target APIs correspond to user profiles.

[0051] In one possible design, if the user portrait is a first user portrait, the multiple target APIs are multiple first APIs; if the user portrait is a second user portrait, and the second user portrait is different from the first user portrait, the multiple target APIs are multiple second APIs, and the multiple second APIs and the multiple first APIs include different APIs.

[0052] In one possible design, the processing unit is also used to determine the API corresponding to the first scene intent based on a preset scene intent set, the API corresponding to the preset scene intent set, and the first scene intent, wherein the preset scene intent set includes multiple scene intents, and the API corresponding to the preset scene intent set is the API corresponding to each scene intent in the multiple scene intents.

[0053] In one possible design, the processing unit is specifically used to input the first information and the API corresponding to the first scene intention into a preset model, and output the second information through the preset model.

[0054] In one possible design, the first information also includes at least one of a security verification code and a type of electronic device. The security verification code is used to verify the first information, and the type of electronic device is used to determine multiple target APIs.

[0055] In a fifth aspect, an electronic device is provided, including a processor and a memory, the memory being coupled to the processor, the memory being used to store computer program code, the computer program code including computer instructions, the processor reading the computer instructions from the memory, so that the electronic device executes the method as described in the first aspect and any one of the designs thereof. Optionally, the memory may be coupled to the processor or may be independent of the memory.

[0056] In one possible design, the electronic device further includes a communication interface, which can be used for the electronic device to communicate with other devices. Exemplarily, the communication interface can be a transceiver, an input / output interface, an interface circuit, an output circuit, an input circuit, a pin or a related circuit, etc.

[0057] In a possible design, the electronic device may further include an output device, which may include but is not limited to a display screen and an audio module. The display screen may be used to output image information. The audio module may be used to output audio information.

[0058] In a sixth aspect, a server is provided, comprising a processor, a memory, and a communication interface, wherein the memory and the communication interface are coupled to the processor, the communication interface is used to communicate with other devices, the memory is used to store computer program codes, the computer program codes include computer instructions, and the processor reads the computer instructions from the memory so that the server executes the method described in the second aspect and any one of the designs thereof. Optionally, the memory may be coupled to the processor or may be independent of the memory.

[0059] In the seventh aspect, a computer-readable storage medium is provided, the computer-readable storage medium comprising a computer program, which, when the computer program is running on an electronic device, enables the electronic device to execute the method described in the first aspect and any one of the designs thereof; or, when the computer program is running on a server, enables the server to execute the method described in the second aspect and any one of the designs thereof.

[0060] In an eighth aspect, a computer program product is provided, the computer program product comprising: a computer program or instructions, which, when the computer program or instructions are run on a computer, enables the computer to execute the method as described in the first aspect and any one of the designs thereof, or enables the computer to execute the method as described in the second aspect and any one of the designs thereof.

[0061] In a ninth aspect, a circuit system is provided, the circuit system comprising a processing circuit, and the processing circuit is configured to execute the method described in any one of the designs of the first aspect or the second aspect.

[0062] In the tenth aspect, a chip system is provided, comprising at least one processor and at least one interface circuit, wherein the at least one interface circuit is used to perform transceiver functions and send instructions to at least one processor, and when the at least one processor executes the instructions, the at least one processor executes the method described in any one of the designs in the first aspect or the second aspect above.

[0063] In an eleventh aspect, a communication system is provided, comprising the electronic device as described in the third aspect and the server as described in the fourth aspect.

[0064] It should be noted that the technical effects brought about by any design in the above-mentioned third to eleventh aspects can refer to the technical effects brought about by the corresponding designs in the first and second aspects, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 It is a schematic diagram of an existing human-computer interaction interface;

[0066] Figure 2 A process diagram of an existing human-computer interaction method is shown;

[0067] Figure 3 A process diagram of another existing human-computer interaction method;

[0068] Figure 4 A schematic diagram of the architecture of a communication system provided in an embodiment of the present application;

[0069] Figure 5 A schematic diagram of the structure of a server provided in an embodiment of the present application;

[0070] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0071] Figure 7 A schematic diagram of the structure of another electronic device and a server provided in an embodiment of the present application;

[0072] Figure 8 A schematic diagram of a human-computer interaction method provided in an embodiment of the present application;

[0073] Fig. 9 Interface diagram provided for the embodiment of the present application Figure 1 ;

[0074] Fig.10 Interface diagram provided for the embodiment of the present application Figure 2 ;

[0075] Fig.11 Interface diagram provided for the embodiment of the present application Figure 3 ;

[0076] Fig.12 A schematic diagram of an API corresponding to a scenario intent provided in an embodiment of the present application;

[0077] Fig.13 Interface diagram provided for the embodiment of the present application Figure 4 ;

[0078] Fig.14 Interface diagram provided for the embodiment of the present application Figure 5 ;

[0079] Fig.15 A flowchart of another human-computer interaction method provided in an embodiment of the present application;

[0080] Fig.16 A schematic diagram of a user portrait provided in an embodiment of the present application;

[0081] Fig.17 A schematic diagram of another user portrait provided in an embodiment of the present application;

[0082] Fig.18 A schematic diagram of a user tag associated with an API provided in an embodiment of the present application;

[0083] Fig.19 A schematic diagram of response information corresponding to different user portraits provided in an embodiment of the present application;

[0084] Fig. 20 A schematic diagram of response information corresponding to different environmental information provided in an embodiment of the present application;

[0085] Fig.21 Interface diagram provided for the embodiment of the present application Figure 6 ;

[0086] Fig. 22 Interface diagram provided for the embodiment of the present application Figure 7 ;

[0087] Fig.23 A schematic diagram of the structure of a device provided in an embodiment of the present application;

[0088] Fig.24 A schematic diagram of the structure of a chip system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0089] The technical solution provided by the embodiments of the present application is described in detail below in conjunction with the accompanying drawings.

[0090] The terms "including" and "having" and any variations thereof mentioned in the description of this application are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but may optionally include other steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices.

[0091] It should be noted that, in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0092] In the description of this application, unless otherwise specified, the meaning of "plurality" refers to two or more. "And / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0093] It can be understood that in the embodiments of the present application, scenario intent can refer to the actual or potential needs of users in different scenarios. An application programming interface (API) is a set of definitions, programs, and protocols that enable mutual communication between computer software. For example, the API defines the types of calls and requests that can be made between different computer software, how to make calls and send requests, and the data format that should be used.

[0094] At present, electronic devices (such as mobile phones, etc.) can respond to scene intentions input by users based on AI technology. For example, users can input scene intentions to electronic devices through various methods such as voice. After receiving the scene intention input by the user, the electronic device will call a single function in response to the scene intention, such as directly displaying text, cards, etc.

[0095] For example, taking a mobile phone as an electronic device, when a user has a travel need, the user can input a voice command such as "I am going to Beijing for a trip at 8:30 tomorrow morning. Please give me some travel suggestions" into the mobile phone. In response to the voice command input by the user, the mobile phone can respond in the form of text or a card. In some examples, the mobile phone may not be able to recognize the user's scene intention based on the voice command input by the user. At this time, the mobile phone may display the following: Figure 1The content of the card 101 shown in (1) may display a text indicating a failed response, such as "I don't quite understand". In other examples, after the mobile phone recognizes the user's scene intention according to the voice command input by the user, if it is determined that the user's scene intention is a travel intention, it may display Figure 1 The content of the card 102 shown in (2) may be a text-based recommendation of some simple information such as scenic spots and food culture to the user.

[0096] The above response methods are limited to the semantic logic of scene intentions, the response results are too simple, cannot meet user needs, and the effect of human-computer dialogue is not good.

[0097] In an existing related scheme, Figure 2 As shown, in the human-computer dialogue scenario, the mobile phone receives the voice information input by the user, and then the mobile phone performs semantic recognition on the aforementioned voice information based on multimodal information (such as the surrounding information of the conversation environment, the user's voice and (facial or body) expressions, etc.) to understand the user's words and output the corresponding voice to respond to the user. In this solution, the user's inquiry is responded to by outputting voice, and the response result is also relatively simple. It is impossible to provide rich, comprehensive and convenient response results based on the user's intentions in different scenarios, and it is impossible to meet user needs. The effect of human-computer dialogue is also not good.

[0098] In another existing related scheme, such as Figure 3 As shown, during the voice interaction process, the voice information input by the user is received, a feedback sentence is generated according to the voice information, and the emotional score of the feedback sentence is calculated according to the set emotional score mechanism, and then the emotional type is determined according to the emotional score, and the feedback sentence is played according to the emotional type. In this scheme, only the corresponding feedback sentence is played according to the voice information input by the user, and the response result is relatively simple. It is impossible to provide rich, comprehensive and convenient response results based on the user's different scene intentions, and it is impossible to meet user needs. The effect of human-computer dialogue is also not good.

[0099] Based on this, the embodiment of the present application provides a human-computer interaction method, which can solve the problem of a single response result to user inquiries, and can provide richer, more comprehensive and convenient response results based on the user's scenario intentions, meet user needs, and improve the effect of human-computer dialogue.

[0100] For example, Figure 4 The schematic diagram of the architecture of a communication system for applying a human-computer interaction method provided in an embodiment of the present application is shown. Figure 4 As shown, the communication system 400 includes an electronic device 401 and a server 402 .

[0101] The electronic device 401 and the server 402 may be connected via wired communication technology and / or wireless communication technology. The wireless communication technology includes but is not limited to at least one of the following: near field communication (NFC), Bluetooth (BT) (e.g., traditional Bluetooth or Bluetooth low energy (BLE)), wireless local area networks (WLAN) (e.g., wireless fidelity (Wi-Fi) networks), Zigbee, frequency modulation (FM), infrared (IR), etc.

[0102] Optionally, the electronic device 401 may be, for example, a mobile phone, a personal computer (PC), a tablet computer (Pad), a laptop computer, a desktop computer, a laptop computer, a computer with a transceiver function, a wearable device, a vehicle-mounted device, an artificial intelligence (AI) device, or other terminal device. The operating system installed in the electronic device 401 includes but is not limited to: Or other operating systems. In some embodiments, the electronic device 401 can be a fixed device or a portable device. The present application does not limit the specific type of the electronic device 401 or the installed operating system.

[0103] Optionally, server 402 may be a device or server with computing functions such as a cloud server or a network server. Server 402 may be a single server, or a server cluster consisting of multiple servers, or a cloud computing service center. For example, a server cluster may be a server cluster consisting of one or more servers in one or more regions, or servers in one or more computer rooms. Optionally, the operating system of server 402 includes but is not limited to Or other operating systems. This application does not impose any restrictions on the specific type of server 402 and the installed operating system.

[0104] In some embodiments, the electronic device 401 may receive an instruction input by a user and determine the user's scene intention based on the instruction. Optionally, the electronic device 401 may send the scene intention to the server 402. Accordingly, after receiving the scene intention, the server 402 may generate response information based on the scene intention. Optionally, the server 402 may send the response information to the electronic device 401. Accordingly, after receiving the response information, the electronic device 401 may output the response information, such as displaying the response information on a display screen, broadcasting the response information by voice, etc.

[0105] It should be understood that Figure 4 The simplified schematic diagram is only for the sake of understanding. The communication system may also include other devices. Figure 4 Not drawn.

[0106] Optionally, the server 402 in the embodiment of the present application can be implemented by different devices. For example, the server 402 in the embodiment of the present application can be implemented by Figure 5 This can be achieved by using the devices in the Figure 5 The hardware structure diagram of the server 402 provided in the embodiment of the present application is shown in FIG. The server 402 includes at least one processor 501 , a communication line 502 , a memory 503 and at least one communication interface 504 .

[0107] The processor 501 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the present application.

[0108] Communication link 502 may include a pathway for transmitting information between the above-mentioned components.

[0109] Communication interface 504, used to communicate with other devices (such as Figure 4The electronic device 401 shown in the present application) communicates. In an embodiment of the present application, the communication interface may be a module, a circuit, a bus, an interface, a transceiver or other device capable of realizing a communication function, for communicating with other devices. Optionally, when the communication interface is a transceiver, the transceiver may be an independently arranged transmitter, which may be used to send information to other devices, and the transceiver may also be an independently arranged receiver, for receiving information from other devices. The transceiver may also be a component that integrates the functions of sending and receiving information, and the embodiments of the present application do not limit the specific implementation of the transceiver. In some embodiments of the present application, the communication interface 504 may be used to receive one or more of the user's scene intention, the environmental scene information of the electronic device, and the user portrait from the electronic device. In other embodiments of the present application, the communication interface 504 may be used to send response information obtained based on one or more of the user's scene intention, the environmental scene information of the electronic device, the user portrait, etc. to the electronic device.

[0110] The memory 503 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 503 may exist independently and be connected to the processor 501 via the communication line 502. The memory 503 may also be integrated with the processor 501.

[0111] The memory 503 is used to store computer-executable instructions for implementing the solution of the present application, and the execution is controlled by the processor 501. The processor 501 is used to execute the computer-executable instructions stored in the memory 503, thereby implementing the method provided in the following embodiments of the present application. For example, the processor 501 determines the corresponding response information based on one or more information of the received user's scene intention, the environmental scene information of the electronic device, and the user portrait.

[0112] Optionally, the computer-executable instructions in the embodiments of the present application may also be referred to as application code, instructions, computer program or other names, which are not specifically limited in the embodiments of the present application.

[0113] In some embodiments of the present application, the memory 503 may also be used to store one or more of a scene intent set, an application programming interface API corresponding to the scene intent, an AI model, an intent framework system, etc. For a detailed introduction to the scene intent set, the API corresponding to the scene intent, the AI ​​model, the intent framework system, etc., please refer to the following description.

[0114] In a specific implementation, as an embodiment, the processor 501 may include one or more CPUs, such as Figure 5 CPU0 and CPU1 in.

[0115] In a specific implementation, as an embodiment, the server 402 may include multiple processors, such as Figure 5 501 and processor 505 in the embodiment of the present invention. Each of these processors may be a single-CPU processor or a multi-CPU processor. The processor herein may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0116] The server 402 may be a general-purpose device or a dedicated device, and the embodiment of the present application does not limit the type of the server 402. The server 402 may be a device having Figure 5 Devices of similar structure.

[0117] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the server 402. In other embodiments of the present application, the server 402 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0118] For example, taking the electronic device 401 as a mobile phone, Figure 6 A schematic diagram of the hardware structure of an electronic device 401 provided in an embodiment of the present application is shown.

[0119] The electronic device 401 includes a processor 610, an external memory interface 620, an internal memory 621, a universal serial bus (USB) interface 630, a charging management module 640, a power management module 641, a battery 642, an antenna 1, an antenna 2, a mobile communication module 650, a wireless communication module 660, an audio module 670, a sensor module 680, a button 690, a motor 691, an indicator 692, a camera 693, a display screen 694, and a subscriber identification module (SIM) card interface 695, etc.

[0120] The processor 610 may include one or more processing units, for example, the processor 610 may include an application processor (AP), a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0121] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0122] The processor 610 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 610 is a cache memory. The memory may store instructions or data that the processor 610 has just used or cyclically used. If the processor 610 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 610, and thus improves the efficiency of the system.

[0123] In some embodiments, processor 610 may include one or more interfaces.

[0124] The charging management module 640 is used to receive charging input from a charger, where the charger can be a wireless charger or a wired charger.

[0125] The power management module 641 is used to connect the battery 642, the charging management module 640 and the processor 610. The power management module 641 receives input from the battery 642 and / or the charging management module 640 to power the processor 610, the internal memory 621, the display screen 694, the camera 693, and the wireless communication module 660.

[0126] The wireless communication function of the electronic device 401 can be implemented through antenna 1, antenna 2, mobile communication module 650, wireless communication module 660, etc.

[0127] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 401 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of the antennas.

[0128] The mobile communication module 650 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 401. The mobile communication module 650 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 650 can receive electromagnetic waves from the antenna 1, and filter, amplify, etc. the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 650 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1.

[0129] The wireless communication module 660 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 401. The wireless communication module 660 can be one or more devices integrating at least one communication processing module. The wireless communication module 660 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 610. The wireless communication module 660 can also receive the signal to be sent from the processor 610, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0130] In some embodiments, the antenna 1 of the electronic device 401 is coupled to the mobile communication module 650, and the antenna 2 is coupled to the wireless communication module 660, so that the electronic device 401 can communicate with the network and other devices (such as Figure 4 The server 402 shown in the figure) communicates.

[0131] In some embodiments of the present application, the electronic device 401 may also sense the environmental scene information of the electronic device 401 through the antenna 1, the mobile communication module 650, the antenna 2, the wireless communication module 660, and the like.

[0132] The electronic device 401 implements the display function through a GPU, a display screen 694, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 694 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 610 may include one or more GPUs, which execute program instructions to generate or change display information.

[0133] The display screen 694 is used to display images, videos, etc. The display screen 694 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 401 may include 1 or N display screens 694, where N is a positive integer greater than 1. In some embodiments of the present application, the display screen 694 can be used to receive instructions input by a user. In other embodiments of the present application, the display screen 694 can also be used to display response information obtained based on the instructions input by the user.

[0134] The electronic device 401 can realize the shooting function through ISP, camera 693, video codec, GPU, display screen 694 and application processor.

[0135] The camera 693 is used to capture static images or videos. In some embodiments, the electronic device 401 may include 1 or N cameras 693, where N is a positive integer greater than 1. In some embodiments of the present application, the camera 693 may also be used to collect images around the electronic device 401 to obtain environmental scene information of the electronic device 401.

[0136] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of electronic device 401 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc. In some embodiments of the present application, NPU can be used for voice recognition of voice commands input by users.

[0137] The external memory interface 620 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 401. The external memory card communicates with the processor 610 through the external memory interface 620 to implement a data storage function. For example, files such as music and videos can be stored in the external memory card.

[0138] The internal memory 621 can be used to store computer executable program codes, which include instructions. The processor 610 executes various functional applications and data processing of the electronic device 401 by running the instructions stored in the internal memory 621. The internal memory 621 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 401 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 621 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0139] In some embodiments of the present application, one or more of a scene intent set, a user portrait, and an intent framework system may be stored in the internal memory 621. For an introduction to the scene intent set, the user portrait, and the intent framework system, please refer to the following description.

[0140] The electronic device 401 can implement audio functions such as music playing and recording through the audio module 670 and the application processor.

[0141] The audio module 670 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signal. The audio module 670 can also be used to encode and decode audio signals. In some embodiments, the audio module 670 can be arranged in the processor 610, or some functional modules of the audio module 670 can be arranged in the processor 610.

[0142] Optionally, the audio module 670 may include at least one of a speaker, a receiver, a microphone, and an earphone interface.

[0143] In some embodiments of the present application, the audio module 670 may be used to receive instructions input by a user. In other embodiments of the present application, the audio module 670 may also be used to output response information obtained based on the instructions input by the user.

[0144] Optionally, the sensor module 680 may include a pressure sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, etc.

[0145] The buttons 690 include a power button, a volume button, etc. The buttons 690 may be mechanical buttons or touch buttons.

[0146] Motor 691 can generate vibration prompts.

[0147] Indicator 692 may be an indicator light, which may be used to indicate charging status, power changes, messages, notifications, etc.

[0148] The SIM card interface 695 is used to connect a SIM card.

[0149] For example, the above only uses a mobile phone as an example to illustrate the structure of the electronic device 401 in the embodiment of the present application, but does not constitute a limitation on the structure and form of the electronic device 401. The embodiment of the present application does not limit the structure and form of the electronic device 401. For example, the structure of the electronic device 401 may also be such as Figure 5 The form of the server 402 is shown, but the present application is not limited thereto.

[0150] In some other embodiments of the present application, the electronic device 401 may include Figure 6 More or fewer components shown, or some components are combined, or some components are split, or some components are replaced, or different component arrangements are arranged. The components shown in the figure can be implemented in hardware, software, or a combination of software and hardware.

[0151] In some embodiments, the electronic device 401 and the server 402 may be divided into functional modules, such as dividing each functional module corresponding to each function, or concentrating two or more functions in one processing unit. The above integrated unit may be implemented in the form of hardware or software functional modules.

[0152] As a possible division, Figure 7 A schematic diagram of the software structure of an electronic device 401 and a server 402 provided in an embodiment of the present application is shown.

[0153] like Figure 7 As shown, the electronic device 401 may include a scene intent recognition module 701 and an intent framework system 702 .

[0154] Among them, the scene intention recognition module 701 can be used to identify whether a user instruction is a scene intention, and specifically which scene intention it is. The user instruction can be a user instruction received by the electronic device 401. For specific implementations of how to identify whether a user instruction is a scene intention, and specifically which scene intention it is, please refer to the following description.

[0155] The intent framework system 702 may be a system for implementing communication between the electronic device 401 and the server 402. In some embodiments, the intent framework system 702 may be used to send the scene intent identified by the scene intent identification module 701 to the server 402. In another embodiment, the intent framework system 702 may be used to receive response information from the server 402.

[0156] Optionally, the electronic device 401 may further include at least one of a user portrait generation module 703 and an environment scene information acquisition module 704 .

[0157] Among them, the user portrait generation module 703 can be used to generate a user portrait. For an introduction to the user portrait, please refer to the following description. In some embodiments, the user portrait can also be used to generate response information. The intent framework system 702 can also be used to send the user portrait generated by the user portrait generation module 703 to the server 402.

[0158] The environment scene information acquisition module 704 can be used to acquire environment scene information, and the environment scene information can be used to characterize at least one of the environment conditions around the electronic device 401, the state of the electronic device 401, etc. In some embodiments, the environment scene information can also be used to generate response information. The intention framework system 702 can also be used to send the environment scene information acquired by the environment scene information acquisition module 704 to the server 402.

[0159] The server 402 may include an intent framework system 711 and a response information generation module 712 .

[0160] Among them, the intent framework system 711 can be a system for realizing communication between the electronic device 401 and the server 402. In some embodiments, the intent framework system 711 can be used to receive one or more of the scene intent, the API corresponding to the scene intent, the user portrait, the environmental scene information, etc. from the electronic device 401. For the introduction of the API corresponding to the scene intent, please refer to the following description. In other embodiments, the intent framework system 711 can also be used to send response information to the electronic device 401.

[0161] In some other embodiments, the intent framework system 711 may also be used to send one or more of the scenario intent, the API corresponding to the scenario intent, the user portrait, the environmental scenario information, etc. to the response information generation module 712. In some other embodiments, the intent framework system 711 may also be used to receive response information from the response information generation module 712.

[0162] In some other embodiments, the intent framework system 711 may also be used to receive notification messages from the electronic device 401, user operation behavior data for response information, etc. Optionally, the notification messages from the electronic device 401, user operation behavior data for response information, etc. may be reported by system applications and / or third-party applications in the electronic device 401. Exemplarily, the notification messages from the electronic device 401 may include, but are not limited to, various notification messages such as weather notifications, flight delay notifications, train delay notifications, commodity discount notifications, and traffic congestion notifications. In some embodiments, the intent framework system 711 may also be used to send notification messages from the electronic device 401, user operation behavior data for response information, etc. to the response information generation module 712.

[0163] The response information generation module 712 can be used to generate response information corresponding to the user instruction. In some embodiments, the response information generation module 712 can receive one or more of the scene intent, the API corresponding to the scene intent, the user portrait, the environmental scene information, the notification message of the electronic device 401, the user's operation behavior data on the response information, etc. from the intention framework system 711, and then generate the response information according to one or more of the scene intent, the API corresponding to the scene intent, the user portrait, the environmental scene information, the notification message of the electronic device 401, the user's operation behavior data on the response information, etc. In some embodiments, the response information generation module 712 can also be used to send the response information to the intention framework system 711.

[0164] In a specific implementation, an AI model (or a big model, or an AI big model, etc.) may be provided in the response information generation module 712. The response information generation module 712 may input one or more of the scene intent, the API corresponding to the scene intent, the user portrait, the environmental scene information, the notification message of the electronic device 401, the user's operation behavior data on the response information, etc. into the AI ​​model, and output the response information through the AI ​​model.

[0165] It can be understood that the AI ​​model can be various models with the ability to understand and generate text. For example, the AI ​​model can be a machine learning model or a deep learning model obtained through training with a large number of parameters and a complex structure. The embodiments of the present application do not impose specific restrictions on the training method of the AI ​​model.

[0166] Understandably, Figure 7 The software modules included in the electronic device 401 and / or server 402 shown are only used as a possible division method. In actual applications, the electronic device 401 and / or server 402 may include more or fewer modules, or there may be other module division methods, which are not limited in the embodiments of the present application.

[0167] The technical solutions involved in the following embodiments can all be implemented in Figure 5 , Figure 6 , Figure 7 The device of the structure shown in Figure 4 The system is implemented in the system with the architecture shown.

[0168] It can be understood that the various interfaces of the electronic device shown in the embodiments of the present application are given by taking the electronic device as a mobile phone as an example.

[0169] It can also be understood that in the embodiments of the present application, the electronic device and / or the server can perform some or all of the steps in the embodiments of the present application, and these steps or operations are only examples, and the embodiments of the present application can also perform other operations or variations of various operations. In addition, the various steps can be performed in different orders presented in the embodiments of the present application, and it is possible that not all operations in the embodiments of the present application need to be performed.

[0170] In some embodiments, response information can be generated based on the scene intent and the API corresponding to the scene intent. Figure 8 FIG. 1 is a flow chart of a human-computer interaction method provided in an embodiment of the present application. Figure 8 As shown, the method comprises the following steps:

[0171] S801. The electronic device receives a user instruction.

[0172] Exemplarily, user instructions (or user operations) may include but are not limited to various types of instructions executed by the user, such as voice instructions, key instructions, gesture instructions, etc.

[0173] In some examples, taking the user instruction as a voice instruction, Fig. 9 A schematic diagram of a scenario in which an electronic device receives a voice command provided by an embodiment of the present application is shown. Fig. 9 As shown in (1), the electronic device may display a voice command receiving interface 900. Optionally, the voice command receiving interface 900 may include a voice mark 901, which facilitates the user to know that the voice command receiving function of the electronic device is turned on. Optionally, the voice command receiving interface 900 may also include one or more shortcut tags, such as "selfie", "turn on the flashlight", etc. The user may also input a command to the electronic device by performing an operation such as clicking on a shortcut tag.

[0174] If the electronic device displays Fig. 9 The voice command receiving interface 900 shown in (1) in the figure, that is, when the voice command receiving function is turned on, if the user says a voice command such as "I am going to travel to Beijing at 8:30 tomorrow morning, please give me some travel advice", the electronic device can receive the voice command. Optionally, after receiving the voice command, the electronic device can also output the recognized voice command, such as but not limited to outputting the recognized voice command in the form of text. For example, the electronic device can display the following: Fig. 9 The command output interface 910 shown in (2) is used to facilitate the user to confirm whether the command recognized by the electronic device is consistent with the command input by the user.

[0175] In other examples, taking the user instruction as a key instruction as an example, Fig.10 A schematic diagram of a scenario in which an electronic device receives a key command provided by an embodiment of the present application is shown. Fig.10 As shown in (1), the electronic device can display a key command receiving interface 1000, wherein the key command receiving interface 1000 includes a keyboard icon 1001, and the user can call the input method application through the keyboard icon 1001 and input the command. Optionally, the key command receiving interface 1000 can also include one or more shortcut tags. For an introduction to these shortcut tags, please refer to Fig. 9 In some examples, users can Fig. 9 The voice mark 901 shown in (1) performs an operation such as clicking, and in response to the operation, the electronic device can display Fig.10 The key command receiving interface 1000 shown in (1) above, of course, the user may also call up the key command receiving interface 1000 in other ways, and the embodiment of the present application does not impose any limitation on this.

[0176] When the electronic device detects that a user Fig.10 In response to the click operation of the keyboard icon 1001 shown in (1), the electronic device may call the input method application and display the following Fig.10 The keyboard 1010 shown in (2) can be used by the user to input instructions. Fig.10 As shown in (2), after the user inputs the key command "I will travel to Beijing at 8:30 tomorrow morning, please give me some travel suggestions" in the input box through the keyboard 1010, the electronic device can receive the key command. Fig.10 As shown in (3), the electronic device can also output the received key command so that the user can know that the electronic device has successfully received the input command.

[0177] In some embodiments, before the electronic device executes step S801, the electronic device may first turn on the function of receiving user instructions. As a possible implementation, the function can be turned on in real time, or in other words, the function is turned on by default after the electronic device is turned on, so that the user can directly input user instructions to the electronic device. As another possible implementation, the function of the electronic device may be in a closed state. At this time, the user can also start the function of the electronic device before inputting user instructions to the electronic device. For example: the user can perform an opening operation. Exemplarily, the opening operation may include but is not limited to voice operation (such as voice input wake-up word), button operation (such as long pressing the power off button), gesture operation, etc. In response to the opening operation, the electronic device starts the function of receiving user instructions.

[0178] For example, Fig.11 A schematic diagram of a scenario for enabling a function of receiving user instructions of an electronic device provided by an embodiment of the present application is shown. Fig.11 As shown in (1), the user inputs the wake-up word "Hi, Xiaox" into the electronic device by voice. In response to the wake-up word, the electronic device starts the function of receiving user instructions. Fig.11 As shown in (2), the user long presses the power button of the electronic device. In response to this operation, the electronic device can also start the function of receiving user instructions.

[0179] Optionally, after the electronic device starts the function of receiving user instructions, the electronic device may display Fig. 9 Alternatively, the voice command receiving interface 900 shown in (1) may be directly displayed. Fig.10 The key command receiving interface 1000 shown in (1) in the figure, etc. Optionally, the electronic device may also output a voice response to the user, so that the user knows that the function of receiving user commands of the electronic device has been turned on.

[0180] Optionally, the electronic device may be in a black screen state or a bright screen state when receiving a user instruction. When in the bright screen state, it may be in a locked screen state or an unlocked state. When in the unlocked state, the running interface of any application may be displayed. The present application embodiment does not impose any restrictions on this.

[0181] In some scenarios, the function of receiving user commands of the electronic device can be integrated into a smart voice assistant. It can be understood that the smart voice assistant can be an intelligent software with a voice recognition function and can respond to user commands through various technical means.

[0182] S802: The electronic device determines the scene intention according to the user instruction.

[0183] In some scenarios, for a user instruction, the user instruction may correspond to a certain scenario intention (that is, the user instruction is an instruction used to implement the scenario intention), or may not correspond to any scenario intention (that is, the user instruction is not an instruction used to implement the scenario intention).

[0184] For example, if the user command is "play a song", "answer the phone for me", "call my mother", "set an alarm for 6:00 in the morning", "introduce me to the climate in the north", etc., then the user command does not correspond to the scene intent. If the user command is "I am going to travel to Beijing at 8:30 tomorrow morning, please give me some travel advice", "What suitable jobs are there at present, please recommend me", "I want to buy a refrigerator, please give me some advice", "I plan to go to Shanghai for medical treatment tomorrow, please give me some advice", etc., then the user command corresponds to the scene intent.

[0185] In some embodiments, the electronic device can identify whether a user instruction corresponds to a scene intent according to the scene intent set, and, if the user instruction corresponds to a scene intent, which scene intent it corresponds to. Figure 7 The scene intent recognition module 701 shown identifies whether a user instruction corresponds to a scene intent and which specific scene intent it is based on the scene intent set.

[0186] It can be understood that in the embodiment of the present application, the scenario intent set can be a set of user intents. The user intent included in the scenario intent set is a scenario intent, and the user intent not included in the scenario intent set is a non-scenario intent, that is, not a scenario intent. In other words, if a user intent is in the scenario intent set, it means that the user intent is a scenario intent, and if a user intent is not in the scenario intent set, it means that the user intent is not a scenario intent.

[0187] In the embodiment of the present application, the user intent included in the scenario intent set is referred to as scenario intent. In some embodiments, the scenario intent set may include multiple scenario intents, and different scenario intents may represent the user's needs in different scenarios.

[0188] For example, the scenario intent set may include, but is not limited to, one or more scenario intents of travel intent, medical treatment intent, learning and education intent, health and fitness intent, shopping and consumption intent, work and career intent, etc. It can be understood that the scenario intents described in the embodiments of the present application are only exemplary explanations made to facilitate understanding of the embodiments of the present application. In actual applications, more or fewer scenario intents, or other types of scenario intents may be included.

[0189] In some implementations, the scene intent set may be sent by a server to the electronic device. Optionally, the electronic device may receive the scene intent set sent by the server before executing step S802. Optionally, the electronic device may also receive the scene intent set sent by the server before executing step S801, or after step S801 and before step S802. In other implementations, the scene intent set may also be obtained by the electronic device through other means, which is not limited in the embodiments of the present application.

[0190] In this embodiment, as a specific implementation, the electronic device can determine whether a user instruction corresponds to a scene intent based on the keywords corresponding to the scene intent contained in the scene intent set. If a user instruction contains a keyword corresponding to a scene intent in the scene intent set, then the user instruction corresponds to the scene intent. If the user instruction does not contain a keyword corresponding to any scene intent in the scene intent set, then the user instruction does not correspond to the scene intent.

[0191] For example, if the scenario intent set includes the travel intent, and the keywords corresponding to the travel intent include "travel", "tourism", "travel outing", etc., then if a user instruction contains any of the keywords "travel", "tourism", "travel outing", etc., it means that the user instruction corresponds to the travel intent. For example, if the user instruction is "I am going to travel to Beijing at 8:30 tomorrow morning, please give me some travel suggestions", the user instruction contains the keywords "tourism" and "travel", which means that the user instruction corresponds to the travel intent.

[0192] For another example: if the scenario intent set includes shopping and consumption intent, the keywords corresponding to the shopping and consumption intent include "buy", "buy", "shopping", etc., then if a user instruction contains any of the keywords "buy", "buy", "shopping", etc., it means that the user instruction corresponds to shopping and consumption intent. For example: if the user instruction "I want to buy a refrigerator, please give me some advice", the user instruction contains the keyword "buy", which means that the user instruction corresponds to shopping and consumption intent.

[0193] For another example: if the scenario intent includes work and career intent, the keywords corresponding to the work and career intent include "work", "career", "employment", etc., then if a user instruction contains any of the keywords "work", "career", "employment", etc., it means that the user instruction corresponds to the work and career intent. For example, if the user instruction is "What suitable jobs are there at present? Please recommend them to me", and the user instruction contains the keyword "work", it means that the user instruction corresponds to the work and career intent.

[0194] If the electronic device determines that the received user instruction corresponds to a certain scene intention, the electronic device may execute step S803 and subsequent steps. If the electronic device determines that the received user instruction corresponds to a non-scene intention, the electronic device may not execute step S803 and subsequent steps. In some embodiments, the electronic device may not respond. In other embodiments, the electronic device may output response information corresponding to the non-scene intention, etc., which is not limited in this application.

[0195] S803: The electronic device sends the scene intent to the server. Correspondingly, the server receives the scene intent from the electronic device.

[0196] For example, electronic devices can use Figure 7 The intent framework system 702 shown in FIG. 702 sends the scene intent to the server. Figure 7 The intent framework system 711 shown receives context intent from an electronic device.

[0197] In some embodiments, the electronic device may first determine the scenario intent corresponding to the user instruction, and then send the corresponding scenario intent to the server. For example, taking the user instruction as "I am going to travel to Beijing at 8:30 tomorrow morning, please give me some travel suggestions", the electronic device identifies the user instruction as a travel intent, and the electronic device may send the "travel intent" to the server.

[0198] Optionally, in this embodiment, the electronic device may also send some key information corresponding to the identified scene intent to the server. For example: taking the travel intent as an example, the key information corresponding to the travel intent may include but is not limited to one or more of the travel location, the number of travelers, the departure time, etc. For another example: taking the medical treatment intent as an example, the attribute information of the medical treatment intent may include but is not limited to one or more of the medical treatment location, the departure time, etc. For another example: taking shopping and consumption intent as an example, the attribute information of shopping and consumption intent may include but is not limited to the types of purchased items, etc. Optionally, the key information corresponding to the scene intent may be obtained according to user instructions.

[0199] In some other embodiments, the electronic device may also send the user instruction and the scene intent identified according to the user instruction to the server. Optionally, in this embodiment, the electronic device may no longer send the key information corresponding to the scene intent.

[0200] It is understandable that the operation of determining the scene intent according to the user instruction can be performed by the electronic device or by the server. For example, the server can determine the scene intent according to the received user instruction. Optionally, the server can also use the same method as the electronic device, such as identifying the user instruction through the scene intent set to determine the scene intent.

[0201] Of course, the electronic device can also send user instructions to other devices besides the server, and the other devices determine the scene intent based on the user instructions, and then send the scene intent to the server directly or indirectly. Optionally, other devices can also use the same method as the electronic device, such as identifying user instructions through a scene intent set to determine the scene intent. Optionally, if the scene intent set is not stored in other devices, other devices can also obtain the scene intent set from the electronic device and / or the server.

[0202] S804: The server generates response information according to the scenario intent.

[0203] In some embodiments, the server can generate response information based on the scene intent and the API corresponding to the scene intent. Figure 7 The response information generation module 712 generates response information according to the scene intent and the API corresponding to the scene intent.

[0204] In some embodiments, a scene intent set and an API corresponding to each scene intent contained in the scene intent set are stored in the server. Optionally, for each scene intent, one or more APIs may correspond, and some or all of the one or more APIs may be used to output the result of calling the one or more APIs to the user after receiving the corresponding scene intent. By calling different APIs, the same or different functions of one or more applications may be implemented, and the functions may be used to meet the user's possible needs under the scene intent.

[0205] In some embodiments, the API corresponding to each scenario intent included in the scenario intent set can be set according to the user's possible needs under the corresponding scenario intent. In some embodiments, the API corresponding to each scenario intent included in the scenario intent set can be an API supported by the electronic device, that is, the electronic device can successfully call the API corresponding to each scenario intent included in the scenario intent set.

[0206] For example, Fig.12 Examples of APIs corresponding to some scenario intents provided in the embodiments of the present application are shown. For example, taking the travel intent as an example, the APIs corresponding to the travel intent may include setting alarm APIs, checking weather APIs, booking air tickets APIs, and booking accommodation APIs. Among them, the setting alarm API may include but is not limited to APIs that can realize the setting alarm function of various applications and / or mini-programs, such as clock application APIs, etc. The weather viewing API may include but is not limited to APIs that can realize the weather viewing function of various applications and / or mini-programs, such as Huawei Weather Application API, Gaogao Weather Application API, Momo Weather Application API, etc. The booking air ticket API may include but is not limited to APIs that can realize the air ticket booking function of various applications and / or mini-programs, such as Feifei Application Air Ticket Query API, Xie Xie Application Air Ticket Query API, Kong Kong Application Air Ticket Query API, etc. The booking accommodation API may include but is not limited to APIs that can realize the accommodation booking function of various applications and / or mini-programs, such as Feifei Application Hotel Query API, Xie Xie Application Hotel Query API, Huahua Application Hotel Query API, etc. For the introduction of APIs corresponding to other scenario intents, please refer to the introduction of APIs corresponding to travel intents.

[0207] It can be understood that the API corresponding to the scenario intent described in the embodiments of the present application is only an exemplary illustration made to facilitate the embodiments of the present application. In actual applications, more or fewer APIs, or other types of APIs may be included.

[0208] In a specific embodiment, the server can first determine the API corresponding to the received scene intent based on the API corresponding to each scene intent contained in the stored scene intent set and the scene intent received from the electronic device, and then input the API corresponding to the received scene intent and the received scene intent into the AI ​​model, and output the response information through the AI ​​model. Please refer to the above for the introduction of the AI ​​model. In this way, by outputting the response information through the AI ​​model, more accurate and rich response information can be obtained.

[0209] In some specific implementations, the server may use a method such as Figure 7 The intent framework system 711 shown in FIG. 7 inputs the received scene intent corresponding to the API and the received scene intent into the AI ​​model. The server can also send the response information output by the AI ​​model to the Figure 7 In the intention framework system 711 shown.

[0210] Of course, in other embodiments, the server may not perform the operation of determining the API corresponding to the received scene intent based on the API corresponding to each scene intent included in the stored scene intent set and the scene intent received from the electronic device, but directly input the API corresponding to each scene intent included in the stored scene intent set and the received scene intent into the AI ​​model, and output response information through the AI ​​model.

[0211] In some embodiments, in combination with step S803, when the scene intent received by the server is a scene intent identified according to a user instruction, and the server also receives the key information corresponding to the user instruction or the scene intent, the server may also input the user instruction or the key information corresponding to the scene intent into the AI ​​model, and output response information through the AI ​​model. In this way, obtaining response information in combination with the user instruction or the key information of the scene intent can improve the accuracy of the obtained response information.

[0212] In some embodiments, the response information may include but is not limited to one or more APIs, calling these one or more APIs can implement one or more functions, and these one or more functions can meet the user's possible needs under the scene intent. Optionally, the API included in the response information can be part or all of the APIs corresponding to the scene intent stored in the server. In this way, the response information can include multiple APIs, calling these APIs can implement multiple functions, and these multiple functions can meet the needs under the scene intent input by the user, providing users with richer and more comprehensive response information, which can improve the effect of human-computer dialogue.

[0213] Optionally, the response information may also include a multimodal display method of the result of calling each API. It is understood that the multimodal display method may refer to a method of displaying the result of calling one or more APIs by integrating multiple media types (such as but not limited to text, images, videos, audio, etc.). In this way, displaying the results of calling multiple APIs through a multimodal display method can make the response information richer and more intuitive.

[0214] For example, taking the scenario intent as travel intent, the generated response information may include one or more APIs such as setting alarm API (such as alarm application API), booking flight tickets API (such as Feifei application flight ticket query API), booking accommodation API (such as Xie Xie application hotel query API, Feifei application hotel query API, Huahua application hotel query API), etc. The multimodal display method of calling these APIs is through text, pictures and videos. In addition, the video can also support voice command playback.

[0215] For another example: Taking the scenario intent of shopping and consumption as an example, the generated response information may include one or more APIs such as shopping API (such as Taotao application API, Dongdong application API, Pinxixi application API), coupon purchase API (such as public account coupon query API, Tuantuan application coupon query API), price comparison API (such as Taotao application price query API, Dongdong application price query API), product evaluation API (such as Xiaolushu application product evaluation display API), etc. The multimodal display method of calling these APIs is through text, pictures and voice display.

[0216] For another example, if the scenario intent is work and career intent, the generated response information may include one or more APIs such as recruitment API (such as direct recruitment application API, job search website application API), career development information viewing API (such as browser application career development information search API), industry news playback API (such as Huawei video application news playback API), etc. The multimodal display method of calling these APIs is to display through text, pictures and videos.

[0217] Optionally, the response information may also include some fixed-format text, preset pictures, etc. The fixed-format text may be used as a reply statement to the user's instruction, making the response information more complete. Exemplarily, the fixed-format text may be such as "OK", "The following are xx planning suggestions", etc.

[0218] S805: The server sends response information to the electronic device. Correspondingly, the electronic device receives the response information from the server.

[0219] For example, the server can Figure 7 The intent framework system 711 shown sends a response message to the electronic device. Accordingly, the electronic device can Figure 7 The intent framework system 702 is shown receiving response information from the electronic device.

[0220] S806. The electronic device outputs response information.

[0221] For example, the electronic device may display the response information through a display screen, or broadcast the response information through a microphone, a handset, or a headset connected to the electronic device. It can be understood that in the embodiments of the present application, the electronic device outputs the response information, which may refer to the electronic device outputting the corresponding information in response to the user's instruction according to the content in the response information, and does not mean that the information output by the electronic device must be exactly the same as the content in the response information. For example, if the response information contains an API, the information output by the electronic device may be the result of calling the API.

[0222] In some embodiments, the display effect of the result of calling the API in the response information output by the electronic device can be determined by the server. As described above, the response information may include a multimodal display method of the result of calling the API, and the electronic device may output the result of calling the API contained in the response information according to the multimodal display method included in the response information. In this way, the display method of the result of calling the API is determined by the server, and the electronic device can directly perform the corresponding display according to the display method determined by the server, without the need to perform additional operations to determine the display method of the result of calling the API, which can save the power consumption of the electronic device. In addition, the computing power of the server is higher than the computing power of the electronic device, so that a better display method can be obtained, so that the display method of the result of calling the API is in line with the current scene intention, which can further improve the display effect of the result of calling the API.

[0223] In other embodiments, the display effect of the result of the API in the call response information output by the electronic device may be determined by the electronic device. For example, after receiving the API contained in the response information, the electronic device may also set a multimodal display method for the result of calling the one or more APIs, and then output the result of calling the one or more APIs according to the set multimodal display method. Optionally, in this embodiment, the response information received by the electronic device may include the multimodal display method determined by the server, or may not include the multimodal display method determined by the server.

[0224] As a possible implementation, the electronic device can randomly set the multimodal display mode. As another possible implementation, the electronic device can set the multimodal display mode according to the environmental scene information of the electronic device. Among them, the environmental scene information can be used to characterize at least one of the environmental conditions around the electronic device and the state of the electronic device. Based on this design, the electronic device sets the multimodal display mode of the API based on the environmental scene information, so that the display result of the set API corresponds to the current environmental scene information. Different display effects can be achieved under different environmental scene information to better meet user needs.

[0225] Exemplarily, the environmental scene information may include, but is not limited to, one or more of the quietness of the environment in which the electronic device is located, the location and / or scene in which the electronic device is located, the communication method used by the electronic device, whether the Bluetooth of the electronic device is turned on, the power of the electronic device, etc. For example, the quietness of the environment in which the electronic device is located may include quiet or noisy. The scene in which the electronic device is located may include, but is not limited to, home scene, in-car scene, shopping mall scene, bookstore scene, subway station scene, company scene, restaurant scene, etc. The communication method used by the electronic device may include, but is not limited to, Wi-Fi network, 5G network, 4G network, etc.

[0226] For example, if the electronic device is in a noisy environment, the user may be more inclined to read text or view pictures because the user may not be able to hear or focus on the video or audio content. In this case, the multimodal display method adopted by the electronic device can be a combination of text and pictures. That is, the electronic device can use a combination of text and pictures to display the results of the API included in the call response information.

[0227] For another example: if the electronic device is in a quiet environment, the user may prefer to watch videos or listen to audio content. In this case, the multimodal display method adopted by the electronic device can be a combination of video and audio. That is, the electronic device can use a combination of video and audio to display the result of calling the API in the response information.

[0228] Optionally, the environmental scene information of the electronic device can be collected by the electronic device, such as the electronic device can obtain the environmental scene information of the electronic device by collecting various information such as audio information, positioning information, environmental image information, and geographic fence information. It can also be determined by other devices and then sent to the electronic device. Of course, the electronic device can also cooperate with other devices to obtain the environmental scene information of the electronic device, and this application does not impose any restrictions on this.

[0229] In some implementations, the electronic device may use artificial intelligence generated content (AIGC) technology to output response information according to a multimodal display method. Of course, the electronic device may also use other technologies to output response information according to a multimodal display method, and the embodiments of the present application are not limited to this.

[0230] Exemplary, combined Fig. 9 , Fig.10 etc., Fig.13 FIG. 1 is a schematic diagram showing an interface of response information output by an electronic device provided in an embodiment of the present application. Fig.13 As shown, the electronic device may display a response interface 1300 , wherein the response interface 1300 includes a card 1302 , and the card 1302 may be used to display the result of calling the API included in the response information. Optionally, the response interface 1300 may also include a user instruction 1301 .

[0231] For example, in card 1302, there are included the result 1303 of calling the setting alarm API, the result 1304 of calling the booking flight API, and the result 1305 of calling the booking accommodation API. It can be understood that the setting alarm API can be an API for implementing the setting alarm function of a certain application, the booking flight API can be an API for implementing the booking flight function of a certain application, and the booking accommodation API can be an API for implementing the booking accommodation function of a certain application.

[0232] The function of setting an alarm can be achieved by calling the result 1303 of calling the alarm setting API. In some examples, when the electronic device displays the result 1303 of calling the alarm setting API, it can automatically set an alarm for the user, such as when the function button 1303a is turned on. In this way, the electronic device has pre-set an alarm for the user at 7 a.m. In other examples, when the electronic device displays the result of calling the alarm setting API, the electronic device has not pre-set an alarm for the user, and the user can make a reservation on his own. For example, the user can directly turn on the function button 1303a, so that the user can reserve an alarm for 7 a.m. Alternatively, the user can also perform an operation such as clicking on the result 1303 of calling the alarm setting API, and in response to the operation, such as Fig.14 As shown in (1), the electronic device can display an alarm interface 1400, and the user can reserve a suitable alarm on the alarm interface 1400.

[0233] The function of booking a flight ticket can be realized by calling the result 1304 of the flight booking API. In some examples, the electronic device can display information of one or more flights by calling the result 1304 of the flight booking API, and the user can select a suitable flight according to the displayed information of one or more flights. For example, if the electronic device detects that the user selects the information of one of the flights, such as the user performs an operation such as clicking on the information of any flight included in the result 1304 of calling the flight booking API. In response to the operation, such as Fig.14 As shown in (2), the electronic device may display the ordering interface 1410 of the flight. The user may complete the ordering of the corresponding flight through the ordering interface 1410. Optionally, the ordering interface 1410 may be a settlement interface for the flight, or may be other interfaces for the flight, such as an economy class or first class selection interface ( Fig.14 (2) is taken as an example), and the embodiments of the present application do not impose any specific limitations on this.

[0234] In other examples, the user can also view more flight information by calling the result 1304 of the flight booking API. For example, the user can perform an operation such as clicking on the result 1304 of the flight booking API. In response to the operation, Fig.14 As shown in (3), the electronic device may display a flight display interface 1420. The user may select a suitable flight to order in the flight display interface 1420.

[0235] The function of accommodation reservation can be realized by calling the result 1305 of booking accommodation API. In some examples, the electronic device can display the information of one or more bookable hotels by calling the result of booking accommodation API, including but not limited to hotel pictures (such as Fig.13In this example, the user can view more hotels by calling the result 1305 of the booking accommodation API. In some examples, the user can perform operations such as clicking on the result 1305 of the booking accommodation API. In response to the operation, such as Fig.14 As shown in (4), the electronic device may display a hotel ordering interface 1430. The user may select a suitable hotel to order through the hotel ordering interface 1430.

[0236] In other examples, the user may also select a suitable hotel by displaying information of one or more bookable hotels from the result of calling the accommodation reservation API. For example, if the electronic device detects an operation such as a user clicking on the information of a hotel, in response to the operation, the electronic device may display the hotel's ordering interface, and the user may select a suitable room to book in the hotel's ordering interface.

[0237] Optionally, card 1302 may also include texts to remind users of weather conditions, such as "Because Beijing is hot and it may rain, we recommend that you bring summer clothes and rain gear to cope with different weather conditions. When going out, pay attention to sun protection and hydration to avoid heat stroke." and / or texts in fixed formats, such as "OK" and "The following are travel planning suggestions."

[0238] Optionally, the card 1302 may also include a button for implementing the functions of one or more results of calling the API included in the card 1302, such as a one-click travel button 1306. The electronic device detects that the user performs an operation such as clicking on the one-click travel button 1306. In response to the operation, the electronic device may book the alarm (such as a 7 o'clock alarm) displayed in the result 1303 of calling the alarm setting API, and / or the electronic device may choose to book a certain flight displayed in the result 1304 of calling the flight booking API, and / or the electronic device may choose to book a certain hotel displayed in the result 1305 of calling the accommodation booking API, etc. Optionally, the electronic device may also display the settlement page of the air ticket, hotel, etc., or directly display the successful booking page, etc.

[0239] Based on the above technical solution, after receiving a user instruction, the electronic device can obtain response information based on the scene intent identified by the user instruction. The response information includes multiple APIs. Calling these multiple APIs can realize multiple functions. These multiple functions can be used to meet the user's needs under the scene intent. Based on the scene intent, the user is provided with the results of calling multiple APIs to realize different functions. The results of calling the APIs are displayed in a multimodal manner, which realizes the recommendation of richer, more comprehensive, convenient and intuitive response information, which can meet user needs and improve the effect of human-computer dialogue.

[0240] In some other embodiments, the response information may be generated based on at least one of the user portrait, the environmental scene information of the electronic device, the scene intent, and the API corresponding to the scene intent. Fig.15 FIG. 2 shows a flow chart of another human-computer interaction method provided in an embodiment of the present application. Fig.15 As shown, the method comprises the following steps:

[0241] S1501. The electronic device receives a user instruction.

[0242] S1502: The electronic device determines the scene intention according to the user instruction.

[0243] For the introduction of step S1501 and step S1502, please refer to Figure 8 The related introduction of step S801 and step S802 is shown.

[0244] S1503: The electronic device sends at least one of the user portrait and the environmental scene information, and the scene intention to the server. Correspondingly, the server receives at least one of the user portrait and the environmental scene information, and the scene intention from the electronic device.

[0245] It can be understood that a user portrait can refer to a digital portrait of an actual user abstracted from the user's behavior data, and the digital portrait can include one or more user tags. Among them, a user tag can be understood as an abstract classification and summary of certain behavioral characteristics of a user.

[0246] For example, if a user often uses an electronic device to watch anime after 12 o'clock in the evening, "late sleep" and "secondary" can be used as user tags for the user. For another example, if a user often uses an electronic device to take a taxi home from the company at 10 o'clock in the evening, "workaholic" can be used as the user tag for the user. For another example, if a user often buys law-related books from a shopping app, the user's major is law, and the user often reads law papers, etc., then the electronic device can use "law" as the user tag for the user. The aforementioned user tags can all be used to build a user profile for the user.

[0247] In some scenarios, when the user does not input any user instruction corresponding to the scenario intent to the electronic device, the electronic device or other device does not have the historical scenario intent corresponding to the user, the response information corresponding to the historical scenario intent, and the user's operation behavior on the response information. It can be understood that in the embodiments of the present application, the historical scenario intent can refer to the scenario intent recognized by the electronic device according to the user instruction before generating the user portrait.

[0248] In this scenario, in some embodiments, one or more user tags can be generated based on the user's behavior data. The user's behavior data can be the behavior data generated by the electronic device before receiving the user instruction corresponding to the scenario intention. For example, the user's behavior data can be generated based on the user's operation record on the electronic device. The user's behavior data can be stored in the electronic device or in other devices.

[0249] Optionally, in this embodiment, different user portraits may correspond to different users.

[0250] In some examples, the electronic device may generate one or more user tags based on the user's behavior data. In other examples, other devices may generate one or more user tags based on the user's behavior data, and the electronic device may obtain the one or more user tags from other devices.

[0251] Furthermore, in some examples, an electronic device may generate a user portrait based on part or all of the above one or more user tags. In other examples, other devices (such as a server) may generate a user portrait based on part or all of the above one or more user tags.

[0252] For example, Fig.16 A schematic diagram of a user portrait provided by an embodiment of the present application is shown. The user labels included in the user portrait include: female, undergraduate, Gemini, sub-healthy, unmarried, lawyer, spicy food, online literature, fitness, pets, weather, anime, two-dimensional, hot pot, etc. It can be understood that Fig.16 The user tags shown in may be only some examples, and the user profile may also include other user tags.

[0253] In other embodiments, the user portrait may also be a user portrait generated by statistically analyzing the behavior data of different users. For example, the electronic device and / or other device may also generate one or more user tags based on the behavior data of different users. Furthermore, the electronic device and / or other device may also generate a user portrait based on part or all of the one or more user tags.

[0254] For example: if 80% of users like to take the high-speed rail, 10% of users like to take the train, and 10% of users like to take the plane, then "like to take the high-speed rail" will be used as the user's user tag. Another example: if 90% of users like to stay in hotels and 20% of users like to stay in homestays, then "like to stay in hotels" will be used as the user's user tag. Another example: if 70% of users often choose to travel with multiple people and 30% of users often choose to travel alone, then "like to travel alone" will be used as the user's user tag. The aforementioned user tags can be used to build a user profile of the user.

[0255] Optionally, in this embodiment, different users may correspond to the same user portrait.

[0256] In other scenarios, after the user inputs one or more user instructions corresponding to scenario intents to the electronic device, the electronic device or other device may store one or more of the historical scenario intents, the response information corresponding to the historical scenario intents, the user's operation behavior on the response information, etc. It is understood that the user's operation behavior on the response information may include but is not limited to the user's operation record on the result of calling the API contained in the response information, etc.

[0257] Optionally, the historical scene intention, the response information corresponding to the historical scene intention, the user's operation behavior on the response information, etc. can be stored in a directory such as Figure 7 In the intent framework system 702 and / or intent framework system 711 shown. For example: after the intent framework system 702 sends a scenario intent to the server, and after receiving the response information of the scenario intent from the server, these scenario intents and the response information corresponding to the scenario intent can be stored. These stored scenario intents are historical scenario intents, and the response information is the response information corresponding to the historical scenario intent. Optionally, the user's operation behavior on the response information can be reported to the intent framework system 702 and / or the intent framework system 711 by the system application and / or the third-party application. For example: after the user operates on the result of calling the API included in the response information, the application corresponding to the API (such as a system application, a third-party application, etc.) can collect the user's operation behavior and report these operation behaviors to the intent framework system 702 and / or the intent framework system 711.

[0258] In this scenario, in some embodiments, the electronic device and / or other device (such as including but not limited to the server, etc.) can generate a user portrait according to one or more of the historical scene intent, the response information corresponding to the historical scene intent, the user's operation behavior on the response information, etc. For example, the electronic device and / or other device can determine one or more user tags according to one or more of the historical scene intent, the response information corresponding to the historical scene intent, the user's operation behavior on the response information, etc., and then generate a user portrait according to part or all of the one or more user tags. In this way, by combining the historical scene intent, the response information corresponding to the historical scene intent, the user's operation behavior on the response information (such as the user's operation on the result of calling the API included in the response information, such as: setting the result of the called API, etc.), etc., the generated user portrait can reflect the user's real preferences or needs under one or more scene intents, and then determine the user's response information under a certain scene intent according to the user portrait that can reflect the user's real preferences or needs under one or more scene intents, so that the response information can better meet the user's needs under the scene intent, and improve the accuracy and pertinence of the response information.

[0259] For example, under the travel intent, if the user often sets an alarm after 7 a.m., often chooses first class, often takes flights of xx airline, often books homestays, often chooses to travel with two people, and books sea view rooms the most, then "don't like to get up before 7:00", "like comfortable seats", "like to take flights of xx airline", "like homestays", "like to travel with two people", "like sea view rooms", etc. can be used as user tags. Another example: under the shopping and consumption intent, the user often buys jade, often watches videos of pet dogs, spends about 1,000 each time shopping, and often goes to xx shopping malls, etc., then "like jade", "like pet dogs", "spend about 1,000 each time", "like to visit xx shopping malls", etc. can be used as user tags. Another example: under the work and career intent, the user often searches for information about programming, often applies for front-end development positions, and the location of the job application is Beijing. The user often buys books related to the computer field, then "good at becoming", "like front-end development positions", "work intention is Beijing", etc. can be used as user tags. The above user tags can also be used to build user portraits of users.

[0260] For example, Fig.17 A schematic diagram of another user portrait provided by an embodiment of the present application is shown. The user labels included in the user portrait include: undergraduate, good at programming, like homestay, etc., which will not be introduced one by one here. It can be understood that Fig.17 The user tags shown in may be only some examples, and the user portrait may also include other user tags.

[0261] Optionally, in this embodiment, the electronic device and / or other devices may also determine more user tags in combination with other user behavior data, and then determine the user portrait, so that the obtained user portrait can be more comprehensive and accurate.

[0262] Optionally, in this embodiment, different users may correspond to different user portraits. In this way, for the same scenario intention, different users correspond to different user portraits, and the response information output according to different user portraits is also different. In other words, different response information can be recommended for different users, which can flexibly respond to the needs of various users and avoid the homogenization of generated response information.

[0263] Optionally, since the user's behavior may be different at different times, that is, the user may have different preferences at different times, the above user tags can be updated regularly or dynamically. Correspondingly, the user portrait composed of user tags can also be updated regularly or dynamically, and this application does not limit the specific update timing and update method. In this way, by updating the user portrait according to the user's current preferences, a more accurate user portrait can be obtained, and then more accurate response information can be obtained when determining the response information based on the user portrait.

[0264] In some embodiments, where there is currently no Fig.17 When the user portrait shown in the example above is Fig.16 The user portraits shown in the following examples are available. Fig.17 When the user portrait shown in the example above is Fig.17 Of course, there are also user portraits such as Fig.16 as well as Fig.17 When using the user portrait shown in the example above, you can also use Fig.17 The user portrait shown. Fig.17 The user portrait can better represent the real needs of the user. Therefore, by adopting the user portrait, the recommended response information can be more accurate.

[0265] For an introduction to the environmental scene information described in step S1503, please refer to Figure 8 The introduction of the environmental scene information described in step S806 is shown.

[0266] For example: electronic devices can also use Figure 7 The intention framework system 702 sends at least one of the user portrait and the environment scene information, as well as the scene intention to the server. Figure 7The intent framework system 711 shown receives at least one of a user portrait and environmental scene information from an electronic device, as well as scene intent.

[0267] In a specific embodiment, the intent framework system 702 may use a preset data structure to send at least one of a user portrait and environmental scene information, as well as a scene intent to a server. That is, the intent framework system 702 may send first information containing at least one of a user portrait and environmental scene information, as well as a scene intent to a server, and the data structure of the first information may use a preset data structure. Exemplarily, the preset data structure may include the following multiple information: EnvIntents: represents the scene intent; EnvParams: represents the environmental scene information; UserPortrait: represents the user portrait information (such as the user tags contained in the user portrait, etc.).

[0268] Optionally, the preset data structure may also include CheckCode: representing a security check code. The security check code can be used by the server to verify messages from the electronic device including at least one of the user portrait and environmental scene information, as well as scene intentions, etc. After the verification is passed, for example, the security check code is correct, the server executes step S1504 and subsequent steps. In this way, the data structure contains a security check code, which can ensure the security of the information sent by the electronic device to the server and prevent the information from being tampered with.

[0269] Optionally, the data structure may also include DeviceType: characterizing the type of electronic device. Exemplarily, the type of electronic device may include, but is not limited to, various types such as mobile phones, tablets, and watches. Optionally, the type of electronic device may also be combined with the information described in the embodiments of the present application to jointly determine the response information, such as including but not limited to the API included in the response information, the multimodal display method of the result of calling the API, etc. It can be understood that for multiple different electronic devices, they may have the same account (such as a Huawei account). For example: for the same user's mobile phone, tablet, watch, etc., the corresponding user account may be the same. Therefore, some data of these different electronic devices may be shared. In order to prevent the subsequent server from determining the response information for these different electronic devices to be exactly the same, that is, in order to output different response information on different electronic devices of the same account, the electronic device can send its own device type to the server to further improve the output effect of the response information.

[0270] Combined with the data structure described above, taking the user instruction "I am going to travel to Beijing at 8:30 tomorrow morning, please give me some travel suggestions" as an example, the information sent by the intent framework system to the server may include the following: CheckCode: 123321; EnvIntents: travel intention, "I am going to travel to Beijing at 8 o'clock tomorrow morning, please help me arrange it"; EnvParams: environment: quiet, network: Wi-Fi, Bluetooth: on...; UserPortrait: "Don't like to get up before 7:00", "Like comfortable seats", "Like jade", "Like pet dogs"...DeviceType: mobile phone.

[0271] It is understood that in step S1503, the electronic device sends the user portrait and environmental scene information to the server as an example. In other embodiments, the server may also obtain the user portrait and environmental scene information from other devices other than the electronic device, or the server may determine the user portrait and environmental scene information by itself, and the embodiments of the present application do not impose any restrictions on this.

[0272] For more information about step S1503, please refer to Figure 8 The relevant introduction of step S803 is shown.

[0273] S1504. The server generates response information based on at least one of the user portrait and environmental scene information, and the scene intention.

[0274] In some embodiments, the server can generate response information based on at least one of the user portrait and the environmental scene information, as well as the scene intent and the API corresponding to the scene intent. Figure 7 The response information generation module 712 generates response information based on at least one of the user portrait and the environmental scene information, as well as the scene intent and the API corresponding to the scene intent. For an introduction to the API corresponding to the scene intent, please refer to Figure 8 Relevant introduction of step S804.

[0275] In a specific embodiment, the server may also input at least one of the user portrait and environmental scene information, as well as the scene intent and the API corresponding to the scene intent into the AI ​​model, and output the response information through the AI ​​model. In this way, by combining the user portrait and at least one of the environmental scene information, as well as the scene intent and the API corresponding to the scene intent to determine the response information, richer and more comprehensive response information can be obtained, which can meet user needs and improve the effect of human-computer dialogue.

[0276] In some specific implementations, the server may use a method such as Figure 7The intent framework system 711 shown will receive at least one of the user portrait and the environmental scene information, as well as the scene intent and the API input AI model corresponding to the scene intent. The server can also send the response information output by the AI ​​model to Figure 7 In the intention framework system 711 shown.

[0277] Optionally, in this specific implementation, before the intent framework system inputs the user portrait, scenario intent, and the API corresponding to the scenario into the AI ​​model, the intent framework system may match the user tags contained in the user portrait with the API corresponding to the scenario intent, that is, determine which user tags are associated with which APIs. Of course, the association operation can also be performed by the AI ​​model, and the embodiments of the present application are not limited to this. In this way, when displaying an API, the specific display effect of the API can be determined based on the user tag associated with the API, so that the response information can be matched with the user portrait, that is, it corresponds to the user's preferences, and the response information can better meet the user's needs.

[0278] For example, Fig.18 As shown, taking the travel scenario intent as an example, the user tags associated with the alarm setting API may include "don't like to get up before 7 o'clock", etc., the user tags associated with the flight booking API may include "like comfortable seats", "often choose first class", "like to take flights of xx airline", etc., and the user tags associated with the accommodation booking API may include "like homestays", "like to travel for two people", "like sea view rooms", etc.

[0279] In some embodiments, the response information corresponds to the user profile. For different user profiles, the response information determined under the same scenario intent may be different, for example, including but not limited to different APIs included in the response information. For example, the scenario intent is a travel intent, and the travel intent is "I am going to travel to Beijing at 8:30 tomorrow morning. Please give me some travel suggestions", etc. Fig.19 As shown, for user portrait A, the determined response information includes API for booking high-speed rail tickets, API for booking star-rated hotels, API for booking an alarm at 6 o'clock, and travel reminders including reminders related to carrying keys. For user portrait B, the determined response information includes API for booking airplane tickets, API for booking homestays, API for booking an alarm at 7 o'clock, and travel reminders including health-related reminders. In this way, for the same scenario intention, different users may have different corresponding user portraits, and then output corresponding response information according to different user portraits. In other words, different response information recommendations can be made for different users, which can flexibly respond to the needs of various users and avoid the homogenization of generated response information.

[0280] In some embodiments, the response information corresponds to the environmental scene information. For different environmental scene information, the response information determined under the same scene intent may be different, such as: including but not limited to different APIs included in the response information, different multimodal display methods of the APIs included in the response information, etc. For example, the scene intent is still a travel intent, and the travel intent is "I am going to travel to Beijing at 8:30 tomorrow morning, please give me some travel suggestions", etc. Fig. 20 As shown, for environmental scene information A, for example, the environmental scene information A represents that the electronic device is in a noisy environment, and the determined response information includes hotel picture API, hotel text introduction API, alarm setting API, weather card introduction API, etc. For environmental scene information B, for example: the environmental scene information B represents that the electronic device is in a quiet environment, the determined response information may include hotel video API, travel blogger video blog API, alarm setting API, weather voice broadcast API, etc.

[0281] In this way, under different environmental scene information, the user's preferences may be different, such as: including but not limited to the volume and interface parameters during API display, and then based on different environmental scene information, the appropriate response information and the display method of the response information are recommended, so that the output response information corresponds to the current environmental scene information, which can meet the user's preferences in different environments under the same scene intention, and achieve a "thousand machines, thousands of faces" display effect.

[0282] For more information about step S1504, please refer to Figure 8 The introduction of step S804 is shown.

[0283] S1505: The server sends response information to the electronic device. Correspondingly, the electronic device receives the response information from the server.

[0284] For example, the server can Figure 7 The intent framework system 711 shown sends a response message to the electronic device. Accordingly, the electronic device can Figure 7 The intent framework system 702 is shown receiving response information from the electronic device.

[0285] In some embodiments, the intent framework system 711 may also use a preset data structure to send response information to the electronic device. Exemplarily, the preset data structure may include the following information: Answers: response information representing the scene intent; APIGroup: representing the API to be displayed; PlayMethod: representing the multimodal display method.

[0286] Optionally, the preset data structure may also include CheckCode: representing a security check code. The security check code may be used by the electronic device to check the response information from the server. After the verification is passed, for example, the security check code is correct, the electronic device executes step S1506. In this way, the data structure includes a security check code, which can ensure the security of the information sent by the server to the electronic device and prevent the information from being tampered with.

[0287] Optionally, to avoid the situation where the server fails to update the content of the API interface on the electronic device side, the electronic device may also perform a validity check on the API contained in the response information after receiving the response information. After determining that it is a valid API, the electronic device may successfully call the API and then output the result of calling the API, so that subsequent users can perform some operations on the result of calling the API, etc., to ensure user experience.

[0288] S1506. The electronic device outputs response information.

[0289] In some embodiments, the electronic device may output the result of calling the API included in the response information according to the multimodal presentation method included in the response information.

[0290] Exemplary, combined Fig.19 The user portrait shown, Fig.21 The following is a schematic diagram showing an interface of response information output by an electronic device provided in an embodiment of the present application. Fig.21 (1) is a schematic diagram of the interface for outputting response information generated based on user profile A. Fig.21 As shown in (1), the electronic device may display a response interface 2100, wherein the response interface 2100 includes a card 2102, and the card 2102 may be used to display the result of calling the API included in the response information. Optionally, the response interface 2100 may also include a user instruction 2101.

[0291] Card 2101 includes a result 2103 of calling the setting alarm API, a result 2104 of calling the booking high-speed rail ticket API, and a result 2105 of calling the booking accommodation API. It can be understood that the setting alarm API can be an API for implementing the setting alarm function of a certain application, the booking high-speed rail ticket API can be an API for implementing the booking high-speed rail ticket function of a certain application, and the booking accommodation API can be an API for implementing the booking accommodation function of a certain application.

[0292] The function of setting an alarm can be realized by calling the result 2103 of setting an alarm API. The function of booking a high-speed rail ticket can be realized by calling the result 2104 of booking a high-speed rail ticket API. The function of booking an accommodation can be realized by calling the result 2105 of booking an accommodation API.

[0293] Optionally, card 2102 may also include texts such as "Because the weather in Beijing is hot and it may rain, it is recommended that you bring summer clothes and rain gear to cope with different weather conditions. When going out, remember to bring your keys." to remind the user of weather conditions and to bring messages, and / or fixed-format texts such as "OK" and "The following are travel planning suggestions."

[0294] Fig.21 (2) is a schematic diagram of the interface for outputting response information generated based on user profile B. Fig.21 As shown in (2), the electronic device may display a response interface 2110, wherein the response interface 2110 includes a card 2112, and the card 2112 may be used to display the result of calling the API included in the response information. Optionally, the response interface 2110 may also include a user instruction 2111.

[0295] For example, in card 2111, there are included the result 2113 of calling the setting alarm API, the result 2114 of calling the booking flight API, and the result 2115 of calling the booking accommodation API. Please refer to the above description for the introduction of the setting alarm API, the booking flight API, and the booking accommodation API.

[0296] The function of setting an alarm can be realized by calling the result 2113 of the setting alarm API. The function of booking a flight ticket can be realized by calling the result 2114 of the booking flight ticket API. The function of booking a stay can be realized by calling the result 2115 of the booking accommodation API.

[0297] For example, if the user tags contained in user profile A reflect that the user does not like to get up later than 6 o'clock, prefers to take the high-speed rail, likes to live in a homestay, etc., then Fig.21 As shown in (1), the result 2103 of calling the setting alarm API contained in the response information recommends an alarm at 6 o'clock for the user, the result 2104 of calling the high-speed rail ticket API recommends a homestay for the user, and the result 2105 of calling the booking accommodation API recommends a homestay for the user.

[0298] If the user profile B contains labels that reflect that the user does not like to get up before 7 o'clock, prefers to take a plane, likes comfortable hotels, etc., then Fig.21 As shown in (2), the result 2113 of calling the setting alarm API contained in the response information recommends a 7 o'clock alarm to the user, the result 2114 of calling the booking flight API recommends a star-rated hotel to the user, and the result 2115 of calling the booking accommodation API recommends a star-rated hotel to the user.

[0299] Optionally, under different user profiles, the user's display habits for the results of calling the API, such as but not limited to the habits of adjusting the volume, etc., may also be different. In this case, the response information may also include indication information, which can be used to indicate the volume of the electronic device. The volume of the electronic device contained in the indication information may also be different. For example: when the electronic device displays Fig.21 When the response interface shown in (1) is displayed, the volume of the electronic device can be 70%. Fig.21 When the response information shown in (2) is output, the volume of the electronic device can be 30%. In this way, the electronic device adjusts the volume of the electronic device when outputting the response information according to the user portrait, so that the volume of the electronic device can meet the actual habits of the user.

[0300] Of course, under the same user profile, for different environmental scene information, the user's display habits for the results of calling the API, such as but not limited to the habits of volume adjustment, etc., may also be different. In this case, for the same user profile, but under different environmental scene information, the volume of the electronic device contained in the indication information in the response information may also be different.

[0301] In this way, generating response information according to user portraits can make the recommendation of response information more meet the actual needs of users, and can implement targeted recommendation of response information for different user portraits, making the response information richer.

[0302] For example, Fig. 22 FIG. 2 is a schematic diagram showing an interface of another response information output by an electronic device provided in an embodiment of the present application. For example, when the environmental scene information of the electronic device indicates that the electronic device is in a noisy environment, the response information output by the electronic device may be, for example, Fig. 22 As shown in (1) in FIG. 2 . The electronic device may display a response interface 2200 , wherein the response interface 2200 includes a card 2202 , and the card 2202 may be used to display the result of calling the API included in the response information. Optionally, the response interface 2200 may also include a user instruction 2201 .

[0303] For example, in card 2201, there are results 2203 of calling the setting alarm API, results 2204 of calling the booking flight API, and results 2205 of calling the booking accommodation API. Due to the noisy environment, the user may not be able to hear or focus on the video or audio content, and the user may prefer to read text or view pictures. Fig. 22 As shown in (1), the result 2203 of calling the alarm setting API, the result 2204 of calling the flight booking API, the result 2205 of calling the accommodation booking API, etc. are displayed using text and pictures.

[0304] For example, when the environmental scene information of the electronic device indicates that the electronic device is in a quiet environment, the response information output by the electronic device may be such as Fig. 22 As shown in (2) in FIG. 2 . The electronic device may display a response interface 2210 , wherein the response interface 2210 includes a card 2212 , and the card 2212 may be used to display the result of calling the API included in the response information. Optionally, the response interface 2210 may also include a user instruction 2211 .

[0305] For example, in card 2212, there are results 2203 of calling the alarm setting API, results 2204 of calling the flight booking API, and results 2205 of calling the accommodation booking API. In a quiet environment, the user may prefer to watch a video or listen to an audio. Fig. 22 As shown in (1), the result 2213 of calling the setting alarm API, the result 2214 of calling the booking flight API, the result 2215 of calling the booking accommodation API, etc. are displayed using audio and video. For example, the electronic device can voice broadcast the content of the result 2213 of calling the setting alarm API "The alarm has been set for you at 7 am", and / or voice broadcast the flight information included in the result 2214 of calling the booking flight API, and / or display the video related to homestay in the result 2215 of calling the booking accommodation API, such as but not limited to the homestay recommendation video, homestay booking guide, etc.

[0306] about Fig.21 as well as Fig. 22 For an introduction to other content, please refer to Fig.13 An introduction to the corresponding content shown.

[0307] In this way, response information is generated according to the environmental scene information, that is, the output response information can be adjusted according to the environmental scene information, which can achieve the recommendation of response information suitable for the current environmental scene information, so that the response information can meet the user's preferences under the current environmental scene information.

[0308] Based on the above technical solution, response information is recommended by combining scene intent, API corresponding to scene intent, user portrait, environmental scene information and other information. The response information can correspond to the user portrait and environmental scene information, and can be more personalized and targeted to meet the user's actual preferences while meeting the user's needs under the input scene intent.

[0309] In other embodiments, the server can also combine various notification messages, user's operation behavior on the response information, etc. to assist in determining the response information. For example: the server can input various notification messages, user's operation behavior data on the response information, and the information described in the above embodiments into the AI ​​model, and output the response information through the AI ​​model. In this way, the server also generates response information in combination with various notification messages, user's operation behavior on the response information, etc., which can make the response information richer and more comprehensive, and further improve the recommendation results of the response information.

[0310] For example, the notification message may include, but is not limited to, weather notification, flight delay notification, train delay notification, commodity discount notification, traffic congestion notification, and other notification messages. The notification message may be reported by a third-party application or a system application to a server such as Figure 7 The intent framework system 702 and / or the intent framework system 711 shown. Of course, in order to ensure the privacy security of the user, the various notification messages that the server can obtain, the user's operation behavior on the response information, etc. can be desensitized data.

[0311] It can be understood that various operations performed by the server in the embodiment of the present application can also be performed electronically, and the embodiment of the present application does not limit this.

[0312] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of the method. It is understandable that, in order to realize the above functions, the device includes a hardware structure and / or software module corresponding to the execution of each function. In combination with the units and algorithm steps of each example described in the embodiment disclosed in this application, the embodiment of the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer-driven hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the technical solution of the embodiment of the present application.

[0313] As another possible division, Fig.23 2 is a schematic diagram of a structure of a device 2300 provided in an embodiment of the present application, and the device 2300 can be used to implement the methods described in the above method embodiments.

[0314] In a possible example, taking the device 2300 as an electronic device, the processing unit 2301 is configured to execute the support device 2300 to execute Figure 8 Step S801, step S802 and step S806 in the above embodiment. And / or, the processing unit 2301 is used to execute the support device 2300 to execute Fig.15 and / or the processing unit 2301 is further configured to support the device 2300 in executing other steps executed by the electronic device in the embodiment of the present application.

[0315] Communication unit 2302, used to support device 2300 to execute Figure 8 Step S803, step S805 in. And / or, the communication unit 2302 is used to support the device 2300 to perform Fig.15 Step S1503 and step S1505 in the embodiment of the present application. And / or, the communication unit 2302 is also used to support the device 2300 to execute other steps executed by the electronic device in the embodiment of the present application.

[0316] In another possible example, taking the device 2300 as a server, the processing unit 2301 is used to execute the support device 2300 to execute Figure 8 and / or, the processing unit 2301 is configured to execute the support device 2300 to execute Fig.15 And / or, the processing unit 2301 is further configured to support the device 2300 in executing other steps executed by the server in the embodiment of the present application.

[0317] Communication unit 2302, used to support device 2300 to execute Figure 8 Step S803, step S805 in. And / or, the communication unit 2302 is used to support the device 2300 to perform Fig.15 Step S1503 and step S1505 in the embodiment of the present application. And / or, the communication unit 2302 is also used to support the device 2300 to execute other steps executed by the server in the embodiment of the present application.

[0318] Optional, Fig.23 The device 2300 shown may also include a storage unit 2303, which stores a program or instruction. When the processing unit 2301 executes the program or instruction, Fig.23 The device 2300 shown can execute the method executed by the electronic device or server in the above-mentioned embodiments.

[0319] Optional, Fig.23 The device 2300 shown may also include an output unit (not shown in the figure). The output unit may be used to support the device 2300 to output various information. Optionally, the output unit may be specifically implemented as a display unit, an audio output unit, etc.

[0320] Fig.23 The technical effects of the device 2300 shown can refer to the technical effects of the method described in the above embodiment, and will not be repeated here. Fig.23The processing unit 2301 involved in the device 2300 shown can be implemented by a processor or a processor-related circuit component, which can be a processor or a processing module. The communication unit 2302 can be implemented by a transceiver or a transceiver-related circuit component, which can be a transceiver or a transceiver module. The output unit can be implemented by related components such as a display screen and an audio module.

[0321] The present application also provides a chip system, such as Fig.24 As shown, the chip system includes at least one processor 2401 and at least one interface circuit 2402. The processor 2401 and the interface circuit 2402 can be interconnected via lines. For example, the interface circuit 2402 can be used to receive signals from other devices. For another example, the interface circuit 2402 can be used to send signals to other devices (such as processor 2401). Exemplarily, the interface circuit 2402 can read instructions stored in the memory and send the instructions to the processor 2401. When the instruction is executed by the processor 2401, the device can execute the various steps performed by the electronic device or server in the above-mentioned embodiment. Of course, the chip system may also include other discrete devices, which are not specifically limited in the embodiments of the present application.

[0322] Optionally, the processor in the chip system may be one or more. The processor may be implemented by hardware or by software. When implemented by hardware, the processor may be a logic circuit, an integrated circuit, etc. When implemented by software, the processor may be a general-purpose processor implemented by reading software code stored in a memory.

[0323] Optionally, the memory in the chip system may be one or more. The memory may be integrated with the processor or may be separately provided with the processor, which is not limited in the present application. Exemplarily, the memory may be a non-transient processor, such as a read-only memory ROM, which may be integrated with the processor on the same chip or may be provided on different chips. The present application does not specifically limit the type of memory and the arrangement of the memory and the processor.

[0324] Exemplarily, the chip system can be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD) or other integrated chips.

[0325] It should be understood that each step in the above method embodiment can be completed by an integrated logic circuit of hardware in a processor or by instructions in the form of software. The method steps disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware processor, or by a combination of hardware and software modules in a processor.

[0326] An embodiment of the present application also provides a computer storage medium, in which computer instructions are stored. When the computer instructions are executed on a device, the device executes the method described in the above method embodiment.

[0327] An embodiment of the present application provides a computer program product, which includes: a computer program or instructions, when the computer program or instructions are executed on a computer, the computer executes the method described in the above method embodiment.

[0328] In addition, an embodiment of the present application also provides a device, which can specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor can execute the computer-executable instructions stored in the memory so that the device executes the methods in the above-mentioned method embodiments.

[0329] An embodiment of the present application also provides a communication system, including an electronic device and a server, wherein the electronic device is used to execute the method executed by the electronic device in the above method embodiment, and the server is used to execute the method executed by the server in the above method embodiment.

[0330] Among them, the equipment, computer storage medium, communication system, computer program product or chip provided in this embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above and will not be repeated here.

[0331] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0332] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The various embodiments can be combined with each other or referenced to each other without conflict. The device embodiments described above are merely schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0333] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0334] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0335] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0336] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A human-computer interaction method, characterized in that: Applied to electronic equipment, the method comprises: Receive user instructions; In a case where the user instruction represents a first scenario intention, acquiring a plurality of target application programming interfaces APIs according to the first scenario intention, the plurality of target APIs corresponding to the first scenario intention; Output the results of calling the multiple target APIs, and the results of calling the multiple target APIs are used to realize the user needs corresponding to the first scenario intention.

2. The method according to claim 1, characterized in that The acquiring the multiple target APIs according to the first scenario intent includes: The multiple target APIs are acquired according to the user portrait and / or the environmental scene information of the electronic device, and the first scene intention, wherein the environmental scene information of the electronic device is used to characterize the environmental conditions around the electronic device and at least one of the states of the electronic device.

3. The method according to claim 2, characterized in that The acquiring the multiple target APIs according to the user portrait and / or the environmental scene information of the electronic device and the first scene intention includes: Sending the user portrait and / or the environmental scene information of the electronic device, and the first scene intention to a server; The plurality of target APIs are received from the server.

4. The method according to any one of claims 1 to 3, characterized in that: The output results of calling the multiple target APIs include: The results of calling the multiple target APIs are output in a multimodal display manner, wherein the multimodal display manner is a method of using multiple display APIs in text, picture, audio, and video.

5. The method according to claim 4, characterized in that Before outputting the results of calling the multiple target APIs in a multimodal display manner of the results of calling the multiple target APIs, the method further includes: A multimodal presentation of results of calling the plurality of target APIs is received from the server.

6. The method according to claim 4 or 5, characterized in that: The multimodal display method corresponds to the environmental scene information of the electronic device.

7. The method according to claim 6, characterized in that If the environmental scene information of the electronic device indicates that the electronic device is in a first environmental state, the multimodal display mode is the first display mode; If the environmental scene information of the electronic device indicates that the electronic device is in a second environmental state, and the second environmental state is different from the first environmental state, then the multimodal display mode is a second display mode, and the second display mode is different from the first display mode.

8. The method according to claim 7, characterized in that The first environmental condition is a quiet environment, and the first display method is to use video and / or audio to display the results of calling the multiple target APIs; the second environmental condition is a noisy environment, and the second display method is to use text and / or pictures to display the results of calling the multiple target APIs.

9. The method according to any one of claims 2 to 8, characterized in that: The multiple target APIs correspond to the user portraits.

10. The method according to claim 9, characterized in that If the user portrait is a first user portrait, the multiple target APIs are multiple first APIs; If the user portrait is a second user portrait, and the second user portrait is different from the first user portrait, then the multiple target APIs are multiple second APIs, and the multiple second APIs and the multiple first APIs include different APIs.

11. The method according to any one of claims 2 to 10, characterized in that: Before receiving the user instruction, the method further includes: According to the second scenario intention, the API corresponding to the second scenario intention, and the user portrait are generated by multiple operation records of the user on the results of calling the API corresponding to the second scenario intention, where the second scenario intention is the scenario intention obtained before receiving the user instruction.

12. The method according to any one of claims 1 to 11, characterized in that: Before acquiring multiple target APIs according to the first scenario intent, the method further includes: The first scene intention is determined according to a preset scene intention set and the user instruction, and the preset scene intention set includes multiple scene intentions.

13. The method according to claim 3, characterized in that The user portrait and / or the environmental scene information of the electronic device, and the first scene intention are included in the first information. The first information also includes a security verification code and at least one of the type of the electronic device. The security verification code is used to verify the first information, and the type of the electronic device is used to determine the multiple target APIs.

14. A human-computer interaction method, characterized in that: Applied to a server, the method comprises: Receiving first information from an electronic device, wherein the first information includes a first scenario intention; Generate second information according to the first information and the API corresponding to the first scenario intent, wherein the second information includes a plurality of target APIs, the plurality of target APIs correspond to the first scenario intent, and the plurality of target APIs are used to implement user needs corresponding to the first scenario intent; The second information is sent to the electronic device.

15. The method according to claim 14, characterized in that The first information also includes a user portrait and / or environmental scene information of the electronic device, wherein the environmental scene information of the electronic device is used to characterize at least one of the environmental conditions around the electronic device and the state of the electronic device.

16. The method according to claim 15, characterized in that The second information also includes a multimodal display method of the results of calling the multiple target APIs, and the multimodal display method is a method of using multiple display APIs in text, pictures, audio, and video.

17. The method according to claim 16, characterized in that The multimodal display method corresponds to the environmental scene information of the electronic device.

18. The method according to claim 17, characterized in that If the environmental scene information of the electronic device indicates that the electronic device is in a first environmental state, the multimodal display mode is the first display mode; If the environmental scene information of the electronic device indicates that the electronic device is in a second environmental state, and the second environmental state is different from the first environmental state, then the multimodal display mode is a second display mode, and the second display mode is different from the first display mode.

19. The method according to claim 18, characterized in that The first environmental condition is a quiet environment, and the first display method is to use video and / or audio to display the results of calling the multiple target APIs; the second environmental condition is a noisy environment, and the second display method is to use text and / or pictures to display the results of calling the multiple target APIs.

20. The method according to any one of claims 15 to 19, characterized in that The multiple target APIs correspond to the user portraits.

21. The method according to claim 20, characterized in that If the user portrait is a first user portrait, the multiple target APIs are multiple first APIs; If the user portrait is a second user portrait, and the second user portrait is different from the first user portrait, then the multiple target APIs are multiple second APIs, and the multiple second APIs and the multiple first APIs include different APIs.

22. The method according to any one of claims 14 to 21, characterized in that Before generating second information according to the first information and the API corresponding to the first scenario intention, the method further includes: According to a preset scene intent set, the API corresponding to the preset scene intent set, the first scene intent determines the API corresponding to the first scene intent, wherein the preset scene intent set includes multiple scene intents, and the API corresponding to the preset scene intent set is the API corresponding to each scene intent in the multiple scene intents.

23. The method according to any one of claims 14 to 22, characterized in that The generating second information according to the first information and an API corresponding to the first scene intention includes: The first information and the API corresponding to the first scene intention are input into a preset model, and the second information is output through the preset model.

24. The method according to any one of claims 14 to 23, characterized in that The first information also includes at least one of a security verification code and a type of the electronic device, the security verification code is used to verify the first information, and the type of the electronic device is used to determine the multiple target APIs.

25. An electronic device, characterized in that: The electronic device comprises a processor and a memory, wherein the memory is coupled to the processor, the memory is used to store computer program code, the computer program code comprises computer instructions, and the processor reads the computer instructions from the memory so that the electronic device executes the method as described in any one of claims 1 to 13.

26. A server, characterized in that: It includes a processor, a memory and a communication interface, the memory and the communication interface are coupled to the processor, the communication interface is used to communicate with other devices, the memory is used to store computer program code, the computer program code includes computer instructions, and the processor reads the computer instructions from the memory so that the server executes the method as described in any one of claims 14-24.

27. A computer-readable storage medium, characterized in that: The computer-readable storage medium comprises a computer program, which, when executed on an electronic device, enables the electronic device to execute the method as described in any one of claims 1 to 13; or, when executed on a server, enables the server to execute the method as described in any one of claims 14 to 24.

28. A computer program product, characterized in that The computer program product comprises: a computer program or instructions, which, when executed on a computer, enables the computer to execute the method according to any one of claims 1 to 13, or enables the computer to execute the method according to any one of claims 14 to 24.

29. A communication system, characterized in that: It comprises an electronic device and a server, wherein the electronic device is used to execute the method as claimed in any one of claims 1 to 13, and the server is used to execute the method as claimed in any one of claims 14 to 24.

Images