Human-computer interaction method and device

Abstract

The embodiment of the invention provides a man-machine interaction method and device. The method comprises the following steps: performing emotion semantic analysis on obtained man-machine conversation data of a current conversation user, and determining a current emotion state of the current conversation user; and generating and displaying a corresponding visual element based on the current emotional state of the current dialogue user. According to the technical scheme provided by the embodiment of the invention, different emotional states of the user can be mapped to different visual elements, and visual feedback expression of the emotions of the user is realized, so that personalized and vivid feedback can be performed on the user in a dialogue process between the user and the AI.

Description

Technical Field

[0001] This specification relates to the field of artificial intelligence technology, and particularly to a human-computer interaction method and an electronic device. Background Art

[0002] With the development of AI (Artificial Intelligence) technology, the use of AI dialogue applications in fields such as mental health and emotional companionship is becoming increasingly widespread.

[0003] In related technical solutions, a dialogue is carried out with the user in the form of text or voice through AI technology. For example, when the user inputs a dialogue question in text or voice form, a text or voice dialogue reply to the dialogue question is generated through AI technology. However, in this technical solution, the interaction form mainly based on text or voice feedback lacks personalized feedback on the user's emotions.

[0004] Therefore, how to provide personalized feedback to the user during the user-AI dialogue process has become a technical problem to be solved urgently.

[0005] The content in the background art section is only the information known to the inventor personally, and does not mean that the above information has entered the public domain before the filing date of this disclosure, nor does it mean that it can become the prior art of this disclosure. Summary of the Invention

[0006] This specification provides a human-computer interaction method and an electronic device, which can provide personalized and vivid feedback to the user during the user-AI dialogue process.

[0007] In a first aspect, this specification provides a human-computer interaction method, including:

[0008] Obtaining human-computer dialogue data corresponding to the current round of dialogue on a human-computer interaction interface, where the human-computer dialogue data includes the dialogue data input by the current dialogue user;

[0009] Performing emotional semantic analysis on the human-computer dialogue data to determine the current emotional state of the current dialogue user; and

[0010] Generating corresponding visual elements based on the current emotional state of the current dialogue user, and displaying the visual elements on the human-computer interaction interface.

[0011] In some example embodiments, based on the above solution, the generating corresponding visual elements based on the current emotional state of the current dialogue user includes:

[0012] Determining the element parameters of the visual elements corresponding to the current emotional state based on the preset correspondence between the current emotional state and the visual elements;

[0013] Generate the visual element corresponding to the current emotional state based on the element parameters of the visual element.

[0014] In some exemplary embodiments, based on the above solution, the element parameters include one or more of element color, element size, element quantity, element transparency, and element movement trajectory.

[0015] In some exemplary embodiments, based on the above solution, the current emotional state includes an emotional type and an emotional intensity. The emotional semantic analysis of the human-machine dialogue data to determine the current emotional state of the current dialogue user includes:

[0016] Perform emotional semantic analysis on the human-machine dialogue data to determine the emotional type and emotional intensity of the current dialogue user.

[0017] The determining the element parameters of the visual element corresponding to the current emotional state based on the preset correspondence between the current emotional state and the visual element includes:

[0018] Based on the preset correspondence between the emotional type and the element parameters, determine the element color and element transparency of the visual element corresponding to the current emotional state.

[0019] Based on the preset correspondence between the emotional intensity and the element parameters, determine the element size of the visual element corresponding to the current emotional state.

[0020] In some exemplary embodiments, based on the above solution, the emotional intensity includes an intensity level and an intensity duration. The determining the element size of the visual element corresponding to the current emotional state based on the preset correspondence between the emotional intensity and the element parameters includes:

[0021] Based on the preset correspondence between the intensity level and the element parameters, determine the element size of the visual element corresponding to the current emotional state.

[0022] Based on the preset correspondence between the intensity duration and the element parameters, determine the element quantity of the visual element corresponding to the current emotional state.

[0023] In some exemplary embodiments, based on the above solution, the generating the visual element corresponding to the current emotional state based on the element parameters of the visual element includes:

[0024] According to the element movement trajectory of the visual element, generate the visual element that changes along the element movement trajectory on the human-computer interaction interface according to the time stamp.

[0025] In some example embodiments, based on the above solution, the visual element is a bubble.

[0026] In some example embodiments, based on the above solution, the visual element is a bubble, the element parameters include the bubble size, and generating the visual element corresponding to the current emotional state based on the element parameters of the visual element includes:

[0027] If the bubble size of the bubble is greater than a predetermined threshold, then segment the bubble size based on the predetermined threshold;

[0028] Generate at least two bubbles corresponding to the current emotional state based on the segmentation result.

[0029] In some example embodiments, based on the above solution, the current emotional state includes an emotional type and an emotional intensity, and performing emotional semantic analysis on the human-computer dialogue data to determine the current emotional state of the current dialogue user includes:

[0030] Performing emotional semantic analysis on the historical human-computer dialogue data to determine the emotional type and emotional intensity of the current dialogue user,

[0031] Generating a corresponding visual element based on the current emotional state of the current dialogue user includes:

[0032] Determining the element type of the visual element corresponding to the current emotional state based on a preset correspondence relationship between the emotional type of the current dialogue user and the element type of the visual element;

[0033] Determining the element size of the visual element corresponding to the current emotional state based on a preset correspondence relationship between the emotional intensity of the current dialogue user and the element size of the visual element;

[0034] Generating the visual element corresponding to the current emotional state based on the element type and the element size.

[0035] In some example embodiments, based on the above solution, the method further includes:

[0036] When superimposing historical visual elements, gradually reduce the size of the previously generated visual element and increase the transparency of the previously generated visual element according to a time decay factor.

[0037] In some example embodiments, based on the above solution, after generating the visual element, the method further includes:

[0038] Arrange the generated visual elements on the human-computer interaction interface according to a preset layout rule.

[0039] In some example embodiments, based on the above solution, the method further includes:

[0040] In response to a trigger operation for generating an image, determining a plurality of the visual elements previously generated on the human-computer interaction interface, where the trigger operation includes an operation for ending a conversation;

[0041] Generating an image corresponding to the human-computer dialogue data based on the plurality of visual elements.

[0042] In some example embodiments, based on the above solution, the method further includes:

[0043] In response to a trigger operation for the visual element, displaying on the human-computer interaction interface a conversation record of the human-computer dialogue data corresponding to the visual element.

[0044] In a second aspect, this specification further provides an electronic device, including: at least one storage medium storing at least one instruction set for performing human-computer interaction processing; and at least one processor communicatively connected to the at least one storage medium, where, when the electronic device runs, the at least one processor reads the at least one instruction set and executes the human-computer interaction method described in the first aspect of this specification according to the instructions of the at least one instruction set.

[0045] As can be seen from the above technical solutions, for the human-computer interaction method and device provided in the embodiments of this specification, on the one hand, by performing emotional semantic analysis on the human-computer dialogue data of the current dialogue user obtained, the current emotional state of the current dialogue user is determined, and the emotional state of the current dialogue user during the dialogue can be accurately and efficiently determined; on the other hand, based on the current emotional state of the current dialogue user, corresponding visual elements are generated and displayed on the human-computer interaction interface, and different current emotional states of the user can be mapped to different visual elements, realizing a visual feedback expression of the user's emotions. Thus, during the process of the user's dialogue with the AI, personalized and vivid feedback can be provided to the user, and further, deep integration of emotional interaction and visual art creation can be achieved, enhancing the user's emotional immersion.

[0046] Other functions of the human-computer interaction method and device provided in this specification will be partially listed in the following description. According to the description, the content introduced by the following numbers and examples will be obvious to those of ordinary skill in the art. The creative aspects of the human-computer interaction method and device provided in this specification can be fully explained by practicing or using the methods, devices, and combinations described in the detailed examples below. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] To more clearly illustrate the technical solutions in the embodiments of this specification, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of this specification. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0048] Figure 1 FIG. shows a schematic diagram of the implementation environment of a human-computer interaction method provided by an embodiment of this specification;

[0049] Figure 2 FIG. shows a hardware structure diagram of an electronic device 200 provided by an embodiment of this specification;

[0050] Figure 3 FIG. shows a schematic flowchart of a human-computer interaction method provided by some embodiments of this specification;

[0051] Figure 4 FIG. shows a schematic diagram of a human-computer interaction interface provided by some embodiments of this specification;

[0052] Figure 5 FIG. shows a schematic diagram of a human-computer interaction interface for generating bubbles provided by an embodiment of this specification;

[0053] Figure 6 FIG. shows a schematic diagram of an interface for splitting bubbles provided by some other embodiments of this specification;

[0054] Figure 7 FIG. shows a schematic diagram of the interface for generating a picture frame provided by some embodiments of this specification; and

[0055] Figure 8 FIG. shows a schematic flowchart of a human-computer interaction method provided by some other embodiments of this specification. Detailed implementation manners

[0056] The following description provides specific application scenarios and requirements of this specification, aiming to enable those skilled in the art to manufacture and use the content in this specification. For those skilled in the art, various partial modifications to the disclosed embodiments are obvious, and the general principles defined here can be applied to other embodiments and applications without departing from the spirit and scope of this specification. Therefore, this specification is not limited to the shown embodiments, but has the broadest scope consistent with the claims.

[0057] The terms used herein are for the purpose of describing particular example embodiments only and are not limiting. For example, unless the context clearly dictates otherwise, as used herein, the singular forms "a", "an", and "the" may also include the plural forms. When used in this specification, the terms "comprising", "including", and / or "having" mean that the associated integers, steps, operations, elements, and / or components exist, but do not preclude the existence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups in the system / method.

[0058] In view of the following description, these features of this specification and other features, as well as the operations and functions of the related elements of the structure, and the combination and manufacturing economy of the components can be significantly improved. Referring to the accompanying drawings, all of which form a part of this specification. However, it should be clearly understood that the drawings are for illustrative and descriptive purposes only and are not intended to limit the scope of this specification. It should also be understood that the drawings are not drawn to scale.

[0059] The flowcharts used in this specification illustrate the operations implemented by the system according to some embodiments in this specification. It should be clearly understood that the operations of the flowchart may not be implemented in sequence. On the contrary, the operations may be implemented in reverse order or simultaneously. In addition, one or more other operations may be added to the flowchart. One or more operations may be removed from the flowchart.

[0060] First, the noun terms related to one or more embodiments of this specification are explained.

[0061] Emotional semantic analysis: Identifying the emotional state in dialogue data such as dialogue text or dialogue audio through natural language processing technology, for example, emotional tendency (positive / negative) and intensity.

[0062] Emotion-element mapping rule: The corresponding relationship between the predefined emotional state and the visual parameters of visual elements such as the color, size / transparency, etc.

[0063] Dynamic frame generation engine: An algorithm module that automatically renders bubbles and combines them into an artistic picture in the interactive interface based on the emotional data of the current dialogue user.

[0064] Time decay factor: Used to reduce the visual weight of historical visual elements such as bubbles (such as shrinking and fading) over time to ensure dynamic update of the picture.

[0065] In the related technical solutions, dialogue is carried out with the user in the form of text or voice through AI technology. However, the form of human-computer interaction in this technical solution still mainly focuses on text or voice, lacking intuitive visual feedback on the user's emotions.

[0066] Based on the above, the embodiments of this specification provide a human-computer interaction method and an electronic device. On the one hand, by performing emotional semantic analysis on the human-computer dialogue data of the current dialogue user obtained, the current emotional state of the current dialogue user can be determined, which can accurately and efficiently determine the emotional state of the current dialogue user during the dialogue. On the other hand, generating corresponding visual elements based on the current emotional state of the current dialogue user and displaying them on the human-computer interaction interface can map different current emotional states of the user to different visual elements, realizing the visual feedback expression of the user's emotions. Thus, during the process of the user's dialogue with the AI, personalized and vivid feedback can be provided to the user, and furthermore, the deep integration of emotional interaction and visual art creation can be achieved, enhancing the user's emotional immersion.

[0067] Next, the technical solutions of the embodiments of this specification will be described in detail with reference to the accompanying drawings.

[0068] Figure 1 FIG. shows a schematic diagram of the implementation environment of a human-computer interaction method provided by the embodiments of this specification.

[0069] See Figure 1 As shown, the implementation environment 100 may include a terminal 110, a server 130, and a database 140.

[0070] The terminal 110 is connected to the server 130 through a wireless network or a wired network 120. The terminal 110 may be a tablet computer, a laptop computer, a desktop computer, etc., but is not limited thereto.

[0071] The terminal 110 may store data or instructions for executing the human-computer interaction method described in this specification. The terminal 110 may include a hardware device with data information processing capabilities and the necessary programs for driving the hardware device to work.

[0072] The server 130 is an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms, etc. The server 130 provides background services for the application programs running on the terminal 110.

[0073] An integrated development platform is installed on the server 130. The integrated development platform, also known as the integrated development environment (IDE), is an application used to provide a program development environment, generally including tools such as a code editor, a compiler, a debugger, and a human-computer interaction interface. Developers can write program codes (i.e., program development) on the integrated development platform. The integrated development platform server can be a computing device specifically used by the integrated development platform to implement the human-computer interaction method. The server 130 can communicate with the terminal 110 and the database 140 respectively for data communication.

[0074] In addition, the server 130 can store data or instructions for executing the human-computer interaction method described in this specification. The server 130 can include a hardware device with data information processing capabilities and the necessary programs required to drive the operation of this hardware device. Of course, the server 130 can also be only a hardware device with data processing capabilities, or only a program running in the hardware device. In some embodiments, the server 130 can also be used as a plug-in and deployed on the terminal 110. At this time, the server 130 stores data or instructions for executing the human-computer interaction method corresponding to the terminal 110 described in this specification.

[0075] The database 140 may store data and / or instructions. In some embodiments, the database 140 may store human-machine dialogue data corresponding to users. In some embodiments, the database 140 may store data and / or instructions for the server 130 to execute or for executing the human-machine interaction method described in this specification. The terminal 110 and the server 130 have access rights to the database 140, and the terminal 110 and the server 130 may access the data or instructions stored in the database 140 through a network. In some embodiments, the database 140 may be directly connected to the terminal 110 and the server 130. In some embodiments, the database 140 may be a part of the server 130. In some embodiments, the database 140 may include mass storage, removable storage, volatile read-write memory, read-only memory (ROM), or the like, or any combination thereof. Exemplary mass storage may include non-transitory storage media such as magnetic disks, optical disks, solid-state drives, etc. Example removable storage may include flash drives, floppy disks, optical disks, memory cards, zip disks, magnetic tapes, etc. Typical volatile read-write memory may include random access memory (RAM). Example RAM may include dynamic RAM (DRAM), double data rate synchronous dynamic RAM (DDR SDRAM), static RAM (SRAM), thyristor RAM (T-RAM), and zero-capacitor RAM (Z-RAM), etc. Exemplary ROM may include masked ROM (MROM), programmable ROM (PROM), virtual programmable ROM (PEROM), electrically programmable ROM (EEPROM), optical disk (CD ROM), and digital versatile disk ROM, etc.

[0076] Those skilled in the art will appreciate that the number of the above terminals may be more or less. For example, there may be only one of the above terminals, or there may be dozens or hundreds of the above terminals, or even more. In this case, other terminals are also included in the above implementation environment. The embodiments of this specification do not limit the number and device types of the terminals.

[0077] After introducing the implementation environment of the embodiments of this specification, the application scenarios of the embodiments of this specification will be introduced below in combination with the above implementation environment. In the following description process, the terminal is the terminal 110 in the above implementation environment, and the server is the server 130 in the above implementation environment. The technical solutions provided by the embodiments of this specification can be applied to various AI dialogue applications or intelligent agents, such as customer service dialogue applications, emotional companionship applications, mental health applications, or social entertainment applications, etc.

[0078] Taking the application of the technical solution provided in the embodiments of this specification in an emotional companionship application as an example, the current conversation user is a user who logs in to the emotional companionship application, and the terminal 110 obtains the human-computer conversation data of the current round of conversation on the human-computer interaction interface of the emotional companionship application, where the human-computer conversation data includes conversation data input by the current conversation user; performs emotional semantic analysis on the human-computer conversation data to determine the current emotional state of the current conversation user; and generates corresponding visual elements based on the current emotional state of the current conversation user, and displays the visual elements on the human-computer interaction interface.

[0079] It should be noted that the above is explained by taking the application of the technical solution provided in the embodiments of this specification in an emotional companionship application as an example. The technical solution provided in the embodiments of this specification can also be applied to other appropriate applications or intelligent agent scenarios, such as mental health intelligent agents or customer service dialogue applications, and the implementation process belongs to the same inventive concept as the above description, which will not be repeated here.

[0080] It should be noted that the steps in the human-computer interaction method in the example embodiment of this specification may be partially executed by the client, partially executed by the server, or entirely executed by the server or entirely by the client, and this specification does not specifically limit this.

[0081] based on Figure 1 The implementation environment shown below will be combined with Figures 2 - 8 , the human-computer interaction method and electronic device provided by the embodiments of this specification are introduced in detail. It should be noted that the above implementation environment is only shown to facilitate understanding of the spirit and principle of this specification, and the embodiments of this specification are not limited in this regard. On the contrary, the embodiments of this specification can be applied to any applicable scenario.

[0082] Figure 2 2 is a schematic diagram of the structure of an electronic device 200 provided according to some embodiments of this specification. The electronic device 200 can execute the human-computer interaction method described in this specification. The human-computer interaction method is introduced in other parts of this specification. The electronic device 200 can be a general-purpose computer or a special-purpose computer. For example, the electronic device 200 can be a server, a personal computer, a portable computer (such as a notebook computer, a tablet computer, etc.), or other electronic devices with computing capabilities. Of course, the electronic device can be Figure 1 The terminal 110 or the server 130 may also be a terminal device used by multiple developers to develop programs on an integrated development platform.

[0083] The electronic device in this specification may include one or more of the following components: a processor 210, a memory 220, an input device 230, an output device 240, and a bus 250. The processor 210, the memory 220, the input device 230, and the output device 240 may be connected through the bus 250.

[0084] The processor 210 may include one or more processing cores. The processor 210 connects various parts within the entire electronic device using various interfaces and lines. By running or executing instructions, programs, code sets, or instruction sets stored in the memory 220, and by invoking data stored in the memory 220, it executes the human-computer interaction method described in this specification. Optionally, the processor 210 may be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), or programmable logic array (PLA). The processor 210 may integrate a combination of one or several of a central processing unit (CPU), a graphics processing unit (GPU), and a modem, etc. Among them, the CPU mainly processes the operating system, user interface, and application programs, etc.; the GPU is responsible for the rendering and drawing of display content; the modem is used to process wireless communication. It can be understood that the above modem may not be integrated into the processor 210 and may be implemented separately through a communication chip.

[0085] The memory 220 may include random access memory (RAM) and may also include read-only memory (ROM). Optionally, the memory 220 includes a non-transitory computer-readable storage medium. The memory 220 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 220 may include a program storage area and a data storage area. Among them, the program storage area may store instructions for implementing the operating system, instructions for implementing at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the following various method embodiments, etc. The operating system may be the Android system, including a system developed based on the Android system in depth, the IOS system, including a system developed based on the IOS system in depth, or other systems.

[0086] In order for the operating system to distinguish the specific application scenarios of third - party applications, it is necessary to establish data communication between the third - party applications and the operating system, so that the operating system can obtain the current scenario information of the third - party applications at any time, and then perform targeted system resource adaptation based on the current scenario.

[0087] Among them, the input device 230 is used to receive input instructions or data. The input device 230 includes, but is not limited to, a keyboard, a mouse, a camera, a microphone, or a touch device. The output device 240 is used to output instructions or data. The output device 240 includes, but is not limited to, a display device and a speaker, etc. In one example, the input device 230 and the output device 240 can be combined, and the input device 230 and the output device 240 are a touch display screen.

[0088] In addition, those skilled in the art can understand that the structure of the electronic device shown in the above figures does not limit the electronic device. The electronic device may include more or fewer components than shown in the figures, or combine some components, or have different component arrangements. For example, the electronic device also includes components such as a radio frequency circuit, an input unit, a sensor, an audio circuit, a Wireless Fidelity (WiFi) module, a power supply, and a Bluetooth module, which will not be elaborated here.

[0089] Figure 3 The flowchart of a human - machine interaction method provided according to an embodiment of this specification is shown. As before, the electronic device 200 can execute the human - machine interaction method of the embodiment of this specification. Specifically, the processor 210 can read the instruction set stored in its local storage medium, and then execute the human - machine interaction method of the embodiment of this specification according to the provisions of the instruction set. Below, steps S310 to S330 in the human - machine interaction method will be described in detail with reference to the accompanying drawings.

[0090] As Figure 3 shown, in step S310, obtain the human - machine dialogue data of the current round on the human - machine interaction interface.

[0091] In the exemplary embodiment, the human - machine interaction interface is the human - machine interaction interface of an AI dialogue application or an agent, such as the graphical user interface of an emotional companionship application. The current dialogue user is the user currently using the AI dialogue agent, such as the user who logs in to the AI dialogue application. The human - machine dialogue data includes the text data or voice data input by the current dialogue user. The electronic device 200 listens for the user's input events, such as inputting text or voice, through an event listener. In response to the monitored input event, it obtains the human - machine dialogue data of the current round on the human - machine interaction interface. The human - machine dialogue data includes the dialogue data input by the user and the corresponding dialogue data output by the AI dialogue application.

[0092] For example, an event listener for a text input box or an audio input button is pre-set on the human-computer interaction interface. When the current conversation user presses the Enter key in the text input box to submit information, the electronic device 200 monitors this input event and obtains the text information entered in the text input box. Refer to Figure 4 As shown, the human-computer interaction interface is the interface of the AI dialogue intelligent agent "Emotional Trash Can". A text input box and an audio input button are set on this human-computer interaction interface, and event listeners for the text input box and the audio input button are also set. When the electronic device 200 monitors a text or audio input event, it obtains the text information entered in the text input box, or the audio input captured by the microphone.

[0093] In some exemplary embodiments, the electronic device 200 captures and records the responses automatically generated by the AI dialogue intelligent agent, such as text responses or voice response information. Further, the electronic device 200 also captures the context information corresponding to the current turn of the current conversation user, such as the conversation timestamp and the conversation ID, etc. The conversation timestamp is used to record the conversation time of each turn of the conversation, and the conversation ID is used to associate multiple turns of the same conversation.

[0094] It should be noted that although the human-computer interaction data is taken as an example of text data or voice data for illustration, those of ordinary skill in the art should understand that the human-computer interaction data can also be image data, which is also within the scope of the embodiments of this specification.

[0095] In step S320, emotional semantic analysis is performed on the human-computer dialogue data to determine the current emotional state of the current conversation user.

[0096] In the exemplary embodiment, emotional semantic analysis refers to identifying and understanding the type of emotional state contained in the text or voice through natural language processing technology. For example, classifying the emotional state contained in the text or voice through a pre-trained emotion classification model, and the emotion classification model can be a pre-trained natural language processing model, such as a BERT (Bidirectional Encoder Representation from Transformers) model, a support vector machine model, or a GPT (Generative Pre-trained Transformer) model, etc. The emotion classification model is pre-trained using a labeled emotion dataset to ensure that the emotion classification model can accurately identify multiple emotional state categories.

[0097] The current emotional state refers to the user's emotional state corresponding to the current input dialogue data of the user, such as the emotional type. The emotional type includes positive emotions, negative emotions, and neutral emotions, etc. The electronic device 200 performs emotional semantic analysis on the human-machine dialogue data through a pre-trained emotion classification model to determine the current emotional state of the current dialogue user. For example, the electronic device 200 obtains the human-machine dialogue data corresponding to the current round of dialogue, such as the dialogue data input by the current dialogue user in the current round of dialogue and the dialogue data output by the AI dialogue application, extracts dialogue features from the human-machine dialogue data through the emotion classification model. The dialogue features include lexical features, emotional features, and context features, etc. Based on the dialogue features, the current emotional state of the current dialogue user is determined. Among them, the lexical features include emotional words such as "happy", "angry", "sad", etc., the emotional features such as emotional intensity, duration and other features, and the context features include the overall meaning of sentences or paragraphs, and understand the actual meaning of emotional words in a specific context.

[0098] Furthermore, in the exemplary embodiment, the current emotional state includes the emotional type and the emotional intensity. The electronic device 200 obtains the human-machine dialogue data input by the current dialogue user in at least one round of dialogue, and performs emotional semantic analysis on the human-machine dialogue data through a pre-trained emotion classification model to determine the emotional type and the emotional intensity of the current dialogue user. For example, assume that the emotional type includes positive emotions, negative emotions, and neutral emotions, and the emotional intensity includes high, medium, low, etc.

[0099] For example, assume that the human-machine dialogue data input by the current dialogue user obtained by the electronic device 200 is "Today's work is not going well and I'm in a terrible mood.", and performs emotional semantic analysis on the human-machine dialogue data through a pre-trained emotion classification model to determine that the emotional type of the current dialogue user is a negative emotion and the emotional intensity is high.

[0100] It should be noted that although the above emotional type and emotional intensity are used as examples for illustration, those of ordinary skill in the art should understand that the emotional type and emotional intensity can also be other appropriate contents. For example, the emotional type can also be slightly positive or slightly negative, etc., and the emotional intensity can also be in the form of a percentage, such as 0-100%, etc., which is also within the scope of the embodiments of this specification.

[0101] In step S330, corresponding visual elements are generated based on the current emotional state of the current dialogue user and the visual elements are displayed on the human-computer interaction interface.

[0102] In an exemplary embodiment, the visual elements can be visual elements of the same predetermined type or different types. The electronic device 200 determines the element parameters of the corresponding visual elements based on the current emotional state of the current conversation user, generates the corresponding visual elements based on the element parameters of the visual elements, and displays the visual elements on the human-computer interaction interface. If the visual elements are visual elements of the same predetermined type, the element parameters of the visual elements include one or more of element color, element size, element quantity, element transparency, and element movement trajectory. If the visual elements are visual elements of different predetermined types, the element parameters of the visual elements include one or more of element type, element color, element size, element quantity, element transparency, and element movement trajectory.

[0103] Further, in an exemplary embodiment, a preset correspondence relationship between the emotional state and the visual elements is pre-set. For example, an emotion-element mapping rule, which represents the correspondence relationship between the predefined emotional state and the visual parameters such as the color, size / transparency of the visual elements. The electronic device 200 determines the element parameters of the visual elements corresponding to the current emotional state based on the preset correspondence relationship between the current emotional state and the visual elements; generates the visual elements corresponding to the current emotional state based on the element parameters of the visual elements.

[0104] For example, the visual elements are visual elements of the same predetermined type, and the current emotional state includes an emotion type and an emotion intensity. The electronic device 200 determines the element color and element transparency of the visual elements corresponding to the current emotional state based on the preset correspondence relationship between the emotion type and the element parameters; determines the element size and element quantity of the visual elements corresponding to the current emotional state based on the preset correspondence relationship between the emotion intensity and the element parameters, generates the corresponding visual elements based on the determined element parameters, and displays the visual elements on the human-computer interaction interface.

[0105] Table 1 below shows the preset correspondence relationship between the emotional state and the visual elements:

[0106] Table 1. Preset Correspondence Relationship between Emotional State and Visual Elements

[0107] Emotion type Element color Transparency Emotion intensity Element size Positive emotion Warm color system such as orange / yellow Lower High Large Neutral emotion Natural color system such as green / white Medium Medium Medium Negative emotion Cool color system such as blue / violet Higher Low Small

[0108] In an exemplary embodiment, the visual elements are bubbles. The electronic device 200 determines the bubble color and bubble transparency corresponding to the current emotional state based on the preset correspondence relationship between the emotion type and the bubble parameters; determines the bubble size corresponding to the current emotional state based on the preset correspondence relationship between the emotion intensity and the bubble parameters, generates the corresponding bubble elements based on the bubble parameters and displays them on the human-computer interaction interface.

[0109] Refer to Figure 5As shown in the left figure, the human-machine dialogue data includes: "Not good". The emotion type of the current emotional state obtained by emotion semantic analysis is negative emotion, and the emotion intensity is medium. The electronic device 200 determines that the bubble color corresponding to the negative emotion is a cold color system and the bubble transparency is relatively high based on the preset correspondence between the emotion type and the bubble parameters; based on the preset correspondence between the emotion intensity and the bubble parameters, it is determined that the bubble size corresponding to the medium emotion intensity is medium, and the corresponding bubble element is generated based on the bubble parameters and displayed on the human-machine interaction interface.

[0110] Referring to Figure 5 As shown in the right figure, the next round of human-machine dialogue data includes: "The work is not bad, hee hee". The emotion type of the current emotional state obtained by emotion semantic analysis is positive emotion, and the emotion intensity is medium. The electronic device 200 determines that the bubble color corresponding to the positive emotion is a warm color system and the bubble transparency is relatively low based on the preset correspondence between the emotion type and the bubble parameters; based on the preset correspondence between the emotion intensity and the bubble parameters, it is determined that the bubble size corresponding to the medium emotion intensity is medium, and the corresponding bubble element is generated based on the bubble parameters and displayed on the human-machine interaction interface.

[0111] According to the technical solution of the above embodiment, by determining the appearance parameters and size parameters of the visual element through the emotion type and emotion intensity, the corresponding visual element can be generated according to different emotion types and emotion intensities, so that different emotions of the user can be expressed by different visual elements, enhancing the emotion expressiveness and further improving the user's emotional immersion.

[0112] In some other exemplary embodiments, the visual element is a different type of visual element, the current emotional state includes an emotion type and an emotion intensity, and the electronic device 200 determines the element type of the visual element corresponding to the current emotional state based on the preset correspondence between the emotion type of the current dialogue user and the element type of the visual element; based on the preset correspondence between the emotion intensity of the current dialogue user and the element size of the visual element, the element size of the visual element corresponding to the current emotional state is determined; the visual element corresponding to the current emotional state is generated based on the element type and the element size.

[0113] For example, in response to the pre-defined mapping rule between the emotion type and the element type, the electronic device 200 maps "positive emotion" to a red star, "neutral emotion" to a green water droplet, and "negative emotion" to a gray cloud.

[0114] By pre-defining the mapping rules between different emotion types and different visual elements, personalized abstract paintings can be generated, increasing the sense of participation in interactive creation.

[0115] According to Figure 3In the technical solution of the exemplary embodiment, on the one hand, by performing emotional semantic analysis on the human-computer dialogue data to determine the current emotional state of the current dialogue user, the current emotional state of the current dialogue user can be accurately and efficiently determined; on the other hand, generating corresponding visual elements based on the current emotional state of the current dialogue user can map different current emotional states of the user to different visual elements, realizing the visual feedback expression of the user's emotions. Thus, during the process of the user's dialogue with the AI, personalized and vivid feedback can be provided to the user, and further, the deep integration of emotional interaction and visual art creation can be achieved, enhancing the user's emotional immersion.

[0116] Further, in some exemplary embodiments, the emotional intensity of the current emotional state includes an intensity level and an intensity duration. The electronic device 200 determines the element size of the visual element corresponding to the current emotional state based on a preset correspondence between the intensity level and the element parameters; and determines the element quantity of the visual element corresponding to the current emotional state based on a preset correspondence between the intensity duration and the element parameters. The intensity level can be determined by emotional intensity vocabulary or the pitch of the audio, and the intensity duration can be determined by the length of the dialogue data input by the user.

[0117] For example, referring to Figure 5 As shown in the left figure, the user's reply is "bad". The electronic device 200 performs emotional semantic analysis on the human-computer dialogue data of the user's reply, determines that the current emotional state is a negative emotion and the emotional intensity is medium intensity. According to the correspondence between the emotional state and the visual element, it is determined that the bubble color corresponding to the negative emotion is a cool color system; based on the preset correspondence between the emotional intensity and the bubble parameters, it is determined that the bubble size corresponding to the medium emotional intensity is medium; according to the length of the dialogue data, the intensity duration is determined to be small, and according to the correspondence between the intensity duration and the bubble quantity, the bubble quantity is determined to be small.

[0118] According to the technical solution in the above exemplary embodiment, by associating the intensity level and the intensity duration of the emotional state with the parameters of the visual element (such as size and quantity), the visual element can be dynamically adjusted according to the user's current emotional state, making the dialogue interface more in line with the user's immediate feelings and providing a more personalized interaction experience.

[0119] In some exemplary embodiments, the visual element is a bubble, and a limit threshold for the size dimension of a single bubble needs to be set to avoid spoiling the picture aesthetics. The element parameters of the visual element include the bubble size. If the bubble size of the bubble is greater than the predetermined threshold, the electronic device 200 divides the bubble size based on the predetermined threshold; and generates at least two bubbles corresponding to the current emotional state based on the division result.

[0120] Referring to Figure 6As shown, a predetermined threshold corresponding to the bubble size is preset, for example, 4.5 cm. The electronic device 200 determines the corresponding bubble size according to the data length of the conversation data replied by the user, for example, 7 cm. If the bubble size of the bubble is greater than the predetermined threshold, for example, 4.5 cm, the electronic device 200 divides the bubble size based on the predetermined threshold; two bubbles corresponding to the current emotional state are generated based on the division result, for example, one bubble size is 4.5 cm and the other bubble size is 2.5 cm.

[0121] According to the technical solution in the above exemplary embodiment, by setting a predetermined threshold for the bubble size and dividing the too large bubbles when necessary, the bubble size can be reasonably controlled, so that each bubble can be better displayed within a limited space, improving the coordination and aesthetics of the overall layout.

[0122] Further, in the exemplary embodiment, after generating the visual elements, the electronic device 200 arranges the generated visual elements on the human-computer interaction interface according to the preset layout rules. For example, a layout rule or layout template for time elements is preset, and the electronic device 200 arranges the generated visual elements on the human-computer interaction interface according to the preset layout rule or layout template. For example, the layout rules may include rules such as alignment method, visual balance method, artistic style, and color contrast.

[0123] For example, when the number of bubbles on the human-computer interaction interface is greater than the predetermined threshold, or after the area ratio of all the bubbles on the human-computer interaction interface to the interface area of the human-computer interaction interface is greater than the predetermined threshold, the electronic device 200 arranges the generated visual elements on the human-computer interaction interface according to the preset layout rules.

[0124] According to the technical solution in the above exemplary embodiment, by setting the preset layout rules, it can be ensured that the bubble layout on the human-computer interaction interface is more aesthetic and can improve the visual attraction.

[0125] In addition, in some exemplary embodiments, the electronic device 200 responds to a trigger operation for generating an image, determines a plurality of visual elements historically generated on the human-computer interaction interface; and generates an image corresponding to the human-computer dialogue data based on the plurality of visual elements and the human-computer interaction interface.

[0126] Refer to Figure 7 As shown, as the bubbles gradually increase, the picture frame is gradually formed. When the multi-round conversation ends, the electronic device 200 generates a picture frame image corresponding to the human-computer dialogue data of the multi-round conversation based on the plurality of bubbles historically generated on the human-computer interaction interface and the human-computer interaction interface through a dynamic picture frame generation engine. The dynamic picture frame generation engine is an algorithm module that automatically renders bubbles in the human-computer interaction interface and combines them into an artistic picture based on the emotional data of the current conversation user.

[0127] Further, in an exemplary embodiment, the electronic device 200 adjusts the element parameters of the visual element in response to the adjustment operation on the visual element on the human-computer interaction interface. For example, in response to the user's operation of adjusting the size of the visual element, the size of the visual element is adjusted; in response to the user's operation of adjusting the position of the visual element, the position of the visual element on the human-computer interaction interface is adjusted, etc.

[0128] According to the technical solution in the above example embodiment, artistic frames are dynamically generated during the conversation between the user and the AI ​​dialogue application, achieving a deep integration of emotional interaction and artistic creation.

[0129] In addition, in some exemplary embodiments, the electronic device 200 displays the conversation record of the human-computer conversation data corresponding to the visual element on the human-computer interaction interface in response to the trigger operation on the visual element on the image. For example, when the user clicks on each bubble, the conversation record between each bubble and the AI ​​can be reviewed, and the experience is enhanced by viewing the historical conversation record through interesting interactions.

[0130] According to the technical solution in the above example embodiment, the user can passively watch the picture generation, or actively adjust parameters or review the conversation to form a closed-loop emotional expression.

[0131] In addition, in the exemplary embodiment, when superimposing historical visual elements, the size of the previously generated visual elements is gradually reduced and the transparency of the previously generated visual elements is increased according to the time decay factor. By introducing the time decay factor to optimize the layout, the picture is avoided from being cluttered, ensuring that the artistic picture frame evolves naturally over time.

[0132] For example, after multiple rounds of conversations, the bubbles on the human-computer interaction interface are relatively full. For example, when the number of bubbles on the human-computer interaction interface is greater than a predetermined threshold, or the bubble area of ​​all bubbles on the human-computer interaction interface accounts for a proportion of the interface area of ​​the human-computer interaction interface that is greater than a predetermined threshold, the electronic device 200 gradually reduces the size of the previously generated visual element and increases the transparency of the previously generated visual element according to the time decay factor.

[0133] According to the technical solution in the above example embodiment, when superimposing historical bubbles, the size of the bubbles is gradually reduced and the transparency is increased according to the "time decay factor" of emotions changing over time, so as to avoid too many and too dense bubbles destroying the aesthetics of the picture.

[0134] Figure 8 A schematic flow chart of a human-computer interaction method provided according to some other embodiments of the present specification is shown.

[0135] Reference Figure 8 As shown, in step S810, in response to the user's login operation to log in to the AI ​​dialogue application, the dialogue human-computer interaction interface is entered.

[0136] In an exemplary embodiment, the human-computer interaction interface is a human-computer interaction interface of an AI dialogue application or an intelligent agent, such as a graphical user interface of an emotional companionship application. The current dialogue user is the user currently using the AI dialogue intelligent agent, such as the user who logs in to the AI dialogue application. The electronic device 200 enters the dialogue human-computer interaction interface in response to the user's login operation for logging in to the AI dialogue application.

[0137] Referring to Figure 4 As shown, the human-computer interaction interface is the interface of the AI dialogue intelligent agent "Emotional Trash Can". A text input box and an audio input button are set on the human-computer interaction interface, and event listeners for the text input box and the audio input button are also set.

[0138] In step S820, the AI dialogue application initiates an opening greeting.

[0139] In an exemplary embodiment, when the user enters the human-computer interaction interface of the AI dialogue application or the intelligent agent, the electronic device 200 calls the AI dialogue application to initiate an opening greeting. Referring to Figure 4 As shown, the human-computer interaction interface is the interface of the AI dialogue intelligent agent "Emotional Trash Can". When the user enters the human-computer interaction interface of the AI dialogue application, the electronic device 200 displays the opening greeting "How are you today?" on the human-computer interaction interface.

[0140] In step S830, the user's reply is obtained.

[0141] In an exemplary embodiment, the electronic device 200 listens for the user's input events such as input text or voice through an event listener, and in response to the monitored input event, obtains the human-computer dialogue data input by the current dialogue user on the human-computer interaction interface.

[0142] Referring to Figure 4 As shown, the human-computer interaction interface is the interface of the AI dialogue intelligent agent "Emotional Trash Can". A text input box and an audio input button are set on the human-computer interaction interface, and event listeners for the text input box and the audio input button are also set. When the electronic device 200 monitors a text or audio input event, it obtains the text information input in the text input box or the audio input captured by the microphone.

[0143] In step S840, emotional semantic analysis is performed on the human-computer dialogue data.

[0144] In an exemplary embodiment, the electronic device 200 obtains the human-computer dialogue data of the current round of dialogue on the human-computer interaction interface, performs emotional semantic analysis on the human-computer dialogue data through a pre-trained emotion classification model, and determines the current emotional state of the current dialogue user. The human-computer dialogue data includes the dialogue data input by the user and the corresponding dialogue data output by the AI dialogue application.

[0145] For example, the electronic device 200 inputs the text of each round of conversation between the current conversation user and the AI conversation application into an emotion classification model such as a BERT or GPT model, and outputs an emotion type or emotion label (e.g., positive / negative / neutral) and a confidence score.

[0146] Furthermore, in the exemplary embodiment, the current emotional state further includes an emotional intensity. The electronic device 200 obtains the human-machine conversation data input by the current conversation user in at least one round of conversation, and performs emotional semantic analysis on the human-machine conversation data through a pre-trained emotion classification model to determine the emotional intensity of the current conversation user. For example, the emotional intensity is 0-100%, where 70%-100% is a high level, 40%-70% is a medium level, and 0-40% is a low level.

[0147] In step S850, a mapping of the emotion and the element parameters of the visual element is performed.

[0148] In the exemplary embodiment, the visual element can be a predetermined same type of visual element or different types of visual elements. The electronic device 200 determines the element parameters of the corresponding visual element based on the current emotional state of the current conversation user, generates the corresponding visual element based on the element parameters of the visual element, and displays the visual element on the human-machine interaction interface.

[0149] For example, a positive emotion is mapped to a warm color system (such as orange / yellow), the bubble size corresponding to the positive emotion is positively correlated with the emotional intensity, and the transparency of the bubble is relatively low; a negative emotion is mapped to a cold color system (such as blue / purple), the bubble size corresponding to the negative emotion is negatively correlated with the emotional intensity, and the transparency of the bubble is relatively high. For example, when the user vents stress, small dark blue translucent bubbles are continuously generated at the four edges of the interface and appear at random positions on the interface edge; when the user expresses a positive emotion, warm color small bubbles are generated, and the warm and cold color system bubbles alternately present an artistic painting.

[0150] In addition, in the exemplary embodiment, the mapping rules between the emotional state and the visual element can be pre-customized, such as the mapping rules between the custom color library, the bubble style library and the emotional state. For example, in response to the mapping definition operation of the user's custom mapping rules, the electronic device 200 maps "joy" to a red star and "calm" to a green water droplet. By customizing the mapping rules between the emotional state and the visual element, a personalized abstract painting can be generated, increasing the sense of participation in interactive creation.

[0151] Further, in an exemplary embodiment, the visual element parameter includes an element motion trajectory, and the electronic device 200 generates a visual element that changes along the element motion trajectory on the human-computer interaction interface according to the element motion trajectory of the visual element and according to the timestamp. For example, assuming that the bubble motion is estimated to be a spiral trajectory or an expansion rule, the electronic device 200 generates bubbles according to the current emotion data and arranges them in a spiral or diffusion trajectory in the human-computer interaction interface according to the timestamp.

[0152] In step S860, a frame is generated.

[0153] In the example embodiment, as the number of bubbles increases, the frame gradually forms. After multiple rounds of conversations are completed, the electronic device 200 generates a frame image corresponding to the human-computer conversation data of the multiple rounds of conversations through a dynamic frame generation engine based on the multiple bubbles historically generated on the human-computer interaction interface and the human-computer interaction interface. The dynamic frame generation engine is an algorithm module that automatically renders bubbles in the human-computer interaction interface and combines them into an artistic picture based on the emotional data of the current conversation user.

[0154] Furthermore, the electronic device 200 provides a summary of emotion-soothing outputs such as emotion-soothing texts through AI applications, so as to achieve multimodal emotional healing through vision and language. Figure 7 As shown, the electronic device 200 uses AI to display “Look, the unhappiness you express is pretty beautiful” on the human-computer interaction interface.

[0155] In addition, in some example embodiments, the electronic device 200 uses GAN (Generative adversarial network) to optimize the bubble layout on the human-computer interaction interface, or has built-in layout templates or layout rules to ensure the aesthetic consistency of the screen. For example, for specific interactive interface requirements (such as bubble layout), the electronic device 200 can input some initial conditions or constraints (such as the amount of information to be displayed, the elements that must be included, etc.) to the generator to generate a layout design scheme that meets the requirements, and optimize the bubble layout on the human-computer interaction interface according to the generated layout design scheme.

[0156] GAN consists of two main parts: a Generator and a Discriminator. The Generator is responsible for generating new layout designs based on input random noise or specific conditions; the Discriminator evaluates whether the generated layout design is real, that is, whether it is close enough to real design samples. For example, a large number of high-quality visual element layout samples of human-computer interaction interfaces are collected in advance, especially those designs widely regarded as beautiful and user-friendly; the visual element layout samples are labeled, and key design elements such as color matching, typesetting methods, and element spacing are extracted. During the training process, the Generator tries to deceive the Discriminator into believing that the generated layout design is real; while the Discriminator tries to distinguish between real and fake designs. Through such an adversarial process, the Generator gradually learns to generate more realistic layout designs.

[0157] By optimizing the bubble layout on the human-computer interaction interface through GAN, it is possible to automatically optimize the aesthetics of the picture frame, ensuring that the visual effect and user experience of the generated picture frame reach an optimal state.

[0158] In step S870, a picture frame display strategy is set.

[0159] In an exemplary embodiment, the picture frame display strategy can be set on the human-computer interaction interface. For example, set the size limit of a single bubble, or set a time decay factor, etc. The time decay factor is used to reduce the visual weight of historical visual elements such as bubbles (such as shrinking and fading) over time, ensuring dynamic update of the screen.

[0160] Furthermore, the electronic device 200 adjusts the element parameters of the visual element in response to an adjustment operation on the visual element on the human-computer interaction interface. For example, in response to a user's operation of adjusting the size of the visual element, the size of the visual element is adjusted; in response to a user's operation of adjusting the position of the visual element, the position of the visual element on the human-computer interaction interface is adjusted, etc.

[0161] In step S880, the picture frame is viewed.

[0162] In an exemplary embodiment, the user is allowed to click on the bubble to view the corresponding conversation content. The electronic device 200 displays the conversation record of the human-machine dialogue corresponding to the visual element on the human-computer interaction interface in response to a trigger operation on the visual element on the image. For example, when the user clicks on each bubble, the conversation record between each bubble and the AI can be viewed, and the historical conversation record can be viewed through interesting interaction to enhance the experience.

[0163] According to the technical solutions in the above exemplary embodiments, the user can either passively view the generation of the picture frame or actively adjust the parameters or recall the conversation, forming a closed-loop emotional expression.

[0164] In step S890, the picture frame is collected / downloaded / shared.

[0165] In the exemplary embodiment, after generating the picture frame, the electronic device 200 provides controls or interfaces for collecting / downloading / sharing the picture frame on the human-computer interaction interface for the user to perform operations of collecting / downloading / sharing the picture frame.

[0166] According to Figure 8 the technical solution of the exemplary embodiment, on the one hand, it realizes the generation of emotion-driven artistic picture frames. For example, it combines the emotion of the dialogue user with dynamic visual art and realizes "dialogue is creation" through the AI model, enhancing the user's emotional immersion. On the other hand, it adopts a multi-dimensional emotion mapping mechanism. For example, in addition to color, it expresses the intensity and evolution of emotions through multi-parameters such as bubble size, transparency, and movement trajectory, enhancing the emotional expressiveness. On the one hand again, it introduces a context-aware dynamic attenuation algorithm. For example, it introduces a time attenuation factor and GAN layout optimization to avoid a cluttered picture and ensure the natural evolution of the artistic picture frame over time. On the one hand again, it conducts a two-way interaction design. For example, the user can either passively view the generation of the picture frame or actively adjust the parameters or backtrack the dialogue to form a closed-loop emotional expression.

[0167] In summary, according to the technical solution of the embodiments of this specification, it is possible to convert an AI dialogue into a collectible emotional art picture frame, which has both healing value and commercial potential and can be applied to fields such as mental health, social entertainment, and digital art.

[0168] On the other hand, this specification provides a non-transitory storage medium storing at least one set of executable instructions for performing human-computer interaction. When the executable instructions are executed by a processor, the executable instructions direct the processor to implement the steps of the human-computer interaction method described in this specification. In some possible implementation manners, various aspects of this specification can also be implemented in the form of a program product, which includes program code. When the program product runs on the electronic device 200, the program code is used to cause the electronic device 200 to execute the steps of the human-computer interaction method described in this specification. The program product for implementing the above method can adopt a portable compact disc read-only memory (CD-ROM) including program code and can run on the electronic device 200. However, the program product of this specification is not limited thereto. In this specification, the readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or combined with an instruction execution system. The program product can adopt any combination of one or more readable media. The readable media can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the readable storage medium include: an electrical connection having one or more wires, a portable disc, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. The computer-readable storage medium can include a data signal propagated in a baseband or as part of a carrier wave, in which the readable program code is carried. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable storage medium can also be any readable medium other than the readable storage medium, and this readable medium can send, propagate, or transmit a program for use by or combined with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium can be transmitted by any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above. The program code for performing the operations of this specification can be written in any combination of one or more programming languages, and the programming languages include object-oriented programming languages - such as Java, C++, etc., and also include conventional procedural programming languages - such as the "C" language or similar programming languages. The program code can be executed entirely on the electronic device 200, partially on the electronic device 200, executed as an independent software package, partially on the electronic device 200 and partially on a remote computing device, or entirely on the remote computing device.

[0169] The above describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the acts or steps recited in the claims may be performed in a different order than in the embodiments and still achieve the desired result. Additionally, the processes depicted in the figures do not necessarily require a particular order or a sequential order to achieve the desired result. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0170] In summary, after reading this detailed disclosure, those skilled in the art will appreciate that the foregoing detailed disclosure may be presented by way of example only and is not necessarily limiting. Although not explicitly stated herein, those skilled in the art will understand that this specification is intended to encompass various reasonable changes, improvements, and modifications to the embodiments. These changes, improvements, and modifications are intended to be proposed by this specification and are within the spirit and scope of the exemplary embodiments of this specification.

[0171] In addition, certain terms in this specification have been used to describe the embodiments of this specification. For example, "one embodiment", "an embodiment", and / or "some embodiments" mean that the specific features, structures, or characteristics described in connection with that embodiment may be included in at least one embodiment of this specification. Thus, it should be emphasized and understood that two or more references to "an embodiment" or "one embodiment" or "alternative embodiments" in various parts of this specification do not necessarily all refer to the same embodiment. Moreover, the specific features, structures, or characteristics may be appropriately combined in one or more embodiments of this specification.

[0172] It should be understood that in the foregoing description of the embodiments of this specification, for the purpose of helping to understand a feature, and for the purpose of simplifying this specification, this specification combines various features in a single embodiment, drawing, or its description. However, this does not mean that the combination of these features is necessary, and those skilled in the art may well mark out some of the devices as separate embodiments for understanding when reading this specification. That is to say, the embodiments in this specification may also be understood as the integration of multiple sub - embodiments. And it also holds when the content of each sub - embodiment contains fewer features than all the features of a single foregoing disclosed embodiment.

[0173] Each patent, patent application, publication of patent application, and other materials cited herein, such as articles, books, specifications, publications, documents, items, etc., may be incorporated herein by reference. For all purposes, the entire content, except for any prosecution file history associated therewith, any identical thereto that may be inconsistent or in conflict with this document, or any identical prosecution file history that may have a limiting effect on the broadest scope of the claims. Now or hereafter associated with this document. By way of example, if there is any inconsistency or conflict between the description, definition, and / or use of a term associated with any of the incorporated materials and the terms, descriptions, definitions, and / or uses in this document, the terms in this document shall govern.

[0174] Finally, it should be understood that the embodiments of the application disclosed herein are illustrative of the principles of the embodiments of this specification. Other modified embodiments are also within the scope of this specification. Therefore, the embodiments disclosed in this specification are merely examples and not limitations. Those skilled in the art can adopt alternative configurations based on the embodiments in this specification to implement the application in this specification. Therefore, the embodiments of this specification are not limited to the embodiments precisely described in the application.

Claims

1. A human-computer interaction method, comprising: Obtaining human-computer dialogue data corresponding to the current round of dialogue on a human-computer interaction interface, where the human-computer dialogue data includes the dialogue data input by the current dialogue user; Performing emotional semantic analysis on the human-computer dialogue data to determine the current emotional state of the current dialogue user; And Generating corresponding visual elements based on the current emotional state of the current dialogue user, and displaying the visual elements on the human-computer interaction interface.

2. The method according to claim 1, wherein, The generating corresponding visual elements based on the current emotional state of the current dialogue user includes: Determining the element parameters of the visual elements corresponding to the current emotional state based on the preset correspondence between the current emotional state and the visual elements; Generating the visual elements corresponding to the current emotional state based on the element parameters of the visual elements.

3. The method according to claim 2, wherein The element parameters include one or more of element color, element size, element quantity, element transparency, and element movement trajectory.

4. The method according to claim 3, wherein, The current emotional state includes an emotional type and an emotional intensity. The performing emotional semantic analysis on the human-computer dialogue data to determine the current emotional state of the current dialogue user includes: Performing emotional semantic analysis on the human-computer dialogue data to determine the emotional type and emotional intensity of the current dialogue user, The determining the element parameters of the visual elements corresponding to the current emotional state based on the preset correspondence between the current emotional state and the visual elements includes: Determining the element color and element transparency of the visual elements corresponding to the current emotional state based on the preset correspondence between the emotional type and the element parameters; Determining the element size of the visual elements corresponding to the current emotional state based on the preset correspondence between the emotional intensity and the element parameters.

5. The method according to claim 4, wherein The emotional intensity includes an intensity level and an intensity duration. The determining the element size of the visual elements corresponding to the current emotional state based on the preset correspondence between the emotional intensity and the element parameters includes: Determining the element size of the visual elements corresponding to the current emotional state based on the preset correspondence between the intensity level and the element parameters; Determining the element quantity of the visual elements corresponding to the current emotional state based on the preset correspondence between the intensity duration and the element parameters.

6. The method according to claim 3, wherein The generating the visual elements corresponding to the current emotional state based on the element parameters of the visual elements includes: Generating, according to the element movement trajectory of the visual elements, the visual elements that change along the element movement trajectory on the human-computer interaction interface according to time stamps.

7. The method according to any one of claims 2 to 6, wherein The visual elements are bubbles.

8. The method according to claim 7, wherein The element parameters include bubble size. The generating the visual elements corresponding to the current emotional state based on the element parameters of the visual elements includes: If the bubble size of the bubble is greater than a predetermined threshold, then dividing the bubble size based on the predetermined threshold; Generating at least two bubbles corresponding to the current emotional state based on the division result.

9. The method according to claim 1, wherein The current emotional state includes an emotion type and an emotion intensity. The emotional semantic analysis of the human-machine dialogue data to determine the current emotional state of the current dialogue user includes: Performing emotional semantic analysis on the historical human-machine dialogue data to determine the emotion type and emotion intensity of the current dialogue user; Generating corresponding visual elements based on the current emotional state of the current dialogue user, including: Determining the element type of the visual element corresponding to the current emotional state based on a preset correspondence between the emotion type and the element type of the visual element; Determining the element size of the visual element corresponding to the current emotional state based on a preset correspondence between the emotion intensity and the element size of the visual element; Generating the visual element corresponding to the current emotional state based on the element type and the element size.

10. The method according to claim 1, wherein, The method further includes: When superimposing historical visual elements, gradually reducing the size of the previously generated visual element and increasing the transparency of the previously generated visual element according to a time decay factor.

11. The method according to claim 1, wherein, After generating the visual element, the method further includes: Arranging the generated visual elements on the human-machine interaction interface according to a preset layout rule.

12. The method according to claim 1, wherein The method further includes: In response to a trigger operation for generating an image, determining a plurality of the visual elements previously generated on the human-machine interaction interface, where the trigger operation includes an operation for ending a dialogue; Generating an image corresponding to the human-machine dialogue data based on the plurality of visual elements and the human-machine interaction interface.

13. The method according to claim 12, wherein, The method further includes: In response to a trigger operation for the visual element on the image, displaying a dialogue record of the human-machine dialogue data corresponding to the visual element on the human-machine interaction interface.

14. An electronic device, including: At least one storage medium storing at least one instruction set for performing human-machine interaction processing; And At least one processor communicatively connected to the at least one storage medium, wherein when the electronic device runs, the at least one processor reads the at least one instruction set and executes the human-machine interaction method according to any one of claims 1-13 based on the indication of the at least one instruction set.

Images