Session processing method and device, electronic equipment and storage medium

Abstract

The invention provides a session processing method and device, electronic equipment and a computer readable storage medium. The method comprises the steps that a virtual space is displayed, the virtual space comprises a virtual image corresponding to each member of the virtual space, and the virtual space can be decorated through virtual materials; and in response to an input operation, displaying an input first message in the virtual space, and displaying a first interaction behavior between the at least two virtual images in the virtual space, the first message including content related to the first interaction behavior. Through the application, chat modes can be enriched, and the man-machine interaction efficiency can be improved.

Description

Technical Field

[0001] The present application relates to artificial intelligence technology, and in particular to a conversation processing method, device, electronic device and storage medium. Background Art

[0002] With the development of Internet technology, the application of instant messaging is becoming more and more widespread. Instant messaging applications are used to provide users with instant messaging services based on the Internet, allowing two or more people to instantly transmit text messages, voice and video messages through the network. With the development of instant messaging applications, instant messaging applications have penetrated into people's lives, and more and more people use instant messaging applications to communicate.

[0003] During a conversation via an instant messaging application, the conversation interface has the function of displaying virtual images of members participating in the conversation. The virtual images have low relevance to the conversation content and occupy the display area of ​​the conversation interface. The relevant technology currently lacks a solution to improve the interaction efficiency of conversation members through virtual images. Summary of the invention

[0004] The embodiments of the present application provide a conversation processing method, device, electronic device, computer-readable storage medium, and computer program product, which can enrich the chat mode and improve the efficiency of human-computer interaction.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] The present application provides a session processing method, the method comprising:

[0007] Displaying a virtual space, wherein the virtual space includes a virtual image corresponding to each member of the virtual space, and the virtual space can be decorated by virtual materials;

[0008] In response to an input operation, displaying the input first message in the virtual space, and

[0009] A first interactive behavior between at least two of the virtual images is displayed in the virtual space, wherein the first message includes content related to the first interactive behavior.

[0010] The present application provides a session processing device, including:

[0011] A display module configured to display a virtual space, wherein the virtual space includes a virtual image corresponding to each member of the virtual space, and the virtual space can be decorated by virtual materials;

[0012] a display module configured to display the input first message in the virtual space in response to an input operation, and

[0013] The display module is configured to display a first interactive behavior between at least two of the virtual images in the virtual space, wherein the first message includes content related to the first interactive behavior.

[0014] An embodiment of the present application provides an electronic device, the electronic device comprising:

[0015] A memory for storing computer executable instructions;

[0016] The processor is used to implement the session processing method provided in the embodiment of the present application when executing the computer executable instructions stored in the memory.

[0017] An embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions for implementing the session processing method provided in the embodiment of the present application when executed by a processor.

[0018] An embodiment of the present application provides a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, the session processing method provided in the embodiment of the present application is implemented.

[0019] The embodiments of the present application have the following beneficial effects:

[0020] By displaying virtual images representing users participating in the chat in the virtual space, the virtual space can be decorated with virtual materials, enriching the form of the chat session. When a message is input, the actions between the virtual images related to the message are displayed, enriching the expression form of the chat. By inputting a message, the virtual image automatically displays related actions, which improves the efficiency of controlling the virtual image, improves the efficiency of human-computer interaction, and improves the user's chat experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a schematic diagram of an application mode of the session processing method provided in an embodiment of the present application;

[0022] Figure 2 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0023] FIG. 3A to FIG. 3E It is a flowchart of a session processing method provided in an embodiment of the present application;

[0024] FIG. 4A to FIG. 4G is a schematic diagram of an interface provided by an embodiment of the present application;

[0025] FIG. 5A to FIG. 5E is a schematic diagram of an interface provided by an embodiment of the present application;

[0026] Figure 6is a schematic diagram of an interface provided by an embodiment of the present application;

[0027] 7A to 7C It is a schematic diagram of the interface provided in the embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0029] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0030] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0031] It should be pointed out that the relevant data collection and processing in this application (for example, collecting chat content input by users into the conversation interface) should be strictly in accordance with the requirements of relevant national laws and regulations when applied in examples, and the informed consent or separate consent of the personal information subject should be obtained. Subsequent data use and processing should be carried out within the scope of authorization of laws and regulations and the personal information subject.

[0032] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0033] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0034] 1) Avatars: objects that interact in virtual scenes, controlled by users or robot programs (e.g., robot programs based on artificial intelligence), objects that can be still, move, and perform various behaviors in virtual scenes, such as various characters in games. Another example: characters that represent the user's personal image in a conversational interface.

[0035] 2) In response, it is used to indicate the conditions or states on which the executed operations depend. When the dependent conditions or states are met, one or more operations executed may be in real time or have a set delay. Unless otherwise specified, there is no restriction on the order of execution of the multiple operations executed.

[0036] 3) Instant messaging (IM) is a terminal service that allows two or more people to use the Internet to instantly transmit text messages, files, voice and video. Instant messaging is divided into enterprise instant messaging and website instant messaging based on its use. It can be divided into mobile instant messaging and PC instant messaging based on the objects it carries. Mobile instant messaging is represented by text messages, websites, and video instant messaging.

[0037] 4) Convolutional Neural Networks (CNN) is a type of feed forward neural network (FNN) that includes convolution calculations and has a deep structure. It is one of the representative algorithms of deep learning. Convolutional neural networks have the ability of representation learning and can perform shift-invariant classification on input images according to their hierarchical structure.

[0038] In the related art, the virtual image in the conversation interface is used to represent the user image. The user can perform activities such as changing the virtual image's clothes and dressing up the virtual image's virtual room. The virtual image has no connection with the chat and cannot convey the user's emotional expression in the chat.

[0039] The embodiments of the present application provide a conversation processing method, a conversation processing device, an electronic device, a computer-readable storage medium, and a computer program product, which can enrich the chat mode and improve the efficiency of human-computer interaction.

[0040] The following describes an exemplary application of an electronic device provided by an embodiment of the present application. The electronic device provided by an embodiment of the present application can be implemented as a laptop, a tablet computer, a desktop computer, a set-top box, a mobile device (for example, a mobile phone, a portable music player, a personal digital assistant, a dedicated messaging device, a portable gaming device), a vehicle terminal, a virtual reality (VR) device, an augmented reality (AR) device, and other types of user terminals), and can also be implemented as a server. Below, an exemplary application when the device is implemented as a terminal device or a server will be described.

[0041] refer to Figure 1, Figure 1 is a schematic diagram of an application mode of the session processing method provided in an embodiment of the present application; for example, Figure 1 The server 200, network 300, terminal device 400-1, terminal device 400-2, and database 500 are involved. The terminal device 400 is connected to the server 200 via the network 300, and the network 300 can be a wide area network or a local area network, or a combination of the two.

[0042] In some embodiments, terminal device 400-1 and terminal device 400-2 are terminal devices used by different users respectively, each terminal device is installed with instant messaging software, server 200 is a server of an instant messaging platform, and users send messages to each other through instant messaging software.

[0043] For example, a user using terminal device 400-1 sends a message in conversation interface 101 (refer to Figure 1 , with the content "Let's watch it together, come in quickly") to the user using terminal device 400-2, and the message is sent to server 200 through network 300. Server 200 calls the interactive behavior data between the virtual images associated with the message from the database according to the message, and sends the interactive behavior data to terminal device 400-1 and terminal device 400-2, and sends the message to terminal device 400-2. In the conversation interface of terminal device 400-1 and terminal device 400-2, the message sent by the user using terminal device 400-1 and the interactive behavior associated with the message formed between the virtual images based on the interactive behavior data can be seen. Figure 1 The middle conversation interface 102 is the conversation interface 101 showing the interactive behavior. In the conversation interface 101, the first virtual image 103 and the second virtual image 104 are sitting together in a daze. In the conversation interface 102, the interactive behavior between the first virtual image 103 and the second virtual image 104 is shown, and the interactive behavior is eye contact.

[0044] The embodiments of the present application can be implemented through database technology. In short, a database can be regarded as an electronic file cabinet where electronic files are stored. Users can add, query, update, delete, etc. data in the files. The so-called "database" is a collection of data that is stored together in a certain way, can be shared with multiple users, has as little redundancy as possible, and is independent of the application program.

[0045] A database management system (DBMS) is a computer software system designed for managing databases. It generally has basic functions such as storage, retrieval, security, and backup. Database management systems can be classified according to the database model they support, such as relational, XML (Extensible Markup Language); or according to the type of computer they support, such as server clusters, mobile phones; or according to the query language used, such as Structured Query Language (SQL), XQuery; or according to performance focus, such as maximum scale, maximum operating speed; or other classification methods. Regardless of the classification method used, some DBMS can cross categories, for example, supporting multiple query languages ​​at the same time.

[0046] The embodiments of the present application can also be implemented through cloud technology. Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on the cloud computing business model application. It can form a resource pool, which is used on demand and is flexible and convenient. Cloud computing technology will become an important support. The background services of the technical network system require a large amount of computing and storage resources, such as video websites, picture websites and more portal websites. With the high development and application of the Internet industry, as well as the promotion of search services, social networks, mobile commerce and open collaboration, each item may have its own hash code identification mark in the future, and all need to be transmitted to the background system for logical processing. Data of different levels will be processed separately, and all kinds of industry data require strong system backing support, which can only be achieved through cloud computing.

[0047] In some embodiments, Figure 1 The server 200 in the example may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The electronic device may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited thereto. The terminal device and the server may be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiments of the present invention.

[0048] As an example, types of graphics processing hardware include a central processing unit (CPU) and a graphics processing unit (GPU).

[0049] The terminal device 400 calculates the data required for display through the graphics computing hardware, and completes the loading, parsing and rendering of the display data, and outputs the video frames that can form a visual perception of the virtual scene through the graphics output hardware, for example, presenting two-dimensional video frames on the display screen of a smartphone, or projecting video frames that achieve a three-dimensional display effect on the lenses of augmented reality / virtual reality glasses; in addition, in order to enrich the perception effect, the terminal device 400 can also use different hardware to form one or more of auditory perception, tactile perception, motion perception and taste perception.

[0050] In some embodiments, the terminal device 400 can implement the session processing method provided in the embodiments of the present application by running a computer program. For example, the computer program can be a native program or software module in the operating system; it can be a native application (APP, APPlication), that is, a program that needs to be installed in the operating system to run, such as an instant messaging APP, a game APP; it can also be a small program that can be embedded in any APP, that is, a program that can be run only by downloading it to a browser environment. In short, the above-mentioned computer program can be an application, module or plug-in in any form.

[0051] Taking a computer program as an application program as an example, in actual implementation, the terminal device 400 installs and runs an application program that supports a virtual scene or a conversation interface. The application program can be any one of a first-person shooter game (FPS), a third-person shooter game, a virtual reality application program, a three-dimensional map program, or a multiplayer survival game, and an instant messaging software. The user uses the terminal device 400 to operate the virtual image in the conversation interface to perform activities, and the activities include but are not limited to: adjusting body posture, crawling, walking, running, riding, jumping, driving, picking up, shooting, attacking, and throwing. Indicatively, the virtual image can be a virtual character, such as a simulated character or an anime character.

[0052] See also Figure 2 , Figure 2 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application, Figure 2The terminal device 400 shown includes: at least one processor 410, a memory 450, at least one network interface 420 and a user interface 430. The various components in the terminal device 400 are coupled together via a bus system 440. It is understood that the bus system 440 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 440 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 440 is not shown in FIG. Figure 2 Various buses are labeled as bus system 440 .

[0053] Processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0054] The user interface 430 includes one or more output devices 431 that enable presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0055] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices that are physically remote from the processor 410.

[0056] The memory 450 includes a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 450 described in the embodiments of the present application is intended to include any suitable type of memory.

[0057] In some embodiments, memory 450 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplarily described below.

[0058] Operating system 451, including system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0059] A network communication module 452, used to reach other electronic devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 include: Bluetooth, Wireless Compatibility Certification (WiFi), and Universal Serial Bus (USB), etc.;

[0060] a presentation module 453 for enabling presentation of information via one or more output devices 431 (e.g., display screen, speaker, etc.) associated with the user interface 430 (e.g., a user interface for operating peripherals and displaying content and information);

[0061] The input processing module 454 is used to detect one or more user inputs or interactions from one of the one or more input devices 432 and translate the detected inputs or interactions.

[0062] In some embodiments, the device provided in the embodiments of the present application can be implemented in software. Figure 2 The session processing device 455 stored in the memory 450 is shown, which can be software in the form of a program or a plug-in, including the following software modules: a display module 4551, an action generation module 4552, and these modules are logical, so they can be arbitrarily combined or further split according to the functions implemented. Figure 2 For the sake of convenience, all the above modules are shown at once, but it should not be considered that the session processing device 455 excludes the implementation that only includes the display module 4551. The functions of each module will be explained below.

[0063] The session processing method provided in the embodiment of the present application will be explained in combination with the exemplary application and implementation of the terminal device provided in the embodiment of the present application.

[0064] The following describes the session processing method provided by the embodiment of the present application. As mentioned above, the electronic device implementing the session processing method of the embodiment of the present application may be a terminal device, or a combination of a terminal device and a server. Therefore, the execution subject of each step will not be repeatedly described below.

[0065] See also Figure 3E , Figure 3E is a flow chart of the session processing method provided in the embodiment of the present application, which will be combined with Figure 3E The steps shown are explained.

[0066] In step 311, the virtual space is displayed.

[0067] Here, the virtual space includes a virtual image corresponding to each member of the virtual space, and the virtual space can be decorated by virtual materials.

[0068] For example, a virtual image is a virtual character set by a user to represent his or her own image. The user can change the clothes and face of the virtual image. For example, if there are two members in the virtual space, two virtual images are displayed in the virtual space, one of which is the virtual image of the first user using the terminal device, and the other is the virtual image of another user (hereinafter referred to as the second user) who is in conversation with the first user. The user can set the virtual materials in the virtual space by himself or herself to decorate the virtual space.

[0069] refer to Fig. 7A , Fig. 7A The virtual space 701A includes a virtual image A and a virtual image B, and the virtual space 701A or the virtual space 702A includes a virtual material 703A (stars) and a virtual material 704A (cabinet) for decoration.

[0070] In step 312, in response to the input operation, the input first message is displayed in the virtual space.

[0071] For example, continue to refer to Fig. 7A , the content of the first message input by the user is “rock-paper-scissors”.

[0072] In step 313, a first interactive behavior between at least two virtual images is displayed in the virtual space.

[0073] Here, the first message includes content related to the first interactive behavior.

[0074] Continue to refer Fig. 7A According to the message "rock-paper-scissors" in the virtual space 701A, the virtual images A and B in the virtual space 702A perform the rock-paper-scissors action. The content of the first message includes the interactive behavior between the virtual images A and B.

[0075] In some embodiments, before step 301, a session interface is displayed; in response to a trigger operation for an entrance to the virtual space, the step of switching from displaying the session interface to displaying the virtual space is performed, wherein members of the session interface include members of the virtual space.

[0076] Example, reference Figure 4A , Figure 4A The conversation interface 401A includes virtual images corresponding to two users participating in the conversation, virtual image A and virtual image B, where virtual image A represents the first user using the current terminal device and virtual image B represents the second user using another terminal device to have a conversation with the first user. Figure 7Cis a schematic diagram of an interface provided by an embodiment of the present application. When the virtual space entrance is triggered, the conversation interface 401A switches to the virtual space 701A. The triggering operation may be a long press operation on the virtual image in the conversation interface.

[0077] In some embodiments, after step 301, in response to an exit operation on the virtual space, a step is performed to switch from displaying the virtual space to displaying the session interface.

[0078] For example, any member of the virtual space can exit. If the member who created the space exits, the virtual images of other members will automatically exit; if a non-creating member exits, the virtual images of other members will not automatically exit.

[0079] In some embodiments, the step of switching from displaying the session interface to displaying the virtual space may be implemented in the following manner:

[0080] Method 1: In response to a trigger operation on the virtual space, and the conversation interface includes two members, directly switching from displaying the conversation interface to displaying the virtual space.

[0081] refer to Figure 7C , Figure 7C 401A is a schematic diagram of an interface provided by an embodiment of the present application. The conversation interface 401A includes two members, and the conversation interface 401A is directly switched to the virtual space 701A.

[0082] Method 2: In response to a trigger operation of the virtual space, and the session interface includes three or more members, a selection control is displayed, which includes all the members of the session interface. In response to a selection operation in the selection control, at least two members are selected as members of the virtual space, and the step of switching from displaying the session interface to displaying the virtual space is executed.

[0083] For example, the selection operation may be a positive selection, that is, the selected virtual object is displayed in the virtual space; or a negative selection, that is, no selected virtual object is displayed in the virtual space. Figure 7B , Figure 7B 701B triggers the triggering operation of the virtual space, and option 702B is displayed. If the user selects group friend B, the session interface 701B switches to the virtual space 703B. The virtual space 703B includes: the virtual image B of group friend B and the user's virtual image A, and virtual materials (for example: virtual materials 704B and virtual materials 705B). Figure 7B It corresponds to the interface of the chat software displayed on the computer. The switching method in method 2 is also applicable to other terminal devices.

[0084] In some embodiments, reference Figure 3A, Figure 3A is a flow chart of a session processing method provided in an embodiment of the present application. Figure 3E Before step 311 in Figure 3A Steps 301 to 303 in the embodiment are described in detail below.

[0085] In step 301, a conversation interface is displayed.

[0086] Here, the conversation interface includes a virtual image corresponding to each member participating in the conversation.

[0087] For example, a virtual image is a virtual character set by a user to represent his or her own image. The user can change the clothes and face of the virtual image. For example, two members participating in a conversation are used as an example. Two virtual images are displayed in the conversation interface, one of which is the virtual image of the first user using the terminal device, and the other is the virtual image of another user (hereinafter referred to as the second user) who is having a conversation with the first user.

[0088] refer to Figure 4A , Figure 4A 402A and 401A are schematic diagrams of the same conversation interface at different times, wherein the time corresponding to conversation interface 402A is after the time corresponding to conversation interface 401A. Conversation interface 401A includes virtual images corresponding to two users participating in the conversation, virtual image A and virtual image B, virtual image A represents the first user using the current terminal device, and virtual image B represents the second user who uses other terminal devices to have a conversation with the first user.

[0089] Continue to refer Figure 3A In step 302, in response to an input operation in the conversation interface, the input first message is displayed.

[0090] For example, the first message is a message input by the first user, and the type of the first message can be a file message (for example:), an invitation message, a voice message, or an emoticon message. Figure 4A , the first user entered the message "Let's watch it together, come in."

[0091] In step 303, a first interactive behavior between at least two virtual images is displayed.

[0092] Here, the first message includes content related to the first interactive behavior.

[0093] For example, continue to refer to Figure 4AAccording to the message in the conversation interface 401A, the virtual image A and the virtual image B in the conversation interface 402A perform an interactive behavior of looking at each other. The first message is: "Let's watch together, come in quickly", and the interactive behavior of the virtual image A and the virtual image B includes the action of looking at each other.

[0094] In some embodiments, the type of the first message includes a file message; step 302 may be implemented by displaying the transmission progress of the file message corresponding to the input operation, and in response to the completion of the file message transmission, displaying an icon representing the file message.

[0095] For example, file messages can be pictures, videos, or audio. Figure 5C , Figure 5C In the conversation interface 501C, the second user sends a file to the first user, and the file transfer progress is 50%, indicating that the file is being transferred.

[0096] For example, when the file transfer is finished, Figure 5C The "File transfer progress 50%" is converted to file transfer 100%, indicating that the file transfer is complete, and the file icon is displayed.

[0097] The first interactive behavior includes: the second interactive behavior and the third interactive behavior, refer to Figure 3B , Figure 3B It is a flowchart of a session processing method provided in an embodiment of the present application; Figure 3A Step 303 in can be Figure 3B Steps 3031 to 3032 in are implemented as described below.

[0098] In step 3031, a second interactive behavior representing file transfer between at least two virtual images is displayed.

[0099] For example, the screen displaying the first interactive behavior includes a virtual vehicle carrying the file message. Figure 5C , a virtual vehicle 502C representing file transfer is displayed between virtual images, Figure 5C The virtual vehicle in is a paper airplane.

[0100] In step 3032, in response to the file message transmission being completed and the file message being a file for viewing, a third interactive behavior of at least two virtual images viewing the file message is displayed.

[0101] For example, the third interactive behavior of viewing a file message may be an interactive behavior of sitting on a chair and browsing the file content together, and the file may be a picture, video, or audio. For example, when the file is a picture or video, the third interactive behavior is the behavior of the virtual image watching the video or picture together; when the file is audio, the third interactive behavior may be the behavior of the virtual image sharing the same pair of wired headphones and listening to music together.

[0102] In the embodiment of the present application, the behavior of sending messages or sharing multimedia information in a real scene is simulated between virtual images, which improves the realism of the virtual images, increases the fun of the conversation process, improves the user's conversation experience, and can promote the friendship between the user and other users participating in the conversation.

[0103] In some embodiments, the type of the first message includes an expression message; step 303 may be implemented by displaying at least two virtual images simulating a first interactive behavior of an expression corresponding to the expression message.

[0104] For example, a simulated expression message means that the face of the avatar and the expression message show a similarity that reaches a similarity threshold or the same expression, see Figure 5D , Figure 5D In the conversation interface 501D, the second user sends a smiling sunglasses expression message to the first user, and the faces of the virtual images A and B simulate the smiling sunglasses expression.

[0105] In the embodiment of the present application, by simulating expression messages between virtual images, the realism of the virtual images is improved, the fun of the conversation process is improved, the user's conversation experience is improved, and the friendship between the user and other users participating in the conversation can be promoted.

[0106] In some embodiments, the type of the first message includes an invitation message, wherein the invitation message is used to invite members participating in the session to browse multimedia information. The multimedia information may be music, video or picture.

[0107] Step 303 may be implemented in the following manner: displaying a first interactive behavior between at least two virtual images representing sharing of multimedia information, and displaying a play control corresponding to the multimedia information, wherein the play control is used to start playing the multimedia information when triggered.

[0108] refer to Figure 5E , Figure 5ESchematic diagram of the interface provided by the embodiment of the present application. A user can also send an invitation message to other users to share multimedia information together. When all users participating in the session agree to share, the virtual images display a screen of browsing multimedia information together, for example: the interactive behavior of virtual images listening to songs together in the session interface 501E. The session interface 501E also includes a prompt message "Listen together: XX song", and a play control 502E is displayed above the prompt message, which is used to start playing the song when triggered and stop playing the song when triggered again.

[0109] In some embodiments, members participating in the conversation include a first member and a second member, the virtual images include a first virtual image corresponding to the first member and a second virtual image of the second member having a conversation with the first member, and the first interactive behavior is initiated by the first virtual image to the second virtual image; after step 301, in response to receiving the second message, the received second message is displayed, and the fourth interactive behavior initiated by the second virtual image to the first virtual image is displayed.

[0110] For example, the second message is generated by the input operation of the second member, and the second message includes content related to the fourth interactive behavior. That is, in the embodiment of the present application, the interactive behavior of the virtual image can be initiated by the virtual image corresponding to the member sending the message. The first message and the second message in the embodiment of the present application are used to distinguish different messages and do not represent the order in which the messages are sent (the third message, the fourth message and the fifth message below also apply to the above description).

[0111] In some embodiments, during the input operation on the conversation interface, the behavior of the first virtual image representing the message input is displayed.

[0112] For example, the behavior representing the message input may be the behavior of the first virtual image typing on a mobile phone, or the behavior of the first virtual image typing on a computer keyboard.

[0113] During the input operation of the second member, the behavior of the second virtual image representing the message input is displayed; in response to not receiving a reply message to the first message, the behavior of the first virtual image representing the waiting is displayed.

[0114] In the embodiment of the present application, both parties participating in the conversation can watch the interactive behavior of the virtual image, which can enhance the realism of the virtual image and improve the user's conversation experience.

[0115] For easy understanding, refer to Figure 4F , Figure 4F401F, the second user is in the input state, then the prompt message "the other party is inputting" and the interactive behavior of the second user's virtual image B representing the input are displayed in the conversation interface 401F of the first user, for example, a bubble is displayed above the head of the virtual image B. When the first user sends a message and waits for the second user to reply, as shown in the conversation interface 402F, the first user's virtual image A presents an action representing waiting, for example, the virtual image A looks at the virtual image B.

[0116] In some embodiments, after step 301, in response to displaying a new message in the conversation interface, and the time interval between the moment when the new message is displayed and the moment when the third message is displayed reaches a duration threshold, an act of displaying a greeting represented by a virtual image associated with the new message is displayed, wherein the virtual image associated with the new message is the virtual image corresponding to the member who inputs the new message, and the third message is the latest message before the new message.

[0117] For example, when the duration between the moment a new message is displayed and the moment a previous message (the third message) is displayed in the conversation interface is greater than or equal to a duration threshold, if a message input by a first user of a first virtual image is received, then the first virtual image is displayed greeting the second virtual image; if a message input by a second user of a second virtual image is received, then the second virtual image is displayed greeting the first virtual image.

[0118] refer to Figure 5B , Figure 5B : is a schematic diagram of the interface provided by the embodiment of the present application. When the time for which no message is sent between the users participating in the conversation reaches a time threshold (pre-set time length, for example: 10 minutes), and one of the users sends a new message, the interactive behavior initiated by the virtual image corresponding to the user who sent the new message to the other virtual image is displayed, such as: greeting. For example: after the second user receives the message "tell you something big" from the first user, he does not reply for a long time. When the time for which the first user and the second user do not send new messages reaches 10 minutes, and the second user replies to the new message, the virtual image B of the second user displays the interactive behavior of greeting.

[0119] In the embodiment of the present application, by displaying the behavior representing the greeting, the prominence of the new message in the conversation interface is improved, the realism between the virtual images is improved, the user's conversation method is enriched, the conversation process is made more interesting, and the user's conversation experience is improved.

[0120] In some embodiments, the members participating in the conversation include a first member and a second member; Figure 3C , Figure 3Cis a flow chart of a session processing method provided in an embodiment of the present application; Figure 3A After step 301 in Figure 3C Steps 304 to 306 in are described in detail below.

[0121] In step 304, in response to a switching operation on the session interface, the session interface is switched to a virtual space.

[0122] For example, the virtual space is an interface for displaying interactive behaviors between virtual images, and the size of the virtual images in the virtual space is larger than the conversation interface. Figure 4D , Figure 4D is a schematic diagram of an interface provided by an embodiment of the present application. Figure 4A The interface switching operation of the middle session interface 401A displays the virtual space ( Figure 4D in the virtual space 401D).

[0123] In some embodiments, after step 304, virtual images of members participating in the conversation and interaction information between members participating in the conversation are displayed in a virtual space, wherein the interaction information includes at least one of the following: number of friend days, an interaction identifier representing the degree of interaction, and common attributes between members participating in the conversation.

[0124] For example, continue to refer to Figure 4D , the virtual space 401D includes the interaction parameters between the virtual image A and the virtual image B. The interaction parameters include: the length of time (in days) that the first user and the second user's accounts have established a friendship, common attributes between the first user and the second user (for example, games played by both, books read, music listened to, etc.), chat summary (for example, favorite conversation messages), interaction identification (for example, icons representing interaction). When the first user triggers the action control 403D in the virtual space 401D, multiple optional action controls 404D are displayed in the virtual space 401D to form a picture in the virtual space 402D.

[0125] In step 305, in response to a triggering operation on an interactive behavior control in the virtual space, a fourth interactive behavior between at least two virtual images is displayed in the virtual space.

[0126] Here, the fourth interactive behavior is the interactive behavior corresponding to the triggered interactive behavior control.

[0127] refer to Figure 4E , Figure 4E is a schematic diagram of an interface provided by an embodiment of the present application. Figure 4DThe interactive behavior between the virtual image A and the virtual image B is displayed in the virtual space by triggering an operation of any action control 404D in the virtual space 402D, for example: Figure 4E In the virtual space 401E, the action control corresponding to the rock-paper-scissors game is triggered, and the virtual image A and the virtual image B perform the interactive behavior of rock-paper-scissors game.

[0128] In step 306, a fourth interactive behavior between at least two virtual images is displayed in the conversation interface of the second member.

[0129] Continue to refer Figure 4E When the interactive behavior of rock-paper-scissors between virtual image A and virtual image B is displayed in the virtual space of the first user, the interactive behavior of rock-paper-scissors between virtual image A and virtual image B is synchronously displayed in the conversation interface 402E of the second user.

[0130] In some embodiments, the virtual materials decorating the virtual space include a virtual space background. In response to a switching operation on the virtual space background, the scene of the virtual space background is switched to a scene corresponding to the switching operation; and according to the time period at the current moment, the virtual space background of the scene matching the time period is displayed.

[0131] For example, the switching operation may be an operation for randomly switching or selecting a background. The switching operation for the virtual space background may be sliding, clicking, or long pressing the position of the virtual space background. Figure 4G , Figure 4G is a schematic diagram of the interface provided by the embodiment of the present application. The background of the virtual space can be switched according to the current time. For example, the background of the virtual space represents different lighting in the morning, noon, and evening. Figure 4G The virtual space in the game represents the images of the virtual space in different time periods. For example, the room is bright in the morning, the setting sun shines through the window in the evening, and the room is dim at night with the lamp shining.

[0132] In some embodiments, in response to the background of the virtual space or the conversation interface being switched to another background, current actions of at least two virtual images are switched to actions matching the other background.

[0133] For example, the action that matches other backgrounds is that there is an interaction between the action of the virtual image and the background scenery, for example: if there is a chair in the background, the virtual image sits on the chair, and for another example: if there is a sofa in the background, at least one of the virtual images sits on the sofa.

[0134] by Figure 5E The session interface 501E and Figure 5A The conversation interface 501A in the example is used for explanation. Figure 5EThere is a chair in the background of the avatar, and the avatar's action is to sit on the chair. Figure 5A , there is no chair in the background and the avatar is in a standing position.

[0135] In the embodiment of the present application, the action of the virtual image is associated with the background, and the action of the virtual image is switched as the background is switched, which enhances the realism of the virtual image and enriches the user's chatting method.

[0136] In some embodiments, after step 301 , in response to a trigger operation on the virtual image, a preconfigured fifth interactive behavior between at least two virtual images is displayed.

[0137] For example, the triggering operation for the virtual image can be a click operation or a long press operation for any virtual image displayed in the conversation interface. The pre-configured fifth interactive behavior can be associated with the application scenario. For example, if the current date of the conversation is during the Spring Festival holiday, the user clicks on the virtual image to trigger the interactive behavior of greeting the virtual images. If the current date of the conversation is the birthday of one of the members participating in the conversation, actions representing birthday wishes are displayed, such as making a wish with a cake, singing a birthday song, and wearing a birthday hat.

[0138] In some embodiments, the conversation interface also includes an interactive behavior control; in response to a trigger operation on the interactive behavior control, a fourth interactive behavior between at least two virtual images is displayed, wherein the fourth interactive behavior is an interactive behavior corresponding to the triggered interactive behavior control.

[0139] refer to Figure 4B , Figure 4B It is a schematic diagram of the interface provided in the embodiment of the present application.

[0140] The conversation interface 402B and the conversation interface 401B are schematic diagrams of the same conversation interface at different times, wherein the time corresponding to the conversation interface 402B is later than the time corresponding to the conversation interface 401B. When neither of the two users participating in the conversation sends a message, the virtual images A and B of the conversation interface 401B are in a pre-configured standby action, and there is no interaction between the two. The first user can click Figure 4A In the conversation interface 401A, the expression bar is displayed, and multiple action controls 403B (for example, rock-paper-scissors, fist-holding, roadblock, etc.) are displayed in the conversation interface 401A to form the screen shown in the conversation interface 401B. Taking the first user triggering the action control corresponding to rock-paper-scissors as an example, in the conversation interface 402B, the message "rock-paper-scissors" corresponding to the action control is displayed in the conversation interface 402B, and the interactive behavior of rock-paper-scissors is performed between the virtual image A and the virtual image B. If the virtual image A and the virtual image B complete the interactive behavior of rock-paper-scissors, the virtual image A and the virtual image B return to the initial action in the conversation interface 401B.

[0141] refer to Figure 4C , Figure 4C 401C and 402C are the session interfaces in the terminal device of the second user, and the session interface 402C and the session interface 401C are schematic diagrams of the same session interface at different times, wherein the time corresponding to the session interface 402C is after the time corresponding to the session interface 401C. Figure 4C The screen displayed in the session interface of the second user's terminal device after the action control is triggered in the first user's terminal device is shown. Since the second user does not trigger the expression bar and the action control, the second user's session interface does not display multiple action controls, but synchronously displays the screen of the virtual image A and the virtual image B performing the interactive behavior of rock-paper-scissors.

[0142] In some embodiments, the members participating in the conversation include a first member and multiple second members; step 301 can be implemented in the following manner: displaying a conversation interface and a first virtual image corresponding to the first member.

[0143] Example, reference Figure 6 The conversation interface 601 is a conversation interface for a multi-person group chat. The first user's virtual image A is displayed in the conversation interface 601. The conversation interface corresponding to each user only displays his or her own virtual image.

[0144] refer to Figure 3D , Figure 3D is a flow chart of a session processing method provided in an embodiment of the present application; Figure 3A After step 301, execute Figure 3D Steps 307 to 308 in are described in detail below.

[0145] In step 307, in response to an input operation on the conversation interface, and in response to the member name of the second member involved in the fourth message corresponding to the input operation, a second virtual image corresponding to the second member involved in the fourth message is displayed on the conversation interface.

[0146] For example, the content of the fourth message may be "@member name", where @ is an identifier used to remind the user in the session, or the fourth message may include the name of the second member. Figure 6 In the conversation interface 602 of the first user, when the first user sends a message "@B group friend" involving group friend B, the virtual image B of group friend B is displayed in the conversation interface 602 of the first user, and virtual image A and virtual image B can perform related interactive behaviors. At the same time, virtual image A is also synchronously displayed in the conversation interface of group friend B.

[0147] In step 308 , a first interaction behavior between at least two virtual images is displayed.

[0148] For example, the fourth message may include content related to the first interactive behavior. The first interactive behavior may be an act of greeting between virtual images, such as waving, shaking hands, etc. The first interactive behavior may be triggered by the fourth message, or by the user clicking on the virtual image or the action control.

[0149] In the embodiment of the present application, by obtaining the virtual image of the corresponding member through the message involving the member name in the group chat, the conversation mode in the group chat format is enriched, the realism of the virtual image is improved, and the user's conversation experience is improved.

[0150] In some embodiments, before step 303, data for forming an interactive behavior can be obtained in the following manner: performing text extraction on the first message input by the input operation to obtain the message text; searching in an action database based on the message text to obtain a target keyword that matches the message text, wherein the action database is used to store the following: interactive behavior data, a mapping relationship between interactive behaviors and keywords; extracting the interactive behavior data of the interactive behavior corresponding to the target keyword, wherein the interactive behavior data is used to configure the virtual image so that the virtual image performs the first interactive behavior.

[0151] For example, the first message is a voice message "Hello". Assume that the voice message is converted into text to obtain the message text "Hello". Based on "Hello", the matching keywords are retrieved in the action database. The interactive behavior data of the matching keywords is extracted, and the model of the virtual image is driven to perform corresponding actions based on the interactive behavior data.

[0152] For example, the degree of match between the message text and the keyword can be determined by the following methods: extracting features from the message text to obtain message text features, extracting features from each keyword to obtain keyword features, calculating the similarity between the message text features and the keyword features, and selecting the keyword with the highest similarity as the matching keyword. The processing of the interactive behavior data related to the message determination can be performed through a convolutional neural network model.

[0153] The embodiments of the present application enrich the form of chat conversations by displaying virtual images representing users participating in the chat on the conversation interface. When a message is input, the actions between the virtual images and the message are displayed, enriching the expression form of the chat. The virtual images are automatically made to display related actions by inputting messages, thereby improving the efficiency of controlling the virtual images, improving the efficiency of human-computer interaction, and improving the user's chat experience.

[0154] The following describes an exemplary application of the session processing method according to an embodiment of the present application in a practical application scenario.

[0155] In the related art, the user's virtual image is displayed on the right side of the conversation interface. The user can customize his or her virtual image by changing clothes, and the virtual image can be configured as a static posture or simple action. In the process of the user chatting with other users, the virtual image is only used for display. The static 2D virtual image of the related art is also very limited in expression and cannot provide vivid actions; the virtual image of the related art is not connected to the chat and cannot convey the emotional state of the user in the chat; the images of the two parties in the conversation are displayed in upper and lower grids, which does not conform to the state of face-to-face chat in a real chat environment, and it is difficult to generate further in-depth interaction.

[0156] The embodiment of the present application proposes a conversation processing method, which closely combines a virtual image with a chat message, and conveys the user's emotions that are difficult to fully express in words during the chat process through the interactive behavior, status, sound, and changes in the chat scene of the virtual image.

[0157] The session processing method provided in the embodiment of the present application is explained below with reference to the accompanying drawings.

[0158] refer to Figure 4A , Figure 4A It is a schematic diagram of the interface provided in the embodiment of the present application.

[0159] Conversation interface 402A and conversation interface 401A are schematic diagrams of the same conversation interface at different times, wherein the time corresponding to conversation interface 402A is after the time corresponding to conversation interface 401A. Conversation interface 401A includes virtual images corresponding to two users participating in the conversation, virtual image A and virtual image B, virtual image A represents the first user using the current terminal device, and virtual image B represents the second user who is conversing with the first user using another terminal device. The first user inputs a message "Let's watch it together, come in quickly." According to the message in conversation interface 401A, virtual image A and virtual image B in conversation interface 402A perform an interactive behavior of looking at each other.

[0160] refer to Figure 4B , Figure 4B It is a schematic diagram of the interface provided in the embodiment of the present application.

[0161] The conversation interface 402B and the conversation interface 401B are schematic diagrams of the same conversation interface at different times, wherein the time corresponding to the conversation interface 402B is later than the time corresponding to the conversation interface 401B. When neither of the two users participating in the conversation sends a message, the virtual images A and B of the conversation interface 401B are in a pre-configured standby action, and there is no interaction between the two. The first user can click Figure 4AIn the conversation interface 401A, the expression bar is displayed, and multiple action controls 403B (for example, rock-paper-scissors, fist-holding, roadblock, etc.) are displayed in the conversation interface 401A to form the screen shown in the conversation interface 401B. Taking the first user triggering the action control corresponding to rock-paper-scissors as an example, in the conversation interface 402B, the message "rock-paper-scissors" corresponding to the action control is displayed in the conversation interface 402B, and the interactive behavior of rock-paper-scissors is performed between the virtual image A and the virtual image B. If the virtual image A and the virtual image B complete the interactive behavior of rock-paper-scissors, the virtual image A and the virtual image B return to the initial action in the conversation interface 401B.

[0162] refer to Figure 4C , Figure 4C 401C and 402C are the session interfaces in the terminal device of the second user, and the session interface 402C and the session interface 401C are schematic diagrams of the same session interface at different times, wherein the time corresponding to the session interface 402C is after the time corresponding to the session interface 401C. Figure 4C The screen displayed in the session interface of the second user's terminal device after the action control is triggered in the first user's terminal device is shown. Since the second user does not trigger the expression bar and the action control, the second user's session interface does not display multiple action controls, but synchronously displays the screen of the virtual image A and the virtual image B performing the interactive behavior of rock-paper-scissors.

[0163] In some embodiments, Figure 4A For example, the first user long presses the virtual image A representing himself in the conversation interface, and the conversation interface switches to the virtual image details page of virtual image A. The virtual image details page displays the enlarged virtual image and the control for switching the appearance of the virtual image. The first user long presses the virtual image B representing himself in the conversation interface, and the virtual image details page of virtual image B is displayed. The virtual image details page of virtual image B displays the control for purchasing the same appearance and the number of likes received by virtual image B.

[0164] refer to Figure 4D , Figure 4D is a schematic diagram of an interface provided by an embodiment of the present application. Figure 4A The interface switching operation of the middle session interface 401A displays the virtual space ( Figure 4DThe virtual space 401D in the first user's virtual space 401D includes interaction parameters between the virtual image A and the virtual image B, including: the length of time (in days) that the first user and the second user's accounts have established a friendship relationship, common attributes between the first user and the second user (for example, games played by both, books read, music listened to, etc.), chat summary (for example, favorite conversation messages), interaction identification (for example, icons representing interaction). When the first user triggers the action control 403D in the virtual space 401D, multiple optional action controls 404D are displayed in the virtual space 401D to form a screen in the virtual space 402D.

[0165] In some embodiments, the background of the virtual space can be changed. When the user changes the background of the virtual space between the user and the friend, the corresponding friend will receive a prompt message. The background of the virtual space displays the social information between the user and the friend, such as: the number of friend days, the message board. The user can enter text messages in the message board, and the message content will be displayed near the virtual image in the conversation interface, or the prompt message "Your friend left a message for you" will be displayed. The virtual space also includes a bookshelf icon for displaying the books that the user has read. Clicking the bookshelf icon switches the virtual space to the reading interface and displays the books that the user has not finished reading.

[0166] refer to Figure 4G , Figure 4G is a schematic diagram of the interface provided by the embodiment of the present application. The background of the virtual space can be switched according to the current time, for example: the virtual space has different lighting in the morning, noon, and evening. Figure 4G The virtual space in the video represents the images of the virtual space in different time periods.

[0167] refer to Figure 4E , Figure 4E is a schematic diagram of an interface provided by an embodiment of the present application. Figure 4D The interactive behavior between the virtual image A and the virtual image B is displayed in the virtual space by triggering an operation of any action control 404D in the virtual space 402D, for example: Figure 4E In the virtual space 401E, the action control corresponding to the rock-paper-scissors game is triggered, and the interactive behavior of rock-paper-scissors is performed between the virtual image A and the virtual image B; when the interactive behavior of rock-paper-scissors is displayed between the virtual image A and the virtual image B in the virtual space of the first user, the interactive behavior of rock-paper-scissors is synchronously displayed between the virtual image A and the virtual image B in the session interface 402E of the second user.

[0168] In some embodiments, preconfigured interactive behaviors can also be triggered by clicking on the virtual image itself. The preconfigured interactive behaviors can be set according to the application scenario. For example, if the current date is the Spring Festival holiday, the user clicks on the virtual image to trigger the interactive behavior of New Year greetings between the virtual images.

[0169] refer to Figure 4F , Figure 4F 401F is a schematic diagram of an interface provided by an embodiment of the present application. In the conversation interface 401F, the second user is in the input state, and the conversation interface 401F of the first user displays a prompt message "the other party is inputting" and the interactive behavior of the second user's virtual image B representing the input, for example: a bubble is displayed above the head of the virtual image B. When the first user sends a message and waits for the second user to reply, as shown in the conversation interface 402F, the first user's virtual image A presents an action representing waiting, for example: the virtual image A looks at the virtual image B.

[0170] refer to Figure 5A , Figure 5A is a schematic diagram of the interface provided by the embodiment of the present application. When the terminal device is a computer, the conversation interface is as follows Figure 5A The first user can click on any avatar (avatar A or avatar B) to trigger the display of the operation panel 502A, which includes the avatar's clothing. The first user can click on the clothing to change the avatar A's clothing.

[0171] refer to Figure 5B , Figure 5B : is a schematic diagram of the interface provided by the embodiment of the present application. When the time for which no message is sent between the users participating in the conversation reaches a time threshold (pre-set time length, for example: 10 minutes), and one of the users sends a new message, the interactive behavior initiated by the virtual image corresponding to the user who sent the new message to the other virtual image is displayed, such as: greeting. For example: after the second user receives the message "tell you something big" from the first user, he does not reply for a long time. When the time for which the first user and the second user do not send new messages reaches 10 minutes, and the second user replies to the new message, the virtual image B of the second user displays the interactive behavior of greeting.

[0172] refer to Figure 5C , Figure 5C In the conversation interface 501C, the second user sends a file to the first user, and the file transfer progress is 50%. A virtual vehicle 502C representing the file transfer is displayed between the virtual images. Figure 5C The virtual vehicle in is a paper airplane.

[0173] In some embodiments, when the transmission is complete, the actions between the virtual images are converted into interactive behaviors of co-viewing.

[0174] refer to Figure 5D , Figure 5D In the conversation interface 501D, the second user sends a smiling sunglasses expression message to the first user, and the faces of the virtual images A and B simulate the smiling sunglasses expression.

[0175] refer to Figure 5E , Figure 5E The user can also send an invitation message to other users to share multimedia information together. If all users participating in the conversation agree to share, the virtual images will display a screen of browsing multimedia information together, for example, the interactive behavior of the virtual images listening to music together in the conversation interface 501E.

[0176] In some embodiments, when the background corresponding to the virtual image in the conversation interface is switched, the action of the virtual image is also switched together with the background. Figure 5E The session interface 501E and Figure 5A The conversation interface 501A in the example is used for explanation. Figure 5E There is a chair in the background of the avatar, and the avatar's action is to sit on the chair. Figure 5A , there is no chair in the background and the avatar is in a standing position.

[0177] In some embodiments, the session processing method provided in the embodiments of the present application is applied in a multi-user turn scenario, referring to Figure 6 , the conversation interface 601 is a conversation interface for a multi-person group chat. The first user's virtual image A is displayed in the conversation interface 601. The conversation interface corresponding to each user only displays his own virtual image. When the first user sends a message "@B group friend" involving group friend B, the first user's conversation interface 602 displays the virtual image B of group friend B. Virtual image A and virtual image B can perform related interactive behaviors. At the same time, virtual image A is also displayed synchronously in the conversation interface of group friend B.

[0178] In some embodiments, art materials of a three-dimensional model of a virtual image are obtained, and a three-dimensional scene is built through a virtual engine to generate a corresponding data packet, which is then transmitted to a content delivery network (CDN) for use by a client of an instant messaging software.

[0179] In some embodiments, data used to form the screen of the interactive behavior of the virtual image is stored by different servers. For example, the scene information server is used to provide scene information, the personal image server is used to store the virtual image of each user, and the business server requests corresponding material resources from the resource server based on the scene information and the virtual images of the users participating in the session. The material resources are used to form the screen of the interactive behavior of the virtual image in the rendering layer of the terminal device.

[0180] For example, the rendering layer can render the material resources by 3D rendering, and the rendering process includes loading the virtual image, loading the static 3D scene and the 3D skeletal animation of the virtual image. The rendering framework of the rendering layer can be Google's Filament rendering framework.

[0181] For example, three-dimensional skeletal animation can be obtained in the following way: load an animation file in gltf format, parse the animation file to obtain animation data, perform mixed calculation based on the animation data, time offset and the three-dimensional skeletal model of the virtual image, obtain each animation frame in the three-dimensional skeletal animation, and combine the animation frames into a three-dimensional skeletal animation according to the order of generation time.

[0182] In some embodiments, the user's click operation on the human-computer interaction interface of the terminal device can be determined in the following manner: when making a click judgment, the click coordinates need to be converted into a ray in three-dimensional space. The ray is calculated based on the x and y values ​​of the viewport coordinates, the camera's FOV, and the direction of the camera, and the obtained ray is characterized by the direction from the origin to the coordinate value.

[0183] For example, a bounding box is an algorithm for solving the optimal bounding space of a discrete point set. The basic idea is to use a geometric body (called a bounding box) that is slightly larger in volume and simpler in characteristics to approximately replace complex geometric objects. An AABB bounding box is used in the embodiment of the present application. Define tmin and tmax, and then traverse the three directions of the bounding box to calculate whether the ray intersects with the bounding box in that direction. If the ray direction has a component in that direction, calculate the intersection parameters and update the tmin and tmax values. If the ray direction has no component in that direction, determine the positional relationship between the ray starting point and the bounding box to determine whether they intersect. If they intersect, the instruction manual receives the corresponding click operation.

[0184] The embodiments of the present application display the virtual images of users participating in the conversation in a scenario-based manner, truly reproducing the face-to-face state of two people in daily chats, and the virtual images will display different actions and states according to the chat content, which can vividly convey the atmosphere of the chat; you can change your favorite scenes to make the chat more immersive; and actively initiate two-person interaction with the virtual images, presenting the interactive content in the form of virtual images, making the chat process closer to reality, more immersive, presenting a real interactive effect, and making the chat more interesting.

[0185] The following is a description of an exemplary structure of the session processing device 455 provided in the embodiment of the present application implemented as a software module. In some embodiments, Figure 2 As shown, the software modules stored in the session processing device 455 of the memory 450 may include: a display module 4551, configured to execute display of a virtual space, wherein the virtual space includes a virtual image corresponding to each member of the virtual space, and the virtual space can be decorated by virtual materials; the display module 4551 is configured to display a first input message in the virtual space in response to an input operation, and to display a first interactive behavior between at least two virtual images in the virtual space, wherein the first message includes content related to the first interactive behavior.

[0186] In some embodiments, the display module 4551 is configured to display a session interface before displaying the virtual space interface; in response to a trigger operation on the entrance to the virtual space, switch from displaying the session interface to displaying the virtual space, wherein members of the session interface include members of the virtual space.

[0187] In some embodiments, the display module 4551 is configured to execute the step of switching from displaying the virtual space to displaying the session interface in response to an exit operation on the virtual space after displaying the virtual space.

[0188] In some embodiments, the display module 4551 is configured to execute the step of switching from displaying the session interface to displaying the virtual space. In response to a trigger operation on the virtual space, and the session interface includes two members, the step of directly switching from displaying the session interface to displaying the virtual space is performed; in response to a trigger operation on the virtual space, and the session interface includes three or more members, a selection control is displayed, which includes all members of the session interface. In response to a selection operation in the selection control, at least two members are selected as members of the virtual space, and the step of switching from displaying the session interface to displaying the virtual space is executed.

[0189] In some embodiments, the type of the first message includes a file message; the display module 4551 is configured to display the transmission progress of the file message corresponding to the input operation, and in response to the completion of the file message transmission, an icon representing the file message is displayed; the first interactive behavior includes: a second interactive behavior and a third interactive behavior, and the display module 4551 is configured to display the second interactive behavior representing the file transmission between at least two virtual images, wherein the screen displaying the first interactive behavior includes a virtual vehicle carrying the file message; in response to the completion of the file message transmission and the file message is a file for viewing, a third interactive behavior of at least two virtual images viewing the file message is displayed.

[0190] In some embodiments, the type of the first message includes an expression message; the display module 4551 is configured to display a first interactive behavior of at least two virtual images simulating an expression corresponding to the expression message.

[0191] In some embodiments, the type of the first message includes an invitation message, wherein the invitation message is used to invite members participating in the conversation to browse the multimedia information; the display module 4551 is configured to display a first interactive behavior between at least two virtual images representing the sharing of multimedia information, and display the playback controls corresponding to the multimedia information, wherein the playback controls are used to start playing the multimedia information when triggered.

[0192] In some embodiments, members participating in the conversation include a first member and a second member, the virtual images include a first virtual image corresponding to the first member and a second virtual image of the second member having a conversation with the first member, and the first interactive behavior is initiated by the first virtual image to the second virtual image; the display module 4551 is configured to, after displaying the conversation interface, display the received second message in response to receiving the second message, and display a fourth interactive behavior initiated by the second virtual image to the first virtual image, wherein the second message is generated by the input operation of the second member, and the second message includes content related to the fourth interactive behavior.

[0193] In some embodiments, the display module 4551 is configured to display the behavior of the first virtual image representing message input during an input operation on the conversation interface; to display the behavior of the second virtual image representing message input during an input operation of the second member; and to display the behavior of the first virtual image representing waiting in response to not receiving a reply message to the first message.

[0194] In some embodiments, the display module 4551 is configured to, after displaying the conversation interface, display a new message in the conversation interface, and the time interval between the moment of displaying the new message and the moment of displaying the third message reaches a duration threshold, display an act of representing a greeting by a virtual image associated with the new message, wherein the virtual image associated with the new message is a virtual image corresponding to the member who inputs the new message, and the third message is the latest message before the new message.

[0195] In some embodiments, members participating in the conversation include a first member and a second member; the display module 4551 is configured to, after displaying the conversation interface, switch the conversation interface to a virtual space in response to a switching operation on the conversation interface; in response to a triggering operation on an interactive behavior control in the virtual space, display a fourth interactive behavior between at least two virtual images in the virtual space, wherein the fourth interactive behavior is an interactive behavior corresponding to the triggered interactive behavior control, and display the fourth interactive behavior between at least two virtual images in the conversation interface of the second member.

[0196] In some embodiments, the display module 4551 is configured to display virtual images of members participating in the conversation and interaction information between members participating in the conversation in the virtual space after switching the conversation interface to the virtual space in response to a switching operation on the conversation interface, wherein the interaction information includes at least one of the following: number of friend days, an interaction identifier representing the degree of interaction, and common attributes between members participating in the conversation.

[0197] In some embodiments, the virtual material includes a virtual space background, and the display module 4551 is configured to respond to a switching operation on the virtual space background, switch the scene of the virtual space background to a scene corresponding to the switching operation; and display the virtual space background of the scene matching the time period according to the current time period.

[0198] In some embodiments, the display module 4551 is configured as a method further comprising: in response to the background of the virtual space or the conversation interface being switched to another background, switching the current actions of at least two virtual images to actions matching the other background.

[0199] In some embodiments, the display module 4551 is configured to display a preconfigured fifth interaction behavior between at least two virtual images in response to a trigger operation on the virtual image after displaying the conversation interface.

[0200] In some embodiments, the conversation interface also includes an interactive behavior control; a display module 4551 is configured to display a fourth interactive behavior between at least two virtual images in response to a trigger operation on the interactive behavior control, wherein the fourth interactive behavior is an interactive behavior corresponding to the triggered interactive behavior control.

[0201] In some embodiments, members participating in the conversation include a first member and multiple second members; the display module 4551 is configured to display a conversation interface and a first virtual image corresponding to the first member; after displaying the conversation interface, in response to an input operation on the conversation interface, and the member name of the second member involved in the fourth message corresponding to the input operation, the second virtual image corresponding to the second member involved in the fourth message is displayed on the conversation interface, and a first interactive behavior between at least two virtual images is displayed, wherein the fourth message includes content related to the first interactive behavior.

[0202] In some embodiments, the display module 4551 is configured to perform text extraction on the first message input by the input operation to obtain the message text before displaying the first interactive behavior between at least two virtual images; search in the action database based on the message text to obtain the target keyword matching the message text, wherein the action database is used to store the following: interactive behavior data, and a mapping relationship between interactive behaviors and keywords; extract the interactive behavior data of the interactive behavior corresponding to the target keyword, wherein the interactive behavior data is used to configure the virtual image so that the virtual image performs the first interactive behavior.

[0203] The embodiment of the present application provides a computer program product, which includes a computer program or a computer executable instruction, and the computer program or the computer executable instruction is stored in a computer-readable storage medium. The processor of the electronic device reads the computer executable instruction from the computer-readable storage medium, and the processor executes the computer executable instruction, so that the electronic device executes the above-mentioned session processing method of the embodiment of the present application.

[0204] The present application embodiment provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions or computer programs are stored. When the computer-executable instructions or computer programs are executed by a processor, the processor will be caused to execute the session processing method provided by the present application embodiment, for example, Figure 3E The session processing method shown.

[0205] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface storage, optical disk, or CD-ROM; or it may be various devices including one or any combination of the above memories.

[0206] In some embodiments, computer executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0207] As an example, computer-executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file storing other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0208] As an example, the executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located at one site, or on multiple electronic devices distributed at multiple sites and interconnected by a communication network.

[0209] In summary, the embodiments of the present application enrich the form of chat conversations by displaying virtual images representing users participating in the chat on the conversation interface. When a message is input, the actions between the virtual images and the message are displayed, enriching the expression form of the chat. By inputting a message, the virtual image automatically displays related actions, thereby improving the efficiency of controlling the virtual image, improving the efficiency of human-computer interaction, and improving the user's chat experience.

[0210] The above is only an embodiment of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A session processing method, characterized in that: The method comprises: Displaying a virtual space, wherein the virtual space includes a virtual image corresponding to each member of the virtual space, and the virtual space can be decorated by virtual materials; In response to an input operation, displaying the input first message in the virtual space, and A first interactive behavior between at least two of the virtual images is displayed in the virtual space, wherein the first message includes content related to the first interactive behavior.

2. The method according to claim 1, characterized in that Before displaying the virtual space interface, the method further includes: Display the session interface; In response to a trigger operation for an entrance to the virtual space, switching from displaying the conversation interface to displaying the virtual space, wherein members of the conversation interface include members of the virtual space; After displaying the virtual space, the method further includes: In response to an exit operation on the virtual space, switching from the display virtual space to the display session interface.

3. The method according to claim 2, characterized in that The step of switching from displaying the conversation interface to displaying the virtual space comprises: In response to a trigger operation on the virtual space, and the conversation interface includes two members, directly switching from displaying the conversation interface to displaying the virtual space; In response to a trigger operation of the virtual space, and the session interface includes three or more members, a selection control is displayed, which includes all members of the session interface. In response to a selection operation in the selection control, at least two members are selected as members of the virtual space, and the step of switching from displaying the session interface to displaying the virtual space is executed.

4. The method according to claim 1, characterized in that: The type of the first message includes a file message; The displaying of the input first message comprises: Displaying the transmission progress of the file message corresponding to the input operation, and in response to the completion of the transmission of the file message, displaying an icon representing the file message; The first interactive behavior includes: a second interactive behavior and a third interactive behavior, and the display of the first interactive behavior between at least two of the virtual images includes: Displaying a second interactive behavior representing file transfer between at least two of the virtual images, wherein the screen displaying the first interactive behavior includes a virtual vehicle carrying the file message; In response to the file message transmission being completed and the file message being a file for viewing, a third interactive behavior of at least two of the virtual images viewing the file message is displayed.

5. The method according to claim 1, characterized in that The type of the first message includes an emoticon message; The displaying of a first interactive behavior between at least two of the virtual images includes: Display at least two of the virtual images to simulate a first interactive behavior of the expression corresponding to the expression message.

6. The method according to claim 1, characterized in that The type of the first message includes an invitation message, wherein the invitation message is used to invite members participating in the session to browse multimedia information; The displaying of a first interactive behavior between at least two of the virtual images includes: A first interactive behavior representing sharing of the multimedia information between at least two of the virtual images is displayed, and a playback control corresponding to the multimedia information is displayed, wherein the playback control is used to start playing the multimedia information when triggered.

7. The method according to claim 2, characterized in that: The members participating in the conversation include a first member and a second member, the virtual images include a first virtual image corresponding to the first member and a second virtual image of the second member having a conversation with the first member, and the first interactive behavior is initiated by the first virtual image to the second virtual image; After displaying the virtual space, the method further includes: In response to receiving the second message, displaying the received second message, and A fourth interactive behavior initiated by the second virtual image to the first virtual image is displayed, wherein the second message is generated by an input operation of the second member, and the second message includes content related to the fourth interactive behavior.

8. The method according to claim 7, characterized in that The method further comprises: During the input operation, displaying the first virtual image's behavior representing the message input; During the input operation of the second member, displaying the behavior of the second virtual image representing the message input; In response to not receiving a reply message to the first message, displaying the first virtual image to represent waiting behavior.

9. The method according to claim 2, characterized in that: After displaying the session interface, the method further includes: In response to displaying a new message in the conversation interface and the time interval between the moment when the new message is displayed and the moment when the third message is displayed reaching a time threshold, an act of displaying a greeting represented by a virtual image associated with the new message is displayed, wherein the virtual image associated with the new message is the virtual image corresponding to the member who inputs the new message, and the third message is the latest message before the new message.

10. The method according to claim 2, characterized in that The members participating in the conversation include a first member and a second member; After displaying the session interface, the method further includes: In response to a switching operation on the session interface, switching the session interface to a virtual space; In response to a triggering operation on an interactive behavior control in the virtual space, displaying a fourth interactive behavior between at least two of the virtual images in the virtual space, wherein the fourth interactive behavior is an interactive behavior corresponding to the triggered interactive behavior control, and In the conversation interface of the second member, a fourth interactive behavior between at least two of the virtual images is displayed.

11. The method according to claim 10, characterized in that After the session interface is switched to the virtual space in response to the switching operation on the session interface, the method further includes: The virtual images of the members participating in the conversation and the interaction information between the members participating in the conversation are displayed in the virtual space, wherein the interaction information includes at least one of the following: the number of days as friends, an interaction identifier representing the degree of interaction, and a common attribute between the members participating in the conversation.

12. The method according to claim 10, characterized in that The virtual material includes a virtual space background, and the method further includes: In response to a switching operation on the virtual space background, switching a scene of the virtual space background to a scene corresponding to the switching operation; According to the time period of the current moment, a virtual space background of a scene matching the time period is displayed.

13. The method according to claim 11, characterized in that The method further comprises: In response to the background of the virtual space or the conversation interface being switched to another background, current actions of at least two of the virtual images are switched to actions matching the other background.

14. The method according to claim 2, characterized in that After displaying the session interface, the method further includes: In response to a trigger operation on the virtual images, a preconfigured fifth interactive behavior between at least two of the virtual images is displayed.

15. The method according to claim 2, characterized in that The conversation interface also includes interactive behavior controls; The method further comprises: In response to a triggering operation on the interactive behavior control, a fourth interactive behavior between at least two of the virtual images is displayed, wherein the fourth interactive behavior is an interactive behavior corresponding to the triggered interactive behavior control.

16. The method according to claim 2, characterized in that The members participating in the conversation include a first member and a plurality of second members; The display session interface includes: Displaying the conversation interface and a first virtual image corresponding to the first member; After displaying the session interface, the method further includes: In response to an input operation on the conversation interface, and the member name of the second member involved in the fourth message corresponding to the input operation is displayed on the conversation interface, a second virtual image corresponding to the second member involved in the fourth message is displayed; and A first interactive behavior between at least two of the virtual images is displayed, wherein the fourth message includes content related to the first interactive behavior.

17. The method according to any one of claims 1 to 13, characterized in that Before displaying the first interactive behavior between at least two of the virtual images, the method further includes: Performing text extraction on the first message input by the input operation to obtain a message text; Based on the message text, a search is performed in an action database to obtain a target keyword matching the message text, wherein the action database is used to store the following contents: interactive behavior data, and a mapping relationship between interactive behaviors and keywords; The interactive behavior data of the interactive behavior corresponding to the target keyword is extracted, wherein the interactive behavior data is used to configure the virtual image so that the virtual image performs the first interactive behavior.

18. A conversation processing device, characterized in that: The device comprises: A display module configured to display a virtual space, wherein the virtual space includes a virtual image corresponding to each member of the virtual space, and the virtual space can be decorated by virtual materials; The display module is configured to display the input first message in the virtual space in response to an input operation, and The display module is configured to display a first interactive behavior between at least two of the virtual images in the virtual space, wherein the first message includes content related to the first interactive behavior.

19. An electronic device, characterized in that: The electronic device comprises: A memory for storing computer executable instructions; The processor is used to implement the session processing method described in any one of claims 1 to 17 when executing the computer executable instructions or computer programs stored in the memory.

20. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that: When the computer executable instructions or computer program are executed by a processor, the session processing method according to any one of claims 1 to 17 is implemented.

21. A computer program product comprising computer executable instructions or a computer program, characterized in that When the computer executable instructions or computer program are executed by a processor, the session processing method according to any one of claims 1 to 17 is implemented.

Images