Chat message processing method and apparatus, electronic device, and readable storage medium

Abstract

This invention discloses a chat message processing method and apparatus. The method includes: establishing an instant messaging group and a corresponding chat window based on a second user or a second user engaging in instant messaging with a first user; creating a message area in the chat window for the user sending the message, and generating an avatar display position in the message area; loading the user's 3D character into the avatar display position as a chat avatar; monitoring emoticon messages sent by users in the instant messaging group during the chat; and, in response to receiving emoticon messages sent by users, integrating the emoticon messages into the 3D character of the user's chat avatar; wherein, when integrating the emoticon messages into the 3D character, the 3D character displays facial expressions and / or actions according to the emoticon messages. This invention enhances the emotions conveyed between users during the chat process, enriches the message display methods, and improves the immersion and engagement of the communication.

Description

Technical Field

[0001] This invention relates to the field of Internet technology and also to the field of social application technology, specifically to a chat message processing method, apparatus, electronic device, computer-readable storage medium, and computer program product in a social application. Background Technology

[0002] With the development of internet technology, people are spending increasingly more time online, and many are transferring their real-world social relationships online. Consequently, applications and tools with social attributes play a vital role in people's lives and work. Social applications maintain connections and communication, provide online entertainment and leisure, and a significant portion of people use social media platforms for business activities and marketing, expanding their social circles to meet new friends and build broader relationships. Therefore, most people currently have a large number of users with whom they have established social relationships on their social applications or tools; these are commonly referred to as friends. To facilitate instant communication between users and their friends, current social applications or tools typically create chat windows. One chat window corresponds to one instant messaging group. When a message sent by a user is displayed in the chat window, a user's avatar is usually added before the message to identify the sender. However, static images used as user identifiers are monotonous, failing to attract users and providing insufficient information to enhance social relationships between users. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a chat message processing method, apparatus, electronic device, computer-readable storage medium, and computer program product to enrich the information displayed in the chat window and enhance the immersion and engagement of the ongoing communication.

[0004] According to one aspect of the present invention, an embodiment of the present invention provides a chat message processing method, applied to a terminal device of a first user, comprising:

[0005] Based on the second user or the second user who communicates with the first user in real time, an instant communication group and corresponding chat window are formed;

[0006] When a message is received from a user, a message area is created for the user in the chat window, and an avatar display slot is generated in the message area. Each message area includes one avatar display slot, and the user is either a first user or a second user.

[0007] Obtain the 3D character of the user who sent the message, and load the user's 3D character into the avatar display position as the chat avatar;

[0008] Display the message sent by the user in the user's current message area;

[0009] Monitor emoticons sent by users in the instant messaging group during the chat process; and

[0010] In response to receiving an emoji message sent by a user, the emoji message is integrated into the user's 3D avatar;

[0011] The 3D character includes a 3D character model. When the expression message is integrated into the 3D character, the 3D character model presents facial expressions and / or actions according to the expression message.

[0012] According to one aspect of the present invention, an embodiment of the present invention provides a chat message processing device, including a communication group creation module, a message area creation module, an avatar processing module, and a chat message module, wherein the communication group creation module is configured to form an instant communication group and a corresponding chat window based on a second user or a second user who is instantly communicating with the first user; the message area creation module is configured to create a message area for the user in the chat window when a user message is received, and generate an avatar display position in the message area, wherein one message area includes one avatar display position; the avatar processing module is configured to acquire... The system generates a 3D avatar of the user who sent the message and loads the 3D avatar into the avatar display area as a chat avatar. It also integrates the emoticon message sent by the user into the 3D avatar. The 3D avatar includes a 3D character model, and when the emoticon message is integrated into the 3D avatar, the 3D character model displays facial expressions and / or actions according to the emoticon message. The chat message module is configured to display the message sent by the user in the user's current message area, and when it detects an emoticon message sent by the user in the instant messaging group during the chat, it sends the emoticon message to the avatar processing module.

[0013] This invention provides an electronic device, including a processor and a memory storing computer program instructions. When the processor executes the computer program instructions, it implements the aforementioned chat message processing method in a social application.

[0014] This invention provides a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the aforementioned chat message processing method in a social application.

[0015] This invention provides a computer program product, which includes computer program instructions. When the computer program instructions are executed by a processor, they implement the aforementioned chat message processing method in a social application.

[0016] Compared to existing technologies such as flat emoticons and static or dynamic emoticons added separately to the message area, the embodiments of the present invention can reduce the area occupied in the message area, and enhance the emotions conveyed between users during the chat through facial expressions, head and body movements of the avatar. The message display methods are rich and the immersion and sense of engagement in the communication are improved. Attached Figure Description

[0017] To more clearly illustrate the implementation of the embodiments of the present invention, the accompanying drawings of the embodiments of the present invention will be briefly described below.

[0018] Figure 1 This diagram illustrates a social application system architecture based on servers and terminal devices.

[0019] Figure 2 This is a flowchart of a chat message processing method in a social application according to an embodiment of the present invention.

[0020] Figure 3 This is a schematic diagram of a chat window according to an embodiment of the present invention.

[0021] Figure 4 This is a flowchart illustrating the process of integrating emoticon messages into the user's 3D avatar according to an embodiment of the present invention.

[0022] Figure 5 This is a flowchart illustrating the processing after emoticon messages are integrated into the user's 3D avatar, according to another embodiment of the present invention.

[0023] Figure 6 This is a flowchart illustrating the processing after a chat window is created, according to another embodiment of the present invention.

[0024] Figure 7 This is a schematic block diagram of a terminal device 20 with a built-in chat message processing device according to an embodiment of the present invention.

[0025] Figure 8 This is a schematic block diagram of a terminal device 20 with a built-in chat message processing device according to another embodiment of the present invention.

[0026] Figure 9 This is a schematic diagram of the structure of an electronic device 60 provided in one embodiment of the present invention.

[0027] Figure 10This is a schematic diagram of the software structure of an exemplary terminal device according to an embodiment of the present invention. Detailed Implementation

[0028] The principles and spirit of the present invention will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided to make the principles and spirit of the present invention clearer and more thorough, enabling those skilled in the art to better understand and implement the principles and spirit of the present invention. The exemplary embodiments provided herein are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments described herein without inventive effort are within the scope of protection of the present invention.

[0029] Those skilled in the art will understand that embodiments of the present invention can be implemented as a system, apparatus, device, method, computer-readable storage medium, or computer program product. Therefore, the present invention can be specifically implemented in at least one of the following forms: entirely hardware, entirely software, or a combination of hardware and software. According to specific embodiments of the present invention, the present invention claims protection for a chat message processing method, apparatus, terminal device, server, computer-readable storage medium, and computer program product.

[0030] In this document, terms such as first, second, and third are used only to distinguish one entity (or operation) from another, and are not intended to require or imply any order or relationship between these entities (or operations).

[0031] Embodiments of the present invention can be applied to servers and terminal devices. Please refer to... Figure 1 , Figure 1 A schematic diagram of a social application system architecture based on a server and a terminal device is shown. The server 10 and the terminal device 20 respectively install the server-side software and client-side software of the social application, and the social application is implemented through the cooperation of the server and the client. The chat message method in the social application provided by this invention can be applied to the user's terminal device. The steps in the method can be implemented jointly by the client of the social application on the terminal device side or by the client of the social application on the terminal device side and the server-side social application server. For distinction, this invention refers to the user of the terminal device as the first user, and the user who establishes a chat activity with the first user as the second user. In the following description, when the described scheme applies to both the first user and the second user, the user is referred to as a user. The chat activity between users is presented on the display interface of the user's terminal device through a chat window.

[0032] The principles and spirit of the present invention will be explained in detail below through several exemplary embodiments or representative implementations.

[0033] Figure 2 This is a flowchart of a chat message processing method in a social application according to an embodiment of the present invention, wherein the method is applied to a first user's terminal device. Figure 3 This is a schematic diagram of a chat window according to an embodiment of the present invention, combined with... Figure 2 and Figure 3 The method described in this embodiment includes the following steps:

[0034] Step S1: Based on the second user or the second user who is in instant communication with the first user, an instant communication group and a corresponding chat window are created. See also Figure 3 , Figure 3 The displayed interface is a chat window created by a communication group. In this embodiment, the communication group has two users. The right side is the first user, which is the user of the user terminal device that presents the current chat window, and the left side is the second user, who is the first user's friend.

[0035] Step S2: Upon receiving a message from a user, a message area is created for the user in the chat window, and an avatar display slot is generated within the message area. Each message area includes one avatar display slot. See also... Figure 3 When two users send messages respectively, a corresponding message area 11 is created for each user, and each message area 11 includes an avatar display position 12.

[0036] Step S3: Obtain the 3D avatar of the user who sent the message, and load the user's 3D avatar into the avatar display position as the chat avatar. The 3D avatar, as the user's avatar, includes a 3D character model, which can be set by the user or randomly assigned by the system. In an optional embodiment, to vividly represent the user avatar, the 3D character model can be dynamically displayed in the avatar display position, for example, changing actions or states according to preset conditions, such as user-defined states or random actions. Dynamic avatars can effectively enhance immersion and engagement.

[0037] Step S4: Display the message sent by the user in the user's current message area. See also Figure 3The message sent by the user is displayed in its corresponding message area 11. In an optional embodiment, each message is displayed in a message box 13, and the shape of the message box 13 is set to give it a pointing direction. In an optional embodiment, in the message area 11, the avatar display position 12 and the message display area are arranged side by side, and the pointing shape of the message box 13 is opposite to that of the avatar display position 12, thereby achieving the effect of the message pointing to the avatar, avoiding the problem of confusion about the source of the message when two users are sending messages. When multiple messages are displayed consecutively, the multiple consecutive messages are arranged from top to bottom, such as... Figure 3 The arrangement of two consecutive messages sent by the second user in the message box 13. Additionally, the background color of message box 13 can be set to further distinguish messages sent by different users.

[0038] Step S5: Monitor whether users in the instant messaging group send emoticon messages during the chat. If a user sends an emoticon message, in step S6, integrate the emoticon message into the user's 3D avatar so that the user's 3D avatar makes corresponding expressions or actions to convey emotions.

[0039] See Figure 4 This is a flowchart illustrating the process of integrating emoji messages into a user's 3D avatar according to an embodiment of the present invention. In this embodiment, when a user selects an emoji and sends it through the input bar in a chat window, the emoji message is integrated into the user's 3D avatar through the following steps:

[0040] Step S61: Obtain the emoji.

[0041] Step S62: Query the text description information of the emoticon.

[0042] Step S63: Determine whether the text description information of the emoticon is obtained. If the text description information of the emoticon is obtained, determine the expression data in step S64 based on the text description information. If there is no text description information, identify the key points of the emoticon in step S65 to obtain the expression data. The expression data includes a time series of one or more facial parts, such as one or more facial bones, facial muscles, etc. Data of the same facial part with multiple time information constitutes a time series of facial part data. When there are multiple facial parts, there are multiple such time series, and these time series have the same time information. In one embodiment, the server or client has a pre-set correspondence table between expression data and expression type (or description information), with each expression corresponding to a set of expression data. For example, as shown in Table 1 below:

[0043] Table 1

[0044]

[0045]

[0046] When the file description information is obtained, the expression type is determined based on the file description information, and then the expression data with the corresponding serial number is extracted from the database based on the expression type.

[0047] Step S66: Obtain the user's 3D character data, such as data on multiple facial parts including facial bones and muscles.

[0048] Step S67: Based on the facial expression data, modify the 3D character data to obtain an image sequence, and generate an facial expression animation file according to a set frame rate, which is a 3D animation file. The first image in the image sequence is the current 3D character image. For example, the facial data in the facial expression data is divided into multiple groups according to time information. Starting from the first group, the changes in the data of the same facial part in two adjacent groups are calculated, and the changes in the facial part data are mapped to the corresponding changes in the facial part data of the 3D character to obtain new data for the corresponding facial part of the 3D character. Similarly, all the new facial part data are obtained. Combining the new facial data of the 3D character constitutes an image. Processing other groups of data in the same way to obtain multiple groups of images forms an image sequence.

[0049] Step S68: Load and play the 3D animation file to the avatar display position.

[0050] In another embodiment, when a user selects to record an expression through the input bar in the chat window, expression data can be generated based on the recorded video. For example, a video is first recorded, and then a certain number of image frames are extracted at certain intervals or frame by frame to obtain the positional data of key facial features with time correspondence, thereby obtaining a time series of key facial feature data, and then the above steps S66 to S68 are executed.

[0051] For example, when a user records an expression, the position of the corresponding facial part of the user can be captured at a certain frequency according to the facial part of the 3D character in the avatar that needs to be changed during fusion. Each time the position of the corresponding facial part of the user is obtained, the position difference of the facial part is calculated based on the position of the corresponding facial part captured in the previous capture, and the position difference of the part is mapped to the position difference of the 3D character's facial part. Based on the difference, the position of the 3D character's facial part is changed, so that the recorded expression can be fused into the 3D character of the avatar in real time.

[0052] For example, the system database may contain pre-set 3D animations and corresponding extracted facial expression data. If pre-set facial expression data is not available, facial data can be extracted from the pre-set 3D animations to obtain a time series of facial features, thus obtaining the facial expression data. Then, by executing steps S66 to S68 as described above, the facial expression animation file is obtained.

[0053] In the foregoing embodiments, the facial expression data only included data on facial features such as bones and muscles. However, in other embodiments, it may also include motion data. The motion may be, for example, a pre-set head movement matched to the expression type, such as tilting the head while smiling or shaking the head back and forth while laughing, displaying a forward and backward tilting motion. The motion may also be a pre-set body movement matched to the expression type, such as raising hands above the head while arching the back in a frightened expression, enhancing the effect through significant body movements.

[0054] Compared to existing technologies such as flat emoticons and static emoticon images added separately to the message area, this invention enhances the sense of immersion in chatting by using facial expressions, head movements, and body movements of the avatar, thus strengthening the effect of the expression.

[0055] exist Figure 3 In the illustrated embodiment, the chat avatar includes an avatar display window and a screenshot of a 3D character's half-body image that protrudes from the boundary of the avatar display window. Since message areas need to be frequently created for users during chat, the 3D character needs to be loaded frequently. Loading a dynamic 3D character requires processing a large amount of data, thus consuming significant memory and CPU resources. When the user's terminal device has insufficient performance, lag may occur, potentially affecting other applications. Therefore, to avoid excessive memory and CPU consumption, this embodiment only captures a half-body image of the 3D character based on the size of the avatar display window and overlays this half-body image onto the avatar display window. This significantly reduces the data processing volume when loading the avatar. Furthermore, this embodiment ensures that the half-body image of the 3D character protrudes from the boundary of the avatar display window, increasing the avatar's three-dimensionality.

[0056] Figure 5 This is a flowchart illustrating the processing after incorporating emoticon messages into the user's 3D avatar, according to another embodiment of the present invention. This embodiment further includes the following steps after incorporating emoticon messages into the user's 3D avatar:

[0057] Step S71: Generate textual descriptions of the facial expressions and / or movements of the 3D characters in the 3D animation file. For example... Figure 3 The text description information in section 14.

[0058] Step S72: Fill the text description information into the text display position next to the chat avatar.

[0059] Step S73: Display the associated identifier 15 in the text display position.

[0060] Step S74: Establish the association relationship between the association identifier 15 and the 3D animation file.

[0061] The text displayed next to the chat avatar can be one or more; when there are multiple, they are arranged next to the avatar in the order of the creation time of their corresponding 3D animation files. For example... Figure 3 As shown, there are four text descriptions of facial expressions and / or actions, arranged in chronological order from top to bottom.

[0062] Step S75: Monitor the user's operations on the associated identifier.

[0063] Step S76: Determine whether a user has performed a playback operation on the associated identifier. If a user has performed a playback operation on the associated identifier, in step S77, load and play the 3D animation file to the avatar display position, and replace it with a screenshot of the 3D character image after the 3D animation file finishes playing.

[0064] This embodiment allows users to repeatedly view emoji messages they send or receive.

[0065] Figure 6 This is a flowchart illustrating the processing after a chat window is created, according to another embodiment of the present invention. (Reference) Figure 3 In this embodiment, after a chat window is established on the user's terminal device display interface, the following steps are further included:

[0066] Step S11: Obtain the 3D character of each user in the instant messaging group.

[0067] Step S12: Load each user's 3D character into the user status field of each user in the chat window. For example... Figure 3 The status card 21 includes two user status bits, which respectively display the 3D characters 20 of the first user and the second user.

[0068] Step S13: Obtain the first state data for each user. The first state data is, for example, data set by the user through a client, which may be a piece of text information, graphic identifier, or audio information set by the user, and its storage location may be, for example, the client and / or the server. For the first user's first state data, the first state data can be obtained by reading the user settings data stored in the client. For the second user's first state data, the first user's client sends a request to the server to obtain the second user's first state data.

[0069] Step S14: The first state data of each user is filled into the corresponding user state bit. For example... Figure 3 As shown, the first user's first state data 22 is a text and graphic identifier, placed below the 3D character.

[0070] Furthermore, it may also include an optional step S15, which synchronously displays the second user's first status data in the second user status bit following the latest chat message. For example... Figure 3 In this system, when the second user changes their first status data, it is displayed in status card 21 and simultaneously shown in the second user status bit 23 after the latest chat message. This ensures that when a user's status changes, it is not only displayed in the status card but also simultaneously displayed after the latest chat message, allowing the other party in the chat to promptly and intuitively perceive the change in the other party's status, further enhancing the sense of immersion in the communication. To obtain the other user's updated status in a timely manner, the following steps are also included:

[0071] Step S16: Monitor the first status data of each user in the instant messaging group. When a user modifies their first status data, the user's client sends this data to the server, which then forwards it to the client of the other user with whom the user is currently chatting.

[0072] Step S17: Determine whether the user's first state data has changed. If the user's first state data has changed, update the first state data in the user's user state bit in step S18.

[0073] Step S19: Generate a status data update notification and display it in the status update notification slot of the current chat window. In one embodiment, the status update notification slot is, for example, the second user status slot in step S15. The two can be in the same position or in different positions. When they are not in the same position, the new first status data is also synchronously displayed in the second user status slot.

[0074] See Figure 3To highlight the social relationship between users, this embodiment includes an intimacy information bit 24 between the user status bits of two users. When adding data to the user status bits, the embodiment also includes obtaining the intimacy data of the first and second users and adding this intimacy data to the intimacy information bit. The intimacy data may include graphic identifiers, such as identifiers with corresponding shapes and colors based on the intimacy relationship. For example, when the two are friends, the identifier might be a smiley face, and when they are lovers, the identifier might be a heart shape. The intimacy data may also include characters such as text and numbers, for example, with numerical values ​​representing the degree of intimacy. The intimacy data is calculated using a certain logical algorithm when the two users establish a social relationship, updated in real-time or periodically, and stored on both the user's client and the server.

[0075] See Figure 3 In one embodiment, the user status bit of the second user may also include a user tag 25, which is used to specifically identify its relationship with the first user, as a supplement to the intimacy data.

[0076] To facilitate users' viewing of personal information in the current chat activity within the chat window, after establishing a user status bit, a relationship is also established between the user status bit and the user's personal information page. When a user performs an operation on the user status bit by clicking or other means, they can be redirected to the user's personal information page, which will then be displayed.

[0077] In this embodiment, after obtaining the 3D characters, first state data, intimacy data, user tag data, etc. of the two users through the aforementioned processing, this information is presented in the chat window in the form of cards, thereby reducing the amount of data processing during the chat process.

[0078] In addition to monitoring messages sent by users during chat, this invention also monitors the time interval between consecutive messages sent by the same user. A consecutive message refers to multiple messages sent by the same user without any messages from other users interspersed. When the time interval between consecutive messages sent by the same user is less than a first threshold, such as 1 minute or 2 minutes, these consecutive messages are displayed in the same current message area in chronological order. Figure 3 As shown, the second user sent one message and then another. Since no other user's messages were inserted in between, these two messages are considered consecutive. The time interval between the two messages can be determined based on their sending times. Because this time interval is less than a set first threshold, such as 1 minute, the second message is displayed in the current message area.

[0079] By displaying consecutive messages that meet the time requirements in the same current message area in this embodiment, the number of operations related to creating message areas is reduced, thereby improving data processing speed and making it easier for users to view messages.

[0080] To avoid message confusion caused by excessively large time spans between consecutive messages sent by the same user, one embodiment monitors the message time span within the current message area. For example, it calculates the difference between the sending time of the latest consecutive message and the sending time of the first message. If this difference is less than a second threshold, a new message area is not needed, and the current message is still displayed in the current message area. If the difference is greater than the second threshold, a new message area is created for the user, and the message is displayed in the newly created message area. Visually, the user's avatar is displayed first, followed by the message next to the avatar.

[0081] From the perspective of message quantity, too many messages in a single message area can cause message confusion. For example, if the messages in a message area cannot fit on the current screen, and the screen moves upwards, the entire screen may be filled with messages without the user's avatar, which can be confusing for the user. To avoid this, the number of consecutive messages in the current message area is monitored. If the number of consecutive messages in the current message area exceeds a third threshold, a new message area is created. Alternatively, the length of the current message area can be monitored; if the length of the current message area exceeds the length of the screen, a new message area is created.

[0082] Of course, in addition to the various displayed contents obtained after the above processing, the chat window also includes other displayed content, such as displaying the second user's username in the window's title bar, and other information such as the second user's geographical location, the entrance to the other party's virtual house, and online status information. It also includes some jump indicators, such as... Figure 3 The top left corner has a return icon, the top right corner has a more information display icon, the bottom of the interface has an input bar, and the input window entry icon in the input bar is used to input various types of information, etc.

[0083] It should be noted that, for clarity, all embodiments of the present invention are described as a combination of a series of actions or processes. Those skilled in the art should understand that the implementation process is not limited by the order of the described actions or processes, and some steps in the embodiments of the present invention may be processed in other orders or simultaneously.

[0084] Corresponding to the method provided by the present invention, the present invention also provides a chat message processing device, which is placed in a terminal device 20. Figure 7This is a schematic block diagram of a terminal device 20 with a built-in chat message processing device according to an embodiment of the present invention. In this embodiment, the terminal device 20 includes at least a display screen 21 and a communication module 22. The display screen is, for example, a mobile phone display screen, with display and touch functions. The communication module 21 is, for example, a 2G / 3G / 4G / 5G communication module, which can communicate with a server. The terminal device also includes other hardware modules (not shown), such as a camera. The chat message processing device 200 includes a communication group creation module 201, a message area creation module 202, an avatar processing module 203, and a chat message module 204. The communication group creation module 201 is configured to create an instant communication group and a corresponding chat window based on a second user or a second user who is communicating instantly with the first user. The message area creation module 202 is configured to create a message area for the user in the chat window when a user message is received, and generate an avatar display slot in the message area, wherein one message area includes one avatar display slot; the avatar processing module 203 is configured to obtain the 3D character of the user who sent the message, load the user's 3D character into the avatar display slot as a chat avatar, and integrate the emoticon message sent by the user into the user's 3D character; wherein the 3D character includes a 3D character model, and when the emoticon message is integrated into the 3D character, the 3D character model presents facial expressions and / or actions according to the emoticon message; the chat message module 204 is configured to display the message sent by the user in the user's current message area, and when monitoring the emoticon message sent by the user in the instant messaging group during the chat, send the emoticon message to the avatar processing module.

[0085] Furthermore, after acquiring the 3D character of the user sending the message, the avatar processing module 203 captures a screenshot of the upper body and loads it into the avatar display position in the message area. When an emoticon message is received, a 3D animation file is generated based on the emoticon message and the 3D character model. The first frame of the 3D animation is a screenshot of the 3D character in the current chat avatar. Loading and playing the 3D animation file into the avatar display position achieves the function of integrating the emoticon message into the 3D character. Further, the avatar processing module 203 also generates text descriptions of the facial expressions and / or actions of the 3D character in the 3D animation file and sends them to the chat message module 204. The chat message module 204 adds the text descriptions to the text display position next to the chat avatar and establishes an association between the text display position and the 3D animation file. Then, an association identifier linking the 3D animation file is displayed in the text display position. Furthermore, the text display position can be one or more. When there are multiple positions, the chat message module 204 is arranged next to the chat avatar in the order of the generation time of the 3D animation file associated with it.

[0086] The avatar processing module 203 monitors the user's operation on the associated identifier, and in response to receiving the user's playback operation on the associated identifier, loads and plays the 3D animation file to the avatar display position, and replaces it with a screenshot of the 3D character image after the 3D animation file finishes playing.

[0087] The message area creation module 202 monitors the time interval between consecutive messages sent by the same user during the chat. In response to a time interval less than a first threshold, the consecutive messages are displayed in the same current message area in chronological order. The message area creation module 202 also monitors the message time span within the current message area, creating a new message area when the message time span exceeds a second threshold. Optionally, the message area creation module 202 can also monitor the number of consecutive messages in the current message area, creating a new message area when the number of consecutive messages exceeds a third threshold. Optionally, the message area creation module 202 can also monitor the length of the current message area, creating a new message area when the length of the current message area exceeds the screen interface length.

[0088] Figure 8 This is a schematic block diagram of a terminal device with a built-in chat message processing device 200 according to another embodiment of the present invention. Figure 7The embodiment shown also includes a status module 205, which obtains the 3D character of each user in the instant messaging group, loads the 3D character of each user into the user status bit of each user in the chat window, obtains the first status data of each user, and fills the first status data of each user into the corresponding user status bit.

[0089] Furthermore, the status module 205 can optionally monitor whether the first status data of each user in the instant messaging group has changed; when the user's first status data changes, it updates the first status data in the user's user status bit and generates a status data update notification, which is displayed in the status update notification bit of the current chat window.

[0090] Furthermore, the status module 205 can optionally acquire the intimacy data of the first user and the second user after establishing the chat window; fill the intimacy data into the intimacy information bit between the status bits of the first user and the second user; establish the association between the user status bit and the user's personal information page; and jump to the user's personal information page when an operation on the user status bit is detected.

[0091] Those skilled in the art will understand that the embodiments described herein are preferred embodiments, and the actions, steps, modules, or units involved are not necessarily essential to the embodiments of the present invention. In the above embodiments, the descriptions of each embodiment have their own emphasis; for parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0092] Figure 9 This is a schematic diagram of the structure of an electronic device 60 according to an embodiment of the present invention. The electronic device 60 includes a processor 61, a memory 62, and a communication bus for connecting the processor 61 and the memory 62. The memory 62 stores a computer program that can run on the processor 61. When the processor 61 runs the computer program, it can execute or implement the steps of the methods in the various embodiments of the present invention. The electronic device 60 also includes a communication interface for receiving and sending data. The electronic device 60 can be a server or a terminal device in the embodiments of the present invention.

[0093] In some embodiments, processor 61 may be a central processing unit (CPU), graphics processing unit (GPU), application processor (AP), modem processor, image signal processor (ISP), controller, video codec, digital signal processor (DSP), baseband processor, neural-network processing unit (NPU), etc. Processor 61 may also be other general-purpose processors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. General-purpose processors may be microprocessors or any conventional processor. The neural network processor (NPU), by drawing inspiration from biological neural network structures, can rapidly process input information and continuously learn itself. The NPU electronic device 60 can realize applications such as intelligent cognition, including image recognition, face recognition, semantic recognition, speech recognition, and text understanding.

[0094] In some embodiments, memory 62 may be an internal storage unit of electronic device 60, such as a hard disk or memory of electronic device 60; memory 62 may also be an external storage device of electronic device 60, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on electronic device 60. Memory 62 may include both internal storage units and external storage devices of electronic device 60. Memory 62 can be used to store operating system, application programs, bootloader, data, and other programs, such as program code of computer programs. Memory 62 includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or compact disc read-only memory (CD-ROM). Memory 62 is used to store program code executed by electronic device 60 and data transmitted. Memory 62 can also be used to temporarily store data that has been output or will be output.

[0095] Those skilled in the art will understand that Figure 9 This is merely an example of electronic device 60 and does not constitute a limitation on electronic device 60. Electronic device 60 may include more or fewer components than shown, or combine certain components, or include different components, such as input / output devices, network access devices, etc.

[0096] Figure 10 This is a schematic diagram of the software structure of a terminal device according to an embodiment of the present invention. Taking the Android operating system as an example, in some embodiments, the Android system is divided into four layers: the application layer, the application framework layer (Framework, FWK), the system layer, and the hardware abstraction layer. The layers communicate with each other through software interfaces.

[0097] First, the application layer can include multiple application packages, which can be various application apps such as calling, camera, video, navigation, weather, instant messaging, education, etc., or AR-based application apps.

[0098] Second, the Application Framework Layer (FWK) provides Application Programming Interfaces (APIs) and a programming framework for applications within the application layer. The application framework layer can include predefined functions, such as functions for receiving events sent by the application framework layer.

[0099] The application framework layer may include a window manager, a resource manager, and a notification manager, among others.

[0100] The window manager manages the windowed applications. It can determine the screen size, the presence of a status bar, lock the screen, and capture screenshots. The content provider stores and retrieves data, making it accessible to applications. This data can include videos, images, audio, made and received phone calls, browsing history and bookmarks, and phonebook entries.

[0101] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0102] The notification manager allows applications to display notifications in the status bar. These notifications can be used to deliver informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the notification manager can be used to notify users of download completion or message alerts. The notification manager can also display notifications as icons or scrolling text in the top status bar, such as notifications from background applications, or as dialog boxes on the screen. Examples include displaying text messages in the status bar, emitting sounds, vibrating electronic devices, and flashing indicator lights.

[0103] In addition, the application framework layer may include a view system, which includes visual controls, such as controls for displaying text and controls for displaying images. The view system can be used to build the application. The display interface can consist of one or more views; for example, the display interface of a text notification icon may include a view for displaying text and a view for displaying images.

[0104] Third, the system layer can include multiple functional modules, such as sensor service modules, physical state recognition modules, 3D graphics processing libraries (e.g., OpenGLES), and so on.

[0105] The sensor service module monitors sensor data uploaded by various sensors at the hardware layer to determine the physical state of the phone; the physical state recognition module analyzes and recognizes user gestures, faces, etc.; and the 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0106] In addition, the system layer may include a surface manager and a media library. The surface manager manages the display subsystem and provides 2D and 3D layer blending for multiple applications. The media library supports playback and recording of various common audio and video formats, as well as still image files.

[0107] Finally, the hardware abstraction layer is the layer between hardware and software. The hardware abstraction layer can include display drivers, camera drivers, sensor drivers, etc., used to drive the relevant hardware in the hardware layer, such as displays, cameras, and sensors.

[0108] This invention also provides a computer-readable storage medium storing a computer program or instructions that, when executed, implement the steps of the methods described in the above embodiments.

[0109] This invention also provides a computer program product, including a computer program or instructions, which, when executed, implement the steps of the methods involved in the above embodiments. For example, the computer program product may be a software installation package.

[0110] Those skilled in the art should understand that the functions of the methods, steps, or related modules / units described in the embodiments of the present invention can be implemented, in whole or in part, by software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product, or by a processor executing computer program instructions. The computer program product includes at least one computer program instruction, which can be composed of corresponding software modules. These software modules can be stored in RAM, flash memory, ROM, EPROM, EEPROM, registers, hard disk, portable hard disk, read-only optical disk (CD-ROM), or any other form of storage medium known in the art. The computer program instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer program instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium, or a semiconductor medium (e.g., SSD).

[0111] Regarding the various devices / products described in the above embodiments, the modules / units included can be software modules / units, hardware modules / units, or a combination of both. For example, for devices / products applied to or integrated into a chip, all of its modules / units can be implemented using hardware methods such as circuits, or at least some modules / units can be implemented using software programs running on a processor integrated within the chip, while the remaining modules / units can be implemented using hardware methods such as circuits. Similarly, for devices / products applied to or integrated into a terminal, all of its modules / units can be implemented using hardware methods such as circuits, or at least some modules / units can be implemented using software programs running on a processor integrated within the terminal, while the remaining modules / units can be implemented using hardware methods such as circuits.

[0112] The above description is merely a specific embodiment of the present invention. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the protection scope of the present invention.

Claims

1. A chat message processing method, characterized in that, The terminal equipment used by the first user includes: Based on the second user or the second user who communicates with the first user in real time, an instant communication group and corresponding chat window are formed; When a message is received from a user, a message area is created for the user in the chat window, and an avatar display slot is generated in the message area. Each message area includes one avatar display slot, and the user is either a first user or a second user. Obtain the 3D character of the user who sent the message, and load the 3D character of the user into the avatar display position as the chat avatar. The chat avatar includes a display window and a half-body image screenshot of the 3D character that protrudes from the boundary of the display window. Display the message sent by the user in the user's current message area; Monitor the emoticon messages sent by the first user in the instant messaging group during the chat process; and In response to receiving an emoji message sent by a first user, the emoji message is integrated into the 3D character of the chat avatar of the user who sent the emoji message; The 3D character includes a 3D character model. When the emoticon message is integrated into the 3D character, a 3D animation file is generated based on the emoticon message and the 3D character model of the user who sent the emoticon message. The 3D animation file is loaded and played in the avatar display position of the user who sent the emoticon message, so that the 3D character model of the user who sent the emoticon message presents facial expressions and / or actions according to the emoticon message. The process of generating 3D animation files further includes: Create an association identifier for the 3D animation file, and establish the association relationship between the association identifier and the 3D animation file; Monitor user operations on the associated identifier; and In response to the user's playback operation on the associated identifier, the 3D animation file is loaded and played to the avatar display position, and replaced with a screenshot of the 3D character image after the 3D animation file finishes playing.

2. The method according to claim 1, characterized in that, The first frame of the 3D animation is a screenshot of the 3D character in the current chat avatar.

3. The method according to claim 1, characterized in that, The process of generating 3D animation files further includes: Generate textual descriptions of the facial expressions and / or movements of the 3D characters in the 3D animation file; and The text description information is added to the text display area next to the chat avatar.

4. The method according to claim 3, characterized in that, The text display area can be one or more. When there are multiple text display areas, they are arranged in the order of the generation time of their corresponding 3D animation files in the area next to the chat avatar.

5. The method according to claim 1, characterized in that, After establishing the chat window, the following further steps are included: Obtain the 3D avatar of each user in the instant messaging group; Load each user's 3D character into the user status bit of each user in the chat window; Obtain the initial state data for each user; and The first state data of each user is respectively filled into the corresponding user state bit.

6. The method according to claim 5, characterized in that, Further includes: The second user's first status data is displayed synchronously in the second user status field following the latest chat message.

7. The method according to claim 5 or 6, characterized in that, The first status data includes text and / or graphic identifiers.

8. The method according to claim 5, characterized in that, Further includes: Monitor whether the initial status data of each user in the instant messaging group has changed; In response to a change in the user's first state data, update the first state data in the user's user state bit; as well as Generate a status data update notification and display it in the status update notification slot of the current chat window.

9. The method according to claim 5, characterized in that, After establishing the chat window, the following further steps are included: Obtain the intimacy data between the first and second users; and The intimacy data is added to the intimacy information bit between the user status bit of the first user and the user status bit of the second user.

10. The method according to claim 9, characterized in that, The intimacy data includes textual and / or graphic identifiers.

11. The method according to claim 5, characterized in that, Further includes: Establish an association between the user status bit and the user's personal information page; in response to an operation on the user status bit, redirect to the user's personal information page.

12. The method according to claim 1, characterized in that, Further includes: Monitor the time interval between consecutive messages sent by the same user during a chat session; as well as In response to the time interval between consecutive messages sent by the same user being less than a first threshold, the consecutive messages are displayed in the same message area in chronological order.

13. The method according to claim 12, characterized in that, Further includes: Monitor the message time span in the current message area; as well as If the message time span in the current message area exceeds the second threshold, a new message area is created. and / or Monitor the number of consecutive messages in the current message area; as well as If the number of consecutive messages in the current message area exceeds the third threshold, a new message area is created. and / or Monitor the length of the current message area; and In response to the current message area exceeding the screen width, a new message area is created.

14. The method according to claim 2, characterized in that, The content of the emoticon message is an animated emoticon or an emoji, and the process of generating a 3D animation file based on the emoticon message and the 3D character model includes: Based on the expression message, obtain the corresponding expression data, wherein the expression data includes at least one or more time series of facial part data; At least obtain facial data of the user's 3D character; The facial feature data is mapped to the facial feature data of the user's 3D avatar based on a time series, to obtain a facial image sequence; and The 3D animation file is generated based on a preset frame rate and the facial image sequence.

15. A chat message processing device, characterized in that, The terminal equipment used by the first user includes: The communication group creation module is configured to create an instant communication group and corresponding chat window based on a second user or a second user who communicates instantly with the first user; The message area creation module is configured to create a message area for the user in the chat window when a user message is received, and generate an avatar display position in the message area. The message area includes one avatar display position, and the user is a first user or a second user. The avatar processing module is configured to obtain the 3D character of the user who sent the message, and load the 3D character of the user into the avatar display position as the chat avatar. The chat avatar includes a display window and a half-body image screenshot of the 3D character that protrudes from the boundary of the display window. The chat message module is configured to display messages sent by the user in the user's current message area, and when it detects an emoticon message sent by the first user in the instant messaging group during the chat, it sends the emoticon message to the avatar processing module. The avatar processing module is further configured to, in response to receiving an emoticon message sent by a first user, integrate the emoticon message into the 3D character of the chat avatar of the user who sent the emoticon message; wherein, the 3D character includes a 3D character model, and when integrating the emoticon message into the 3D character, a 3D animation file is generated based on the emoticon message and the 3D character model of the user who sent the emoticon message, and the 3D animation file is loaded and played into the avatar display position of the user who sent the emoticon message, so that the 3D character model of the user who sent the emoticon message presents facial expressions and / or actions according to the emoticon message; The avatar processing module is further configured to: create an association identifier for the 3D animation file and establish an association relationship between the association identifier and the 3D animation file when generating the 3D animation file; monitor the user's operation on the association identifier; and in response to the user's playback operation on the association identifier, load and play the 3D animation file to the avatar display position, and replace it with an image screenshot of the 3D character after the 3D animation file has finished playing.

16. An electronic device, characterized in that, It includes a processor and a memory storing computer program instructions, wherein the processor, when executing the computer program instructions, implements the chat message processing method as described in any one of claims 1-14.

17. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the chat message processing method as described in any one of claims 1-14.

18. A computer program product, characterized in that, It includes computer program instructions that, when executed by a processor, implement the chat message processing method as described in any one of claims 1-14.

Images