Information interaction method and device, electronic equipment and computer readable storage medium

Abstract

The invention provides an information interaction method and device, electronic equipment and a computer readable storage medium, and the method comprises the steps: carrying out the real-time collection of voice information in response to a first touch operation of a clock-in button, so as to generate voice collection data; performing semantic recognition on the voice acquisition data to determine a semantic recognition result of the voice acquisition data; determining an updating strategy for the first virtual character module according to the semantic recognition result; and updating the first virtual role module according to the determined updating strategy, so that an updating result can be displayed in graphical user interfaces of the first user and other users. Through the method, the accuracy, the convenience and the diversity of the card punching information are enhanced.

Description

Technical Field

[0001] This application relates to the field of interactive technology, and in particular to an information interaction method, apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] With technological advancements and societal progress, social media offers users increasingly diverse avenues for self-expression. Users can post their statuses through social media apps, allowing others to see and interact with them. Furthermore, users can engage with each other in a variety of ways within these apps.

[0003] Among them, posting users' status updates through check-ins is a major way for users to showcase their information on social media apps. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide an information interaction method, device, electronic device and computer-readable storage medium to enhance the accuracy, convenience and diversity of check-in information.

[0005] In a first aspect, embodiments of this application provide an information interaction method, which displays at least a portion of a virtual interaction scene in a graphical user interface. The virtual interaction scene includes a first virtual character module controlled by a first user and a second virtual character module controlled by other users. The graphical user interface also displays a check-in button, which floats on the virtual interaction scene. The information interaction method includes:

[0006] In response to the first touch operation of the check-in button, voice information is collected in real time to generate voice collection data;

[0007] Perform semantic recognition on the voice acquisition data to determine the semantic recognition result of the voice acquisition data;

[0008] Based on the semantic recognition results, an update strategy for the first virtual character module is determined;

[0009] The first virtual character module is updated according to the determined update strategy so that the update result can be displayed in the graphical user interface of the first user and other users.

[0010] In conjunction with the first aspect, this application provides a first possible implementation of the first aspect, wherein the step of responding to a first touch operation on the check-in button and collecting voice information in real time to generate voice collection data includes:

[0011] In response to the press of the check-in button, voice information collection begins;

[0012] In response to the release operation after the check-in button is pressed, the collection of voice information ends, and the voice information acquired after the start and before the end is used as voice collection data.

[0013] In conjunction with the first aspect, this application provides a second possible implementation of the first aspect, wherein the step of responding to a first touch operation on the check-in button and collecting voice information in real time to generate voice collection data includes:

[0014] Upon first touch of the check-in button, it begins real-time collection of voice information and timing.

[0015] When the timer reaches the predetermined duration, the collection of voice information ends, and the voice information acquired after the start and before the end is used as voice collection data.

[0016] In conjunction with the first aspect, this application provides a third possible implementation of the first aspect, wherein the method further includes:

[0017] In response to a second touch operation on the check-in button, a virtual character input control is displayed in the graphical user interface;

[0018] The semantic recognition result is generated in response to a touch operation on the virtual character input control.

[0019] In conjunction with the third possible implementation of the first aspect, this application provides a fourth possible implementation of the first aspect, wherein the step of performing semantic recognition on the voice acquisition data to determine the semantic recognition result of the voice acquisition data includes:

[0020] The voice data collected is subjected to initial semantic recognition using the current user terminal.

[0021] If the first semantic recognition is successful, then the semantic recognition result of the voice collection data is generated based on the result of the first semantic recognition.

[0022] If the first semantic recognition fails, the voice data is sent to the cloud server for a second semantic recognition.

[0023] If the second semantic recognition is successful, then the semantic recognition result of the voice acquisition data is generated based on the result of the second semantic recognition.

[0024] In conjunction with the first aspect, this application provides a fifth possible implementation of the first aspect, wherein the update strategy for the first virtual character module includes at least one of the following:

[0025] The animation of the first virtual character model, the facial expressions of the first virtual character model, the clothing of the first virtual character model, the accessories corresponding to the first virtual character model, the semantic recognition results corresponding to the first virtual character model, the state information of the first virtual character model, and the voice information corresponding to the voice acquisition data.

[0026] In conjunction with the first or second possible implementation of the first aspect, this application provides a sixth possible implementation of the first aspect, which further includes:

[0027] After the real-time acquisition of voice information begins, the acquired voice information is subjected to speech recognition in real time, and the results of the speech recognition are updated and displayed in the graphical user interface.

[0028] In conjunction with the first aspect, this application provides a seventh possible implementation of the first aspect, wherein, before updating the first virtual character module according to the determined update strategy, the method further includes:

[0029] Display the update strategy to be updated in the graphical user interface;

[0030] In response to a user's touch operation regarding the update strategy to be updated, the update strategy to be updated is adjusted.

[0031] Secondly, embodiments of this application also provide an information interaction device that displays at least a portion of a virtual interaction scene in a graphical user interface. The virtual interaction scene includes a first virtual character module controlled by a first user and a second virtual character module controlled by other users. The graphical user interface also displays a check-in button, which floats on the virtual interaction scene. The information interaction device includes:

[0032] The data acquisition module is used to respond to the first touch operation of the check-in button and collect voice information in real time to generate voice data.

[0033] The first recognition module is used to perform semantic recognition on the voice acquisition data in order to determine the semantic recognition result of the voice acquisition data;

[0034] The determining module is used to determine an update strategy for the first virtual character module based on the semantic recognition result;

[0035] An update module is used to update the first virtual character module according to the determined update strategy, so that the update result can be displayed in the graphical user interface of the first user and other users.

[0036] In conjunction with the second aspect, this application provides a first possible implementation of the second aspect, wherein, when the acquisition module is used to acquire voice information in real time in response to a first touch operation on the check-in button to generate voice acquisition data, it is specifically used for:

[0037] In response to the press of the check-in button, voice information collection begins;

[0038] In response to the release operation after the check-in button is pressed, the collection of voice information ends, and the voice information acquired after the start and before the end is used as voice collection data.

[0039] In conjunction with the second aspect, this application provides a second possible implementation of the second aspect, wherein, when the acquisition module is used to acquire voice information in real time in response to a first touch operation on the check-in button to generate voice acquisition data, it is specifically used for:

[0040] Upon first touch of the check-in button, it begins real-time collection of voice information and timing.

[0041] When the timer reaches the predetermined duration, the collection of voice information ends, and the voice information acquired after the start and before the end is used as voice collection data.

[0042] In conjunction with the second aspect, this application provides a third possible implementation of the second aspect, wherein the apparatus further includes:

[0043] The first display module is used to respond to the second touch operation of the check-in button and display virtual character input controls in the graphical user interface;

[0044] The generation module is used to generate the semantic recognition result in response to touch operations on the virtual character input control.

[0045] In conjunction with the third possible implementation of the second aspect, this application provides a fourth possible implementation of the second aspect, wherein the first recognition module, when performing semantic recognition on the voice acquisition data to determine the semantic recognition result of the voice acquisition data, is specifically used for:

[0046] The voice data collected is subjected to initial semantic recognition using the current user terminal.

[0047] If the first semantic recognition is successful, then the semantic recognition result of the voice collection data is generated based on the result of the first semantic recognition.

[0048] If the first semantic recognition fails, the voice data is sent to the cloud server for a second semantic recognition.

[0049] If the second semantic recognition is successful, then the semantic recognition result of the voice acquisition data is generated based on the result of the second semantic recognition.

[0050] In conjunction with the second aspect, this application provides a fifth possible implementation of the second aspect, wherein the update strategy of the first virtual character module includes at least one of the following:

[0051] The animation of the first virtual character model, the facial expressions of the first virtual character model, the clothing of the first virtual character model, the accessories corresponding to the first virtual character model, the semantic recognition results corresponding to the first virtual character model, the state information of the first virtual character model, and the voice information corresponding to the voice acquisition data.

[0052] In conjunction with the first or second possible implementation of the second aspect, this application provides a sixth possible implementation of the second aspect, which further includes:

[0053] The second recognition module is used to perform speech recognition on the collected speech information in real time after the speech information collection begins, and to update and display the speech recognition result in the graphical user interface.

[0054] In conjunction with the second aspect, this application provides a seventh possible implementation of the second aspect, which further includes:

[0055] The second display module is used to display the update strategy to be updated in the graphical user interface;

[0056] The adjustment module is used to respond to the user's touch operation on the update strategy to be updated and to adjust the update strategy to be updated.

[0057] Thirdly, embodiments of this application also provide an electronic device, including: a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the memory via the bus, and when the machine-readable instructions are executed by the processor, the steps in any of the possible implementations of the first aspect described above are performed.

[0058] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps in any of the possible implementations of the first aspect described above.

[0059] This application provides an information interaction method, device, electronic device, and computer-readable storage medium. In this method, when a user checks in, they can complete the check-in by inputting voice information. This allows the update strategy of the first virtual character module to be generated based on the semantic recognition results of the user's input voice. The information published during the check-in is not limited to text information but can also include other relevant information about the virtual character, thereby enhancing the accuracy, convenience, and diversity of the check-in information.

[0060] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0061] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0062] Figure 1 A schematic diagram of the graphical user interface of the mobile terminal provided in the embodiments of this application is shown;

[0063] Figure 2 This illustration shows the content displayed in the graphical user interface when switching to an AR scene, as provided in an embodiment of this application.

[0064] Figure 3 A flowchart of an information interaction method provided in an embodiment of this application is shown;

[0065] Figure 4 A schematic diagram of the background portion in the graphical user interface provided in the embodiments of this application is shown;

[0066] Figure 5 This illustration shows a schematic diagram of displaying semantic recognition results in an AR interface, as provided in an embodiment of this application.

[0067] Figure 6 A schematic diagram of the complete check-in process provided in the embodiments of this application is shown;

[0068] Figure 7 This illustration shows the group state after check-in is completed, as provided in an embodiment of this application.

[0069] Figure 8 This paper shows a schematic diagram of the structure of an information interaction device provided in an embodiment of this application;

[0070] Figure 9A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0071] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0072] With the development of technology, some social media software has developed more and more products in terms of "helping users express and present themselves", including text, images, videos and even more immersive VR / AR interaction methods. Each new expression tool has its unique advantages and corresponding disadvantages.

[0073] In related technologies, users primarily post information about themselves in text form to check in. More advanced technologies allow users to add images, GIFs, or videos. The text portion of the check-in is mainly manually entered by the user, while images, GIFs, and videos are uploaded from their local devices. After uploading, posting the information completes the check-in.

[0074] However, applicants found this check-in method to be neither entirely accurate nor convenient. Therefore, the application itself provides an information exchange method, such as... Figure 1 The diagram shows the graphical user interface of the mobile terminal provided in this application. The graphical user interface displays at least a portion of the virtual interactive scene. The virtual interactive scene includes a first virtual character module controlled by a first user and a second virtual character module controlled by other users. The graphical user interface also displays a check-in button, which is floating on the virtual interactive scene.

[0075] The first virtual character module can be a virtual character module named "Youzi" as shown in the figure, while the other virtual character modules in the figure are second virtual character modules controlled by other users. Generally, the position of each virtual character module in this virtual interactive scene can be determined based on the real-time movement of the virtual character by the user in the virtual interactive scene, or it can be determined based on the user's position in the real scene (that is, the user's position in the Earth coordinate system).

[0076] When the virtual character's position in the virtual interactive scene is determined based on the user's position in the real scene, the user needs to activate the mobile terminal's (the mobile terminal logged into with the virtual character) location function. The mobile terminal obtains its current location in real time and uploads it to the server. The server can then update the position of the user-controlled virtual character in the virtual scene based on each user's actual location information in the Earth coordinate system. In other words, the position of the user-controlled virtual character in the virtual scene is a mapping of the user (mobile terminal) in the Earth coordinate system.

[0077] Users can adjust the area of ​​the virtual interactive scene displayed in the graphical user interface (GUI) by swiping their fingers. The distance and speed of the swipe affect the content of the virtual interactive scene displayed in the GUI. For example, if a user swipes their finger to the right, the left-hand side of the virtual interactive scene (currently...) will be displayed in the GUI. Figure 1 All virtual character modules will move to the right.

[0078] In the graphical user interface, the three buttons on the lower right are the AR toggle button, refresh button, and repositioning button. The AR toggle button is used to switch between two virtual interactive scenes (a purely virtual scene created by the server and an augmented reality (AR) scene). Figure 2 The image shows the content displayed in the graphical user interface when switching to the AR scene. In this AR scene interface, the circular button at the bottom is the check-in button, the camera button to the right of the check-in button is used to trigger the photo-taking function, and the back arrow button to the left of the check-in button is used to return to the pure virtual scene. Users can click the camera button to publish the real-time photo as check-in information.

[0079] The check-in button in the graphical user interface is located at Figure 1 The button in the lower middle section. The circular button in the center, displaying a virtual character, is the main button for checking in. Users can trigger the check-in button by clicking, double-clicking, multiple clicks, or long-pressing. The two circular buttons to the left and right of the check-in button are other function buttons; for example, the left button is for querying information, and the right button is for posting text messages.

[0080] like Figure 3 As shown, this information interaction method includes the following steps:

[0081] S101 responds to the first touch operation of the check-in button and collects voice information in real time to generate voice collection data;

[0082] S102, perform semantic recognition on the voice acquisition data to determine the semantic recognition result of the voice acquisition data;

[0083] S103, determine the update strategy for the first virtual character module based on the semantic recognition result;

[0084] S104, the first virtual character module is updated according to the determined update strategy so that the update result can be displayed in the graphical user interface of the first user and other users.

[0085] In step S101, the first touch operation has two usage methods: long press for voice input and short press for voice input.

[0086] When using long press for voice input, step S101 can be implemented as follows:

[0087] In response to the press of the check-in button, voice information collection begins;

[0088] In response to the release operation after the check-in button is pressed, the collection of voice information ends, and the voice information acquired after the start and before the end is used as voice collection data.

[0089] A single press and an adjacent release complete a full long press operation. Voice recording begins after the user presses the button (using an audio capture device on the mobile device, such as a phone's microphone) and stops after the user releases the button. The voice information collected during the long press of the check-in button is used as the voice data.

[0090] When using short presses for voice input, step S101 can be implemented as follows:

[0091] Upon first touch of the check-in button, it begins real-time collection of voice information and timing.

[0092] When the timer reaches the predetermined duration, the collection of voice information ends, and the voice information acquired after the start and before the end is used as voice collection data.

[0093] In this scenario, the user only needs to press the check-in button once to complete the voice information collection. Specifically, the timer can start when the check-in button is clicked and stop after a predetermined duration. The voice information collected during the time interval between the start and end of the timer is used as the voice data. For example, clicking the check-in button could start the timer for 10 seconds, 20 seconds, or one minute, then stop the timer and terminate the voice information collection.

[0094] The timing method can also be as follows: Timing begins when the check-in button is clicked, and the duration of no valid voice information received is calculated in real time (if the collected voice information does not contain the user's voice, it is considered that no valid voice information has been received). Once the predetermined time for no valid voice information has been received, timing stops, and voice information collection ceases. Specifically, if the user stops speaking 5 seconds after pressing the check-in button, and the system detects 2 seconds of no valid voice information received after 7 seconds, then timing stops, and voice information collection ceases (2 seconds is the predetermined duration of no valid voice information received).

[0095] In addition to voice input, you can also trigger a check-in button to input text (usually a single click to input text).

[0096] When using triggers for text input, the above method may also include the following steps:

[0097] In response to a second touch operation on the check-in button, a virtual character input control is displayed in the graphical user interface;

[0098] The semantic recognition result is generated in response to a touch operation on the virtual character input control.

[0099] The second touch operation is usually a single click, but it can also be a double click, multiple clicks, or other touch methods to trigger text input (such as swiping or triggering a specified gesture graphic).

[0100] After the user issues a second touch operation, a virtual character input control (soft keyboard) will be displayed in the graphical user interface. The user can then input text information using this virtual character input control. The input text information can then be directly used as the semantic recognition result for further message publishing in subsequent steps.

[0101] In step S102, the recognition performed is semantic recognition, not speech recognition, which is one of the main features of this solution. Speech recognition can only convert the user's input speech content into text, but semantic recognition can identify the emotional information of the user's input speech, such as whether the user is in a depressed state, a happy state, etc. This state information cannot be displayed by speech recognition alone. When performing semantic recognition, it is preferable to use a local model. However, since the size of the semantic recognition model used in the mobile terminal is limited (mainly due to insufficient computing power of the mobile terminal), the accuracy and precision of recognition are limited. Therefore, a dual recognition method can be adopted, that is, first perform recognition using a small semantic recognition model in the mobile terminal. If the recognition result is not ideal, then perform recognition using a large semantic recognition model in the server, thereby ensuring the efficiency and accuracy of recognition.

[0102] That is, step S102 can be implemented in the following way:

[0103] Step 1021: Perform the first semantic recognition on the voice collection data through the current user terminal;

[0104] Step 1022: If the first semantic recognition is successful, then generate the semantic recognition result of the voice acquisition data based on the result of the first semantic recognition.

[0105] Step 1023: If the first semantic recognition fails, the voice collection data is sent to the cloud server for a second semantic recognition.

[0106] Step 1024: If the second semantic recognition is successful, then generate the semantic recognition result of the voice acquisition data based on the result of the second semantic recognition.

[0107] In step 1021, the first semantic recognition is performed using a mobile terminal. The model used for the first semantic recognition is a small model stored in the mobile terminal.

[0108] If the result of the first semantic recognition is unsatisfactory, in step 1023, the voice data can be sent to the server for recognition by a large semantic recognition model. If the result of the first semantic recognition is satisfactory, the result of that semantic recognition can be directly output. Conversely, if the result of the second semantic recognition is successful, the result of the second semantic recognition can be directly used as the semantic recognition result of the voice data.

[0109] It should be noted that both the model used for the first semantic recognition (first semantic recognition model) and the model used for the second semantic recognition (second semantic recognition model) are trained before use. During use, you only need to input the voice collection data into the first semantic recognition model or the second semantic recognition model, and the corresponding semantic recognition model can directly output the semantic recognition result.

[0110] The computing power required to run the first semantic recognition model is less than that required to run the second semantic recognition model, or in other words, the space occupied by the first semantic recognition model is less than that occupied by the second semantic recognition model.

[0111] If the second semantic recognition fails, a prompt message indicating the failure can be directly sent to the user. This prompt message can be displayed as text on the graphical user interface, or it can be sent to the user via voice or other means.

[0112] Generally, whether the first or second semantic recognition fails, the possible reasons may be the same or different, but they can be broadly categorized into two types: the first is insufficient effective speech information, and the second is the inability to analyze the semantic result. In the first case, the most common cause is that the user's voice is too soft or the environment is noisy. In such situations, the model cannot effectively extract speech information, leading to recognition failure. In the second case, the most common cause is that the speech duration is particularly short, or the speech content contains illogical parts.

[0113] Therefore, in response to the above two situations, the solution provided in this application can perform a data validity check before the first speech recognition or before sending the voice data to the cloud server. Validity check can include: signal-to-noise ratio (SNR) judgment, effective speech duration judgment, etc. If any of these judgments fail, a prompt message can be directly returned to the user indicating speech recognition failure and requesting re-entry of the voice. Generally, if the SNR is too low, it indicates excessive noise. In this case, even with noise reduction, the result is difficult to perform semantic recognition, so a prompt message can be directly returned to the user requesting re-entry of the voice. Similarly, if the effective speech duration is too short, a prompt message can also be directly returned to the user requesting re-entry of the voice.

[0114] Furthermore, when generating semantic recognition results, besides relying solely on the user's input voice information and corresponding semantic recognition results, other user information can also be combined for semantic recognition. This information can include user reference information, such as the user's location, the images posted by the user, environmental information about the user's location, and the user's identity information. The reason for using these methods to assist in generating semantic recognition results is that this information can also carry information expressing the user's thoughts. For example, through the user's location information, we can understand the user's current location. The information expressed by users will inevitably differ in different locations, and there are patterns in the information expressed in specific types of locations. For instance, in tourist areas, the information expressed by users is more related to relaxation or promotional messages. In workplaces, the information expressed is more work-related. Similarly, the images posted by users (such as those posted for check-ins) can also carry similar directional information. If the check-in image is related to the workplace, the posted voice information will likely be work-related. Environmental information can include weather conditions, time information, user's physical condition, etc. This information can also reflect what the user is thinking, and therefore all contribute to the generation of semantic recognition results.

[0115] Similarly, the above-mentioned information can complement each other. For example, if the time information shows that it is late on a weekday, and the posted picture and / or location shows that the user is in the office, it can be determined that the user is working overtime. Therefore, the semantic recognition result can be generated in the direction of working overtime at the company at night.

[0116] In other words, in the solution provided in this application, the step of generating semantic recognition results of the voice collection data based on the results of the first semantic recognition can be performed as follows:

[0117] Based on the results of the initial semantic recognition and the user's reference information, a semantic recognition result for the voice collection data is generated. The user's reference information includes at least one of the following: the user's location information, the images posted by the user, and environmental information about the user's location.

[0118] Correspondingly, in the solution provided in this application, the step of generating semantic recognition results of the voice collection data based on the results of the first semantic recognition can be performed as follows:

[0119] Based on the results of the second semantic recognition and the user's reference information, a semantic recognition result for the voice collection data is generated. The user's reference information includes at least one of the following: the user's location information, the images posted by the user, and environmental information of the user's location.

[0120] In addition to providing the above functions, the system records the user's location information after each check-in. Therefore, the system can create a footprint map to record where the user has been, and the user can also actively view the footprint map.

[0121] In order to fully express the semantic recognition results, the solution provided in this application sets up a variety of different expression methods for the first virtual character module, that is, the update strategy of the first virtual character module, including at least one of the following:

[0122] The animation of the first virtual character model, the expression of the first virtual character model, the clothing of the first virtual character model, the accessories corresponding to the first virtual character model, the semantic recognition result corresponding to the first virtual character model, and the state information of the first virtual character model.

[0123] Among them, the animation action of the first virtual character model refers to the physical actions that the first virtual character can perform, such as raising hands, yawning, running, etc. These actions also correspond to the semantic recognition results. For example, if the voice collection data shows that the user is expressing content related to exercise, and the semantic recognition result shows that the user is breathing heavily, it can be determined that the user is engaged in vigorous exercise. Then the animation action of the first virtual character model can be running or certain specified sports (the specific sports can be determined based on the semantic recognition results).

[0124] The facial expressions of the first virtual character model are primarily used to reflect the emotional information in the semantic recognition results, which is also the main difference between semantic recognition and speech recognition. Through semantic recognition, the user's current emotional state can be identified. For example, if a user says "I'm happy" but their tone is melancholic, a fake smile can be used as the facial expression for the first virtual character model.

[0125] The clothing of the first virtual character model mainly refers to the clothes that affect the appearance of the virtual character. The accessories corresponding to the first virtual character model refer to items worn on the body and pets walked by, etc., which are non-character objects that can display the user's status, apart from clothing.

[0126] Clothing can also be generated based on semantic recognition results. Besides semantic recognition, clothing can also be generated based on the user's reference information mentioned earlier. For example, if the user's location information shows they are on a beach, the virtual character can wear beachwear; if the location information shows they are near their workplace, the virtual character can wear a suit. If the environment information shows the user is in the rain or outdoors in the rain, an umbrella can be used as an accessory. If the user's posted picture shows them at a barbecue stall, the virtual character can be equipped with barbecue skewers and casual wear. If the user's location shows they are in their own neighborhood after get off work, and the semantic recognition results detect the sound of a dog barking, the user can be equipped with loungewear and a dog to indicate they are walking their dog.

[0127] The semantic recognition results for the first virtual character model are primarily displayed in character form. These characters are divided into two types: text and symbols. Text is simply the textual content of the semantic recognition result, while symbols can display the emotional state of the user's speech. For example, if the user is excited, more exclamation marks can be used; if the user is questioning, question marks can be used. Or, for instance, if the user's identity is that of a rock music enthusiast, special rock-themed characters can be used to display the semantic recognition results. Figure 2 In this example, the words "mushroom" above the virtual character's head are a display of the semantic recognition results. If the time indicates Halloween, or if the user's current location is related to Halloween, a Halloween pumpkin head can be displayed on the image. For example... Figure 5 As shown, this is how semantic recognition results are displayed in the AR interface.

[0128] The first virtual character model's state information consists of the state information posted by the user. In this state information, the user can display their current state, such as... Figure 1 In the image, the "love" symbol above the pomelo's head indicates its status. Among the three people in the upper left corner of the pomelo, "drinking tea" indicates their status.

[0129] The voice information corresponding to the voice acquisition data is actually the voice emitted by the user. This voice can be the original recording, the original voice after noise reduction, or a virtual voice (such as a cartoon voice) that plays the text content after semantic / speech recognition of the user's voice through specific virtual sound effects. Alternatively, based on the user's voice, the timbre information can be determined and the corresponding virtual voice can be played using the timbre information.

[0130] It should be noted that the above update strategy is generally automatically output by the trained model. In other words, the model can automatically output the update strategy based on the semantic recognition results.

[0131] After the update strategy is determined, step S104 can be executed to update the first virtual character module according to the determined update strategy, so that the update result can be displayed in the graphical user interface of the first user and other users.

[0132] After the update is completed, the update result is directly displayed in the first user's graphical user interface (the refresh is completed automatically). Other users need to refresh the interface by timed refresh, manual refresh, or other refresh methods before they can see the updated result of the first user in the first virtual character module.

[0133] Generally, users update their information to display their own information. Therefore, after the first user completes the check-in (executes step S104), the surrounding virtual characters can be gathered together so that the user can see the status of other users (see the status of the virtual characters controlled by other users in the graphical user interface), thereby enhancing interactivity.

[0134] Furthermore, to improve user interaction efficiency, after performing step S104, the solution of this application may also include the following steps:

[0135] Based on the content of the update strategy, a second virtual character module that is similar to the updated first virtual character module is queried as the target virtual character module.

[0136] The target virtual character module is highlighted and restricted to the current graphical user interface.

[0137] The module for finding other similar second virtual characters aims to let the first user know who else is similar to them (similar users are more likely to interact). For example, users who are both working overtime or eating are more likely to communicate. In fact, the update strategy and the semantic recognition result have a certain mapping relationship. Therefore, using either the semantic recognition result or the update strategy for querying in this step is equivalent.

[0138] Specifically, to determine which second virtual character modules are similar to the first virtual character module, a direct comparison can be made based on various information in the update strategy or semantic recognition results.

[0139] There are two ways to highlight a virtual character. One is to prominently display the target virtual character module without hiding the non-target virtual character modules (e.g., through highlighting or enlarging). The second is to hide the secondary virtual character modules of the non-target virtual character modules, so that the user can only see the target virtual character modules that are similar to themselves. When highlighting, a clustered display method can be used to improve the user's viewing efficiency. Specifically, this can be achieved by zooming in or out of the virtual interactive scene space displayed in the graphical user interface to ensure that the number of target virtual character modules displayed in the graphical user interface reaches a predetermined number.

[0140] To ensure that users clearly understand what they are saying, the solution provided in this application also includes the following:

[0141] After the real-time acquisition of voice information begins, the acquired voice information is subjected to speech recognition in real time, and the results of the speech recognition are updated and displayed in the graphical user interface.

[0142] like Figure 4 As shown, the background of this graphical user interface displays the text currently being entered by the user. By showing this text, the user can see what they have said and what they haven't finished saying, making it easier for them to decide what to say next. The "37°46'N 122°25W" above the text in the graphical user interface represents the user's current latitude and longitude.

[0143] like Figure 6 As shown, the complete process of clocking in is illustrated. Figure 6 The first image shows the user's state in the graphical user interface before triggering the check-in button. In this image, the first virtual character module controlled by the first user is named "Youzi," and the other virtual character modules surrounding Youzi are the second virtual character modules. The circular button with a cartoon character at the bottom of the image is the check-in button.

[0144] After the user clicks the check-in button, they are redirected to... Figure 6In the second image, the user hasn't yet pressed and held the speak button (the circular button with a microphone icon at the bottom of the image) to start speaking, so no text is displayed on the background. After the user starts speaking, the screen jumps to the third image, where the background displays the text corresponding to the user's current words. The fourth image shows the user having finished speaking; the user clicked the cancel button to the left of the speak button but hasn't lifted their hand. The fifth image shows the user manually correcting their previous statement or re-entering the text to check in. The seventh image shows the check-in information before publication, allowing the user to confirm beforehand. During this process, any adjustments to the update strategy can be made. After confirmation, the screen moves to the eighth image, where the first virtual character module is updated according to the update strategy. An animation effect is then displayed; since the image is static, only a single frame of the animation is shown. In this animation, the first virtual character module is singing and dancing, while the surrounding target virtual character modules are highlighted (the second virtual character module, which is not a target virtual character module, is hidden). The image in Chapter 9 shows the state after the animation has finished playing, and the overall state has returned to that of the first image (only the content updated according to the update strategy has been retained).

[0145] like Figure 7 As shown, the group's status after completing the check-in is displayed. Figure 7 The first image in the document is an example of an image used when checking in (the interface before step S101). The second and third images show the changes in the interface before and after step S104. As you can see, in the third image, the second virtual character module displayed in the graphical user interface has changed significantly, meaning that it has been filtered. Also, in the third image, the displayed area has been enlarged.

[0146] Overall, the solution provided in this application has the following characteristics:

[0147] In terms of expression, this solution retains the advantages of traditional Chinese and image-based expression tools, while increasing the weight of expression in the voice dimension and adding sufficient fun. It allows users to have the basic advantages of text and images when expressing themselves, as well as a more novel experience and expression, quickly "speaking out" and "talking" anytime, expressing themselves in a more natural and direct way.

[0148] In terms of the expression of results, this solution breaks through the WYSIWYG user experience. Compared to the filters, background music, or special effects commonly used in traditional expression tools, this solution uses a more vivid approach—the rich emotions and movements of the puppet. Depending on the content being expressed, the user's puppet will react differently, including changes in emotion and movement, more realistically and directly mapping the user's expression onto the puppet, making the puppet express itself more vividly and realistically. At the same time, the addition of the user's input voice ensures that the puppet doesn't detach itself from reality. The vivid performance after intelligent analysis, combined with the user's authentic and direct voice, creates the unique result of this expression tool.

[0149] Additional information on unique features:

[0150] Unique Action Performance: In this solution, everything from subtle emotions like facial expressions and lip movements to auxiliary expressions like actions and props is presented in a unique way. It doesn't simply replicate the emotions or actions users would make in real life, but rather processes them, presenting them in a more exaggerated, surreal, humorous, or whimsical manner. For example, the voice input "I give up" is a common expression among many users. In this solution, a similar presentation is "I float," where the entire puppet floats up, along with surrounding objects, using a homophone to express the feeling of "I give up," and visually conveying a sense of "uncontrollable helplessness."

[0151] Unique Understanding and Analysis: In this solution, users don't need to manually select corresponding actions; they simply need to speak or type. Based on the user's expression, the solution will select emotions and actions that match the user's mood. This precise understanding of the user's intended message is a unique feature of this solution. It comprehensively judges the user's expression based on tone, context, environment, time, and even location, giving the user a clear and authentic experience.

[0152] The solution provided in this application also has the following value:

[0153] 1. Tool value: A simple, direct, versatile, accurate, vivid, and effective expression tool.

[0154] Simple and direct: Users only need to say a sentence or type a paragraph to complete their expression, and a fun Bondee performance can be generated quickly.

[0155] Rich in features: In each expression, users can adjust their voice effects, change their puppet's movements, modify the description of images and text, and even customize the expression of props at any time.

[0156] Accurate and vivid: Understand the user's tone and context, combine the user's time / location, fully analyze the user's expression, and select the emotions and actions that best match the user's expression.

[0157] Rich in effects: It has the basic effects of traditional graphic expression tools, as well as 3D, dynamic puppet performances, and the support of sound performance, forming a rich visual and auditory expression effect.

[0158] 2. Emotional value: Discover other users similar to the user.

[0159] Understanding You: Every "Boop!" expression from a user is recorded in the app. Through continuous expression, the "Boop!" tool learns more and more about the user's language, expression style, and common tone, thus selecting the most suitable emotions and actions to express the user. It personalizes every user experience, understanding both E-people and i-people, and catering to various styles such as trendy, street, home, and Y2K!

[0160] Record Your Moments: Every "Boop!" is stored on a footprint map within the app. The location of each "Boop!" is recorded on this map, allowing users to easily review their thoughts at any given time and place. It also allows for management and adjustment, truly recording each user's moments at different times and locations.

[0161] Connect with more "you"s similar to you: In the final stage of "Boop!", if there are "Boop!"s around you with similar content, these people will also appear around you, like an echo "Echo". After each "Boop!", you connect with every echo similar to you and resonate with others.

[0162] 3. Social and community value: A genuine, fun, and warm community atmosphere.

[0163] Authenticity: When a user posts a "Boop!", the text, images, and voice are all unique and authentic. The location where a "Boop!" appears is also unique and authentic. Every authentic "Boop!" makes up an authentic Bondee.

[0164] Fun: The actions and emotions during each "Boop!" are ever-changing, and the unique outfits of each person make Bondee's "Boop!" content fun and unique.

[0165] Warm Resonance: Each "Boop!" is accompanied by a unique Echo, and the "Boop!" dolls in the same area will gather together, creating a lively and warm atmosphere.

[0166] Based on the same technical concept, embodiments of this application also provide an information interaction device that displays at least a portion of a virtual interaction scene in a graphical user interface. The virtual interaction scene includes a first virtual character module controlled by a first user and a second virtual character module controlled by other users. A check-in button is also displayed in the graphical user interface, and the check-in button floats on the virtual interaction scene. Figure 8 As shown, the device includes:

[0167] The data acquisition module 801 is used to respond to the first touch operation of the check-in button and collect voice information in real time to generate voice acquisition data.

[0168] The first recognition module 802 is used to perform semantic recognition on the voice acquisition data in order to determine the semantic recognition result of the voice acquisition data.

[0169] The determining module 803 is used to determine an update strategy for the first virtual character module based on the semantic recognition result;

[0170] The update module 804 is used to update the first virtual character module according to the determined update strategy, so that the update result can be displayed in the graphical user interface of the first user and other users.

[0171] Optionally, when the acquisition module 801 is used to acquire voice information in real time in response to the first touch operation of the check-in button to generate voice acquisition data, it is specifically used for:

[0172] In response to the press of the check-in button, voice information collection begins;

[0173] In response to the release operation after the check-in button is pressed, the collection of voice information ends, and the voice information acquired after the start and before the end is used as voice collection data.

[0174] Optionally, when the acquisition module 801 is used to acquire voice information in real time in response to the first touch operation of the check-in button to generate voice acquisition data, it is specifically used for:

[0175] Upon first touch of the check-in button, it begins real-time collection of voice information and timing.

[0176] When the timer reaches the predetermined duration, the collection of voice information ends, and the voice information acquired after the start and before the end is used as voice collection data.

[0177] Optionally, the device further includes:

[0178] The first display module is used to respond to the second touch operation of the check-in button and display virtual character input controls in the graphical user interface;

[0179] The generation module is used to generate the semantic recognition result in response to touch operations on the virtual character input control.

[0180] Optionally, when the first recognition module performs semantic recognition on the voice acquisition data to determine the semantic recognition result of the voice acquisition data, it is specifically used for:

[0181] The voice data collected is subjected to initial semantic recognition using the current user terminal.

[0182] If the first semantic recognition is successful, then the semantic recognition result of the voice collection data is generated based on the result of the first semantic recognition.

[0183] If the first semantic recognition fails, the voice data is sent to the cloud server for a second semantic recognition.

[0184] If the second semantic recognition is successful, then the semantic recognition result of the voice acquisition data is generated based on the result of the second semantic recognition.

[0185] Optionally, the update strategy for the first virtual character module includes at least one of the following:

[0186] The animation of the first virtual character model, the facial expressions of the first virtual character model, the clothing of the first virtual character model, the accessories corresponding to the first virtual character model, the semantic recognition results corresponding to the first virtual character model, the state information of the first virtual character model, and the voice information corresponding to the voice acquisition data.

[0187] Optional, also includes:

[0188] The second recognition module is used to perform speech recognition on the collected speech information in real time after the speech information collection begins, and to update and display the speech recognition result in the graphical user interface.

[0189] Optional, also includes:

[0190] The second display module is used to display the update strategy to be updated in the graphical user interface;

[0191] The adjustment module is used to respond to the user's touch operation on the update strategy to be updated and to adjust the update strategy to be updated.

[0192] Figure 9A schematic diagram of an electronic device provided in this application embodiment includes: a processor 901, a memory 902, and a bus 903. The memory 902 stores machine-readable instructions executable by the processor 901. When the electronic device runs the above-described information processing method, the processor 901 communicates with the memory 902 through the bus 903. The processor 901 executes the machine-readable instructions to perform the steps of the method described in Embodiment 1.

[0193] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps described in Embodiment 1.

[0194] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, electronic devices, and computer-readable storage media described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0195] In the several embodiments provided in this application, it should be understood that the disclosed methods, apparatuses, electronic devices, and computer-readable storage media can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or modules may be electrical, mechanical, or other forms.

[0196] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0197] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0198] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0199] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The scope of protection of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of protection of the claims.

Claims

1. An information interaction method, characterized in that, At least part of a virtual interactive scene is displayed in a graphical user interface, the virtual interactive scene having a first virtual character module controlled by a first user and a second virtual character module controlled by other users; A check-in button is also displayed in the graphical user interface, the check-in button being displayed floating on the virtual interactive scene; the information interaction method comprises: In response to a first touch operation on the check-in button, real-time voice information is collected to generate voice collection data; Semantic recognition is performed on the voice collection data to determine a semantic recognition result of the voice collection data; An update strategy for the first virtual character module is determined according to the semantic recognition result; The first virtual character module is updated according to the determined update strategy, so that the update result can be displayed in the graphical user interfaces of the first user and the other users.

2. The method of claim 1, wherein, The step of collecting real-time voice information to generate voice collection data in response to a first touch operation on the check-in button comprises: In response to a press operation on the check-in button, the collection of voice information is started; In response to a lift operation after the press operation on the check-in button, the collection of voice information is ended, and the voice information collected after the start and before the end is taken as the voice collection data.

3. The method of claim 1, wherein, The step of collecting real-time voice information to generate voice collection data in response to a first touch operation on the check-in button comprises: In response to a first touch operation on the check-in button, real-time voice information is collected, and timing is started; In response to the timing reaching a preset time length, the collection of voice information is ended, and the voice information collected after the start and before the end is taken as the voice collection data.

4. The method of claim 1, wherein, The method further comprises: In response to a second touch operation on the check-in button, a virtual character input control is displayed in the graphical user interface; In response to a touch operation on the virtual character input control, the semantic recognition result is generated.

5. The method of claim 4, wherein, The step of performing semantic recognition on the voice collection data to determine a semantic recognition result of the voice collection data comprises: The voice collection data is subjected to first semantic recognition by a current user terminal; If the first semantic recognition is successful, a semantic recognition result of the voice collection data is generated according to the result of the first semantic recognition; If the first semantic recognition fails, the voice collection data is sent to a cloud server for second semantic recognition; If the second semantic recognition is successful, a semantic recognition result of the voice collection data is generated according to the result of the second semantic recognition.

6. The method of claim 1, wherein, The update strategy of the first virtual character module comprises at least one of the following: Action animation of the first virtual character model, expression of the first virtual character model, dressing of the first virtual character model, accessories corresponding to the first virtual character model, semantic recognition result corresponding to the first virtual character model, state information of the first virtual character model, and voice information corresponding to the voice collection data.

7. The method of claim 2 or 3, wherein, Further comprising: After the real-time collection of voice information is started, real-time voice recognition is performed on the collected voice information, and the result of the voice recognition is updated and displayed in the graphical user interface.

8. The method of claim 1, wherein, Before the updating of the first virtual role module according to the determined updating strategy in the step, the method further comprises: displaying an updating strategy to be updated in the graphical user interface; adjusting the updating strategy to be updated in response to a touch operation of a user on the updating strategy to be updated.

9. An information interaction device, characterized in that, The virtual interactive scene has at least part of a graphical user interface, and the virtual interactive scene has a first virtual role module controlled by a first user and a second virtual role module controlled by other users. The graphical user interface further displays a punch-in button, and the punch-in button is displayed on the virtual interactive scene; and the information interaction device comprises: a collection module configured to collect voice information in real time to generate voice collection data in response to a first touch operation on the punch-in button; a first recognition module configured to perform semantic recognition on the voice collection data to determine a semantic recognition result of the voice collection data; a determination module configured to determine an updating strategy for the first virtual role module according to the semantic recognition result; an updating module configured to update the first virtual role module according to the determined updating strategy, so that an updating result can be displayed in the graphical user interfaces of the first user and the other users.

10. An electronic device, comprising: comprise: a processor, a memory, and a bus, the memory stores machine readable instructions executable by the processor, when the electronic device is running, the processor and the memory communicate through the bus, and the machine readable instructions are executed by the processor to perform the steps of the method according to any one of claims 1 to 8.

11. A computer readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to perform the steps of the method according to any one of claims 1 to 8.

Images