Interactive information processing method, device, terminal and storage medium
By obtaining imext and sound data from the server side in the online live broadcast platform, generating and sending interactive content, the problem of user update and expansion is solved, and rich interactive information and high user experience is achieved.
Patent Information
- Application Number
- CN202110181775.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-02-08
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2041-02-08
AI Technical Summary
In the online live broadcast platform, it is difficult for users to update and expand pictures and audio, which affects the user experience.
Emoji or emoji and sound data from the data server through the terminal, generate interactive content based on the interactive information processing mode, and send it to the live broadcast server to display it in other terminals. Emoji includes emojis and sound data includes MIDI Audio and wave table information are stored in XMF files to establish the association relationship between the epitome ID and the XMF file.
It realizes the update and expansion of graphic text and sound data, supports a large amount of audio concurrency, provides rich sound and light effects, does not affect the broadcast effect of the anchor, and improves the user experience.
Smart Images

Figure CN114915853B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of Internet technology, and in particular to an interactive information processing method, device, terminal and storage medium. Background Art
[0002] With the development of network technology, the application of online live broadcast platforms is becoming more and more extensive, and can be applied to business areas such as video conferencing, RCS, live broadcast, and online education. The online live broadcast platform can include anchors and users. An anchor can broadcast online in the live broadcast room, and users can enter the live broadcast room and watch the content broadcast by the anchor. During the user's viewing process, the user can like, follow, or give virtual gifts to the anchor, and can interact with information. The interactive information may include pictures, audio, etc. Since pictures and audio are usually preset by users locally, it is difficult for users to update and expand pictures and audio, which affects the user experience. Summary of the invention
[0003] In view of this, a technical problem to be solved by the present invention is to provide an interactive information processing method, device, terminal and storage medium.
[0004] According to a first aspect of the present disclosure, there is provided an interactive information processing method, comprising: during a live video broadcast, a terminal determines an interactive information processing mode; the terminal obtains emojis or emojis and corresponding sound data from a data server; wherein the data server pre-stores the emojis and the sound data; based on the interactive information processing mode, the terminal generates and displays interactive content using the emojis or the emojis and the corresponding sound data; the terminal sends the interactive content to a live broadcast server, so that the live broadcast server sends the interactive content to other terminals joining the live video broadcast for display.
[0005] Optionally, the emoji includes: emoticons; the sound data includes: MIDI audio, wavetable information; the sound data is stored in a corresponding XMF file; the data server pre-establishes an association relationship between the emoji ID and the XMF file, and stores the association relationship.
[0006] Optionally, the interactive information processing mode includes: adding an emoji display mode; the terminal obtains emojis or emojis and corresponding sound data from a data server, including: the terminal obtains an emoji ID, and obtains the emoji from the data server according to the emoji ID; the terminal generates interactive content based on the interactive information processing mode using the emoji or the emoji and corresponding sound data and displays it, including: the terminal generates the interactive content based on the adding emoji mode using the emoji and displays it in the display area.
[0007] Optionally, the interactive information processing mode includes: a background overlay emoji display mode; the terminal is a live broadcast host terminal; the terminal obtains emojis or emojis and corresponding sound data from a data server, including: the live broadcast host terminal obtains an emoji ID, and obtains the emoji and an associated XMF file from the data server according to the emoji ID; the terminal generates and displays interactive content using the emojis or the emojis and corresponding sound data based on the interactive information processing mode, including: the live broadcast host terminal generates the interactive content according to the emojis and the associated XMF file; the live broadcast host terminal determines a background layer and a foreground layer in the local video image; the live broadcast host terminal adds an interactive content layer between the background layer and the foreground layer based on the background overlay emoji display mode, and displays the interactive content in the interactive content layer.
[0008] Optionally, the interactive information processing mode includes: an emoji and audio synthesis mode; the terminal obtains emojis or emojis and corresponding sound data from a data server, including: the terminal obtains an emoji ID, and obtains the emoji and an associated XMF file from the data server according to the emoji ID; the terminal generates and displays interactive content based on the interactive information processing mode using the emojis or the emojis and corresponding sound data, including: the terminal generates and displays the interactive content based on the emoji and audio synthesis mode using the emojis and associated XMF files.
[0009] Optionally, the interactive information processing mode includes: an emoji sound effect processing mode; the terminal is a live broadcast client terminal, and the terminal obtains emojis or emojis and corresponding sound data from a data server, including: the live broadcast client terminal obtains an emoji ID, and obtains the emoji and an associated XMF file from the data server according to the emoji ID; the terminal generates and displays interactive content based on the interactive information processing mode using the emojis or the emojis and corresponding sound data, including: the live broadcast client terminal extracts wave table information from the XMF file, randomly assigns pitches based on the wave table information and performs chord processing to generate audio information; the live broadcast client terminal generates and displays the interactive content based on the emoji sound effect processing mode using the emojis and the audio information.
[0010] Optionally, the terminal generates and displays barrage content according to the interactive content; the terminal sends the barrage content to a live broadcast server, so that the live broadcast server sends the barrage content to the other terminals for display.
[0011] According to a second aspect of the present disclosure, there is provided an interactive information processing device, which is applied to a terminal, and includes: a mode determination module, which is used to determine an interactive information processing mode during a live video broadcast; a data acquisition module, which is used to obtain emojis or emojis and corresponding sound data from a data server; wherein the data server pre-stores the emojis and the sound data; a content processing module, which is used to generate and display interactive content based on the interactive information processing mode using the emojis or the emojis and the corresponding sound data; and a content sending module, which is used to send the interactive content to a live broadcast server, so that the live broadcast server sends the interactive content to other terminals joining the live video broadcast for display.
[0012] Optionally, the emoji includes: emoticons; the sound data includes: MIDI audio, wavetable information; the sound data is stored in a corresponding XMF file; the data server pre-establishes an association relationship between the emoji ID and the XMF file, and stores the association relationship.
[0013] Optionally, the interactive information processing mode includes: adding an emoji display mode; the data acquisition module is used to obtain an emoji ID and obtain the emoji from the data server according to the emoji ID; the content processing module is used to generate the interactive content based on the emoji adding mode and display it in the display area using the emoji.
[0014] Optionally, the interactive information processing mode includes: a background overlay emoji display mode; the terminal is a live host terminal; the data acquisition module is used to obtain the emoji ID, and obtain the emoji and the associated XMF file from the data server according to the emoji ID; the content processing module is used for the live host terminal to generate the interactive content according to the emoji and the associated XMF file; determine the background layer and the foreground layer in the local video image; based on the background overlay emoji display mode, add an interactive content layer between the background layer and the foreground layer, and display the interactive content in the interactive content layer.
[0015] Optionally, the interactive information processing mode includes: an emoji and audio synthesis mode; the data acquisition module is used to obtain an emoji ID, and obtain the emoji and the associated XMF file from the data server according to the emoji ID; the content processing module is used to generate and display the interactive content based on the emoji and audio synthesis mode, using the emoji and the associated XMF file.
[0016] Optionally, the interactive information processing mode includes: an emoji sound effect processing mode; the terminal is a live broadcast client terminal; the data acquisition module is used to obtain the emoji ID, and obtain the emoji and the associated XMF file from the data server according to the emoji ID; the content processing module is used to extract wavetable information from the XMF file, randomly assign pitches based on the wavetable information and perform chord processing to generate audio information; based on the emoji sound effect processing mode, the emoji and the audio information are used to generate the interactive content and display it.
[0017] Optionally, the content processing module is further used to generate and display barrage content based on the interactive content; the content sending module is used to send the barrage content to the live broadcast server, so that the live broadcast server sends the barrage content to the other terminals for display.
[0018] According to a third aspect of the present disclosure, there is provided an interactive information processing device, including:
[0019] A memory; and a processor coupled to the memory, the processor being configured to execute the method as described above based on instructions stored in the memory.
[0020] According to a fourth aspect of the present disclosure, a terminal is provided, comprising: the interactive information processing device as described above.
[0021] According to a fifth aspect of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium non-temporarily stores computer instructions, and the instructions are executed by a processor to perform the method as described above.
[0022] The interactive information processing method, device, terminal and storage medium disclosed in the present invention obtain emojis or emojis and corresponding sound data from a server, use emojis or emojis and corresponding sound data to generate interactive content and display it; send the interactive content to a live broadcast server, so that the live broadcast server sends the interactive content to other terminals that join the video live broadcast for display; by storing the emojis and sound data on the server side, the transmission traffic requirement is relatively low, and the emojis and sound data can be easily updated and expanded; it can support a large amount of audio concurrency, provide rich sound and light effects, and will not affect the broadcast effect of the host; it enriches the interactive information between users and improves the user experience. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0024] Figure 1 is a flow chart of an embodiment of an interactive information processing method according to the present disclosure;
[0025] Figure 2 A schematic diagram of a process of generating interactive content in one embodiment of an interactive information processing method according to the present disclosure;
[0026] Figure 3 A schematic diagram of a flow chart of generating interactive content in another embodiment of the interactive information processing method according to the present disclosure;
[0027] Figure 4 A schematic diagram of a process of generating interactive content in another embodiment of the interactive information processing method according to the present disclosure;
[0028] Figure 5 A schematic diagram of a process of generating interactive content in another embodiment of the interactive information processing method according to the present disclosure;
[0029] Figure 6 A schematic diagram of an application scenario of an embodiment of the interactive information processing method according to the present disclosure;
[0030] Figure 7 is a module diagram of an embodiment of an interactive information processing device according to the present disclosure;
[0031] Figure 8 It is a module diagram of another embodiment of the interactive information processing device according to the present disclosure. DETAILED DESCRIPTION
[0032] The present disclosure is described more fully below with reference to the accompanying drawings, in which exemplary embodiments of the present disclosure are described. The technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure. The technical solutions of the present disclosure are described in many aspects below in conjunction with various figures and embodiments.
[0033] Figure 1FIG. 1 is a flow chart of an embodiment of the interactive information processing method according to the present disclosure, as shown in FIG. Figure 1 As shown:
[0034] Step 101: During the live video broadcast, the terminal determines an interactive information processing mode.
[0035] In one embodiment, the terminal may be a mobile phone, a tablet computer, etc. The terminal pre-installs a video live broadcast APP, and uses the video live broadcast APP to publish, play, and watch videos. The interactive information may be likes, applause, comment information, etc. There are multiple interactive information processing modes, and corresponding options may be set in the video live broadcast APP. The interactive information processing mode is determined by the user selecting an option on the video live broadcast APP.
[0036] Step 102: The terminal obtains the emoji or the emoji and the corresponding sound data from the data server; wherein the data server pre-stores the emoji and the sound data.
[0037] In one embodiment, an emoji is a visual emotional symbol used in a text, which is encoded using Unicode and is usually rendered as an icon in a computer device. An emoji may be a graphic emoticon, such as a QQ emoticon, a WeChat emoticon, etc. The sound data may be a variety of sound files.
[0038] Step 103: The terminal generates and displays interactive content using emojis or emojis and corresponding sound data based on the interactive information processing mode. The terminal can use a live video APP to display interactive content, display emoticons corresponding to emojis, and play sound data.
[0039] Step 104: The terminal sends the interactive content to the live broadcast server, so that the live broadcast server sends the interactive content to other terminals that join the video live broadcast for display.
[0040] In one embodiment, the terminal uses a live video broadcast APP to send the interactive content to a live broadcast server, and the live broadcast server sends the interactive content to the live video broadcast APP installed in other terminals that join the live video broadcast, and the live video broadcast APP displays it.
[0041] Danmu refers to the content displayed directly on the video. It can appear on the video in the form of scrolling, staying, or even more action effects. It can be comments or feelings sent by the video viewers. Danmu will appear on the video screen in real time in the form of sliding subtitles to ensure that all viewers can notice it, thereby achieving interaction between viewers. The terminal generates and displays the Danmu content based on the interactive content. The terminal sends the Danmu content to the live broadcast server so that the live broadcast server sends the Danmu content to other terminals for display. Existing methods can be used to generate and display Danmu content based on interactive content.
[0042] In one embodiment, the emoji includes emoticons, etc.; the sound data includes MIDI audio, wavetable information, etc., and the sound data is stored in a corresponding XMF file. The data server pre-establishes an association between the emoji ID and the XMF file, and stores the association.
[0043] XMF (Extensible Music Format) is a tree-like video file format used to bundle music as a guide. MIDI (Musical Instrument Digital Interface) uses digital control signals of notes to record music. A complete MIDI music is only a few dozen KB and can contain dozens of music tracks.
[0044] Wavetable (WAVE TABLE) records all the sounds that can be produced by various real musical instruments (including various ranges and tones) and stores them as a wavetable file. When playing, it sends instructions to the wavetable according to the music information recorded in the MIDI file, finds the corresponding sound information one by one from the "table", and plays it back after synthesis and processing.
[0045] like Figure 6 As shown, in the interactive information processing method disclosed in the present invention, emoticons are uniformly managed by emojis, associated with XMF data, and stored in the data server 64, which effectively reduces the data size and facilitates management and storage. In the data server 64, emoticons are stored in the font library as emojis and associated with corresponding XMFs, and MIDI, WAVE data, wave tables, etc. are stored in XMF. Terminals 61, 62, and 63 can obtain emojis, XMF files, etc. from the data server 64. The data server 64 and the live broadcast server 65 can be integrated.
[0046] In one embodiment, the interactive information processing mode includes adding an emoji display mode. Figure 2 FIG. 1 is a flow chart of generating interactive content in one embodiment of the interactive information processing method according to the present disclosure, as shown in FIG. Figure 2 As shown:
[0047] Step 201: The terminal obtains an emoji ID, and obtains the emoji from a data server according to the emoji ID.
[0048] In one embodiment, an icon corresponding to the emoji is set on the live video APP, and when the user selects the icon on the live video APP, the emoji ID corresponding to the icon is obtained. The terminal obtains the emoji corresponding to the emoji ID on the data server through the live video APP.
[0049] Step 202: Based on the emoji adding mode, the terminal generates interactive content using emoji and displays it in the display area.
[0050] When the interactive information processing mode is to add an emoji display mode, the terminal generates interactive content, i.e., bullet screen content, using emojis through the live video APP, and displays it through the live video APP. For example, using the live video APP, emojis are displayed in a dialog box in the video. Existing methods can be used to generate interactive content using emojis and display them in the display area.
[0051] In one embodiment, the interactive information processing mode includes a background overlay emoji display mode, and the terminal is a live host terminal. Figure 3 FIG. 1 is a flow chart of generating interactive content in another embodiment of the interactive information processing method according to the present disclosure, as shown in FIG. Figure 3 As shown:
[0052] Step 301: The live host terminal obtains an emoji ID, and obtains the emoji and the associated XMF file from a data server according to the emoji ID.
[0053] In one embodiment, the live host terminal is a terminal used by the host and the anchor. When the host and the anchor select an icon on the live video APP, the emoji ID corresponding to the icon is obtained. The live host terminal obtains the emoji corresponding to the emoji ID and the associated XMF file on the data server through the live video APP.
[0054] Step 302: The live host terminal generates interactive content according to the emoji and the associated XMF file. The interactive content may be a bullet screen content, etc., and the existing method is used to generate the bullet screen content according to the emoji and the associated XMF file.
[0055] Step 303: The live host terminal determines the background layer and the foreground layer in the local video image. In the live video APP, the existing dynamic cutout technology can be used to determine the background layer and the foreground layer in the local video image.
[0056] Step 304: The live host terminal adds an interactive content layer between the background layer and the foreground layer based on the background overlay emoji display mode, and displays the interactive content in the interactive content layer. In the live video APP, the existing technology can be used to add an interactive content layer between the background layer and the foreground layer, and display the interactive content in the interactive content layer.
[0057] When displaying text or emoji on the local video image, the existing dynamic cutout technology is used to separate the foreground and background of the video image, and the text or emoji is superimposed between the foreground and the background, and can be covered by the foreground. For example, in online teaching, for students who answer questions well, the host superimposes a red star on their local video and displays the word "winner" in a scrolling manner.
[0058] In one embodiment, the interactive information processing mode includes emoji and audio synthesis modes. Figure 4 FIG. 1 is a flow chart of generating interactive content in another embodiment of the interactive information processing method according to the present disclosure, as shown in FIG. Figure 4 As shown:
[0059] Step 401: The terminal obtains an emoji ID, and obtains the emoji and the associated XMF file from a data server according to the emoji ID.
[0060] Step 402: The terminal generates and displays interactive content using emoji and associated XMF files based on the emoji and audio synthesis mode. Existing technologies can be used to generate and display interactive content using emoji and associated XMF files. The interactive content may be bullet screen content or the like.
[0061] The emoji is associated with an XMF data (file), which contains a MIDI or wavedata, and can be used in scenarios that do not require multiple mixing. Application scenarios can be: one-to-many, one-to-one, when the teacher asks questions and students need to answer, or when reminding students to pay attention.
[0062] In one embodiment, the interactive information processing mode includes an emoji sound effect processing mode, and the terminal is a live broadcast client terminal. The live broadcast client terminal is a terminal of a client participating in the live broadcast, and the client is not a host or an anchor. Figure 5 FIG. 1 is a flow chart of generating interactive content in another embodiment of the interactive information processing method according to the present disclosure, as shown in FIG. Figure 5 As shown:
[0063] Step 501, the live broadcast client terminal obtains the emoji ID, and obtains the emoji and the associated XMF file from the data server according to the emoji ID.
[0064] Step 502: The live broadcast client terminal extracts wave table information from the XMF file, randomly assigns pitches based on the wave table information, performs chord processing, and generates audio information. Existing methods can be used to randomly assign pitches based on the wave table information and perform chord processing to generate audio information.
[0065] Step 503: The live broadcast client terminal generates and displays interactive content using emoji and audio information based on the emoji sound effect processing mode.
[0066] The XMF file associated with the emoji contains a built-in wave table or wave data, which randomly assigns pitches, performs chord processing, and performs sound effects processing. Application scenarios include: many-to-one, student response, applause, flower presentation, etc., and can process chords such as applause.
[0067] In one embodiment, Figure 7 As shown, the present disclosure provides an interactive information processing device 70, which is applied to a terminal, and includes a mode determination module 71, a data acquisition module 72, a content processing module 73 and a content sending module 74. The mode determination module 71 determines the interactive information processing mode during the video live broadcast. The data acquisition module 72 obtains emojis or emojis and corresponding sound data from a data server; wherein the data server pre-stores emojis and sound data.
[0068] The content processing module 73 generates and displays interactive content using emoji or emoji and corresponding sound data based on the interactive information processing mode. The content sending module 74 sends the interactive content to the live broadcast server so that the live broadcast server sends the interactive content to other terminals participating in the video live broadcast for display.
[0069] Emojis include emoticons, etc.; sound data include MIDI audio, wavetable information, etc., and sound data is stored in the corresponding XMF file. The data server pre-establishes the association between the emoji ID and the XMF file, and stores the association.
[0070] The content processing module 73 generates and displays the bullet screen content according to the interactive content. The content sending module 74 sends the bullet screen content to the live broadcast server so that the live broadcast server sends the bullet screen content to other terminals for display.
[0071] In one embodiment, the interactive information processing mode includes an emoji display mode. The data acquisition module 72 acquires an emoji ID and acquires the emoji from a data server according to the emoji ID. The content processing module 73 generates interactive content using the emoji based on the emoji mode and displays it in the display area.
[0072] The interactive information processing mode includes a background superimposed emoji display mode, and the terminal is a live host terminal. The data acquisition module 72 obtains the emoji ID, and obtains the emoji and the associated XMF file from the data server according to the emoji ID. The content processing module 73 of the live host terminal generates interactive content according to the emoji and the associated XMF file, and determines the background layer and the foreground layer in the local video image. Based on the background superimposed emoji display mode, the content processing module 73 adds an interactive content layer between the background layer and the foreground layer, and displays the interactive content in the interactive content layer.
[0073] The interactive information processing mode includes an emoji and audio synthesis mode. The data acquisition module 72 acquires an emoji ID and acquires the emoji and the associated XMF file from the data server according to the emoji ID. The content processing module 73 generates and displays interactive content using the emoji and the associated XMF file based on the emoji and audio synthesis mode.
[0074] The interactive information processing mode includes an emoji sound effect processing mode, and the terminal is a live client terminal. The data acquisition module 72 obtains the emoji ID, and obtains the emoji and the associated XMF file from the data server according to the emoji ID. The content processing module 74 extracts the wave table information from the XMF file, randomly assigns pitches based on the wave table information and performs chord processing to generate audio information. Based on the emoji sound effect processing mode, the content processing module 74 uses the emoji and audio information to generate interactive content and display it.
[0075] In one embodiment, Figure 8 FIG. 1 is a schematic diagram of a module of another embodiment of an interactive information processing device according to the present disclosure. Figure 8 As shown, the device may include a memory 81, a processor 82, a communication interface 83 and a bus 84. The memory 81 is used to store instructions, the processor 82 is coupled to the memory 81, and the processor 82 is configured to execute the above-mentioned interactive information processing method based on the instructions stored in the memory 81.
[0076] The memory 81 may be a high-speed RAM memory, a non-volatile memory, etc. The memory 81 may also be a memory array. The memory 81 may also be divided into blocks, and the blocks may be combined into virtual volumes according to certain rules. The processor 82 may be a central processing unit CPU, or an application-specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the interactive information processing method of the present disclosure.
[0077] In one embodiment, the present disclosure provides a terminal, including the interactive information processing device in any of the above embodiments. The terminal may be a mobile phone, a tablet computer, etc.
[0078] In one embodiment, the present disclosure provides a computer-readable storage medium, which stores computer instructions. When the instructions are executed by a processor, the interactive information processing method in any of the above embodiments is implemented.
[0079] The interactive information processing method, device, terminal and storage medium provided in the above embodiments obtain emojis or emojis and corresponding sound data from the server side, use emojis or emojis and corresponding sound data to generate interactive content and display it; send the interactive content to the live broadcast server, so that the live broadcast server sends the interactive content to other terminals that join the video live broadcast for display; by storing the emojis and sound data on the server side, the transmission traffic requirements are relatively low, and the emojis and sound data can be easily updated and expanded; it can support a large amount of audio concurrency, provide rich sound and light effects, and will not affect the broadcast effect of the host; it enriches the interactive information between users and improves the user experience.
[0080] The method and system of the present disclosure may be implemented in many ways. For example, the method and system of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present disclosure may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present disclosure. Therefore, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.
[0081] The description of the present disclosure is given for the purpose of illustration and description, and is not intended to be exhaustive or to limit the present disclosure to the disclosed form. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiments are selected and described in order to better illustrate the principles and practical applications of the present disclosure, and to enable those of ordinary skill in the art to understand the present disclosure and thereby design various embodiments with various modifications suitable for specific uses.
Claims
1. A method for processing interactive information, include: During the live video broadcast, the terminal determines the interactive information processing mode; The terminal obtains the emoji or the emoji and the corresponding sound data from the data server; The data server pre-stores the emoji and the sound data; the emoji includes: emoticons; the sound data includes: MIDI audio, wave table information; the sound data is stored in the corresponding XMF file; the interactive information processing mode includes: background overlay emoji display mode; the terminal is a live host terminal; the live host terminal obtains the emoji ID, and obtains the emoji and the associated XMF file from the data server according to the emoji ID; the interactive information processing mode includes: emoji sound effect processing mode; the terminal is a live client terminal, the live client terminal obtains the emoji ID, and obtains the emoji and the associated XMF file from the data server according to the emoji ID; The terminal generates and displays interactive content using the emoji or the emoji and corresponding sound data based on the interactive information processing mode; The live host terminal generates the interactive content according to the emoji and the associated XMF file; the live host terminal determines the background layer and the foreground layer in the local video image; the live host terminal adds an interactive content layer between the background layer and the foreground layer based on the background superimposed emoji display mode, and displays the interactive content in the interactive content layer; the live client terminal extracts the wave table information from the XMF file, randomly assigns pitches based on the wave table information and performs chord processing to generate audio information; the live client terminal generates and displays the interactive content using the emoji and the audio information based on the emoji sound effect processing mode; The terminal sends the interactive content to a live broadcast server, so that the live broadcast server sends the interactive content to other terminals that join the video live broadcast for display.
2. The method according to claim 1, in, The data server pre-establishes an association relationship between the emoji ID and the XMF file, and stores the association relationship.
3. The method according to claim 2, wherein the interactive information processing mode include: Added emoji display mode; The terminal obtains the emoji or the emoji and the corresponding sound data from the data server, including: The terminal obtains an emoji ID, and obtains the emoji from the data server according to the emoji ID; The terminal generates and displays interactive content using the emoji or the emoji and corresponding sound data based on the interactive information processing mode, including: Based on the added emoji display mode, the terminal generates the interactive content using the emoji and displays it in the display area.
4. The method according to claim 2, wherein the interactive information processing mode include: Emoji and audio synthesis modes; The terminal obtains the emoji or the emoji and the corresponding sound data from the data server, including: The terminal obtains an emoji ID, and obtains the emoji and the associated XMF file from the data server according to the emoji ID; The terminal generates and displays interactive content using the emoji or the emoji and corresponding sound data based on the interactive information processing mode, including: The terminal generates and displays the interactive content using the emoji and the associated XMF file based on the emoji and audio synthesis mode.
5. The method according to any one of claims 1 to 4, in, The terminal generates and displays bullet screen content according to the interactive content; The terminal sends the barrage content to a live broadcast server, so that the live broadcast server sends the barrage content to the other terminals for display.
6. An interactive information processing device, applied in a terminal, include: A mode determination module is used to determine the interactive information processing mode during the video live broadcast; A data acquisition module is used to acquire emojis or emojis and corresponding sound data from a data server; wherein the data server pre-stores the emojis and the sound data; the emojis include: emoticons; the sound data include: MIDI audio, wave table information; the sound data is stored in a corresponding XMF file; A content processing module, configured to generate and display interactive content using the emoji or the emoji and corresponding sound data based on the interactive information processing mode; The interactive information processing mode includes: a background superimposed emoji display mode; the terminal is a live host terminal; the data acquisition module is used to obtain an emoji ID, and obtain the emoji and the associated XMF file from the data server according to the emoji ID; the content processing module is used for the live host terminal to generate the interactive content according to the emoji and the associated XMF file; determine the background layer and the foreground layer in the local video image; based on the background superimposed emoji display mode, add an interactive content layer between the background layer and the foreground layer, and display the interactive content in the interactive content layer; The interactive information processing mode includes: an emoji sound effect processing mode; the terminal is a live broadcast client terminal; the data acquisition module is used to obtain an emoji ID, and obtain the emoji and the associated XMF file from the data server according to the emoji ID; the content processing module is used to extract wave table information from the XMF file, randomly assign pitches based on the wave table information and perform chord processing to generate audio information; based on the emoji sound effect processing mode, the interactive content is generated and displayed using the emoji and the audio information; The content sending module is used to send the interactive content to the live broadcast server, so that the live broadcast server sends the interactive content to other terminals participating in the video live broadcast for display.
7. The device according to claim 6, in, The data server pre-establishes an association relationship between the emoji ID and the XMF file, and stores the association relationship.
8. The device according to claim 7, wherein the interactive information processing mode include: Added emoji display mode; The data acquisition module is used to obtain an emoji ID, and obtain the emoji from the data server according to the emoji ID; The content processing module is used to generate the interactive content using the emoji based on the added emoji display mode and display it in the display area.
9. The device according to claim 7, wherein the interactive information processing mode include: Emoji and audio synthesis modes; The data acquisition module is used to obtain an emoji ID, and obtain the emoji and the associated XMF file from the data server according to the emoji ID; The content processing module is used to generate and display the interactive content using the emoji and the associated XMF file based on the emoji and audio synthesis mode.
10. The device according to any one of claims 6 to 9, in, The content processing module is further used to generate and display bullet screen content according to the interactive content; The content sending module is used to send the barrage content to the live broadcast server, so that the live broadcast server sends the barrage content to the other terminals for display.
11. An interactive information processing device, include: Memory; and a processor coupled to the memory, wherein the processor is configured to execute the method according to any one of claims 1 to 5 based on instructions stored in the memory.
12. A terminal, include: An interactive information processing device as claimed in any one of claims 6 to 11. 13 . A computer-readable storage medium, wherein the computer-readable storage medium non-transitorily stores computer instructions, wherein the instructions are executed by a processor to execute the method according to claim 1 .
