Interactive information processing method and device, electronic equipment, computer readable storage medium and computer program product

Abstract

The application provides an interactive information processing method and device, electronic equipment, computer readable storage medium and computer program product. The method comprises the following steps: in the case that a client in a background running state plays target audio, in response to a state switching trigger operation for the client, the client is switched from the background running state to a foreground running state, wherein the target audio is used for recommending a target object, and in the interface of the client in the foreground running state, interactive information corresponding to the target object is displayed. Through the application, the interaction rate of the advertisement and the participation of the user can be significantly improved.

Description

Technical Field

[0001] This application relates to the field of Internet technology, and in particular to an interactive information processing method, apparatus, electronic device, computer-readable storage medium, and computer program product. Background Technology

[0002] With the booming development of the internet industry, especially the mobile internet, information promotion via the internet has become a new and widely used form of information dissemination. Advertising, as a typical technology for information promotion on the internet, has become an important way for users to obtain information and for advertisers to enhance brand awareness and promote products. Audio ad inserts refer to short audio advertising segments inserted during audio content playback, typically used for brand promotion or product advertising. However, in the solutions provided by related technologies, users can only hear the audio ad inserts while listening to audio in the background and cannot see the advertising content, resulting in users being unable to effectively interact with the ads, thus affecting the conversion rate and user engagement. Summary of the Invention

[0003] This application provides an interactive information processing method, apparatus, electronic device, computer-readable storage medium, and computer program product, which can significantly improve the conversion rate of advertisements and user engagement.

[0004] The technical solution of this application embodiment is implemented as follows:

[0005] This application provides an interactive information processing method, including:

[0006] When a target audio is played on a client running in the background, in response to a state switching trigger operation for the client, the client is switched from the background running state to the foreground running state, wherein the target audio is used to recommend target objects, and

[0007] In the client interface, which is running in the foreground, interactive information corresponding to the target object is displayed.

[0008] In the above scheme, before displaying the interactive information corresponding to the target object, the method further includes: sending an information acquisition request to the server; receiving the interactive information corresponding to the target object returned by the server, wherein the interactive information is obtained by the server in the following way: based on the target text corresponding to the target audio, calling the text-generated graph model for generation processing to obtain the interactive information corresponding to the target object.

[0009] This application provides an interactive information processing device, including:

[0010] A switching module is used to switch the client from the background running state to the foreground running state in response to a state switching trigger operation for the client when the target audio is played in the background running state, wherein the target audio is used to recommend target objects;

[0011] The display module is used to display interactive information corresponding to the target object in the interface of the client that is in the foreground running state.

[0012] This application provides another interactive information processing method, including:

[0013] Play multimedia data;

[0014] In response to the fulfillment of audio playback conditions, the playback of the multimedia data is paused, the target audio is played, and the target object recommended by the target audio is displayed;

[0015] In response to a trigger operation on the target object, interactive information corresponding to the target object is displayed.

[0016] In the above scheme, displaying interactive information corresponding to the target object includes performing one of the following processes: displaying a virtual container including the object identifier of the target object, wherein the virtual container encapsulates incentive information; displaying at least one virtual card including the object identifier of the target object, wherein different virtual cards are associated with different incentive information; or displaying a virtual image related to the target object, wherein the virtual image is associated with incentive information.

[0017] In the above scheme, after displaying the interactive information corresponding to the target object, the method further includes: in response to an interactive operation based on the interactive information, displaying an interactive result page, wherein the interactive result page includes incentive information allocated to a first account, the first account being the currently logged-in account of a client used to play the multimedia data.

[0018] In the above scheme, when the interactive information is a virtual container encapsulating incentive information, the step of displaying an interactive result page in response to an interactive operation based on the interactive information includes: displaying an interactive result page in response to an interactive operation based on the virtual container, and playing first multimedia data for recommending the target object on the interactive result page; and displaying the incentive information assigned to the client's currently logged-in account on the interactive result page in response to the completion of the first multimedia data playback.

[0019] In the above scheme, the step of displaying an interaction result page in response to an interactive operation based on the virtual container includes: in response to a trigger operation on the virtual container, controlling the virtual container to deform during the period before the trigger operation is released, wherein the degree of deformation is positively correlated with the pressure parameter or duration of the trigger operation; in response to the degree of deformation of the virtual container being greater than a degree threshold, playing an animation of the virtual container breaking, and displaying the interaction result page after the animation finishes playing.

[0020] In the above scheme, the step of displaying the interaction result page in response to the interactive operation based on the virtual container includes: in response to a motion-sensing operation on the terminal device where the client is located, and the operation parameter of the motion-sensing operation is greater than the operation parameter threshold, playing an animation of the virtual container breaking; and in response to the completion of the animation playback, displaying the interaction result page.

[0021] In the above scheme, when the interaction information is at least one virtual card, the step of displaying the interaction result page in response to the interaction operation based on the interaction information includes: in response to the trigger operation for any of the virtual cards, displaying the interaction result page, and displaying the incentive information assigned to the current login account of the client on the interaction result page, wherein the incentive information is the incentive information associated with the triggered virtual card, and the incentive information associated with different virtual cards is different.

[0022] In the above scheme, when the interactive information is a virtual avatar, the step of displaying an interactive result page in response to an interactive operation based on the interactive information includes: performing one of the following processes: controlling the virtual avatar to output guidance information, wherein the guidance information is used to instruct the implementation of an interactive action; in response to obtaining an interactive event corresponding to the interactive action, displaying an interactive result page and displaying incentive information assigned to the client's current login account on the interactive result page, wherein different interactive actions are associated with different incentive information; in response to controlling the virtual avatar to complete a set task, displaying an interactive result page and displaying incentive information assigned to the client's current login account on the interactive result page, wherein different tasks are associated with different incentive information.

[0023] In the above scheme, after displaying the interactive information corresponding to the target object, the method further includes: in response to no interactive operation based on the interactive information being received within a set time period, canceling the display of the interactive information in the interface of the client in the foreground running state; or, in response to the display time of the interactive information reaching a time threshold, automatically displaying an interactive result page, and displaying incentive information allocated to the client's current login account in the interactive result page.

[0024] In the above scheme, the step of pausing the playback of the multimedia data and playing the target audio in response to meeting the audio playback conditions includes: performing one of the following processes: pausing the playback of the multimedia data and playing the target audio in response to the playback duration of the multimedia data exceeding a duration threshold; pausing the playback of the multimedia data and playing the target audio in response to the playback progress of the multimedia data reaching a set time point; or pausing the playback of the multimedia data and playing the target audio in response to the occurrence of a set scene in the multimedia data.

[0025] In the above scheme, playing the target audio includes: obtaining account feature data of the currently logged-in account of the client used to play the multimedia data; for each audio in a pre-created audio library, determining the similarity between the audio feature data of the audio and the account feature data, wherein different audios are used to recommend different objects; taking the audio with the highest similarity as the target audio and playing the target audio.

[0026] In the above scheme, playing the target audio includes: sending an audio acquisition request to the server so that the server generates target audio that matches the content of the multimedia data; receiving the target audio returned by the server and playing the target audio.

[0027] In the above scheme, playing the target audio includes: obtaining the status data of the terminal device where the client used to play the multimedia data is located, and determining the current listening scenario based on the status data; filtering out audio that matches the current listening scenario from a pre-created audio library, using the matched audio as the target audio, and playing the target audio.

[0028] In the above scheme, playing the target audio includes: sending an audio acquisition request to the server, wherein the audio acquisition request carries account feature data of the currently logged-in account of the client used to play the multimedia data; receiving the target audio returned by the server and playing the target audio, wherein the target audio is obtained by the server through the following methods: obtaining a target object matching the account feature data from an object library and generating target text for recommending the target object; and converting the target text into the target audio through a text-to-speech service.

[0029] In the above scheme, playing the target audio includes: acquiring the voiceprint information of the target character in the multimedia data; and playing the target audio according to the voiceprint information.

[0030] In the above scheme, after displaying the interactive information corresponding to the target object, the method further includes: acquiring behavioral data, wherein the behavioral data includes: the triggering timing for the target object and the interaction status for the interactive information; and adjusting the playback timing of the target audio and the interactive information based on the behavioral data.

[0031] This application provides another interactive information processing device, including:

[0032] The playback module is used to play multimedia data;

[0033] The playback module is also configured to pause the playback of the multimedia data and play the target audio in response to the fulfillment of audio playback conditions;

[0034] The display module is used to display the target object recommended by the target audio;

[0035] The display module is also used to display interactive information corresponding to the target object in response to a trigger operation on the target object.

[0036] This application provides an electronic device, including:

[0037] Memory is used to store executable instructions for a computer;

[0038] The processor, when executing computer-executable instructions stored in the memory, implements the interactive information processing method provided in the embodiments of this application.

[0039] This application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the interactive information processing method provided in this application.

[0040] This application provides a computer program product, including a computer program or computer executable instructions, which, when executed by a processor, implements the interactive information processing method provided in this application.

[0041] The embodiments of this application have the following beneficial effects:

[0042] When a user hears the target audio used to recommend a specific object in the background, if they are interested in the target object, they can switch the client from background to foreground. At this time, the interface of the client in foreground can display interactive information corresponding to the target object. In this way, a dual stimulation of auditory and visual senses can be formed, which can significantly improve the conversion rate of the advertisement and the user's engagement. Attached Figure Description

[0043] Figure 1This is a schematic diagram of the architecture of the interactive information processing system 100 provided in the embodiments of this application;

[0044] Figure 2 This is a schematic diagram of the structure of the electronic device 500 provided in the embodiments of this application;

[0045] Figure 3 This is a first flowchart illustrating the interactive information processing method provided in the embodiments of this application;

[0046] Figure 4 This is a second flowchart illustrating the interactive information processing method provided in the embodiments of this application;

[0047] Figure 5 This is a schematic diagram of the third process of the interactive information processing method provided in the embodiments of this application;

[0048] Figure 6 This is a schematic diagram illustrating an application scenario of the interactive information processing method provided in the embodiments of this application;

[0049] Figure 7 This is a schematic diagram of the architecture of the interactive information processing system provided in the embodiments of this application;

[0050] Figure 8 This is a schematic diagram of the fourth process of the interactive information processing method provided in the embodiments of this application. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0052] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0053] It is understood that in the embodiments of this application, data such as user information are involved. When the embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with relevant laws, regulations and standards.

[0054] In the following description, the terms “first, second, ...” are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that “first, second, ...” may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0055] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0056] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0057] 1) Responding to: used to indicate the conditions or states on which the operation is performed depends. When the conditions or states on which it depends are met, one or more operations can be performed in real time or with a set delay. Unless otherwise specified, there is no restriction on the order in which the multiple operations are performed.

[0058] 2) Background running state: This refers to the state in which the client continues to execute tasks without occupying the foreground interface. For example, when the client is in the background running state, it can execute tasks in a mode invisible to the user, allowing the user to operate other clients simultaneously without being affected. In the background running state, the client's interface can be minimized (e.g., minimized to a component) or completely hidden, without occupying screen display space.

[0059] 3) Foreground running state: This refers to the state where the client interface directly occupies the user's current operating interface, and the user can interact through the screen interface. In the foreground running state, the client interface can be directly displayed in front of the screen, such as when the window is maximized, and it usually exclusively occupies the user's input device, which may block other operations until the task is completed.

[0060] 4) Text-to-Speech (TTS): This refers to the technology that converts text content into natural speech output. TTS is widely used in various scenarios, such as voice assistants, navigation systems, e-book reading, educational software, and providing information services for visually impaired people.

[0061] 5) Audio ad overlay: also known as audio ad or audio advertisement, refers to a short audio advertisement segment inserted during the playback of audio content (or video content), usually used for brand promotion or product promotion.

[0062] This application provides an interactive information processing method, apparatus, electronic device, computer-readable storage medium, and computer program product, which can significantly improve the interaction rate of advertisements and user engagement. The electronic device provided in this application embodiment is described below. The electronic device provided in this application embodiment can be implemented as a terminal device.

[0063] For example, see Figure 1 , Figure 1 This is a schematic diagram of the architecture of the interactive information processing system 100 provided in the embodiments of this application, as shown below. Figure 1 As shown, the interactive information processing system 100 provided in this application embodiment includes: a server 200, a network 300, and a terminal device 400. The network 300 can be a local area network or a wide area network, or a combination of the two. The terminal device 400 is a terminal device associated with the user. A client 410 runs on the terminal device 400. The client 410 can be various types of clients, such as video playback clients, audio playback clients, and browsers.

[0064] In some embodiments, taking client 410 as an audio playback client as an example, audio (such as crosstalk, audiobooks, etc.) can be played through client 410 running in the background. For example, a user can listen to crosstalk in the background. When the audio reaches a preset advertising trigger time (such as when the audio playback duration reaches 30 minutes), the audio playback can be paused and an audio overlay advertisement (i.e., the target video) can be played. When the user is interested in the audio overlay advertisement, client 410 can be switched from the background running state to the foreground running state. At this time, the interface of client 410 running in the foreground state can display interactive information corresponding to the audio overlay advertisement (such as virtual blind boxes, virtual gifts, virtual packages, virtual cards, etc.). In this way, a dual stimulation of auditory and visual senses can be formed, thereby significantly improving the advertising interaction rate and user engagement.

[0065] It should be noted that, Figure 1The server 200 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. The terminal device 400 can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, in-vehicle terminal, etc., but is not limited to these. The terminal device 400 and the server 200 can be directly or indirectly connected via wired or wireless communication, which is not limited in this embodiment.

[0066] In other embodiments, the terminal device may also implement the interactive information processing method provided in this application embodiment by running various computer-executable instructions or computer programs. For example, computer-executable instructions may be microprogram-level commands, machine instructions, or software instructions. Computer programs may be native programs or software modules in an operating system; they may be native applications (APPs), i.e., programs that need to be installed in the operating system to run, such as an audio playback APP; or they may be applets that can be embedded in any APP, i.e., programs that only need to be downloaded to a browser environment to run. In summary, the aforementioned computer-executable instructions may be any form of instruction, and the aforementioned computer programs may be any form of application, module, or plugin.

[0067] The structure of the electronic device provided in the embodiments of this application will be further described below. Taking the electronic device as a terminal device as an example, see... Figure 2 , Figure 2 This is a schematic diagram of the structure of the electronic device 500 provided in the embodiments of this application. Figure 2 The illustrated electronic device 500 includes at least one processor 510, a memory 550, at least one network interface 520, and a user interface 530. The various components in the electronic device 500 are coupled together via a bus system 540. It is understood that the bus system 540 is used to implement communication between these components. In addition to a data bus, the bus system 540 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 2 The general labeled all buses as Bus System 540.

[0068] The processor 510 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0069] User interface 530 includes one or more output devices 531 that enable the presentation of media content, including one or more speakers and / or one or more visual displays. User interface 530 also includes one or more input devices 532, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0070] The memory 550 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 550 may optionally include one or more storage devices physically located away from the processor 510.

[0071] The memory 550 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 550 described in this application embodiment is intended to include any suitable type of memory.

[0072] In some embodiments, memory 550 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0073] Operating system 551 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0074] The network communication module 552 is used to reach other computing devices via one or more (wired or wireless) network interfaces 520, exemplary network interfaces 520 including: Bluetooth, WiFi, and Universal Serial Bus (USB), etc.

[0075] Presentation module 553 is configured to enable the presentation of information (e.g., a user interface for operating peripheral devices and displaying content and information) via one or more output devices 531 (e.g., a display screen, a speaker, etc.) associated with user interface 530;

[0076] The input processing module 554 is used to detect and translate one or more user inputs or interactions from one or more input devices 532.

[0077] In some embodiments, the apparatus provided in this application can be implemented in software. Figure 2 An interactive information processing device 555 stored in memory 550 is shown. This device can be software in the form of programs and plug-ins, including the following software modules: a switching module 5551, a display module 5552, a playback module 5553, an acquisition module 5554, a determination module 5555, a sending module 5556, a receiving module 5557, a filtering module 5558, and an adjustment module 5559. These modules are logically connected and can therefore be arbitrarily combined or further separated according to the functions implemented. It should be noted that... Figure 2 For ease of explanation, all the above modules are shown at once. However, this should not be construed as excluding the implementation of the interactive information processing device 555, which may include only the switching module 5551 and the display module 5552, or only the playback module 5553 and the display module 5552. The functions of each module will be described below.

[0078] The interactive information processing method provided in this application will be specifically described below with reference to exemplary applications and implementations of the terminal devices provided in the embodiments of this application.

[0079] For example, see Figure 3 , Figure 3 This is a first flowchart illustrating the interactive information processing method provided in this application embodiment, which will be combined with... Figure 3 The steps shown are explained.

[0080] It should be noted that, Figure 3 The method illustrated can be executed by various forms of computer programs running on the terminal device, and is not limited to a client. For example, it can also be the operating system, software module, script, and applet mentioned above. Therefore, the client-side examples used below should not be considered as limiting the embodiments of this application. Furthermore, for ease of description, no specific distinction will be made between the terminal device and the client running on the terminal device below.

[0081] In step 101, when the target audio is played on a client running in the background, the client is switched from the background running state to the foreground running state in response to a state switching trigger operation for the client.

[0082] Here, target audio can be used to recommend target objects, which can be a specific product (such as items, services, etc.) or a brand (such as an advertiser). In other words, target audio can be audio used for brand promotion or product promotion. For example, target audio can be an audio ad (simply referred to as an audio ad or audio advertisement). For instance, when a user hears an audio ad in the background, if they are interested in the currently playing audio ad, they can click on a component of the client displayed on the screen (for example, it could be a floating window, i.e., when the client is running in the background, only a floating window can be displayed on the screen), thereby switching the client from background running state to foreground running state.

[0083] It should be noted that, for cases where the client is completely hidden, when playing the target audio through the client running in the background, a notification message can be displayed on the screen. After the user clicks on the notification message, the client can be switched from the background to the foreground.

[0084] In some embodiments, during execution Figure 3 Prior to step 101 shown, the following processes may also be performed: playing second multimedia data (e.g., video or audio, i.e., inserting audio ads during video playback or audio playback) through a client running in the background; pausing the playback of the second multimedia data in response to the audio playback condition being met, and playing the target audio through the client running in the background.

[0085] For example, taking audiobook 1 as the second multimedia data, users can listen to audiobook 1 in the background (i.e., the client can play audiobook 1 while running in the background). At the same time, audio ad overlays (i.e., target audio) can be inserted into audiobook 1. When the playback of audiobook 1 reaches the preset ad trigger time (e.g., the playback time of audiobook 1 exceeds 30 minutes), the playback of audiobook 1 can be paused and an audio ad overlay can be inserted. That is, when the user listens to audiobook 1 for more than 30 minutes, the user may become fatigued. At this time, light and interesting ad content can be inserted to alleviate the user's fatigue.

[0086] It should be noted that the audio ad inserts in audiobooks can be generated based on artificial intelligence (AI) technology. For example, the content of the audiobook currently being listened to can be analyzed to extract the user's interest tags and preference characteristics. Then, based on the user's interest tags and preference characteristics, matching ad content (such as ad copy) can be selected from the ad creative library. Subsequently, the ad copy can be converted into corresponding audio ad inserts through TTS services. In this way, personalized audio ad inserts generated using AI technology make the ad content highly relevant to the user's interests, thereby enhancing the attractiveness of the ad and the user experience.

[0087] In some embodiments, following the foregoing, the above-described playback of second multimedia data through a client running in the background can be achieved in the following ways: displaying a playback control in the interface of a client running in the foreground; playing the second multimedia data through the client running in the foreground in response to a trigger operation on the playback control; switching the client from the foreground running state to the background running state in response to a state switching trigger operation on the client, and continuing to play the second multimedia data through the client running in the background.

[0088] For example, taking an audio playback client as an example, the playback interface of the audio playback client running in the foreground can display the audiobook to be played (such as the audiobook currently selected by the user) and a play button. When the user clicks the play button, the audiobook can be played through the audio playback client running in the foreground. If the user needs to operate other clients, the audio playback client can be switched from the foreground to the background. For example, when the user triggers an upward swipe operation in the playback interface of the audio playback client, the audio playback client can be switched from the foreground to the background (for example, the interface of the audio playback client can be minimized into a small widget or completely hidden from the screen), and the audiobook can continue to play in the background. In this way, the user can perform other operations while listening to the audiobook in the background, such as opening an instant messaging client to chat, which meets the user's need to perform multiple tasks at the same time.

[0089] In other embodiments, following the foregoing, the above-described response to meeting audio playback conditions, pausing the playback of the second multimedia data and playing the target audio through a client running in the background, can be achieved in any of the following ways: pausing the playback of the second multimedia data when its duration exceeds a duration threshold (e.g., 30 minutes), and playing the target audio through a client running in the background; pausing the playback of the second multimedia data when its playback progress reaches a set time point (e.g., the insertion time point of the target audio), and playing the target audio through a client running in the background; pausing the playback of the second multimedia data when a set scenario appears in the second multimedia data, and playing the target audio through a client running in the background.

[0090] It should be noted that in practical applications, the insertion time of the target audio within the second multimedia data can also be determined based on the characteristic information of the target audio and the characteristic information of the second multimedia data to which the target audio is inserted. For example, taking the target audio as an audio overlay advertisement and the second multimedia data as audio (such as crosstalk, audiobooks, etc.) as an example, the insertion time of the audio overlay advertisement can be selected based on the characteristic information of the audio overlay advertisement to be inserted and the characteristic information of the audio overlay advertisement to which the audio overlay advertisement is inserted. The time point where the matching degree between the characteristic information corresponding to adjacent time periods in the audio and the characteristic information of the audio overlay advertisement is greater than the matching degree threshold can be selected as the insertion time point of the audio overlay advertisement during audio playback. For example, assuming that the characteristic information of the audio overlay advertisement reflects that the audio overlay advertisement is an advertisement for introducing musical instruments, and based on the characteristic information of the audio, it is determined that the music one minute before and one minute after the time point 9:00 is soothing music, then the time point 9:00 can be selected as the insertion time point of the audio overlay advertisement during audio playback. Alternatively, one can first determine the time range within the audio where audio ad overlays can be inserted, based on the audio's characteristic information. Then, based on the user's characteristic information, the insertion point within the determined time range can be selected. For example, one can first determine the time range corresponding to audio segments with volumes below a predetermined volume threshold based on the audio's characteristic information. Then, based on the user's current time and location information, it can be determined that the user is at work. Therefore, an earlier point from the determined time range can be selected as the insertion point for the audio ad overlay. This allows audio ad overlays to be inserted into the audio at a more appropriate time, reducing the abruptness of the ads and enhancing the user experience. Furthermore, this embodiment does not embed the audio ad overlay into the audio; instead, it pauses the playback of the original audio when inserting the ad overlay. This allows the original audio to be inserted normally without modification.

[0091] For example, taking audiobook 1 as the second multimedia data and audio ad as the target audio, a user can listen to audiobook 1 in the background (i.e., the audio playback client can play audiobook 1 while running in the background). When the playback time of audiobook 1 reaches the duration threshold (e.g., 30 minutes), the user may become fatigued due to listening to audiobook 1 for a long time. Therefore, the playback of audiobook 1 can be paused and a light and interesting audio ad can be inserted. This can effectively alleviate the user's fatigue.

[0092] For example, taking audiobook 1 as the second multimedia data and audio ad as the target audio, users can listen to audiobook 1 in the background (i.e., the audio playback client can play audiobook 1 while running in the background). At the same time, audio ad can be inserted into audiobook 1. For example, audio ad can be inserted between different chapters of audiobook 1. That is to say, when any chapter of audiobook 1 is finished playing, audiobook 1 can be paused and an audio ad can be inserted. In this way, the user's continuous listening experience will not be interrupted, thereby reducing the user's resistance to advertising.

[0093] For example, taking a short video (e.g., Video 1) as the second multimedia data and an audio overlay ad as the target audio, a user can listen to Video 1 in the background (i.e., the video playback client can play Video 1 while running in the background). When a set scenario (e.g., a dinner party or a date) is detected in Video 1, the playback of Video 1 can be paused, and an audio overlay ad matching the set scenario in Video 1 can be inserted. For example, assuming the set scenario in Video 1 is a dinner party, an audio overlay ad recommending a certain food can be inserted. In other words, the content of the audio overlay ad matches the content the user is currently listening to. This can effectively reduce user resistance to advertising and allow the ad format to be more naturally integrated into the user's usage scenario.

[0094] In some embodiments, the above-mentioned playback of target audio through a client running in the background can be achieved by: obtaining the account feature data of the currently logged-in account of the client; for each audio (e.g., an audio ad) in a pre-created audio library (e.g., an audio ad library), determining the similarity between the audio feature data and the account feature data, wherein different audios can be used to recommend different objects; taking the audio with the highest similarity as the target audio, and playing the target audio through the client running in the background.

[0095] For example, taking an audio playback client as an example, when the ad triggering time is met, the client can obtain the account feature data of the currently logged-in account (e.g., account 1 registered by user 1). This data includes the identity information filled in by user 1 when registering account 1, as well as search and listening records during the use of the audio playback client. Then, the client can send an ad retrieval request carrying the account feature data of account 1 to the server (e.g., an ad server). After receiving the ad retrieval request from the audio playback client, the ad server can first extract the account feature data of account 1 from the request. Then, it can calculate the similarity (e.g., cosine similarity) between the ad feature data of each audio ad in the ad library and the account feature data of account 1. The ad server can then return the audio ad with the highest similarity (e.g., audio ad 1, i.e., the audio ad that best matches user 1's interests) to the audio playback client, so that the audio playback client can play audio ad 1. In this way, the ad content is highly matched with the user's interests, thereby enhancing the ad's attractiveness and user experience.

[0096] In other embodiments, the above-mentioned playback of target audio via a client running in the background can also be achieved by: sending an audio acquisition request to the server so that the server generates target audio that matches the content of the second multimedia data; receiving the target audio returned by the server and playing the target audio via a client running in the background.

[0097] For example, taking audiobook 1 as the second multimedia data, when the ad triggering time is met, the audio playback client can send an ad retrieval request to the ad server. Upon receiving the ad retrieval request from the audio playback client, the ad server can generate an audio ad that matches the content of the audiobook 1 currently being listened to, based on AI technology. For instance, the ad server can input the content of the audiobook 1 currently being listened to into a large language model, allowing the model to generate ad copy related to the content of the audiobook 1. Then, the ad server can convert the ad copy into a corresponding audio ad through a TTS service. Subsequently, the ad server can return the generated audio ad to the audio playback client, allowing the client to play the ad returned by the ad server. In this way, customized audio ads are generated based on the content currently being listened to, ensuring a high degree of match between the ad content and the user's listening experience. This effectively reduces user resistance to ads, thereby enhancing ad appeal and user experience. Furthermore, using AI technology to generate audio ads can also effectively improve ad generation efficiency.

[0098] In some embodiments, the above-mentioned playback of target audio through a client running in the background can also be achieved by: obtaining the status data of the terminal device where the client is located (e.g., including geographical location and movement speed), and determining the user's current listening scenario based on the status data (e.g., listening while driving, listening while running indoors, listening while running outdoors, listening while walking, listening while sitting, etc.); filtering out audio (e.g., audio sticker ads) that matches the user's current listening scenario from a pre-created audio library (e.g., an audio sticker ad library), and using the matched audio as the target audio, which can then be played through a client running in the background.

[0099] For example, taking an audio playback client as an example, location technology can be used to obtain the geographical location and movement speed of the terminal device (such as the user's smartphone) where the audio playback client is located. This location and movement speed can then be used as the user's current geographical location and movement speed. Based on the user's current geographical location and movement speed, the user's current listening scenario can be determined. For example, based on the user's current geographical location and movement speed, combined with the current time, the user can be categorized into specific listening scenarios such as listening while driving (i.e., the user is on a road and moving quickly), listening while walking outdoors (i.e., the user is outdoors and moving continuously at a walking speed), and listening while running outdoors (i.e., the user is outdoors and moving continuously at a running speed). For instance, if the user's current location is detected as a road and their movement speed is greater than a first speed threshold, the user can be categorized as listening while driving. Subsequently, audio ads that match the listening scenario while driving can be selected from the pre-created audio ad library (e.g., audio ad 1). Audio ad 1 is then played through an audio playback client running in the background. In this way, the ad content inserted into the audio matches the user's current listening scenario, meaning that the ad content inserted into the audio meets the user's needs. This can effectively reduce the user's resistance to the ad, thereby increasing the ad's interaction rate and user engagement.

[0100] In other embodiments, following the examples above, the status data may include geographical location and movement speed. The current listening scenario can be determined based on the status data in the following way: input the geographical location and movement speed of the client's terminal device into a pre-trained scene recognition model to obtain the user's current listening scenario.

[0101] For example, taking an audio playback client as an example, the location and speed of the terminal device (such as the user's smartphone) where the audio playback client is located can be obtained through positioning technology. This location and speed can then be used as the user's current location and speed. The location and speed can then be input into a pre-trained scene recognition model so that the scene recognition model can output the user's current listening scene. The scene recognition model can be trained based on sample state data and listening scenes labeled with the sample state data.

[0102] The training process of the scene recognition model will be explained in detail below.

[0103] For example, sample state data can be input into the initialized scene recognition model so that the initialized scene recognition model outputs the corresponding listening scene. Then, the difference between the listening scene output by the initialized scene recognition model and the listening scene labeled for the sample data can be calculated, and backpropagation can be performed based on the difference. Finally, the parameters of the initialized scene recognition model can be updated layer by layer during the backpropagation process to obtain the trained scene recognition model.

[0104] It should be noted that the scene recognition model mentioned above can be various types of machine learning models, such as convolutional neural network models, support vector machines, deep neural network models, etc. This application embodiment does not specifically limit the type of scene recognition model.

[0105] In some embodiments, following the above example, the above-mentioned method of filtering audio from a pre-created audio library that matches the current listening scenario can be achieved in the following way: query the pre-created audio library based on the scene identifier of the current listening scenario to obtain audio that matches the current listening scenario. In the audio library, multiple scene identifiers and multiple audios corresponding to the multiple scene identifiers can be stored in key-value pairs. That is, the scene identifier can be used as the key and the audio as the value, and stored in key-value pairs.

[0106] For example, taking an audio ad library as an example, the ad library can store multiple ad messages, each belonging to a corresponding listening scenario. For instance, suppose there are three listening scenarios: scenario 1 (e.g., listening while driving), scenario 2 (e.g., listening while running), and scenario 3 (e.g., listening while sitting). Each listening scenario can be associated with at least one ad message. The ad library can store the scenario identifiers for these three listening scenarios, as well as the ad messages associated with each scenario, in key-value pairs. Thus, after determining the user's current listening scenario (e.g., listening while driving) based on their current location and speed, the ad library can be queried using the scenario identifier to obtain ad messages matching the user's current listening scenario. This key-value pair approach to scene identification and ad message storage effectively improves subsequent query efficiency.

[0107] In other embodiments, the above-mentioned playback of target audio via a client running in the background can also be achieved by: obtaining the user's current location and scene tags; determining multiple audio tracks (e.g., audio ad tracks) to be recommended from an audio library (e.g., an audio ad library) based on the user's feature information and current location; determining the score of each audio track to be recommended based on the current location, scene tags, and push data of each audio track to be recommended (e.g., data that can characterize the popularity of the audio track to be recommended, such as click-through rate, conversion rate, etc.); selecting the audio track with the highest score as the target audio track and playing the target audio via a client running in the background.

[0108] For example, location technology can be used to obtain the geographical location of the client's terminal device as the user's current location. This current location can be a geographic coordinate. When the geographic coordinate is a shopping mall, the user's scenario tag could be shopping or dining; when the geographic coordinate is a road (such as a highway or street), the user's scenario tag could be driving; and when the geographic coordinate is a gym or playground, the user's scenario tag could be exercising. Then, based on the user's characteristic information (such as data collected during registration, including but not limited to: account nickname, gender, age, interests, etc.) and current location, multiple audio ad ads to be recommended can be selected from a pre-created audio ad library. For example, a pre-created audio ad library can store multiple audio ads. Some of these ads are for non-targeted regions, while others are for targeted regions (e.g., a province or city). Based on the user's characteristics and current location, multiple ads matching these characteristics can be selected from the library. In other words, all selected ads for recommendation are for targeted regions. Then, based on the current location, contextual tags, and push data for each ad, a score is calculated. This score represents a comprehensive evaluation of the ad's location, context, and popularity. Finally, the highest-scoring ad can be played through an audio playback client running in the background. Thus, by first obtaining the user's current location and scene tags representing the user's current behavioral state, multiple audio ad clips to be recommended can be identified from a pre-created audio ad library based on the user's feature information and current location. Then, based on the current location, scene tags, and push data of each audio ad clip to be recommended, a score is calculated for each audio ad clip to be recommended. Finally, the audio ad clip with the highest score is played, ensuring that the advertising content inserted into the audio is highly matched with the user's interests, which can effectively improve the interaction rate of the ads and the user experience.

[0109] In some embodiments, the above-mentioned playback of target audio through a client running in the background can also be achieved by: sending an audio acquisition request to the server, wherein the audio acquisition request may carry account feature data of the client's currently logged-in account; receiving the target audio returned by the server and playing the target audio through the client running in the background, wherein the target audio may be obtained by the server by: obtaining a target object matching the account feature data from an object library and generating target text for recommending the target object (for example, target text for recommending the target object may be generated by a large language model); and converting the target text into target audio through a text-to-speech service.

[0110] For example, taking an audio playback client as an example, when the ad triggering time is met, the audio playback client can send an audio ad retrieval request to the server (e.g., an ad server). This request can carry account feature data of the currently logged-in account of the audio playback client (e.g., account 1 registered by user 1). This data includes, for example, the identity information user 1 filled in when registering account 1, and search and listening records during the use of the audio playback client. Upon receiving the audio ad retrieval request from the audio playback client, the ad server can first extract the account feature data of account 1 from the request. Then, it can filter from the object library (e.g., an item library) to find objects that match the account feature data of account 1 (e.g., item 1, i.e., items that match user 1's interests). Next, the ad server can generate ad copy for recommending item 1 by calling a large language model, and then convert the ad copy into the corresponding audio ad by calling a TTS service. Finally, the ad server can return the generated audio ad to the audio playback client, so that the audio playback client running in the background can play the audio ad returned by the server. In this way, since the ad content is highly matched with the user's interests, it can effectively reduce the user's resistance to the ad, thereby increasing the ad interaction rate and user engagement.

[0111] In other embodiments, following the examples above, after generating target text (e.g., advertising copy) for recommending target objects, the server can also call a text-to-image model to generate interactive information (e.g., advertising images) corresponding to the target object based on the target text. Then, when the client is detected to switch from background running state to foreground running state, the interactive information corresponding to the target object can be displayed in the interface of the client running in the foreground state. In this way, a dual stimulus of auditory and visual senses can be formed, thereby effectively improving the conversion rate of the advertisement.

[0112] In some embodiments, the above-mentioned playback of target audio through a client running in the background can also be achieved by: obtaining the voiceprint information of the target character (e.g., the protagonist) in the second multimedia data; and playing the target audio through a client running in the background according to the voiceprint information.

[0113] For example, taking the second multimedia data as crosstalk 1, a user can listen to crosstalk 1 in the background (for example, an audio playback client can play crosstalk 1 while running in the background). When crosstalk 1 reaches the preset advertising trigger time (for example, the playback time of crosstalk 1 exceeds 30 minutes), the playback of crosstalk 1 can be paused, and an audio ad can be played according to the voice of the actor in crosstalk 1. For example, the voiceprint information of the actor in crosstalk 1 can be obtained first, and then the audio ad can be played according to the voice corresponding to the voiceprint information. In this way, the integration and naturalness between the audio ad and the audio content currently being listened to by the user can be greatly improved, thereby effectively reducing the user's resistance to advertising.

[0114] It should be noted that the target audio mentioned above can also include an introduction to incentive information. In this way, setting up an incentive mechanism can effectively enhance users' enthusiasm for participation and interaction.

[0115] In step 102, interactive information corresponding to the target object is displayed on the interface of the client running in the foreground.

[0116] Here, interactive information can include virtual containers (such as virtual blind boxes, virtual gifts, virtual packages, etc.), virtual cards, virtual avatars (such as the brand's IP), etc.

[0117] It's important to note that the interactive information here can be interactive advertising, which refers to a form of advertising that combines user participation and feedback. When users interact with the advertising content (e.g., clicking, swiping, or participating in a game), personalized advertising feedback can be generated based on user behavior data. This two-way communication approach can improve user engagement and advertising effectiveness.

[0118] In some embodiments, the above-mentioned display of interactive information corresponding to the target object can be achieved in any of the following ways: displaying a virtual container including the object identifier of the target object (e.g., the brand's logo), wherein the virtual container may contain incentive information; displaying at least one virtual card including the object identifier of the target object, wherein different virtual cards may be associated with different incentive information; displaying a virtual image related to the target object, wherein the virtual image may be associated with incentive information.

[0119] For example, taking an audio playback client as an example, after switching the audio playback client from background to foreground, a virtual container containing the object identifier of the target object (such as the logo of brand 1) can be displayed on the playback interface of the audio playback client in the foreground state. The virtual container can encapsulate incentive information provided by brand 1. For example, virtual blind boxes, virtual gifts, or virtual packages with the logo of brand 1 can be displayed on the playback interface of the audio playback client in the foreground state. The virtual blind boxes, virtual gifts, or virtual packages can contain incentive information provided by brand 1 (such as coupons, VIP benefits, etc.). In other words, after users hear the audio ad in the background, they can see the relevant ad content on the foreground interface. In this way, a dual stimulation of auditory and visual senses can be formed, effectively improving the conversion rate of the ad and user engagement.

[0120] It should be noted that the interactive information displayed in the client interface when it is running in the foreground matches the target audio. For example, if the content of the target audio (i.e., the audio ad) is "XX (i.e., the name of brand 1) has prepared a big gift for you. Open the audio playback client and tap the blind box sponsored by XX", then the interactive information displayed in the client interface when it is running in the foreground is a virtual blind box with the logo of brand 1.

[0121] For example, taking an audio playback client as an example, after switching the audio playback client from background to foreground, multiple virtual cards including the object identifier of the target (such as the logo of brand 1) can be displayed on the interface of the audio playback client in the foreground state. For example, multiple virtual cards with the logo of brand 1 can be displayed on the playback interface of the audio playback client in the foreground state. The incentive information associated with different virtual cards can be different. That is to say, users can obtain the incentive information provided by brand 1 by drawing cards. In this way, after hearing the audio ad in the background, users can see the relevant ad content on the foreground interface of the client, thus forming a dual stimulation of hearing and vision, effectively improving the ad interaction rate and user engagement.

[0122] For example, taking an audio playback client as an example, after switching the audio playback client from background to foreground, a virtual avatar related to the target object (such as the IP avatar of brand 1) can be displayed on the playback interface of the audio playback client in the foreground state. At the same time, the IP avatar of brand 1 can be controlled to output guiding information or tasks. After the user performs the corresponding actions according to the guiding information output by the IP avatar of brand 1, or controls the IP avatar of brand 1 to complete the set tasks, they can obtain incentive information provided by brand 1. In this way, after the user hears the audio ad in the background, they can see the relevant ad content on the foreground interface of the client, thus forming a dual stimulation of hearing and vision, effectively improving the ad interaction rate and user engagement.

[0123] In some embodiments, see Figure 4 , Figure 4 This is a second flowchart illustrating the interactive information processing method provided in the embodiments of this application, as shown below. Figure 4 As shown, after execution Figure 3 After step 102 shown, you can also execute... Figure 4 Step 103 shown will combine Figure 4 The steps shown are explained.

[0124] In step 103, in response to an interactive action based on interactive information, an interactive result page is displayed.

[0125] Here, the interaction results page (such as an ad landing page) can display incentive information assigned to the currently logged-in account of the client. The incentive information can be coupons, VIP benefits, electronic red envelopes, etc.

[0126] In some embodiments, when the aforementioned interactive information is a virtual container encapsulating incentive information, step 103 can be implemented as follows: in response to an interactive operation based on the virtual container, an interactive result page is displayed, and first multimedia data (e.g., video or audio) for recommending target objects is played on the interactive result page; in response to the completion of the first multimedia data playback, incentive information assigned to the client's currently logged-in account is displayed on the interactive result page.

[0127] For example, taking a virtual container as a virtual blind box, the virtual blind box can contain coupons (i.e., incentive information) provided by Brand 1 (i.e., the target audience). After responding to the user's interactive operation based on the virtual blind box, the client (e.g., an audio playback client) can display an advertising landing page and play a video introducing Brand 1 on the advertising landing page. After watching the video, the user can see the coupons issued by Brand 1 on the advertising landing page. In this way, through the interactive incentive mechanism, the conversion rate of the advertisement can be effectively improved, while reducing the user's resistance to the advertisement, making the advertisement format more naturally integrated into the user's usage scenario.

[0128] In other embodiments, following the examples above, the above-mentioned response to interactive operations based on virtual containers and the display of an interactive result page can be implemented in the following way: In response to a trigger operation (e.g., a press operation) on the virtual container, during the period before the trigger operation is released, the virtual container is controlled to deform, wherein the degree of deformation may be positively correlated with the pressure parameter or duration of the trigger operation (e.g., the longer the user presses the virtual container, the greater the deformation of the virtual container; or, the greater the force of the user pressing the virtual container, the greater the deformation of the virtual container); In response to the degree of deformation of the virtual container being greater than a degree threshold, an animation of the virtual container breaking is played, and the interactive result page is displayed after the animation finishes playing.

[0129] For example, taking a flexible virtual blind box as an example, when the user presses the virtual blind box displayed on the interface, the deformation of the virtual blind box can be controlled according to the duration or force of the press. For instance, the longer the user presses the virtual blind box, the greater the deformation; or the greater the force of the press, the greater the deformation. When the deformation exceeds a threshold, an animation of the virtual blind box breaking can be played on the playback interface of the client running in the foreground (e.g., an audio playback client). After the animation finishes playing, an interaction result page (e.g., an advertising landing page) is displayed. In this way, users need to press the virtual blind box to obtain the incentive information encapsulated within, enriching the interaction methods and improving the user experience.

[0130] It should be noted that in practical applications, the virtual blind box mentioned above can also be broken with a single click. For example, when a user clicks on the virtual blind box, an animation of the virtual blind box breaking can be played directly, and an advertising landing page can be displayed after the animation finishes playing. This application embodiment does not specifically limit this.

[0131] In some embodiments, following the above examples, the above-mentioned response to interactive operations based on virtual containers and the display of an interactive result page can also be achieved in the following ways: in response to a motion-sensing operation (e.g., a shaking operation) on the terminal device where the client is located, and the operation parameters of the experience operation (e.g., the amplitude or duration of shaking) are greater than the operation parameter threshold, an animation of the virtual container breaking is played; in response to the completion of the animation playback, the interactive result page is displayed.

[0132] For example, taking a virtual container as a virtual blind box, users can also open the virtual blind box by shaking their terminal device. For instance, after switching a client (e.g., an audio playback client) from background to foreground, and displaying a virtual blind box with the brand's logo on the playback interface of the foreground audio playback client, the user can open the virtual blind box by shaking the terminal device (e.g., the user's smartphone). For example, cracks can be displayed on the virtual blind box as the user shakes the smartphone, and the number of cracks gradually increases with the duration of shaking. When the user shakes the smartphone for a certain duration (e.g., 2 minutes), an animation of the virtual blind box breaking can be played on the playback interface of the foreground audio playback client. After the animation finishes, an interactive result page (e.g., an advertising landing page) is displayed. In other words, users can also interact with the virtual blind box through motion controls, enriching the interaction methods and further enhancing the user experience.

[0133] In other embodiments, when the above-mentioned interactive information is at least one virtual card, step 103 can also be implemented in the following way: in response to a triggering operation (e.g., a click operation) for any virtual card, display an interaction result page, and display the incentive information assigned to the client's currently logged-in account on the interaction result page, wherein the incentive information may be the incentive information associated with the triggered virtual card, and the incentive information associated with different virtual cards may be different.

[0134] For example, taking an audio playback client as an example, after switching the audio playback client from background to foreground, various virtual cards bearing the logo of Brand 1 can be displayed on the playback interface of the foreground audio playback client. Different virtual cards can be associated with different incentive information. For instance, taking three virtual cards as examples, let's say virtual card 1, virtual card 2, and virtual card 3. Virtual card 1 could be associated with a coupon offered by Brand 1, virtual card 2 with VIP benefits offered by Brand 1, and virtual card 3 with an electronic red envelope offered by Brand 1. Users can select one virtual card from these. For example, if a user clicks on virtual card 1, an interaction results page (e.g., an ad landing page) can be displayed, showing the coupon offered by Brand 1 to the user. In this way, on the one hand, the interactive incentive mechanism can effectively reduce user resistance to advertising; on the other hand, it enriches the interactive methods of advertising and further enhances the user experience.

[0135] In some embodiments, when the above-mentioned interactive information is a virtual avatar, step 103 can also be implemented in the following way: controlling the virtual avatar to output guidance information, wherein the guidance information is used to instruct the user to perform an interactive action (e.g., blinking, nodding, clapping, etc.); in response to obtaining an interactive event corresponding to the interactive action (e.g., detecting the user blinking, nodding, or clapping, etc.), displaying an interaction result page, and displaying the incentive information assigned to the client's current login account on the interaction result page, wherein different interactive actions can be associated with different incentive information, that is, when the user performs different interactive actions, the incentive information obtained can be different.

[0136] For example, taking an audio playback client as an example, after switching the audio playback client from background to foreground, the IP image of Brand 1 can be displayed on the playback interface of the audio playback client in the foreground state. At the same time, Brand 1's IP image can be controlled to output guiding information, such as "There's a reward after blinking!" Then, when the user blinks, an interaction result page (such as an ad landing page) can be displayed, and the incentive information (such as coupons) provided by Brand 1 to the user can be displayed on the interaction result page. In this way, the interactive methods of the advertisement are enriched, and the user's participation experience is further enhanced.

[0137] In other embodiments, when the above-mentioned interactive information is a virtual avatar, step 103 can also be implemented in the following way: in response to controlling the virtual avatar to complete the set task, display the interaction result page, and display the incentive information assigned to the client's current login account on the interaction result page. The incentive information associated with different tasks can be different. That is, when the user controls the virtual avatar to complete different tasks, the incentive information obtained can be different.

[0138] For example, taking an audio playback client as an example, after switching the audio playback client from background to foreground, the IP image of Brand 1 can be displayed on the playback interface of the audio playback client in the foreground state. At the same time, task descriptions for multiple tasks can also be displayed on the playback interface of the audio playback client in the foreground state. After detecting that the user controls the IP image of Brand 1 to complete a task, an interaction result page can be displayed, showing the incentive information (such as a coupon) provided by Brand 1 to the user that is associated with the task currently completed by the user. In this way, the interactive methods of advertising are enriched, and the user's participation experience is further enhanced.

[0139] In some embodiments, after execution Figure 3 Following step 102, the following processing may also be performed: in response to no interactive operation based on interactive information being received within a set time period (e.g., 3 minutes), the interactive information is canceled from being displayed on the interface of the client running in the foreground; or, in response to the display time of the interactive information reaching a time threshold, the interactive result page is automatically displayed, and the incentive information assigned to the client's currently logged-in account is displayed on the interactive result page.

[0140] For example, taking an audio playback client as an example, after switching the audio playback client from background to foreground, interactive information corresponding to the target object (e.g., brand 1) can be displayed on the playback interface of the audio playback client in the foreground state. This could be a virtual blind box, virtual gift, virtual package, virtual card, etc., with brand 1's logo printed on it. If the user does not interact with the interactive information within a set time (e.g., 3 minutes), it means that the user is not interested in the currently displayed interactive information, and the display of the interactive information can be canceled. In this way, the display of interactive information for a long time can be avoided, thus avoiding interference to the user.

[0141] For example, taking an audio playback client as an example, after switching the audio playback client from background to foreground, interactive information corresponding to the target object (e.g., brand 1) can be displayed on the playback interface of the audio playback client in the foreground state. This interactive information could include virtual blind boxes, virtual gifts, virtual packages, virtual cards, etc., bearing brand 1's logo. If the display time of the interactive information on the playback interface exceeds a time threshold, an interactive results page (e.g., an advertising landing page) can be automatically displayed. The interactive results page will then display incentive information provided by brand 1 to the user (e.g., coupons, VIP benefits, etc.). In this way, users can automatically receive corresponding rewards when they are unable to interact with the interactive information (e.g., while eating), effectively improving the user experience.

[0142] In some embodiments, where the target audio is inserted into second multimedia data, after execution... Figure 3 After step 102 shown, the following processing can also be performed: in response to the completion of target audio playback, the interactive information is canceled from being displayed in the interface of the client running in the foreground, and the second multimedia data continues to be played through the client running in the foreground.

[0143] For example, taking the second multimedia data as crosstalk 1 and the target audio as an audio overlay advertisement, a user can listen to crosstalk 1 in the background (for example, an audio playback client can play crosstalk 1 while running in the background). When crosstalk 1 reaches the preset advertisement trigger time, it can be paused, and an audio overlay advertisement can be inserted. When the user is interested in the audio overlay advertisement, they can switch the audio playback client from the background to the foreground. In the playback interface of the audio playback client in the foreground, the advertisement content matching the audio overlay advertisement can be displayed. Subsequently, when it is detected that the audio overlay advertisement has finished playing, the advertisement content matching the audio overlay advertisement can be hidden in the playback interface of the audio playback client in the foreground, and crosstalk 1 can continue to play through the audio playback client in the foreground. In other words, when the audio overlay advertisement finishes playing, the advertisement content matching the audio overlay advertisement displayed in the foreground interface can be hidden, and the previously paused crosstalk 1 can continue to play. In this way, the audio content previously listened to by the user can be resumed, effectively improving the user experience.

[0144] In other embodiments, after execution Figure 3 After step 102 shown, the following processing can also be performed: acquiring user behavior data, wherein the behavior data may include: the triggering time of the state switching trigger operation (i.e., when the user opens the APP), the interaction status with the interactive information; and adjusting the playback timing of the target audio and the interactive information based on the behavior data.

[0145] For example, taking an audio playback client as an example, the audio playback client can send user behavior data (such as when the user opens the app, interaction status, etc.) to the advertising server. After receiving the user behavior data sent by the audio playback client, the advertising server can adjust the advertising strategy in real time based on the user behavior data. For example, it can adjust the playback timing of audio overlay ads within audio or video, or it can adjust the ad content displayed on the front-end interface based on the user behavior data. In this way, the subsequent ads can be more in line with the user's needs.

[0146] The following will continue to combine Figure 5 The interactive information processing method provided in the embodiments of this application will be described.

[0147] For example, see Figure 5 , Figure 5 This is a schematic diagram of the third process of the interactive information processing method provided in the embodiments of this application, which will be combined with Figure 5 The steps shown are explained.

[0148] In step 201, multimedia data is played.

[0149] Here, multimedia data can include video and audio. That is, embodiments of this application can insert audio overlay advertisements (i.e., target audio) during video playback, or audio overlay advertisements can be inserted during audio (e.g., crosstalk or audiobook) playback. Furthermore, the aforementioned process of playing multimedia data can be performed while the user's terminal device (e.g., smartphone) is in standby mode (e.g., locked screen), or it can be performed while the client is running in the background. For example, a user can place the client currently used for playing multimedia data (e.g., audio playback client or video playback client) in the background and browse the content displayed on the interface of other clients.

[0150] In step 202, in response to the audio playback conditions being met, the playback of multimedia data is paused, the target audio is played, and the target object recommended by the target audio is displayed.

[0151] Here, target audio can refer to audio ad overlays used for brand promotion or product advertising. The target audience recommended by the target audio can be a brand or a specific product.

[0152] In some embodiments, the above-mentioned display of the target object recommended by the target audio can be achieved by: displaying a notification message corresponding to the target audio in the system notification bar; and, in response to a triggering operation on the notification message, displaying the playback interface of the client for playing multimedia data, and displaying the target object recommended by the target audio in the playback interface.

[0153] For example, when a user's terminal device (e.g., a smartphone) is in standby mode, and the target audio (e.g., an audio ad promoting brand 1) is playing, the user's smartphone can access system permissions to provide a corresponding notification through the system notification bar. For instance, it can display a notification message corresponding to the currently playing audio ad in the system notification bar. Upon receiving a click action from the user on this notification message, a client for playing multimedia data (e.g., an audio playback client or a video playback client) can be opened. For example, the playback interface of the multimedia data playback client can be displayed on the screen, and the logo of brand 1 recommended by the currently playing audio ad can be displayed in the playback interface.

[0154] In some embodiments, the above-mentioned response to meeting audio playback conditions (e.g., advertising triggering timing), pausing the playback of multimedia data and playing the target audio, can be achieved in any of the following ways: pausing the playback of multimedia data and playing the target audio in response to the playback duration of multimedia data exceeding a duration threshold; pausing the playback of multimedia data and playing the target audio in response to the playback progress of multimedia data reaching a set time point; or pausing the playback of multimedia data and playing the target audio in response to the appearance of a set scene in the multimedia data.

[0155] In some embodiments, the above-mentioned playback of target audio can be achieved by: obtaining account feature data of the currently logged-in account of the client used to play multimedia data; determining the similarity between the audio feature data and the account feature data of each audio in a pre-created audio library, wherein different audios are used to recommend different objects; and selecting the audio with the highest similarity as the target audio and playing the target audio.

[0156] In some embodiments, the above-mentioned playback of target audio can be achieved by: sending an audio acquisition request to the server so that the server generates target audio that matches the content of the multimedia data; receiving the target audio returned by the server and playing the target audio.

[0157] In some embodiments, the above-mentioned playback of target audio can be achieved by: obtaining the status data of the terminal device where the client used to play multimedia data is located, and determining the current listening scenario based on the status data; filtering out audio that matches the current listening scenario from a pre-created audio library, using the matched audio as the target audio, and playing the target audio.

[0158] In some embodiments, the above-mentioned playback of target audio can be achieved by: sending an audio acquisition request to the server, wherein the audio acquisition request carries account feature data of the currently logged-in account of the client for playing multimedia data; receiving the target audio returned by the server and playing the target audio, wherein the target audio is obtained by the server by: obtaining a target object matching the account feature data from an object library and generating target text for recommending the target object; and converting the target text into target audio through a text-to-speech service.

[0159] In some embodiments, the above-mentioned playback of target audio can be achieved by: obtaining the voiceprint information of the target character in the multimedia data; and playing the target audio according to the voiceprint information.

[0160] In step 203, in response to a trigger operation on the target object, interactive information corresponding to the target object is displayed.

[0161] In some embodiments, the target object mentioned above may be directly displayed in the system notification bar. In this case, step 203 can be implemented as follows: in response to a trigger operation on the target object displayed in the system notification bar, the playback interface of the client for playing multimedia data is displayed, and interactive information corresponding to the target object is displayed in the playback interface.

[0162] For example, taking an audio playback client as an example, when playing target audio (such as an audio ad for promoting Brand 1), Brand 1's logo can be directly displayed in the system notification bar. Upon receiving a user's click on Brand 1's logo displayed in the system notification bar, the audio playback client's playback interface can be opened, displaying interactive information corresponding to the target (such as virtual blind boxes, virtual gifts, virtual packages, or virtual cards provided by Brand 1) for user interaction. In other words, after clicking on Brand 1's logo displayed in the system notification bar, the user can be directly redirected to the corresponding page for interaction.

[0163] In some embodiments, the above-mentioned display of interactive information corresponding to the target object can be achieved in any of the following ways: displaying a virtual container including the object identifier of the target object, wherein the virtual container encapsulates incentive information; displaying at least one virtual card including the object identifier of the target object, wherein different virtual cards are associated with different incentive information; displaying a virtual image related to the target object, wherein the virtual image is associated with incentive information.

[0164] In some embodiments, after displaying interactive information corresponding to the target object, the following processing may also be performed: in response to an interactive operation based on the interactive information, displaying an interactive result page, wherein the interactive result page includes incentive information assigned to a first account, the first account being the currently logged-in account of a client used to play multimedia data.

[0165] In some embodiments, when the interactive information is a virtual container encapsulating incentive information, the above-mentioned response to the interactive operation based on the interactive information and the display of the interactive result page can be achieved in the following ways: in response to the interactive operation based on the virtual container, the interactive result page is displayed, and the first multimedia data for recommending the target object is played on the interactive result page; in response to the completion of the playback of the first multimedia data, the incentive information assigned to the client's currently logged-in account is displayed on the interactive result page.

[0166] In some embodiments, the above-mentioned response to the interactive operation based on the virtual container and the display of the interactive result page can be achieved in the following ways: in response to the trigger operation on the virtual container, during the period before the trigger operation is released, the virtual container is controlled to deform, wherein the degree of deformation is positively correlated with the pressure parameter or duration of the trigger operation; in response to the degree of deformation of the virtual container being greater than the degree threshold, an animation of the virtual container breaking is played, and the interactive result page is displayed after the animation finishes playing.

[0167] In some embodiments, the above-mentioned response to interactive operations based on virtual containers and display of the interactive result page can also be achieved in the following ways: in response to a motion-sensing operation on the terminal device where the client is located, and the operation parameter of the motion-sensing operation is greater than the operation parameter threshold, an animation of the virtual container breaking is played; in response to the completion of the animation playback, the interactive result page is displayed.

[0168] In some embodiments, when the interaction information is at least one virtual card, the above-mentioned response to the interaction operation based on the interaction information and display of the interaction result page can be achieved in the following way: in response to the trigger operation for any virtual card, the interaction result page is displayed, and the incentive information assigned to the client's currently logged-in account is displayed on the interaction result page, wherein the incentive information is the incentive information associated with the triggered virtual card, and the incentive information associated with different virtual cards is different.

[0169] In some embodiments, when the interactive information is a virtual avatar, the above-mentioned response to the interactive operation based on the interactive information and the display of the interactive result page can be achieved in any of the following ways: controlling the virtual avatar to output guidance information, wherein the guidance information is used to instruct the implementation of the interactive action; in response to obtaining the interactive event corresponding to the interactive action, displaying the interactive result page and displaying the incentive information assigned to the client's current login account on the interactive result page, wherein different interactive actions are associated with different incentive information; in response to controlling the virtual avatar to complete the set task, displaying the interactive result page and displaying the incentive information assigned to the client's current login account on the interactive result page, wherein different tasks are associated with different incentive information.

[0170] In some embodiments, after displaying the interactive information corresponding to the target object, the following processing may also be performed: in response to no interactive operation based on the interactive information being received within a set time period, the interactive information is canceled from being displayed in the interface of the client running in the foreground; or, in response to the display time of the interactive information reaching a time threshold, the interactive result page is automatically displayed, and the incentive information assigned to the client's currently logged-in account is displayed on the interactive result page.

[0171] In the above scheme, after displaying the interactive information corresponding to the target object, the following processing can also be performed: obtaining user behavior data, wherein the behavior data includes: the triggering time for the target object, the interaction status for the interactive information (i.e., whether the user interacts with the interactive information, and the duration of the interaction, etc.); and adjusting the playback time of the target audio and the interactive information based on the behavior data.

[0172] The following will describe an exemplary application of the embodiments of this application in a real-world application scenario.

[0173] Users of audio playback clients spend relatively little time focusing on the interface. Industry research shows an average daily usage time of about 20 minutes, averaging 3 minutes per session, and 7 times a day. The typical usage pattern is for the client to run in the background while the user listens. Furthermore, in the solutions provided by related technologies, users cannot see the ad content when they hear audio overlay ads while listening to audio content in the background, resulting in a lack of effective interaction and thus no ad conversion. In other words, in the solutions provided by these technologies, audio overlay ads are difficult to generate ad clicks and cannot support bidding-based ads (where the ad objective is to assess ad conversion effectiveness), resulting in limited effectiveness for audio overlay ads.

[0174] Therefore, this application provides an interactive information processing method that combines the response timing of audio ad overlays with the instantaneous return to the app (i.e., switching the app from background to foreground running state) to achieve coordinated listening and viewing behaviors. For example, when a user hears an audio ad overlay in the background, opening the app can display a limited-time blind box ad on the foreground interface, thus creating a dual stimulation of auditory and visual senses. Furthermore, this application can also incorporate AI technology. For instance, AI technology can be used to customize and tune the voice for audiobooks, or AI technology can be used to write advertising copy (i.e., scripts). For example, for a user listening to Guo XX's crosstalk, the following audio ad overlay can be played through Guo XX's voice:

[0175] Script 1: Alright everyone, young and old, listen up! XX (the advertiser, such as a brand) has prepared a great gift for you—just open XXXX (the name of the audio playback client), tap on the blind box ad sponsored by XX, and wow! 30 minutes of VIP access for free! It's like unwrapping a package; with just a flick of your finger, a wonderful show will immediately unfold in your ears!

[0176] Script 2: Guess what? Quickly open XXXX (the name of the audio playback client)! This blind box doesn't just contain VIP, it might also hide extra discounts from XX (the advertiser, such as a brand), or a chance to snag great deals for just 1 cent! (Flicks fan) Alright, without further ado, let's open the blind box and let XX take you to continue listening to VIP premium content.

[0177] It should be noted that the technical solutions provided in this application are applicable not only to audiobook apps and audio apps, but also to short video apps (such as video listening functions).

[0178] In some embodiments, when a user hears an audio ad in the background, opening the app (i.e., switching the app from the background to the foreground) will display ad content matching the audio ad on the app's foreground interface (e.g., virtual blind boxes, virtual boxes, virtual gifts, virtual packages, virtual coupons, etc.). Furthermore, if the user does not interact with the ad content displayed on the foreground interface, this ad content can be automatically destroyed after a set time (e.g., 3 minutes); alternatively, the user can be directly redirected to an ad landing page, which is not specifically limited in this embodiment. Additionally, the physical form of the audio ad and the ad content displayed on the foreground interface can be highly relevant to the audio content the user is currently listening to. For example, AI-generated images and audio can be combined with the user's current audio content and the brand to be promoted to create an ad, allowing AI to generate ad formats and creative materials most suitable for the current moment and creative concept, thus achieving personalized creative experiences.

[0179] In other words, the technical solution provided in this application can, on the one hand, awaken users and reactivate applications (i.e., encourage users to open the app) and improve advertising monetization efficiency; on the other hand, it solves the problem that the monetization efficiency of audio ads is affected by insufficient ad exposure and interaction based solely on listening in related technologies, forming a dual stimulation of auditory and visual senses, thereby improving ad interaction rates and user engagement. Furthermore, this application embodiment can also combine AI technology to enable personalized audio ad overlays, greatly improving the integration and naturalness between ads and the content the user is currently listening to, while also effectively improving ad production efficiency.

[0180] The interactive information processing method provided in the embodiments of this application will be described in detail below.

[0181] In some embodiments, see Figure 6 , Figure 6 This is a schematic diagram illustrating an application scenario of the interactive information processing method provided in the embodiments of this application, such as... Figure 6As shown, users can listen to audio (such as an audiobook) in the background. For example, when the audio playback client is running in the background, its interface can be minimized to a small widget 601, meaning the background audio playback client only occupies a small portion of the screen. Then, when the audio playback reaches a time set by the developer, the audio can be paused, and an audio overlay advertisement can be inserted. When the user shows interest in the audio overlay advertisement, the audio playback client can be switched from the background to the foreground. For example, upon receiving a click on widget 601, the audio playback client's playback interface 602 can be displayed in full screen, along with interactive content 603 matching the audio overlay advertisement (i.e., the aforementioned interactive information, such as a virtual blind box), awaiting user interaction. If the user does not interact with the interactive content 603 within 3 minutes, the interactive content 603 can be automatically hidden from the playback interface 602.

[0182] In other embodiments, see Figure 7 , Figure 7 This is a schematic diagram of the architecture of the interactive information processing system provided in the embodiments of this application, such as... Figure 7 As shown, the interactive information processing system provided in this application embodiment includes: a user terminal (e.g., a smartphone, tablet, etc.), an audio content server (for providing audio content), an advertising delivery server (for managing, distributing, and processing interactive logic for advertising content), and an AI content generation server (for generating creative content such as AI-generated text, text-generated images, and text-generated audio).

[0183] For example, when a user terminal receives an audio playback operation triggered by the user (e.g., a click on the play button), it can request audio content from the audio content server. Upon receiving the request from the user terminal, the audio content server can return an audio stream to the user terminal. The user can listen to the audio content in the background; for example, the user can switch the app from foreground to background operation and control the app to continuously play audio content in the background. When the audio content reaches a preset advertising trigger point (e.g., the audio content playback duration has exceeded 30 minutes), the system initiates the advertising-related process. Specifically, the audio content server can send a request to the ad delivery server to prepare ad content. After receiving the request from the audio content server, the ad delivery server can send a request to the AI ​​content generation server, for example, requesting the AI ​​content generation server to generate customized ad creatives. After receiving the request from the ad delivery server, the AI ​​content generation server can generate customized ad creatives (e.g., including ad text, images, etc.) related to the user's interests based on the content the user is listening to, and can also convert the ad text into audio-visual ads through TTS services. The AI ​​content generation server can return the generated audio ad to the ad delivery server. Upon receiving the audio ad from the AI ​​content generation server, the ad delivery server can push the audio ad to the user's terminal. After hearing the audio ad, the user can open the app (switching the app from background to foreground) and interact with it. The app interface can display interactive content related to the audio ad (i.e., ad content, such as virtual blind boxes, virtual gifts, virtual coupons, virtual packages, etc.). The ad interaction module can detect whether the user interacts with the interactive content within a set time (e.g., 3 minutes). If the user interacts with the interactive content, the ad interaction process is completed, and the user receives an incentive reward; if the user does not interact with the interactive content, the interactive content is automatically destroyed and no longer displayed.

[0184] The following will continue to combine Figure 8 The interactive information processing method provided in the embodiments of this application will be described in detail.

[0185] For example, see Figure 8 , Figure 8 This is a schematic diagram of the fourth process of the interactive information processing method provided in the embodiments of this application, which will be combined with Figure 8 The steps shown are explained.

[0186] In step 301, the user terminal requests audio content from the audio content server.

[0187] In step 302, the audio content server returns an audio stream to the user terminal.

[0188] In step 303, the audio content server requests advertising materials from the advertising delivery server.

[0189] In some embodiments, the timing of ad triggering can be intelligently determined based on the audio content listened to by the user, the duration of the audio, the user's characteristic data, and historical behavior. For example, when a user listens to audio for too long (e.g., more than 30 minutes), the user may experience fatigue. At this time, light and interesting audio ad overlays can be inserted into the audio to effectively alleviate the user's fatigue.

[0190] In step 304, the ad delivery server requests the AI ​​content generation server to generate a customized ad.

[0191] In step 305, the AI ​​content generation server returns the generated advertising materials to the advertising delivery server.

[0192] In some embodiments, the AI ​​content generation server can utilize AI models (such as large language models, stable diffusion models, and TTS speech synthesis models) to generate customized audio-visual ad content, including audio, images, text, and ad content interfaces. This can be achieved using interfaces provided by large model services or by training the model independently. Subsequently, the AI ​​content generation server can push the generated ad materials to user terminal devices. The user terminal can also collect user interaction data, enabling the ad delivery server to analyze this data and optimize subsequent ad delivery strategies.

[0193] In step 306, the advertising delivery server pushes audio sticker advertisements to the user terminal.

[0194] In step 307, the user terminal switches the APP from background running state to foreground running state.

[0195] In some embodiments, the user terminal can control the APP to continuously play the audio stream in the background and detect the triggering timing of the audio overlay advertisement. After the audio overlay advertisement is played, if the user is interested in the currently playing audio overlay advertisement, they can open the APP (i.e., switch the APP from background running state to foreground running state). In the interface of the APP running in the foreground state, advertising content matching the audio overlay advertisement can be displayed, such as virtual blind boxes, virtual gifts, virtual coupons, and virtual packages. After the advertising content appears, a countdown can be started (e.g., 3 minutes). If the user does not interact with the advertising content before the countdown ends, the advertising content can be automatically destroyed after the countdown ends.

[0196] In step 308, the user terminal sends interactive data back to the advertising server.

[0197] In some embodiments, the user terminal can transmit user behavior data (i.e., interaction data, such as when the user opens the app, the interaction with advertising content, etc.) back to the advertising delivery server in real time. After receiving the user behavior data uploaded by the user terminal, the advertising delivery server can adjust the advertising delivery strategy in real time based on the user behavior data. For example, it can adjust the playback timing of audio overlay ads in audio or video based on the user behavior data, or it can adjust the advertising content displayed on the front-end interface to better meet the user's needs.

[0198] In summary, the interactive information processing method provided in this application has the following beneficial effects:

[0199] 1) This application embodiment combines audio-visual ads with the user's app opening behavior to accurately capture user interaction opportunities, thereby significantly improving the ad interaction rate. Furthermore, compared to related technologies where audio-visual ads rely solely on user hearing and have a low interaction rate, this application embodiment combines audio-visual ads with visual interaction, creating a dual stimulus of hearing and sight, thereby significantly improving the interaction rate.

[0200] 2) This application embodiment utilizes AI technology to generate personalized advertising materials based on the content currently being listened to by the user, such as sound, images, and interfaces, so that the advertising content is highly relevant to the user's interests, thereby enhancing the attractiveness of the advertisement.

[0201] 3) This application embodiment uses interactive incentives to enable users to receive rewards (such as blind box rewards, VIP benefits, etc.) after interaction, which further improves the advertising conversion rate.

[0202] 4) The embodiments of this application, through intelligent advertising triggering timing and creative design, make the advertising format more naturally integrated into the user's usage scenario, thereby reducing the user's resistance to advertising.

[0203] The following description continues to illustrate the exemplary structure of the interactive information processing device 555 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 2 As shown, the software modules stored in the interactive information processing device 555 in the memory 550 may include: a switching module 5551 and a display module 5552.

[0204] The switching module 5551 is used to switch the client from the background running state to the foreground running state in response to the state switching trigger operation of the client when the target audio is played in the background running state. The target audio is used to recommend target objects. The display module 5552 is used to display interactive information corresponding to the target object in the interface of the client running in the foreground running state.

[0205] In some embodiments, the display module 5552 is further configured to perform one of the following processes: displaying a virtual container including an object identifier of the target object, wherein the virtual container encapsulates incentive information; displaying at least one virtual card including an object identifier of the target object, wherein different virtual cards are associated with different incentive information; and displaying a virtual image associated with the target object, wherein the virtual image is associated with incentive information.

[0206] In some embodiments, after displaying interactive information corresponding to the target object, the display module 5552 is further configured to display an interactive result page in response to an interactive operation based on the interactive information, wherein the interactive result page includes incentive information assigned to the client's currently logged-in account.

[0207] In some embodiments, when the interactive information is a virtual container encapsulating incentive information, the display module 5552 is further configured to, in response to the interactive operation based on the virtual container, display an interactive result page and play first multimedia data for recommending target objects on the interactive result page; and, in response to the completion of the playback of the first multimedia data, display incentive information assigned to the client's currently logged-in account on the interactive result page.

[0208] In some embodiments, the display module 5552 is further configured to, in response to a trigger operation on the virtual container, control the virtual container to deform during the period before the trigger operation is released, wherein the degree of deformation is positively correlated with the pressure parameter or duration of the trigger operation; and, in response to the degree of deformation of the virtual container being greater than a degree threshold, play an animation of the virtual container breaking, and display an interactive result page after the animation finishes playing.

[0209] In some embodiments, the display module 5552 is further configured to, in response to a motion-sensing operation on the terminal device where the client is located, and the operation parameter of the motion-sensing operation is greater than the operation parameter threshold, play an animation of a virtual container breaking; and in response to the completion of the animation playback, display an interactive result page.

[0210] In some embodiments, when the interaction information is at least one virtual card, the display module 5552 is further configured to display an interaction result page in response to a triggering operation for any virtual card, and display incentive information assigned to the client's currently logged-in account on the interaction result page, wherein the incentive information is the incentive information associated with the triggered virtual card, and different virtual cards are associated with different incentive information.

[0211] In some embodiments, when the interactive information is a virtual avatar, the display module 5552 is further configured to control the virtual avatar to output guidance information, wherein the guidance information is used to instruct the implementation of an interactive action; in response to obtaining an interactive event corresponding to the interactive action, an interactive result page is displayed, and incentive information assigned to the client's current login account is displayed on the interactive result page, wherein different interactive actions are associated with different incentive information.

[0212] In some embodiments, when the interactive information is a virtual avatar, the display module 5552 is further configured to display an interactive result page in response to controlling the virtual avatar to complete a set task, and to display incentive information assigned to the client's currently logged-in account on the interactive result page, wherein different tasks are associated with different incentive information.

[0213] In some embodiments, after displaying the interactive information corresponding to the target object, the display module 5552 is further configured to, in response to the absence of an interactive operation based on the interactive information within a set time period, cancel the display of the interactive information in the interface of the client running in the foreground; or, in response to the display time of the interactive information reaching a time threshold, automatically display the interactive result page and display the incentive information allocated to the client's currently logged-in account in the interactive result page.

[0214] In some embodiments, the interactive information processing device 555 further includes a playback module 5553, configured to, when the switching module 5551 is playing target audio in a client running in the background, before switching the client from a background running state to a foreground running state in response to a state switching trigger operation for the client; and to, in response to meeting the audio playback conditions, to pause the playback of the second multimedia data and play the target audio in the client running in the background.

[0215] In some embodiments, the playback module 5553 is further configured to perform one of the following processes: in response to the playback duration of the second multimedia data exceeding a duration threshold, pausing the playback of the second multimedia data and playing the target audio through a client running in the background; in response to the playback progress of the second multimedia data reaching a set time point, pausing the playback of the second multimedia data and playing the target audio through a client running in the background; in response to the occurrence of a set scenario in the second multimedia data, pausing the playback of the second multimedia data and playing the target audio through a client running in the background.

[0216] In some embodiments, the interactive information processing device 555 further includes an acquisition module 5554 and a determination module 5555, wherein the acquisition module 5554 is used to acquire account feature data of the currently logged-in account of the client; the determination module 5555 is used to determine the similarity between the audio feature data and the account feature data of each audio in a pre-created audio library, wherein different audios are used to recommend different objects; the audio with the highest similarity is selected as the target audio; and the playback module 5553 is further used to play the target audio through the client running in the background.

[0217] In some embodiments, the interactive information processing device 555 further includes a sending module 5556 and a receiving module 5557, wherein the sending module 5556 is used to send an audio acquisition request to the server so that the server generates target audio that matches the content of the second multimedia data; the receiving module 5557 is used to receive the target audio returned by the server; and the playback module 5553 is further used to play the target audio through a client running in the background.

[0218] In some embodiments, the acquisition module 5554 is further configured to acquire status data of the terminal device where the client is located; the determination module 5555 is further configured to determine the current listening scenario based on the status data; the interactive information processing device 555 further includes a filtering module 5558, configured to filter out audio that matches the current listening scenario from a pre-created audio library; the determination module 5555 is further configured to use the matched audio as the target audio; and the playback module 5553 is further configured to play the target audio through the client running in the background.

[0219] In some embodiments, the state data includes geographic location and movement speed; the determination module 5555 is further configured to input the geographic location and movement speed into a pre-trained scene recognition model to obtain the current listening scene; the filtering module 5558 is further configured to query a pre-created audio library based on the scene identifier of the current listening scene to obtain audio that matches the current listening scene, wherein the audio library stores multiple scene identifiers and multiple audios corresponding to the multiple scene identifiers in key-value pairs.

[0220] In some embodiments, the sending module 5556 is further configured to send an audio acquisition request to the server, wherein the audio acquisition request carries account feature data of the client's currently logged-in account; the receiving module 5557 is further configured to receive the target audio returned by the server; and the playing module 5553 is further configured to play the target audio through the client running in the background, wherein the target audio is obtained by the server by: obtaining a target object matching the account feature data from an object library and generating target text for recommending the target object; and converting the target text into target audio through a text-to-speech service.

[0221] In some embodiments, the acquisition module 5554 is further configured to acquire the voiceprint information of the target character in the second multimedia data; the playback module 5553 is further configured to play the target audio through a client running in the background according to the sound that matches the voiceprint information.

[0222] In some embodiments, the display module 5552 is further configured to display a playback control in the interface of the client running in the foreground; the playback module 5553 is further configured to play second multimedia data through the client running in the foreground in response to a trigger operation on the playback control; the switching module 5551 is further configured to switch the client from the foreground running state to the background running state in response to a state switching trigger operation on the client; and the playback module 5553 is further configured to continue playing the second multimedia data through the client running in the background.

[0223] In some embodiments, after displaying interactive information corresponding to the target object in the interface of the client running in the foreground state, the display module 5552 is further configured to cancel the display of interactive information in response to the completion of target audio playback; the playback module 5553 is further configured to continue playing the second multimedia data through the client running in the foreground state.

[0224] In some embodiments, after the display module 5552 displays the interactive information corresponding to the target object, the acquisition module 5554 is further configured to acquire behavioral data, wherein the behavioral data includes: the triggering timing of the state switching triggering operation and the interaction status in response to the interactive information; the interactive information processing device 555 further includes an adjustment module 5559, configured to adjust the playback timing of the target audio and the interactive information based on the behavioral data.

[0225] The following description continues to illustrate the exemplary structure of the interactive information processing device 555 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 2 As shown, the software modules stored in the interactive information processing device 555 in the memory 550 may include: a playback module 5553 and a display module 5552.

[0226] The playback module 5553 is used to play multimedia data; the playback module 5553 is also used to pause the playback of multimedia data and play the target audio in response to the fulfillment of audio playback conditions; the display module 5552 is used to display the target object recommended by the target audio; the display module 5552 is also used to display interactive information corresponding to the target object in response to a trigger operation on the target object.

[0227] In some embodiments, the display module 5552 is further configured to display a notification message corresponding to the target audio in the system notification bar; in response to a triggering operation for the notification message, display a playback interface of the client for playing multimedia data, and display the target object recommended by the target audio in the playback interface.

[0228] In some embodiments, the target object is displayed in the system notification bar. The display module 5552 is also configured to, in response to a trigger operation on the target object displayed in the system notification bar, display a playback interface of the client for playing multimedia data, and display interactive information corresponding to the target object in the playback interface.

[0229] In some embodiments, the display module 5552 is further configured to perform one of the following processes: displaying a virtual container including an object identifier of the target object, wherein the virtual container encapsulates incentive information; displaying at least one virtual card including an object identifier of the target object, wherein different virtual cards are associated with different incentive information; and displaying a virtual image associated with the target object, wherein the virtual image is associated with incentive information.

[0230] In some embodiments, after displaying interactive information corresponding to the target object, the display module 5552 is further configured to display an interactive result page in response to an interactive operation based on the interactive information. The interactive result page includes incentive information assigned to a first account, which is the currently logged-in account of the client used to play multimedia data.

[0231] In some embodiments, when the interactive information is a virtual container encapsulating incentive information, the display module 5552 is further configured to, in response to the interactive operation based on the virtual container, display an interactive result page and play first multimedia data for recommending target objects on the interactive result page; and, in response to the completion of the playback of the first multimedia data, display incentive information assigned to the client's currently logged-in account on the interactive result page.

[0232] In some embodiments, the display module 5552 is further configured to, in response to a trigger operation on the virtual container, control the virtual container to deform during the period before the trigger operation is released, wherein the degree of deformation is positively correlated with the pressure parameter or duration of the trigger operation; and, in response to the degree of deformation of the virtual container being greater than a degree threshold, play an animation of the virtual container breaking, and display an interactive result page after the animation finishes playing.

[0233] In some embodiments, the display module 5552 is further configured to, in response to a motion-sensing operation on the terminal device where the client is located, and the operation parameter of the motion-sensing operation is greater than the operation parameter threshold, play an animation of a virtual container breaking; and in response to the completion of the animation playback, display an interactive result page.

[0234] In some embodiments, when the interaction information is at least one virtual card, the display module 5552 is further configured to display an interaction result page in response to a triggering operation for any virtual card, and display incentive information assigned to the client's currently logged-in account on the interaction result page, wherein the incentive information is the incentive information associated with the triggered virtual card, and different virtual cards are associated with different incentive information.

[0235] In some embodiments, when the interactive information is a virtual avatar, the display module 5552 is further configured to perform one of the following processes: control the virtual avatar to output guidance information, wherein the guidance information is used to instruct the implementation of an interactive action; in response to obtaining an interactive event corresponding to the interactive action, display an interactive result page and display incentive information assigned to the client's current login account on the interactive result page, wherein different interactive actions are associated with different incentive information; in response to controlling the virtual avatar to complete a set task, display an interactive result page and display incentive information assigned to the client's current login account on the interactive result page, wherein different tasks are associated with different incentive information.

[0236] In some embodiments, after displaying the interactive information corresponding to the target object, the display module 5552 is further configured to, in response to the absence of an interactive operation based on the interactive information within a set time period, cancel the display of the interactive information in the interface of the client running in the foreground; or, in response to the display time of the interactive information reaching a time threshold, automatically display the interactive result page and display the incentive information allocated to the client's currently logged-in account in the interactive result page.

[0237] In some embodiments, the playback module 5553 is further configured to perform one of the following processes: in response to the playback duration of the multimedia data exceeding a duration threshold, pausing the playback of the multimedia data and playing the target audio; in response to the playback progress of the multimedia data reaching a set time point, pausing the playback of the multimedia data and playing the target audio; in response to the occurrence of a set scene in the multimedia data, pausing the playback of the multimedia data and playing the target audio.

[0238] In some embodiments, the interactive information processing device 555 further includes an acquisition module 5554 and a determination module 5555, wherein the acquisition module 5554 is used to acquire account feature data of the currently logged-in account of the client used to play multimedia data; the determination module 5555 is used to determine the similarity between the audio feature data and the account feature data for each audio in a pre-created audio library, wherein different audios are used to recommend different objects; the playback module 5553 is further used to select the audio with the highest similarity as the target audio and play the target audio.

[0239] In some embodiments, the interactive information processing device 555 further includes a sending module 5556 and a receiving module 5557, wherein the sending module 5556 is used to send an audio acquisition request to the server so that the server generates target audio that matches the content of the multimedia data; the receiving module 5557 is used to receive the target audio returned by the server; and the playback module 5553 is used to play the target audio.

[0240] In some embodiments, the acquisition module 5554 is further configured to acquire status data of the terminal device where the client for playing multimedia data is located; the determination module 5555 is further configured to determine the current listening scenario based on the status data; the interactive information processing device 555 further includes a filtering module 5558, configured to filter out audio that matches the current listening scenario from a pre-created audio library; and the playback module 5553 is further configured to use the matched audio as the target audio and play the target audio.

[0241] In some embodiments, the sending module 5556 is further configured to send an audio acquisition request to the server, wherein the audio acquisition request carries account feature data of the currently logged-in account of the client for playing multimedia data; the receiving module 5557 is further configured to receive target audio returned by the server; and the playing module 5553 is further configured to play the target audio, wherein the target audio is obtained by the server by: obtaining a target object matching the account feature data from an object library and generating target text for recommending the target object; and converting the target text into target audio through a text-to-speech service.

[0242] In some embodiments, the acquisition module 5554 is further configured to acquire the voiceprint information of the target character in the multimedia data; the playback module 5553 is further configured to play the target audio according to the sound matching the voiceprint information.

[0243] In some embodiments, the acquisition module 5554 is further configured to acquire behavioral data, wherein the behavioral data includes: the triggering timing for the target object and the interaction status for the interactive information; the interactive information processing device 555 further includes an adjustment module 5559, configured to adjust the playback timing of the target audio and the interactive information based on the behavioral data.

[0244] It should be noted that the description of the apparatus in this application embodiment is similar to the description of the method embodiment above, and has similar beneficial effects as the method embodiment, therefore it will not be repeated. For any technical details not covered in the interactive information processing apparatus provided in this application embodiment, please refer to... Figure 3 , Figure 4 ,or Figure 5 The meaning is understood in accordance with the description of any of the accompanying drawings.

[0245] This application provides a computer program product comprising a computer program or computer-executable instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, causing the computer device to perform the interactive information processing method described above in this application.

[0246] This application provides a computer-readable storage medium storing computer-executable instructions. When these computer-executable instructions are executed by a processor, they cause the processor to execute the interactive information processing method provided in this application. For example, ... Figure 3 , Figure 4 ,or Figure 5 The interactive information processing method is shown.

[0247] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0248] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0249] As an example, executable instructions can be deployed to execute on a single electronic device, or on multiple electronic devices located in one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0250] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. An interactive information processing method, characterized in that, The method includes: When a target audio is played on a client running in the background, in response to a state switching trigger operation for the client, the client is switched from the background running state to the foreground running state, wherein the target audio is used to recommend target objects, and In the client interface, which is in the foreground running state, interactive information corresponding to the target object is displayed.

2. The method according to claim 1, characterized in that, The display of interactive information corresponding to the target object includes: Perform one of the following processes: Display a virtual container that includes the object identifier of the target object, wherein the virtual container encapsulates incentive information; Display at least one virtual card including the object identifier of the target object, wherein different virtual cards are associated with different incentive information; Display a virtual avatar related to the target object, wherein the virtual avatar is associated with incentive information.

3. The method according to claim 1 or 2, characterized in that, After displaying the interactive information corresponding to the target object, the method further includes: In response to an interactive action based on the interactive information, an interactive result page is displayed, wherein the interactive result page includes incentive information assigned to the currently logged-in account of the client.

4. The method according to claim 3, characterized in that, When the interactive information is a virtual container encapsulating incentive information, the step of displaying an interactive result page in response to an interactive operation based on the interactive information includes: In response to an interactive operation based on the virtual container, an interactive results page is displayed, and first multimedia data for recommending the target object is played on the interactive results page; In response to the completion of the first multimedia data playback, the incentive information assigned to the currently logged-in account of the client is displayed on the interactive results page.

5. The method according to claim 4, characterized in that, The step of displaying an interaction result page in response to an interactive operation based on the virtual container includes: In response to a trigger operation on the virtual container, during the period before the trigger operation is released, the virtual container is controlled to deform, wherein the degree of deformation is positively correlated with the pressure parameter or duration of the trigger operation; In response to the virtual container deforming to a degree greater than a threshold, an animation of the virtual container breaking is played, and an interactive results page is displayed after the animation finishes playing.

6. The method according to claim 4, characterized in that, The step of displaying an interaction result page in response to an interactive operation based on the virtual container includes: In response to a motion-sensing operation on the terminal device where the client is located, and the operation parameter of the motion-sensing operation is greater than the operation parameter threshold, an animation of the virtual container breaking is played. Upon completion of the animation playback, the interactive results page is displayed.

7. The method according to claim 3, characterized in that, When the interactive information is at least one virtual card, the step of displaying the interactive result page in response to an interactive operation based on the interactive information includes: In response to a triggering operation for any of the virtual cards, an interaction result page is displayed, and the interaction result page displays incentive information assigned to the currently logged-in account of the client, wherein the incentive information is the incentive information associated with the triggered virtual card, and the incentive information associated with different virtual cards is different.

8. The method according to claim 3, characterized in that, When the interactive information is a virtual avatar, the step of responding to the interactive operation based on the interactive information and displaying the interactive result page includes: Perform one of the following processes: The virtual avatar is controlled to output guidance information, wherein the guidance information is used to instruct the implementation of an interactive action; in response to obtaining an interactive event corresponding to the interactive action, an interactive result page is displayed, and incentive information allocated to the current login account of the client is displayed on the interactive result page, wherein different interactive actions are associated with different incentive information; In response to controlling the virtual avatar to complete the set task, an interaction result page is displayed, and the incentive information assigned to the client's currently logged-in account is displayed on the interaction result page, wherein different tasks are associated with different incentive information.

9. The method according to claim 1, characterized in that, After displaying the interactive information corresponding to the target object, the method further includes: In response to no interactive operation based on the interactive information being received within a set time period, the interactive information is not displayed on the interface of the client running in the foreground state; or, In response to the display duration of the interactive information reaching a duration threshold, the interactive results page is automatically displayed, and the incentive information assigned to the client's currently logged-in account is displayed on the interactive results page.

10. The method according to claim 1, characterized in that, In the case where the target audio is played on a client running in the background, before switching the client from the background running state to the foreground running state in response to a state switching trigger operation for the client, the method further includes: Play the second multimedia data through the client running in the background; In response to the fulfillment of the audio playback conditions, the playback of the second multimedia data is paused, and the target audio is played through the client running in the background.

11. The method according to claim 10, characterized in that, The step of pausing playback of the second multimedia data and playing the target audio through the client running in the background in response to the audio playback condition being met includes: Perform one of the following processes: In response to the playback duration of the second multimedia data exceeding a duration threshold, the playback of the second multimedia data is paused, and the target audio is played through the client running in the background. In response to the playback progress of the second multimedia data reaching a set time point, the playback of the second multimedia data is paused, and the target audio is played through the client running in the background. In response to the occurrence of a set scenario in the second multimedia data, the playback of the second multimedia data is paused, and the target audio is played through the client running in the background.

12. The method according to claim 10, characterized in that, Playing the target audio through the client running in the background includes: Obtain the account characteristic data of the currently logged-in account of the client; For each audio file in a pre-created audio library, the similarity between the audio feature data of the audio file and the account feature data is determined, wherein different audio files are used to recommend different objects; The audio with the highest similarity is selected as the target audio, and the target audio is played through the client running in the background.

13. The method according to claim 10, characterized in that, Playing the target audio through the client running in the background includes: Send an audio retrieval request to the server so that the server generates target audio that matches the content of the second multimedia data; The system receives the target audio returned by the server and plays the target audio through the client running in the background.

14. The method according to claim 10, characterized in that, Playing the target audio through the client running in the background includes: Obtain the status data of the terminal device where the client is located, and determine the current listening scenario based on the status data; Audio matching the current listening scenario is selected from a pre-created audio library, and the matched audio is used as the target audio. The target audio is then played through the client running in the background.

15. The method according to claim 10, characterized in that, Playing the target audio through the client running in the background includes: Send an audio retrieval request to the server, wherein the audio retrieval request carries account feature data of the client's currently logged-in account; The server receives the target audio returned by the server and plays the target audio through the client running in the background. The target audio is obtained by the server in the following ways: obtaining a target object that matches the account feature data from the object library and generating target text for recommending the target object; and converting the target text into the target audio through a text-to-speech service.

16. The method according to claim 10, characterized in that, Playing the target audio through the client running in the background includes: Obtain the voiceprint information of the target character from the second multimedia data; The target audio is played through the client running in the background, based on the voiceprint information that matches the voiceprint information.

17. The method according to any one of claims 1 to 16, characterized in that, After displaying the interactive information corresponding to the target object, the method further includes: Acquire behavioral data, wherein the behavioral data includes: the triggering timing of the state switching triggering operation and the interaction status in response to the interactive information; Based on the behavioral data, the timing of the playback of the target audio and the interactive information are adjusted.

18. An interactive information processing method, characterized in that, The method includes: Play multimedia data; In response to the fulfillment of audio playback conditions, the playback of the multimedia data is paused, the target audio is played, and the target object recommended by the target audio is displayed; In response to a trigger operation on the target object, interactive information corresponding to the target object is displayed.

19. The method according to claim 18, characterized in that, The recommended target objects for displaying the target audio include: Display a notification message corresponding to the target audio in the system notification bar; In response to a triggering operation for the notification message, a playback interface of the client for playing the multimedia data is displayed, and the target object recommended by the target audio is displayed in the playback interface.

20. The method according to claim 18, characterized in that, The target object is displayed in the system notification bar; The step of displaying interactive information corresponding to the target object in response to a trigger operation on the target object includes: In response to a trigger operation on the target object displayed in the system notification bar, a playback interface for playing the multimedia data is displayed on the client, and interactive information corresponding to the target object is displayed on the playback interface.

21. An interactive information processing device, characterized in that, The device includes: A switching module is used to switch the client from the background running state to the foreground running state in response to a state switching trigger operation for the client when the target audio is played in the background running state, wherein the target audio is used to recommend target objects; The display module is used to display interactive information corresponding to the target object in the interface of the client that is in the foreground running state.

22. An interactive information processing device, characterized in that, The device includes: The playback module is used to play multimedia data; The playback module is also configured to pause the playback of the multimedia data and play the target audio in response to the fulfillment of audio playback conditions; The display module is used to display the target object recommended by the target audio; The display module is also used to display interactive information corresponding to the target object in response to a trigger operation on the target object.

23. An electronic device, characterized in that, include: Memory is used to store executable instructions for a computer; A processor, when executing computer-executable instructions stored in the memory, implements the interactive information processing method according to any one of claims 1 to 17 or any one of claims 18 to 20.

24. A computer-readable storage medium storing computer-executable instructions, characterized in that, When the computer-executable instructions are executed by the processor, they implement the interactive information processing method according to any one of claims 1 to 17 or any one of claims 18 to 20.

25. A computer program product comprising a computer program or computer-executable instructions, characterized in that, When the computer program or computer-executable instructions are executed by a processor, they implement the interactive information processing method according to any one of claims 1 to 17 or any one of claims 18 to 20.

Images