Information recommendation method, information recommendation device, electronic device, and computer program

Abstract

The present application provides an information recommendation method, an information recommendation device, an electronic device, and a storage medium, the method including the steps of: playing a video on a video playback interface; displaying a first barrage on the video playback interface, the first barrage including recommendation information; and playing first material related to the story of the first video clip on the video playback interface in response to a trigger operation on the first barrage, the first video clip being a video clip corresponding to a display time period of the first barrage in the video.

Description

[Technical Field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application claims priority from a Chinese patent application bearing application number 202310286312.2, filed with the China Patent Office on March 15, 2023, the entire contents of which are incorporated herein by reference.

[0002] The present application relates to the technical field of human-computer interaction, and in particular to an information recommendation method, an information recommendation device, an electronic device, and a storage medium. [Background technology]

[0003] Danmaku advertising is a type of advertising format in which advertising content is displayed in the form of a barrage of bullets while a video is being played, and is already being used on some video sites as a new form of advertising.

[0004] However, in the solutions provided in the related art, the barrage advertisements are usually mainly displayed with characters or pictures, and the user can only change the font size, color, moving speed, density, etc. of the characters, which lacks effective user interaction, thereby reducing the efficiency and accuracy of information recommendation. Summary of the Invention

[0005] The embodiments of the present application provide an information recommendation method, an information recommendation device, an electronic device, a computer-readable storage medium, and a computer program product that can enrich the expression forms of barrage advertisements and further improve the efficiency and accuracy of information recommendation.

[0006] The technical solution of the embodiment of the present application is realized as follows:

[0007] An embodiment of the present application provides an information recommendation method executed by an electronic device, the method comprising: playing the video on a video playback interface; displaying a first barrage on the video playback interface, the first barrage including recommendation information; In response to a trigger operation on the first barrage, playing first material related to the story of the first video clip on the video playback interface, the first video clip being a video clip corresponding to a display time period of the first barrage within the video.

[0008] An embodiment of the present application provides an information recommendation device, the device comprising: a playback module configured to play the video on the video playback interface; the playback module is further configured to display a first barrage on the video playback interface, the first barrage including recommendation information; The playback module is further configured to play first material related to a story of a first video clip on the video playback interface in response to a trigger operation on the first barrage, the first video clip being a video clip corresponding to a display time period of the first barrage within the video.

[0009] An embodiment of the present application provides an electronic device, the electronic device comprising: a memory for storing executable instructions; and a processor that, when executing executable instructions stored in the memory, implements the information recommendation method provided in the embodiments of the present application.

[0010] An embodiment of the present application provides a computer-readable storage medium having computer-executable instructions stored thereon, which, when executed by a processor, implements the information recommendation method provided in the embodiment of the present application.

[0011] An embodiment of the present application provides a computer program product including a computer program or computer-executable instructions, which, when executed by a processor, implements the information recommendation method provided in the embodiment of the present application.

[0012] The embodiments of the present application have the following beneficial effects:

[0013] During the video playback process, a first barrage containing recommendation information is displayed on the video playback interface, and when a user's trigger operation on the first barrage is recognized, a first material related to the story of the first video clip is displayed on the video playback interface, thereby enriching the presentation format of the barrage advertisement, achieving a correlation effect between the advertisement and the video story, and promoting two-way interaction between the user and the advertisement. In other words, the technology provided in the embodiments of the present application enriches the presentation format of the barrage advertisement, thereby reducing the failure rate of information recommendation, avoiding invalid recommendations, and further improving the efficiency and accuracy of information recommendation. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a schematic diagram of the architecture of an information recommendation system 100 according to an embodiment of the present application. [Figure 2] 1 is a schematic diagram illustrating a configuration of an electronic device 500 according to an embodiment of the present invention. [Figure 3] 1 is a flowchart of an information recommendation method according to an embodiment of the present application; [Figure 4] 1 is a schematic diagram of an application scene of an information recommendation method according to an embodiment of the present application; [Figure 5] 1 is a flowchart of an information recommendation method according to an embodiment of the present application; [Figure 6A] 1 is a schematic diagram of an application scene of an information recommendation method according to an embodiment of the present application; [Figure 6B] 1 is a schematic diagram of an application scene of an information recommendation method according to an embodiment of the present application; [Figure 6C]1 is a schematic diagram of an application scene of an information recommendation method according to an embodiment of the present application; [Figure 6D] 1 is a schematic diagram of an application scene of an information recommendation method according to an embodiment of the present application; [Figure 6E] 1 is a schematic diagram of an application scene of an information recommendation method according to an embodiment of the present application; [Figure 6F] 1 is a schematic diagram of an application scene of an information recommendation method according to an embodiment of the present application; [Figure 7] 1 is a flowchart of an information recommendation method according to an embodiment of the present application; [Figure 8] 1 is a diagram illustrating the principle of realizing a special effect video according to an embodiment of the present invention; DETAILED DESCRIPTION OF THE INVENTION

[0015] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings. The described embodiments do not limit the present application, and all other embodiments that can be obtained by those skilled in the art without any creative efforts are included in the protection scope of the present application.

[0016] In the following description, the term "some embodiments" describes a subset of all possible embodiments, but it is understood that "some embodiments" may be the same or different subsets of all possible embodiments and may be combined with each other without conflict.

[0017] It is to be understood that embodiments of the present application involve associated data such as user information, and where the embodiments of the present application are applicable to a particular product or technology, user permission or consent may be required, and the collection, use, and processing of associated data may be in compliance with the relevant laws, regulations, and standards of the relevant country or region.

[0018] The use of terms such as "first / second / ..." in the specification of the present application does not limit a particular order but distinguishes between similar objects, and it will be understood that "first / second / ..." can be used to interchange a particular order or order where appropriate, so that the embodiments of the present application described herein can be performed in orders other than those shown or described herein.

[0019] In the present application, the term "module" or "unit" refers to a computer program or a portion of a computer program having a certain function, working in conjunction with other related portions to achieve a certain goal, and may be implemented in whole or in part using software, hardware (e.g., processing circuitry or memory), or a combination thereof. Similarly, one or more modules or units may be implemented using a processor (or multiple processors or memories). Furthermore, each module or unit may be part of an overall module or unit that includes the functionality of that module or unit.

[0020] Unless otherwise specified, all technical and scientific terms used in the examples of this application have the same meaning as commonly understood by those skilled in the art. The terms used in the examples of this application are used only to describe the examples of this application and are not intended to limit the application.

[0021] Before describing the embodiments of the present application in detail, the nouns and terms related to the embodiments of the present application will be explained. The explanation of the nouns and terms related to the embodiments of the present application is as follows.

[0022] 1) Danmaku: This is content that pops up on the video playback interface. This content is provided by various users and is used for comments on video content and interactions based on the video content. For example, while watching a video through a video playback platform, users can post comments on the screen, and the comments will be displayed in real time on the video screen in the form of scrolling subtitles.

[0023] 2) Danmaku Ads: A type of advertising format in which advertising content is displayed in the form of a barrage of advertisements while a video is being played. For example, the barrage of advertisements in movies and TV dramas can be used to display text information or pictures.

[0024] 3) Respond to: Represents a condition or state on which an operation to be executed depends, and when the dependent condition or state is met, one or more operations to be executed may be in real time or may have a set delay. Unless otherwise specified, there is no restriction on the execution order of multiple operations to be executed.

[0025] When a video is played, video application operators need to add various recommendation information (e.g., advertisements) to the video to display to users. Here, a barrage advertisement is a type of advertisement presentation format in which advertising content is displayed in the form of a barrage of advertisements while the video is being played, and is already being used on some video sites as a new advertisement presentation format.

[0026] In related technologies, barrage advertisements are usually mainly displayed using text or pictures, and users can only change the font size, color, movement speed, density, etc. of the text. The display method is usually full-width scrolling from right to left within a video playback box. The display method is relatively monotonous and lacks effective user interaction, resulting in a low conversion effect of the advertisement.

[0027] In view of the above, embodiments of the present application provide an information recommendation method, an information recommendation device, an electronic device, a computer-readable storage medium, and a computer program product that can enrich the expression forms of barrage advertisements, realize two-way interaction between users and advertisements, and further improve the advertisement conversion rate and user experience. The electronic device provided in the embodiments of the present application will be described below. The electronic device provided in the embodiments of the present application may be implemented as a terminal device, or may be implemented by a terminal device and a server working together. An example in which a terminal device and a server work together to implement the information recommendation method provided in the embodiments of the present application will be described below.

[0028] Illustratively, refer to Fig. 1, which is a schematic diagram of the architecture of an information recommendation system 100 according to an embodiment of the present application. In order to support applications that enrich the expression form of barrage advertisements and improve the efficiency and accuracy of information recommendation, as shown in Fig. 1, the information recommendation system 100 includes a server 200, a network 300, and a terminal device 400, where the network 300 may be a local area network, a wide area network, or a combination thereof, the terminal device 400 is a terminal device associated with a user, and a client 410 runs on the terminal device 400, and the client 410 may be various types of clients, including, for example, a video playback client, a live streaming client, a browser, etc.

[0029] In some embodiments, taking the case where client 410 is a video playback client as an example, in response to a video playback operation triggered by a user (e.g., receiving a user click operation on a play button), client 410 sends a request to the server via network 300 to obtain video data to be played, so that server 200 returns the corresponding video data to terminal device 400. Next, client 410 can play a video in a video playback interface based on the received video data, and then, during the video playback process, client 410 can display a first barrage (e.g., a barrage advertisement) including recommendation information in the video playback interface. Finally, client 410 can play a first material related to the story of the first video clip in the video playback interface in response to a user trigger operation (e.g., a click operation, a swipe operation, etc.) on the first barrage. Here, the first video clip is a video clip corresponding to the display time period of the first barrage in the video. This allows for a richer range of expressive forms for barrage ads, promotes two-way interaction between users and ads, and improves the conversion rate of ads and the user experience.

[0030] In some other embodiments, the embodiments of the present application may be realized by cloud technology, which is a hosting technology that integrates a set of resources such as hardware, software, and networks into a wide area network or a local area network to realize data calculation, storage, processing, and sharing.

[0031] Cloud technology is a collective term for network technology, information technology, integration technology, management platform technology, and application technology based on the cloud computing business model, which forms a resource pool that can be used flexibly and conveniently according to requirements. Cloud computing technology is an important support. The background services of technical network systems require large amounts of computing and storage resources.

[0032] 1 may be an independent physical server, a server cluster or a distributed system configured with multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), big data, and artificial intelligence platforms. The terminal device 400 may be, but is not limited to, a smartphone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, an in-vehicle terminal, a virtual reality (VR) device, an augmented reality (AR) device, etc. The terminal device 400 and the server 200 may be directly or indirectly connected via wired or wireless communication, and the present application is not limited thereto.

[0033] In some embodiments, the terminal device may implement the information recommendation method provided in the embodiments of the present application by executing various computer-executable instructions or computer programs. For example, the computer-executable instructions may be microprogram-level instructions, machine instructions, or software instructions. The computer program may be a native program or software module within an operating system, a local application (APP), i.e., a program that needs to be installed on an operating system to run, such as a live streaming APP or a video playback APP, a mini-program, i.e., a program that can be run simply by downloading it to a browser environment, or a mini-program that can be embedded in any APP. In short, the computer-executable instructions may be instructions in any form, and the computer program may be an application, module, or plug-in in any form.

[0034] The configuration of an electronic device provided in an embodiment of the present application will be described below. Taking the electronic device as a terminal device as an example, FIG. 2 is a schematic diagram showing the configuration of an electronic device 500 according to an embodiment of the present application. The electronic device 500 shown in FIG. 2 includes at least one processor 510, a memory 550, at least one network interface 520, and a user interface 530. The components in the electronic device 500 are coupled via a bus system 540. It should be understood that the bus system 540 is configured to realize connection and communication between these components. The bus system 540 includes a data bus as well as a power bus, a control bus, and a status signal bus. However, for clarity of explanation, various buses are represented as the bus system 540 in FIG. 2.

[0035] The processor 510 may be an integrated circuit chip with signal processing capabilities, such as a general purpose processor, a digital signal processor (DSP), a programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc., where the general purpose processor may be a microprocessor or any conventional processor, etc.

[0036] The user interface 530 includes one or more output devices 531 that enable the presentation of media content, including one or more speakers and / or one or more visual displays. The user interface 530 further includes one or more input devices 532 that facilitate user input, such as a keyboard, mouse, microphone, touchscreen display, camera, or other input buttons or controls.

[0037] Memory 550 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. Memory 550 includes one or more storage devices physically remote from processor 510.

[0038] The memory 550 may include volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 550 described in the embodiments herein may include any suitable type of memory.

[0039] In some embodiments, memory 550 may store data to support various operations, examples of which include programs, modules, data structures, or a subset or superset thereof, such as those illustrated below.

[0040] The operating system 551 includes system programs for processing various basic system services and executing hardware-related tasks, such as a frame layer, a core library layer, and a drive layer, and is configured to realize various basic operations and process hardware-based tasks.

[0041] The network communication module 552 is configured to reach other computing devices via one or more (wired or wireless) network interfaces 520, example network interfaces 520 including Bluetooth®, Wireless Fidelity (WiFi)®, and Universal Serial Bus (USB), etc.

[0042] The display module 553 is configured to enable presentation of information (e.g., a user interface for operating peripherals and presenting content and information) via one or more output devices 531 (e.g., a display, a speaker) associated with the user interface 530.

[0043] The input processing module 554 is configured to detect one or more user inputs or interactions from one or more input devices 532 and to interpret the detected inputs or interactions.

[0044] In some embodiments, the device provided in the embodiments of the present application may be realized by software. FIG. 2 shows an information recommendation device 555 stored in a memory 550. The information recommendation device 555 may be software in the form of a program or plug-in, and includes software modules such as a playback module 5551, an identification module 5552, a determination module 5553, an encoding module 5554, an enhancement module 5555, a virtualization module 5556, and an acquisition module 5557. These modules are logical and can be arbitrarily combined or further divided according to the functions to be realized. It should be noted that, for the sake of convenience, FIG. 2 shows all the above modules collectively, but this should not be considered to exclude an implementation in which the information recommendation device 555 includes only the playback module 5551. The functions of each module will be described later.

[0045] Hereinafter, the information recommendation method provided in the embodiment of the present application will be described in detail in combination with exemplary applications and implementations of the terminal device provided in the embodiment of the present application.

[0046] For example, please refer to FIG. 3, which is a flowchart of an information recommendation method according to an embodiment of the present application, which will be described with reference to the steps shown in FIG.

[0047] It should be noted that the method illustrated in FIG. 3 may be performed by various forms of computer programs executed on a terminal device, and is not limited to a client. For example, it may be the above-mentioned operating system, software module, script, mini-program, etc. Therefore, the following client example should not be considered to limit the embodiments of the present application. Also, for convenience of explanation, no particular distinction will be made hereinafter between a terminal device and a client executed on a terminal device.

[0048] In step 101, a video is played on a video playback interface.

[0049] In some embodiments, step 101 can be implemented by playing a video on a video playback interface in response to a video playback trigger operation. For example, a video playback button can be displayed on the video playback interface (e.g., the video playback button can be displayed below the video playback interface), and when a user clicks on the video playback button, the terminal device can send a request to the server to obtain video data of the video waiting to be played, and after receiving the video data returned from the server, play the video on the video playback interface.

[0050] In step 102, the first barrage is displayed on the video playback interface.

[0051] Here, the first danmaku may include recommendation information (such as an advertisement), and at least one of the movement speed, size, and display effect parameters (such as color and font size) of the first danmaku (such as a danmaku advertisement) may be different from that of the second danmaku, where the second danmaku is a danmaku that does not include recommendation information (i.e., a general danmaku). That is, the display style of the danmaku advertisement displayed in the video playback interface may be different from that of a normal danmaku advertisement, thereby enticing the user to click on the danmaku advertisement.

[0052] For example, if the first danmaku is a danmaku advertisement, the moving speed of the danmaku advertisement in the video playback interface may be slower than that of the second danmaku (i.e., a normal danmaku that does not include advertising content, for example, a danmaku sent by another user watching the video), thereby preventing the danmaku advertisement from moving too fast for the user to click on it. Alternatively, the size of the danmaku advertisement may be larger than that of the second danmaku, i.e., the danmaku advertisement displayed in the video playback interface is more conspicuous and stands out compared to a normal danmaku, thereby increasing the user's attention. Alternatively, the color of the danmaku advertisement may be distinguished from that of the second danmaku. For example, by applying a color to the danmaku advertisement and a white color to the second danmaku advertisement, the user can quickly select the danmaku advertisement from among multiple danmaku advertisements and easily perform trigger operations such as clicking or swiping on the danmaku advertisement.

[0053] It should be noted that the first barrage may be sent by the operator of the video application or by a user (e.g., a privileged user) watching the video, and the embodiment of the present application does not particularly limit this. Furthermore, it should be noted that the recommendation information included in the first barrage may be other types of information such as news, inquiry information, etc. in addition to advertisements, and the embodiment of the present application does not particularly limit the type of recommendation information.

[0054] In some embodiments, the above step 102 can be implemented in the following manner: in response to the video playback progress reaching a predetermined point, start displaying a first barrage from a first end of the video playback interface (e.g., the right side of the video playback interface), and gradually move the first barrage toward a second end of the video playback interface (e.g., the left side of the video playback interface) until the first barrage exits the video playback interface.

[0055] For example, if the first barrage is a barrage advertisement, when it is detected that the video playback progress has reached a predetermined point (for example, 10 seconds), the barrage advertisement starts to be displayed from the right side of the video playback interface, and is controlled to gradually move toward the left side of the video playback interface until the barrage advertisement leaves the video playback interface.

[0056] To make it easier for users to click on the barrage ad, the barrage ad can be controlled to move to the left side of the video playback interface, then stopped on the left side of the video playback interface for a predetermined period of time (e.g., 5 seconds), and then the barrage ad can be controlled to exit the video playback interface.

[0057] It should be further noted that the server sets at least one time point for each video, so that when the terminal device detects that the playback progress of the current video has reached a predetermined time point, it can send a barrage advertisement content request for the current time point to the server, and after receiving the barrage advertisement content returned from the server, display the corresponding barrage advertisement in the video playback interface. Of course, the terminal device can also send a barrage advertisement content request for the current predetermined time point to the server before the playback progress of the video reaches the predetermined time point (e.g., 10 seconds before), i.e., preload the barrage advertisement, and the embodiments of the present application are not particularly limited thereto.

[0058] For example, for each video, the server can set at least one time point for the video in the following manner: the video can be divided into multiple video clips, for example, according to different scenes appearing on the video screen, the video can be divided into multiple different video clips, and at least one time point can be set for each video clip, that is, a corresponding barrage advertisement can be displayed during the playback of each video clip.

[0059] In some other embodiments, the above step 102 can be further implemented in the following manner: in response to a predetermined scene appearing in the video, start displaying a first barrage from a first end of the video playback interface, and gradually move the first barrage toward a second end of the video playback interface until the first barrage exits the video playback interface, where the recommendation information included in the first barrage is associated with a scene, and the recommendation information associated with different scenes is different.

[0060] For example, if the first barrage of scenes is a barrage advertisement, a trained scene identification model can be invoked to perform scene identification processing on the video (e.g., the video can be identified frame by frame or once every predetermined period, such as once every 10 seconds). If a predetermined scene is detected in the video, a barrage advertisement containing an advertisement associated with the current scene can be displayed from the right side of the video playback interface, and the barrage advertisement can be controlled to gradually move toward the left side of the video playback interface until it disappears from the video playback interface. For example, if the scene in the video is a family having a meal, a barrage advertisement containing a beverage advertisement can be displayed in the video playback interface. This associates the advertising content included in the barrage advertisement with the scene in the video, reducing user resistance to the advertisement and further improving the conversion rate of the advertisement.

[0061] The following describes the training process of the above scene classification model.

[0062] In some embodiments, the server invokes the initialized scene identification model based on the sample video to perform a prediction process to obtain the scenes that appear in the sample video, then substitutes the scenes obtained by the prediction and the label data marked for the sample video (e.g., the scene information marked for the sample video) into a loss function to perform a backpropagation process, updates the parameters of the scene identification model during the backpropagation process, and obtains a trained scene identification model, and the server can then transmit the trained scene identification model to the terminal device.

[0063] The following continues to explain the configuration of the above scene classification model.

[0064] In some embodiments, an exemplary configuration of a scene identification model may include an input layer (i.e., an embedding layer), an encoding layer (e.g., which may be composed of multiple cascaded convolutional layers), a fully-connected layer, and an output layer (including an activation function such as a Softmax function). For example, some frames of a video (e.g., multiple frames extracted from a portion of a video currently being played) are input to the input layer for an embedding process, then an encoding process is performed on the embedded feature vectors output from the input layer via the encoding layer to obtain hidden layer feature vectors, and then a fully-connected process is performed on the hidden layer feature vectors via the fully-connected layer. Finally, the fully-connected results output from the fully-connected layer are input to the output layer, and an activation process is performed via the output layer to obtain the scenes appearing in the video.

[0065] The above scene identification model may be a neural network model (e.g., a convolutional neural network, a deep convolutional neural network, or a fully connected neural network), a decision tree model, a gradient boosting tree, a multi-layer perceptron, a support vector machine, etc., and the embodiments of the present application do not particularly limit the type of scene identification model.

[0066] In some other embodiments, the above step 102 can be further implemented as follows: in response to the appearance of a related object in the video that meets a matching condition with the recommendation information, start displaying a first barrage from a first end of the video playback interface, and gradually move the first barrage to a second end of the video playback interface until the first barrage exits the video playback interface, and the matching condition includes: the label of the related object includes a keyword in the recommendation information, and the label of the related object includes a recommendation theme of the recommendation information.

[0067] For example, if the first barrage advertisement is a barrage advertisement, an image recognition model can be invoked to perform image recognition processing on the video (e.g., the video can be recognized frame by frame, or recognized once every 10 seconds, or other predetermined period). If a related object that meets a matching condition with an advertisement included in the barrage advertisement is detected in the video—for example, if the advertisement included in the barrage advertisement is an ice cream advertisement, the corresponding recommendation theme can be gourmet food—and if an object related to gourmet food (e.g., dessert, cake, ice cream, etc., the corresponding label for these objects is gourmet food) is detected in the video, the barrage advertisement including the ice cream advertisement can be displayed from the right side of the video playback interface and can be controlled to gradually move toward the left side of the video playback interface until the barrage advertisement disappears from the video playback interface. In this way, because the advertising content included in the barrage advertisement is related to the objects that appear in the video, the barrage advertisement does not appear abrupt, reducing user resistance to the advertisement and further improving the ad conversion rate and user experience.

[0068] It should be noted that the above-mentioned image identification model may be a neural network model (e.g., a convolutional neural network, a deep convolutional neural network, or a fully connected neural network), a decision tree model, a gradient boosting tree, a multilayer perceptron, a support vector machine, etc., and the embodiment of the present application does not particularly limit the type of scene identification model. Furthermore, it should be noted that to avoid leakage, the image identification model may be invoked to perform detection for each video and frame, and of course, to reduce resource consumption of the terminal device, the image identification model may be invoked to perform detection once per predetermined period, and the embodiment of the present application does not particularly limit this.

[0069] In some other embodiments, the above step 102 can be further implemented as follows: in response to a playback progress setting operation on the video time axis, start playing the video from the set playback progress, start displaying a first barrage from a first end of the video playback interface, and gradually move the first barrage to a second end of the video playback interface until the first barrage exits the video playback interface.

[0070] For example, taking the case where the first barrage is a barrage advertisement, reference is made to FIG. 4, which is a schematic diagram illustrating an application scenario of the information recommendation method according to an embodiment of the present application. As shown in FIG. 4, a video playback interface 401 displays a timeline 402 of the currently playing video, and a button 403 for adjusting the playback progress is displayed on the timeline 402. Assume that the current playback progress of the video is 20 seconds, and then a user drags the button 403 displayed on the video timeline 402. For example, if the user drags the button 403 to a position corresponding to 10 seconds on the timeline 402, the video is replayed from 10 seconds, and the barrage advertisement 404 starts to be displayed from the right side of the video playback interface. The barrage advertisement 404 is controlled to gradually move toward the left side of the video playback interface until it disappears from the video playback interface. In this way, displaying a barrage advertisement while the user is repeatedly watching a great video clip can reduce the user's resistance to advertisements and effectively improve the conversion rate of advertisements.

[0071] In step 103, in response to a trigger operation on the first barrage, a first material related to the story of the first video clip is played on the video playback interface.

[0072] Here, the first material can be used to enhance the story atmosphere of the first video clip, where the type of the first material may include pictures, text, emojis, virtual images, etc.

[0073] It should be noted that the number of first materials displayed in the video playback interface may be one or more. For example, the number of first materials displayed in the video playback interface may be dynamically adjusted based on the size of a blank area on the currently played video screen. For example, if it is detected that the blank area included in the currently played video screen is small, fewer first materials may be displayed in the video playback interface. If it is detected that the currently played video screen has a large blank area, more first materials may be displayed in the video playback interface. Of course, the number of first materials displayed in the video playback interface may be manually set by the user, and the embodiment of the present application is not particularly limited thereto.

[0074] It should be further noted that the display position of the video playback interface may be fixed, for example, the first material may always be displayed in the lower area of ​​the video playback interface. Of course, the position of the first material in the video playback interface may also be dynamically adjusted, for example, the first material may move according to the movement of a person appearing on the video screen. Of course, the position of the first material displayed in the video playback interface may be set by a user, for example, the first material may be displayed at a position set by the user in the video playback interface, and the embodiments of the present application are not particularly limited thereto.

[0075] In some embodiments, taking the first barrage advertisement as an example, assuming that the display time of the barrage advertisement is from 10 seconds to 15 seconds, i.e., when the video is played up to 10 seconds, the barrage advertisement appears from one side of the video playback interface, and when the video is played up to 15 seconds, the barrage advertisement disappears from the other side of the video playback interface. When a user's trigger operation (e.g., a click operation, a swipe operation, etc.) for the barrage advertisement is received, a first material related to the story of the first video clip (i.e., the video clip from 10 seconds to 15 seconds) is played in the video playback interface. For example, assuming that the story of the first video clip is about a male protagonist confessing his love to a female protagonist, the first material may be a material with a style of falling flower petals to enhance the story atmosphere of the first video clip, thereby realizing interaction between the advertisement and the video story and improving the user's viewing experience.

[0076] In addition, when a user's trigger operation for the first barrage is received, a new layer may be added and the first material related to the story of the first video clip may be played on the newly added layer, and the layer may be overlaid on the video playback interface, thereby achieving the visual effect of playing the first material on the video playback interface.

[0077] In some other embodiments, continuing from the above example, the first material may be selected from a plurality of candidate materials, and before playing the first material related to the story of the first video clip, the following process can be further performed: a story identification process is performed on the first video clip to obtain story information of the first video clip, and material that matches the story information of the first video clip (i.e., first material for enhancing the story atmosphere of the first video clip) is determined from the plurality of candidate materials.

[0078] For example, the step of performing a story identification process on the first video clip and obtaining story information of the first video clip can be implemented by calling a story identification model to perform a story identification process on the first video clip and obtain the story information of the first video clip. For example, if the first video clip is video clip 1, which is a portion from 10 seconds to 15 seconds of the video, video clip 1 can be cut out from the video, and then a trained story identification model can be called to perform a story identification process on video clip 1, thereby obtaining the story information of video clip 1. For example, if video clip 1 includes a scene of kissing or handing over flowers, the corresponding story information can be that the male protagonist confesses his love to the female protagonist, or that the two are on a date.

[0079] Note that the above story identification model may be a neural network model (e.g., a convolutional neural network, a deep convolutional neural network, or a fully connected neural network), a decision tree model, a gradient boosting tree, a multilayer perceptron, a support vector machine, etc., and the embodiments of the present application do not particularly limit the type of story identification model.

[0080] For example, the step of determining a first material that matches the story information of the first video clip from the above-mentioned plurality of candidate materials can be realized by the steps of: performing an encoding process on the styles of the plurality of candidate materials respectively to obtain a style feature vector corresponding to each material; performing an encoding process on the story information of the first video clip to obtain a story feature vector corresponding to the first video clip; sequentially determining the similarity between the story feature vector corresponding to the first video clip and the style feature vector corresponding to each material; and selecting the material corresponding to the greatest similarity as the material that matches the story information of the first video clip (i.e., the first material).

[0081] For example, if the multiple candidate materials are assumed to be four materials, namely material 1, material 2, material 3, and material 4, the style of material 1 may be a style of falling petals as shown in FIG. 6B, the style of material 2 may be a style of spreading light spots as shown in FIG. 6C, the style of material 3 may be a style of rising bubbles as shown in FIG. 6D, and the style of material 4 may be a style of colorful fireworks as shown in FIG. 6E. A model such as a multilayer perceptron or a convolutional neural network may be invoked to perform an encoding process on each of the styles of the four materials, thereby obtaining a style feature vector corresponding to each material. Next, a model such as a multilayer perceptron or a convolutional neural network may be subsequently invoked to perform an encoding process on the story information of a first video clip (e.g., video clip 1, which is the portion of the video from 10 seconds to 15 seconds) to obtain a story feature vector corresponding to video clip 1. Similarity (e.g., the binary norm or Euclidean distance between the two feature vectors) between the story feature vector corresponding to video clip 1 and the style feature vectors corresponding to each of the four materials is then calculated. The material (assumed to be material 1) corresponding to the greatest similarity is used as the first material of the story information matching video clip 1, i.e., material 1 may be used as a material to enhance the story atmosphere of video clip 1. Since the style of the first material is associated with the story of the first video clip, this can improve the user's viewing experience, promote user interaction with the advertisement, and further increase the advertisement conversion rate.

[0082] In some embodiments, the first material may be extracted from the first video clip, and before playing the first material related to the story of the first video clip, the following process may further be performed: identifying a target object from the first video clip, the target object including any of the object that has appeared the longest in the first video clip, the object that has appeared the most times in the first video clip, or the object that is most numerous in the first video clip, and using at least one of the target object and the virtualized target object as the first material to enhance the story atmosphere of the first video clip.

[0083] For example, if the first video clip is video clip 1 from 10 seconds to 15 seconds into the video, an image recognition model can be invoked to identify a target object from video clip 1. Here, the target object may be the object (e.g., assumed to be a flower) that appears the longest other than a person in video clip 1, the object (e.g., assumed to be a white cloud) that appears the most frequently other than a person in video clip 1, or the object (e.g., assumed to be trees) that appears the most frequently in video clip 1. Then, at least one of the target object or the virtualized target object can be used as a first material to enhance the story atmosphere of video clip 1. In this way, since the first material is associated with the story of video clip 1, visual inconsistency between the first material and the video content can be avoided, effectively improving the user's viewing experience.

[0084] For example, if the target object is a flower, the flower that appears in video clip 1 can be directly used as a material (i.e., a first material) to enhance the story atmosphere of video clip 1. Of course, a virtualization process can be performed on the flower to obtain a virtualized flower, for example, a cartoon flower, and the cartoon flower can be used as a material to enhance the story atmosphere of video clip 1. Furthermore, both a flower and a cartoon flower can be used as a material to enhance the story atmosphere of video clip 1 at the same time, and the embodiments of the present application are not particularly limited thereto. In other embodiments, the first video clip may include multiple story units, and the step of playing the first material related to the story of the first video clip can be realized by synchronously playing the first material in a video playback interface when different story units of the first video clip are played, and applying display effect parameters corresponding to the story units to the first material.

[0085] For example, if the first video clip is video clip 1, which is the portion from 10 seconds to 20 seconds of the video, video clip 1 can be divided into multiple story units (e.g., assume video clip 1 is divided into three story units, namely story unit 1, story unit 2, and story unit 3) based on the people appearing in video clip 1, the topics discussed by the people, or the location (e.g., including indoors, outdoors, etc.). In this case, when different story units of video clip 1 are played, the first material is synchronously played in the video playback interface, and display effect parameters (e.g., including the number, size, movement speed, transparency, etc.) corresponding to the currently played story unit are applied to the first material. For example, when story unit 1 in video clip 1 is played, the number of first materials is 30, the movement speed is 5 m / s, and the transparency is 50%. When story unit 2 in video clip 1 is played, the number of first materials increases to 100, the movement speed increases to 10 m / s, and the transparency decreases to 30%, and when story unit 3 in video clip 1 is played, the number of first materials decreases to 50, the movement speed decreases to 6 m / s, and the transparency increases to 60%. In this way, by applying different display effect parameters to different story units in the first video clip when playing the first materials, the display effect of the first materials changes according to changes in the story, further improving the user's viewing experience.

[0086] In addition to dividing the first video clip into multiple story units based on the people who appear in the first video clip, the topics they discuss, or the locations they visit, the first video clip can also be divided into multiple stages based on time. For example, if the first video clip is video clip 1, which is a portion from 10 seconds to 40 seconds of the video, video clip 1 can be divided into three stages in advance, where stage 1 corresponds to the portion from 10 seconds to 20 seconds in video clip 1, stage 2 corresponds to the portion from 20 seconds to 30 seconds in video clip 1, and stage 3 corresponds to the portion from 30 seconds to 40 seconds in video clip 1. When the first material is played, a display effect parameter corresponding to the current stage is applied to the first material, and different display effect parameters correspond to different stages. For example, when playing stage 1 in video clip 1, the number of first materials is 30 and the transparency is 50%, when playing stage 2 in video clip 1, the number of first materials increases to 100 and the transparency decreases to 30% (i.e., the display effect of the first materials is improved), and when playing stage 3 in video clip 1, the number of first materials decreases to 20 and the transparency increases to 60%. Thus, when the first materials are synchronously played for different stages of the first video clip, different display effect parameters are applied to the first materials, thereby further enriching the expression forms of the first materials and improving the user's viewing experience.

[0087] In some other embodiments, refer to Fig. 5, which is a flowchart of an information recommendation method according to an embodiment of the present application. As shown in Fig. 5, after performing step 103 shown in Fig. 3, steps 104 to 106 shown in Fig. 5 can be further performed, which will be described with reference to the steps shown in Fig. 5.

[0088] In step 104, the recommendation information is displayed at a set position in the video playback interface.

[0089] In some implementations, after the first barrage leaves the video playback interface, the recommendation information included in the first barrage may be displayed in a set location (eg, the bottom right corner) of the video playback interface.

[0090] For example, if the first barrage is a barrage advertisement, if the display conditions are met, the barrage advertisement can be controlled to be visible from one end of the video playback interface and gradually move toward the other end of the video playback interface, and after the barrage advertisement leaves the video playback interface, the advertising content included in the barrage advertisement can be displayed in the lower right corner of the video playback interface, thereby realizing secondary exposure of the advertisement.

[0091] In step 105, the display effect parameters of the first material are improved in response to a trigger operation on the recommendation information.

[0092] In some embodiments, the step of improving the display effect parameters of the first material can be achieved by performing at least one of the following processes: increasing the number of first materials displayed in the video playback interface; increasing the movement speed of the first material in the video playback interface; decreasing the transparency of the first material in the video playback interface; and increasing the size of the first material displayed in the video playback interface.

[0093] For example, if the first barrage advertisement is a barrage advertisement, after the barrage advertisement exits the video playback interface, the advertisement content included in the barrage advertisement and an interaction button (e.g., a "click to scatter flowers" button) associated with the advertisement content may be displayed in the lower right corner of the video playback interface. When a user clicks on the interaction button, the display effect parameters of the first material played in the video playback interface may be improved. For example, the number of first materials may be increased, the movement speed of the first materials may be increased, or the transparency of the first materials may be reduced in the video playback interface. This may promote user interaction with the advertisement and further improve the conversion rate of the advertisement.

[0094] In step 106, after the first material has been played, a detailed page of the recommendation information is displayed.

[0095] In some embodiments, after detecting that the first material has finished playing, a recommendation details page can be automatically displayed in the video playback interface to facilitate user purchases. Of course, the recommendation details page can also be displayed in a split-screen format to avoid affecting the user's video viewing. For example, the recommendation details page can be displayed in an area independent of the video playback interface, and the embodiments of the present application do not limit the display method of the recommendation details page.

[0096] 6A , which is a schematic diagram illustrating an application scenario of the information recommendation method according to an embodiment of the present application. As shown in FIG. 6A , when a video playback progress reaches a predetermined point, a barrage advertisement 604 can be displayed in a video playback interface 602. When a user clicks on the barrage advertisement 604, a first material 605 related to the story of a first video clip can be displayed in the video playback interface. For example, if the story of the first video clip is about a male protagonist confessing his love to a female protagonist, the first material 605 can be a material with a falling petal style to enhance the story atmosphere of the first video clip. After the barrage advertisement 604 exits the video playback interface, advertising content 606 included in the barrage advertisement 604 and an interaction button (e.g., a "click to scatter flowers" button 607) associated with the advertising content 606 are displayed in the lower right corner of the video playback interface. When a user clicks on the "click to scatter flowers" button 607, the display effect of the first material 605 displayed on the video playback interface can be enhanced, for example, by reducing the transparency of the first material 605 or increasing the moving speed of the first material 605, and then controlling the first material 605 to gradually disappear, and automatically displaying the advertisement details page 608.

[0097] In addition, when a user's trigger operation for the first barrage is received, in addition to playing the first material in the video playback interface, a detailed page of the recommendation information can also be displayed, and the embodiment of the present application is not particularly limited thereto.

[0098] In some other embodiments, the type of the first material may be a virtual image, and the step of playing the first material related to the story of the first video clip in the above-mentioned video playback interface can further be realized by a step of playing a virtual image related to the story of the first video clip in the video playback interface, wherein the display method of the virtual image includes a static display method and a dynamic display method.

[0099] For example, if the first barrage is a barrage advertisement, when a user clicks on the barrage advertisement, a virtual image related to the story of the first video clip can be played in the video playback interface. Here, the virtual image is associated with an advertisement included in the barrage advertisement, and the advertisement is related to the story of the first video clip. For example, if the story of the first video clip is about having a family dinner, the corresponding advertisement may be a beverage advertisement. The virtual image may be an intellectual property (IP) image of an advertising brand set by an advertiser for the advertisement. Of course, the virtual image may be related to the video story, for example, a virtual object extracted from the video. Furthermore, the display method of the virtual image may be a static display method, for example, a static image. Of course, the display method of the virtual image may be a dynamic display method, for example, a moving image.

[0100] It should be noted that, assuming that a user trigger operation (e.g., a click operation, a swipe operation, etc.) on the virtual image is received while the virtual image is being played, a detailed page of the recommendation information can be displayed. Here, the detailed page of the recommendation information may be floating on the video playback interface or may be displayed independently of the video playback interface in a split-screen manner. The embodiments of the present application do not limit the display method of the detailed page of the recommendation information.

[0101] In some other embodiments, the virtual image may be obtained from a first video clip, and before playing the virtual image related to the story of the first video clip, the following process can be further performed: identify a target object from the first video clip, the first video clip being a video clip corresponding to the display time period of the first barrage in the video, perform a virtualization process on the target object, and obtain a virtual image, the target object including one of the object that has appeared the longest in the first video clip, the object that has appeared the most times in the first video clip, or the object that is most numerous in the first video clip.

[0102] For example, if the first video clip is video clip 1, which is a portion from 10 seconds to 15 seconds into the video, video clip 1 can be cut out from the video, and then an image recognition model can be invoked to identify a target object from video clip 1. Here, the target object may be the object (e.g., a flower) that appears the longest other than a person in video clip 1. Of course, the target object may also be the object (e.g., a chick) that appears the most frequently other than a person in video clip 1. Furthermore, the target object may be the object (e.g., a tree) that appears the most frequently in video clip 1. After identifying the target object from video clip 1, a virtualization process can be performed on the identified target object to obtain a virtual image. For example, if the target object is a chick, a virtualization process can be performed on the identified chick from video clip 1 to obtain a cartoonized chick, and the cartoonized chick can be used as a virtual image related to the story of video clip 1. This effectively improves the user's viewing experience because the virtual image is associated with the story of the first video clip.

[0103] In some embodiments, the virtual image may be obtained based on the recommendation information, and before playing the virtual image related to the story of the first video clip, the following processing can be further performed: obtain feature information of the recommendation information, where the recommendation information is related to the story of the first video clip, and the feature information includes at least one of keywords of the recommendation information and a recommendation theme; obtain a virtual image with the feature information of the recommendation information as a label, where different virtual images are pre-associated with different labels.

[0104] For example, if the first barrage is a barrage advertisement, before playing a virtual image related to the story of the first video clip, feature information of the advertisement included in the barrage advertisement can be first obtained, and the advertisement can be related to the story of the first video clip. For example, if the story of the first video clip is about a family dining out, the corresponding advertisement can be a beverage advertisement. For example, the advertisement's keywords and recommendation theme can be obtained, and then a matching target virtual object, such as a virtual image with a cute label or a gourmet label, can be obtained from multiple candidate virtual images based on at least one of the obtained advertisement's keywords (e.g., assumed to be cute) and recommendation theme (e.g., assumed to be gourmet). This improves the consistency between the virtual image and the recommendation information included in the first barrage, as the virtual image is associated with the recommendation information, and further improves the user's viewing experience.

[0105] In addition, the above-mentioned recommended theme may be obtained by extracting keywords of the recommended information. For example, if there are multiple keywords of the recommended information, any keyword randomly extracted from the multiple keywords can be used as the recommended theme of the recommended information.

[0106] According to the information recommendation method provided in the embodiments of the present application, during the video playback process, a first barrage (e.g., a barrage advertisement) is displayed on the video playback interface, and after identifying a user's trigger operation on the barrage advertisement, a first material related to the story of the first video clip is displayed on the video playback interface, thereby enriching the presentation format of the barrage advertisement, achieving a correlation effect between the advertisement and the video story, and promoting two-way interaction between the user and the advertisement. In other words, the technology provided in the embodiments of the present application enriches the presentation format of the barrage advertisement, thereby reducing the failure rate of information recommendation, avoiding invalid recommendations, and further improving the efficiency and accuracy of information recommendation.

[0107] In the following, the first barrage advertisement is taken as an example to describe an exemplary application example of the embodiment of the present application in an actual application scene.

[0108] The present embodiment provides a new advertising format that generates special effects (corresponding to the first material) through interaction with a barrage ad. The special effects can have a certain correlation with the video story, thereby achieving a correlation effect between the barrage ad and the video story. By identifying a user's click, swipe, or other operation on the barrage ad via a terminal device (e.g., a mobile phone associated with the user) and triggering the display of special effects, the barrage ad's presentation format is enriched, two-way interaction between the user and the ad is realized, and the presentation effect of brand advertising is improved. Furthermore, from the perspective of advertisement delivery, the present embodiment rationally adjusts the delivery time of the barrage ad—for example, by delivering a barrage ad that can interact with the story in a scene that matches the product features in the video—to achieve a correlation between the advertisement and the video content, thereby more easily connecting with the viewer, increasing user interest in participating in the interaction, and further improving the click-through rate, interaction rate, and conversion rate of the advertisement.

[0109] The information recommendation method provided in the embodiment of the present application will be described in detail below.

[0110] In some implementations, a user realizes story interaction by clicking on a barrage ad to trigger an atmospheric special effect (corresponding to the first material described above).

[0111] Illustratively, refer to FIG. 6A, which is a schematic diagram of an application scene of the information recommendation method according to an embodiment of the present application. As shown in FIG. 6A, when watching a video on a video playback platform, a user needs to turn on a barrage display 601, so that the video content can be played in a video playback interface 602 while simultaneously displaying a barrage 603 sent by other users. When the video content is played up to a certain point or scene, a barrage advertisement 604 can be displayed in the video playback interface 602. For example, the barrage advertisement 604 can appear from the right side of the video playback interface 602, gradually move to the left side of the video playback interface 602, and pause on the left side for N seconds (e.g., 10 seconds). When a user clicks on the barrage advertisement 604 (e.g., a user clicks on the "Chase Love and Scatter Flowers" button in the barrage advertisement 604), a petal-falling atmosphere special effect 605 can be displayed in the video playback interface 602. For example, atmospheric special effects may be displayed on the story atmosphere layer, and the story atmosphere layer may be displayed overlaid on the video playback interface. If a user click operation on the barrage advertisement 604 is not received, the barrage advertisement 604 may be displayed for N seconds and then canceled, advertising content 606 (e.g., reduced advertising content) included in the barrage advertisement 604 may be displayed in the lower right corner of the video playback interface, and atmospheric special effects 605 may be automatically displayed on the video playback interface. Thereafter, if a user click operation is received on an advertisement interaction button (e.g., a "click to scatter flowers" button 607) displayed in the lower right corner of the video playback interface 602, the display effect of the atmospheric special effects 605 displayed on the video playback interface may be enhanced (e.g., by increasing the number of petals in the atmospheric special effects 605 or increasing the moving speed of the petals in the atmospheric special effects 605), and after the display of the atmospheric special effects 605 is completed (e.g., by controlling the atmospheric special effects 605 to gradually disappear), an advertisement detail page 608 may be automatically displayed.

[0112] Note that, when a user clicks on the advertising content included in the barrage ad 604, a direct jump to the advertisement details page 608 can be made. It should be noted that the material style of the atmospheric special effect can be, in addition to the petal falling style 609 shown in FIG. 6B, a light spot spreading style 610 shown in FIG. 6C, a bubble rising style 611 shown in FIG. 6D, or a fireworks coloring style 612 shown in FIG. 6E. A special effect atmosphere suitable for the style of the story can be displayed on the video playback interface according to the current video story. For example, if the current video story is about a male protagonist confessing his love to a female protagonist, after the user clicks on the barrage ad, a petal falling style atmospheric special effect can be displayed to enhance the atmosphere.

[0113] In some other implementations, a user can also achieve story interaction by clicking on a barrage ad to trigger an IP image (corresponding to the virtual image above).

[0114] For example, refer to FIG. 6F, which is a schematic diagram illustrating an application scenario of the information recommendation method according to an embodiment of the present application. As shown in FIG. 6F, when watching a video on a video playback platform, a user needs to turn on the barrage display 601, which allows the video content to be played on the video playback interface 602 while simultaneously displaying the barrage 603 sent by other users. When the video content is played up to a specific point or scene, a barrage advertisement 613 can be displayed on the video playback interface 602. If a user clicks on the barrage advertisement 613, an IP image 614 of the advertising brand (the material format of the IP image may be a static image or a moving image) can be displayed. After the barrage advertisement 613 is displayed on the video playback interface 602 for N seconds (e.g., 10 seconds), it can move off-screen according to the normal barrage movement direction. Then, M seconds (e.g., 6 seconds) after the barrage advertisement disappears, the IP image 614 of the advertising brand is displayed again on the video playback interface, and the barrage advertisement 613 is pushed again, thereby achieving a secondary exposure of the advertisement. Thereafter, when a user clicks on the barrage advertisement 613 or the IP image 614, a detailed advertisement page 615 can be displayed on the screen.

[0115] It should be noted that an IP image 614 of the advertising brand may also be displayed together with the barrage advertisement 613. It should be further noted that interaction methods for the barrage advertisement in the embodiment of the present application include, but are not limited to, clicking, swiping, shaking, flipping, tilting, pinching, etc., and the special effects in the embodiment of the present application may be expressed in various ways and forms, such as VR, AR, and three-dimensional (3D) video.

[0116] In some embodiments, after a user enters the detail page of a video, the terminal device associated with the user can initiate a request for the markpoint position of the corresponding barrage advertisement from the server. After receiving this request, the server can query the markpoint position (corresponding to the above-mentioned predetermined time point) of the corresponding video identifier (e.g., VID: Video ID). If there is markpoint position information of the barrage advertisement corresponding to this VID, it returns it to the terminal device. After receiving the markpoint position information returned from the server, the terminal device plays the main video. When the playback progress of the main video reaches N seconds before the markpoint position (e.g., 10 seconds, this advance amount can be changed by a configuration system), the terminal device can initiate a request for detailed barrage advertisement content for this markpoint position from the server. The server searches for corresponding barrage advertisement content based on the markpoint position and returns it to the terminal device. At the same time, the main video continues to be played. When the playback progress of the main video reaches the markpoint position, the terminal device checks whether there is barrage advertisement content at this time point, and if there is, displays the barrage advertisement. The display method of the barrage ad is the same as normal barrage ad, scrolling from one side of the screen to the other at the speed set by the barrage ad. If there is no barrage ad content, or after the current barrage ad is displayed, a check is made to see if there is a next mark point position. If there is a mark point position, the process enters a loop to process the next mark point position; if there is not, the process ends.

[0117] For example, please refer to FIG. 7, which is a flowchart of an information recommendation method according to an embodiment of the present application, which will be described with reference to the steps shown in FIG.

[0118] In step 201, the terminal device displays a video detail page.

[0119] In step 202, the terminal device transmits a request for the location of the mark point of the barrage advertisement to the server.

[0120] In step 203, the server queries the markpoint location based on the VID of the video.

[0121] In step 204, the server returns the mark point location list to the terminal device.

[0122] In step 205, the terminal device plays the main video.

[0123] In step 206, the terminal device determines that the main program time has reached N seconds before the mark point position.

[0124] In step 207, the terminal device sends a request for barrage advertising content to the server.

[0125] In step 208, the server retrieves the barrage advertisement content at the corresponding markpoint location.

[0126] In some implementations, the danmaku ad content may include text, pictures, special effects content, and the like.

[0127] In step 209, the server returns the corresponding barrage advertisement content to the terminal device.

[0128] In step 210, the terminal device determines that the time progress of the main story has reached a markpoint position.

[0129] In step 211, the terminal device checks whether there is any barrage advertising content, and if there is, executes step 212; if there is no barrage advertising content, executes step 213.

[0130] In step 212, the terminal device displays the barrage advertisement.

[0131] In some embodiments, the danmaku ads provided in the embodiments of the present application support user interaction and can combine with the ads to create corresponding interaction effects. The above display and interaction effects can be achieved by special danmaku and layer conversion animation. This special danmaku (i.e., danmaku ads) refers to a danmaku that mixes pictures and rich text by inserting icon-style pictures into any position of the text based on the existing plain text danmaku and supporting rich text. The terminal device triggers the display of special effects by identifying the user's actions, such as clicking or swiping, on the danmaku ad.

[0132] For example, refer to Fig. 8, which is a diagram illustrating the principle of realizing a special effect video according to an embodiment of the present application. As shown in Fig. 8, the layer density can be controlled by one variable v, and can be changed (e.g., increased or decreased) through an interaction button on the advertisement. At time T after the special effect is displayed, the number of randomly generated layers L is n × v, the position of the special effect material is P(x, y) = (random() % screenWidth, random() % screenHeight), the size S(x) = random() % 100 / 100.f × rawScale, and the transparency O(x) = random() % 100 / 100.f × rawOpacity, where n is a variable greater than 1, random() represents a random function, screenWidth represents the screen width (i.e., the width of the video playback interface), screenHeight represents the screen height (i.e., the height of the video playback interface), rawScale represents the original size of the special effect material, and rawOpacity represents the original transparency of the special effect material. Next, after time N, i.e., time T' = T + N, the steps performed at time T are repeated. This allows different layers to be generated at different times, and by randomly changing the position, size, and transparency of the layer generated at the previous time T, it is possible to achieve random video special effects.

[0133] In step 213, the terminal device checks whether there is a next mark point position, and if there is, executes step 206; if not, executes step 214.

[0134] In step 214, the terminal device ends the process.

[0135] As described above, the information recommendation method provided in the embodiment of the present application has the following beneficial effects.

[0136] (1) From the advertiser's perspective, the technical solutions provided in the embodiments of the present application are useful for increasing brand advertising awareness, cultivating users' favorable impressions of the brand, and enhancing marketing effectiveness.

[0137] (2) From the user's perspective, the technical solutions provided in the embodiments of the present application can fully arouse the user's curiosity, encourage the user to voluntarily watch advertisements, reduce the user's subjective resistance to advertisements, and realize two-way communication between the brand and the viewer.

[0138] (3) From the perspective of advertising strategy, by analyzing the behavioral data generated by users participating in interactions, it can provide brands with reference to consumer preferences and make guiding suggestions for brand distribution strategies.

[0139] (4) From the perspective of a video playback platform, the technical solutions provided in the embodiments of the present application can bring about an interactive experience, thereby increasing the interest of the video story and improving the user's viewing experience.

[0140] The following continues to describe an exemplary configuration in which the information recommendation device 555 provided in the embodiments of the present application is implemented as a software module. In some embodiments, as shown in FIG. 2, the software modules in the information recommendation device 555 stored in the memory 550 may include a playback module 5551.

[0141] The playback module 5551 is configured to play a video on the video playback interface, the playback module 5551 is further configured to display a first barrage on the video playback interface, the first barrage including recommendation information, and the playback module 5551 is further configured to play first material related to the story of the first video clip on the video playback interface in response to a trigger operation on the first barrage, the first video clip being a video clip corresponding to the display time period of the first barrage in the video.

[0142] In some embodiments, the information recommendation device 555 further comprises an identification module 5552, which is configured to perform a story identification process on the first video clip to obtain story information of the first video clip before the playback module 5551 plays the first material related to the story of the first video clip, and the information recommendation device 555 further comprises a determination module 5553, which is configured to determine the first material that matches the story information from a plurality of candidate materials.

[0143] In some embodiments, the information recommendation device 555 further includes an encoding module 5554, which is configured to perform an encoding process on each of the styles of a plurality of candidate materials to obtain a style feature vector corresponding to each material, and the encoding module 5554 is further configured to perform an encoding process on story information of a first video clip to obtain a story feature vector corresponding to the first video clip, and the determination module 5553 is further configured to sequentially determine a similarity between the story feature vector corresponding to the first video clip and the style feature vector corresponding to each material, and use the material corresponding to the maximum similarity as the first material matching the story information.

[0144] In some embodiments, the identification module 5552 is further configured to identify a target object from the first video clip before the playback module 5551 plays the first material related to the story of the first video clip, the target object including any of the object that has appeared the longest in the first video clip, the object that has appeared the most times in the first video clip, or the object that is most numerous in the first video clip, and the determination module 5553 is further configured to use at least one of the target object and the virtualized target object as the first material related to the story of the first video clip.

[0145] In some embodiments, after the first barrage leaves the video playback interface, the playback module 5551 is further configured to display the recommended information at a set position in the video playback interface, the information recommendation device 555 further includes an improvement module 5555, the improvement module 5555 is configured to improve the display effect parameters of the first material in response to a trigger operation on the recommended information, and the playback module 5551 is further configured to display a detail page of the recommended information after the playback of the first material is completed.

[0146] In some embodiments, the enhancement module 5555 is further configured to perform at least one of the following processes: increasing the number of first materials displayed in the video playback interface; increasing the speed of movement of the first materials in the video playback interface; decreasing the transparency of the first materials in the video playback interface; and increasing the size of the first materials displayed in the video playback interface.

[0147] In some embodiments, the first video clip includes multiple story units, and the playback module 5551 is further configured to synchronously play the first material in the video playback interface when playing different story units of the first video clip, and apply display effect parameters corresponding to the story units to the first material.

[0148] In some embodiments, the first material includes a virtual image, and the playback module 5551 is further configured to play the virtual image related to the story of the first video clip in the video playback interface, and the display manner of the virtual image includes a static display manner and a dynamic display manner.

[0149] In some embodiments, before the playback module 5551 plays the virtual image related to the story of the first video clip, the identification module 5552 is further configured to identify a target object from the first video clip, and the information recommendation device 555 further includes a virtualization module 5556, which is configured to perform a virtualization process on the target object to obtain a virtual image related to the story of the first video clip, and the target object includes one of the object that has appeared the longest in the first video clip, the object that has appeared the most times in the first video clip, or the object that is most numerous in the first video clip.

[0150] In some embodiments, the information recommendation device 555 further includes an acquisition module 5557, where the acquisition module 5557 is configured to acquire feature information of the recommendation information before the playback module 5551 plays a virtual image related to the story of the first video clip, where the recommendation information is related to the story of the first video clip and the feature information includes at least one of a keyword of the recommendation information and a recommendation theme, and the acquisition module 5557 is further configured to acquire a virtual image with the feature information of the recommendation information as a label, where different virtual images are pre-associated with different labels.

[0151] In some embodiments, the playback module 5551 is further configured, in response to the playback progress of the video reaching a predetermined point, to begin displaying the first barrage from a first end of the video playback interface and gradually move the first barrage toward a second end until the first barrage moves out of the video playback interface.

[0152] In some embodiments, the playback module 5551 is further configured to, in response to a predetermined scene appearing in the video, begin displaying a first barrage from a first end of the video playback interface and gradually move the first barrage toward a second end until the first barrage moves out of the video playback interface, wherein the recommendation information included in the first barrage is associated with a scene, and the recommendation information associated with different scenes is different.

[0153] In some embodiments, the playback module 5551 is further configured to, in response to a related object appearing in the video that meets a matching condition with the recommendation information, start displaying a first barrage from a first end of the video playback interface and gradually move the first barrage to a second end until the first barrage moves out of the video playback interface, where the matching condition includes that a label of the related object includes a keyword in the recommendation information and that a label of the related object includes a recommendation theme of the recommendation information.

[0154] In some embodiments, the playback module 5551 is further configured to, in response to a playback progress setting operation on a timeline of the video, start playing the video from the set playback progress, start displaying a first barrage from a first end of the video playback interface, and gradually move the first barrage to a second end until the first barrage moves out of the video playback interface.

[0155] In some embodiments, at least one of the movement speed, size, and display effect parameters of the first barrage is different from that of the second barrage, and the second barrage is a barrage that does not include recommendation information.

[0156] It should be noted that the description of the device in the embodiment of the present application is similar to the description of the method embodiment above, and has similar beneficial effects as the method embodiment, so it will not be described again here. The technical details of the information recommendation device provided in the embodiment of the present application that have not yet been described can be understood by referring to the description of either FIG. 3 or FIG. 5.

[0157] An embodiment of the present application provides a computer program product including a computer program or computer-executable instructions, the computer program or computer-executable instructions being stored in a computer-readable storage medium, a processor of a computing device reading the computer-executable instructions from the computer-readable storage medium, and the processor executing the computer-executable instructions to cause the computing device to perform the information recommendation method described in the embodiment of the present application.

[0158] An embodiment of the present application provides a computer-readable storage medium having computer-executable instructions stored thereon, which, when executed by a processor, causes the processor to perform an information recommendation method provided in an embodiment of the present application, for example, the information recommendation method shown in FIG. 3 or FIG. 5.

[0159] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic memory, an optical disk, or a CD-ROM, or may be various devices including one or any combination of the above memories.

[0160] In some embodiments, the executable instructions take the form of a program, software, software module, script, or code, which may be written in any type of programming language (including compiled or interpreted, declarative or procedural languages), and may be arranged in any form, including as an independent program, or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0161] As an example, the executable instructions may be arranged to be executed on a single electronic device, may be arranged to be executed on multiple electronic devices located at a single location, or may be arranged to be executed on multiple electronic devices distributed across multiple locations and interconnected via a communications network.

[0162] The above is only an example of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and scope of the present application shall be included in the scope of protection of the present application.

Claims

1. An information recommendation method executed by an electronic device, comprising: playing the video on a video playback interface; displaying a first barrage on the video playback interface, the first barrage including recommendation information; In response to a trigger operation on the first barrage, a step of playing first material related to the story of the first video clip on the video playback interface, the first video clip being a video clip corresponding to a display time period of the first barrage within the video.

2. Before playing a first material related to the story of the first video clip, the information recommendation method includes: performing a story identification process on the first video clip to obtain story information of the first video clip; determining the first material that matches the story information from a plurality of candidate materials; The information recommendation method according to claim 1 .

3. The step of determining the first material that matches the story information from the plurality of candidate materials includes: performing an encoding process for each of the styles of a plurality of candidate materials to obtain a style feature vector corresponding to each of the plurality of candidate materials; performing an encoding process on story information of the first video clip to obtain a story feature vector corresponding to the first video clip; sequentially determining a similarity between a story feature vector corresponding to the first video clip and a style feature vector corresponding to each of the plurality of candidate materials; and using the candidate material corresponding to the highest similarity as the first material that matches the story information. The information recommendation method according to claim 2 .

4. Before playing a first material related to the story of the first video clip, the information recommendation method includes: identifying a target object from the first video clip, the target object comprising one of the object that has appeared the longest in the first video clip, the object that has appeared the most times in the first video clip, or the object that is most numerous in the first video clip; and using at least one of the target object and the virtualized target object as the first material related to the story of the first video clip. The information recommendation method according to claim 1 .

5. After the first barrage has left the video playback interface, the information recommendation method includes: displaying the recommendation information at a set position on the video playback interface; improving a display effect parameter of the first material in response to a trigger operation on the recommendation information; and displaying a detailed page of the recommendation information after the playback of the first material is completed. The information recommendation method according to claim 1 .

6. The step of improving the display effect parameters of the first material includes: increasing the number of said first materials displayed on said video playback interface; increasing the speed of movement of the first material in the video playback interface; reducing the transparency of the first material in the video playback interface; performing at least one of the following operations: increasing the size of the first material displayed on the video playback interface; The information recommendation method according to claim 5.

7. the first video clip includes a plurality of story units; The step of playing a first material related to the story of the first video clip on the video playback interface includes: When playing different story units of the first video clip, synchronously playing a first material on the video playback interface, and applying display effect parameters corresponding to the story units to the first material; The information recommendation method according to claim 1 .

8. the first material includes a virtual image; The step of playing a first material related to the story of the first video clip on the video playback interface includes: Playing the virtual image related to the story of the first video clip on the video playback interface, wherein the display manner of the virtual image includes a static display manner and a dynamic display manner; The information recommendation method according to claim 1 .

9. Before playing the virtual image related to the story of the first video clip, the information recommendation method includes: identifying a target object from the first video clip, the target object comprising one of the object that has appeared the longest in the first video clip, the object that has appeared the most times in the first video clip, or the object that is most numerous in the first video clip; performing a virtualization process on the target object to obtain the virtual image related to the story of the first video clip; The information recommendation method according to claim 8.

10. Before playing the virtual image related to the story of the first video clip, the information recommendation method includes: acquiring characteristic information of the recommendation information, the recommendation information being related to a story of the first video clip, and the characteristic information including at least one of a keyword of the recommendation information and a recommendation theme; acquiring the virtual images using the feature information of the recommendation information as labels, wherein different virtual images are pre-associated with different labels; The information recommendation method according to claim 8.

11. The step of displaying a first barrage on the video playback interface includes: In response to the video playback progress reaching a predetermined point, starting to display a first barrage from a first end of the video playback interface, and gradually moving the first barrage toward a second end until the first barrage moves out of the video playback interface. The information recommendation method according to any one of claims 1 to 10.

12. The step of displaying a first barrage on the video playback interface includes: in response to a predetermined scene appearing in the video, starting to display a first barrage from a first end of the video playback interface and gradually moving the first barrage toward a second end until the first barrage moves out of the video playback interface, wherein recommendation information included in the first barrage is associated with the scene, and recommendation information associated with different scenes is different; The information recommendation method according to any one of claims 1 to 10.

13. The step of displaying a first barrage on the video playback interface includes: in response to a related object appearing in the video that satisfies a matching condition with the recommendation information, starting to display a first barrage from a first end of the video playback interface and gradually moving the first barrage to a second end until the first barrage exits the video playback interface, wherein the matching condition includes that a label of the related object includes a keyword in the recommendation information and that a label of the related object includes a recommended theme of the recommendation information; The information recommendation method according to any one of claims 1 to 10.

14. The step of displaying a first barrage on the video playback interface includes: In response to a playback progress setting operation on a time axis of the video, starting playback of the video from the set playback progress, starting display of a first barrage from a first end of the video playback interface, and gradually moving the first barrage to a second end until the first barrage exits the video playback interface. The information recommendation method according to any one of claims 1 to 10.

15. At least one of the movement speed, size, and display effect parameters of the first barrage is different from that of the second barrage, and the second barrage does not include recommendation information. The information recommendation method according to any one of claims 1 to 10.

16. An information recommendation device, a playback module configured to play the video on the video playback interface; the playback module is further configured to display a first barrage on the video playback interface, the first barrage including recommendation information; The playback module is further configured to play first material related to a story of a first video clip on the video playback interface in response to a trigger operation on the first barrage, the first video clip being a video clip corresponding to a display time period of the first barrage within the video.

17. An electronic device, a memory for storing executable instructions; and a processor that, when executing executable instructions stored in the memory, implements the information recommendation method according to any one of claims 1 to 10.

18. A computer program that causes a computer to execute the information recommendation method according to any one of claims 1 to 10.

Images