Video display control method, device and medium

Abstract

The present invention discloses a video display control method, device and medium, the method comprising the following steps: receiving user payment information, determining whether the user has paid; if the user has paid, displaying the video content in full screen on the terminal screen; if the user has not paid, displaying the video content in non-full screen on the terminal screen, and displaying advertisement content around the video content on the terminal screen. The method disclosed by the present invention realizes flexible display control of video content according to user payment status, and ensures that the user can watch the video content even if the user has not paid, and compensates the loss of the video provider through advertisement revenue.

Description

Technical Field

[0001] The invention relates to a video display control method, belonging to the technical field of image control. Background Art

[0002] With the rapid development of information technology and broadcasting and television technology, users can now watch various types of video content through terminal devices. However, in some cases, users need to pay to watch some video content.

[0003] Existing methods generally directly block video content when the user has not paid, which brings a bad experience to the user.

[0004] Advertisements are the main source of income for video providers. They are usually played before the video content is played. This method requires users to watch the advertisement before they can watch the video content, which makes users wait for a long time and creates a sense of disgust. In addition, the advertisement content has no correlation with the video content and is not user-specific, resulting in a low conversion rate of the advertisement.

[0005] Therefore, it is necessary to conduct a more in-depth study on the existing video display control method to solve the above problems. Summary of the invention

[0006] In order to overcome the above problems, in-depth research was conducted and a video display control method was designed, including the following steps:

[0007] Receive user payment information and determine whether the user has paid;

[0008] If the user pays, the video content will be displayed in full screen on the terminal screen;

[0009] If the user has not paid, the video content will be displayed in a non-full screen on the terminal screen, and advertising content will be displayed around the video content on the terminal screen.

[0010] In a preferred embodiment, a product in a video content is identified, and an advertisement highly related to the product is played when the video plays the product.

[0011] In a preferred embodiment, before determining whether the user has paid, the method further includes:

[0012] S1. Parse the video content to obtain product terms contained in the video, as well as the time and duration of the product terms appearing in the video;

[0013] Establish an advertisement library to store cooperative advertisement entries, the broadcast time of each advertisement, and the price coefficient of the advertisement;

[0014] S2, obtaining the relevance between different product terms and different advertising terms, and generating a relevance matrix;

[0015] S3. Based on the analysis results, obtain the continuous influence parameters of the product entry on the user during the video playback process;

[0016] S4, splitting the video into multiple continuous segments, and encoding any segment based on the product terms appearing in the segment and the continuous influence parameters corresponding to the product terms to obtain the product code of the segment;

[0017] S5, obtaining a set of segment advertisements for the segment based on the product terms and the relevance matrix that appear in the segment;

[0018] Encoding the segment advertisement set based on the advertisement broadcast time and the advertisement price coefficient to obtain the advertisement code of the segment;

[0019] S6. Using product codes and advertising codes as nodes and relevance matrices as edges, a graph neural network is used for matching to obtain the best advertising terms for the segment.

[0020] In a preferred embodiment, if the user has not paid, while the video content of the segment is played in a non-full screen on the terminal screen, the advertising material corresponding to the best advertising term of the segment is played around the video content.

[0021] In a preferred embodiment, in S2, the correlation matrix It is expressed as:

[0022]

[0023] in, Indicates different product terms. Represents the set of all product terms in the video. Indicates different advertising terms. Represents a collection of advertising terms. Indicates Product entries and The relevance between the advertising terms.

[0024] In a preferred embodiment, in S3, the continuous influence parameter of the product is obtained based on the duration of the product appearing in the video content, the number of times the product appears, and the average interval time between adjacent appearances.

[0025] In a preferred embodiment, the continuous influence parameter is also modified by a centrality weight, and the centrality weight is used to characterize the position of the product in the video.

[0026] In a preferred embodiment, in S4, encoding is performed by a multilayer perceptron to obtain a product code, including the following sub-steps:

[0027] S41, masking the area except the product image in the video clip by using a mask to obtain a pure product image;

[0028] S42, extracting video features from the pure product image using a video encoder;

[0029] S43, fusing the video feature with the continuous influence parameter to obtain a fusion feature;

[0030] S44. Encode the fused features through a multi-layer perceptron to obtain a product code for the segment.

[0031] The present invention also provides an electronic device, comprising:

[0032] at least one processor; and

[0033] A memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any of the methods described above.

[0034] The present invention also provides a computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute any of the above-mentioned methods.

[0035] The beneficial effects of the present invention include:

[0036] (1) Flexible display control of video content based on user payment status is achieved, ensuring that users can still watch video content even if they have not paid, and compensating the losses of video providers through advertising revenue;

[0037] (2) The ability to link advertising materials with video content not only improves the fun of the video, reduces users’ aversion to advertising, but also increases the advertising’s publicity effect and improves the conversion rate of advertising. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A schematic diagram showing the overall flow of a video display control method according to a preferred embodiment of the present invention is shown;

[0039] Figure 2 A schematic diagram of the process of linking video content and advertising material in a video display control method according to a preferred embodiment of the present invention is shown. DETAILED DESCRIPTION

[0040] The present invention will be further described in detail below through the accompanying drawings and embodiments. Through these descriptions, the characteristics and advantages of the present invention will become more clear and distinct.

[0041] The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments. Although various aspects of the embodiments are shown in the drawings, the drawings are not necessarily drawn to scale unless otherwise noted.

[0042] According to a video display control method provided by the present invention, Figure 1 As shown, the following steps are included:

[0043] Receive user payment information and determine whether the user has paid;

[0044] If the user pays, the video content will be displayed in full screen on the terminal screen;

[0045] If the user has not paid, the video content will be displayed in a non-full screen on the terminal screen, and advertising content will be displayed around the video content on the terminal screen.

[0046] In this way, the present invention realizes flexible display control of video content according to the user's payment situation, providing a better user experience. When the user has not paid, he can still watch the video content, and the advertising content is displayed in the form of pictures around the video, which increases the advertising revenue. After the user pays, he can enjoy the video content played in full screen, and the surrounding advertising space disappears, providing a clearer and more comfortable viewing experience.

[0047] Furthermore, although the video content provider can recover the video playback cost by playing advertisements, this method is not user-friendly, and in this way, the user's attention will be focused on the video content, making the advertising effect poor. How to improve the user's viewing experience and improve the advertising effect is the difficulty of the present invention.

[0048] In the present invention, the advertising material is linked with the video content. By identifying the product in the video and playing an advertisement highly related to the product when the product is played in the video, the advertisement and the video content are integrated to achieve the effect of video and advertisement linkage, which not only improves the play interest and reduces the user's aversion to the advertisement, but also increases the publicity effect of the advertisement and improves the conversion rate of the advertisement. The questionnaire survey shows that linking the advertising material with the video content can not only reduce the user's aversion to the advertisement, but even generate heat in certain plots and increase the user's interest in watching the advertisement.

[0049] According to the present invention, before determining whether the user has paid, preferably before the server provides the video to the user, such as Figure 2 As shown, it also includes:

[0050] S1. Parse the video content to obtain product terms contained in the video, as well as the time and duration of the product terms appearing in the video;

[0051] Establish an advertisement library to store cooperative advertisement entries, the broadcast time of each advertisement, and the price coefficient of the advertisement;

[0052] According to the present invention, the price coefficient of the advertisement is used to represent the fees provided by different advertisers after a single broadcast of the advertisement.

[0053] S2, obtaining the relevance between different product terms and different advertising terms, and generating a relevance matrix;

[0054] S3. Based on the analysis results, obtain the continuous influence parameters of the product entry on the user during the video playback process;

[0055] S4, splitting the video into multiple continuous segments, and encoding any segment based on the product terms appearing in the segment and the continuous influence parameters corresponding to the product terms to obtain the product code of the segment;

[0056] S5, obtaining a set of segment advertisements for the segment based on the product terms and the relevance matrix that appear in the segment;

[0057] Encoding the segment advertisement set based on the advertisement broadcast time and the advertisement price coefficient to obtain the advertisement code of the segment;

[0058] S6. Using product codes and advertising codes as nodes and relevance matrices as edges, a graph neural network is used for matching to obtain the best advertising terms for the segment.

[0059] Preferably, the above process is implemented on the server side of the video content provider, thereby reducing the amount of computation on the user side and avoiding freezes and other phenomena on the user side.

[0060] Furthermore, if the user has not paid, while the video content of the segment is played in a non-full screen on the terminal screen, the advertising material corresponding to the best advertising term of the segment is played around the video content.

[0061] According to the present invention, in S1, the specific method of the analysis is not limited, and those skilled in the art can freely choose according to actual needs. It can be performed by manual labeling or automatic identification and labeling through a neural network, such as a CNN neural network.

[0062] In S2, the correlation matrix It is expressed as:

[0063]

[0064] in, Indicates different product terms. Represents the set of all product terms in the video. Indicates different advertising terms. Represents a collection of advertising terms. Indicates Product entries and The relevance between the advertising terms.

[0065] In the prior art, there are many methods for obtaining term similarity, which are not limited in the present invention. Those skilled in the art can freely choose from the prior art as needed. For example, traditional statistical-based text similarity calculation methods, such as vector space model, latent Dirichlet distribution model, etc., can also be used to pre-train word embeddings using methods such as word2vec, and then use deep neural networks to extract sentence representations and their interactions, and train the final multi-layer perceptron from the representations and interactions to adapt to text semantic similarity labels.

[0066] In S3, the continuous influence parameter of the product is obtained based on the duration of the product's appearance in the video content, the number of times the product appears, and the average interval between adjacent appearances.

[0067] Generally speaking, the longer a product appears in the video content, the more likely it is that the audience will pay attention to the product. For example, in a video clip of cleaning a room, there will be many products in the video, such as brooms, tables and chairs, and computers. However, the broom appears the longest in the video clip, so the audience is most likely to pay attention to the broom.

[0068] Furthermore, in actual video content, due to reasons such as storyboards and camera movements, a product will not appear continuously but multiple times. Using the number of times a product appears and the average interval between adjacent appearances as generation factors for the product's continuous influence parameters can more comprehensively obtain all products that the audience may pay attention to, thereby finding more suitable advertising materials in the advertising library for playback, avoiding the problem of poor correlation between advertising and video content, which affects the effectiveness of advertising delivery.

[0069] Preferably, an influence network is set up, and the duration of a product's continuous appearance in the video content, the number of times the product appears, and the average interval between adjacent appearances are used as inputs of the influence network, and the influence network outputs the continuous influence parameters of the product.

[0070] Preferably, the influence network adopts a fully connected neural network.

[0071] In a preferred embodiment, product salient features are also obtained and used as one of the input parameters of the influence network.

[0072] In actual video playback, the audience's attention to the product in the video will be affected by product movement, camera movement, etc. By obtaining the product's significant features, the accuracy of the product's sustained influence parameters can be improved to avoid the advertising material appearing abrupt due to inconsistency with the video content.

[0073] In a preferred embodiment, the saliency feature is obtained using the method disclosed in the document Dahou, Yasser, et al. "ATSal: An Attention Based Architecture for Saliency Prediction in 360Videos." International Conference on Pattern Recognition. Cham: Springer International Publishing, 2021.

[0074] In a preferred embodiment, the continuous influence parameter is also modified by a centrality weight, and the centrality weight is used to characterize the position of the product in the video.

[0075] The probability of being noticed by users for products at the edge of the video is significantly reduced, and the probability of being noticed for products closer to the center of the video is significantly increased, as measured by the center weight.

[0076] The centrality weight It is expressed as:

[0077]

[0078] in, represents the average position of the product in the video at all times, Indicates the midpoint of the video. W Indicates the width of the video. H Indicates the height of the video.

[0079] In S4, encoding is performed through a multi-layer perceptron to obtain the product code.

[0080] The multilayer perceptron is a commonly used neural network, and its specific structure is not described in detail in the present invention.

[0081] Preferably, encoding is performed through a multi-layer perceptron to obtain the product code.

[0082] Specifically, the method includes the following sub-steps:

[0083] S41, masking the area except the product image in the video clip by using a mask to obtain a pure product image;

[0084] S42, extracting video features from the pure product image using a video encoder;

[0085] S43, fusing the video feature with the continuous influence parameter to obtain a fusion feature;

[0086] S44. Encode the fused features through a multi-layer perceptron to obtain a product code for the segment.

[0087] Preferably, in S42, the video encoder adopts I3D.

[0088] In S43, after performing dot multiplication on the video features and the continuous influence parameters, they are fused through a fusion module, which includes a self-attention mechanism of three linear layers, a Norm layer and a linear layer to realize an attention mechanism for multi-time step prediction.

[0089] In S44, the multilayer perceptron is a commonly used neural network model, and its specific structure is not described in detail in the present invention.

[0090] In S5, a relevance threshold is set, and when the relevance between the product term and the advertisement term in the segment is greater than the threshold, the advertisement is considered to belong to the segment advertisement of the segment.

[0091] Furthermore, the playing time and the price coefficient of the advertisements in the segment advertisement set are passed through a fully connected network to obtain the advertisement coding.

[0092] In S6, the graph neural network is a commonly used deep learning network for graph structured data. It is good at finding the deep rules and semantic features hidden behind the nodes and edges in the graph data. It is widely used in different fields and can obtain accurate planning.

[0093] Various embodiments of the method described above in the present invention can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs, which may be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general programmable processor, which may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0094] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0095] In the context of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0096] To provide interaction with a user, the methods and apparatus described herein may be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0097] The methods and apparatus described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communications network). Examples of communications networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0098] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0099] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in the present disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution disclosed in the present invention can be achieved, and this document is not limited here.

Claims

1. A video display control method, characterized in that: The following steps are involved: Receive user payment information and determine whether the user has paid; If the user pays, the video content will be displayed in full screen on the terminal screen; If the user has not paid, the video content will be displayed in a non-full screen on the terminal screen, and advertising content will be displayed around the video content on the terminal screen; Identify products in video content and play ads that are highly relevant to the product when the video plays the product; Before determining whether the user has paid, it also includes: S1. Parse the video content to obtain product terms contained in the video, as well as the time and duration of the product terms appearing in the video; Establish an advertisement library to store cooperative advertisement entries, the broadcast time of each advertisement, and the price coefficient of the advertisement; S2, obtaining the relevance between different product terms and different advertising terms, and generating a relevance matrix; S3. Based on the analysis results, obtain the continuous influence parameters of the product entry on the user during the video playback process; S4, splitting the video into multiple continuous segments, and encoding any segment based on the product terms appearing in the segment and the continuous influence parameters corresponding to the product terms to obtain the product code of the segment; S5, obtaining a set of segment advertisements for the segment based on the product terms and the relevance matrix that appear in the segment; Encoding the segment advertisement set based on the advertisement broadcast time and the advertisement price coefficient to obtain the advertisement code of the segment; S6, using product codes and advertising codes as nodes and relevance matrices as edges, a graph neural network is used for matching to obtain the best advertising terms for the segment; In S3, the continuous influence parameter is obtained based on the duration of the product's appearance in the video content, the number of times the product appears, and the average interval between adjacent appearances; The continuous influence parameter is also modified by a centrality weight, which is used to characterize the position of the product in the video.

2. The video display control method according to claim 1, characterized in that: If the user has not paid, the video content of the segment will be played in a non-full screen on the terminal screen, and the advertising material corresponding to the best advertising term of the segment will be played around the video content.

3. The video display control method according to claim 1, characterized in that: In S2, the correlation matrix It is expressed as: ； in, Indicates different product terms. Represents the set of all product terms in the video. Indicates different advertising terms. Represents a collection of advertising terms. Indicates Product entries and The relevance between the advertising terms.

4. The video display control method according to claim 1, characterized in that: In S4, encoding is performed through a multi-layer perceptron to obtain a product code, which includes the following sub-steps: S41, masking the area except the product image in the video clip by using a mask to obtain a pure product image; S42, extracting video features from the pure product image using a video encoder; S43, fusing the video feature with the continuous influence parameter to obtain a fusion feature; S44. Encode the fused features through a multi-layer perceptron to obtain a product code for the segment.

5. An electronic device comprising: at least one processor; as well as A memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor so that the at least one processor can execute the method of any one of claims 1-4.

6. A computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-4.

Images