Video display method and video generation method

Abstract

The embodiment of the invention provides a video display method and a video generation method, and belongs to the field of video processing. The video display method comprises the following steps: playing a target video; under the condition that a preset condition is met, obtaining a popularization article in a current playing picture in the target video; displaying an article information pop-up window of the promoted article; and in response to a triggering operation for the article information popup window, skipping to an interaction page of the promoted article. According to the technical scheme provided by the embodiment of the invention, the advertisement content and the original video can be well fused together, so that the watching experience of audiences is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of video advertising technology, and in particular to a video display method, a video generation method, a video display device, a video generation device, a computer device, a computer-readable storage medium, and a computer program product. Background Art

[0002] With the rapid development of Internet technology, product advertising and marketing has gradually gained favor among the general audience.

[0003] At present, one of the main forms of product promotion is voice-over advertising. However, voice-over advertising is not only expensive to produce, but also difficult to integrate with the original video content, which can easily arouse the audience's disgust and affect the audience's viewing experience.

[0004] It should be noted that the above content is not necessarily prior art, nor is it intended to limit the scope of patent protection of this application. Summary of the invention

[0005] The embodiments of the present application provide a video display method, a video generation method, a video display device, a video generation device, a computer device, a computer-readable storage medium, and a computer program product to solve or alleviate one or more of the technical problems raised above.

[0006] One aspect of an embodiment of the present application provides a video display method, the method comprising: Play the target video; When a preset condition is met, obtaining a promotional item in a currently playing screen of the target video; Displaying a pop-up window of item information of the promotional item; and In response to a triggering operation on the item information pop-up window, jump to an interactive page of the promoted item.

[0007] Optionally, the current playing screen includes a video pause frame screen; when a preset condition is met, obtaining the promotional item in the current playing screen in the target video includes: When the target video is paused, the promotional item in the pause frame of the video is obtained; the promotional item is pre-marked with item information, and the item information includes name, location, description information and jump information.

[0008] Optionally, displaying a pop-up window of item information of the promoted item includes: Generating a highlight display frame according to the position of the promotional item, wherein the highlight display frame is used to indicate the corresponding position of the promotional item; In response to a click operation on a corresponding position of the promotional item, a pop-up window of item information of the promotional item is popped up.

[0009] Optionally, generating a highlight display frame according to the position of the promotional item includes: Obtaining the edge contour and position of the promotional item; Determine the highlight display frame parameters based on the edge contour and position of the promotional item; wherein the highlight display frame parameters include: the size, shape and / or position of the highlight display frame; The highlight display frame is drawn according to the highlight display frame parameters.

[0010] Optionally, obtaining the promotional item in the current playing screen of the target video includes: Acquire a first feature vector of the promotional item in the current playing picture from a server; Obtaining a second feature vector of each of the one or more objects in the current playing picture; Matching the first feature vector and the second feature vector to obtain a matching result; According to the matching result, the promotional items in the current playing picture are determined.

[0011] Another aspect of the embodiments of the present application provides a video generation method, the method comprising: Performing object recognition on a target video frame in the video to be processed to extract promotional items and identification information of the promotional items; Displaying the information editing interface of the promotional item; receiving input information through the information editing interface, wherein the input information and the identification information are used to form item information of the promotion item; In response to a preset trigger operation on the video to be processed, a target video with embedded object information is generated.

[0012] Optionally, displaying the information editing interface of the promotional item includes: In response to a click operation on the promotional item, displaying the information editing box associated with the promotional item; The information editing box includes: an item name box, an item description box, and a jump indication box.

[0013] Optionally, performing object recognition on a target video frame in the video to be processed to extract the promotional item and the identification information of the promotional item includes: Inputting the target video frame into a pre-trained object recognition model to obtain the promoted object and structured recognition information; The identification information includes the item identifier of the promotion item, the first feature vector of the promotion item, the time point of the target video frame in the video to be processed, and the location area of ​​the promotion item in the target video frame.

[0014] Another aspect of an embodiment of the present application provides a video display device, the device comprising: A playback module, used to play the target video; An acquisition module, used to acquire the promotional items in the current playing screen of the target video when a preset condition is met; A display module, used to display a pop-up window of item information of the promoted item; The jump module is used to jump to the interactive page of the promotional item in response to the triggering operation on the item information pop-up window.

[0015] Another aspect of an embodiment of the present application provides a video generation device, the device comprising: An identification module, used to identify objects in a target video frame in a processed video to extract promotional objects and identification information of the promotional objects; A display module, used to display the information editing interface of the promotional items; A receiving module, used for receiving input information through the information editing interface, wherein the input information and the identification information are used to form the item information of the promotion item; The generating module is used to generate a target video with embedded object information in response to a preset triggering operation on the video to be processed.

[0016] Another aspect of an embodiment of the present application provides a computer device, including: at least one processor; and a memory communicatively coupled to the at least one processor; Wherein: the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method as described above.

[0017] Another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method described above is implemented.

[0018] Another aspect of an embodiment of the present application provides a computer program product, including a computer program, which implements the method described above when executed by a processor.

[0019] The embodiments of the present application using the above technical solution may include the following advantages: by obtaining the promotional items in the current playback screen of the target video when the preset conditions are met. Display a pop-up window of item information corresponding to the promotional item. In response to a click operation on the item information pop-up window, jump to the interactive page of the promotional item, for example, jump to the purchase page of the promotional item and guide the audience to buy. Different from the host's oral broadcast, this embodiment better integrates the product advertising with the original video content. When the preset conditions are met, at least part of the item information of the promotional item (such as the product advertising item) embedded in the current playback screen will be popped up in the form of a pop-up window in combination with the current playback screen, thereby realizing embedded non-intrusive item promotion and improving the audience's viewing experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings exemplarily illustrate the embodiments and constitute a part of the specification, and together with the text description of the specification, are used to explain the exemplary implementation of the embodiments. The embodiments shown are for illustrative purposes only and do not limit the scope of the claims. In all drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0021] Figure 1 A diagram schematically shows an operating environment of a video display method according to Embodiment 1 of the present application; Figure 2 A flowchart of a video display method according to Embodiment 1 of the present application is schematically shown; Figure 3 Schematically shows Figure 1 Flow chart of sub-steps of step S202; Figure 4 Schematically shows Figure 1 Flow chart of sub-steps of step S204; Figure 5 Schematically shows Figure 4 Flow chart of sub-steps of step S400; Figure 6 The flowchart of the video generation method according to the second embodiment of the present application is schematically shown; Fig. 7A and 7B The following schematically shows an exemplary application flow chart of the present application; Figure 8 A block diagram of a video display device according to a third embodiment of the present application is schematically shown; and Fig. 9 A block diagram schematically shows a video generating device according to a fourth embodiment of the present application; and Fig.10 The hardware architecture diagram of the computer device according to the fifth embodiment of the present application is schematically shown. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0023] It should be noted that the descriptions involving "first", "second", etc. in the embodiments of the present application are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0024] In the description of the present application, it should be understood that the numerical labels before the steps do not indicate the order in which the steps are executed, but are only used to facilitate the description of the present application and to distinguish each step, and therefore should not be understood as a limitation on the present application.

[0025] First, the following terms are explained: Video creation: the process of using various tools and techniques to shoot, edit, cut and process video materials to produce a complete video work.

[0026] Product promotion: a form of advertising that displays and promotes items through videos or live broadcasts to attract viewers to buy.

[0027] Embedded advertising: a form of advertising content that is subtly integrated into a video, movie or program without interrupting the audience's normal viewing of the video.

[0028] Voice-over advertising: Video creators create a segment introducing and recommending an item, and insert it into the video at the appropriate time to guide viewers to buy.

[0029] Secondly, in order to facilitate those skilled in the art to understand the technical solutions provided in the embodiments of the present application, the relevant technologies are described below: With the rapid development of Internet technology, product-driven advertising marketing has gradually gained favor among the general public. At present, one of the main forms of product-driven advertising is voice-over advertising. However, the applicant understands that: (1) the production cost and threshold of voice-over advertising are relatively high, requiring the creator to have attractive descriptions and explanations of the items. (2) Although voice-over advertising does not interrupt the playback of the entire video, it is difficult for the advertising content to be perfectly integrated with the original video, which will affect the audience's viewing experience.

[0030] To this end, the embodiments of the present application provide a video display technology solution and a video generation technology solution. In the above technical solutions: (1) In the video creation stage, the embodiments of the present application intelligently identify the items in the video that can be used for product advertising, and the creator can choose to add corresponding item details (descriptions, purchase links, etc.). (2) In the video playback stage, when the viewer pauses the video, the items in the pause frame are identified, and the previously marked item positions and corresponding item information are displayed on the screen. The viewer can choose to click to jump to purchase. The entire process has a very low threshold for creators to make advertisements, and the audience has a lossless viewing experience. See below for details.

[0031] Finally, for ease of understanding, an exemplary operating environment is provided below.

[0032] like Figure 1 As shown, the operating environment diagram includes: Figure 1 As shown, the operating environment diagram includes: a service platform 2, a client 4 and a video production terminal 8.

[0033] The service platform 2 can connect the client 4 and the video production terminal 8 through the network.

[0034] The service platform 2 may be a single server, a server cluster or a cloud computing service center.

[0035] The service platform 2 can provide video information storage services, video information reading services, etc. to the client.

[0036] The service platform 2 may be located in a data center such as a single location, or distributed in different geographical locations (e.g., in multiple locations). The service platform 2 may provide services via a network. The network includes various network devices such as routers, switches, multiplexers, hubs, modems, bridges, repeaters, firewalls, proxy devices, and / or the like. The network may include physical links such as coaxial cable links, twisted pair cable links, optical fiber links, combinations thereof, and the like, or wireless links such as cellular links, satellite links, Wi-Fi links, and the like.

[0037] The client 4 and the video production end 8 can be configured separately to access the content and services of the service platform 2. The client 4 and the video production end 8 can each separately include an electronic device carrying or externally connected to a display panel, such as a mobile device, a tablet device, a laptop computer, a workstation, a virtual reality device, a gaming device, a digital streaming device, a vehicle terminal, a smart TV, a set-top box, etc., and can also include a virtualized computing instance. The virtualized computing instance can include a virtual machine, such as a simulation of a computer system, an operating system, a server, etc. The computing device can load a virtual machine by the computing device based on a virtual image and / or other data defining specific software (e.g., an operating system, a dedicated application, a server) for simulation. As the demand for different types of processing services changes, different virtual machines can be loaded and / or terminated on one or more computing devices.

[0038] The client 4 may be associated with one or more viewers. A single viewer may also use one or more of the clients 4 to access the service platform 2. The client 4 may travel to various locations and use different networks to access the service platform 2.

[0039] The video production terminal 8 can be associated with one or more video creators. A single video creator can also use the video production terminal 8 to access the service platform 2. The video production terminal 8 can travel to various locations and use different networks to access the service platform 2. It should be noted that the client 4 and the video production terminal 8 can be the same or different devices.

[0040] The client / video production terminal 8 may include an interface. The interface may include a touch pad, a touch screen, a mouse, a keyboard, or other sensing elements. For example, the input element may be configured to receive audience instructions, which may enable the client to perform various operations, such as pausing a video, playing a video, generating (producing) a video, etc.

[0041] It should be noted that the above devices are exemplary, and the number and type of devices are adjustable in different scenarios or according to different needs.

[0042] Embodiment 1 This method embodiment can be executed in the client 4. In the following, the client 4 is taken as the execution subject of this process.

[0043] Figure 2 The flowchart of the video display method according to the first embodiment of the present application is schematically shown.

[0044] like Figure 2 As shown, the video display method may include steps S200 to S206, wherein: Step S200, playing the target video; Step S202, obtaining the promotional items in the current playing screen of the target video when a preset condition is met; Step S204, displaying a pop-up window of item information of the promoted item; Step S206, in response to the triggering operation on the item information pop-up window, jump to the interactive page of the promotion item.

[0045] The video display method provided in this embodiment can be applied to a client (such as 4A), by obtaining the promotional items in the current playback screen of the target video when the preset conditions are met. Display the item information pop-up window corresponding to the promotional item. In response to a click operation on the item information pop-up window, jump to the interactive page of the promotional item. For example, you can jump to the purchase page of the promotional item and guide the audience to buy. Different from the anchor's oral broadcast, this embodiment better integrates the product advertisement with the original video content. When the preset conditions are met, the name, description information and jump information (such as product advertisement) of the promotional item embedded in the current playback screen will be combined with the current playback screen to pop up in the form of a pop-up window, so as to realize embedded non-intrusive product promotion and improve the audience's viewing experience.

[0046] The following combination Figure 2 , each step in steps S200~S206 and other optional steps are described in detail.

[0047] Step S200 , play the target video.

[0048] The operation object (such as the audience) can play the target video through the web page, APP, etc. in the client 4A. The target video is a video created by the video creator using the video material he collected and uploaded to the video playback platform. The items in some / a certain video frame in the target video are pre-set as promotional items.

[0049] The target video can be transmitted to the client 4A via CDN or other means. The client provides a playback control interface to support viewers to perform operations such as pause, play, fast forward, fast rewind, and adjust the volume. For example, viewers can click a play button (such as a "play" icon, click a video cover, etc.) or trigger other playback-related viewer interactive operations to play the target video. Viewers can also adjust the playback status by clicking buttons or sliders on the playback control interface.

[0050] Step S202 , when a preset condition is met, obtaining the promotional items in the current playing screen of the target video.

[0051] When the target video is being created, preset conditions can be set.

[0052] The preset condition can be pause, trigger a specific button, enter a specific time point (such as the 30th second or 1st minute of the video), etc. The specific time point can be the climax of the video, the product display link, or the time point when the audience focuses their attention. In addition, specific patterns, colors, shapes, texts, or specific characters can be set as trigger conditions. When the target video plays to the screen containing these features, the recognition of the promotional items is triggered.

[0053] Promotional items refer to items that can be advertised and promoted, and can be various commodities or services such as clothing, furniture, daily necessities, televisions, etc. It should be noted that the promotional items in the current playback screen can be one or more.

[0054] In an optional embodiment, the current playing screen includes a video pause frame screen, and the step S202 can be implemented in the following manner: when the target video is paused, obtaining the promotional item in the video pause frame screen. The promotional item is pre-marked with item information. The item information includes name, location, description information and jump information.

[0055] Exemplarily, the promotional items are pre-tagged with item information, wherein the item information includes name, location, description information, jump information, etc. The item information of the promotional items may be pre-stored in the server. In some embodiments, the viewing history, interest preferences, and interactive behaviors of the audience can be analyzed by a machine learning algorithm to build a personalized promotional item recommendation model for the audience. When the target video is paused, the promotional items that the audience is most interested in are identified from the pause frame of the video and displayed first, thereby improving the audience's viewing experience. In other embodiments, the identification of promotional items can also be triggered according to the specific behavior of the audience. For example, when the audience actively drags the video progress bar to skip or review a certain part of the content, the video screen that the audience may be interested in can be analyzed, and the promotional items in these video screens can be displayed, so as to more accurately judge the current interest points of the audience, so as to display the relevant promotional items at the right time. This method reduces the insertion of advertisements when the audience is not interested or is concentrating on watching the video, and reduces the interference with the audience's viewing experience.

[0056] In an optional embodiment, if Figure 3 As shown, step S202 may include: Step S300, obtaining a first feature vector of the promotional item in the current playing picture from a server; Step S302, obtaining a second feature vector of each of one or more objects in the current playing picture; Step S304, matching the first feature vector with the second feature vector to obtain a matching result; Step S306: Determine the promotional items in the current playing picture according to the matching result.

[0057] Obtain and load the first feature vector of the promoted item from the server. The first feature vector may be the output result of the promoted item input into a pre-trained item recognition model. The first feature vector is used to represent the features of the promoted item. During video playback, the currently played frame image is captured at a certain frame rate (e.g., 30 frames per second). The captured frame image is preprocessed, including resizing, denoising, contrast enhancement, etc., to improve the accuracy of subsequent processing. The preprocessed frame image is scanned using an item recognition model (e.g., a target detection model based on deep learning) to obtain a second feature vector of one or more items. The second feature vector is used to obtain the features of one or more items in the currently played screen. The first feature vector is matched with the second feature vector of each item respectively. A similarity calculation method (e.g., cosine similarity) is used to evaluate the similarity between feature vectors. According to the set similarity threshold, the first feature vector that is highly similar to the second feature vector is screened out, and then the promoted item in the currently played screen is determined according to the matching value. In the case of multiple matching results, a priority strategy (e.g., position priority, etc.) can be used to determine the most likely promoted item.

[0058] In this embodiment, the promotion item can be accurately identified by acquiring the first feature vector of the promotion item from the server and matching the second feature vector of the item in the current playing picture according to the first feature vector of the promotion item.

[0059] Step S204 , a pop-up window of item information of the promoted item is displayed.

[0060] The item information pop-up window may include various information about the promoted item, such as name, description information, and jump information (such as item link, etc.). The description information may include key information such as the function, characteristics, material, size, etc. of the item, so as to help the audience quickly understand the item. In some embodiments, the item information pop-up window may also display high-definition product pictures or short videos. Pictures and videos can intuitively show the usage scenarios and effects of the item, etc., thereby enhancing the audience's perception and interest in the promoted item. If there are promotional activities (such as discounts, coupons, limited-time sales, etc.), the item information pop-up window may display these activity information to stimulate the audience's desire to buy. In order to facilitate the audience to buy quickly, a one-click purchase or add to shopping cart button may be set in the item information pop-up window. The audience only needs to click the button to complete the purchase or collection operation, simplifying the operation process. In order to provide a good browsing experience, the item information pop-up window may also provide an obvious close button to support the audience to close the item information pop-up window at any time when they are not interested. In some embodiments, the item information pop-up window may be displayed and hidden using smooth animation effects to increase the audience experience. The embodiment of the present application can not only improve the promotion efficiency by setting up a pop-up window for item information, but also provide the audience with comprehensive and timely item information, promotional information, etc., thereby stimulating the audience's desire to buy.

[0061] In an optional embodiment, if Figure 4 As shown, step S204 may include: Step S400, generating a highlight display frame according to the position of the promotional item, wherein the highlight display frame is used to indicate the corresponding position of the promotional item; Step S402: In response to a click operation on a corresponding position of the promotional item, a pop-up window of item information of the promotional item is popped up.

[0062] The highlight frame can use a color that contrasts sharply with the video background to achieve a striking effect. You can adjust the transparency of the highlight frame to reduce the obstruction of important parts of the video content. To further highlight the highlight frame, you can add borders and shadow effects to make it more three-dimensional. You can also configure the highlight frame to have a flashing animation effect. For example, by using a timer or animation framework to periodically adjust the display properties of the highlight frame (such as color brightness, border flashing, etc.) within a specified time period, you can effectively attract the user's attention.

[0063] In some embodiments, when the viewer clicks on the corresponding position of the promoted item, partial prompt information may be displayed first, such as preview information of the promoted item (such as brand, item introduction), without immediately popping up the item information pop-up window.

[0064] In this embodiment, a highlight display frame is generated, and the highlight display frame prompts the audience to click on a specific area to pop up a pop-up window of item information, making it easier for users to notice the promoted items and achieving highly targeted display.

[0065] In an optional embodiment, if Figure 5 As shown, step S400 may include: S500, obtaining the edge contour and position of the promotional item; S502, determining highlight display frame parameters based on the edge contour and position of the promotional item; wherein the highlight display frame parameters include: size, shape and / or position of the highlight display frame; S504: Draw the highlight display frame according to the highlight display frame parameters.

[0066] In some embodiments, the video frame can be converted from the original color space (such as RGB) to a color space (such as grayscale or HSV) that is more suitable for edge detection. Grayscale images can remove color information, making edge detection simpler and more efficient. HSV color space can separate color features and more accurately separate and identify color features of promotional items. Apply filtering algorithms such as Gaussian filtering, median filtering, etc. to the converted video frames to remove noise in the image, thereby reducing misjudgment during edge detection. In the case of low contrast of the video frame, the saturation of the video frame can be enhanced by histogram equalization, gamma correction, etc. to improve the clarity of the edge. Before edge detection, the approximate position of the promotional item in the video frame is located by a target detection algorithm (such as YOLO, SSD, etc. based on deep learning), thereby narrowing the scope of edge detection and improving processing speed and accuracy. According to the results of target detection, the regional image containing the promotional item is cropped to reduce the amount of edge detection calculations. Canny edge detectors, Sobel operators, Prewitt operators, etc. can be used to identify the edges of promotional items in video frames. To further improve accuracy, you can adjust the parameters of the edge detection algorithm, such as the standard deviation of the Gaussian filter, high and low thresholds, etc. After edge detection of the promoted items, thicker edge lines will be obtained. These thick lines can be refined into single-pixel width edges through edge thinning algorithms (such as non-maximum suppression, Hysteresis threshold processing, etc.). Find continuous edge pixels in the edge image and connect them into closed contours. For example, you can use contour tracking algorithms (such as Freeman chain code, Moore-Neighbor tracking, etc.).

[0067] The size of the promotional item is calculated based on the number of pixels of the edge contour of the promotional item or the size of the pixel area covered. The size of the highlight display frame is determined based on the size of the promotional item. In some embodiments, the highlight display frame can be configured to be slightly larger than the size of the promotional item so that the highlight display frame can completely cover the promotional item.

[0068] Get the shape features of the edge contour of the promotional item, such as rectangle, circle, irregular polygon, etc. Generate a highlight display frame of approximate shape based on these features. For irregular shaped items, the closest polygon shape can be selected for approximation. Alternatively, the highlight display frame can be generated by calculating the minimum circumscribed rectangle of the edge contour of the promotional item.

[0069] A highlight display box can be drawn on the video frame through a graphics drawing library (such as HTML5 Canvas, OpenGL, DirectX, etc.).

[0070] In this embodiment, by extracting the edge contour of the promoted item, a highlight display frame similar to the edge contour of the item is determined and drawn, so that the highlight display frame will not block important parts of the item and can effectively attract the user's attention.

[0071] Step S206 , in response to a triggering operation on the item information pop-up window, jump to the interactive page of the promoted item.

[0072] The triggering operation for the item information pop-up window may be a click operation, a voice command, a screen sliding gesture, an air gesture, etc.

[0073] The interactive page may be a product purchase page, a service purchase page, etc.

[0074] In some embodiments, the item information pop-up window may include a "Buy" button, which will trigger the page to jump to the product payment page or enter the purchase page after the viewer clicks it. The item information pop-up window may also include a "View Details" button, which will trigger the page to jump to the detailed interactive page of the promoted item after the viewer clicks it. The item information pop-up window may also include other clickable areas, such as "View More Reviews", "View Instructions for Use", etc. Clicking these areas will jump to the corresponding interactive page. The item information pop-up window can also set other interactive methods. For example, you can slide to view more details, rotate to view 3D models, use augmented reality (AR) technology to preview products, etc. These interactive methods can improve the efficiency of interaction and enhance the fun and interactivity of advertisements.

[0075] The video display method provided in this embodiment can be applied to a client (such as 4A), by obtaining the promotional items in the current playback screen of the target video when the preset conditions (such as pause) are met. Display the item information pop-up window corresponding to the promotional item. In response to a click operation on the item information pop-up window, jump to the interactive page of the promotional item. For example, you can jump to the purchase page of the promotional item and guide the audience to buy. Different from the anchor's oral broadcast, this embodiment better integrates the product advertisement with the original video content. When the preset conditions are met, the name, description information and jump information (such as product advertisement) of the promotional item embedded in the current playback screen will be combined with the current playback screen. In the form of a pop-up window, it realizes embedded non-intrusive item promotion and improves the audience's viewing experience.

[0076] Embodiment 2 This method embodiment can be executed in the video production terminal 8. The video production terminal 8 is taken as the execution subject of this process.

[0077] Figure 6 The flowchart of the video generation method according to the second embodiment of the present application is schematically shown.

[0078] like Figure 6 As shown, the video generation method may include steps S600 to S606, wherein: Step S600, performing object recognition on the target video frame in the video to be processed to extract the promotional item and the identification information of the promotional item; Step S602, displaying the information editing interface of the promotional item; Step S604, receiving input information through the information editing interface, wherein the input information and the identification information are used to form item information of the promotion item; Step S606: In response to a preset trigger operation for the video to be processed, a target video with embedded object information is generated.

[0079] The video generation method provided in this embodiment performs object recognition on the target video frame in the video to be processed to obtain the identification information of the object. An information editing interface for the object is displayed. Input information from the video creator is received through the information editing interface. The input information and the identification information are used to form the item information of the promoted item. In response to a preset trigger operation for the video to be processed, a target video with embedded item information is generated (i.e., a target video with embedded advertisements is generated), thereby solving the problem of high production costs and thresholds for traditional voice-over advertisements, and better integrating the advertisement content with the original video.

[0080] The following combination Figure 6 , each step in steps S600~S606 and other optional steps are described in detail.

[0081] Step S600 , object recognition is performed on the target video frame in the processed video to extract the promotional items and the identification information of the promotional items.

[0082] The video to be processed may include daily life recording videos, object usage experience sharing videos, talent show videos, teaching videos, movie clips, TV series clips, live broadcast clips, etc. In some embodiments, the target video frames and video frames within the target time period that need to be identified can be selected according to the input of the video creator, or they can be identified frame by frame. The target video frame can be a specific time point, a specific scene, or an automatically detected key frame in the video. Use an artificial intelligence model (such as a convolutional neural network CNN) to perform object identification on the selected video frames. Input the target video frame into the pre-trained model, and the model will output identification information. An exemplary scheme is provided below.

[0083] In an optional embodiment, step S600 may further include: inputting the target video frame into a pre-trained item recognition model to obtain the promoted item and structured recognition information, wherein the recognition information includes the item identifier of the promoted item, the time point of the target video frame in the video to be processed, and the location area of ​​the promoted item in the target video frame.

[0084] Exemplarily, the object recognition module can be a convolutional neural network (CNN). CNN automatically extracts features in the image, such as edges, textures, etc., through multiple layers of convolutional layers, and increases the nonlinear ability of the network through nonlinear activation functions such as ReLU. Subsequently, the feature map is downsampled through the pooling layer to reduce the amount of calculation and the number of parameters, while maintaining the spatial hierarchy of the features. After the features extracted by the convolution layer and the pooling layer are flattened, they are input into the fully connected layer for classification or regression. In this embodiment, by inputting the target video frame into the CNN, through multiple layers of convolution, pooling and full connection operations, the item identification, location area and item features of the promoted item are finally output. The model can be trained with a large amount of data to improve the recognition ability of the item recognition model. It can also identify various promotional items in complex video scenes, such as clothing, electronic products, food, etc. In addition, other item recognition models can also be used, such as ResNet, EfficientNet, etc.

[0085] In this embodiment, the pre-trained object recognition model can automatically and efficiently recognize the identification information of the objects in the video, thereby improving the processing speed and accuracy.

[0086] Step S602 , displaying the information editing interface of the promotional item.

[0087] After the promotional item is identified and the identification information is obtained, the information editing interface of the promotional item is displayed. On the information editing interface, interactive methods such as dragging and clicking can be added to enable the video creator to sort, add or delete the item information. In some embodiments, an intelligent recommendation function can be provided for the video creator, such as recommending relevant item descriptions, keywords, pictures, etc., to reduce the input of the video creator and improve the editing efficiency. For common items or brands, a preset template framework can be used, and the video creator only needs to make fine adjustments on this basis to quickly complete the editing. A real-time preview function can also be provided. Each piece of information edited by the video creator can be instantly reflected in the preview area, allowing the video creator to intuitively see the editing effect. At the same time, a variety of preview modes (such as full screen, thumbnail, etc.) can also be configured to meet the needs of different scenarios. In some embodiments, the basic information of the promotional item (such as name, category) can be automatically filled into the editing box according to the recognition results, reducing the workload of the video creator's manual input. It should be noted that the information editing interface can be displayed immediately after the promotional item is identified. The information editing interface can also be displayed only when the preset conditions are met. An exemplary solution is provided below.

[0088] In an optional embodiment, step S602 may further include: in response to a click operation on the promotional item, displaying an information editing box associated with the promotional item, wherein the information editing box may include: an item name box, an item description box, and a jump indication box.

[0089] Exemplarily, the information editing interface may be an information editing box. The information editing box may be a template set by the system by default. The information editing box may also be a template automatically generated based on information such as item category, historical sales data, etc. For example, when a video creator clicks on a promotional item, the category of the item (such as clothing, electronic products, household items, etc.) is first identified. The layout and elements of the information editing box are adjusted according to the item category. For example, for clothing items, the information editing box may additionally include a "size information box", "color selection box", etc. For electronic products, it may include a "technical parameter box", "evaluation summary box", etc. It is also possible to generate a corresponding information editing box based on historical sales data by analyzing the market demand focus of the item or similar items during the sales process.

[0090] Step S604 , receiving input information through the information editing interface, and the input information and the identification information are used to form the item information of the promotion item.

[0091] The video creator provides detailed information about the promoted item through various interactive methods of the information editing interface (such as text box input, drop-down selection, dragging pictures, etc.). These detailed information may include a detailed description of the item, specification parameters, usage scenarios, precautions, price range, promotional information, etc. The input information can be entered by the video creator or automatically supplemented by the system. For example, if historical data shows that the camera performance of a certain mobile phone is the focus of audience attention, the system can automatically insert or highlight the description information related to the camera in the "Item Description Box". After the video creator completes the input information and saves it, the video creator's input information will be obtained from the information editing interface and integrated with the previously identified item identification information to obtain the item information of the promoted item.

[0092] Step S606 , in response to a preset trigger operation for the video to be processed, generating a target video with embedded object information.

[0093] After the video creator completes entering the detailed information of the promotional item through the information editing interface and confirms that it is correct, a target video with embedded item information can be generated when the preset trigger operation is met, that is, a target video with embedded advertising is generated. The preset trigger operation can be the video creator clicking the "Generate Video" button, automatic triggering after completing the editing, or any other predefined interactive action of the video creator. After the target video is generated, a preview function can be provided so that the video creator can check whether the embedded effect of the information is satisfactory. If the video creator is not satisfied, he can return to the information editing interface to make changes and re-trigger the generation process of the target video. After the video creator completes the editing, the target video can be uploaded to the service platform.

[0094] In order to make the present application easier to understand, an exemplary application is provided below in conjunction with FIG. 7 .

[0095] Target video generation (production) stage S700, obtaining a video to be processed, which is composed of multiple video frames.

[0096] S702, identifying one or more objects in the target video frame of the video to be processed by an object recognition model to obtain structured identification information of the objects, wherein the identification information includes a feature vector, an object identifier, a location region in the target video frame, and the like.

[0097] S704, displaying the obtained structured identification information to the video creator.

[0098] S706, the video creator selects a promotional item from one or more items.

[0099] S708, after the video creator clicks, the information editing interface of the promotional item is displayed.

[0100] S710, the video creator inputs information (eg, item description, purchase link, etc.) into the information editing interface. The identification information of the promoted item is integrated with the input information to obtain item information of the promoted item, and the item information is uploaded to the server.

[0101] S712, generating a target video associated with the above-mentioned item information, that is, obtaining a target video with an embedded advertisement, and publishing it.

[0102] Target video playback (viewing) stage S714, when the viewer watching the target video clicks pause, obtaining a second feature vector of the object in the current video screen (video pause frame screen).

[0103] S716, obtaining a first feature vector of the promotional item from the server, matching the first feature vector with the second feature vector to obtain a matching value, and determining the promotional item in the pause frame of the video when the matching value is greater than a preset value.

[0104] S718, drawing a highlight display frame according to the edge contour of the promotional item to prompt the audience to the corresponding position of the promotional item in the pause frame of the video.

[0105] S720, in response to the viewer clicking on the promoted item, the item information of the promoted item is obtained from the server, thereby popping up an item information pop-up window.

[0106] S722, when the viewer clicks on the item information pop-up window, the viewer will be redirected to the interactive page (such as the purchase page) of the promoted item.

[0107] Embodiment 3 Figure 8 The block diagram of the video display device according to the third embodiment of the present application is schematically shown. The device can be divided into one or more program modules, one or more program modules are stored in a storage medium and executed by one or more processors to complete the embodiment of the present application. The program module referred to in the embodiment of the present application refers to a series of computer program instruction segments that can complete specific functions. The following description will specifically introduce the functions of each program module in this embodiment. Figure 8 As shown, the device 1000 may include: a playing module 1100, an acquisition module 1200, a display module 1300, and a jump module 1400, wherein: Playing module 1100, used for playing the target video; An acquisition module 1200 is used to acquire the promotional items in the current playing screen of the target video when a preset condition is met; The display module 1300 is used to display a pop-up window of item information of the promotion item; The jump module 1400 is used to jump to the interactive page of the promotional item in response to the triggering operation on the item information pop-up window.

[0108] As an optional embodiment, the current playing screen includes a video pause frame screen; when a preset condition is met, obtaining the promotional item in the current playing screen in the target video includes: When the target video is paused, the promotional item in the pause frame of the video is obtained; the promotional item is pre-marked with item information, and the item information includes name, location, description information and jump information.

[0109] As an optional embodiment, displaying a pop-up window of item information of the promoted item includes: Generating a highlight display frame according to the position of the promotional item, wherein the highlight display frame is used to indicate the corresponding position of the promotional item; In response to a click operation on a corresponding position of the promotional item, a pop-up window of item information of the promotional item is popped up.

[0110] As an optional embodiment, generating a highlight display frame according to the position of the promotional item includes: Obtaining the edge contour and position of the promotional item; Determine the highlight display frame parameters based on the edge contour and position of the promotional item; wherein the highlight display frame parameters include: the size, shape and / or position of the highlight display frame; The highlight display frame is drawn according to the highlight display frame parameters.

[0111] As an optional embodiment, obtaining the promotional item in the current playing screen of the target video includes: Acquire a first feature vector of the promotional item in the current playing picture from a server; Obtaining a second feature vector of each of the one or more objects in the current playing picture; Matching the first feature vector and the second feature vector to obtain a matching result; According to the matching result, the promotional items in the current playing picture are determined.

[0112] Embodiment 4 Fig. 9The block diagram of the video generation device according to the fourth embodiment of the present application is schematically shown. The device can be divided into one or more program modules, one or more program modules are stored in a storage medium and executed by one or more processors to complete the embodiment of the present application. The program module referred to in the embodiment of the present application refers to a series of computer program instruction segments that can complete specific functions. The following description will specifically introduce the functions of each program module in this embodiment. Fig. 9 As shown, the device 2000 may include: an identification module 2100, a display module 2200, a receiving module 2300, and a generating module 2400, wherein: The recognition module 2100 is used to perform object recognition on the target video frame in the video to be processed to extract the promotional item and the identification information of the promotional item; Display module 2200, used to display the information editing interface of the promotional item; A receiving module 2300, configured to receive input information through the information editing interface, wherein the input information and the identification information are used to form item information of the promotion item; The generating module 2400 is used to generate a target video with embedded object information in response to a preset triggering operation on the video to be processed.

[0113] As an optional embodiment, the information editing interface of the promotional item is displayed, including: In response to a click operation on the promotional item, displaying the information editing box associated with the promotional item; The information editing box includes: an item name box, an item description box, and a jump indication box.

[0114] As an optional embodiment, object recognition is performed on a target video frame in the video to be processed to extract the promotional item and the identification information of the promotional item, including: The target video frame is input into a pre-trained item recognition model to obtain the promoted item and structured recognition information; wherein the recognition information includes the item identifier of the promoted item, the first feature vector of the promoted item, the time point of the target video frame in the video to be processed, and the location area of ​​the promoted item in the target video frame.

[0115] Embodiment 5 Fig.10The schematic diagram of the hardware architecture of a computer device 10000 suitable for implementing the video display / generation method according to the fifth embodiment of the present application is shown schematically. In some embodiments, the computer device 10000 may be a terminal device such as a smart phone, a wearable device, a tablet computer, a personal computer, a vehicle terminal, a game console, a virtual device, a workbench, a digital assistant, a set-top box, a robot, etc. In other embodiments, the computer device 10000 may be a rack server, a blade server, a tower server, or a cabinet server (including an independent server, or a server cluster composed of multiple servers), etc. Fig.10 As shown, the computer device 10000 includes but is not limited to: a memory 10010, a processor 10020, and a network interface 10030 that can communicate with each other through a system bus. Among them: The memory 10010 includes at least one type of computer-readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (such as SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 10010 can be an internal storage module of the computer device 10000, such as a hard disk or memory of the computer device 10000. In other embodiments, the memory 10010 can also be an external storage device of the computer device 10000, such as a plug-in hard disk equipped on the computer device 10000, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the memory 10010 can also include both the internal storage module of the computer device 10000 and its external storage device. In this embodiment, the memory 10010 is generally used to store the operating system and various application software installed in the computer device 10000, such as program codes of the video display / generation method, etc. In addition, the memory 10010 can also be used to temporarily store various data that have been output or will be output.

[0116] In some embodiments, the processor 10020 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other chips. The processor 10020 is generally used to control the overall operation of the computer device 10000, such as performing control and processing related to data interaction or communication with the computer device 10000. In this embodiment, the processor 10020 is used to run the program code stored in the memory 10010 or process data.

[0117] The network interface 10030 may include a wireless network interface or a wired network interface, and the network interface 10030 is generally used to establish a communication link between the computer device 10000 and other computer devices. For example, the network interface 10030 is used to connect the computer device 10000 to an external terminal through a network, and to establish a data transmission channel and a communication link between the computer device 10000 and the external terminal. The network may be a wireless or wired network such as an intranet, the Internet, the Global System of Mobile communication (GSM), Wideband Code Division Multiple Access (WCDMA), 4G network, 5G network, Bluetooth, Wi-Fi, etc.

[0118] It should be pointed out that Fig.10 Only a computer device having components 10010 - 10030 is shown, but it should be understood that implementation of all of the components shown is not a requirement, and more or fewer components may alternatively be implemented.

[0119] In this embodiment, the video display / generation method stored in the memory 10010 can also be divided into one or more program modules and executed by one or more processors (such as processor 10020) to complete the embodiment of the present application.

[0120] Embodiment 6 An embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the video display / generation method in the embodiment.

[0121] In this embodiment, the computer-readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the computer-readable storage medium can be an internal storage unit of a computer device, such as a hard disk or memory of the computer device. In other embodiments, the computer-readable storage medium can also be an external storage device of a computer device, such as a plug-in hard disk equipped on the computer device, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the computer-readable storage medium can also include both the internal storage unit of the computer device and its external storage device. In this embodiment, the computer-readable storage medium is generally used to store an operating system and various application software installed on the computer device, such as the program code of the video display / generation method in the embodiment. In addition, the computer-readable storage medium can also be used to temporarily store various types of data that have been output or are to be output.

[0122] Embodiment 7 An embodiment of the present application also provides a computer program product, including a computer program, which implements the method in the above embodiment when executed by a processor.

[0123] Obviously, those skilled in the art should understand that the modules or steps of the above-mentioned embodiments of the present application can be implemented by general-purpose computer devices, they can be concentrated on a single computer device, or distributed on a network composed of multiple computer devices, optionally, they can be implemented by executable program codes of computer devices, so that they can be stored in a storage device and executed by the computer device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the embodiments of the present application are not limited to any specific combination of hardware and software.

[0124] It should be noted that the above are only preferred embodiments of the present application, and the patent protection scope of the present application is not limited thereto. Any equivalent structure or equivalent process transformation made using the contents of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A video display method, It is characterized in that The method comprises: Play the target video; When a preset condition is met, obtaining a promotional item in a currently playing screen of the target video; Displaying a pop-up window of item information of the promotional item; and In response to a triggering operation on the item information pop-up window, jump to an interactive page of the promoted item.

2. The method according to claim 1, It is characterized in that The current playing screen includes a video pause frame screen; when a preset condition is met, obtaining the promotional item in the current playing screen in the target video includes: When the target video is paused, the promotional item in the pause frame of the video is obtained; the promotional item is pre-marked with item information, and the item information includes name, location, description information and jump information.

3. The method according to claim 1, It is characterized in that A pop-up window showing item information of the promoted item is displayed, including: Generating a highlight display frame according to the position of the promotional item, wherein the highlight display frame is used to indicate the corresponding position of the promotional item; In response to a click operation on a corresponding position of the promotional item, a pop-up window of item information of the promotional item is popped up.

4. The method according to claim 3, It is characterized in that Generate a highlight display frame according to the position of the promotional item, including: Obtaining the edge contour and position of the promotional item; Determine the highlight display frame parameters based on the edge contour and position of the promotional item; wherein the highlight display frame parameters include: the size, shape and / or position of the highlight display frame; The highlight display frame is drawn according to the highlight display frame parameters.

5. The method according to claim 1, It is characterized in that Obtaining the promotional items in the current playing screen of the target video, including: Acquire a first feature vector of the promotional item in the current playing picture from a server; Obtaining a second feature vector of each of the one or more objects in the current playing picture; Matching the first feature vector and the second feature vector to obtain a matching result; According to the matching result, the promotional items in the current playing picture are determined.

6. A video generation method, It is characterized in that The method comprises: Performing object recognition on a target video frame in the video to be processed to extract promotional items and identification information of the promotional items; Displaying the information editing interface of the promotional item; receiving input information through the information editing interface, wherein the input information and the identification information are used to form item information of the promotion item; In response to a preset trigger operation on the video to be processed, a target video with embedded object information is generated.

7. The method according to claim 6, It is characterized in that The information editing interface for displaying the promotional items includes: In response to a click operation on the promotional item, displaying the information editing box associated with the promotional item; The information editing box includes: an item name box, an item description box, and a jump indication box.

8. The method according to claim 6, It is characterized in that Performing object recognition on the target video frame in the processed video to extract the promotional item and the identification information of the promotional item, including: Inputting the target video frame into a pre-trained object recognition model to obtain the promoted object and structured recognition information; The identification information includes the item identifier of the promotion item, the first feature vector of the promotion item, the time point of the target video frame in the video to be processed, and the location area of ​​the promotion item in the target video frame.

9. A video generating device, It is characterized in that The device comprises: A playback module, used to play the target video; An acquisition module, used to acquire the promotional items in the current playing screen of the target video when a preset condition is met; A display module, used to display a pop-up window of item information of the promoted item; The jump module is used to jump to the interactive page of the promotional item in response to the triggering operation on the item information pop-up window.

10. A video generating device, It is characterized in that The device comprises: An identification module, used to identify objects in a target video frame in a processed video to extract promotional objects and identification information of the promotional objects; A display module, used to display the information editing interface of the promotional items; A receiving module, used for receiving input information through the information editing interface, wherein the input information and the identification information are used to form the item information of the promotion item; The generating module is used to generate a target video with embedded object information in response to a preset triggering operation on the video to be processed.

11. A computer device, It is characterized in that include: at least one processor; and a memory communicatively connected to the at least one processor; wherein: The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.

12. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

13. A computer program product comprising a computer program, It is characterized in that When the computer program is executed by a processor, the steps of the method according to claims 1 to 8 are implemented.

Images