A method and system for automated ad recommendation based on video content awareness

Abstract

This invention discloses an automated advertising recommendation method and system based on video content awareness, relating to the field of image processing. The system includes a recommendation center communicatively connected to a video data acquisition module, a video data processing module, a feature extraction module, and an ad matching module. The video data acquisition module collects video data viewed by the user. The video data processing module processes the collected video data to obtain a noise-free reference video image. The feature extraction module extracts features from the noise-free reference video image to obtain its characteristics. The ad matching module performs ad matching based on the characteristics of the noise-free reference video image and recommends matched ads to the user. Furthermore, the system recommends ads based on the type of video viewed by the user, significantly improving ad click-through rates.

Description

Technical Field

[0001] This invention relates to the field of image processing, specifically to an automated advertising recommendation method and system based on video content awareness. Background Technology

[0002] With the development of internet technology and the popularization of broadband networks, video content has become a major component of internet traffic; the number of users and viewing time of video streaming platforms are constantly growing, which has brought huge market opportunities for advertising; traditional advertising methods are usually based on users' browsing history, search records and demographic information, and the advertising content is not very relevant to the video content that users are watching, which can easily arouse user resentment, and the accuracy of advertising is not high, making it difficult to maximize the advertising effect.

[0003] How to obtain corresponding advertisements based on the video content watched by users, and insert the obtained advertisements into the video content watched by users to improve the accuracy of advertisement delivery and enhance the user experience is a problem we need to solve. To this end, we now provide an automated advertisement recommendation method and system based on video content perception. Summary of the Invention

[0004] To address the aforementioned technical problems, the present invention aims to provide an automated advertising recommendation method and system based on video content perception.

[0005] The objective of this invention can be achieved through the following technical solution: an automated advertising recommendation system based on video content perception, comprising a recommendation center, wherein the recommendation center is communicatively connected to a video data acquisition module, a video data processing module, a feature extraction module, and an advertising matching module;

[0006] The video data acquisition module is used to collect video data watched by the user.

[0007] The video data processing module is used to process the video data collected from the user to obtain a reference video image after removing noise.

[0008] The feature extraction module is used to extract features from the obtained noise-free reference video image to obtain the features of the noise-free reference video image.

[0009] The ad matching module is used to perform ad matching based on the features of the obtained reference video image after removing noise, and recommend the matched ads to the user.

[0010] Furthermore, the process by which the video data acquisition module acquires video data viewed by the user includes:

[0011] The video data acquisition module consists of several video data acquisition terminals;

[0012] By configuring each video data acquisition terminal and generating a corresponding configuration channel, and connecting to the corresponding server through the generated configuration channel, the video data acquisition terminal obtains the video data watched by the user on the corresponding server and sends the obtained video data watched by the user to the video data processing module.

[0013] Furthermore, the process by which the video data processing module processes the collected video data watched by the user includes:

[0014] Frames are extracted from the video data watched by the user to obtain several frames of video data, and each frame of video data is recorded as a reference video image.

[0015] Set a Gaussian filter, input the reference video image into the Gaussian filter, the Gaussian filter processes the reference video image to remove noise in the reference video image, while retaining the overall structure and edge information of the reference grayscale video image, the Gaussian filter outputs, and obtains the reference video image after removing the influence of noise.

[0016] Furthermore, the feature extraction module performs feature extraction on the obtained noise-removed reference video image, including:

[0017] The feature extraction module contains a feature extraction model, which consists of an input layer, a scene extraction layer, an edge layer, an object extraction layer, a person extraction layer, and an output layer.

[0018] The noise-removed reference video image is input into the feature extraction model. The reference video image reaches the output layer of the feature extraction model. The input layer of the feature extraction model sends the reference video image to the scene extraction layer of the feature extraction model.

[0019] The scene extraction layer stores several scene images, each scene image representing a scene. The scenes include a first scene, a second scene, a third scene, and so on. The reference video image is matched with the scene images stored in the scene extraction layer to obtain the scene in the reference video image.

[0020] The scene extraction layer in the feature extraction model sends the reference video image to the edge layer and the scene in the reference video image to the output layer;

[0021] The edge layer processes the reference video image to obtain an edge reference video grayscale image;

[0022] The edge layer sends the edge reference video grayscale image to the object extraction layer. The object extraction layer stores several edge object grayscale images, each edge object grayscale image representing an object, including a first object, a second object, a third object, and so on. The edge reference video grayscale image is matched with the edge object grayscale images stored in the object extraction layer to obtain the object in the edge reference video grayscale image.

[0023] The object extraction layer sends the objects in the edge reference video grayscale image to the output layer, and sends the edge reference video grayscale image to the person extraction layer;

[0024] The character extraction layer stores several edge character grayscale images, each edge character grayscale image representing a character, including a first character, a second character, a third character, and so on. The edge reference video grayscale image is matched with the edge character grayscale images stored in the character extraction layer to obtain the character in the edge reference video grayscale image.

[0025] The person extraction layer sends the person from the edge reference video grayscale image to the output layer;

[0026] The output layer outputs the scene, objects, and people from the reference video image.

[0027] Furthermore, the edge layer's processing of the reference video image includes:

[0028] The edge layer converts the reference video image into a reference video grayscale image, obtains the gradient magnitude of the pixels in the reference video grayscale image through the Sobel operator, and sets the gradient magnitude threshold range, which is denoted as (Y0, Y1).

[0029] If the gradient magnitude of a pixel in the reference video grayscale image is greater than the maximum value in the gradient magnitude threshold range, the pixel is marked as a first edge pixel. If the gradient magnitude of a pixel is less than or equal to the maximum value in the gradient magnitude threshold range, but greater than or equal to the minimum value in the gradient magnitude threshold range, the pixel is marked as a second edge pixel. If the gradient magnitude of a pixel is less than the minimum value in the gradient magnitude threshold range, the pixel is marked as a non-edge pixel.

[0030] Get the pixels in the top, bottom, left, right and four diagonal directions of the second edge pixel. If the first edge pixel exists among the obtained pixels, then mark the second edge pixel as the first edge pixel.

[0031] If the first edge pixel does not exist among the acquired pixels, then the second edge pixel is marked as a non-edge pixel;

[0032] Set the brightness value of the first edge pixel to "255" to obtain the edge reference video grayscale image.

[0033] Furthermore, the process of ad matching by the ad matching module based on the features of the obtained noise-removed reference video image includes:

[0034] The most frequently occurring scene, object, and person in the reference video image is identified. The most frequently occurring scene in the acquired reference video image is identified as the reference scene, the most frequently occurring object in the acquired reference video image is identified as the reference object, and the most frequently occurring person in the acquired reference video image is identified as the reference person.

[0035] Construct an ad matching chain, which consists of scene nodes, object nodes, character nodes, and ad nodes;

[0036] Connect scene nodes to object nodes, connect object nodes to character nodes, and connect character nodes to advertisement nodes.

[0037] Input the reference scene into the scene node, input the reference object into the object node, and input the reference person into the person node to obtain the reference advertisement matching chain;

[0038] The ad matching module is equipped with an ad matching chain database. Based on the reference ad matching chain and the ad matching chain database, it obtains the ads recommended to the user and inserts the obtained ads into the video watched by the user.

[0039] Furthermore, the process of obtaining recommended ads to users based on the reference ad matching chain and the ad matching chain database includes:

[0040] The advertising matching chain database stores several advertising matching chains. Based on historical advertising push data, it obtains advertisements corresponding to different scenarios, objects, and people, and inputs the advertisements corresponding to different scenarios, objects, and people into the advertising matching chain.

[0041] The obtained reference ad matching chain is compared with the ad matching chains stored in the ad matching chain database. Ad matching chains whose content in scene nodes, object nodes, and character nodes is the same as the content in scene nodes, object nodes, and character nodes in the reference ad matching chain are recorded as standard ad matching chains. Ads in the ad nodes of the standard ad matching chains are then retrieved. The retrieved ads are the ads recommended to the user.

[0042] This invention also discloses a recommendation method for an automated advertising recommendation system based on video content awareness, comprising the following steps:

[0043] Step 1: Obtain video data watched by the user;

[0044] Step 2: Process the acquired video data viewed by the user to obtain a reference video image after removing noise.

[0045] Step 3: Extract features from the noise-removed reference video image to obtain its features.

[0046] Step 4: Perform ad matching based on the features of the obtained noise-removed reference video image, and recommend the matched ads to the user.

[0047] Compared with existing technologies, the beneficial effects of this invention are as follows: It collects video data viewed by users, processes the collected video data to obtain a reference video image after removing noise, extracts features from the obtained reference video image after removing noise, performs ad matching based on the features of the obtained reference video image after removing noise, and recommends matched ads to users. It also recommends corresponding ads to users based on the type of video they watch, greatly improving ad click-through rates, increasing the accuracy of ad placement, maximizing ad effectiveness, and enhancing user experience. Attached Figure Description

[0048] Figure 1 This is a schematic diagram of the present invention. Detailed Implementation

[0049] like Figure 1 As shown, an automated advertising recommendation system based on video content perception includes a recommendation center, which is communicatively connected to a video data acquisition module, a video data processing module, a feature extraction module, and an advertising matching module.

[0050] The video data acquisition module is used to collect video data watched by the user.

[0051] It should be further explained that, in the specific implementation process, the process by which the video data acquisition module collects the video data watched by the user includes:

[0052] The video data acquisition module consists of several video data acquisition terminals;

[0053] By configuring each video data acquisition terminal and generating a corresponding configuration channel, and connecting to the corresponding server through the generated configuration channel, the video data acquisition terminal obtains the video data watched by the user on the corresponding server and sends the obtained video data watched by the user to the video data processing module.

[0054] The video data processing module is used to process the video data collected from the user to obtain a reference video image after removing noise.

[0055] It should be further explained that, in the specific implementation process, the video data processing module processes the collected video data watched by the user, including:

[0056] Frame extraction is performed on the collected video data watched by users to obtain several frames of video data. Each frame of video data is recorded as a reference video image and labeled as j, j = 1, 2, 3, ..., m, where m is a positive integer.

[0057] Set a Gaussian filter, input the reference video image into the Gaussian filter, the Gaussian filter processes the reference video image to remove noise in the reference video image, while retaining the overall structure and edge information of the reference grayscale video image, the Gaussian filter outputs, and obtains the reference video image after removing the influence of noise.

[0058] The feature extraction module is used to extract features from the obtained noise-free reference video image to obtain the features of the noise-free reference video image.

[0059] It should be further explained that, in the specific implementation process, the feature extraction module performs feature extraction on the obtained reference video image after removing the influence of noise, including:

[0060] The feature extraction module contains a feature extraction model, which consists of an input layer, a scene extraction layer, an edge layer, an object extraction layer, a person extraction layer, and an output layer.

[0061] The input layer is used to input a reference video image, the scene extraction layer is used to extract the scene in the reference video image, the edge layer is used to extract the edge information in the reference video image, the object extraction layer is used to extract the object in the reference video image, the person extraction layer is used to extract the person in the reference video image, and the output layer is used to output the features of the reference video image.

[0062] The noise-removed reference video image is input into the feature extraction model. The reference video image reaches the output layer of the feature extraction model. The input layer of the feature extraction model sends the reference video image to the scene extraction layer of the feature extraction model.

[0063] The scene extraction layer stores several scene images, each scene image representing a scene. The scenes include a first scene, a second scene, a third scene, and so on. The reference video image is matched with the scene images stored in the scene extraction layer to obtain the scene in the reference video image.

[0064] The scene extraction layer in the feature extraction model sends the reference video image to the edge layer and the scene in the reference video image to the output layer;

[0065] The edge layer converts the reference video image into a reference video grayscale image, obtains the gradient magnitude of the pixels in the reference video grayscale image through the Sobel operator, and sets the gradient magnitude threshold range, which is denoted as (Y0, Y1).

[0066] If the gradient magnitude of a pixel in the reference video grayscale image is greater than the maximum value in the gradient magnitude threshold range, the pixel is marked as a first edge pixel. If the gradient magnitude of a pixel is less than or equal to the maximum value in the gradient magnitude threshold range, but greater than or equal to the minimum value in the gradient magnitude threshold range, the pixel is marked as a second edge pixel. If the gradient magnitude of a pixel is less than the minimum value in the gradient magnitude threshold range, the pixel is marked as a non-edge pixel.

[0067] Get the pixels in the top, bottom, left, right and four diagonal directions of the second edge pixel. If there is a first edge pixel among the obtained pixels, then mark the second edge pixel as the first edge pixel.

[0068] If the first edge pixel does not exist among the acquired pixels, then the second edge pixel is marked as a non-edge pixel;

[0069] Set the brightness value of the first edge pixel to 255 to obtain the edge reference video grayscale image;

[0070] The edge layer sends the edge reference video grayscale image to the object extraction layer. The object extraction layer stores several edge object grayscale images, each edge object grayscale image representing an object, including a first object, a second object, a third object, and so on. The edge reference video grayscale image is matched with the edge object grayscale images stored in the object extraction layer to obtain the object in the edge reference video grayscale image.

[0071] The object extraction layer sends the objects in the edge reference video grayscale image to the output layer, and sends the edge reference video grayscale image to the person extraction layer;

[0072] The character extraction layer stores several edge character grayscale images, each edge character grayscale image representing a character, including a first character, a second character, a third character, and so on. The edge reference video grayscale image is matched with the edge character grayscale images stored in the character extraction layer to obtain the character in the edge reference video grayscale image.

[0073] The person extraction layer sends the person from the edge reference video grayscale image to the output layer;

[0074] The output layer outputs the scene, objects, and people from the reference video image.

[0075] The ad matching module is used to perform ad matching based on the features of the obtained reference video image after removing noise, and recommend the matched ads to the user.

[0076] It should be further explained that, in the specific implementation process, the process by which the ad matching module performs ad matching based on the features of the obtained reference video image after removing the noise effect includes:

[0077] The most frequently occurring scene, object, and person in the reference video image is identified. The most frequently occurring scene in the acquired reference video image is identified as the reference scene, the most frequently occurring object in the acquired reference video image is identified as the reference object, and the most frequently occurring person in the acquired reference video image is identified as the reference person.

[0078] Construct an ad matching chain, which consists of scene nodes, object nodes, character nodes, and ad nodes;

[0079] Connect scene nodes to object nodes, connect object nodes to character nodes, and connect character nodes to advertisement nodes.

[0080] Input the reference scene into the scene node, input the reference object into the object node, and input the reference person into the person node to obtain the reference advertisement matching chain;

[0081] The ad matching module is equipped with an ad matching chain database, which stores several ad matching chains. Based on historical ad push data, it obtains ads corresponding to different scenarios, objects, and people, and inputs the ads corresponding to different scenarios, objects, and people into the ad matching chain.

[0082] The obtained reference ad matching chain is compared with the ad matching chains stored in the ad matching chain database. Ad matching chains whose content in scene nodes, object nodes, and character nodes is the same as the content in scene nodes, object nodes, and character nodes in the reference ad matching chain are recorded as standard ad matching chains. Ads in the ad nodes of the standard ad matching chains are obtained, recommended to users, and inserted into the videos watched by users.

[0083] This invention also discloses a recommendation method for an automated advertising recommendation system based on video content awareness, comprising the following steps:

[0084] Step 1: Obtain video data watched by the user;

[0085] Step 2: Process the acquired video data viewed by the user to obtain a reference video image after removing noise.

[0086] Step 3: Extract features from the noise-removed reference video image to obtain its features.

[0087] Step 4: Perform ad matching based on the features of the obtained noise-removed reference video image, and recommend the matched ads to the user.

[0088] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. An automated advertising recommendation system based on video content awareness, comprising a recommendation center, characterized in that, The recommendation center is connected to a video data acquisition module, a video data processing module, a feature extraction module, and an ad matching module. The video data acquisition module is used to collect video data watched by the user. The video data processing module is used to process the video data collected from the user to obtain a reference video image after removing noise. The feature extraction module is used to extract features from the obtained noise-free reference video image to obtain the features of the noise-free reference video image. The ad matching module is used to perform ad matching based on the features of the obtained reference video image after removing noise, and recommend the matched ads to the user. The process by which the video data acquisition module acquires video data watched by the user includes: The video data acquisition module consists of several video data acquisition terminals; By configuring each video data acquisition terminal and generating a corresponding configuration channel, and connecting to the corresponding server through the generated configuration channel, the video data acquisition terminal obtains the video data watched by the user on the corresponding server through the configuration channel and sends the obtained video data watched by the user to the video data processing module. The process by which the video data processing module processes the collected video data watched by the user includes: Frames are extracted from the video data watched by the user to obtain several frames of video data, and each frame of video data is recorded as a reference video image. Set a Gaussian filter, input the reference video image into the Gaussian filter, the Gaussian filter processes the reference video image to remove noise in the reference video image, while retaining the overall structure and edge information of the reference grayscale video image, the Gaussian filter outputs, and obtains the reference video image after removing the influence of noise. The feature extraction module performs feature extraction on the noise-removed reference video image, including the following steps: The feature extraction module contains a feature extraction model, which consists of an input layer, a scene extraction layer, an edge layer, an object extraction layer, a person extraction layer, and an output layer. The reference video image after removing noise is input into the feature extraction model. The reference video image reaches the output layer of the feature extraction model. The input layer of the feature extraction model sends the reference video image to the scene extraction layer of the feature extraction model. The scene extraction layer stores several scene images, each scene image representing a scene. The reference video image is matched with the scene images stored in the scene extraction layer to obtain the scene in the reference video image. The scene extraction layer in the feature extraction model sends the reference video image to the edge layer and the scene in the reference video image to the output layer; The edge layer processes the reference video image to obtain an edge reference video grayscale image; The edge layer sends the edge reference video grayscale image to the object extraction layer. The object extraction layer stores several edge object grayscale images, each edge object grayscale image representing an object. The edge reference video grayscale image is matched with the edge object grayscale images stored in the object extraction layer to obtain the object in the edge reference video grayscale image. The object extraction layer sends the objects in the edge reference video grayscale image to the output layer, and sends the edge reference video grayscale image to the person extraction layer; The character extraction layer stores several edge character grayscale images, each edge character grayscale image representing a character. The edge reference video grayscale image is matched with the edge character grayscale images stored in the character extraction layer to obtain the character in the edge reference video grayscale image. The person extraction layer sends the person from the edge reference video grayscale image to the output layer; The output layer outputs the scene, objects, and people from the reference video image. The reference video images are labeled, denoted as j, j=1, 2, 3, ..., m, where m is a positive integer.

2. The automated advertising recommendation system based on video content perception according to claim 1, characterized in that, The edge layer processes the reference video image as follows: The edge layer converts the reference video image into a reference video grayscale image, obtains the gradient magnitude of the pixels in the reference video grayscale image through the Sobel operator, and sets a gradient magnitude threshold range, which is denoted as (Y0, Y1). If the gradient magnitude of a pixel in the reference video grayscale image is greater than the maximum value in the gradient magnitude threshold range, the pixel is marked as a first edge pixel. If the gradient magnitude of a pixel is less than or equal to the maximum value in the gradient magnitude threshold range, but greater than or equal to the minimum value in the gradient magnitude threshold range, the pixel is marked as a second edge pixel. If the gradient magnitude of a pixel is less than the minimum value in the gradient magnitude threshold range, the pixel is marked as a non-edge pixel. Get the pixels in the top, bottom, left, right and four diagonal directions of the second edge pixel. If the first edge pixel exists among the obtained pixels, then mark the second edge pixel as the first edge pixel. If the first edge pixel does not exist among the acquired pixels, then the second edge pixel is marked as a non-edge pixel; Set the brightness value of the first edge pixel to "255" to obtain the edge reference video grayscale image.

3. The automated advertising recommendation system based on video content perception according to claim 2, characterized in that, The process of ad matching by the ad matching module based on the features of the noise-removed reference video image includes: The most frequently occurring scene, object, and person in the reference video image is identified. The most frequently occurring scene in the acquired reference video image is identified as the reference scene, the most frequently occurring object in the acquired reference video image is identified as the reference object, and the most frequently occurring person in the acquired reference video image is identified as the reference person. Construct an ad matching chain, which consists of scene nodes, object nodes, character nodes, and ad nodes; Connect scene nodes to object nodes, connect object nodes to character nodes, and connect character nodes to advertisement nodes. Input the reference scene into the scene node, input the reference object into the object node, and input the reference person into the person node to obtain the reference advertisement matching chain; The ad matching module is equipped with an ad matching chain database. Based on the reference ad matching chain and the ad matching chain database, it obtains the ads recommended to the user and inserts the obtained ads into the video watched by the user.

4. The automated advertising recommendation system based on video content perception according to claim 3, characterized in that, The process of obtaining ads recommended to users based on reference ad match chains and ad match chain databases includes: The advertising matching chain database stores several advertising matching chains. Based on historical advertising push data, it obtains advertisements corresponding to different scenarios, objects, and people, and inputs the advertisements corresponding to different scenarios, objects, and people into the advertising matching chain. The obtained reference ad matching chain is compared with the ad matching chains stored in the ad matching chain database. Ad matching chains whose content in scene nodes, object nodes, and character nodes is the same as the content in scene nodes, object nodes, and character nodes in the reference ad matching chain are recorded as standard ad matching chains. Ads in the ad nodes of the standard ad matching chains are then retrieved. The retrieved ads are the ads recommended to the user.

5. A method for automated ad recommendation based on video content awareness, specifically applied to the automated ad recommendation system based on video content awareness as described in any one of claims 1 to 4, characterized in that, Includes the following steps: Step 1: Obtain video data watched by the user; Step 2: Process the acquired video data viewed by the user to obtain a reference video image after removing noise. Step 3: Extract features from the noise-free reference video image to obtain its features. Step 4: Based on the features of the obtained noise-removed reference video image, perform ad matching and recommend the matched ads to the user.

Images