Advertisement automatic recommendation method and system based on video content perception

Abstract

The invention discloses an automatic advertisement recommendation method and system based on video content perception, and relates to the field of image processing, the system comprises a recommendation center, the recommendation center is in communication connection with a video data acquisition module, a video data processing module, a feature extraction module and an advertisement matching module; video data watched by a user are collected through the video data collection module, the collected video data watched by the user are processed through the video data processing module, a reference video image with noise influence removed is obtained, feature extraction is conducted on the obtained reference video image with noise influence removed through the feature extraction module, and the reference video image with noise influence removed is obtained. According to the method, the reference video image is subjected to noise influence removal to obtain the features of the reference video image subjected to noise influence removal, the advertisement matching module performs advertisement matching according to the obtained features of the reference video image subjected to noise influence removal, the matched advertisement is recommended to the user, the corresponding advertisement is recommended to the user according to the type of the video watched by the user, and the click rate of the advertisement is greatly improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to an automatic advertisement recommendation method and system based on video content perception. Background Art

[0002] With the development of Internet technology and the popularization of broadband networks, video content has become a major component of Internet traffic; the number of users and viewing time of video streaming platforms are growing, which has brought huge market opportunities for advertising; traditional advertising methods are usually based on users' browsing history, search history and demographic information. The advertising content is less relevant to the video content that users are watching, which is easy to cause user disgust. The accuracy of advertising is not high, and it is difficult to maximize the advertising effect;

[0003] How to obtain corresponding advertisements according to the video content watched by users, insert the obtained advertisements into the video content watched by users, improve the accuracy of advertisement delivery, and enhance the user experience is a problem we need to solve. To this end, an automatic advertisement recommendation method and system based on video content perception are now provided. Summary of the invention

[0004] In order to solve the above technical problems, the purpose of the present invention is to provide an automatic advertisement recommendation method and system based on video content perception.

[0005] The object of the present invention can be achieved by the following technical solutions: an automated advertising recommendation system based on video content perception, comprising a recommendation center, wherein the recommendation center is communicatively connected to a video data acquisition module, a video data processing module, a feature extraction module, and an advertising matching module;

[0006] The video data acquisition module is used to acquire video data watched by users;

[0007] The video data processing module is used to process the collected video data watched by the user to obtain a reference video image after removing the noise influence;

[0008] The feature extraction module is used to extract features from the reference video image after the noise effect is removed, so as to obtain features of the reference video image after the noise effect is removed;

[0009] The advertisement matching module is used to match advertisements according to the features of the reference video image obtained after removing the influence of noise, and recommend the matched advertisements to the user.

[0010] Furthermore, the process of the video data acquisition module acquiring the video data watched by the user includes:

[0011] The video data acquisition module is composed of several video data acquisition terminals;

[0012] By configuring each video data acquisition terminal and generating a corresponding configuration channel, the corresponding server is linked through the generated configuration channel. The video data acquisition terminal obtains the video data watched by the user in the corresponding server through the configuration channel and sends the obtained video data watched by the user to the video data processing module.

[0013] Furthermore, the process of the video data processing module processing the collected video data watched by the user includes:

[0014] Perform frame extraction on the collected video data watched by the user to obtain a number of frames of video data, and record each frame of video data as a reference video image;

[0015] A Gaussian filter is set, and the reference video image is input into the Gaussian filter. The Gaussian filter processes the reference video image to remove the noise in the reference video image while retaining the overall structure and edge information of the reference grayscale video image. The Gaussian filter is output to obtain the reference video image after the noise is removed.

[0016] Furthermore, the process of extracting features from the reference video image obtained after removing the noise influence by the feature extraction module includes:

[0017] A feature extraction model is constructed in the feature extraction module, and the feature extraction model is composed of an input layer, a scene extraction layer, an edge layer, an object extraction layer, a character extraction layer and an output layer;

[0018] The reference video image after removing the noise influence is input into the feature extraction model, the reference video image reaches the output layer in the feature extraction model, and the input layer in the feature extraction model sends the reference video image to the scene extraction layer in the feature extraction model;

[0019] The scene extraction layer stores a plurality of scene images, each of which represents a scene, and the scenes include a first scene, a second scene, a third scene, ..., matching the reference video image with the scene images stored in the scene extraction layer to obtain the scene in the reference video image;

[0020] The scene extraction layer in the feature extraction model sends the reference video image to the edge layer and sends the scene in the reference video image to the output layer;

[0021] The edge layer processes the reference video image to obtain an edge reference video grayscale image;

[0022] The edge layer sends the edge reference video grayscale image to the object extraction layer, wherein the object extraction layer stores a plurality of edge object grayscale images, each edge object grayscale image represents an object, wherein the objects include a first object, a second object, a third object, ..., and the edge reference video grayscale image is matched with the edge object grayscale images stored in the object extraction layer to obtain the object in the edge reference video grayscale image;

[0023] The object extraction layer sends the objects in the edge reference video grayscale image to the output layer, and sends the edge reference video grayscale image to the person extraction layer;

[0024] The character extraction layer stores a plurality of edge character grayscale images, each edge character grayscale image represents a character, and the characters include a first character, a second character, a third character, ..., and the edge reference video grayscale image is matched with the edge character grayscale image stored in the character extraction layer to obtain the character in the edge reference video grayscale image;

[0025] The character extraction layer sends the characters in the edge reference video grayscale image to the output layer;

[0026] The output layer outputs the scenes, objects and characters in the reference video image.

[0027] Furthermore, the process of the edge layer processing the reference video image includes:

[0028] The edge layer converts the reference video image into a reference video grayscale image, obtains the gradient amplitude of the pixels in the reference video grayscale image through the Sobel operator, and sets the gradient amplitude threshold range, and the set gradient threshold range is recorded as (Y0, Y1);

[0029] If the gradient amplitude of a pixel in the reference video grayscale image is greater than the maximum value in the gradient amplitude threshold range, the pixel is marked as a first edge pixel; if the gradient amplitude of the pixel is less than or equal to the maximum value in the gradient amplitude threshold range and greater than or equal to the minimum value in the gradient amplitude threshold range, the pixel is marked as a second edge pixel; if the gradient amplitude of the pixel is less than the minimum value in the gradient amplitude threshold range, the pixel is marked as a non-edge pixel;

[0030] Obtain pixels in the upper, lower, left, right, and four diagonal directions of the second edge pixel, and if the first edge pixel exists in the obtained pixels, mark the second edge pixel as the first edge pixel;

[0031] If the first edge pixel does not exist in the acquired pixels, the second edge pixel is marked as a non-edge pixel;

[0032] The brightness value of the first edge pixel is set to "255" to obtain an edge reference video grayscale image.

[0033] Furthermore, the process of performing advertisement matching by the advertisement matching module according to the features of the reference video image obtained after removing the influence of noise includes:

[0034] Obtain the scenes, objects, and persons that appear most frequently in the reference video image, record the scenes that appear most frequently in the reference video image as reference scenes, record the objects that appear most frequently in the reference video image as reference objects, and record the persons that appear most frequently in the reference video image as reference persons;

[0035] Constructing an advertisement matching chain, wherein the advertisement matching chain is composed of a scene node, an object node, a character node, and an advertisement node;

[0036] Connect scene nodes with object nodes, connect object nodes with character nodes, and connect character nodes with advertisement nodes;

[0037] Input the reference scene into the scene node, input the reference object into the object node, input the reference person into the person node, and obtain the reference advertisement matching chain;

[0038] The advertisement matching module is provided with an advertisement matching chain database, and obtains advertisements recommended to the user according to the reference advertisement matching chain and the advertisement matching chain database, and inserts the obtained advertisements into the video watched by the user.

[0039] Furthermore, the process of obtaining an advertisement recommended to a user according to the reference advertisement matching chain and the advertisement matching chain database includes:

[0040] The advertisement matching chain database stores a plurality of advertisement matching chains, obtains advertisements corresponding to different scenes, objects and characters according to historical advertisement push data, and inputs different scenes, objects, characters and advertisements corresponding to different scenes, objects and characters into the advertisement matching chain;

[0041] The obtained reference advertisement matching chain is compared with the advertisement matching chain stored in the advertisement matching chain database, and an advertisement matching chain whose contents in scene nodes, object nodes and character nodes are identical to those in the reference advertisement matching chain is obtained; the advertisement matching chain whose contents in scene nodes, object nodes and character nodes are identical to those in the reference advertisement matching chain is recorded as a standard advertisement matching chain; advertisements in advertisement nodes in the standard advertisement matching chain are obtained, and the obtained advertisements are the advertisements recommended to users.

[0042] The present invention also discloses a recommendation method of an automatic advertisement recommendation system based on video content perception, comprising the following steps:

[0043] Step 1: Obtain the video data watched by the user;

[0044] Step 2: Process the acquired video data watched by the user to obtain a reference video image after removing the noise influence;

[0045] Step 3: extracting features from the reference video image after removing the noise, and obtaining features of the reference video image after removing the noise;

[0046] Step 4: Perform advertisement matching based on the features of the reference video image after noise is removed, and recommend the matched advertisement to the user.

[0047] Compared with the prior art, the beneficial effects of the present invention are: collecting video data watched by users, processing the collected video data watched by users, obtaining a reference video image after removing the influence of noise, performing feature extraction on the obtained reference video image after removing the influence of noise, obtaining the features of the reference video image after removing the influence of noise, matching advertisements according to the features of the obtained reference video image after removing the influence of noise, and recommending matched advertisements to users, recommending corresponding advertisements to users according to the types of videos watched by users, thereby greatly improving the click-through rate of advertisements, improving the accuracy of advertisement delivery, maximizing the advertising effect, and improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a schematic diagram of the present invention. DETAILED DESCRIPTION

[0049] like Figure 1 As shown, an automatic advertisement recommendation system based on video content perception includes a recommendation center, wherein the recommendation center is communicatively connected with a video data acquisition module, a video data processing module, a feature extraction module, and an advertisement matching module;

[0050] The video data acquisition module is used to acquire video data watched by users;

[0051] It should be further explained that, in a specific implementation process, the process of the video data acquisition module acquiring the video data watched by the user includes:

[0052] The video data acquisition module is composed of several video data acquisition terminals;

[0053] By configuring each video data acquisition terminal and generating a corresponding configuration channel, the corresponding server is linked through the generated configuration channel. The video data acquisition terminal obtains the video data watched by the user in the corresponding server through the configuration channel and sends the obtained video data watched by the user to the video data processing module.

[0054] The video data processing module is used to process the collected video data watched by the user to obtain a reference video image after removing the noise influence;

[0055] It should be further explained that, in a specific implementation process, the process in which the video data processing module processes the collected video data watched by the user includes:

[0056] Frame extraction is performed on the collected video data watched by the user to obtain a number of frames of video data, each frame of video data is recorded as a reference video image, and the reference video image is labeled as j, where j=1, 2, 3, ..., m, where m is a positive integer;

[0057] A Gaussian filter is set, and the reference video image is input into the Gaussian filter. The Gaussian filter processes the reference video image to remove the noise in the reference video image while retaining the overall structure and edge information of the reference grayscale video image. The Gaussian filter is output to obtain the reference video image after the noise is removed.

[0058] The feature extraction module is used to extract features from the reference video image after the noise effect is removed, so as to obtain features of the reference video image after the noise effect is removed;

[0059] It should be further explained that, in a specific implementation process, the process of the feature extraction module extracting features from the reference video image obtained after removing the noise influence includes:

[0060] A feature extraction model is constructed in the feature extraction module, and the feature extraction model is composed of an input layer, a scene extraction layer, an edge layer, an object extraction layer, a character extraction layer and an output layer;

[0061] The input layer is used to input a reference video image, the scene extraction layer is used to extract the scene in the reference video image, the edge layer is used to extract edge information in the reference video image, the object extraction layer is used to extract objects in the reference video image, the character extraction layer is used to extract characters in the reference video image, and the output layer is used to output features of the reference video image;

[0062] The reference video image after removing the noise influence is input into the feature extraction model, the reference video image reaches the output layer in the feature extraction model, and the input layer in the feature extraction model sends the reference video image to the scene extraction layer in the feature extraction model;

[0063] The scene extraction layer stores a plurality of scene images, each of which represents a scene, and the scenes include a first scene, a second scene, a third scene, ..., matching the reference video image with the scene images stored in the scene extraction layer to obtain the scene in the reference video image;

[0064] The scene extraction layer in the feature extraction model sends the reference video image to the edge layer and sends the scene in the reference video image to the output layer;

[0065] The edge layer converts the reference video image into a reference video grayscale image, obtains the gradient amplitude of the pixels in the reference video grayscale image through the Sobel operator, and sets the gradient amplitude threshold range, and the set gradient threshold range is recorded as (Y0, Y1);

[0066] If the gradient amplitude of a pixel in the reference video grayscale image is greater than the maximum value in the gradient amplitude threshold range, the pixel is marked as a first edge pixel; if the gradient amplitude of the pixel is less than or equal to the maximum value in the gradient amplitude threshold range and greater than or equal to the minimum value in the gradient amplitude threshold range, the pixel is marked as a second edge pixel; if the gradient amplitude of the pixel is less than the minimum value in the gradient amplitude threshold range, the pixel is marked as a non-edge pixel;

[0067] Obtain pixels in the upper, lower, left, right, and four diagonal directions of the second edge pixel, and if the first edge pixel exists in the obtained pixels, mark the second edge pixel as the first edge pixel;

[0068] If the first edge pixel does not exist in the acquired pixels, the second edge pixel is marked as a non-edge pixel;

[0069] The brightness value of the first edge pixel is set to 255 to obtain an edge reference video grayscale image;

[0070] The edge layer sends the edge reference video grayscale image to the object extraction layer, wherein the object extraction layer stores a plurality of edge object grayscale images, each edge object grayscale image represents an object, wherein the objects include a first object, a second object, a third object, ..., and the edge reference video grayscale image is matched with the edge object grayscale images stored in the object extraction layer to obtain the object in the edge reference video grayscale image;

[0071] The object extraction layer sends the objects in the edge reference video grayscale image to the output layer, and sends the edge reference video grayscale image to the person extraction layer;

[0072] The character extraction layer stores a plurality of edge character grayscale images, each edge character grayscale image represents a character, and the characters include a first character, a second character, a third character, ..., and the edge reference video grayscale image is matched with the edge character grayscale image stored in the character extraction layer to obtain the character in the edge reference video grayscale image;

[0073] The character extraction layer sends the characters in the edge reference video grayscale image to the output layer;

[0074] The output layer outputs the scenes, objects and characters in the reference video image.

[0075] The advertisement matching module is used to match advertisements according to the features of the reference video image obtained after removing the influence of noise, and recommend the matched advertisements to the user;

[0076] It should be further explained that, in a specific implementation process, the process of the advertisement matching module performing advertisement matching according to the features of the reference video image obtained after removing the influence of noise includes:

[0077] Obtain the scenes, objects, and persons that appear most frequently in the reference video image, record the scenes that appear most frequently in the reference video image as reference scenes, record the objects that appear most frequently in the reference video image as reference objects, and record the persons that appear most frequently in the reference video image as reference persons;

[0078] Constructing an advertisement matching chain, wherein the advertisement matching chain is composed of a scene node, an object node, a character node, and an advertisement node;

[0079] Connect scene nodes with object nodes, connect object nodes with character nodes, and connect character nodes with advertisement nodes;

[0080] Input the reference scene into the scene node, input the reference object into the object node, input the reference person into the person node, and obtain the reference advertisement matching chain;

[0081] The advertisement matching module is provided with an advertisement matching chain database, in which a plurality of advertisement matching chains are stored, and advertisements corresponding to different scenes, objects and characters are obtained according to historical advertisement push data, and advertisements corresponding to different scenes, objects, characters and different scenes, objects and characters are input into the advertisement matching chain;

[0082] The obtained reference advertisement matching chain is compared with the advertisement matching chain stored in the advertisement matching chain database, and an advertisement matching chain whose contents in scene nodes, object nodes and character nodes are identical to those in scene nodes, object nodes and character nodes in the reference advertisement matching chain is obtained; the advertisement matching chain whose contents in scene nodes, object nodes and character nodes are identical to those in scene nodes, object nodes and character nodes in the reference advertisement matching chain is recorded as a standard advertisement matching chain; advertisements in advertisement nodes in the standard advertisement matching chain are obtained, the obtained advertisements are recommended to users, and the obtained advertisements are inserted into videos watched by users.

[0083] The present invention also discloses a recommendation method of an automatic advertisement recommendation system based on video content perception, comprising the following steps:

[0084] Step 1: Obtain the video data watched by the user;

[0085] Step 2: Process the acquired video data watched by the user to obtain a reference video image after removing the noise influence;

[0086] Step 3: extracting features from the reference video image after removing the noise, and obtaining features of the reference video image after removing the noise;

[0087] Step 4: Perform advertisement matching based on the features of the reference video image after noise is removed, and recommend the matched advertisement to the user.

[0088] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. An automated advertising recommendation system based on video content perception, comprising a recommendation center, characterized in that: The recommendation center is communicatively connected to a video data acquisition module, a video data processing module, a feature extraction module, and an advertisement matching module; The video data acquisition module is used to acquire video data watched by users; The video data processing module is used to process the collected video data watched by the user to obtain a reference video image after removing the noise influence; The feature extraction module is used to extract features from the reference video image after the noise effect is removed, so as to obtain features of the reference video image after the noise effect is removed; The advertisement matching module is used to match advertisements according to the features of the reference video image obtained after removing the influence of noise, and recommend the matched advertisements to the user.

2. The automatic advertising recommendation system based on video content perception according to claim 1, characterized in that: The process of the video data acquisition module acquiring the video data watched by the user includes: The video data acquisition module is composed of several video data acquisition terminals; By configuring each video data acquisition terminal and generating a corresponding configuration channel, the corresponding server is linked through the generated configuration channel. The video data acquisition terminal obtains the video data watched by the user in the corresponding server through the configuration channel and sends the obtained video data watched by the user to the video data processing module.

3. The automatic advertising recommendation system based on video content perception according to claim 2, characterized in that: The process of the video data processing module processing the collected video data watched by the user includes: Perform frame extraction on the collected video data watched by the user to obtain a number of frames of video data, and record each frame of video data as a reference video image; A Gaussian filter is set, and the reference video image is input into the Gaussian filter. The Gaussian filter processes the reference video image to remove the noise in the reference video image while retaining the overall structure and edge information of the reference grayscale video image. The Gaussian filter is output to obtain the reference video image after the noise is removed.

4. The automatic advertisement recommendation system based on video content perception according to claim 3, characterized in that: The process of extracting features from the reference video image after removing the noise influence by the feature extraction module includes: A feature extraction model is constructed in the feature extraction module, and the feature extraction model is composed of an input layer, a scene extraction layer, an edge layer, an object extraction layer, a character extraction layer and an output layer; The reference video image after removing the noise influence is input into the feature extraction model, the reference video image reaches the output layer in the feature extraction model, and the input layer in the feature extraction model sends the reference video image to the scene extraction layer in the feature extraction model; The scene extraction layer stores a plurality of scene images, each of which represents a scene, and the scenes include a first scene, a second scene, a third scene, ..., matching the reference video image with the scene images stored in the scene extraction layer to obtain the scene in the reference video image; The scene extraction layer in the feature extraction model sends the reference video image to the edge layer and sends the scene in the reference video image to the output layer; The edge layer processes the reference video image to obtain an edge reference video grayscale image; The edge layer sends the edge reference video grayscale image to the object extraction layer, wherein the object extraction layer stores a plurality of edge object grayscale images, each edge object grayscale image represents an object, wherein the objects include a first object, a second object, a third object, ..., and the edge reference video grayscale image is matched with the edge object grayscale images stored in the object extraction layer to obtain the object in the edge reference video grayscale image; The object extraction layer sends the objects in the edge reference video grayscale image to the output layer, and sends the edge reference video grayscale image to the person extraction layer; The character extraction layer stores a plurality of edge character grayscale images, each edge character grayscale image represents a character, and the characters include a first character, a second character, a third character, ..., and the edge reference video grayscale image is matched with the edge character grayscale image stored in the character extraction layer to obtain the character in the edge reference video grayscale image; The character extraction layer sends the characters in the edge reference video grayscale image to the output layer; The output layer outputs the scenes, objects and characters in the reference video image.

5. The automatic advertisement recommendation system based on video content perception according to claim 4, characterized in that: The process of edge layer processing the reference video image includes: The edge layer converts the reference video image into a reference video grayscale image, obtains the gradient amplitude of the pixels in the reference video grayscale image through the Sobel operator, and sets the gradient amplitude threshold range, and the set gradient threshold range is recorded as (Y0, Y1); If the gradient amplitude of a pixel in the reference video grayscale image is greater than the maximum value in the gradient amplitude threshold range, the pixel is marked as a first edge pixel; if the gradient amplitude of the pixel is less than or equal to the maximum value in the gradient amplitude threshold range and greater than or equal to the minimum value in the gradient amplitude threshold range, the pixel is marked as a second edge pixel; if the gradient amplitude of the pixel is less than the minimum value in the gradient amplitude threshold range, the pixel is marked as a non-edge pixel; Obtain pixels in the upper, lower, left, right, and four diagonal directions of the second edge pixel, and if the first edge pixel exists in the obtained pixels, mark the second edge pixel as the first edge pixel; If the first edge pixel does not exist in the acquired pixels, the second edge pixel is marked as a non-edge pixel; The brightness value of the first edge pixel is set to "255" to obtain an edge reference video grayscale image.

6. The automatic advertisement recommendation system based on video content perception according to claim 5, characterized in that: The process of performing advertisement matching by the advertisement matching module according to the features of the reference video image obtained after removing the influence of noise includes: Obtain the scenes, objects, and persons that appear most frequently in the reference video image, record the scenes that appear most frequently in the reference video image as reference scenes, record the objects that appear most frequently in the reference video image as reference objects, and record the persons that appear most frequently in the reference video image as reference persons; Constructing an advertisement matching chain, wherein the advertisement matching chain is composed of a scene node, an object node, a character node, and an advertisement node; Connect scene nodes with object nodes, connect object nodes with character nodes, and connect character nodes with advertisement nodes; Input the reference scene into the scene node, input the reference object into the object node, input the reference person into the person node, and obtain the reference advertisement matching chain; The advertisement matching module is provided with an advertisement matching chain database, and obtains advertisements recommended to the user according to the reference advertisement matching chain and the advertisement matching chain database, and inserts the obtained advertisements into the video watched by the user.

7. The automatic advertisement recommendation system based on video content perception according to claim 6, characterized in that: The process of obtaining an advertisement recommended to a user based on the reference advertisement matching chain and the advertisement matching chain database includes: The advertisement matching chain database stores a plurality of advertisement matching chains, obtains advertisements corresponding to different scenes, objects and characters according to historical advertisement push data, and inputs different scenes, objects, characters and advertisements corresponding to different scenes, objects and characters into the advertisement matching chain; The obtained reference advertisement matching chain is compared with the advertisement matching chain stored in the advertisement matching chain database, and an advertisement matching chain whose contents in scene nodes, object nodes and character nodes are identical to those in the reference advertisement matching chain is obtained; the advertisement matching chain whose contents in scene nodes, object nodes and character nodes are identical to those in the reference advertisement matching chain is recorded as a standard advertisement matching chain; advertisements in advertisement nodes in the standard advertisement matching chain are obtained, and the obtained advertisements are the advertisements recommended to users.

8. The method for automatic advertising recommendation based on video content perception according to claim 7 is specifically applied to the automatic advertising recommendation system based on video content perception according to any one of claims 1 to 7, characterized in that: The following steps are involved: Step 1: Obtain the video data watched by the user; Step 2: Process the acquired video data watched by the user to obtain a reference video image after removing the noise influence; Step 3: extracting features from the reference video image after removing the noise, and obtaining features of the reference video image after removing the noise; Step 4: Perform advertisement matching based on the features of the reference video image after noise is removed, and recommend the matched advertisement to the user.

Images