Advertisement embedding method and device

Abstract

The present invention provides an advertisement implantation method and device, and relates to the field of video processing technology. The method comprises: obtaining key points of a court corresponding to a court simulation model, and a first position corresponding to an implanted advertisement; obtaining a video to be implanted with advertisements, and determining a standard frame image under each lens corresponding to the video to be implanted with advertisements, wherein the standard frame image is a video frame image where an implanted advertisement first appears under each lens; determining a first spatial conversion relationship between the court simulation model and the key points of the court according to the standard frame image and the key points of the court; and implanting advertisements into the video to be implanted with advertisements according to the first spatial conversion relationship and the first position. The solution of the present invention selects standard frames from multiple lenses, and provides an accurate transformation space template for subsequent frames, thereby improving the accuracy of advertisement delivery.

Description

Technical Field

[0001] The present invention relates to the technical field of video processing, and in particular to an advertisement embedding method and device. Background Art

[0002] In the prior art, there are two main ways to implement advertising in sports events: one is to play advertisements at fixed positions in the stadium through LED (light emitting diode) display screens or projections, but this requires the support of hardware equipment and consumes more resources in switching advertisements; the other is to fix the advertisements at a certain position in the video and place the advertisements in the video screen, but this will block the playback process of the video, affecting the viewing experience and creating a sense of disharmony.

[0003] In order to find a good dynamic balance between user experience and video ad exposure times, video advertising urgently needs a "soft" ad embedding method to replace these traditional "hard" embedding methods. Summary of the invention

[0004] The purpose of the present invention is to provide an advertisement placement method and device to achieve refined time management, realistic and accurate delivery of stadium advertisements, and improve the exposure and acceptance of advertisements.

[0005] To achieve the above object, an embodiment of the present invention provides an advertisement embedding method, comprising:

[0006] Obtaining key points of the court corresponding to the court simulation model and the first position of the corresponding embedded advertisement;

[0007] Acquire a video to be embedded with advertisements, and determine a standard frame image under each shot corresponding to the video to be embedded with advertisements, wherein the standard frame image is a video frame image under each shot where an embedded advertisement appears for the first time;

[0008] Determining a first spatial conversion relationship between the court simulation model and the standard frame image according to the standard frame image and the court key points;

[0009] Advertisements are embedded in the video to be embedded with advertisements according to the first spatial conversion relationship and the first position.

[0010] Optionally, obtaining a video to be embedded with advertisements and determining a standard frame image under each shot corresponding to the video to be embedded with advertisements includes:

[0011] Classifying the video to be embedded with advertisements according to the shot categories, and determining video data corresponding to N shot types, where N is an integer greater than or equal to 1;

[0012] According to the N video data, a video data list corresponding to the N lens types is established;

[0013] The standard frame image under each lens is determined according to the video data list.

[0014] Optionally, the method further includes:

[0015] Acquire the non-standard frame image of the to-be-embedded advertisement video; the non-standard frame image is other images in the non-standard frame image except the standard frame image;

[0016] Determine, according to the non-standard frame image and the video data list, a first standard frame image under a lens corresponding to the non-standard frame image;

[0017] Advertisements are embedded in the video to be embedded with advertisements according to the first standard frame image, the first spatial conversion relationship, and the first position.

[0018] Optionally, embedding advertisements into the video to be embedded with advertisements according to the first standard frame image, the first spatial conversion relationship, and the first position includes:

[0019] Determine, according to the non-standard frame image and the first standard frame image, a second spatial conversion relationship between the standard frame image and the non-standard frame image under each lens;

[0020] Determining a third spatial conversion relationship between the court simulation model and the non-standard frame image according to the second spatial conversion relationship and the first spatial conversion relationship;

[0021] Advertisements are embedded in the video to be embedded with advertisements according to the third spatial conversion relationship and the first position.

[0022] Optionally, embedding advertisements into the video to be embedded with advertisements according to the third spatial conversion relationship and the first position includes:

[0023] Determining a first area range covered by the non-standard frame image in the court simulation model according to the third spatial conversion relationship;

[0024] According to the first area range, predicting a second area range covered by a next frame of image in the court simulation model;

[0025] Advertisements are embedded in the video to be embedded with advertisements according to the second area range and the first position.

[0026] Optionally, predicting a second area range covered by a next frame of image in the stadium simulation model according to the first area range includes:

[0027] Determine a time sequence corresponding to a video frame image according to the video to be embedded with an advertisement; the time sequence includes a region range corresponding to each frame image;

[0028] Determining a moving speed of the non-standard frame image in the court simulation model according to the time sequence and the first area range;

[0029] According to the moving speed, a second area range covered by the next frame of image in the court simulation model is predicted.

[0030] Optionally, embedding advertisements into the video to be embedded with advertisements according to the second area range and the first position includes:

[0031] Determine a positional relationship between the second area range and the first position according to the second area range and the first position;

[0032] Advertisements are embedded based on the positional relationship.

[0033] Optionally, when the position relationship is that the second area range has no intersection with the first position,

[0034] According to the position relationship, the embedded advertisement is performed, including:

[0035] Determining the timing of pasting the advertisement and the type of the advertisement according to the position relationship;

[0036] Advertisements are embedded according to the pasting timing of the advertisement and the type of the advertisement.

[0037] Optionally, when the position relationship is that the second area completely includes the first position,

[0038] According to the position relationship, the embedded advertisement is performed, including:

[0039] According to the positional relationship, determining that the frame image corresponding to the second area range is the updated standard frame image;

[0040] Determine, according to the updated standard frame image, the first spatial conversion relationship, and the second spatial conversion relationship, that the frame image corresponding to the second area range is located at a second position corresponding to the advertisement implantation in the stadium simulation model;

[0041] Determine a first material layer of the frame image and a second material layer to be embedded with an advertisement corresponding to the second area range;

[0042] The first material layer, the second material layer and the second position layer are merged to embed the advertisement.

[0043] Optionally, determining the standard frame image under each shot corresponding to the video to be embedded with the advertisement further includes:

[0044] Obtaining the camera position information corresponding to the advertisement video to be embedded and the corresponding image information under the current lens;

[0045] When any of the following preset conditions is met, the standard frame corresponding to the image information under the current lens is updated;

[0046] Wherein, the preset conditions are:

[0047] The resolution of the image information under the current lens is lower than the first preset range;

[0048] The current shot switching corresponding to the video to be embedded with the advertisement;

[0049] The corresponding camera position information exceeds a first preset distance.

[0050] Optionally, build a stadium simulation model, including:

[0051] Obtain stadium information and advertising information to be embedded;

[0052] Determine key points of the court according to the court venue information, and determine the location of the implanted advertisement according to the advertisement information to be implanted;

[0053] A stadium simulation model is constructed according to the key points of the stadium and the positions of the embedded advertisements.

[0054] To achieve the above object, an embodiment of the present invention further provides an advertisement embedding device, comprising:

[0055] An acquisition module, used for acquiring key points of the court corresponding to the court simulation model and a first position corresponding to the embedded advertisement;

[0056] The first processing module is used to obtain a video to be embedded with advertisements, and determine a standard frame image under each shot corresponding to the video to be embedded with advertisements, wherein the standard frame image is a video frame image under each shot where an embedded advertisement appears for the first time;

[0057] A second processing module, configured to determine a first spatial conversion relationship between the court simulation model and the standard frame image according to the standard frame image and the key points of the court;

[0058] The third processing module is used to embed advertisements into the video to be embedded with advertisements according to the first spatial conversion relationship and the first position.

[0059] To achieve the above object, an embodiment of the present invention further provides a readable storage medium on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps in the advertisement implantation method as described in any one of the above items are implemented.

[0060] The beneficial effects of the above technical solution of the present invention are as follows:

[0061] The embodiment of the present invention obtains the key points of the court corresponding to the court simulation model, and the first position of the corresponding embedded advertisement; obtains the video to be embedded with advertisement, and determines the standard frame image under each lens corresponding to the video to be embedded with advertisement, wherein the standard frame image is the video frame image where the embedded advertisement appears for the first time under each lens, and the present invention adopts a standard frame determination mechanism for multiple lenses; according to the standard frame image and the key points of the court, determines the first spatial conversion relationship between the court simulation model and the standard frame image, and pastes the advertisement in the court simulation model into the standard frame according to the first spatial conversion relationship; and implants the advertisement into the video to be embedded with advertisement according to the first spatial conversion relationship and the first position. Subsequent advertisement pasting and delivery will give priority to the classification of lenses, and perform spatial transformation operations on each frame image under the lens and the corresponding standard frame, which is conducive to finding the best standard frame for each lens, reducing the spatial transformation cost of the image, and improving the accuracy and stability of advertisement delivery. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 A flowchart of an advertisement embedding method according to an embodiment of the present invention;

[0063] Figure 2 A second flowchart of the advertisement embedding method provided by an embodiment of the present invention;

[0064] Figure 3 A schematic diagram of modules of an advertisement embedding device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0065] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0066] It should be understood that the references to "one embodiment" or "an embodiment" throughout the specification mean that the specific features, structures, or characteristics associated with the embodiment are included in at least one embodiment of the present invention. Therefore, the references to "in one embodiment" or "in an embodiment" appearing throughout the specification do not necessarily refer to the same embodiment. In addition, these specific features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0067] In various embodiments of the present invention, it should be understood that the size of the serial numbers of the following processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0068] Additionally, the terms "system" and "network" are often used interchangeably herein.

[0069] In the embodiments provided in the present application, it should be understood that "B corresponding to A" means that B is associated with A, and B can be determined according to A. However, it should also be understood that determining B according to A does not mean determining B only according to A, and B can also be determined according to A and / or other information.

[0070] In order to solve the above problems, this proposal proposes an advertising implantation method and device, which is based on real stadium information modeling, combined with the switching of video lenses, to determine the timing and type of advertising pasting, find the best conversion standard frame, and achieve refined time management and realistic stadium advertising precision delivery.

[0071] The technical solution of the present invention is based on artificial intelligence technology and is suitable for scenes such as advertising placement in stadium game videos / live broadcasts (including but not limited to basketball, volleyball, football, etc.). The present invention performs scene modeling of real stadium information, and places advertisements at fixed virtual positions in the model, so that the advertisements are integrated with the scene (such as pasted on the ground of the stadium, etc.), and matches the stadium modeling information through video analysis from the camera's perspective, combines lens classification with the spatial relationship between the scene under the video lens and the real scene, determines the timing and type of advertisement placement, and migrates the virtual advertisements in the model to the stadium video from the camera's perspective, so as to perform virtual placement of advertisements in quasi-real scenes of the video.

[0072] It should be noted that the present invention uses artificial intelligence technology to deliver immersive advertising. It models the real stadium and advertisements based on fixed positions, and migrates the advertisements to the stadium scene under the camera's perspective, so that the advertisements and the scene are integrated, which reduces the sense of incongruity of advertising delivery, improves user experience, and solves the industry's pain point problem that advertising delivery is too single and affects the viewing experience.

[0073] like Figure 1 As shown, an advertisement embedding method according to an embodiment of the present invention includes:

[0074] Step 100, obtaining key points of the court corresponding to the court simulation model and a first position corresponding to the embedded advertisement;

[0075] It is understandable that in order to embed advertisements in a video, the advertisement content needs to be embedded in the video frame and displayed as the video frame is played. It is necessary to determine the key points of the court P_model and the first position D_ad corresponding to the embedded advertisement. The key points of the court are data of the court simulation model that reflects the real court field information; the first position corresponding to the embedded advertisement is within the range of the key points of the court, and the first position can be one position or multiple positions.

[0076] Step 200, obtaining a video to be embedded with advertisements, and determining a standard frame image under each shot corresponding to the video to be embedded with advertisements, wherein the standard frame image is a video frame image under each shot where an embedded advertisement appears for the first time;

[0077] In this embodiment, the video to be embedded in the advertisement is played in the form of online on-demand, local video playback or live broadcast. The storage location and transmission method of the video to be embedded in the advertisement are not limited. For example, the storage location can be a local hard disk, a mobile storage device, a network server, etc. of the video playback terminal; the transmission method can be network transmission, data line transmission, Bluetooth wireless transmission, etc. All video frames of the video to be embedded in the advertisement are classified by lens, and by decoding the video to be embedded in the advertisement, the video frame image under the corresponding at least one lens can be obtained. In step 200, all video frame images under each lens are determined according to the video to be embedded in the advertisement, and the first frame under the lens where the advertisement is fully embedded for the first time appears in the picture is taken as the initialized standard frame image. Each of the standard frames can be set with a corresponding standard frame key point, and the standard frame key point is used as the basis for subsequent processing.

[0078] Step 300, determining a first spatial conversion relationship between the court simulation model and the standard frame image according to the standard frame image and the key points of the court;

[0079] In this embodiment, after the standard frame image LMn-standard is initialized for each shot and the standard frame key points are calculated, the standard frame key points are matched with the court key points to determine the first spatial conversion relationship Ln-K between the court simulation model Model and the standard frame image LMn-standard.

[0080] Step 400: embed advertisements into the video to be embedded with advertisements according to the first spatial conversion relationship and the first position.

[0081] In this embodiment, the advertisement space is converted according to the first space conversion relationship Ln-K and the first position D_ad, the advertisement position D_ad' corresponding to the standard frame image is determined, and the advertisement is inserted into the video to be inserted according to the advertisement position D_ad'.

[0082] It should be noted that all frame images under each camera perspective are based on the image space where the standard frame image is located, and can also be used as a reference frame for key point detection on the court to replace and place advertisements.

[0083] The invention changes the advertisement with the camera angle of view, thus realizing the scene-based advertisement. At the same time, diversified advertisement delivery can be carried out according to different lens types and the spatial position of lens shooting, thus realizing the advantages of refined advertisement delivery time, low delivery cost, convenient advertisement replacement, free control of advertisement playing time and duration, wide portability, etc.

[0084] Specifically, before step 100, a stadium simulation model is constructed, including:

[0085] Step 110, obtaining stadium information and advertisement information to be implanted;

[0086] Step 120, determining key points of the court according to the court venue information, and determining the location of the implanted advertisement according to the advertisement information to be implanted;

[0087] Step 130: construct a stadium simulation model according to the key points of the stadium and the positions of the embedded advertisements.

[0088] In this embodiment, the field information, that is, the real field information, size, and advertisement information, size, and insertion position are used to establish a field simulation model, thereby obtaining a field simulation model (Model), and determining the field key points that need to be detected and the position of the embedded advertisement; the field key points are represented by: P_model = [P0, P1, P2, ... Pq] (q is the number of key points), and the position of the embedded advertisement is represented by: D_ad = [x1, y1; x2, y2] (D_ad allows multiple). The position of the embedded advertisement is preferably located in the target area of ​​the field key point, and the shape of the target area can be determined according to the needs of the advertisement content, for example, it can be a rectangle, trapezoid, quadrilateral, irregular trapezoid, circle, sector, arch, etc.

[0089] On the basis of steps 110 to 130, in combination with the above steps 100 to 400, the present invention adopts a standard frame determination mechanism for multiple lenses, classifies the lenses of the video, and obtains the standard frames under the corresponding classifications. According to the relationship between the real stadium simulation model and the standard frames, a corresponding spatial conversion relationship is established. Subsequent embedded advertising will give priority to the classification of lenses, and perform spatial conversion operations on each frame image under the lens and the corresponding standard frame, which is conducive to finding the best standard frame for each lens, reducing the spatial conversion cost of the image, and improving the accuracy and stability of advertising delivery.

[0090] Optionally, the step 200 includes:

[0091] Step 210, classifying the video to be embedded with advertisements according to the shot types, and determining video data corresponding to N shot types; N is an integer greater than or equal to 1;

[0092] Here, the lens categories include but are not limited to lens types such as different shooting positions and angles.

[0093] Step 220, establishing a video data list corresponding to N lens types according to the N video data;

[0094] Step 230: Determine the standard frame image under each lens according to the video data list.

[0095] In this embodiment, according to the video data corresponding to N lens types, all video frame images of the to-be-embedded advertising video are classified into lens categories, and a video data list L_list corresponding to the N lens types is prepared, where L_list = [L1, L2, ..., Ln], where n is the total number of lenses. According to the video data list, all frame image data under each lens can be determined, including the standard frame image under each lens, and also other images under each lens except the standard frame image, thereby providing data support for subsequent advertising placement.

[0096] Optionally, the method further includes:

[0097] Step 500, obtaining the non-standard frame image of the to-be-embedded advertisement video; the non-standard frame image is the other image in the non-standard frame image except the standard frame image;

[0098] Step 600, determining the first standard frame image under the lens corresponding to the non-standard frame image according to the non-standard frame image and the video data list;

[0099] Step 700: embed advertisements into the video to be embedded with advertisements according to the first standard frame image, the first spatial conversion relationship and the first position.

[0100] The present invention also provides a processing method for obtaining the non-standard frame image of the video to be embedded with advertisements. Specifically, the non-standard frame image M of the video to be embedded with advertisements is classified into the corresponding lens type according to the video data list and assigned to L_list, and the first standard frame image under the lens corresponding to the non-standard frame image M is determined, that is, the standard frame Ln-standard under the lens corresponding to the lens of the non-standard frame image M is determined. In step 700, advertisements are embedded in the video to be embedded with advertisements according to the first standard frame image determined according to the non-standard frame image M and the first spatial conversion relationship and the first position mode of step 400. The present invention embeds advertisements in the video to be embedded with advertisements in two ways: standard frame images and non-standard frame images, thereby increasing the diversity of advertisement delivery and making the time management of advertisement delivery more refined.

[0101] Optionally, step 700 includes:

[0102] Step 710, determining a second space conversion relationship between the standard frame image and the non-standard frame image under each lens according to the non-standard frame image and the first standard frame image;

[0103] In this embodiment, the key points of the standard frame are detected based on the non-standard frame image and the first standard frame image, the key points P_M=[P0, P1,..., Pn] of the non-standard frame image M are determined, and the key points P_M of the non-standard frame image M are matched with the key points of the standard frame Ln-standard to obtain the second spatial conversion relationship Ln-M between the standard frame image and the non-standard frame image.

[0104] Step 720, determining a third space conversion relationship between the court simulation model and the non-standard frame image according to the second space conversion relationship and the first space conversion relationship;

[0105] Step 730: embed advertisements into the video to be embedded with advertisements according to the third spatial conversion relationship and the first position.

[0106] In this embodiment, according to the second spatial conversion relationship Ln-M and the first spatial conversion relationship Ln-K, the third spatial conversion relationship Ln-model between the court simulation model and the non-standard frame image is determined by matching the court key point P_model with the key point P_M of the non-standard frame image M, and according to the conversion relationship of the third spatial conversion relationship Ln-model and the first position, advertising is implanted in the image space of the non-standard frame image M.

[0107] Optionally, step 730 includes:

[0108] Step 731, determining a first area range covered by the non-standard frame image in the court simulation model according to the third space conversion relationship;

[0109] In this embodiment, according to the third space conversion relationship Ln-model, the image space of the current non-standard frame image M is converted into the court simulation model Model to obtain the first area range M-area = (x1, y1), (x2, y2), ... (xn, yn)] covered by the visual space where the non-standard frame image M is located in the court simulation model Model.

[0110] Step 732, predicting a second area range covered by the next frame of image in the stadium simulation model according to the first area range;

[0111] Step 733: embed advertisements into the video to be embedded with advertisements according to the second area range and the first location.

[0112] It should be noted that the first area range and the second area range can be understood as the size of the court contained in the current video frame image, which is less than or equal to the court range determined by the court key points. The shape of the area range can be determined according to the needs of the video content under the current lens, for example, it can be a rectangle, a regular quadrilateral, a circle, a sector, etc.

[0113] In this embodiment, in order to solve the problem of jitter of key points between frames, the present invention can adopt a point registration method to predict the second area range of the next frame image in the stadium simulation model based on the first area range, that is, using the key points of the previous frame as a reference, searching around the key points detected in the current frame for the position that best matches the points corresponding to the previous frame to ensure the visual effect of the entire advertising implantation solution.

[0114] The present invention uses the standard frame obtained by lens classification as the basis for subsequent frame conversion, calculates the positional relationship between the first area range in the simulation model under each lens and the advertisement placement, and performs advertisement placement. Compared with the matching scheme based on the real stadium simulation model, the image space of the standard frame is more similar to the image space of the subsequent frame, and can be used as the reference frame for key point detection of the subsequent frame. The calculation of the space conversion matrix is ​​more accurate, which is conducive to the accuracy of advertisement pasting.

[0115] Optionally, the step 732 includes:

[0116] Step 7321, determining a time sequence corresponding to the video frame images according to the video to be embedded with the advertisement; the time sequence includes a region range corresponding to each frame image;

[0117] Step 7322, determining the moving speed of the non-standard frame image in the stadium simulation model according to the time sequence and the first area range;

[0118] Step 7323: predicting the second area covered by the next frame of image in the court simulation model according to the moving speed.

[0119] In this embodiment, a time series corresponding to a video frame image is established, and the time series includes an area range corresponding to each frame image, which is expressed as: List_area = [M-area1, M-area2, ..., M-arean], and the speed at which the visual space where the non-standard frame image corresponding to the first area range moves in the court simulation model Model after the lens is classified is calculated based on the points in the areas of the previous and next frames, which is denoted as V, and the second area range M-area' covered by the next frame image in the court simulation model is predicted based on M-area and speed V of the non-standard frame image corresponding to the first area range.

[0120] Optionally, the step 733 includes:

[0121] Step 7331: determining a positional relationship between the second area range and the first position according to the second area range and the first position;

[0122] Step 7332: embed advertisements based on the positional relationship.

[0123] In this embodiment, the positional relationship between the first position D_ad and the second area range M-area' is determined in combination with the first position D_ad in the stadium simulation model Model, and the following situations exist: complete inclusion, intersection (incomplete inclusion), and no intersection. Based on their positional relationship, the timing of pasting the advertisement and the type of advertisement are determined. The present invention obtains the converted advertisement transformation map and coordinates based on the shooting perspective based on the advertisement coordinate range in the stadium simulation model Model and the calculated transformation information, providing a position and image basis for subsequent advertisement migration.

[0124] In the above technical solution, the present invention converts the area displayed by the non-standard image into the stadium simulation model according to the spatial relationship between the non-standard image and the stadium simulation model, establishes a regional monitoring list, calculates the speed and position of the regional movement, and monitors the area displayed by the camera in real time. Combined with the location information of the advertisement to be placed in the stadium, the timing and type of advertisement placement are determined, the diversity of advertisement placement is increased, and the time management of advertisement placement is made more refined.

[0125] Specifically, when the position relationship is that the second area range has no intersection with the first position, step 7332 includes:

[0126] Determining the timing of pasting the advertisement and the type of the advertisement according to the position relationship;

[0127] Advertisements are embedded according to the pasting timing of the advertisement and the type of the advertisement.

[0128] In this embodiment, when the first position relationship is that the second area range and the first position have no intersection, the advertisement posting timing and advertisement type are determined based on the position relationship, so that advertisements can be replaced and selected according to actual conditions, thereby increasing the diversity of advertisement placement.

[0129] Specifically, when the position relationship is that the second area completely includes the first position, step 7332 includes:

[0130] According to the positional relationship, determining that the frame image corresponding to the second area range is the updated standard frame image;

[0131] In this embodiment, according to the position of the current image display area in the stadium simulation model Model, combined with the position where the advertisement needs to be implanted, the first frame in which the area completely contains the advertisement is updated to the standard frame under the lens, and the standard frame is updated in real time, that is, the corresponding video frame image when the second area range completely contains the first position is determined as the updated standard frame image, so that the standard frame better meets the conversion requirements. It can also be understood that the first frame that is completely contained is replaced by the standard frame under its lens classification, which is recorded as LMn-standard' (this is the updated standard frame image).

[0132] Determine, according to the updated standard frame image, the first spatial conversion relationship, and the second spatial conversion relationship, that the frame image corresponding to the second area range is located at a second position corresponding to the advertisement implantation in the stadium simulation model;

[0133] In this embodiment, according to the first spatial conversion relationship Ln-K and the first position D_ad determined above, spatial conversion is performed to determine the advertising position D_ad' corresponding to the standard frame image, and then according to the second spatial conversion relationship Ln-M and the advertising position D_ad', the second position D_ad_M corresponding to the advertisement implanted in the stadium simulation model of the frame image corresponding to the second area is determined.

[0134] Determine a first material layer of the frame image and a second material layer to be embedded with an advertisement corresponding to the second area range;

[0135] The first material layer, the second material layer and the second position layer are merged to embed the advertisement.

[0136] In this embodiment, the first material layer of the frame image corresponding to the second area range is determined, that is, the person and sphere information is detected and the contour mask is extracted, and the second material layer of the advertisement to be embedded is determined, that is, the advertisement image is extracted. Finally, the second position D_ad_M, the advertisement image (the second material layer), and the contour mask of the person and sphere in the current non-standard frame image (the first material layer) are combined and fused according to a certain hierarchical relationship (Poisson fusion or other fusion methods, with no limitation to the method) to obtain the final advertisement presentation effect and complete the embedded advertisement.

[0137] It should be noted that the present invention uses a classification network to determine whether a human portrait exists in the image of the advertising area based on the obtained advertising area to be converted. If there is no human information, the advertisement is directly pasted. If there is human information, subsequent portrait segmentation operations are performed. This operation can improve the running speed of the overall solution and make it real-time.

[0138] It should also be noted that the present invention separates the portrait from the background through a portrait segmentation algorithm, separates the sphere through a sphere detection algorithm, and finally integrates the advertisement image, portrait, sphere, and background in layers. Here, because of the portrait and sphere detection and identification, the advertisement will not block the video content and affect the viewing quality; and the placement of advertisements can be customized according to user needs in terms of pattern content, time placement, duration, etc.

[0139] In a specific embodiment, the advertisement placement of the present invention is based on the modeling of real stadium scenes, and the advertisements are placed in fixed positions of the stadium simulation model according to demand. The video shots are classified, and a standard frame for advertisement space conversion is established for each shot to provide a more accurate spatial reference for subsequent frames. At the same time, the shot classification is combined with the area range of the current area image under the video shot in the real stadium simulation model to calculate the timing and type of advertisement placement, realize the precise migration of advertisements from different perspectives and at different times, and detect and identify people and balls in the stadium to prevent advertisements from blocking video content, so that the advertisements are realistically embedded in the video.

[0140] In summary, the present invention has the following advantages in advertising:

[0141] The present invention selects standard frames for multiple lenses, provides an accurate transformation space template for subsequent frames, and improves the accuracy of advertising placement; the present invention can accurately calculate the timing and type of advertising placement according to lens classification and the regional range of the current video lens image in the real stadium simulation model, so that the advertising placement time can be controlled more freely and the placement means can be more abundant; the present invention updates the standard frame under the corresponding lens according to the lens viewing angle range and the position of the advertisement, improves the accuracy of advertising space conversion, and makes the advertisement pasting more realistic; the advertisement placed by the present invention can be adjusted according to different viewing angles in the video, so that the advertisement changes with the position of the viewing angle, and is fixedly placed at the inherent position of the stadium, so that the advertisement placement does not feel inconsistent.

[0142] Optionally, in step 200, determining the standard frame image under each shot corresponding to the video to be embedded with the advertisement further includes:

[0143] Determining the standard frame image under each shot corresponding to the video to be embedded with the advertisement also includes:

[0144] Obtaining the camera position information corresponding to the advertisement video to be embedded and the corresponding image information under the current lens;

[0145] When any of the following preset conditions is met, the standard frame corresponding to the image information under the current lens is updated;

[0146] Wherein, the preset conditions are:

[0147] The resolution of the image information under the current lens is lower than the first preset range;

[0148] In this embodiment, when determining the standard frame corresponding to the current lens angle of view according to the image information under the current lens corresponding to the to-be-embedded advertisement video, it can be understood that when the current lens angle of view is a one-shot view, the resolution of the current lens angle of view is lower than the first preset range, that is, when the clarity of the current lens angle of view is lower than the threshold, the standard frame corresponding to the current lens angle of view is updated. The first preset range is a preset range value for video resolution, which can be set according to specific needs.

[0149] The current shot switching corresponding to the video to be embedded with the advertisement;

[0150] Here, when the lens is switched, the standard frame corresponding to the current lens viewing angle is updated.

[0151] The corresponding camera position information exceeds a first preset distance.

[0152] It should be noted that, when the current lens angle of view is a one-shot shot, moving from the leftmost to the rightmost of the court, when the leftmost court key point away from the court exceeds the first preset distance, that is, when the corresponding camera position information is switched, the standard frame corresponding to the current lens angle of view is updated. The first preset distance is a preset value about the distance. For example, the real court is determined as four court key points, that is, the four corners of the real court, which are recorded as points A, B, C, and D in sequence. The first preset distance is the preset distance value set at each corner.

[0153] The present invention is based on real stadium information modeling, which combines the switching of video lenses and determines the timing and type of advertisement pasting based on the positioning of the stadium simulation position, finds the best conversion standard frame, and realizes refined time management and realistic stadium advertisement precise delivery.

[0154] like Figure 2 As shown in a specific overall flow chart, the method includes:

[0155] Modeling is performed based on real stadium data to determine the key points P_model and the first position D_ad of the stadium. For the input live football game video, a list of lens types L_list is established to classify the lenses for each acquired frame. The standard LMn-standard is initialized for each lens: the first frame in which the advertisement appears completely in the picture under the lens is taken as the initialization standard frame, the second key point of the standard frame is matched with the key point in the real stadium simulation model, the first space conversion relationship Ln-K between the stadium simulation model and the standard frame is established, the advertisement space is converted, and the advertisement position D_ad' under the standard frame is obtained.

[0156] For each non-standard frame image M of the video video, the corresponding lens type is classified and assigned to L_list, the standard frame Ln-standard belonging to the lens is determined, the corresponding key points of M are detected to obtain P_M, and the key points are matched with the key points of the standard frame Ln-standard to obtain the spatial conversion relationship between the standard frame and the current non-standard frame image M; the spatial conversion relationship Ln-model is matched with the stadium key points P_model in the stadium simulation model model to obtain the spatial conversion relationship Ln-model between the stadium simulation model and the current non-standard frame image M.

[0157] According to the conversion relationship of Ln-model, the image space of the current non-standard frame image M is converted to the court simulation model model to obtain the range M-area covered by the visual space where the non-standard frame image M is located in the court simulation model model.

[0158] Establish a time series List_area, calculate the speed V of the non-standard frame image M in the visual space in the stadium simulation model model according to the points in the areas of the previous and next frames, infer the M-area' of the next frame according to the M-area and speed V of the current non-standard frame image M, and judge the positional relationship between D_ad and M-area' in combination with the first position D_ad in the stadium simulation model model, and then judge the timing of pasting the advertisement and the type of advertisement based on their positional relationship.

[0159] When the positional relationship between D_ad and M-area' is complete inclusion, the first frame M' that is completely included is replaced with a standard frame under its lens classification, which is recorded as LMn-standard'.

[0160] The image space of the advertisement is transformed according to the second space conversion relationship Ln-M between the standard frame and the current non-standard frame image M, the person and sphere information in the current non-standard frame image M is detected and the contour mask is extracted, and the advertisement position D_ad' in the standard frame is converted through the space conversion relationship Ln-M to obtain the second position D_ad_M where the current image needs to be pasted.

[0161] Combined with the second position D_ad_M, the advertisement image, the contour mask of the person and the sphere in the current non-standard frame image M, the multiple elements are fused according to a certain hierarchical relationship to obtain the final advertisement presentation effect.

[0162] In summary, the video advertising implantation method of the present invention can realize batch automated analysis of massive videos, and the video advertising delivery efficiency is high; and different advertising contents can be implanted in the same video according to the needs of advertising delivery, and the reuse rate is high; in addition, the overall solution processing time is no more than 30ms, which can ensure real-time processing; and it supports the implantation of various forms of picture advertisements, and is not limited to the advertising content, and the delivery content is flexible.

[0163] like Figure 3 As shown, the optional real-time example of the present invention provides an advertisement embedding device, including:

[0164] An acquisition module 10 is used to acquire a key point of the court corresponding to the court simulation model and a first position corresponding to the embedded advertisement;

[0165] The first processing module 20 is used to obtain a video to be embedded with advertisements, and determine a standard frame image under each shot corresponding to the video to be embedded with advertisements, wherein the standard frame image is a video frame image under each shot where an embedded advertisement appears for the first time;

[0166] A second processing module 30 is used to determine a first spatial conversion relationship between the court simulation model and the standard frame image according to the standard frame image and the key points of the court;

[0167] The third processing module 40 is configured to embed advertisements into the video to be embedded with advertisements according to the first spatial conversion relationship and the first position.

[0168] Optionally, the first processing module 20 includes:

[0169] The first determination submodule is used to classify the video to be embedded with advertisements according to the shot type, and determine the video data corresponding to N shot types; N is an integer greater than or equal to 1;

[0170] Establishing a submodule for establishing a video data list corresponding to N lens types according to N video data;

[0171] The second determination submodule is used to determine the standard frame image under each lens according to the video data list.

[0172] Optionally, the device further comprises:

[0173] A second acquisition module is used to acquire the non-standard frame image to be embedded in the advertisement video; the non-standard frame image is other images in the non-standard frame image except the standard frame image;

[0174] A first determining module, configured to determine, according to the non-standard frame image and the video data list, a first standard frame image under a lens corresponding to the non-standard frame image;

[0175] The fourth processing module is used to embed advertisements into the video to be embedded with advertisements according to the first standard frame image, the first spatial conversion relationship and the first position.

[0176] Optionally, the fourth processing module includes:

[0177] A third determining submodule, configured to determine a second spatial conversion relationship between the standard frame image and the non-standard frame image under each lens according to the non-standard frame image and the first standard frame image;

[0178] a fourth determining submodule, configured to determine a third spatial conversion relationship between the court simulation model and the non-standard frame image according to the second spatial conversion relationship and the first spatial conversion relationship;

[0179] The fifth determining submodule is configured to embed advertisements into the video to be embedded with advertisements according to the third spatial conversion relationship and the first position.

[0180] Optionally, the fifth determining submodule includes:

[0181] A first determining unit, configured to determine a first area range covered by the non-standard frame image in the court simulation model according to the third spatial conversion relationship;

[0182] A prediction unit, configured to predict a second area range covered by a next frame of image in the court simulation model according to the first area range;

[0183] The first processing unit is configured to embed advertisements into the video to be embedded with advertisements according to the second area range and the first position.

[0184] Optionally, the prediction unit includes:

[0185] A first determining subunit is used to determine a time sequence corresponding to a video frame image according to the video to be embedded with an advertisement; the time sequence includes a region range corresponding to each frame image;

[0186] A second determining subunit is used to determine the moving speed of the non-standard frame image in the stadium simulation model according to the time sequence and the first area range;

[0187] The prediction subunit is used to predict the second area covered by the next frame of image in the stadium simulation model according to the moving speed.

[0188] Optionally, the first processing unit includes:

[0189] A third determining subunit, configured to determine a positional relationship between the second area range and the first position according to the second area range and the first position;

[0190] The first processing subunit is used to insert advertisements according to the position relationship.

[0191] Optionally, when the position relationship is that the second area range has no intersection with the first position, the first processing subunit is specifically configured to:

[0192] Determining the timing of pasting the advertisement and the type of the advertisement according to the position relationship;

[0193] Advertisements are embedded according to the pasting timing of the advertisement and the type of the advertisement.

[0194] Optionally, when the position relationship is that the second area range completely includes the first position, the first processing subunit is specifically configured to:

[0195] According to the positional relationship, determining that the frame image corresponding to the second area range is the updated standard frame image;

[0196] Determine, according to the updated standard frame image, the first spatial conversion relationship, and the second spatial conversion relationship, that the frame image corresponding to the second area range is located at a second position corresponding to the advertisement implantation in the stadium simulation model;

[0197] Determine a first material layer of the frame image and a second material layer to be embedded with an advertisement corresponding to the second area range;

[0198] The first material layer, the second material layer and the second position layer are merged to embed the advertisement.

[0199] Optionally, the first processing module 20 further includes:

[0200] An acquisition unit, used for acquiring the camera position information corresponding to the advertisement video to be embedded and the image information under the corresponding current lens;

[0201] An updating unit, used to update the standard frame corresponding to the image information under the current lens when any of the following preset conditions is met;

[0202] Wherein, the preset conditions are:

[0203] The resolution of the image information under the current lens is lower than the first preset range;

[0204] The current shot switching corresponding to the video to be embedded with the advertisement;

[0205] The corresponding camera position information exceeds a first preset distance.

[0206] Optionally, the device further comprises:

[0207] The third acquisition module is used to acquire the stadium information and the advertisement information to be implanted;

[0208] A second determination module is used to determine key points of the court according to the court venue information, and to determine the location of the implanted advertisement according to the advertisement information to be implanted;

[0209] A construction module is used to construct a stadium simulation model according to the key points of the stadium and the positions of the embedded advertisements.

[0210] A readable storage medium according to an embodiment of the present invention stores a program or instruction thereon. When the program or instruction is executed by a processor, the steps in the advertising implantation method as described above are implemented and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0211] The processor is the processor in the advertisement embedding method described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0212] In the embodiment of the present invention, module can be implemented with software so that it can be executed by various types of processors. For example, an executable code module of an identification can include one or more physical or logical blocks of computer instructions, for example, it can be constructed as an object, process or function. Nevertheless, the executable code of the identified module does not need to be physically located together, but can include different instructions stored in different positions, and when these instructions are logically combined together, it constitutes a module and realizes the specified purpose of the module.

[0213] In fact, executable code module can be a single instruction or many instructions, and can even be distributed on a plurality of different code segments, distributed among different programs, and distributed across a plurality of memory devices. Similarly, operating data can be identified in the module, and can be implemented and organized in the data structure of any appropriate type according to any appropriate form. The operating data can be collected as a single data set, or can be distributed in different locations (including on different storage devices), and can only be present on a system or network as an electronic signal at least in part.

[0214] When a module can be implemented by software, considering the level of existing hardware technology, a person skilled in the art can build a corresponding hardware circuit to implement the corresponding function of the module that can be implemented by software without considering the cost. The hardware circuit includes a conventional very large scale integration (VLSI) circuit or gate array and existing semiconductors such as logic chips, transistors, or other discrete components. The module can also be implemented by a programmable hardware device, such as a field programmable gate array, a programmable array logic, a programmable logic device, etc.

[0215] The above exemplary embodiments are described with reference to the accompanying drawings, and many different forms and embodiments are feasible without departing from the spirit and teachings of the present invention. Therefore, the present invention should not be constructed as a limitation of the exemplary embodiments proposed herein. More specifically, these exemplary embodiments are provided so that the present invention will be perfect and complete, and the scope of the present invention will be conveyed to those who are familiar with the technology. In these figures, the component sizes and relative sizes may be exaggerated for clarity. The terms used here are only based on the purpose of describing specific exemplary embodiments and are not intended to be limiting. As used herein, unless the text clearly indicates otherwise, the singular forms "one", "an" and "the" are intended to include these multiple forms. It will be further understood that the terms "including" and / or "comprising" when used in this specification indicate the presence of the features, integers, steps, operations, components and / or components, but do not exclude the presence or increase of one or more other features, integers, steps, operations, components, components and / or their groups. Unless otherwise indicated, when stated, a range of values ​​includes the upper and lower limits of that range and any subranges therebetween.

[0216] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. An advertisement embedding method, characterized in that: include: Obtaining key points of the court corresponding to the court simulation model and the first position of the corresponding embedded advertisement; Acquire a video to be embedded with advertisements, and determine a standard frame image under each shot corresponding to the video to be embedded with advertisements, wherein the standard frame image is a video frame image under each shot where an embedded advertisement appears for the first time; Determining a first spatial conversion relationship between the court simulation model and the standard frame image according to the standard frame image and the court key points; According to the first spatial conversion relationship and the first position, inserting advertisements into the video to be inserted with advertisements; Among them, obtaining the video to be embedded with advertisements and determining the standard frame image under each lens corresponding to the video to be embedded with advertisements, includes: classifying the video to be embedded with advertisements according to the lens category, and determining the video data corresponding to N lens types; N is an integer greater than or equal to 1; establishing a video data list corresponding to N lens types according to the N video data; and determining the standard frame image under each lens according to the video data list.

2. The method according to claim 1, characterized in that: The method further comprises: Acquire the non-standard frame image of the advertisement video to be embedded; the non-standard frame image is other images except the standard frame image in all frame image data under each lens; Determine, according to the non-standard frame image and the video data list, a first standard frame image under a lens corresponding to the non-standard frame image; Advertisements are embedded in the video to be embedded with advertisements according to the first standard frame image, the first spatial conversion relationship, and the first position.

3. The method according to claim 2, characterized in that According to the first standard frame image, the first spatial conversion relationship and the first position, inserting advertisements into the video to be inserted with advertisements includes: Determine, according to the non-standard frame image and the first standard frame image, a second spatial conversion relationship between the standard frame image and the non-standard frame image under each lens; Determining a third spatial conversion relationship between the court simulation model and the non-standard frame image according to the second spatial conversion relationship and the first spatial conversion relationship; Advertisements are embedded in the video to be embedded with advertisements according to the third spatial conversion relationship and the first position.

4. The method according to claim 3, characterized in that According to the third spatial conversion relationship and the first position, inserting advertisements into the video to be inserted with advertisements includes: Determining a first area range covered by the non-standard frame image in the court simulation model according to the third spatial conversion relationship; According to the first area range, predicting a second area range covered by a next frame of image in the court simulation model; Advertisements are embedded in the video to be embedded with advertisements according to the second area range and the first position.

5. The method according to claim 4, characterized in that Predicting a second area range covered by a next frame of image in the stadium simulation model according to the first area range includes: Determine a time sequence corresponding to a video frame image according to the video to be embedded with an advertisement; the time sequence includes a region range corresponding to each frame image; Determining a moving speed of the non-standard frame image in the court simulation model according to the time sequence and the first area range; According to the moving speed, a second area range covered by the next frame of image in the court simulation model is predicted.

6. The method according to claim 4, characterized in that According to the second area range and the first position, inserting advertisements into the video to be inserted with advertisements includes: Determine a positional relationship between the second area range and the first position according to the second area range and the first position; Advertisements are embedded based on the positional relationship.

7. The method according to claim 6, characterized in that When the position relationship is that the second area range has no intersection with the first position, According to the position relationship, the embedded advertisement is performed, including: Determining the timing of pasting the advertisement and the type of the advertisement according to the position relationship; Advertisements are embedded according to the pasting timing of the advertisement and the type of the advertisement.

8. The method according to claim 6, characterized in that When the position relationship is that the second area completely includes the first position, According to the position relationship, the embedded advertisement is performed, including: According to the positional relationship, determining that the frame image corresponding to the second area range is the updated standard frame image; Determine, according to the updated standard frame image, the first spatial conversion relationship, and the second spatial conversion relationship, that the frame image corresponding to the second area range is located at a second position corresponding to the advertisement implantation in the stadium simulation model; Determine a first material layer of the frame image and a second material layer to be embedded with an advertisement corresponding to the second area range; The first material layer, the second material layer and the second position layer are merged to embed the advertisement.

9. The method according to claim 1, characterized in that: Determining the standard frame image under each shot corresponding to the video to be embedded with the advertisement also includes: Obtaining the camera position information corresponding to the advertisement video to be embedded and the corresponding image information under the current lens; When any of the following preset conditions is met, the standard frame corresponding to the image information under the current lens is updated; Wherein, the preset conditions are: The resolution of the image information under the current lens is lower than the first preset range; The current shot switching corresponding to the video to be embedded with the advertisement; The corresponding camera position information exceeds a first preset distance.

10. The method according to claim 1, characterized in that Build a stadium simulation model, including: Obtain stadium information and advertising information to be embedded; Determine key points of the court according to the court venue information, and determine the location of the implanted advertisement according to the advertisement information to be implanted; A stadium simulation model is constructed according to the key points of the stadium and the positions of the embedded advertisements.

11. An advertisement placement device, characterized in that: include: An acquisition module, used for acquiring key points of the court corresponding to the court simulation model and a first position corresponding to the embedded advertisement; The first processing module is used to obtain the video to be embedded with advertisements, and determine the standard frame image under each lens corresponding to the video to be embedded with advertisements, wherein the standard frame image is the video frame image in which the embedded advertisement appears for the first time under each lens; wherein the first processing module includes: a first determination submodule, used to classify the video to be embedded with advertisements according to the lens category, and determine the video data corresponding to N lens types; N is an integer greater than or equal to 1; an establishment submodule, used to establish a video data list corresponding to N lens types according to the N video data; and a second determination submodule, used to determine the standard frame image under each lens according to the video data list; A second processing module, configured to determine a first spatial conversion relationship between the court simulation model and the standard frame image according to the standard frame image and the key points of the court; The third processing module is used to embed advertisements into the video to be embedded with advertisements according to the first spatial conversion relationship and the first position.

12. A readable storage medium having a program or instruction stored thereon, characterized in that: When the program or instruction is executed by a processor, the steps in the advertisement implantation method according to any one of claims 1 to 10 are implemented.

Images