Image processing method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides an image processing method and device, electronic equipment and a storage medium, and is applied to the technical field of image processing. The image processing method comprises the steps that a video frame to be played, advertisement materials and candidate advertisement display positions are acquired, and the candidate advertisement display positions represent candidate display areas for displaying the advertisement materials in the video frame; non-transparent pixel points of the candidate display area and non-transparent pixel points of the advertisement material are determined; calculating an overlapping index of the non-transparent pixel points of the candidate display area and the non-transparent pixel points of the advertisement material; and under the condition that the overlapping index does not meet the preset index, taking the candidate advertisement display position as a target advertisement display position. By applying the image processing method and device, the electronic equipment and the storage medium provided by the embodiment of the invention, the video watching experience of a user can be improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image processing method, device, electronic equipment and storage medium. Background Art

[0002] In the related art, video players usually obtain advertising materials from the server and overlay them in a specific area of ​​the video playback interface. However, this method of displaying advertisements in a fixed area during video playback often blocks the video screen. For example, a video player displays an advertisement in the lower right corner of the video playback interface, and the main character of a video is also fixed in the lower right corner of the video playback interface, so blockage occurs.

[0003] Furthermore, if the blocked area is the part containing the main content of the video, it will make it difficult for the user to obtain the key information of the video, affecting the user's experience of watching the video. Summary of the invention

[0004] The purpose of the embodiments of the present invention is to provide an image processing method, device, electronic device and storage medium to improve the user's experience of watching videos. The specific technical solution is as follows:

[0005] In a first aspect of the present invention, an image processing method is provided, which is applied to a client, and the method comprises:

[0006] Acquire a video frame to be played, an advertisement material, and a candidate advertisement display position, wherein the candidate advertisement display position represents a candidate display area for displaying the advertisement material in the video frame;

[0007] Determine the non-transparent pixels of the candidate display area and the non-transparent pixels of the advertising material;

[0008] Calculate the overlap index of the non-transparent pixels of the candidate display area and the non-transparent pixels of the advertising material;

[0009] When the overlap index does not meet a preset index, the candidate advertisement display position is used as a target advertisement display position.

[0010] Optionally, the method further comprises:

[0011] When the overlap index meets the preset index, determining the adjusted advertisement display position;

[0012] The adjusted ad display position is used as the current candidate ad display position, and the execution is returned: determining the non-transparent pixels of the candidate display area and the non-transparent pixels of the advertising material; calculating the overlapping index of the non-transparent pixels of the candidate display area and the non-transparent pixels of the advertising material, until the overlapping index calculated based on a candidate ad display position does not meet the preset index, then the candidate ad display position corresponding to the candidate ad display position whose calculated overlapping index does not meet the preset index is used as the target ad display position.

[0013] Optionally, obtaining the video frame to be played includes:

[0014] Acquire a plurality of video frames that are not played in the currently played video, wherein the candidate display area for displaying the advertisement material in each video frame in the plurality of video frames is the same;

[0015] Select a video frame as the video frame to be played;

[0016] After calculating the overlap index of the non-transparent pixels of the candidate display area and the non-transparent pixels of the advertising material, the method further includes:

[0017] Determining whether the overlap index meets a preset index;

[0018] If the overlapping index does not meet the preset index, return to the execution: select a video frame as the step of the video frame to be played, until the overlapping index calculated based on each video frame in the multiple video frames does not meet the preset index, and the candidate display area corresponding to the multiple video frames is used as the target advertising display position.

[0019] Optionally, determining the adjusted advertisement display position includes:

[0020] receiving a move operation on the advertising material, and / or a scaling operation on the advertising material in a virtual container;

[0021] Based on the moving operation and / or the zooming operation, an adjusted advertisement display position is determined.

[0022] Optionally, before determining the non-transparent pixels of the candidate display area and the non-transparent pixels of the advertising material, the method further includes:

[0023] Constructing a virtual container; the size of the virtual container is the same as the size of the video playback window playing the video frame;

[0024] Corresponding to the position of the video frame in the video playback window, displaying the video frame in the virtual container;

[0025] Corresponding to the position of the candidate display area in the video playback window, displaying the advertising material in the virtual container;

[0026] The receiving a moving operation on the advertising material and / or a scaling operation on the advertising material in the virtual container includes:

[0027] A move operation on the advertising material in the virtual container and / or a zoom operation on the advertising material in the virtual container are received.

[0028] Optionally, after determining the target advertisement display position, the method further includes:

[0029] When the playing time of the video frame is reached, the advertising material is displayed at the target advertising display position.

[0030] Optionally, the determining the non-transparent pixels of the candidate display area and the non-transparent pixels of the advertising material includes:

[0031] Determine the color value of each pixel in the candidate display area and the color value of each pixel in the advertising material;

[0032] Pixels whose color values ​​are not less than the preset transparency threshold are regarded as non-transparent pixels.

[0033] Optionally, the overlap index indicates the overlap ratio between the non-transparent pixels of the advertising material and the non-transparent pixels of the candidate display area when the advertising material is superimposed on the candidate display area; if the overlap index does not meet the preset index, it indicates that the overlap ratio does not exceed the preset ratio; or,

[0034] The overlap index represents the ratio of the number of non-transparent pixels of the advertising material to the number of non-transparent pixels of the candidate display area; if the overlap index does not meet the preset index, it means that the ratio does not exceed the preset ratio.

[0035] In a second aspect of the present invention, an image processing device is provided, which is applied to a client. The method includes:

[0036] An acquisition module, used to acquire a video frame to be played, an advertisement material and a candidate advertisement display position, wherein the candidate advertisement display position represents a candidate display area for displaying the advertisement material in the video frame;

[0037] A first determination module, used to determine the non-transparent pixels of the candidate display area and the non-transparent pixels of the advertising material;

[0038] A calculation module, used for calculating the overlap index of the non-transparent pixels of the candidate display area and the non-transparent pixels of the advertising material;

[0039] The second determining module is configured to use the candidate advertisement display position as a target advertisement display position when the overlap index does not meet a preset index.

[0040] According to a third aspect of an embodiment of the present invention, there is provided an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0041] The memory is used to store computer programs;

[0042] The processor is used to implement the image processing method described in the first aspect when executing the program stored in the memory.

[0043] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the image processing method described in the first aspect is implemented.

[0044] According to another aspect of the present invention, there is provided a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the image processing method described in the first aspect.

[0045] In the embodiment of the present invention, by obtaining the video frame to be played, the advertising material, and the candidate display area candidate advertising display position indicating the display of the advertising material in the video frame, and determining the non-transparent pixels of the candidate display area and the non-transparent pixels of the advertising material, the overlap index of the non-transparent pixels of the candidate display area and the non-transparent pixels of the advertising material is calculated, so that when the overlap index does not meet the preset index, the candidate advertising display position is used as the target advertising display position. That is, only when the overlap index does not meet the preset index, the candidate advertising display position is used as the target advertising display position, so that when the advertising material is displayed in the target advertising display position during the video playback process, the occlusion of the content in the video frame can be reduced, thereby improving the user's experience of watching the video. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.

[0047] Figure 1 A flow chart of an image processing method provided by an embodiment of the present invention;

[0048] Figure 2a A schematic diagram of a video playback window display screen provided by an embodiment of the present invention;

[0049] Figure 2b Another schematic diagram of a video playback window display screen provided by an embodiment of the present invention;

[0050] Figure 3 A schematic diagram of a specific area in a video frame provided by an embodiment of the present invention;

[0051] Figure 4 A schematic diagram of an advertisement material blocking a video frame content provided by an embodiment of the present invention;

[0052] Figure 5 Another flow chart of the image processing method provided by the embodiment of the present invention;

[0053] Figure 6a A schematic diagram of the location of mobile advertising materials provided by an embodiment of the present invention;

[0054] Figure 6b A schematic diagram of changing the size of an advertisement material provided by an embodiment of the present invention;

[0055] Figure 6c A flowchart of a specific embodiment provided for an embodiment of the present invention;

[0056] Figure 7 A schematic diagram of the structure of an image processing device provided by an embodiment of the present invention;

[0057] Figure 8 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0058] The technical solutions in the embodiments of the present invention will be described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0059] Overlaying ads in videos can effectively increase ad exposure, thereby enhancing ad effects and promoting ad monetization. In the current conventional video ad delivery method, the video player obtains ad materials from the server at a specific time and overlays them with the video in a specific area. These ad materials are usually regular shapes with transparent and non-transparent parts, such as square images.

[0060] However, when related technologies overlay advertisements during video playback, these advertisements may block the main content of the video and affect the user's viewing experience.

[0061] In view of the above problems, an embodiment of the present invention provides an image processing method. The image processing method provided by the embodiment of the present invention can be applied to a client, for example, a browser, a player that is playing a video, etc.

[0062] like Figure 1As shown, the image processing method provided by the embodiment of the present invention may include:

[0063] S101, obtaining a video frame to be played, an advertisement material, and a candidate advertisement display position, where the candidate advertisement display position represents a candidate display area for displaying the advertisement material in the video frame;

[0064] S102, determining non-transparent pixels of the candidate display area and non-transparent pixels of the advertising material;

[0065] S103, calculating the overlap index of the non-transparent pixels of the candidate display area and the non-transparent pixels of the advertising material;

[0066] If the overlap index does not meet the preset index, execute S104;

[0067] S104: Use the candidate advertisement display position as the target advertisement display position.

[0068] In the embodiment of the present invention, by obtaining the video frame to be played, the advertising material, and the candidate display area candidate advertising display position indicating the display of the advertising material in the video frame, and determining the non-transparent pixels of the candidate display area and the non-transparent pixels of the advertising material, the overlap index of the non-transparent pixels of the candidate display area and the non-transparent pixels of the advertising material is calculated, so that when the overlap index does not meet the preset index, the candidate advertising display position is used as the target advertising display position. That is, only when the overlap index does not meet the preset index, the candidate advertising display position is used as the target advertising display position, so that when the advertising material is displayed in the target advertising display position during the video playback process, the occlusion of the content in the video frame can be reduced, thereby improving the user's experience of watching the video.

[0069] The preset index may represent an index of video frame occlusion by displaying the ad material at the ad display position. The candidate ad display position may also be understood as the current ad display position. It is assumed that the ad material is displayed at the current ad display position to calculate the corresponding overlap index, and only when the overlap index does not meet the preset index, the current ad display position is used as the target ad display position.

[0070] In S101, the video frame to be played may be a video frame in a video currently being played.

[0071] In one possible implementation, a video segment in a currently playing video may be obtained, for example, a video segment including multiple video frames played after a current moment in the video.

[0072] In a specific implementation process, the video frame to be played can be selected from the acquired video clips. For example, a frame in the video clip is randomly selected as the video frame to be played, or each frame in the video frame can be selected in sequence as the video frame to be played.

[0073] Different videos may have different aspect ratios, such as 4:3, 16:9, etc. Therefore, when playing different videos, the presentation effect of the video playback window may also be different. For example, videos with certain aspect ratios can occupy the entire screen of the video playback window, such as Figure 2a As shown, some proportions of videos will display a blank area when played in the video player window, such as Figure 2b As shown, the oblique line areas in the upper and lower parts of the picture are areas without picture.

[0074] At the same time, since the user can also adjust the display ratio of the video screen in the video playback window during the video viewing process, for example, full, 75%, 50%, etc., the presentation effect of the same video in the video playback window may also change.

[0075] Therefore, for the image processing process under different presentation effects, the embodiment of the present invention can obtain the set of each picture presented in the video playback window after the current moment in the video being played as a video clip. It can be understood that the picture presented in the video playback window includes the video frame image of the video being played. Therefore, the video clip obtained in the embodiment of the present invention also includes the set of multiple video frames played after the current moment in the video being played. In this way, even if the presentation effects are different, the aspect ratio of the obtained video clip is the same, which improves the adaptability of the image processing method to different videos and different presentation effects.

[0076] During video playback, you can obtain video clips in real time, that is, obtain video clips with the current time as the starting time.

[0077] You can also obtain video clips at a specific time point. A specific time point refers to a time point after the current moment in the video; this time point is usually a certain time interval from the current moment, such as 30 seconds. Therefore, this acquisition method can also be called pre-acquiring video clips. In the scenario of pre-acquiring video clips, the device acquires video clips with the specific time point as the starting moment. For example, the video clip is acquired starting from 30 seconds after the current moment. Pre-acquiring video clips in this way reserves time for subsequent processing and improves the continuity of displaying advertisements in the video.

[0078] The obtained advertising materials can be of different types. For example, pictures, small clips, etc. The advertising materials displayed during video playback can be compared to stickers added to the video playback interface. During video playback, you can request advertising materials from the server; you can also obtain advertising materials cached by yourself.

[0079] S102, the color values ​​of the pixels can be used to calculate whether the pixels in the candidate display area are non-transparent pixels and whether the pixels in the advertising material are non-transparent pixels, thereby determining the non-transparent pixels in the candidate display area and the non-transparent pixels in the advertising material.

[0080] Specifically, the color value of each pixel in the candidate display area and the color value of each pixel in the advertising material can be determined; and the pixels whose color values ​​are not less than a preset transparency threshold are regarded as non-transparent pixels.

[0081] Calculating whether a pixel is a non-transparent pixel can also be understood as calculating the transparency information of the pixel.

[0082] Transparency information is used to indicate how much effective information the pixel contains. The less effective information a pixel contains, the higher the transparency of the pixel; the more effective information a pixel contains, the lower the transparency of the pixel. In an embodiment of the present invention, pixels are divided into transparent pixels that do not contain effective information and non-transparent pixels that contain effective information.

[0083] In one achievable method, the amount of effective information contained in a pixel can be measured by the color value of the pixel. The higher the color value of the pixel, the more effective information the pixel contains; the lower the color value of the pixel, the less effective information the pixel contains.

[0084] For example, the degree of transparency of the pixel points can be determined according to the color values ​​of the pixel points in the video frames and the advertising material included in the video clip. Detecting the pixel information of each video frame and the advertising material includes: detecting the color value of each pixel point in each video frame and the advertising material in the video clip; setting the transparency information of the pixel points whose color value is less than the transparency threshold to transparent; and setting the transparency information of the pixel points whose color value is not less than the transparency threshold to non-transparent.

[0085] The transparency threshold can also be understood as a valid information threshold. If the color value is less than the transparency threshold, it is considered that the pixel contains less valid information. At this time, the transparency information of the pixel can be set to transparent, that is, the pixel is a transparent pixel; if the color value is not less than the transparency threshold, it is considered that the pixel contains more valid information. At this time, the transparency information of the pixel can be set to non-transparent, that is, the pixel is a non-transparent pixel. The specific value of the transparency threshold can be determined based on actual needs, and the embodiment of the present invention is not limited.

[0086] In one feasible method, the RGBA color representation (additional transparency color representation) can be used to represent the pixel information of each pixel point, and the pixel information of each pixel point is stored in a two-dimensional array (hereinafter referred to as a two-dimensional array storing pixel information as a pixel set). Among them, every four elements in the array represent the red (R), green (G), blue (B) and transparency (Alpha, referred to as A) values ​​of a pixel. If the A value of a pixel is greater than 0, it means that the pixel is visible, that is, non-transparent. When the device detects that the A value of a pixel is greater than 0, it can determine that it is a non-transparent pixel point.

[0087] The image processing method provided by the embodiment of the present invention can be applied to a browser, and the player that is playing the video can include a player in the browser.

[0088] In one possible implementation, the component that plays the video in the browser, that is, the player, can be understood as a secondary packaging component of a Video element. The embodiment of the present invention can use the Canvas element in the browser to construct a virtual container, and use the virtual container to determine the non-transparent pixels of the candidate display area and the non-transparent pixels of the advertising material.

[0089] Construct a virtual container. In order to better restore the actual video playback scene, the size of the virtual container can be the same as the video playback window of the currently playing video, that is, the size of each video frame. Place each video frame of the video clip in the virtual container; thereby fully restoring the actual video playback scene.

[0090] Place the creative at its initial position within the virtual container.

[0091] The initial position of the advertising material in the virtual container can also be understood as the initial position of the advertising material in the video frame to be played.

[0092] In the embodiment of the present invention, a specific area in the video frame can be pre-selected as the area where the advertising material can be superimposed. Usually, the area around the video screen, that is, the area around the virtual container in the embodiment of the present invention, is often set as a blank display area. The probability of screen occlusion when displaying advertisements in this area is low. Therefore, the specific area can include the video frame, that is, the edge area of ​​the virtual container with a preset ratio. For example, when the preset ratio is 1 / 4, Figure 3 As shown, the specific area may include an area of ​​the upper 1 / 4 area, a lower 1 / 4 area, a left 1 / 4 area, and a right 1 / 4 area of ​​the video frame, that is, Figure 3 The dashed shaded area.

[0093] The specific area may also include a designated non-edge area, for example, only the area in the lower right 1 / 4 of the video frame is selected as the specific area.

[0094] The initial position may include any position in the specific area, or may include a position in the specific area that contains less effective information in most video frames obtained through big data statistics, for example, positions at the four corners of the video frame.

[0095] After placing the video frame to be played and the advertising material in the virtual container, the non-transparent pixels of the candidate display area and the non-transparent pixels of the advertising material can be determined through the virtual container, which can also be understood as detecting the pixel information of the video frame to be played and the advertising material.

[0096] In one practicable manner, the color value of each pixel can be detected by the preset function of the virtual container, so as to determine the transparency of each pixel according to the color value and obtain the pixel information of the video frame and the advertising material. The specific determination method can refer to the above content and will not be repeated here.

[0097] The embodiment of the present invention only detects the pixel information of the video frame and the advertising material itself, which requires few computing resources and has low computing cost. The pixel information can be used to determine the overlap index of the advertising material and the video frame at low cost to judge whether the video content is blocked.

[0098] In S103, the overlap index represents the overlap ratio between the non-transparent pixels of the advertising material and the non-transparent pixels of the candidate display area when the advertising material is superimposed on the candidate display area; if the overlap index does not meet the preset index, it means that the overlap ratio does not exceed the preset ratio; or, the overlap index represents the ratio of the number of non-transparent pixels of the advertising material to the number of non-transparent pixels of the candidate display area; if the overlap index does not meet the preset index, it means that the ratio does not exceed the preset ratio.

[0099] The preset proportion and preset ratio may be determined based on actual needs or experience.

[0100] For example, the advertising material can be aligned with the candidate display area, and the proportion of the entire display area occupied by non-transparent pixels in all pixels of the advertising material and the candidate display area is calculated, which is the overlap ratio.

[0101] The overlap index indicates the overlap ratio between the non-transparent pixels of the creative and the non-transparent pixels of the candidate display area when the creative is superimposed on the candidate display area.

[0102] For example, the overlap metric represents the number of non-transparent pixels in the overlapped area of ​​the video frame relative to the number of non-transparent pixels in the creative.

[0103] Since non-transparent pixels are pixels with valid information, when the transparent pixels are blocked, the outside world will not be able to know the valid information contained in the non-transparent pixels. Therefore, if the non-transparent pixels in the video frame are blocked, the user will not be able to know the valid information contained in these blocked non-transparent pixels. Furthermore, if there are a large number of non-transparent pixels, such as Figure 4 As shown, the advertisement material 401 blocks the video character, which greatly affects the viewing experience of the user.

[0104] The overlapping area is the overlapping part of the video frame and the advertising material. The more non-transparent pixels there are in the advertising material, the stronger the ability of the advertising material to block the overlapping area of ​​the video frame at the current position, and the higher the possibility that the non-transparent pixels in the overlapping area of ​​the video frame are blocked. Therefore, in the embodiment of the present invention, the overlap index can also be understood as an index reflecting the occlusion of the video frame in the overlapping area. When the overlap index is not less than the preset overlap index, it is considered that the advertising material blocks the main content of the video, where the main content of the video can also be understood as the effective information in the video.

[0105] In S104, the target advertisement display position may represent a display area in the video frame where the advertisement material is displayed, and the display area may also be understood as position information corresponding to the display advertisement material. Simply put, it means locating where in the video frame the advertisement material is displayed.

[0106] After obtaining the target ad display position, the ad creative can be displayed at the target ad display position when the video frame playback time is reached.

[0107] In one achievable manner, after obtaining the target advertisement display position, the target advertisement display position may be saved first, and when the playback time of the video frame is reached, the advertisement material may be displayed at the target advertisement display position.

[0108] Specifically, the advertising material, the video frame, and the target advertising display position may be saved correspondingly.

[0109] In the embodiment of the present invention, the current advertisement display position is used as the target advertisement display position only when the overlap index does not meet the preset index, and when the overlap index meets the preset index, the current advertisement display position is adjusted until the overlap index does not meet the preset index, and then the current advertisement display position corresponding to the adjusted overlap index not meeting the preset index is used as the target advertisement display position.

[0110] The main content in the video frame can also be understood as the effective information in the video frame. Specifically, the main content can include people, objects, etc. in the video frame.

[0111] In one approach, a main content recognition model may be obtained by training a plurality of sample images in advance, and a video frame may be input into the main content recognition model, and the main content of the video frame may be determined by the main content recognition model.

[0112] Specifically, the main content recognition model can be obtained by training multiple sample images in the following ways:

[0113] Obtaining multiple sample images and marking information of the main content of each sample image, such as marking people and objects in a video frame by a rectangular frame, wherein the image can be marked by a marking method in the related art, which is not limited in the embodiment of the present invention;

[0114] A sample image is selected, and the sample image is input into the initial model. The initial model outputs the recognition information of the main content in the sample image, and the recognition information is compared with the marking information. If the difference between the recognition information and the marking information exceeds the preset difference, the parameters of the initial model are adjusted until the difference between the recognition information and the marking information is less than the preset difference. Then, the method returns to execute the selection of a sample image and subsequent steps until the number of training times reaches the preset number, and the training is terminated to obtain a trained main content recognition model, wherein the preset difference, the preset number of times, etc. are determined according to actual needs or experience.

[0115] The identification information may be a picture marked with the main content of the sample, or may be position information of the identified main content in the video frame.

[0116] The initial model can be any type of artificial intelligence model, such as a deep learning model. The specific model training process is carried out with reference to the training method of the artificial intelligence model in the relevant technology, which will not be repeated here.

[0117] In the embodiment of the present invention, the main content in the video frame is represented by non-transparent pixels in the video frame.

[0118] In an optional embodiment, as Figure 5 As shown, when the overlap index meets the preset index, S105 is executed: determining the adjusted advertisement display position;

[0119] S106: The adjusted ad display position is used as the current candidate ad display position, and the process returns to the execution of: determining the non-transparent pixels of the candidate display area and the non-transparent pixels of the ad material; calculating the overlap index of the non-transparent pixels of the candidate display area and the non-transparent pixels of the ad material, until the overlap index calculated based on a candidate ad display position does not meet the preset index, and then the candidate ad display position corresponding to the candidate ad display position whose calculated overlap index does not meet the preset index is used as the target ad display position.

[0120] It can also be understood that after S104, if the overlap index does not meet the preset index, S104 is executed; and if the overlap index meets the preset index, S105 and S106 are executed, and the process returns to execute S102 and S103, until the overlap index calculated based on a candidate advertising display position does not meet the preset index, then the candidate advertising display position corresponding to the calculated overlap index that does not meet the preset index is used as the target advertising display position.

[0121] Determining the adjusted advertisement display position includes: receiving a move operation on the advertisement material and / or a zoom operation on the advertisement material; and determining the adjusted advertisement display position based on the move operation and / or the zoom operation.

[0122] In one possible implementation, before determining the non-transparent pixels of the candidate display area and the non-transparent pixels of the advertising material, the method further includes:

[0123] Construct a virtual container; the size of the virtual container is the same as the size of the video playback window for playing the video frame; the video frame is displayed in the virtual container corresponding to the position of the video frame in the video playback window; the advertising material is displayed in the virtual container corresponding to the position of the candidate display area in the video playback window;

[0124] In this case, receiving a move operation on the advertising material and / or a scaling operation on the advertising material in the virtual container includes: receiving a move operation on the advertising material in the virtual container and / or a scaling operation on the advertising material in the virtual container.

[0125] When the overlap index meets the preset index, it means that displaying the advertisement at the current position will block the video content. In this case, the position and / or size of the advertisement creative needs to be adjusted to avoid blocking.

[0126] Among them, the overlap index represents the overlap ratio of the non-transparent pixels of the advertising material and the non-transparent pixels of the candidate display area when the advertising material is superimposed on the candidate display area; the overlap index meeting the preset index indicates that the overlap ratio exceeds the preset ratio; or, the overlap index represents the ratio of the number of non-transparent pixels of the advertising material to the number of non-transparent pixels of the candidate display area; the overlap index meeting the preset index indicates that the ratio exceeds the preset ratio.

[0127] The preset area range may be the aforementioned specific area. Adjusting the position of the advertising material based on the preset area range may include moving the advertising material within the specific area, and more specifically, translating the advertising material in a preset direction by a preset number of unit distances based on the specific area. For example, Figure 6aAs shown, in a specific area, the advertising creative is translated upward by 50 units, wherein the advertising creative 501 with a dotted line shadow represents the advertising creative before being moved, and the advertising creative 502 without a dotted line shadow represents the advertising creative after being moved.

[0128] The preset size range indicates the adjustable size range of the advertising material. The lower limit of the size can be the minimum size that allows users to clearly know the information contained in the advertising material; the upper limit of the size can be determined according to the preset area range, for example, the length does not exceed the minimum length within the preset area range, and the width does not exceed the minimum width within the preset area range. Figure 2b For example, when the preset ratio is 1 / 4, the upper limit of the preset size range is 1 / 4 of the video frame length, and the upper limit of the width is 1 / 4 of the video frame width. When adjusting the size of the creative, in order to avoid distortion of the creative, you can enlarge or reduce the creative in equal proportion. Figure 6b As shown, the solid frame 503 with dotted shadows represents the original size of the advertising material, and the reduced advertising material 504 can be Figure 6b shown.

[0129] Due to the different sizes of the ad materials provided by the advertisers, the sizes of some ad materials may be close to or even larger than the size of the video frame. Obviously, no matter how these ads are moved in the video frame, they will block the video content.

[0130] In one example, after determining that the overlap index is not less than the preset overlap index (the overlap index satisfies the preset index), the embodiment of the present invention can first construct the two-dimensional array corresponding to the video frame (video frame pixel set) and the two-dimensional array corresponding to the advertising material (advertising pixel set) according to the size relationship between the video frame size and the advertising material size. When using the two-dimensional array to store pixel information, the size relationship between the video frame pixel set and the advertising pixel set can also be determined. Then choose to adjust the advertising material position or the advertising material size, or adjust the advertising material position and size at the same time.

[0131] When the video frame pixel set is much larger than the ad pixel set, it is considered that the video frame size is much larger than the ad material size, and initially, the ad material will not cover the video screen to a large extent. At this time, the ad material position can be adjusted first.

[0132] When the video frame pixel set is close to the ad pixel set, the video frame size is considered to be close to the ad creative size. When the initial positions are different, the ad creative may or may not block the video screen. At this time, the ad creative position and ad creative size can be adjusted at the same time.

[0133] When the video frame pixel set is smaller than the ad pixel set, it is considered that the video frame size is smaller than the ad creative size. Initially, the ad creative is likely to block the video screen. At this time, you can adjust the ad creative size first.

[0134] After adjusting the position and / or size of the advertising creative, the position of the advertising creative in the video frame changes, and then for the adjusted current position, each video frame of the video clip is a video frame for which the overlapping index has not been calculated at the current position. That is, after adjusting the position and / or size of the advertising creative, it is necessary to redetermine, for each video frame of the video clip, the overlapping index between the advertising creative and each video frame at the current position.

[0135] In one example, after determining that the overlap index is not less than the preset overlap index (the overlap index satisfies the preset index), the embodiment of the present invention can first construct a two-dimensional array (video screen pixel set) corresponding to the above-mentioned specific area and a two-dimensional array (advertising pixel set) corresponding to the advertising material according to the size relationship between the size of the above-mentioned selected specific area in the video frame and the size of the advertising material, and when using the two-dimensional array to store pixel information, the size relationship between the video screen pixel set and the advertising pixel set can also be determined. Then choose to adjust the advertising material position or the advertising material size, or adjust the position and size of the advertising material at the same time.

[0136] In one achievable method, the process of adjusting the position of the advertising material and adjusting the size of the advertising material can be performed sequentially. For example, under the premise of keeping the size unchanged, all positions are first traversed within the preset area to determine whether there is an advertising display position at this size that can make the overlap index of each video frame less than the preset overlap index. If the advertising display position does not exist, the size of the advertising material is changed, and each position is traversed again to determine the overlap index. Alternatively, under the premise of keeping the position unchanged, all sizes are first traversed within the preset size range, and then the position of the advertising material is adjusted. The specific method is not limited.

[0137] In another achievable manner, the process of adjusting the position of the advertising material and adjusting the size of the advertising material can also be performed alternately. That is, when the overlap index between the advertising material and a video frame is not less than the preset overlap index, the position of the advertising material can be adjusted first. If the overlap index determined at the adjusted current position is still not less than the preset overlap index, the size of the advertising material can be adjusted again. In this way, the position of the advertising material and the size of the advertising material are adjusted alternately until the overlap index between each video frame and the advertising material is less than the preset overlap index at a certain position and a certain size. Among them, the process of adjusting the position of the advertising material and adjusting the size of the advertising material is not prioritized, and the frequency of adjusting the position of the advertising material and adjusting the size of the advertising material is not limited.

[0138] After adjusting the position and / or size of the advertising material, the determined overlap index for the adjusted current position may still not be less than the preset overlap index. At this time, the advertising material can continue to be moved in a certain direction and / or the size of the advertising material can continue to be adjusted to obtain a new current position, and continue to determine whether the overlap index is less than the preset overlap index at the new current position, until the overlap index is less than the preset overlap index at a certain current position, and select other video frames in the video clip whose overlap index has not been calculated at the current position to determine the overlap index.

[0139] In an optional embodiment, obtaining the video frame to be played in S101 includes: obtaining multiple video frames that are not played in the currently played video, wherein the candidate display area for displaying the advertising material in each video frame in the multiple video frames is the same; selecting a video frame as the video frame to be played;

[0140] In this case, after S103, it can also include: judging whether the overlapping index meets the preset index; if the overlapping index does not meet the preset index, returning to the execution: selecting a frame of video as the step of the video frame to be played, until the overlapping index calculated based on each video frame in the multiple frames of video does not meet the preset index, and taking the candidate display area corresponding to the multiple frames of video as the target advertising display position.

[0141] During video playback, the video picture is constantly changing. The video picture at the same position in different video frames may be different, that is, the pixel information may be different.

[0142] Therefore, when determining the overlap index, the embodiment of the present invention can determine the overlap index frame by frame for each frame in the currently acquired video clip. It can include: selecting any video frame in the video clip whose overlap index has not been calculated at the current position; calculating the ratio of the number of non-transparent pixels of the advertising material to the number of non-transparent pixels of the candidate display area when the advertising material is at the current position, and obtaining the corresponding overlap index of the currently selected video frame; if the overlap index between the currently selected video frame and the advertising material is less than the preset overlap index, then returning to the execution step: selecting any video frame in the video clip whose overlap index has not been calculated at the current position, until the overlap index between each video frame and the advertising material is less than the preset overlap index, and obtaining the current position as the advertising display position. Wherein, when the advertising material is displayed, the advertising material will not cover the main content of the video clip.

[0143] In one achievable manner, for a video clip corresponding to a video screen, in an embodiment of the present invention, only when the overlap index calculated for each video frame in the video clip does not meet the preset index (the ratio of the number of non-transparent pixels of the advertising material to the number of non-transparent pixels of the candidate display area does not exceed the preset ratio), the candidate display area corresponding to multiple video frames is used as the target advertising display position. For a video clip, if the overlap index corresponding to one frame meets the preset index (the ratio of the number of non-transparent pixels of the advertising material to the number of non-transparent pixels of the candidate display area exceeds the preset ratio), the overlap index corresponding to other video frames in the video clip is no longer continued; and the adjusted advertising display position is determined; the adjusted advertising display position is used as the current candidate advertising display position, and the execution is returned: determining the non-transparent pixels of the candidate display area and the non-transparent pixels of the advertising material; calculating the overlap index of the non-transparent pixels of the candidate display area and the non-transparent pixels of the advertising material, until the overlap index calculated based on a candidate advertising display position does not meet the preset index, then the candidate advertising display position corresponding to the calculated overlap index that does not meet the preset index is used as the target advertising display position.

[0144] In the embodiment of the present invention, when determining the overlap index, any video frame whose overlap index has not been calculated at the current position is selected in the video clip; when the advertising material is at the current position, the ratio of the first number of advertising materials to the second number of the currently selected video frames is calculated to obtain the overlap index of the currently selected video frame; if the overlap index between the currently selected video frame and the advertising material is less than the preset overlap index, the execution step is returned to: any video frame whose overlap index has not been calculated at the current position is selected in the video clip, until the overlap index between each video frame and the advertising material is less than the preset overlap index, and the current position is obtained as the advertising material display position. Wherein, when the advertising material is displayed, the advertising material will not cover the main content of the video clip.

[0145] For example, a first number representing the number of pixels with non-transparent transparency information in the advertising material and a second number representing the number of pixels with non-transparent transparency information in the overlapping area of ​​the currently selected video frame can be used to determine the overlap index between the advertising material and the currently selected video frame by calculating the ratio of the first number to the second number.

[0146] An embodiment of the present invention can utilize a first number representing the number of pixels with non-transparent transparency information in an advertising material, and a second number representing the number of pixels with non-transparent transparency information in an overlapping area in a currently selected video frame, and determine the overlap index between the advertising material and the currently selected video frame by calculating the ratio of the first number to the second number.

[0147] In an achievable manner, when the overlap index between the video frame and the advertising material is first determined for the video clip, the first selected video frame whose overlap index has not been calculated at the current position may be the first frame of the video clip. If the overlap index between the first frame and the advertising material is less than a preset overlap index, the second frame may be selected to determine the overlap index, and if the overlap index is less than the preset overlap index, the third frame, the fourth frame, and so on, the last frame may be selected.

[0148] In this selection scenario, if the overlap index between the last frame and the advertising creative at the current position is still smaller than the preset overlap index, it means that the current position can make the overlap index between all video frames of the video clip and the advertising creative smaller than the preset overlap index. At this time, the current position of the advertising creative in the last frame can be used as the target advertising display position.

[0149] In another achievable method, when the overlap index between the video frame and the advertising material is first determined in the video clip, the first selected video frame whose overlap index has not been calculated at the current position can also be any frame in the video clip, and when the determined overlap index is less than the preset overlap index, continue to select any other video frame in the video clip whose overlap index has not been calculated at the current position, until the overlap index between each video frame and the advertising material is less than the preset overlap index, at which time the current position of the advertising material in the last selected video frame whose overlap index has been determined can be used as the target advertising display position.

[0150] For the above two selection methods, if the overlapping indexes of the advertising material and each video frame are less than the preset overlapping indexes when the advertising material is at the initial position, then the initial position is the target advertising display position.

[0151] However, due to the continuous changes in the video screen, in the process of determining the overlap index frame by frame for the current position, it may not be that the overlap index determined for all video frames of the video clip for the current position is less than the preset overlap index. If the overlap index between the currently selected video frame and the advertising material is not less than the preset overlap index, the position of the advertising material is adjusted within the preset area range, and / or the size of the advertising material is adjusted within the preset size range, and the position of the adjusted advertising material is used as the current position of the advertising material; return to the execution step: select any video frame in the video clip whose overlap index has not been calculated at the current position.

[0152] In an embodiment of the present invention, the overlap index between the currently selected video frame and the advertising material may be not less than the preset overlap index when the video frame is selected from the video clip for the first time, or the overlap index between a certain video frame and the advertising material may be not less than the preset overlap index only after multiple selections.

[0153] In another possible implementation, for each adjusted current position, when the overlap index between the video frame and the advertising material is not less than the preset overlap index at each current position, or in other words, no matter how the advertising material position and / or the size of the advertising material is adjusted, a current position cannot be found such that the overlap index of each video frame in the video clip at the current position is less than the preset overlap index. The advertising material may not be displayed in the video clip.

[0154] Not displaying the advertising material in the video clip may include not displaying any advertising material when the video clip is played; or may include obtaining other advertising materials, determining whether there is an advertising display position in the video clip that can display other advertising materials, and if so, displaying the other advertising materials when the video clip is played. The process of determining whether there is an advertising display position in the video clip that can display other advertising materials can be described in the above embodiment, and will not be repeated here.

[0155] The embodiment of the present invention can determine the overlap index between the advertisement material and the video frame at a low cost and judge whether the video content is blocked by only using pixel information (transparency information).

[0156] In the above-mentioned method of determining the overlap index, only one current position and / or size of the advertising material and one video frame in the video clip are targeted each time, and the position of the advertising material in the video frame is adjusted in real time according to the overlap index, until at a certain current position and / or size, the overlap index determined for each video frame in the video clip is smaller than the preset overlap index, or, after traversing all positions and / or sizes, no position and / or size can make the overlap index determined for each video frame smaller than the preset overlap index at the same time.

[0157] In another achievable manner, for each video frame of the video clip, all positions and / or sizes of the advertising material in the video frame can be traversed to determine the overlap index, and all positions and / or sizes that can satisfy the overlap index being less than the preset overlap index can be selected. Then, the intersection is taken for all positions and / or sizes determined from each video frame to obtain the positions and / or sizes that can simultaneously make the overlap index of each video frame less than the preset overlap index, or when the intersection is empty, that is, any position and / or size cannot make the overlap index determined for each video frame less than the preset overlap index at the same time.

[0158] Through the above process, after the target advertisement display position corresponding to the video clip is determined, the advertisement material is displayed in the advertisement display position while the video clip is played.

[0159] The advertising material can be displayed in the advertising display position of the video playback window according to the position information of each pixel point of the advertising material. The pixel information can also include the position information of the pixel point;

[0160] After determining the target advertisement display position, the method may further include: recording the position information of each pixel of the advertisement material when the advertisement material is placed in the advertisement display position; and then, while playing the video clip, displaying the advertisement material corresponding to the position information.

[0161] The position information may include the coordinate information of the pixel point in the video frame. For example, a coordinate system is constructed with the lower left corner of the video frame as the origin, and the positive directions of the y-axis and the x-axis are upward and rightward respectively, to determine the coordinate information of the pixel point. Since the video frame is the same size as the video playback window, when the determined advertising display position displays the advertising material at the same position in the video playback window, the advertising material will not shift and thus block the video content, thereby ensuring that the main content of the video clip is clearly visible.

[0162] In another achievable manner, after determining the target ad display position, the position information of the center position of the ad material when the ad material is placed in the target ad display position can also be recorded. When playing the video clip, the position information of the center position is used to locate the display position of the ad material in the video clip, and the ad material is displayed at the display position.

[0163] Using the image processing method provided by the embodiment of the present invention to display advertisements during video playback can be simply summarized as obtaining video frames, extracting pixel information of specific areas, calculating the overlap ratio, and adjusting the position of stickers in real time. By seamlessly integrating real-time video frame capture and pixel information extraction during video playback, rapid response and analysis of the advertising material delivery process is guaranteed, and real-time detection of whether the advertising material covers the main content of the video screen is achieved, the position and size of the advertising material are dynamically adjusted, and the display position of the advertising material in the main body of the video is reasonably allocated to ensure that the advertising material can avoid the main content of the video when displayed, thereby improving the user viewing experience.

[0164] Furthermore, under the premise that the advertising material does not block the main content of the video, the probability of users closing the advertisement can be reduced, and the effectiveness of advertising, advertising conversion rate and brand image can be improved. At the same time, it can also improve advertisers' satisfaction with the advertising effect and enhance the competitiveness of companies that provide video advertising technology, such as advertising platforms and marketing companies that provide video player SDK (software development kit) and advertising sticker services in the advertising field.

[0165] In a feasible way, the video frame analysis algorithm can be further optimized to improve the detection accuracy and speed, so that the image processing method can adapt to more complex video content scenarios. Machine learning technology can be introduced to train the model to automatically identify the main content in the video and improve the intelligence of the adjustment of the advertising sticker position.

[0166] The image processing method provided by the embodiment of the present invention can be applied in the scenario of video playback. During the video playback process, it can determine the advertising display position that does not block the video content for different advertising materials and different video clips. Therefore, during the video playback process, the advertising display position can be flexibly adjusted for different video clips to ensure that the displayed advertisements will never block the video content while the user is watching the video, thereby improving the user's video viewing experience.

[0167] A specific embodiment, such as Figure 6c As shown, it may include:

[0168] S601, creating a virtual player container.

[0169] The virtual container can be initialized through the browser API using the preset function of the browser. The size of the virtual container is the same as the size of the video playback window.

[0170] S602: Capture video frames and write them into a virtual container.

[0171] The video segments of the advertising creative to be delivered are obtained, and each video frame in each video segment is written into a virtual container.

[0172] S601-S602 can be understood as the steps of obtaining the player video frame.

[0173] S603, loading the video frame into the memory and determining the specific area.

[0174] The embodiment of the present invention utilizes Canvas technology, and the process of loading the video frame in the virtual container and the subsequent parsing of the video frame is performed in the memory and will not be displayed to the user. As mentioned above, the specific area is the area in the video frame that can be used for displaying advertising materials.

[0175] S604: Parse the video frame in the virtual container.

[0176] Parsing the video frame includes detecting pixel information of each pixel point of the video frame, wherein the pixel information includes transparency information, and the parsed pixel information can be stored in a global pixel two-dimensional array.

[0177] In a feasible manner, the pixel information of each pixel point in a specific area may also be stored only in a two-dimensional array.

[0178] S603-S604 can be understood as the steps of extracting pixels in a specific area of ​​the video frame.

[0179] The embodiment of the present invention can place the advertisement material in a virtual container and use the virtual container to parse the pixel information of the advertisement material.

[0180] S605 , determining whether the coverage ratio of the video frame at the default position in the global two-dimensional array exceeds a preset threshold.

[0181] The default position can include the overlapping area of ​​the ad material and the video frame. The transparency information of each pixel is stored in the two-dimensional array. The coverage ratio of the default position corresponds to Figure 1 The illustrated embodiment is an overlap indicator of a video frame and an advertisement material in an overlap area.

[0182] When the coverage ratio does not exceed the preset threshold, that is, the overlap index is not greater than the preset overlap index, other video frames in the video clip that have not been judged for coverage ratio at the default position can be selected to continue to execute S605. If the coverage ratios of all video frames judged at the default position do not exceed the preset threshold, S606 can be executed. At this time, the default position is the advertisement display position.

[0183] When the coverage ratio exceeds the preset threshold, that is, when the overlap index is greater than the preset overlap index, it is considered that displaying the advertisement material at the default position will block the video content, and S607 can be executed at this time.

[0184] S606, end the judgment.

[0185] S607: Move the advertisement material and / or change the size of the advertisement material.

[0186] Move the position of the advertising material within a preset area; and / or change the size of the advertising material within a preset size range. For example, move from position 1 to position 2, position 3, etc. from left to right horizontally.

[0187] S608: Determine whether there is a non-overlapping area during the process of moving the advertisement material and / or changing the size of the advertisement material.

[0188] The non-overlapping area corresponds to the aforementioned advertising display position. Each time the advertising material is moved and / or changed, the adjusted position of the advertising material is used as the current position, and the same method as S605 is used to determine whether there is an advertising display position that can make the overlapping index of all video frames less than the preset overlapping index based on the coverage ratio, that is, the overlapping index. If so, execute S610; if not, execute S609.

[0189] S609, do not display the advertising material. That is, do not display the advertising material when playing to the video segment.

[0190] S610: Obtain the coordinates of the advertisement material in the virtual container.

[0191] Gets the specific coordinates of the creative in the coordinate system established by the virtual container.

[0192] S611, display the advertising material at the corresponding coordinates of the player.

[0193] When the video clip is played, the ad creative is displayed at the corresponding coordinates of the player.

[0194] Since the virtual container is the same size as the video playback window, the creative can be displayed at the same position in the video playback window using the specific coordinates of the creative in the coordinate system established by the virtual container, so that the creative will not block the video content.

[0195] The embodiment of the present invention can automatically and efficiently adjust the advertisement display position, ensure the reasonable allocation of advertisements and the main content of the video, and ensure that the main content of the video will not be disturbed during the advertisement display process, thereby improving the user's viewing experience and the effectiveness of the advertisement.

[0196] like Figure 7 As shown, an embodiment of the present invention provides an image processing device, which is applied to a client, including:

[0197] The acquisition module 701 is used to acquire a video frame to be played, an advertisement material and a candidate advertisement display position, wherein the candidate advertisement display position represents a candidate display area for displaying the advertisement material in the video frame;

[0198] A first determination module 702, configured to determine the non-transparent pixels of the candidate display area and the non-transparent pixels of the advertising material;

[0199] A calculation module 703, used to calculate the overlap index of the non-transparent pixels of the candidate display area and the non-transparent pixels of the advertising material;

[0200] The second determining module 704 is configured to use the candidate advertisement display position as a target advertisement display position when the overlap index does not meet a preset index.

[0201] Optionally, the device further comprises:

[0202] A third determination module, configured to determine an adjusted advertisement display position when the overlap index satisfies a preset index;

[0203] The adjustment module is used to use the adjusted advertising display position as the current candidate advertising display position and return to execute: determining the non-transparent pixels of the candidate display area and the non-transparent pixels of the advertising material; calculating the overlapping index of the non-transparent pixels of the candidate display area and the non-transparent pixels of the advertising material, until the overlapping index calculated based on a candidate advertising display position does not meet the preset index, then the candidate advertising display position corresponding to the calculated overlapping index that does not meet the preset index is used as the target advertising display position.

[0204] Optionally, the acquisition module 701 is specifically used to: acquire a plurality of video frames that are not played in the currently played video, wherein the candidate display area for displaying the advertising material in each video frame in the plurality of video frames is the same;

[0205] A selection module is used to select a video frame as a video frame to be played;

[0206] A judgment module is used to judge whether the overlapping index of the non-transparent pixels of the candidate display area and the non-transparent pixels of the advertising material meets the preset index after calculating the overlapping index; if the overlapping index does not meet the preset index, return to the execution step of selecting a frame of video as the video frame to be played, until the overlapping index calculated based on each video frame in the multiple video frames does not meet the preset index, and the candidate display area corresponding to the multiple video frames is used as the target advertising display position.

[0207] Optionally, the third determination module is specifically configured to: receive a move operation on the advertising material, and / or a zoom operation on the advertising material in the virtual container; and determine an adjusted advertisement display position based on the move operation and / or the zoom operation.

[0208] Optionally, the device further comprises:

[0209] A construction module, used for constructing a virtual container before determining the non-transparent pixels of the candidate display area and the non-transparent pixels of the advertising material; the size of the virtual container is the same as the size of the video playback window playing the video frame;

[0210] A display module, configured to display the video frame in the virtual container corresponding to the position of the video frame in the video playback window; and display the advertising material in the virtual container corresponding to the position of the candidate display area in the video playback window;

[0211] The third determining module is specifically configured to: receive a move operation on the advertising material in the virtual container, and / or a zoom operation on the advertising material in the virtual container.

[0212] Optionally, the display module is used to display the advertising material at the target advertising display position after determining the target advertising display position and when the playback time of the video frame is reached.

[0213] Optionally, the first determination module 702 is specifically used to: determine the color value of each pixel in the candidate display area and the color value of each pixel in the advertising material; and regard the pixel whose color value is not less than a preset transparency threshold as a non-transparent pixel.

[0214] Optionally, the overlap index represents the overlap ratio of the non-transparent pixels of the advertising material and the non-transparent pixels of the candidate display area when the advertising material is superimposed on the candidate display area; if the overlap index does not meet the preset index, it means that the overlap ratio does not exceed the preset ratio; or, the overlap index represents the ratio of the number of non-transparent pixels of the advertising material to the number of non-transparent pixels of the candidate display area; if the overlap index does not meet the preset index, it means that the ratio does not exceed the preset ratio.

[0215] The embodiment of the present invention further provides an electronic device, such as Figure 8 As shown, it includes a processor 801, a communication interface 802, a memory 803 and a communication bus 804, wherein the processor 801, the communication interface 802, and the memory 803 communicate with each other through the communication bus 804.

[0216] Memory 803, used for storing computer programs;

[0217] The processor 801 is used to implement any image processing method described in the above embodiments when executing the program stored in the memory 803.

[0218] The communication bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0219] The communication interface is used for communication between the above terminal and other devices.

[0220] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0221] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0222] In another embodiment of the present invention, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the image processing method described in any one of the above embodiments is implemented.

[0223] In another embodiment of the present invention, a computer program product including instructions is provided. When the computer program product is run on a computer, the computer executes the image processing method described in any one of the above embodiments.

[0224] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0225] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0226] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, electronic device, computer-readable storage medium, and computer program product embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0227] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. An image processing method, characterized in that: Applied to a client, the method comprises: Acquire a video frame to be played, an advertisement material, and a candidate advertisement display position, wherein the candidate advertisement display position represents a candidate display area for displaying the advertisement material in the video frame; Determine the non-transparent pixels of the candidate display area and the non-transparent pixels of the advertising material; Calculate the overlap index of the non-transparent pixels of the candidate display area and the non-transparent pixels of the advertising material; When the overlap index does not meet a preset index, the candidate advertisement display position is used as a target advertisement display position.

2. The method according to claim 1, characterized in that The method further comprises: When the overlap index meets the preset index, determining the adjusted advertisement display position; The adjusted ad display position is used as the current candidate ad display position, and the execution is returned: determining the non-transparent pixels of the candidate display area and the non-transparent pixels of the advertising material; calculating the overlapping index of the non-transparent pixels of the candidate display area and the non-transparent pixels of the advertising material, until the overlapping index calculated based on a candidate ad display position does not meet the preset index, then the candidate ad display position corresponding to the candidate ad display position whose calculated overlapping index does not meet the preset index is used as the target ad display position.

3. The method according to claim 1, characterized in that The step of obtaining the video frame to be played includes: Acquire a plurality of video frames that are not played in the currently played video, wherein the candidate display area for displaying the advertisement material in each video frame in the plurality of video frames is the same; Select a video frame as the video frame to be played; After calculating the overlap index of the non-transparent pixels of the candidate display area and the non-transparent pixels of the advertising material, the method further includes: Determining whether the overlap index meets a preset index; If the overlapping index does not meet the preset index, return to the execution: select a video frame as the step of the video frame to be played, until the overlapping index calculated based on each video frame in the multiple video frames does not meet the preset index, and the candidate display area corresponding to the multiple video frames is used as the target advertising display position.

4. The method according to claim 2, characterized in that: The step of determining the adjusted advertisement display position includes: receiving a move operation on the advertising material, and / or a scaling operation on the advertising material in a virtual container; Based on the moving operation and / or the zooming operation, an adjusted advertisement display position is determined.

5. The method according to claim 4, characterized in that Before determining the non-transparent pixels of the candidate display area and the non-transparent pixels of the advertising material, the method further includes: Constructing a virtual container; the size of the virtual container is the same as the size of the video playback window playing the video frame; Corresponding to the position of the video frame in the video playback window, displaying the video frame in the virtual container; Corresponding to the position of the candidate display area in the video playback window, displaying the advertising material in the virtual container; The receiving a moving operation on the advertising material and / or a scaling operation on the advertising material in the virtual container includes: A move operation on the advertising material in the virtual container and / or a zoom operation on the advertising material in the virtual container are received.

6. The method according to any one of claims 1 to 5, characterized in that: After determining the target advertisement display position, the method further includes: When the playing time of the video frame is reached, the advertising material is displayed at the target advertising display position.

7. The method according to any one of claims 1 to 5, characterized in that: The determining of the non-transparent pixels of the candidate display area and the non-transparent pixels of the advertising material includes: Determine the color value of each pixel in the candidate display area and the color value of each pixel in the advertising material; Pixels whose color values ​​are not less than the preset transparency threshold are regarded as non-transparent pixels.

8. The method according to any one of claims 1 to 5, characterized in that: The overlap index represents the overlap ratio between the non-transparent pixels of the advertising material and the non-transparent pixels of the candidate display area when the advertising material is superimposed on the candidate display area; if the overlap index does not meet the preset index, it means that the overlap ratio does not exceed the preset ratio; or The overlap index represents the ratio of the number of non-transparent pixels of the advertising material to the number of non-transparent pixels of the candidate display area; if the overlap index does not meet the preset index, it means that the ratio does not exceed the preset ratio.

9. An image processing device, characterized in that: Applied to a client, the method comprises: An acquisition module, used to acquire a video frame to be played, an advertisement material and a candidate advertisement display position, wherein the candidate advertisement display position represents a candidate display area for displaying the advertisement material in the video frame; A first determination module, used to determine the non-transparent pixels of the candidate display area and the non-transparent pixels of the advertising material; A calculation module, used for calculating the overlap index of the non-transparent pixels of the candidate display area and the non-transparent pixels of the advertising material; The second determining module is configured to use the candidate advertisement display position as a target advertisement display position when the overlap index does not meet a preset index.

10. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing the steps of any method described in claims 1-8 when executing a program stored in a memory.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any one of the methods of claims 1-8 are implemented.

Images