Advertisement quality detection method and device, and computer device

Abstract

The specification discloses an advertisement quality detection method and device and a computer device, which can be applied to the technical field of computer vision. The method comprises the following steps: extracting a plurality of video frames in an advertisement video; detecting the interface color of the advertisement according to the plurality of video frames to obtain color index data; detecting the interface element of the advertisement according to the plurality of video frames to obtain element index data; detecting the timing information of the advertisement according to the plurality of video frames to obtain timing index data; and determining the quality detection result of the advertisement video according to the color index data, the element index data and the timing index data, wherein the quality detection result is used to represent the user experience of the advertisement. The embodiment of the specification can accurately and efficiently evaluate the user experience of the advertisement.

Description

Technical Field

[0001] This specification relates to the field of computer vision technology, and in particular to an advertising quality detection method, apparatus, and computer equipment. Background Technology

[0002] With the rapid development of the mobile internet, mobile advertising has become an important part of digital marketing. To ensure user experience and advertising effectiveness, effective monitoring of mobile ad quality is becoming increasingly crucial.

[0003] In existing technologies, advertising platforms can use SDKs (Software Development Kits) to track and log data to collect behavioral metrics such as ad impressions, clicks, and conversions. However, these metrics primarily characterize ad performance and conversion rates, and are difficult to directly reflect the user's actual experience at the screen level. Alternatively, user experience can be evaluated through manual sampling. However, manual sampling lacks objectivity and is inefficient. Summary of the Invention

[0004] This specification provides an advertising quality detection method, apparatus, and computer equipment to accurately and efficiently detect and evaluate the user experience of advertisements.

[0005] This specification provides an embodiment of an advertising quality detection method, including: Extract multiple video frames from the advertisement video; Color index data is obtained by detecting the interface color of the advertisement based on the multiple video frames; Based on the multiple video frames, the interface elements of the advertisement are detected to obtain element index data; Based on the temporal information of the advertisements detected from the multiple video frames, temporal indicator data is obtained; Based on the color index data, the element index data, and the time sequence index data, the quality inspection result of the advertisement video is determined, and the quality inspection result is used to represent the user experience of the advertisement.

[0006] In some embodiments, color-abnormal video frames can be identified among the plurality of video frames. Each color-abnormal video frame includes a color-abnormal region, which is a connected region, and its color features satisfy a first condition, while its area satisfies a second condition. Color index data can be statistically analyzed based on the color-abnormal video frames. The color index data includes at least one of the following: a first duration and a second duration; the first duration represents the cumulative duration of color abnormalities in the advertising video, and the second duration represents the duration of color abnormalities in the advertising video.

[0007] In some embodiments, the color feature includes brightness, and the first condition includes: the mean brightness is greater than or equal to a first brightness threshold and the brightness variance is less than or equal to a first variance threshold. The color-abnormal area is a white area, and the color-abnormal video frame is a white video frame.

[0008] In some embodiments, the color feature includes brightness, and the second condition includes: the mean brightness is less than or equal to a second brightness threshold, and the brightness variance is less than or equal to a second variance threshold; The color-abnormal area is a black area, and the color-abnormal video frame is a black video frame.

[0009] In some embodiments, the video frames can be input into a computer vision model to detect specified advertising interface elements in the video frames; the element index data can be determined based on the detection results of the multiple video frames.

[0010] The element index data includes at least one of the following: the probability that the advertisement video contains a specified advertisement interface element, the location information of the specified advertisement interface element, the screen ratio of the specified advertisement interface element, and the margin reachability of the specified advertisement interface element.

[0011] In some embodiments, the number of state cycles experienced by the close control across the plurality of video frames can be counted; wherein, a state cycle includes the duration from the appearance to the disappearance of the close control; the number of ad turns in the advertisement video can be determined based on the number of state cycles. The time-series metric data includes the number of ad turns.

[0012] In some embodiments, the number of state cycles is multiple; in the multiple state cycles, the time interval between two adjacent state cycles is greater than or equal to a set duration threshold, and the time interval includes the interval between the time when the close control disappears in the previous state cycle and the time when the close control appears in the next state cycle.

[0013] In some embodiments, time information of a specified advertising interface element can be detected based on the plurality of video frames. The time information includes at least one of the following: first appearance time and duration. The time-series metric data includes the time information.

[0014] In some embodiments, the plurality of video frames are sequential in time. Multiple target video frames that are consecutive and contain the specified advertising interface element can be selected from the plurality of video frames; a first video frame and a second video frame can be selected from the plurality of target video frames, wherein the first video frame is the target video frame in which the specified advertising interface element first appears, and the second video frame is the target video frame in which the specified advertising interface element last appears; the first appearance time can be determined based on the position of the first video frame in the advertising video; the duration can be determined based on the positions of the first video frame and the second video frame in the advertising video.

[0015] In some embodiments, a first sub-detection result representing color quality can be determined based on the color index data and preset color quality rules; a second sub-detection result representing element quality can be determined based on the element index data and preset element quality rules; a third sub-detection result representing time-series quality can be determined based on the time-series index data and preset time-series quality rules; and the quality detection result can be determined based on the first sub-detection result, the second sub-detection result, and the third sub-detection result.

[0016] This specification also provides an advertising quality detection device, comprising: Extraction unit, used to extract multiple video frames from an advertisement video; The first detection unit is used to detect the interface color of the advertisement based on the multiple video frames and obtain color index data; The second detection unit is used to detect the interface elements of the advertisement based on the multiple video frames and obtain element index data. The third detection unit is used to detect the timing information of the advertisement based on the multiple video frames and obtain timing index data; The determining unit is configured to determine the quality inspection result of the advertisement based on the color index data, the element index data, and the time sequence index data, wherein the quality inspection result is used to represent the user experience of the advertisement.

[0017] This specification also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described advertising quality detection method.

[0018] This specification also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described advertising quality detection method.

[0019] This specification also provides a computer program product, which includes a computer program that, when executed by a processor, implements the above-described advertising quality detection method.

[0020] The technical solution of this specification embodiment can extract multiple video frames from an advertisement video; detect the interface color of the advertisement based on the multiple video frames to obtain color index data; detect the interface elements of the advertisement based on the multiple video frames to obtain element index data; detect the timing information of the advertisement based on the multiple video frames to obtain timing index data; and determine the quality detection result of the advertisement video based on the color index data, the element index data, and the timing index data, wherein the quality detection result is used to represent the user experience of the advertisement. Therefore, advertisements can be detected based on video data that can represent the advertisement display process. By performing detection at the pixel display level, the detection result can directly reflect the user's actual screen experience. Furthermore, this specification embodiment can detect the user experience of the advertisement from multiple dimensions such as interface color, interface elements, and timing information, thereby improving the accuracy of advertisement user experience detection. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. The drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart illustrating the advertising quality detection method in the embodiments of this specification; Figure 2 This is a schematic diagram of the advertising quality detection device in the embodiments of this specification. Detailed Implementation

[0023] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. The specific embodiments described herein are only used to explain this disclosure, and not to limit this disclosure. All other embodiments obtained by those skilled in the art based on the described embodiments of this disclosure are within the scope of protection of this disclosure. In addition, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. In addition, the acquisition, transmission, storage, use, and processing of data in the technical solutions of this application comply with the relevant provisions of national laws and regulations. In addition, in the embodiments of this application, certain software, components, models, and other existing solutions in the industry may be mentioned. These should be considered as exemplary, and their purpose is only to illustrate the feasibility of implementing the technical solutions of this application, but it does not mean that the applicant has used or necessarily used such solutions.

[0024] This specification provides a detection system. The detection system may include terminal devices and servers. The terminal devices include, but are not limited to, smartphones, tablets, portable computers, desktop computers, and wearable devices. The server may be a single server or a server cluster comprising multiple servers. The terminal devices can collect advertising videos and send them to the server. The server can receive the advertising videos and determine the quality detection result of the advertisement based on the videos. The quality detection result is used to represent the user experience of the advertisement. For example, the quality detection result may include a detection report. Thus, advertisements can be detected based on video data from the advertisement display process. By performing detection at the pixel display level, the detection results can reflect the user's actual screen experience.

[0025] In some embodiments, the terminal device can also obtain metadata of the advertising video and send the metadata of the advertising video to the server. The server can receive the metadata of the advertising video. The metadata is used to represent the attribute information corresponding to the advertising video. The metadata includes at least one of the following: advertising source, advertising position, advertising creative identifier, application version, regional information, etc. The advertising source can be an advertiser and / or an advertising platform, etc. The advertising position can be the display location of the advertisement (e.g., the display location in an application). The advertising creative identifier is used to identify the advertising creative. The application version can be the application version used to carry the advertising video. The regional information is used to represent the geographical region of the terminal device, and can also represent the geographical region where the advertisement is disseminated.

[0026] In some embodiments, the advertising video may include an advertising video in a test environment, the terminal device may include testing equipment for testers, and the server may include an advertising monitoring platform. The server may send an advertising detection request to the terminal device. The terminal device may receive the advertising detection request, invoke an application to play the advertisement, and capture the video of the advertisement playing; it may also send the advertising video to the server. The server may receive the advertising video. For example, the terminal device may record the screen until the advertisement finishes playing to obtain the advertising video. Optionally, the advertising detection request may include an advertising source. The terminal device may play the advertisement from the advertising source and capture the corresponding advertising video. Optionally, the terminal device may also obtain metadata of the advertising video and may send the advertising video metadata to the server. The server may receive the advertising video metadata.

[0027] This allows for monitoring the user experience of ads in a test environment.

[0028] In some embodiments, the advertising video may include an online advertising video, the terminal device may include a user-facing device, and the server may include an advertising monitoring platform. The terminal device can trigger a scheduled task to invoke an application to play an advertisement, and can collect the advertising video; it can also send the advertising video to the server. The server can receive the advertising video. For example, the terminal device can record the screen until the advertisement finishes playing to obtain the advertising video. Optionally, the terminal device can obtain metadata of the advertising video and can send the metadata to the server. The server can receive the metadata.

[0029] This allows for monitoring the user experience of ads in an online environment.

[0030] In some embodiments, the detection system may include one or more terminal devices. These terminal devices can collect one or more advertising videos and send them to a server. A single terminal device can collect one or more advertising videos and send them to a server. The server can receive the one or more advertising videos. The metadata of advertising videos collected by different terminal devices may be the same or different. The metadata of different advertising videos collected by a single terminal device may be the same or different. The server can perform quality detection on each advertising video separately to obtain the quality detection results. The server can perform statistical analysis on the quality detection results of one or more advertising videos based on the metadata. For example, the server can statistically analyze the quality detection results of advertising videos according to dimensions such as advertising source, advertising position, advertising creative identifier, application version, and regional information. This allows for obtaining statistical results for a specified advertising source, advertising position, advertising creative identifier, application version, or regional information.

[0031] This specification provides an embodiment of an advertising quality detection method. The method can be applied to a server.

[0032] Please see Figure 1 The method may include the following steps.

[0033] Step 11: Extract multiple video frames from the advertisement video; Step 12: Detect the interface color of the advertisement based on the multiple video frames to obtain color index data; Step 13: Detect the interface elements of the advertisement based on the multiple video frames to obtain element index data; Step 14: Detect the timing information of the advertisements based on the multiple video frames to obtain timing indicator data; Step 15: Determine the quality inspection result of the advertisement video based on the color index data, the element index data, and the time sequence index data. The quality inspection result is used to represent the user experience of the advertisement.

[0034] The technical solution of this specification embodiment can extract multiple video frames from an advertisement video; detect the interface color of the advertisement based on the multiple video frames to obtain color index data; detect the interface elements of the advertisement based on the multiple video frames to obtain element index data; detect the timing information of the advertisement based on the multiple video frames to obtain timing index data; and determine the quality detection result of the advertisement video based on the color index data, the element index data, and the timing index data, wherein the quality detection result is used to represent the user experience of the advertisement. Therefore, advertisements can be detected based on video data that can represent the advertisement display process. By performing detection at the pixel display level, the detection result can reflect the user's actual screen experience. Furthermore, this specification embodiment can detect the user experience of the advertisement from multiple dimensions such as interface color, interface elements, and timing information, improving the accuracy of advertisement user experience detection.

[0035] In some embodiments, the advertising video can be video data of an advertisement playback. The advertisement may include mobile advertisements. Advertisements may include, for example, product advertisements, service advertisements, brand advertisements, etc. Advertisements may also include, for example, promotional advertisements, splash screen advertisements, interstitial advertisements, rewarded advertisements, etc. The advertising video can be captured by methods such as screen recording. The video format of the advertising video includes, but is not limited to, MPEG format, AVI format, MOV format, 3GP format, H.264 format, etc. The advertising video may include multiple frames of image data. Each frame of image data can be called a video frame. The frame rate of the advertising video can be 24fps, 30fps, or 60fps, etc. The color space of the video frame includes, but is not limited to, RGB, YUV, etc. The terminal device can send the advertising video to the server. The server can receive the advertising video.

[0036] In some embodiments, the server may extract multiple video frames from an advertising video. These multiple video frames are sequential in time and can form a video frame sequence. The video frame sequence may include multiple consecutive video frames.

[0037] The server can extract all video frames from the ad video. Alternatively, the server can randomly extract multiple video frames from the ad video. Or, considering that adjacent video frames in an ad video are often similar, the server can evenly extract multiple video frames from the ad video. For example, the server can also extract video frames from the ad video at preset intervals of a certain number of frames. This results in a greater difference between the extracted video frames, thus saving computational load and time in ad quality detection without affecting its accuracy.

[0038] The server can use tools such as FFmpeg to extract video frames from the advertisement video. Optionally, the server can also obtain the time information (e.g., a timestamp) for each video frame. The time information is used to indicate the point in time within the advertisement video. For example, the time information for a video frame includes the duration from the start of the video to that video frame.

[0039] In some embodiments, the server can identify one or more video frames with abnormal color among the plurality of video frames; and can statistically analyze color index data of the advertising video based on the one or more video frames with abnormal color. The color index data may include at least one of the following: a first duration and a second duration. The first duration is used to represent the cumulative duration of the color abnormality in the advertising video, and the second duration is used to represent the duration of the color abnormality in the advertising video.

[0040] A video frame with color anomalies may contain one or more color anomaly regions. Color anomaly regions are connected regions.

[0041] The color characteristics of the color anomaly region satisfy a first condition. The first condition is used to determine whether the color is abnormal, such as determining whether the color is white and / or black. Color characteristics may include, for example, brightness. The color anomaly region may include, for example, a white region and / or a black region. Optionally, the first condition is used to determine whether the color is white. If the brightness of the color anomaly region satisfies the first condition, it indicates that the color anomaly region is a white region. The first condition may include, for example, a brightness mean greater than or equal to a first brightness threshold and a brightness variance less than or equal to a first variance threshold. The first brightness threshold may be, for example, 245, and the first variance threshold may be, for example, 15. Optionally, the first condition is used to determine whether the color is black. If the brightness of the color anomaly region satisfies the first condition, it indicates that the color anomaly region is a black region. The first condition may include, for example, a brightness mean less than or equal to a second brightness threshold and a brightness variance less than or equal to a second variance threshold. The second brightness threshold may be, for example, 10, and the second variance threshold may be, for example, 10. Optionally, the first condition is used to determine whether the color is white or black. The first condition may include a first sub-condition and a second sub-condition. The first sub-condition is used to determine whether the color is white. The first sub-condition may include, for example, a mean brightness value greater than or equal to a first brightness threshold, and a brightness variance less than or equal to a first variance threshold. If the brightness of a color anomaly region meets the first sub-condition, then the color anomaly region is a white region. The second sub-condition is used to determine whether the color is black. The second sub-condition may include, for example, a mean brightness value less than or equal to a second brightness threshold, and a brightness variance less than or equal to a second variance threshold. If the brightness of a color anomaly region meets the second sub-condition, then the color anomaly region is a black region.

[0042] The area of ​​the color anomaly region satisfies the second condition. This second condition determines the size of the area. If the area of ​​the color anomaly region satisfies the second condition, it means that the area of ​​the color anomaly region is large enough to affect the user's viewing experience of the video frame. For example, the second condition could include the area of ​​the color anomaly region being greater than or equal to an area threshold. Another example is that the second condition could include the ratio of the area of ​​the color anomaly region to the area of ​​the video frame being greater than a certain threshold. This threshold could be, for example, 80%.

[0043] The server can identify one or more color-abnormal video frames among the multiple video frames using OpenCV (Open Source Computer Vision Library) technology. Alternatively, for each of the multiple video frames, the server can detect whether it contains a color-abnormal region; if a color-abnormal region is found, the video frame can be determined to be a color-abnormal video frame. For example, the server can select several color-abnormal pixels in the video frame that satisfy a first condition; it can connect spatially adjacent color-abnormal pixels to obtain several independent connected components. The server can select the connected component with the largest area from the several independent connected components; it can determine whether the connected component with the largest area satisfies a second condition; if it satisfies the second condition, the largest connected component can be identified as a color-abnormal region, thus identifying the video frame as a color-abnormal video frame; if it does not satisfy the second condition, it can be determined that the video frame does not contain a color-abnormal region, thus identifying the video frame as not being a color-abnormal video frame. Alternatively, the server can also determine whether the area of ​​each independent connected component satisfies the second condition; if the area of ​​one or more independent connected components satisfies the second condition, the video frame can be identified as a color-abnormal video frame. If none of the connected components meet the second condition, the server can determine that the video frame does not contain a color abnormality region, and thus can determine that the video frame does not belong to a color abnormality video frame.

[0044] During the playback of the advertisement video, each video frame is displayed for a certain duration. The display duration of a video frame is related to the frame rate of the advertisement video. Taking an advertisement video with a frame rate of 30fps as an example, the display duration of a video frame is 33.3 milliseconds. Therefore, the server can calculate the sum of the display durations of all color-abnormal video frames as the first duration. The first duration can be understood as the overall duration of color abnormalities in the advertisement video. Additionally, when there is only one color-abnormal video frame, the server can obtain the display duration of that color-abnormal video frame as the second duration. Alternatively, when there are multiple color-abnormal video frames, the server can further determine whether the multiple color-abnormal video frames contain multiple target color-abnormal video frames that are temporally consecutive. If the multiple color-abnormal video frames do not contain the multiple target color-abnormal video frames, the server can obtain the display duration of one color-abnormal video frame as the second duration. If the multiple color-abnormal video frames contain the multiple target color-abnormal video frames, the server can calculate the sum of the display durations of the multiple target color-abnormal video frames as the second duration. The second duration can be understood as the duration of a single color anomaly in the video data, specifically the duration of a group of time-continuous color anomaly video frames.

[0045] Of course, in addition to the first and second durations, color indicator data can also include other indicator data.

[0046] For example, the server can calculate a first ratio between a first duration and the ad video playback duration. Color metric data may also include this first ratio. As another example, the server can also calculate a second ratio between a second duration and the ad video playback duration. Color metric data may also include this second ratio. Furthermore, the server can also calculate a third ratio between the first and second durations. Color metric data may also include this third ratio.

[0047] In some embodiments, the server can input each video frame into a computer vision model to detect one or more specified advertising interface elements in the video frame, and obtain the detection results output by the computer vision model. The server can determine the element index data of the advertising video based on the detection results of multiple video frames. The element index data may include at least one of the following: the probability that the advertising video contains a specified advertising interface element, the location information of the specified advertising interface element, the screen ratio of the specified advertising interface element, and the margin reachability of the specified advertising interface element.

[0048] Computer vision models can include models such as YOLO and Faster R-CNN that can be used for object detection. Specified ad interface elements include key interface elements that help improve the user experience of the ad. Specified ad interface elements include, but are not limited to, countdown controls, close controls, CTA (Call-To-Action) controls, volume controls, ad logo controls, progress bar controls, mute controls, unmute controls, watermark controls, loading indicators, etc. CTA controls are used to guide user interaction, such as controls for "Download Now," "Learn More," "Claim Now," "Register for a Trial," and "Watch Video." The forms of specified ad interface elements include, but are not limited to, buttons, arrows, text, and graphics.

[0049] The server can directly input video frames into the computer vision model. Alternatively, the server can perform data augmentation on the video frames and then input the augmented video frames into the computer vision model. Data augmentation can improve the accuracy of advertising interface element detection. Data augmentation includes, but is not limited to, scaling, rotation, blurring, color cast, contrast adjustment, noise reduction, affine transformation, random occlusion, brightness jitter, and compression artifact enhancement. The output of the computer vision model can include the detection results of the video frames. The detection results of the video frames can include at least one of the following: the probability that the video frame contains a specified advertising interface element, the position information of the specified advertising interface element in the video frame, the screen ratio of the specified advertising interface element in the video frame, and the margin reachability of the specified advertising interface element in the video frame. The probability of the advertising interface element can be a confidence score. The confidence score can be a value between 0 and 1. The higher the confidence score, the higher the probability that the video frame contains the advertising interface element. The position information of the advertising interface element can include the coordinates of the advertising interface element in the video frame, such as the coordinates of the bounding box of the advertising interface element. For example, the bounding box can be a rectangular bounding box. Then the coordinates can include the coordinates of the top-left vertex of the rectangular bounding box. The positional information of ad elements can be used to characterize the layout rationality of ad elements. A more rational layout of ad elements results in a better user experience. The screen-to-body ratio of ad elements can include the area ratio between the ad element and the video frame. The screen-to-body ratio of ad elements characterizes their visual prominence and operability. The margin accessibility of ad elements can include the shortest distance from the edge of the ad element to the corresponding edge of the video frame. The margin accessibility of ad elements characterizes the interactivity of the ad elements.

[0050] The detection results for video frames can include the detection results of one or more specified interface elements. The detection results for different specified interface elements can be the same or different. For example, the detection results for a CTA control can include: the probability that the video frame contains the CTA control, the CTA control's position information, the CTA control's screen ratio, and the CTA control's margin reachability. The detection results for a progress bar control can include: the probability that the video frame contains the progress bar control and the progress bar control's position information. The detection results for an ad label control can include: the probability that the video frame contains the ad label control and the ad label control's position information.

[0051] The server can obtain the detection results of video frames as element indicator data for the advertising video. Alternatively, to avoid the instability of single-frame detection results due to video encoding, image noise, instantaneous model misjudgment, etc., the server can also statistically analyze the detection results of multiple video frames to obtain element indicator data for the advertising video.

[0052] Optionally, the detection results for each video frame may include the probabilities of one or more specified advertising interface elements. For each specified advertising interface element, the server can select the maximum or minimum probability from multiple probabilities for that element as the probability of that specified advertising interface element in the element index data; alternatively, it can calculate the average of the multiple probabilities as the probability of that specified advertising interface element in the element index data.

[0053] Optionally, the detection results for each video frame may include the location information of one or more specified advertising interface elements. Then, for each specified advertising interface element, the server can calculate the average of multiple location information values ​​for that specified advertising interface element, using this average as the location information of that specified advertising interface element in the element index data.

[0054] Optionally, the detection results for each video frame may include the screen ratio of one or more specified advertising interface elements. For each specified advertising interface element, the server can select the largest or smallest screen ratio from multiple screen ratios of that specified advertising interface element as the screen ratio of that specified advertising interface element in the element index data; alternatively, it can calculate the average of the multiple screen ratios as the screen ratio of that specified advertising interface element in the element index data.

[0055] Optionally, the detection results for each video frame may include the margin reachability of one or more specified advertising interface elements. For each specified advertising interface element, the server can select the largest or smallest margin reachability from multiple margin reachability values ​​for that specified advertising interface element as the margin reachability of that specified advertising interface element in the element index data; alternatively, it can calculate the average of the multiple margin reachability values ​​as the margin reachability of that specified advertising interface element in the element index data.

[0056] In some embodiments, an advertising video may include one or more sub-videos. Each sub-video corresponds to one or more advertising contents (ad creatives), and each sub-video corresponds to an independent advertising content; different sub-videos correspond to different advertising contents. The advertising video is formed by encapsulating one or more sub-videos together. Thus, when the advertising video is played, the one or more sub-videos are played sequentially as a whole. One playback of an advertising content is called one round of advertising, and the switching of advertising rounds corresponds to the switching of advertising content. Therefore, the number of advertising rounds of an advertising video is equal to the number of advertising contents or sub-videos it contains. The switching of advertising content or sub-videos can be triggered by the user. For example, after the user triggers the close control in the current advertising content, playback jumps to the next advertising content. Alternatively, the switching of advertising content or sub-videos can also be triggered automatically. For example, after the current advertising content finishes playing, the next advertising content plays automatically.

[0057] The timing metrics for ad videos include the number of ad turns. The server can count the number of ad turns based on multiple video frames. Considering that a close control appears during ad playback, and appears at the start of an ad turn and disappears at the end, the server can count the number of state cycles the close control experiences across multiple video frames. The number of ad turns can then be determined based on this number of state cycles. A state cycle includes the duration from the appearance of the close control to its disappearance. Optionally, in the presence of multiple state cycles, the time interval between two adjacent state cycles must be greater than or equal to a set duration threshold. The time interval refers to the difference between the time when the close control disappears in the previous state cycle and the time when it appears in the next state cycle. By setting a duration threshold, different ad turns can be distinguished, improving the accuracy of ad turn detection.

[0058] The server can detect whether each video frame contains a close control, thus obtaining the control's state for that frame. The control state can be selected from a first state and a second state. The first state indicates that the video frame contains a close control. The second state indicates that the video frame does not contain a close control. The server can use a computer vision model to detect whether a video frame contains a close control. Based on the control states and timing information of each video frame, the server can count the number of state cycles.

[0059] The multiple video frames form a video frame sequence in chronological order. The server can start scanning from the beginning of the video frame sequence or the end of the previous candidate state period to sequentially identify the control state of each video frame. When a first specified video frame that meets the start condition is identified, the server can determine a start time based on the time information of the first specified video frame. Scanning continues, and when a second specified video frame that meets the end condition is identified, the server can determine an end time based on the time information of the second specified video frame. The time period defined by the start time and the end time constitutes a candidate state period. In a candidate state period, the start time corresponds to the appearance time of the closed control, and the end time corresponds to the disappearance time of the closed control. The server can repeat the above process until the entire video frame sequence has been traversed.

[0060] The starting conditions may include at least one of the following: 1) There are no video frames before the current video frame, and its control state is in the first state; 2) The control state of the adjacent video frame before the current video frame is the second state, and the control state of the current frame is the first state.

[0061] Termination conditions may include: The control state of the adjacent video frame before the current video frame is in the first state, and the control state of the current frame is in the second state.

[0062] Optionally, to avoid jitter caused by video encoding, image noise, or momentary model misjudgment, the server can also use a first steady-state condition to identify the first specified video frame. The first steady-state condition may include: when the control states of K consecutive video frames are all in the first state, the first frame in that consecutive sequence is taken as the first specified video frame. Alternatively, the server can use a second steady-state condition to identify the second specified video frame. The second steady-state condition may include: when the control states of K consecutive video frames are all in the second state, the first frame in that consecutive sequence is taken as the second specified video frame. The first specified video frame is used to determine the start time. The second specified video frame is used to determine the end time. This improves the accuracy of the start time determination and the accuracy of the ad roll number determination. K is a positive integer, for example, a positive integer between 1 and 10. For example, K is 5.

[0063] The server can directly treat candidate state periods as independent state periods, thus determining the number of ad rounds based on the number of candidate state periods. For example, the server can use the number of candidate state periods as the number of ad rounds. Alternatively, to avoid jitter caused by video encoding, image noise, or instantaneous model misjudgments, the server can merge candidate state periods to obtain independent state periods; the number of ad rounds can then be determined based on the number of independent state periods. For example, the server can use the number of independent state periods as the number of ad rounds. This improves the accuracy of ad round detection through jitter reduction.

[0064] For example, for each candidate state period, the server can designate that state period as the previous candidate state period; it can also obtain the adjacent candidate state period that follows it as the next candidate state period. The server can calculate the time interval between the end time of the previous candidate state period and the start time of the next candidate state period. If this time interval is greater than or equal to a set duration threshold, the server can determine that the two candidate state periods are two independent state periods. If the time interval is less than the set duration threshold, the server can merge the two candidate state periods into one independent state period. The server can determine the number of ad rounds based on the final number of independent state periods.

[0065] The following is an example of a scenario where the number of ad rounds is determined.

[0066] The server can initialize the ad round counter R=0. Starting from the first video frame in the video frame sequence, the server checks each video frame for a close control. When K consecutive video frames contain the close control, the server designates the first of these K frames as the first specified video frame and determines the close control's time t1 based on its timing information. The server then continues checking. When K consecutive video frames do not contain the close control, the server designates the first of these K frames as the second specified video frame and determines the close control's time t2 based on its timing information. The server then obtains a state cycle and increments R by 1.

[0067] Afterward, the server can continue detection from the second specified video frame. If K consecutive video frames containing the close control are detected again, the server can calculate the time interval Δt between the first video frame of the K video frames and the previous second specified video frame. As an example, if Δt is greater than or equal to a set duration threshold, the server can use the first video frame of the K video frames as the new first specified video frame; the server can determine the new time t1 of the close control based on the time information of the new first specified video frame. The server continues subsequent detection to determine the new time t2, and then increments R by 1. As another example, if Δt is less than the set duration threshold, the server can consider the appearance of the close control to be a brief fluctuation within the same advertising round, ignore the K video frames, and continue detection from the last video frame of the K video frames.

[0068] The server repeats the above steps until the entire video frame sequence has been traversed. The final value of R is the number of ad turns.

[0069] In some embodiments, the server can detect the timing information of a specified advertising interface element based on multiple video frames. The timing metric data may include the timing information. The timing information includes at least one of the following: first appearance time and duration. The first appearance time indicates the moment when the specified advertising interface element first appears in the advertising video, and can determine how long a user needs to wait before observing the specified advertising interface element in order to interact with the advertising video. The duration indicates the duration for which the specified advertising interface element continuously appears in the advertising video, and can determine the size of the time window for user interaction with the advertising video.

[0070] The server can detect whether each video frame contains a specified advertising element. The server can select multiple target video frames from the plurality of video frames that are temporally consecutive and all contain the specified advertising element. The server can select a first video frame and a second video frame from the multiple target video frames. The first video frame can be the first target video frame in which the specified advertising element appears. The second video frame can be the last target video frame in which the specified advertising element appears. The server can determine the first appearance time of the specified advertising element based on the time information of the first video frame. For example, the time information includes the duration from the start of the video to the second video frame; the server can use the duration corresponding to the first video frame as the first appearance time. The server can determine the duration of the specified advertising element based on the time information of the first and second video frames. For example, the time information includes the duration from the start of the video to the second video frame; the server can calculate the difference between the durations corresponding to the first and second video frames as the duration.

[0071] By detecting time information based on multiple target video frames that are sequential in time and all contain specified interface elements, jitter caused by video encoding, image noise, and instantaneous model misjudgment can be suppressed, thereby improving detection accuracy.

[0072] The server can detect the timing information of one or more specified advertising interface elements based on multiple video frames. The timing information of each specified advertising interface element includes at least one of the following: first appearance time and duration.

[0073] Whether video frames are sequential in time can be determined based on the time information corresponding to each video frame. If there is no time information of other video frames between the time information of two video frames, then the two video frames can be determined to be sequential. For example, the server arranges and compares the video frames based on their time information. When the difference in time information between two video frames equals the display duration of a single frame of the advertisement video, the server can determine that the two video frames are sequential. Alternatively, when two video frames are in adjacent positions in a chronologically arranged frame sequence, the server can also determine that the two video frames are sequential.

[0074] Optionally, the ad video has one ad round. The server can select a set of target video frames from multiple video frames; it can directly select the first and second video frames from this set of video frames. Optionally, the ad video has multiple ad rounds. The server can select multiple sets of target video frames from multiple video frames. Each set of target video frames corresponds to one ad round. For each set of target video frames, the server can select the first and second video frames from that set of video frames. Thus, the server can obtain the time information (e.g., first appearance time, duration) of the specified interface elements in each ad round. Optionally, as mentioned above, the server can count the number of state cycles experienced by the close control in multiple video frames. A state cycle includes the duration from the appearance to the disappearance of the close control, thus corresponding to one ad round. The server can then obtain one or more state cycles. For each state cycle, the server can select video frames located within that state cycle from the multiple video frames; it can select multiple target video frames that are temporally continuous and all contain the specified ad interface elements from the videos within that state cycle. By narrowing the search range of target video frames, computational efficiency can be improved.

[0075] A set of target video frames may include multiple target video frames that are sequential in time and all contain the specified advertising interface elements.

[0076] In some embodiments, the quality detection result is used to represent the user experience of the advertisement. The quality detection result may include a first score. The first score is used to represent the overall quality of the user experience. The larger the first score, the better the overall user experience of the advertisement video. Alternatively, the quality detection result may also include multiple second scores, which are used to represent the quality of the user experience from multiple dimensions. For example, the multiple second scores may include a color score, an element score, and a time-series score. The color score is used to represent the quality of the user experience from the dimension of interface color, and can be specifically determined based on color index data. The element score is used to represent the quality of the user experience from the dimension of interface elements, and can be specifically determined based on element index data. The time-series score is used to represent the quality of the user experience from the time-series dimension, and can be specifically determined based on time-series index data. Of course, the quality detection result can also be other forms of results. For example, the server can use color index data, element index data, and time-series index data as the quality detection result. In this way, the quality detection result can directly include color index data, element index data, and time-series index data.

[0077] In some embodiments, the server can input color index data, element index data, and time series index data into the advertising quality assessment model to obtain the quality detection result output by the advertising quality assessment model. The advertising quality assessment model can be a machine learning model. Alternatively, the server can determine a first sub-detection result based on the color index data; determine a second sub-detection result based on the element index data; determine a third sub-detection result based on the time series index data; and determine the quality detection result based on the first, second, and third sub-detection results. For example, the first, second, and third sub-detection results can each be a score. The score is used to represent the degree of user experience. The server can then add the first, second, and third sub-detection results or perform a weighted sum to obtain the quality detection result.

[0078] For example, a server can input color index data into a color quality assessment model to obtain the first sub-detection result output by the color quality assessment model; it can input element index data into an element quality assessment model to obtain the second sub-detection result output by the element quality assessment model; and it can input time-series index data into a time-series quality assessment model to obtain the third sub-detection result output by the time-series quality assessment model. As another example, the server can pre-configure color quality rules, element quality rules, and time-series quality rules. Pre-configured color quality rules are used to evaluate the user experience of an advertisement from a color perspective. Element quality rules are used to evaluate the user experience of an advertisement from the perspective of its interface elements. Time-series quality rules are used to evaluate the user experience of an advertisement from a time-series perspective. The server can determine the first sub-detection result representing color quality based on color index data and pre-configured color quality rules; it can determine the second sub-detection result representing element quality based on element index data and pre-configured element quality rules; and it can determine the third sub-detection result representing time-series quality based on time-series index data and pre-configured time-series quality rules.

[0079] For example, color quality rules may include: a cumulative duration evaluation function for color anomalies and a duration evaluation function for color anomalies. The cumulative duration evaluation function for color anomalies represents the correspondence between the cumulative duration of color anomalies and the score. By inputting the first duration from the color index data into this function, the corresponding score can be obtained. The duration evaluation function for color anomalies represents the correspondence between the duration of color anomalies and the score. By inputting the second duration from the color index data into this function, the corresponding score can be obtained.

[0080] The first sub-detection result may include at least one of the following: the score output by the cumulative duration evaluation function for color anomalies, and the score output by the duration evaluation function for color anomalies. Alternatively, the first sub-detection result may also include: the sum of the scores output by the cumulative duration evaluation function for color anomalies and the duration evaluation function for color anomalies.

[0081] For example, element quality rules can include: a detection probability evaluation function, a location information evaluation function, a screen-to-body ratio evaluation function, and a margin accessibility evaluation function. The detection probability evaluation function represents the correspondence between probability and score; the probability is the likelihood that the ad video contains a specified ad interface element. By inputting the probability of the ad video containing the specified ad interface element into the detection probability evaluation function, the corresponding score can be obtained. The location information evaluation function represents the correspondence between location information and score; the location information is the location information of the specified ad interface element. By inputting the location information of the specified ad interface element into the location information evaluation function, the corresponding score can be obtained. The screen-to-body ratio evaluation function represents the correspondence between screen-to-body ratio and score. By inputting the screen-to-body ratio of the specified ad interface element into the screen-to-body ratio evaluation function, the corresponding score can be obtained. The margin accessibility evaluation function represents the correspondence between margin accessibility and score. By inputting the margin accessibility of the specified ad interface element into the margin accessibility evaluation function, the corresponding score can be obtained.

[0082] The second sub-detection result may include at least one of the following: the score output by the detection probability evaluation function, the score output by the location information evaluation function, the score output by the screen occupancy evaluation function, and the score output by the margin accessibility evaluation function. Alternatively, the second sub-detection result may also include the sum of the scores output by the detection probability evaluation function, the location information evaluation function, the screen occupancy evaluation function, and the margin accessibility evaluation function.

[0083] For example, time-series quality rules include: ad roll number evaluation function, first appearance time evaluation function, and duration evaluation function. The ad roll number evaluation function represents the correspondence between ad roll number and score. By inputting the ad roll number into the ad roll number evaluation function, the corresponding score can be obtained. The first appearance time evaluation function represents the correspondence between first appearance time and score. By inputting the first appearance time into the first appearance time evaluation function, the corresponding score can be obtained. The duration evaluation function represents the correspondence between duration and score. By inputting the duration into the duration evaluation function, the corresponding score can be obtained.

[0084] The third sub-detection result may include at least one of the following: the score output by the ad roll number evaluation function, the score output by the first appearance time evaluation function, and the score output by the duration evaluation function. Alternatively, the third sub-detection result may also include: the sum of the scores output by the ad roll number evaluation function, the first appearance time evaluation function, and the duration evaluation function.

[0085] In some embodiments, the server may pre-configure a rule set. The rule set includes one or more rules. Each rule corresponds to a type of metadata. Each rule includes color quality rules, element quality rules, and time-series quality rules, etc.

[0086] The server can match the corresponding target rules in the rule set based on the metadata of the advertisement video; it can determine the first sub-detection result to represent color quality based on color index data and color quality rules in the target rule set; it can determine the second sub-detection result to represent element quality based on element index data and element quality rules in the target rule set; it can determine the third sub-detection result to represent time quality based on time series index data and time series quality rules in the target rule set; and it can determine the quality detection result based on the first, second, and third sub-detection results.

[0087] In some embodiments, the server can acquire multiple video data sets. These multiple video data sets can be sent to the server by one or more terminal devices. The server can receive the multiple video data sets sent by the one or more terminal devices.

[0088] The element data of the multiple video data sets may be the same, different, or partially the same. The server can classify the multiple video data sets to obtain one or more video datasets. Each video dataset may include one or more video data sets. Each video data set in each video dataset may correspond to the same metadata. Therefore, for each video dataset, the server can perform statistical analysis on the detection results of each video data set to obtain the detection results of metadata (such as ad source, ad position, ad creative identifier, application version, regional information, etc.).

[0089] Please see Figure 2 This specification also provides an advertising quality testing device, comprising: Extraction unit 21 is used to extract multiple video frames from the advertising video; The first detection unit 22 is used to detect the interface color of the advertisement based on the multiple video frames and obtain color index data; The second detection unit 23 is used to detect the interface elements of the advertisement based on the multiple video frames and obtain element index data; The third detection unit 24 is used to detect the timing information of the advertisement based on the multiple video frames and obtain timing index data; The determining unit 25 is used to determine the quality inspection result of the advertisement based on the color index data, the element index data and the time sequence index data, wherein the quality inspection result is used to represent the user experience of the advertisement.

[0090] This specification also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described advertising quality detection method.

[0091] This specification also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described advertising quality detection method.

[0092] This specification also provides a computer program product, which includes a computer program that, when executed by a processor, implements the above-described advertising quality detection method.

[0093] Those skilled in the art will understand that this specification can be provided as a method, system, or computer program product. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0094] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments thereof. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. The computer may be a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0095] The functional units in the embodiments of this specification can be integrated into one processing unit, or each functional unit can exist physically separately, or two or more functional units can be integrated into one processing unit.

[0096] Those skilled in the art will understand that the descriptions of the various embodiments in this specification have different focuses, and parts not described in detail in a certain embodiment can be referred to in the relevant descriptions of other embodiments. Furthermore, it is understood that those skilled in the art, after reading this specification, can conceive of any combination of some or all of the embodiments listed in this specification without creative effort, and such combinations are also within the scope of disclosure and protection of this specification.

[0097] Although this specification has been described through embodiments, those skilled in the art will understand that the above embodiments are merely illustrative of the core ideas of this specification. Those skilled in the art will appreciate that many variations and modifications are possible with this specification. It is intended that the appended claims encompass these variations and modifications without departing from the spirit of this specification.

Claims

1. A method for detecting advertising quality, characterized in that, include: Extract multiple video frames from the advertisement video; Color index data is obtained by detecting the interface color of the advertisement based on the multiple video frames; Based on the multiple video frames, the interface elements of the advertisement are detected to obtain element index data; Based on the temporal information of the advertisements detected from the multiple video frames, temporal indicator data is obtained; Based on the color index data, the element index data, and the time sequence index data, the quality inspection result of the advertisement video is determined, and the quality inspection result is used to represent the user experience of the advertisement.

2. The method according to claim 1, characterized in that, The step of detecting the interface color of the advertisement based on the multiple video frames to obtain color index data includes: Identify color-abnormal video frames among the plurality of video frames. The color-abnormal video frames include color-abnormal regions. The color-abnormal regions are connected regions, and their color features satisfy a first condition and their areas satisfy a second condition. Based on the video frames with abnormal colors, the color index data is statistically analyzed; The color index data includes at least one of the following: a first duration and a second duration; the first duration is used to represent the cumulative duration of color anomalies in the advertisement video, and the second duration is used to represent the duration of color anomalies in the advertisement video.

3. The method according to claim 2, characterized in that, The color feature includes brightness, and the first condition includes: the mean brightness is greater than or equal to a first brightness threshold, and the brightness variance is less than or equal to a first variance threshold; The color-abnormal area is a white area, and the color-abnormal video frame is a white video frame.

4. The method according to claim 2, characterized in that, The color feature includes brightness, and the second condition includes: the mean brightness is less than or equal to a second brightness threshold, and the brightness variance is less than or equal to a second variance threshold; The color-abnormal area is a black area, and the color-abnormal video frame is a black video frame.

5. The method according to claim 1, characterized in that, The step of detecting the interface elements of the advertisement based on the multiple video frames and obtaining element index data includes: The video frame is input into a computer vision model to detect specified advertising interface elements in the video frame; Based on the detection results of the multiple video frames, the element index data is determined; The element index data includes at least one of the following: the probability that the advertisement video contains a specified advertisement interface element, the location information of the specified advertisement interface element, the screen ratio of the specified advertisement interface element, and the margin reachability of the specified advertisement interface element.

6. The method according to claim 1, characterized in that, The step of detecting the timing information of advertisements based on the multiple video frames to obtain timing indicator data includes: The number of state cycles experienced by the close control across the multiple video frames is counted; wherein, a state cycle includes the duration from the appearance to the disappearance of the close control; The number of ad rounds for the ad video is determined based on the number of state cycles. The time-series metric data includes the number of ad rounds.

7. The method according to claim 6, characterized in that, The number of state cycles is multiple; in the multiple state cycles, the time interval between two adjacent state cycles is greater than or equal to a set duration threshold, and the time interval includes the interval between the time when the close control disappears in the previous state cycle and the time when the close control appears in the next state cycle.

8. The method according to claim 1, characterized in that, The step of detecting the timing information of advertisements based on the multiple video frames to obtain timing indicator data includes: Based on the multiple video frames, detect the time information of the specified advertising interface element; The time information includes at least one of the following: the first appearance time and the duration; The time-series indicator data includes the time information.

9. The method according to claim 8, characterized in that, The multiple video frames are sequential in time; The time information for detecting the specified advertising interface element includes: Select multiple consecutive target video frames from the plurality of video frames that contain the specified advertising interface elements; Select a first video frame and a second video frame from multiple target video frames. The first video frame is the target video frame in which the specified advertising interface element first appears, and the second video frame is the target video frame in which the specified advertising interface element last appears. The first appearance time is determined based on the position of the first video frame in the advertisement video; The duration is determined based on the positions of the first and second video frames in the advertisement video.

10. The method according to claim 1, characterized in that, Determining the quality inspection result of the advertising video includes: Based on the color index data and the preset color quality rules, a first sub-detection result is determined to represent the color quality; Based on the element index data and the preset element quality rules, a second sub-detection result is determined to represent the element quality; Based on the time series index data and the preset time series quality rules, a third sub-detection result is determined to represent the time series quality; The quality inspection result is determined based on the first sub-detection result, the second sub-detection result, and the third sub-detection result.

11. An advertising quality testing device, characterized in that, include: Extraction unit, used to extract multiple video frames from an advertisement video; The first detection unit is used to detect the interface color of the advertisement based on the multiple video frames and obtain color index data; The second detection unit is used to detect the interface elements of the advertisement based on the multiple video frames and obtain element index data. The third detection unit is used to detect the timing information of the advertisement based on the multiple video frames and obtain timing index data; The determining unit is configured to determine the quality inspection result of the advertisement based on the color index data, the element index data, and the time sequence index data, wherein the quality inspection result is used to represent the user experience of the advertisement.

12. A computer device, comprising a memory, a processor, and computer programs / instructions stored in the memory, characterized in that, The processor executes the computer program / instructions to implement the steps of the method according to any one of claims 1 to 10.

13. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 10.

Images