OTT advertisement optimization system based on pointing behavior of remote controller

Abstract

The invention relates to the technical field of internet television services, in particular to an OTT advertisement optimization system based on a pointing behavior of a remote controller, which comprises a data acquisition module for distinguishing a cruising state, a staring state and a shaking state and extracting a user intention track by adopting a track processing algorithm based on speed and spatial-temporal clustering; the semantic analysis module is used for identifying advertisement creative elements by means of first frame analysis tracking and the like and generating advertisement semantic element map data; and the instruction generation module is used for performing space-time correlation mapping on the user intention track and a semantic map, calculating element attention scores including high-cost activation times and staring micro-motion amplitudes, finally generating a creative optimization instruction, and automatically adjusting the layout of advertisement creative elements. According to the method, refined, quantifiable and automatic OTT advertisement closed-loop optimization is realized.

Description

Technical Field

[0001] This invention relates to the field of Internet TV service technology, specifically to an OTT advertising optimization system based on remote control pointing behavior. Background Technology

[0002] In the OTT (Over-The-Top) advertising sector, creative optimization is a crucial step in improving advertising effectiveness. Currently, mainstream advertising optimization methods in the industry have serious flaws. First, they rely on traditional manual A / B testing, which has a long setup, execution, and analysis cycle, resulting in low efficiency and delays. Second, they depend on vague post-click data. This data is coarse-grained; it only reflects whether a user made a final click, but it cannot explain which specific element of the ad creative attracted the user, thus failing to guide advertisers in refined creative iterations.

[0003] In the field of PC / mobile internet advertising analytics, technologies already exist that utilize mouse movement trajectories and hover heatmaps to analyze ad placement and optimize user experience. This method has proven effective on PCs because it provides richer procedural data than click-through rates (CTR). However, when attempting to directly apply this mature advertising analytics model to OTT large-screen scenarios, a fundamental obstacle arises: PC mouse interactions are precise, and their trajectory data effectively reflects user intent. But OTT interactions are "backward-leaning," with users operating directional remotes in the air, inevitably resulting in a large amount of "unconscious trembling" noise from natural hand shaking. Therefore, the commercial challenge in this field is that the "high noise" characteristic of OTT remote control data renders mature "trajectory-based ad attention attribution" methods on PCs completely ineffective. Without effectively separating "unconscious trembling" from "conscious gaze" in noisy trajectory data, it's impossible to accurately attribute attention signals to specific ad creative elements.

[0004] In conclusion, how to implement an automated, refined, and quantifiable OTT advertising creative optimization system to replace inefficient manual A / B testing and vague click data is a technical problem that urgently needs to be solved in this field.

[0005] To address this, an OTT advertising optimization system based on remote control pointing behavior is proposed. Summary of the Invention

[0006] The purpose of this invention is to provide an OTT advertising optimization system based on remote control pointing behavior. The system includes a data acquisition module that employs a trajectory processing algorithm based on speed and spatiotemporal clustering to distinguish between cruising, staring, and jittery states, extracting the user's intent trajectory; a semantic analysis module that identifies advertising creative elements through first-frame analysis and tracking, generating an advertising semantic element map; and an instruction generation module that spatiotemporally maps the user's intent trajectory with the semantic map, calculates element attention scores including high-cost activation counts and staring micro-motion amplitude, and finally generates creative optimization instructions to automatically adjust the layout of advertising creative elements. This invention achieves refined, quantifiable, and automated closed-loop optimization of OTT advertising.

[0007] To achieve the above objectives, the present invention provides the following technical solution: An OTT advertising optimization system based on remote control pointing behavior includes: Data acquisition module: Receives raw trajectory data streams collected by a directional remote controller; employs a trajectory processing algorithm based on velocity and spatiotemporal clustering to extract dormant, staring, and cruising trajectory data from the raw trajectory data stream; distinguishes between staring and jittery dynamic data by calculating spatial variance; applies a Kalman filter algorithm to smooth and reduce noise in the staring and cruising trajectory data to generate user intent trajectory data; Semantic analysis module: Receives advertising visual content data, applies an image recognition model to perform semantic understanding on the advertising visual content data, identifies advertising creative elements in the advertising visual content, and generates advertising semantic element map data containing the advertising creative elements and their corresponding location coordinates; Instruction generation module: It performs spatial and temporal association mapping between user intent trajectory data and advertising semantic element map data; based on the association mapping, it calculates element attention scores for advertising creative elements, generates creative optimization instructions based on the element attention scores, and automatically adjusts the layout of advertising creative elements in the advertising visual content.

[0008] Preferably, the trajectory processing algorithm based on speed and spatiotemporal clustering specifically includes: receiving the original trajectory data stream; when the remote control pointer speed is higher than a preset speed threshold, it is determined to be cruise-state trajectory data; when the pointer speed is not higher than the preset speed threshold, the set of coordinate points of the original trajectory data stream within a preset time window is obtained; when the spatial variance of the coordinate points in the spatial distribution is less than a clustering threshold, it is determined to be staring-state trajectory data; when the spatial variance of the coordinate points in the spatial distribution is greater than a clustering threshold, it is determined to be jitter-state data; the spatial variance of the staring-state trajectory data is extracted as micro-intention data; and the Kalman filter algorithm is applied to smooth the identified staring-state and cruise-state data. Sleep mode indicates that the remote control is stationary and has no movement within a preset time; jitter mode indicates the noise data of the user's unconscious hand shaking; cruise mode indicates that the remote control pointer moves from point A to point B; staring mode indicates that the user is staring at an element.

[0009] Preferably, when the advertising visual content data is a static image, the semantic understanding implementation process includes: Subject Detection: Applying a subject detection model to select the main elements in the material and output their category and bounding box coordinates; Text Recognition: Applying optical character recognition technology, including text detection and text recognition, to extract all visible text information; Face Recognition: Applying a face recognition model to extract facial features from detected "human" elements and compare them with a preset "celebrity material library" to automatically label the "celebrity" identity; Key Element Recognition: Including a dedicated logo detection model and rule-based call-to-action button recognition.

[0010] Preferably, when the advertising visual content data is video, first-frame analysis and tracking are used, and the specific implementation steps include: Keyframe extraction: The video is segmented into shots using a scene boundary detection algorithm; First frame analysis: On the first keyframe of each shot, a semantic understanding process for static images is executed to identify all initial advertising creative elements within the shot; Tracking: A target tracker is initialized for each identified initial advertising creative element; Trajectory generation and calibration: In subsequent frames of the shot, the target tracker is run at high speed to predict and update the continuously changing bounding box positions of the elements, and the semantic understanding process for static images is periodically re-run to calibrate the tracker position deviation and discover new elements.

[0011] Preferably, the specific implementation of the spatial and temporal dimension association mapping includes: traversing each gaze state event in the user intent trajectory data and obtaining its timestamp and coordinates; temporal dimension mapping: using the timestamp to query the advertising semantic element map data, retrieving the advertising creative elements that are in the screen display state within the corresponding time segment as candidate elements; spatial dimension mapping: obtaining the position coordinates of the candidate elements corresponding to the timestamp; association determination: when the coordinates of the gaze state event fall within the bounding box defined by the position coordinates of the candidate element, the attention contribution of this gaze state event is attributed to the advertising creative element.

[0012] Preferably, the association determination step in the spatial dimension mapping further includes: before checking whether the coordinates fall within the bounding box, the system first dynamically generates a gravity range for the position coordinates of the candidate element; the gravity range is an expansion of a preset pixel value on each of the four sides of the original bounding box; the final association determination is to check whether the coordinates of the staring event fall within the expanded gravity range.

[0013] Preferably, the calculation of the element attention score involves obtaining the following attention metrics for the advertising creative element: Total gaze duration: the total number of milliseconds of all gaze events associated with the element, reflecting the time the user spends on the element; First discovery speed: the number of milliseconds from the start of the advertisement to the first time the element is gazed at, reflecting the element's prominence; High-cost activation count: the number of times the user picks up the remote control and points it at the element for the first time after the remote control has been in a dormant state, reflecting the element's appeal; Gaze micro-motion amplitude: the statistical data of micro-intent on the element, reflecting the user's gaze stability and focus.

[0014] Preferably, the process of calculating the attention score includes: applying an independent weight value to the total gaze duration, first-time discovery speed, number of high-cost activations, and the reciprocal of the gaze micro-movement amplitude for weighted combination; The weights are derived through training using a multi-task learning architecture, which includes: a shared representation layer: constructing a shared underlying network that receives four metrics as input and learns a combined high-dimensional feature representation; a multi-task head: building multiple independent output layers in parallel on top of the shared representation layer, wherein: a main task head: predicting the click-through rate corresponding to ad exposure through a logistic regression layer based on the shared representation; and an auxiliary task head: an additional logistic regression layer that predicts the probability of occurrence of high-value sparse signals based on the same shared representation; the sparse signals include high-cost activation counts and gaze micro-motion amplitude. The entire model is trained by jointly optimizing the loss function of all task heads, forcing the shared representation layer to learn the features of sparse signals; the weight values ​​calculated by the final element attention score are the corresponding weights learned by the main task head after the multi-task learning model is trained.

[0015] Preferably, the creative optimization instructions specifically include: element-level optimization instructions: based on the element attention score, automatically select the element with the highest score from multiple candidate ad creative elements for combined delivery; layout dynamic adjustment instructions: based on the score ranking of the element attention score and historical intent trajectory heatmap data, automatically fine-tune key ad creative elements to the area determined by the historical intent trajectory heatmap data where the user attention hover frequency is higher than a preset frequency threshold; automated A / B testing instructions: automatically generate an optimized version of ad visual content enhanced based on the element attention score, and automatically conduct a small-scale A / B test with the original ad visual content.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention employs a trajectory processing algorithm based on velocity and spatiotemporal clustering through a data acquisition module. It distinguishes between staring and jittery states by calculating spatial variance and applies Kalman filtering for smoothing and noise reduction. This approach effectively separates the user's true gaze intent from noisy trajectory data, addressing the core technical challenges mentioned in the background section and laying a data foundation for subsequent accurate attribution.

[0017] 2. This invention identifies specific elements in advertising materials through a semantic analysis module, and then uses an instruction generation module to perform spatiotemporal mapping between gaze data and elements. More importantly, it constructs a multi-dimensional element attention score that includes total gaze duration, first-time discovery speed, number of high-cost activations, and the amplitude of gaze micro-movements, thereby achieving refined and quantifiable in-depth analysis of user attention.

[0018] 3. Based on the calculated element attention scores, this invention automatically generates creative optimization instructions through an instruction generation module. These instructions include automatically selecting and combining the elements with the highest scores, automatically fine-tuning key elements to the user's golden attention zone, and automatically generating optimized creative materials and initiating small-scale A / B testing. This replaces inefficient manual operations, forming a highly efficient, closed-loop, and automated advertising creative optimization system. Attached Figure Description

[0019] Figure 1 This invention provides a structural diagram of an OTT advertising optimization system based on remote control pointing behavior. Figure 2 A flowchart of an OTT advertising optimization system based on remote control pointing behavior is provided for an embodiment of the present invention; Figure 3 A flowchart illustrating the implementation of element attention scoring in an embodiment of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Please see Figures 1 to 3 This invention provides an OTT advertising optimization system based on remote control pointing behavior, the technical solution of which is as follows: An OTT advertising optimization system based on remote control pointing behavior includes: Data acquisition module: Receives raw trajectory data streams collected by a directional remote controller; employs a trajectory processing algorithm based on velocity and spatiotemporal clustering to extract dormant, staring, and cruising trajectory data from the raw trajectory data stream; distinguishes between staring and jittery dynamic data by calculating spatial variance; applies a Kalman filter algorithm to smooth and reduce noise in the staring and cruising trajectory data to generate user intent trajectory data; Semantic analysis module: Receives advertising visual content data, applies an image recognition model to perform semantic understanding on the advertising visual content data, identifies advertising creative elements in the advertising visual content, and generates advertising semantic element map data containing the advertising creative elements and their corresponding location coordinates; Instruction generation module: It performs spatial and temporal association mapping between user intent trajectory data and advertising semantic element map data; based on the association mapping, it calculates element attention scores for advertising creative elements, generates creative optimization instructions based on the element attention scores, and automatically adjusts the layout of advertising creative elements in the advertising visual content.

[0022] Example 1: This embodiment will elaborate in detail on the specific application of the present invention in future automotive advertising A / B testing scenarios.

[0023] As one embodiment of the present invention, refer to Figure 1 A structural diagram of an OTT advertising optimization system based on remote control pointing behavior, refer to... Figure 2 A flowchart of an OTT advertising optimization system based on remote control pointing behavior, refer to Figure 3 The flowchart for implementing element attention scoring.

[0024] Furthermore, the trajectory processing algorithm based on speed and spatiotemporal clustering specifically includes: receiving the original trajectory data stream; when the remote control pointer speed is higher than a preset speed threshold, it is determined to be cruise-state trajectory data; when the pointer speed is not higher than the preset speed threshold, the set of coordinate points of the original trajectory data stream within a preset time window is obtained; when the spatial variance of the coordinate points in the spatial distribution is less than a clustering threshold, it is determined to be staring-state trajectory data; when the spatial variance of the coordinate points in the spatial distribution is greater than a clustering threshold, it is determined to be jitter-state data; the spatial variance of the staring-state trajectory data is extracted as micro-intention data; and the Kalman filter algorithm is applied to smooth the identified staring-state and cruise-state data. Sleep mode indicates that the remote control is stationary and has not moved for a preset time; jitter mode indicates the noise data of the user's unconscious hand shaking; cruise mode indicates that the remote control pointer moves from point A to point B; staring mode indicates that the user is staring at an element. The speed threshold is set in the range of (30, 200) pixels / second; the duration of the time window is in the range of (100, 500) milliseconds, preferably (200, 300) milliseconds, to balance noise filtering effect and real-time response capability; the clustering threshold of the spatial variance is a radius of R pixels, and the value of R is in the range of (5, 30) pixels.

[0025] To implement the Kalman filter algorithm, the system constructs a four-dimensional state vector, including x-coordinate, y-coordinate, x-direction velocity, and y-direction velocity. The state transition matrix is ​​set based on a uniform motion model. The observation matrix is ​​set to observe only the x and y coordinates. The process noise covariance (Q) and observation noise covariance (R) are set as follows: when the system is determined to be in cruise mode, the process noise covariance Q is increased to allow for rapid trajectory changes; when the system is determined to be in staring mode, the process noise covariance Q is decreased and the observation noise covariance R is increased to enhance the smoothing effect and suppress jitter.

[0026] Specifically, the system receives the user's raw trajectory data stream and determines it to be in sleep mode if the remote control remains motionless for 5 consecutive seconds. When a celebrity appears in the A version of the advertisement, the user picks up the remote control, and the pointer speed exceeds 100 pixels per second; the system classifies this as cruise trajectory data. When the pointer moves to the celebrity's face area, and the pointer speed falls below this threshold, the system receives a set of coordinate points within 300 milliseconds. Because the user is "gazing" at the celebrity, these points are highly convergent in space, and their spatial variance is less than the clustering threshold, meaning they fall within a circle with a radius of 10 pixels. The system classifies this as gaze-based trajectory data. The system extracts the spatial variance of this gaze-based data as micro-intent data. If the user's hand is simply shaking unconsciously, the coordinate points will be divergent, and the spatial variance will exceed the clustering threshold; this is then classified as jerky data and discarded. Finally, the system applies a Kalman filter algorithm to smooth and denoise the identified gaze-based and cruise-based data to generate intent trajectory data.

[0027] This invention achieves effective differentiation between gaze-based and jittery movements in OTT scenarios by combining speed thresholding and spatiotemporal clustering. The application of Kalman filtering further improves the smoothness and accuracy of intent trajectory data, laying a data foundation for subsequent accurate attribution.

[0028] Furthermore, when the advertising visual content data is a static image, the semantic understanding implementation process includes: Subject Detection: Applying a subject detection model, the main elements in the material are selected and their category and bounding box coordinates are output; Text Recognition: Applying optical character recognition technology, including text detection and text recognition, to extract all visible text information; Face Recognition: Applying a face recognition model, facial features are extracted from detected "human" elements and compared with a preset "celebrity material library" to automatically label the "celebrity" identity; Key Element Recognition: Including a dedicated trademark detection model and a rule-based call-to-action button recognition.

[0029] Specifically, the subject detection module runs a pre-trained Faster R-CNN to quickly select two main elements in the image: e1 (category: "person") and e2 (category: "car"), and outputs their respective bounding box coordinates. The text detection model DBNet first locates the area containing the advertising slogan "New Interstellar S1 Launched" and the button text "Book Now"; then, the text recognition model CRNN extracts these specific text contents. At the same time, the ArcFace model automatically performs facial feature extraction on the detected "person" element and immediately compares it with a pre-set "celebrity material library" and automatically labels it. Finally, the key element recognition module is activated: a dedicated trademark detection model identifies the "Future Car" brand logo located in the corner; simultaneously, a rule-based model combines the "Book Now" text extracted by OCR and its visual button shape to determine it as a call-to-action (CTA) button. Through this process, the system accurately deconstructs a static image into a set of structured data containing coordinates, categories, and semantic labels.

[0030] This invention, by clearly identifying specific creative elements such as the advertising subject, promotional text, spokesperson, and brand identity, transforms vague attention hotspots into clear semantic attribution tables. This allows advertisers to quantify the specific contribution of each creative element in a refined manner for the first time.

[0031] Furthermore, when the advertising visual content data is video, first-frame analysis and tracking are employed. Specific implementation steps include: Keyframe extraction: The video is segmented into shots using a scene boundary detection algorithm; First frame analysis: On the first keyframe of each shot, a semantic understanding process for static images is executed to identify all initial advertising creative elements within the shot; Tracking: A target tracker is initialized for each identified initial advertising creative element; Trajectory generation and calibration: In subsequent frames of the shot, the target tracker is run at high speed to predict and update the continuously changing bounding box positions of the elements, and the semantic understanding process for static images is periodically re-run to calibrate the tracker position deviation and discover new elements.

[0032] Specifically, taking the 15-second video advertisement for Future Car A version as an example, the system first initiates keyframe extraction: using a scene boundary detection algorithm, the video is automatically segmented into different shots. For example, it accurately identifies that at t=12s, the advertisement switches from a shot of a celebrity driving to a close-up of the CTA button. Next, the system performs first-frame analysis: on the first keyframe of the first shot (0-12 seconds), a complete static image semantic understanding process is executed to identify all initial advertising creative elements within that shot, such as e1 (celebrity) and e2 (car). Subsequently, the system initializes the DeepSORT target tracker for e1 and e2. In the trajectory generation and calibration phase: in subsequent frames of the first shot, for example from t=0.03s to t=11.9s, the target tracker is run at high speed to predict and update the continuously changing bounding box positions of e1 (celebrity) and e2 (car). Simultaneously, the system reruns the static semantic understanding process every 30 frames to calibrate the tracker's positional deviation and discover any newly emerging elements. When a new shot is detected at t=12s, the system performs "first frame analysis" again, at which point it identifies a new element e4 (the "Reserve Now" button) and initializes a new tracker for it.

[0033] The advertising semantic element map data is specifically a hash table or time-series data structure with timestamps as keys. Each key corresponds to a list containing all visible advertising creative elements on the screen at that timestamp. Each item in the list (representing an element) includes: a unique element identifier (ID), an element category (e.g., "celebrity"), and the element's bounding box coordinates ([x_min, y_min, x_max, y_max]). For video, the target tracker updates its bounding box coordinates in each frame and writes them into the map data at the corresponding timestamp, thus achieving continuous recording of the dynamic element positions.

[0034] The semantic analysis module further includes a parallel audio semantic analysis submodule, which analyzes the audio tracks accompanying the advertising visual content data and identifies and labels the timestamps of key audio events, including brand sound effects, promotional voices, or specific sound effects; the spatial and temporal dimension association mapping in the instruction generation module further includes performing audio-visual cross-modal association, associating the gaze trajectory data with the audio events that are immediately adjacent on the timestamp.

[0035] In the relaxed, reclining environment of the OTT living room, audio often serves as a leading signal to capture the user's attention. This invention, through audio-visual cross-modal association, enables the system to determine for the first time whether the user's gaze was triggered by an adjacent audio event. This allows element attention scores to quantify the true contribution of audio creative to visual attention, significantly improving the depth and accuracy of attribution for advertising effectiveness.

[0036] This invention employs a first-frame analysis and tracking strategy, where the system runs a high-cost full suite of AI analysis only in the first frame of a shot, while using a lightweight target tracker in subsequent frames. This combination of tracking and periodic calibration significantly improves the computational efficiency of semantic analysis for video advertisements while ensuring tracking accuracy.

[0037] Furthermore, the specific implementation of the spatial and temporal dimension association mapping includes: traversing each gaze state event in the user intent trajectory data and obtaining its timestamp and coordinates; temporal dimension mapping: using the timestamp to query the advertising semantic element map data, retrieving the advertising creative element that is in the screen display state within the corresponding time segment as a candidate element; spatial dimension mapping: obtaining the position coordinates of the candidate element corresponding to the timestamp; association determination: checking whether the coordinates of the gaze state event fall within the bounding box defined by the position coordinates of the candidate element. If they do, the attention contribution of this gaze state event is attributed to the advertising creative element.

[0038] Specifically, taking the Future Auto A version advertisement as an example, when the system traverses the user's intent trajectory data, it obtains a user's gaze event with a timestamp of t=5000ms, coordinates (300, 450), and a duration of 900ms. During time-dimensional mapping, the system uses t=5000ms to query the advertisement's semantic element map data, retrieving candidate elements currently on screen including e1 (the celebrity) and e2 (the car itself). Next, in spatial-dimensional mapping, the system obtains their position coordinates at 5000ms; for example, the bounding box of e1 is [200, 400, 400, 500], and the bounding box of e2 is [450, 400, 800, 600]. Finally, an association determination is performed: the system checks whether the gaze coordinates (300, 450) fall within the bounding box defined by the candidate element's position coordinates. The determination finds that the coordinates fall within the bounding box of e1 (the celebrity) [200, 400, 400, 500]. If the match is successful, the system will attribute the attention contribution of this 900ms gaze event to the advertising creative element e1 (celebrity).

[0039] Furthermore, the association determination step in the spatial dimension mapping further includes: before checking whether the coordinates fall within the bounding box, the system first dynamically generates a gravity range for the position coordinates of the candidate element; the gravity range is an expansion of a preset pixel value on each of the four sides of the original bounding box; the final association determination is to check whether the coordinates of the staring event fall within the expanded gravity range.

[0040] Specifically, the gaze event F1 is captured with a timestamp of t=2300ms and coordinates (x: 650, y: 720). After time mapping, one of the candidate elements is the "Book Now" CTA button (e3), whose bounding box at t=2300ms is [600, 700, 800, 800]. To address the low pointing accuracy issue caused by the backward tilting operation of the OTT remote control, before checking whether the coordinates fall within the bounding box, the system first dynamically generates a gravity range (GBB) for the candidate element's position coordinates. This GBB expands each of the four sides of the original bounding box by a preset pixel value; for example, expanding each side by 20 pixels, generating a new GBB of [580, 680, 820, 820]. The final association determination checks whether the coordinates (x: 650, y: 720) of the gaze event fall within the expanded gravity range. In this case, the coordinate point falls within [580, 680, 820, 820], so the system determines that the attention contribution of this gaze event is attributed to e3 (CTA button).

[0041] This invention introduces a gravitational range—that is, dynamically expanding a buffer zone on top of the original bounding box—to robustly capture gaze events with clear intent but slightly off-target orientation. This significantly improves the accuracy and recall of attention attribution in noisy environments.

[0042] Furthermore, the calculation of the element attention score involves obtaining the following metrics for advertising creative elements: Total gaze duration: the total number of milliseconds of all gaze events associated with the element, reflecting the time users spend on the element; First discovery speed: the number of milliseconds from the start of the advertisement to the first time the element is gazed at, reflecting the element's prominence; High-cost activation counts: the number of times a user picks up the remote control and points it at the element for the first time after it has been in a dormant state, reflecting the element's appeal; Gaze micro-motion amplitude: the statistical data of micro-intent on the element, reflecting the stability and focus of the user's gaze.

[0043] In the data acquisition module, a spatial variance value is calculated for each set of coordinate points identified as a gaze state. This variance value is stored along with the timestamp and center coordinates of the gaze state event. When calculating the gaze micro-motion amplitude (MMT_j) of element j, the system retrieves all gaze state events that are spatiotemporally associated with element j, extracts the spatial variance values ​​corresponding to these events, and finally calculates the arithmetic mean of these variance values ​​as the MMT_j index, reflecting the average degree of user observation initiative on element j.

[0044] Specifically, in the evaluation of future car A-version advertisements, attention metrics are calculated for advertising creative elements, such as the "star" e1. First, the total gaze duration is calculated as 850,000 milliseconds, derived by summing the total milliseconds of all gaze events associated with e1, reflecting the user's time spent on e1. Second, the initial discovery speed is recorded as 1200 milliseconds, the time from the start of the advertisement to the first gaze on e1, reflecting the prominence of the star element. Next, the system counts 300 high-cost activations, the number of times a user picks up the remote control from its dormant state and points it at e1, reflecting the e1's appeal. Finally, the system also counts the micro-motion amplitude of gazes on e1, the statistical value of micro-intent data captured while gazing at the star, to reflect the user's gaze stability and level of focus.

[0045] The acquisition of high-cost activation counts further includes: when an event is recorded in which a user activates an element from a remote control's sleep state, the duration of the sleep state before activation is also acquired; and the number of events is weighted based on the duration to generate a weighted activation cost score, wherein the duration of the sleep state is proportional to the assigned weight.

[0046] By incorporating the duration of the dormant state before activation as a weight, this invention can more precisely differentiate the costs of different activation events. For example, it can distinguish between activation after shallow interaction (e.g., 30 seconds of dormancy) and activation after deep immersion (e.g., 10 minutes of dormancy). This allows the system to more accurately quantify the true appeal of advertising creative elements in breaking immersion and awakening users, providing a more precise signal for attention scores.

[0047] This invention constructs a multi-dimensional, deep attention evaluation system that far exceeds traditional dwell time. It innovatively introduces initial discovery speed (reflecting element prominence), high-cost activation frequency (reflecting user's willingness to act), and gaze micro-motion amplitude (reflecting the degree of active observation by the user). This indicator system enables refined and quantifiable in-depth analysis of user attention.

[0048] Furthermore, the process of calculating the attention score includes: applying an independent weight value to the total gaze duration, first-time discovery speed, number of high-cost activations, and the reciprocal of the gaze micro-motion amplitude for weighted combination; The weights are derived through training using a multi-task learning architecture, which includes: a shared representation layer: constructing a shared underlying network that receives four metrics as input and learns a combined high-dimensional feature representation; a multi-task head: building multiple independent output layers in parallel on top of the shared representation layer, wherein: a main task head: predicting the click-through rate corresponding to ad exposure through a logistic regression layer based on the shared representation; and an auxiliary task head: an additional logistic regression layer that predicts the probability of occurrence of high-value sparse signals based on the same shared representation; the sparse signals include high-cost activation counts and gaze micro-motion amplitude. The entire model is trained by jointly optimizing the loss function of all task heads, forcing the shared representation layer to learn the features of sparse signals; the weight values ​​calculated by the final element attention score are the corresponding weights learned by the main task head after the multi-task learning model is trained.

[0049] Specifically, the system takes four metrics as inputs: total gaze duration, the reciprocal of the first discovery speed, the number of high-cost activations, and the amplitude of gaze micro-movements. These are fed into a shared underlying network to learn a combined high-dimensional feature representation.

[0050] Above this shared representation layer, the system constructs multiple independent output layers in parallel. The main task head, through a logistic regression layer, predicts the final business outcome corresponding to ad impressions, i.e., click-through rate (CTR), based on the shared representation. Simultaneously, the system sets up auxiliary task heads, which, based on the same shared representation, predict the probability of occurrence of highly valuable signals that are very sparse in the data, such as the number of high-cost activations or the amplitude of gaze micro-motions.

[0051] In the tens of millions of historical exposures of future cars, the high-cost activation event of picking up the remote control might only occur in 0.1%, which the main task head might ignore when predicting CTR. The existence of the auxiliary task head, by jointly optimizing the loss function of all task heads to train the entire model, forces the shared representation layer to learn to effectively extract features from these sparse signals. After training, the weights used for EAS calculation are the corresponding weights of the various metrics learned by the main task head. For example, the training results might show that the weight of high-cost activations, such as w3=0.6, is much higher than that of total gaze duration, such as w1=0.2, because it has been shown to have the strongest correlation with the actual click-through rate.

[0052] This invention employs a multi-task learning architecture and uses ad click-through rate as the primary optimization objective. This ensures that the weights of various metrics are no longer subjectively set, but rather derived through reverse training using machine learning. This guarantees that the final element attention score is highly correlated with actual business conversion results, making it a highly reliable predictive conversion contribution score.

[0053] Furthermore, the creative optimization instructions specifically include: element-level optimization instructions: based on the element attention score, automatically select the element with the highest score from multiple candidate ad creative elements for combined delivery; dynamic layout adjustment instructions: based on the score ranking of the element attention score and historical intent trajectory heatmap data, automatically adjust key ad creative elements to the area determined by the historical intent trajectory heatmap data where the user attention hover frequency is higher than a preset frequency threshold; automated A / B testing instructions: automatically generate an optimized version of ad visual content enhanced based on the element attention score, and automatically conduct a small-scale A / B test with the original ad visual content.

[0054] Specifically, in future automotive advertising scenarios, the system automatically generates creative optimization instructions based on the calculated Element Attention Score (EAS). First, the element-level optimization instruction is activated: the system compares the CTA button in version A (EAS score of only 30) with the CTA button in version B (EAS score as high as 92), automatically selecting the highest-scoring CTA element from version B for combined placement. Next, the dynamic layout adjustment instruction is triggered: based on the high-scoring ranking of the celebrity in version A (EAS=85) and combined with historical intent trajectory heatmap data, the system determines the screen area where the celebrity is located as the golden area; the instruction fine-tunes the key advertising creative element (i.e., the high-scoring CTA in version B) to this golden area where user attention most frequently lingers. Finally, the automated A / B testing instruction takes effect: the system automatically generates a "Version C" optimization plan, which strengthens the two high-EAS-scoring elements: the celebrity in version A (high attractiveness) and the CTA in version B (high activation); the visual content of version C is automatically generated and immediately subjected to a small-scale A / B test with the original versions (versions A and B) to verify the optimization effect.

[0055] The instruction generation module further acquires intent delay metrics. By analyzing the user's remote control behavior within a preset time after the end of the ad visual content, it evaluates the entire ad creative and uses it as a supplementary basis for the creative optimization instruction. The remote control behavior includes: immediate activation, where the remote control performs rewind or volume adjustment; and delayed activation, where the remote control pauses.

[0056] In OTT scenarios, the user's immediate reaction at the moment an ad ends is the most direct signal for measuring its intrusiveness and impact. This solution, by tracking intent delay, provides a new and high-value evaluation dimension for the friendliness and effectiveness of ad creatives.

[0057] This invention achieves an automated closed-loop process for ad optimization. Based on the calculated high-reliability EAS score, the system can automatically generate three specific, executable optimization instructions: element-level optimization, dynamic layout adjustment, and automated A / B testing. This replaces traditional, inefficient manual analysis and operations, greatly improving the iteration efficiency and commercial effectiveness of OTT ad creatives.

[0058] This invention provides a complete and automated OTT advertising optimization solution. Through the collaborative work of three modules, it solves the core problems of low efficiency in manual testing, coarse-grained click data, and the failure of PC-based heatmap technology in noisy OTT environments. This invention creates a highly efficient, refined, and quantifiable closed-loop system from noise data acquisition to semantic understanding and then to automatic optimization instructions.

[0059] Example 2: Embodiment 2 of the present invention provides a specific implementation process for obtaining element attention scores.

[0060] As one embodiment of the present invention, refer to Figure 1 A structural diagram of an OTT advertising optimization system based on remote control pointing behavior, refer to... Figure 2 A flowchart of an OTT advertising optimization system based on remote control pointing behavior, refer to Figure 3 The flowchart for implementing element attention scoring.

[0061] The attention score calculation process is initiated after the instruction generation module completes the spatiotemporal mapping of user intent trajectory data and ad semantic element map data. For each creative element in the ad visual content (taking element j as an example), the system will acquire indicators, build a model, and calculate the final score step by step.

[0062] The system first calculates the following four core metrics for element j from all gaze events associated with it: Total Gaze Duration (TDT_j): This metric is the sum of all gaze events attributed to element j in milliseconds. For example, in 1 million impressions of a future car A-version advertisement, element e1 (the celebrity) accumulated a gaze duration of 850,000 milliseconds. This metric directly reflects the amount of time users spend on elements and is a fundamental indicator of an element's "eye-catching" ability.

[0063] First Detection Speed ​​(TFF_j): This metric records the number of milliseconds taken from the start of the advertisement until element j is first hit by a gaze event. For example, the TFF for element e1 (the star) is 1200 milliseconds. This metric reflects the "salience" or "visibility" of an element. The lower the TFF value, the easier it is for the element to be detected.

[0064] High-cost activation count (AE_j): This metric represents the total number of times a user picks up the remote control and points it at element j the first time it leaves the remote control in a dormant state (i.e., the remote control has been idle for N seconds). In OTT contexts, this is a high-value signal, reflecting the appeal of element j or its ability to activate deeper user intent. For example, e4 (CTA button) in version B ads achieved an extremely high number of AEs.

[0065] Gaze Micromotion Amplitude (MMT_j): This metric is the micro-intent data of all gaze-state events associated with element j, specifically the spatial variance value calculated and stored in the data acquisition module. This metric reflects the user's level of focus when gazing at this element. A lower gaze micromotion amplitude indicates more stable hand control and a more focused gaze, representing higher quality focus rather than a slight tremor indicating distraction or impending shift in attention.

[0066] After obtaining the above four metrics, the system uses an independent weight value (w) to perform a weighted combination to calculate the final element attention score (EAS_j). Specifically, the system weights and sums the total gaze duration, the number of high-cost activations, the gaze micro-motion amplitude, and the reciprocal of the first-detection speed (1 / TFF_j). The reciprocal is used because the lower the TFF value, the faster the detection, and the higher its contribution to the score should be.

[0067] The weight values ​​(w1, w2, w3, w4) are derived through a multi-task learning (MTL) architecture, based on massive amounts of historical exposure data and real business results such as click-through rate, through reverse training. This ensures that the EAS score is not only an "attractiveness score," but also a highly reliable "predictive conversion contribution score."

[0068] The training implementation process of the MTL architecture specifically includes: Shared Representation Layer: First, a shared, fully connected neural network (DNN) is constructed, for example, containing two hidden layers, each with 64 units, using the ReLU activation function. This network receives the four metrics (total gaze duration, reciprocal of first-time detection speed, number of high-cost activations, and gaze micro-motion amplitude) as input, and learns a combined high-dimensional feature representation.

[0069] Above the shared representation layer, the system constructs multiple independent output layers in parallel.

[0070] Multi-task Header: Above the shared representation layer, multiple independent output layers (task heads) are constructed in parallel: 1. Main Task Header: A logistic regression layer that predicts the click-through rate (CTR) corresponding to ad impressions based on the shared representation. The training labels for this task are binary classification labels (0 = no click, 1 = click), using the binary cross-entropy loss function. 2. Auxiliary Task Header: An additional logistic regression layer that predicts the probability of sparse signal occurrence based on the same shared representation.

[0071] In the tens of millions of historical exposures of future cars, high-cost activations (AEs) and gaze micro-motion amplitudes (MMTs) are extremely sparse events, with an occurrence probability of less than 0.1%. If the model only uses the main task head to predict CTR, it is likely to ignore these sparse signal features to minimize the overall loss, as their contribution to the overall CTR prediction is statistically insignificant. However, once these sparse signals, such as a user picking up the remote control, occur, their correlation with the final conversion is extremely high. Therefore, the purpose of setting the auxiliary task head is to force the model to learn these sparse signals. Sparse signals are processed as binary labels during training (0 = no high-cost activation occurred, 1 = at least one high-cost activation occurred). This task also uses the binary cross-entropy loss function.

[0072] The system trains the entire model by jointly optimizing the loss function of all task heads. This mechanism forces the shared representation layer to learn a general high-dimensional feature that can be used to predict both CTR and the occurrence of sparse signals such as high-cost activations. Once the entire model training converges, the system uses only the weights learned by the main task head as the weight values ​​(w1, w2, w3, w4) for the final Elementary Attention Score (EAS) calculation. For example, in the training data for future cars, the model finds that the number of high-cost activations (AE) contributes the most to the prediction of CTR (i.e., weight w3), while the weight w2 for the first-time detection speed (TFF_j) is relatively low.

[0073] This embodiment obtains multi-dimensional indicators and uses an MTL architecture with real business response (CTR) as the main task and sparse signals as the auxiliary task to calculate objective and reliable element attention scores, providing reliable data support for subsequent "creative optimization instructions".

[0074] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An OTT advertisement optimization system based on a remote controller pointing behavior, characterized by, Comprise: Data acquisition module: obtain the original trajectory data stream collected by the directional remote controller; Trajectory processing is adopted on the original trajectory data stream to extract dormant state data, gaze state trajectory data and cruise state trajectory data and generate user intention trajectory data; Semantic analysis module: obtain advertisement visual content data, perform semantic understanding on the advertisement visual content through first frame analysis tracking, identify advertisement creative elements, and generate advertisement semantic element map data containing advertisement creative elements and corresponding position coordinates; Instruction generation module: associate and map user intention trajectory data and advertisement semantic element map data in spatial and temporal dimensions; based on the association and mapping, calculate the attention index of the advertisement creative elements, take the predicted advertisement click rate as the main task, predict the sparse signal occurrence probability in the attention index as the auxiliary task, jointly optimize the main task and the auxiliary task and extract the weight learned by the main task, combine the weight and the attention index to generate element attention score, generate creative optimization instructions based on the element attention score, and automatically adjust the layout of the advertisement creative elements.

2. The OTT advertisement optimization system based on a remote controller pointing behavior according to claim 1, wherein, The trajectory processing specifically comprises: receiving the original trajectory data stream, determining cruise state trajectory data when the remote controller pointer speed is higher than a preset speed threshold, obtaining a set of coordinate points of the original trajectory data stream within a preset time window when the pointer speed is not higher than the preset speed threshold, determining gaze state trajectory data when the spatial variance of the coordinate points in spatial distribution is less than a clustering threshold, determining dithering state data when the spatial variance of the coordinate points in spatial distribution is greater than the clustering threshold, extracting the spatial variance of the gaze state trajectory data as micro-intention data, and applying a Kalman filter algorithm to smooth the identified gaze state and cruise state data to generate user intention trajectory data; The dormant state indicates that the remote controller is stationary and has no movement within a preset time period; the dithering state indicates unconscious dithering noise data of the user's hand; the cruise state indicates that the remote controller pointer moves from point A to point B; and the gaze state indicates that the user gazes at a certain advertisement element.

3. The OTT advertisement optimization system based on a remote controller pointing behavior according to claim 1, wherein, The creative element types contained in the advertisement semantic element map data include: advertisement subject: frame out the subject element in the advertisement material and output its category and bounding box coordinates; promotion text: extract advertisement text information; spokesperson: identify the "person" element in the advertisement, compare with a preset "star material library" and automatically label; brand logo: including trademark identification and "action call button" identification in the advertisement.

4. The OTT advertisement optimization system based on a remote controller pointing behavior according to claim 1, wherein, When the advertisement visual content data is a video, first frame analysis tracking is adopted, and the specific implementation steps comprise: Divide the video into shots through a scene boundary detection algorithm; identify all initial advertisement creative elements in the first key frame of each shot; initialize a target tracker for each initial advertisement creative element; in the subsequent frames of the shot, run the target tracker, predict and update the bounding box position of the continuously changing elements, and periodically re-identify the advertisement creative elements to calibrate the tracker position deviation and discover new advertisement creative elements.

5. The OTT advertisement optimization system based on a remote controller pointing behavior according to claim 1, wherein, The specific implementation of the correlation mapping of the spatial and time dimensions includes: traversing each gaze state event in the user intention trajectory data, and obtaining the timestamp and coordinates thereof; time dimension mapping: querying the advertisement semantic element map data using the timestamp to retrieve the advertisement creative element within the corresponding time segment as a candidate element; spatial dimension mapping: obtaining the position coordinates of the candidate element corresponding to the timestamp; correlation determination: when the coordinates of the gaze state event fall within the boundary box defined by the position coordinates of the candidate element, attributing the attention contribution of the gaze state event to the advertisement creative element; The correlation determination step further includes: before checking whether the coordinates fall within the boundary box, the system first dynamically generates a gravity range for the position coordinates of the candidate element; the gravity range is an original boundary box expanded by a preset pixel value on each side; the final correlation determination checks whether the coordinates of the gaze state event fall within the expanded gravity range.

6. The OTT advertisement optimization system based on a remote controller pointing behavior according to claim 1, wherein, The calculation of the element attention score obtains the following attention indicators for the advertisement creative element: total gaze duration: accumulates the total millisecond number of gaze events associated with the element, reflecting the time consumption of the user on the element; first discovery speed: records the millisecond number spent from the start of the advertisement to the first gaze on the element, reflecting the eye-catching degree of the element; high-cost activation number: counts the number of events in which the user picks up the remote control to point to the element from the first time the remote control is in the dormant state, reflecting the charm degree of the element; gaze micro-motion amplitude: counts the micro-intention data on the element, reflecting the stability and concentration of the user's gaze.

7. The OTT advertisement optimization system based on a remote controller pointing behavior according to claim 1, wherein, The implementation process of the element attention score calculation includes: applying an independent weight value to each of the total gaze duration, the first discovery speed, the high-cost activation number, and the inverse of the gaze micro-motion amplitude for weighted combination; Each weight value is optimized through a multi-task learning architecture, including: shared representation layer: a shared bottom layer network is constructed, which receives the four indicators as input and learns the combined high-dimensional feature representation; multi-task head: a plurality of independent output layers are constructed in parallel on the shared representation layer, wherein: main task head: based on the shared representation, the click rate corresponding to the advertisement exposure is predicted through a logistic regression layer; auxiliary task head: an additional logistic regression layer based on the same shared representation predicts the occurrence probability of a sparse signal; the sparse signal includes the high-cost activation number and the gaze micro-motion amplitude; the loss functions of all task heads are optimized and the entire model is trained; the weight value output by the main task head is the contribution degree of each indicator to the advertisement click rate.

8. The OTT advertisement optimization system based on a remote controller pointing behavior according to claim 1, wherein, The creative optimization instruction specifically comprises: an element level selection instruction: based on the element attention score, automatically selecting the highest scoring element among multiple alternative advertising creative elements for combination publishing; a layout dynamic adjustment instruction: based on the score ranking of the element attention score and historical intention trajectory heat map data, automatically adjusting key advertising creative elements to an area determined by the historical intention trajectory heat map data, in which the user attention hovering frequency is higher than a preset frequency threshold; an automated A / B test instruction: automatically generating advertising visual content optimized based on the element attention score, and automatically performing traffic A / B test with the original advertising visual content.

Images