An advertising instruction system and method

By collecting human-computer interaction information to generate a combined visual and auditory attention map, and adjusting the playback parameters of audiovisual advertisements, the problem of traditional advertisements being unable to be personalized is solved, thus improving the communication effect of advertisements.

CN120013605BActive Publication Date: 2026-11-17GUANGZHOU ZHUOBANG ADVERTISING PROD CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202510111573.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2025-01-23
Publication Date
2026-11-17
Estimated Expiration
2045-01-23

AI Technical Summary

Technical Problem

Traditional audiovisual advertising cannot be personalized based on the audience's immediate feedback, resulting in visual and auditory elements failing to effectively attract the audience's attention and affecting advertising effectiveness.

Method used

By collecting human-computer interaction information, extracting audience dwell characteristics, determining visual interest areas and auditory salience, generating visual attention maps and auditory salience maps, and combining audiovisual elements to adjust advertising playback parameters.

Benefits of technology

It improved the perception and appeal of audiovisual advertisements, optimized the presentation of visual and audio content, and enhanced the communication effect of advertisements.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120013605B_ABST
    Figure CN120013605B_ABST
Patent Text Reader

Abstract

This application provides an advertising instruction system and method. It extracts audience dwell characteristics from human-computer interaction information within the playback environment of a smart terminal, and uses these dwell characteristics to filter multiple advertising interest points from audiovisual advertisements. It then extracts all visual interest regions of the audience during the playback of the audiovisual advertisement on the smart terminal using the dwell characteristics of all audiences within each advertising interest point, determines the audience attention level for each visual interest region, and determines the visual attention map of the audience in the playback environment based on all audience attention levels. It also determines the auditory saliency of each advertising interest point, generates an auditory saliency map of the audiovisual advertisement during playback using all auditory saliencies, and combines the visual attention map and auditory saliency map to obtain the fused interactive features of the audiovisual advertisement during playback. These fused interactive features are then used to adjust the playback parameters of the audiovisual advertisement. Based on the above scheme, audiovisual joint attention adjustment during audiovisual advertisement playback can be achieved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of audiovisual advertising technology, and more specifically, to an advertising instruction system and method. Background Technology

[0002] Audiovisual advertising, using video and sound as the primary means of disseminating advertising information, is characterized by its highly concentrated and intuitive communication, making it a modern and eye-catching form of advertising. With the advent of the digital age, the production techniques of audiovisual advertising art have become more diversified, and its forms of expression have become richer. The development of digital technology has allowed audiovisual advertising to be freed from the limitations of actual shooting space, expanding the creative space of creators. Through technologies such as animation, special effects, and digital music, it has achieved higher artistry and stronger visual appeal.

[0003] In traditional advertising formats, visual and auditory elements are typically static and fixed, unable to be personalized based on immediate audience feedback. Audiences exhibit significant differences in the distribution of their visual and auditory attention when watching ads, potentially focusing on specific parts while ignoring others. This overlooks the differences in audience attention across different time and space dimensions, resulting in some ad content failing to effectively capture attention. Consequently, important information may be played when audience attention is low, leading to poor information delivery and significantly reduced advertising effectiveness. Therefore, achieving combined visual and auditory attention regulation in audiovisual advertising has become a challenging problem for the industry. Summary of the Invention

[0004] This application provides an advertising instruction system and method that can achieve combined audiovisual attention adjustment during the playback of audiovisual advertisements.

[0005] Firstly, this application provides a method for interactive playback of audiovisual advertisements, including:

[0006] Collect human-computer interaction information between smart terminals and audiences when audiovisual advertisements are played, extract audience dwell characteristics in the playback environment of the smart terminal from the human-computer interaction information, and filter out multiple advertising interest points from the audiovisual advertisements through the dwell characteristics;

[0007] By extracting the dwell characteristics of all audiences in each advertising interest point, all visual interest areas of the audience are extracted when the audiovisual advertisement is played on the smart terminal, and then the audience attention of each visual interest area is determined. Based on the audience attention, the visual attention map of the audience in the playback environment of the smart terminal is determined.

[0008] The auditory saliency of each advertising interest point is determined based on the self-attention mechanism, and then an auditory saliency map is generated when the audiovisual advertisement is played on the smart terminal by using all the auditory saliency.

[0009] The visual attention map and the auditory salience map are combined to obtain the fused interactive features of the audiovisual advertisement when it is played on a smart terminal. The playback parameters of the audiovisual advertisement are then adjusted using the fused interactive features.

[0010] In some embodiments, extracting the audience dwell characteristics in the playback environment of the smart terminal from the human-computer interaction information specifically includes:

[0011] Filter all audience users from all users in the playback environment of the smart terminal;

[0012] Extract the dwell time and dwell area of ​​each audience user from the human-computer interaction information;

[0013] The dwell characteristics of the audience in the playback environment of the smart terminal are determined by all dwell time periods and all dwell areas.

[0014] In some embodiments, filtering multiple advertising interest points from audiovisual advertisements using the dwell characteristics specifically includes:

[0015] Retrieve all significant playback nodes in audiovisual advertisements;

[0016] The dwell time period in the dwell feature is used to determine the dwell value of each significant playback node;

[0017] Multiple advertising interest points are selected from audiovisual ads based on various dwell times.

[0018] In some embodiments, extracting all visual interest regions of the audience when the audiovisual advertisement is played on a smart terminal by using the dwell characteristics of all audiences in each advertising interest point specifically includes:

[0019] Obtain the dwell area from the dwell characteristics of all audiences in each ad interest point;

[0020] Identify all areas of visual interest for the audience when audiovisual advertisements are played on smart terminals by identifying all the areas where the advertisements reside.

[0021] In some embodiments, determining the audience attention of each visual interest region specifically includes:

[0022] For each visual interest region, the dwell time, dwell frequency, and visual field density of the visual interest region are extracted from the human-computer interaction information.

[0023] The audience attention of visual interest regions is determined by the dwell time, dwell frequency, and visual field density, thereby obtaining the audience attention of each visual interest region.

[0024] In some embodiments, determining the auditory salience of each advertising interest point based on a self-attention mechanism specifically includes:

[0025] For each ad interest point, obtain the dwell time period of the ad interest point;

[0026] Extract the audio signal of the audiovisual advertisement within the specified dwell time period;

[0027] The saliency features of the audio signal are extracted and quantified based on the self-attention mechanism to obtain the auditory saliency of the advertising interest points, and then the auditory saliency of each advertising interest point is obtained.

[0028] In some embodiments, the smart terminal is a human-computer interaction device.

[0029] Secondly, this application provides an advertising instruction system, including a playback interaction unit, the playback interaction unit comprising:

[0030] The data collection module is used to collect human-computer interaction information between the smart terminal and the audience when the audiovisual advertisement is played, extract the audience's dwell characteristics in the playback environment of the smart terminal from the human-computer interaction information, and filter out multiple advertising interest points from the audiovisual advertisement through the dwell characteristics.

[0031] The processing module is used to extract all visual interest regions of the audience when the audiovisual advertisement is played on the smart terminal by using the dwell characteristics of all audiences in each advertising interest point, thereby determining the audience attention of each visual interest region, and determining the visual attention map of the audience in the playback environment of the smart terminal based on all audience attention.

[0032] The processing module is also used to determine the auditory saliency of each advertising interest point based on the self-attention mechanism, and then generate an auditory saliency map of the audiovisual advertisement when it is played on the smart terminal through all the auditory saliency.

[0033] The execution module is used to combine the visual attention map and the auditory salience map to obtain the fused interaction features when the audiovisual advertisement is played on the smart terminal, and to adjust the playback parameters in the audiovisual advertisement using the fused interaction features.

[0034] Thirdly, this application provides a computer device, the computer device including a memory and a processor, the memory for storing a computer program, and the processor for calling and running the computer program from the memory, so that the computer device performs the above-described interactive method for playing audiovisual advertisements.

[0035] Fourthly, this application provides a computer-readable storage medium storing instructions or code that, when executed on a computer, cause the computer to implement the aforementioned interactive method for playing audiovisual advertisements.

[0036] The technical solutions provided by the embodiments disclosed in this application have the following beneficial effects:

[0037] This application provides an advertising indication system and method, which collects human-computer interaction information between a smart terminal and the audience during the playback of audiovisual advertisements. The system extracts the audience's dwell characteristics in the playback environment of the smart terminal from the human-computer interaction information, and uses these dwell characteristics to filter out multiple advertising interest points from the audiovisual advertisements. It then extracts all visual interest regions of the audience during the playback of the audiovisual advertisements on the smart terminal using the dwell characteristics of all audiences in each advertising interest point, thereby determining the audience attention level of each visual interest region. Based on all audience attention levels, it determines the visual attention map of the audience in the playback environment of the smart terminal. Based on a self-attention mechanism, it determines the auditory saliency of each advertising interest point, and then generates an auditory saliency map of the audiovisual advertisements during playback on the smart terminal using all auditory saliencies. Finally, it combines the visual attention map and the auditory saliency map to obtain fused interactive features of the audiovisual advertisements during playback on the smart terminal, and uses these fused interactive features to adjust the playback parameters of the audiovisual advertisements.

[0038] Therefore, this application combines visual attention maps and auditory saliency maps to obtain the fused interactive features of audiovisual advertisements played on smart terminals. These fused interactive features are then used to adjust playback parameters in the audiovisual advertisements. First, determining the visual attention map yields a saliency distribution graph of the audience's attention areas during the audiovisual advertisement playback. This allows understanding how the audience allocates their attention during playback and which visual elements require more emphasis. Leveraging the advantages of self-attention mechanisms, the visual attention map can dynamically adjust based on specific changes in audience attention. This dynamic adjustment of visual content not only enhances the audience's perception of the audiovisual advertisement but also strengthens the display of the audience's visual focus points, thereby optimizing the audiovisual advertisement's effect. Furthermore, in subsequent audiovisual joint attention adjustment, the visual attention map can guide the adjustment of the audiovisual advertisement content based on the audience's attention characteristics, increasing the attractiveness of the audiovisual advertisement. This approach reduces the visual neglect area of ​​the audience, thereby improving the communication effect of audiovisual advertising. Then, by determining the auditory saliency map, the spatial distribution and temporal characteristics of the audience's saliency response to audio content during audiovisual advertising playback can be obtained. This helps identify key points of the audio content in audiovisual advertising, such as particularly attention-grabbing voice prompts, sound effects, or musical segments. Simultaneously, it provides more comprehensive audience attention feedback for audiovisual advertising, facilitating the simultaneous optimization of audio and visual parameters. By integrating the auditory saliency map into audiovisual joint attention regulation, the volume, sound effects, or voice presentation can be adjusted according to the audience's immediate audio attention during audiovisual advertising playback, ensuring the optimal presentation of key audio information in the audiovisual advertising. This further enhances the persuasiveness and attractiveness of audiovisual advertising, improving audience participation and overall advertising effectiveness. In summary, the above solution enables audiovisual joint attention regulation during audiovisual advertising playback. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is an exemplary flowchart of an interactive method for playing audiovisual advertisements according to some embodiments of this application;

[0041] Figure 2 This is a basic flowchart of audiovisual advertising delivery shown in some embodiments of this application;

[0042] Figure 3 This is a flowchart illustrating the process of determining fusion interaction features according to some embodiments of this application;

[0043] Figure 4 This is a schematic diagram of the structure of the playback interaction unit shown in some embodiments of this application;

[0044] Figure 5 This is a schematic diagram of the structure of a computer device for implementing an interactive method for playing audiovisual advertisements according to some embodiments of this application. Detailed Implementation

[0045] To better understand the technical solution of this application, the technical solution of this application will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0046] refer to Figure 1 The figure is an exemplary flowchart of an interactive method for playing audiovisual advertisements according to some embodiments of this application. The interactive method for playing audiovisual advertisements mainly includes the following steps:

[0047] In step 101, human-computer interaction information between the smart terminal and the audience is collected when the audiovisual advertisement is played. The audience's dwell characteristics in the playback environment of the smart terminal are extracted from the human-computer interaction information. Multiple advertising interest points are selected from the audiovisual advertisement through the dwell characteristics.

[0048] It should be noted that, in this application, the audiovisual advertisement is an intelligent advertisement based on a multimodal attention mechanism; human-computer interaction information is information used to describe the interaction process between the user and the intelligent terminal, and the intelligent terminal is a human-computer interaction device; in specific implementation, the monitoring system in the intelligent terminal can be used to monitor the time period, gaze area and click area of ​​each user operation when the audiovisual advertisement is played on the intelligent terminal within a specified time period (default is the most recent week) as the user's interaction behavior, and the set of all interaction behaviors can be used as the human-computer interaction information between the intelligent terminal and the audience when the audiovisual advertisement is played.

[0049] In some embodiments, reference Figure 2 The diagram described herein is a basic flowchart of audiovisual advertising placement according to some embodiments of this application. The diagram illustrates the basic structure of the advertising placement process. First, the advertiser (i.e., a company or individual wishing to promote their products or services) establishes a cooperative relationship with an advertising agency. The advertiser entrusts the advertising agency to handle the advertising, and the advertising agency is responsible for publishing the advertising content on appropriate media. In this process, the media acts as advertising space providers, possessing advertising spaces that the audience is likely to notice. These advertising spaces can be television, radio, newspapers, magazines, outdoor billboards, or online platforms, etc.

[0050] After an advertisement is published, the audience "consumes" it through the media. The audience are the recipients of the advertising information; they learn about the advertiser's products or services by watching, listening to, or reading the advertisement. The effectiveness of an advertisement depends on several factors, including the appeal of the advertising content, the reach of the media, audience preferences, and the timing of the ad placement.

[0051] The diagram also reflects the different roles involved in the advertising process and their relationships. Advertisers are the initiators of advertising campaigns, advertising agencies are the bridge connecting advertisers and media, and media are the channels through which advertising information is delivered to the audience. Audience reactions and behaviors are key indicators for measuring the success of advertising. This structured process helps ensure that advertising information can be effectively delivered from advertisers to the target audience, thereby achieving the purpose of advertising.

[0052] In some embodiments, extracting the audience dwell characteristics in the playback environment of the smart terminal from the human-computer interaction information can be achieved through the following steps:

[0053] Filter all audience users from all users in the playback environment of the smart terminal;

[0054] Extract the dwell time and dwell area of ​​each audience user from the human-computer interaction information;

[0055] The dwell characteristics of the audience in the playback environment of the smart terminal are determined by all dwell time periods and all dwell areas.

[0056] In specific implementation, firstly, selecting all audience users from all users in the playback environment of the smart terminal can be achieved as follows: obtain all users in the playback environment of the smart terminal from the monitoring system of the smart terminal. All users involved in the human-computer interaction information can be considered as audience users, thus obtaining all audience users. Secondly, extracting the dwell time period and dwell area of ​​each audience user from the human-computer interaction information can be achieved as follows: for each audience user, select all interactive behaviors of the audience user from the human-computer interaction information. Then, take the set of time periods in each interactive behavior as the dwell time period of the audience user, and take the set of clicked areas in each interactive behavior as the dwell area of ​​the audience user. The dwell time period and dwell area of ​​each audience user can be obtained through the above method. Finally, determining the dwell characteristics of the audience in the playback environment of the smart terminal through all dwell time periods and all dwell areas can be achieved as follows: take the set of all dwell time periods as dwell time characteristics, and take the set of all dwell areas as dwell space characteristics. The set of dwell time characteristics and dwell space characteristics can be taken as the dwell characteristics of the audience in the playback environment of the smart terminal.

[0057] It should be noted that, in this application, dwell characteristics refer to the characteristics of audience dwelling behavior during the playback of audiovisual advertisements; audience users refer to the target group that watches and listens to audiovisual advertisement content; dwell time period refers to the time interval during which the audience concentrates on a certain point of interest in the advertisement during the playback of audiovisual advertisements; dwell area refers to the specific area that the audience focuses on during the playback of audiovisual advertisements.

[0058] In some embodiments, filtering multiple advertising interest points from audiovisual advertisements using the dwell characteristics can be achieved through the following steps:

[0059] Retrieve all significant playback nodes in audiovisual advertisements;

[0060] The dwell time period in the dwell feature is used to determine the dwell value of each significant playback node;

[0061] Multiple advertising interest points are selected from audiovisual ads based on various dwell times.

[0062] It should be noted that, in this application, "advertising interest points" refer to elements in the advertising content that are highly attractive to the audience. Specifically, firstly, obtaining all prominent playback nodes in the audiovisual advertisement can be achieved by: obtaining all prominent playback nodes in the audiovisual advertisement from the advertisement design specification; these prominent playback nodes refer to the time points in the audiovisual advertisement used to highlight the advertising content. Then, determining the dwell value of each prominent playback node through the dwell time period in the dwell features can be achieved by: for prominent playback nodes, filtering the prominent playback nodes from the dwell features... For each prominent node, the average duration of all dwell time periods can be used as the dwell value of the prominent node. The dwell value of each prominent node can be obtained through the above method. This dwell value is a quantitative value used to measure the intensity of audience attention to the ad interest point. Finally, multiple ad interest points can be selected from the audiovisual ads based on each dwell value. The average of all dwell values ​​can be used as the dwell threshold of the audiovisual ad. Thus, the prominent nodes corresponding to each dwell value higher than the dwell threshold can be used as ad interest points, thereby obtaining multiple ad interest points.

[0063] In step 102, all visual interest regions of the audience are extracted when the audiovisual advertisement is played on the smart terminal by using the dwell characteristics of all audiences in each advertising interest point, thereby determining the audience attention of each visual interest region, and determining the visual attention map of the audience in the playback environment of the smart terminal based on all audience attention.

[0064] In some embodiments, extracting all visual interest regions of the audience when an audiovisual advertisement is played on a smart terminal by using the dwell characteristics of all audiences at each advertising point of interest can be achieved through the following steps:

[0065] Obtain the dwell area from the dwell characteristics of all audiences in each ad interest point;

[0066] Identify all areas of visual interest for the audience when audiovisual advertisements are played on smart terminals by identifying all the areas where the advertisements reside.

[0067] It should be noted that, in this application, the visual interest region refers to the area of ​​the screen that the audience pays attention to during the playback of the audiovisual advertisement. In specific implementation, firstly, the dwell region in the dwell characteristics of all audiences in each advertisement interest point is obtained; then, the visual interest regions of the audience when the audiovisual advertisement is played on the smart terminal can be determined by the following method: all dwell regions can be deduplicated and used as the visual interest regions when the audiovisual advertisement is played on the smart terminal, thus obtaining all the visual interest regions of the audience when the audiovisual advertisement is played on the smart terminal.

[0068] In some embodiments, determining the audience attention level for each visual interest region can be achieved through the following steps:

[0069] For each visual interest region, the dwell time, dwell frequency, and visual field density of the visual interest region are extracted from the human-computer interaction information.

[0070] The audience attention of visual interest regions is determined by the dwell time, dwell frequency, and visual field density, thereby obtaining the audience attention of each visual interest region.

[0071] It should be noted that, in this application, audience attention refers to the intensity of interest and attention that an audience generates towards specific advertising content during the playback of audiovisual advertisements. Specifically, for each visual interest region, the dwell time, dwell frequency, and visual field density of the visual interest region can be extracted from the human-computer interaction information in the following manner: For each visual interest region, all time periods of user operation within the visual interest region are obtained from the human-computer interaction information. The total duration of all time periods can be used as the dwell time of the visual interest region, representing the cumulative time the audience spends at the advertising point of interest. The number of all time periods can be used as the dwell frequency of the visual interest region, representing the number of times the audience pays attention to the advertising point of interest. Simultaneously, a self-attention mechanism is used to analyze the visual characteristics within the visual interest region. The density of features is used as visual field density, which represents the density of the audience's eye movement trajectory in a specific area. Areas with higher density are often the focus of the audience's attention, helping to accurately locate the advertising focus. Then, the audience attention of the visual interest area is determined by the dwell time, dwell frequency, and visual field density. The audience attention of each visual interest area can be obtained in the following way: a multivariate regression model is constructed to combine dwell time, dwell frequency, and visual field density to calculate the audience attention of the visual interest area, that is, audience attention = a*dwell time + b*dwell frequency + c*visual field density + d, where a, b, c, and d are the parameters of the multivariate regression model, which are obtained by training with historical running data. The audience attention of each visual interest area can be obtained in the above way.

[0072] In some embodiments, determining the visual attention map of the audience in the playback environment of the smart terminal based on all audience attention can be achieved by the following steps:

[0073] By merging all visual interest regions into adjacent regions, multiple merged regions for the audiovisual advertisement are obtained.

[0074] Determine the visual attention intensity of each merged area by considering all audience attention levels.

[0075] By mapping all visual attention intensity onto the full-screen image, a visual attention map of the audience in the playback environment of the smart terminal is obtained.

[0076] In specific implementation, firstly, all visual interest regions are merged into adjacent regions to obtain multiple merged regions for the audiovisual advertisement. This can be achieved by merging adjacent visual interest regions into one merged region. Secondly, the visual attention intensity of each merged region is determined by the audience attention levels. This can be achieved by obtaining the audience attention levels of each visual interest region within each merged region, and then using the average of all audience attention levels as the visual attention intensity of the merged region. Finally, all visual attention intensities are mapped onto a full-screen image to obtain the visual attention map of the audience in the playback environment of the smart terminal. This can be achieved by initializing an empty full-screen image, dividing the full-screen image according to the position coordinates of the merged regions, and then mapping each visual attention intensity onto the full-screen image. The mapped full-screen image can then be used as the visual attention map of the audience in the playback environment of the smart terminal.

[0077] It should be noted that, in this application, the visual attention map represents the salience distribution of the area of ​​visual content that the audience pays attention to during the playback of audiovisual advertisements; the merged region is used to describe the comprehensive area where visual interests are close together in the visual attention map; and the visual attention intensity represents a numerical indicator of the degree of attention that the audience pays to a specific area in the visual content.

[0078] In step 103, the auditory saliency of each advertising interest point is determined based on the self-attention mechanism, and then an auditory saliency map is generated when the audiovisual advertisement is played on the smart terminal using all the auditory saliency.

[0079] In some embodiments, determining the auditory salience of each advertising interest point based on a self-attention mechanism can be achieved through the following steps:

[0080] For each ad interest point, obtain the dwell time period of the ad interest point;

[0081] Extract the audio signal of the audiovisual advertisement within the specified dwell time period;

[0082] The saliency features of the audio signal are extracted and quantified based on the self-attention mechanism to obtain the auditory saliency of the advertising interest points, and then the auditory saliency of each advertising interest point is obtained.

[0083] It should be noted that, in this application, auditory saliency represents the intensity of the audio signal attracting the audience's attention during the playback of an audiovisual advertisement. Specifically, firstly, for each advertising point of interest, the dwell time period of that advertising point of interest is obtained. Then, the audio signal of the audiovisual advertisement within the dwell time period can be extracted in the following way: a video segment of the audiovisual advertisement played within the dwell time period is obtained, and the audio signal in that video segment can be used as the audio signal of the audiovisual advertisement within the dwell time period. Finally, the saliency features of the audio signal are extracted and quantified based on a self-attention mechanism to obtain the auditory saliency of the advertising point of interest. The auditory saliency of each advertising point of interest can then be obtained in the following way: a feature extraction algorithm based on a self-attention mechanism (e.g., the Transformer model) is used to extract the temporal features of the audio signal as saliency features, and a normalization algorithm is used to normalize these saliency features. The result of the normalized saliency features can be used as the auditory saliency of the advertising point of interest. The auditory saliency of each advertising point of interest can be obtained through the above methods.

[0084] In some embodiments, generating an auditory saliency map of an audiovisual advertisement played on a smart terminal using all auditory saliency can be achieved through the following steps:

[0085] All auditory saliences are integrated chronologically according to the dwell time of each advertising interest point to generate a salience time series of audiovisual advertisements;

[0086] By constructing a multi-level saliency map of the audiovisual advertisement using the saliency time series, an auditory saliency map of the audiovisual advertisement when it is played on a smart terminal is obtained.

[0087] In specific implementation, firstly, all auditory saliency is integrated temporally according to the dwell time of each advertising interest point to generate the saliency time series of the audiovisual advertisement. This can be achieved by: using the proportion of the dwell time of each auditory saliency corresponding to the advertising interest point to the total duration of the audiovisual advertisement as a weight, calculating the weighted value of each auditory saliency, and then sorting each weighted value in chronological order according to the midpoint of the dwell time of the corresponding advertising interest point to obtain the saliency time series covering the entire playback process; then, the saliency map of the audiovisual advertisement is constructed in multiple layers using the saliency time series to obtain the auditory saliency map of the audiovisual advertisement when played on the smart terminal. This can be achieved by: initializing a graph theory model based on a saliency neural network, using the saliency time series as the time weight nodes in the graph theory model, and using the graph theory model to construct the saliency map of the audiovisual advertisement in multiple layers. The result of the multi-layer construction of the graph theory model can then be used as the auditory saliency map of the audiovisual advertisement when played on the smart terminal.

[0088] It should be noted that in this application, the auditory saliency map represents the spatial distribution and temporal characteristics of the audience's saliency response to audio content during audiovisual advertising playback; the saliency time series represents the dynamic characteristic sequence of auditory saliency changes over time during audiovisual advertising playback; the graph theory model is a model based on a saliency neural network. This graph theory model constructs a dynamically changing graph structure by using the saliency time series as time weight nodes. Each node represents the intensity of auditory saliency during audiovisual advertising playback within a specific time period. The edges in the graph represent the temporal correlation between nodes. A self-attention mechanism is used to dynamically adjust the weights between nodes, thereby achieving multi-level information transmission and aggregation. Through the multi-level construction of the graph theory model, the temporal evolution of auditory saliency features during audiovisual advertising playback can be accurately captured, thereby generating a complete auditory saliency map that reflects the changes in the audience's attention to the audio content of audiovisual advertising.

[0089] In step 104, the visual attention map and the auditory salience map are combined to obtain the fused interactive features of the audiovisual advertisement when it is played on the smart terminal, and the playback parameters of the audiovisual advertisement are adjusted using the fused interactive features.

[0090] In some embodiments, the visual attention map and the auditory saliency map are combined audiovisually to obtain the fused interactive features of the audiovisual advertisement when it is played on a smart terminal, with reference to... Figure 3 The diagram is a flowchart illustrating the process of determining fusion interaction features in some embodiments of this application. In this embodiment, determining fusion interaction features can be achieved through the following steps:

[0091] In step 1041, the visual attention map and the auditory saliency map are weighted and fused to obtain a fused feature map;

[0092] In step 1042, the interactive features in the fused feature map are extracted as the fused interactive features when the audiovisual advertisement is played on the smart terminal.

[0093] It should be noted that, in this application, the fused interaction feature represents the dynamic distribution of audience attention and response characteristics under the comprehensive influence of multimodal information in audiovisual advertising. Specifically, the visual attention map and the auditory saliency map are first weighted and fused to obtain the fused feature map. This can be achieved by obtaining the actual contribution ratio of visual and auditory elements to audience attraction from the advertising design specification of the audiovisual advertisement as weights. A weighted feature fusion algorithm (e.g., a dynamic weight allocation algorithm based on attention mechanisms) can then be used to fuse the spatial distribution characteristics of the visual attention map with the temporal variation characteristics of the auditory saliency map. The fused feature map can then be used as the fused feature map. The fusion feature map is used to describe the combined influence of visual attention and auditory saliency on audience attention when advertising content is played on a smart terminal. Then, the interaction features extracted from the fusion feature map as the fusion interaction features when audiovisual advertisements are played on a smart terminal can be implemented in the following way: the spatiotemporal correlation analysis algorithm can be used to extract and identify the multimodal attention changes of the audience during the playback of audiovisual advertisements from the fusion feature map as interaction features. These interaction features characterize the audience's comprehensive response to the audiovisual advertisement content, so these interaction features can be used as the fusion interaction features when audiovisual advertisements are played on a smart terminal, providing support for subsequent playback optimization and advertising strategies.

[0094] In some embodiments, adjusting playback parameters in audiovisual advertisements using the fused interactive features can be achieved in the following ways: by analyzing the visual and auditory attention distribution in the fused interactive features, identifying differences in audience attention to different content, and dynamically adjusting the advertisement playback parameters, for example: image parameter adjustment: adjusting image brightness, contrast, and color saturation according to the distribution of visual attention to highlight content in high-attention areas while appropriately weakening low-attention areas; audio parameter adjustment: optimizing audio volume, frequency enhancement, and sound positioning in conjunction with changes in auditory saliency features to make key sound effects or speech more prominent; playback sequence optimization: optimizing playback speed or content switching rhythm based on temporal saliency in the interactive features to ensure that the audience can fully understand the content in high-saliency areas; multimodal synchronization adjustment: optimizing the temporal synchronization of audiovisual content through collaborative analysis of visual and auditory features, for example, aligning the switching points of audio and video to improve sensory consistency; interactive response enhancement: adjusting interactive prompts at specific moments in conjunction with interactive features, such as pop-up prompt boxes or adding click hotspots to enhance user engagement.

[0095] Furthermore, in another aspect of this application, in some embodiments, this application provides an advertising instruction system, which includes a playback interaction unit, referencing... Figure 4The figure is a schematic diagram of the structure of a playback interaction unit according to some embodiments of this application. The playback interaction unit includes: a collection module 201, a processing module 202, and an execution module 203, which are described below:

[0096] The acquisition module 201 in this application is mainly used to collect human-computer interaction information between the smart terminal and the audience when the audiovisual advertisement is played, extract the audience's dwell characteristics in the playback environment of the smart terminal from the human-computer interaction information, and filter out multiple advertising interest points from the audiovisual advertisement through the dwell characteristics.

[0097] Processing module 202 in this application is used to extract all visual interest regions of the audience when the audiovisual advertisement is played on the smart terminal by using the dwell characteristics of all audiences in each advertising interest point, thereby determining the audience attention of each visual interest region, and determining the visual attention map of the audience in the playback environment of the smart terminal based on all audience attention.

[0098] It should be noted that the processing module 202 is also used to determine the auditory saliency of each advertising interest point based on the self-attention mechanism, and then generate an auditory saliency map of the audiovisual advertisement when it is played on the smart terminal through all the auditory saliency.

[0099] The execution module 203 in this application is mainly used to combine the visual attention map and the auditory salience map to obtain the fused interactive features when the audiovisual advertisement is played in the smart terminal, and to adjust the playback parameters in the audiovisual advertisement using the fused interactive features.

[0100] The foregoing has detailed examples of the advertising instruction system and method provided in the embodiments of this application. It is understood that the corresponding apparatus, in order to achieve the above functions, includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0101] In some embodiments, this application also provides a computer device, the computer device including a memory and a processor, the memory for storing a computer program, and the processor for calling and running the computer program from the memory, so that the computer device performs the above-described interactive method for playing audiovisual advertisements.

[0102] In some embodiments, reference Figure 5 The dashed lines in the figure indicate that the unit or module is optional. This figure is a schematic diagram of the structure of a computer device implementing an interactive method for playing audiovisual advertisements according to an embodiment of this application. The interactive method for playing audiovisual advertisements described in the above embodiments can be achieved through… Figure 5 The computer device shown is used to implement this, and the computer device includes at least one processor 301, a memory 302 and at least one communication unit 305. The computer device may be a terminal device, a server or a chip.

[0103] Processor 301 can be a general-purpose processor or a special-purpose processor. For example, processor 301 can be a central processing unit (CPU), which can be used to control computer devices, execute software programs, and process data from software programs. The computer device may also include a communication unit 305 for inputting (receiving) and outputting (transmitting) signals.

[0104] For example, the computer device may be a chip, and the communication unit 305 may be the input and / or output circuit of the chip, or the communication unit 305 may be the communication interface of the chip, which may be a component of a terminal device, network device or other device.

[0105] For example, the computer device may be a terminal device or a server, and the communication unit 305 may be a transceiver of the terminal device or the server, or the communication unit 305 may be a transceiver circuit of the terminal device or the server.

[0106] The computer device may include one or more memories 302 storing a program 304. The program 304 can be executed by a processor 301 to generate instructions 303, causing the processor 301 to execute the method described in the above method embodiments according to the instructions 303. Optionally, the memory 302 may also store data (such as a target audit model). Optionally, the processor 301 may also read data stored in the memory 302, which may be stored at the same storage address as the program 304, or it may be stored at a different storage address than the program 304.

[0107] The processor 301 and memory 302 can be configured separately or integrated together, for example, integrated on the system on chip (SOC) of the terminal device.

[0108] It should be understood that each step of the above method embodiment can be completed by hardware logic circuits or software instructions in the processor 301. The processor 301 can be a CPU, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, such as discrete gates, transistor logic devices, or discrete hardware components.

[0109] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0110] For example, in some embodiments, this application also provides a computer-readable storage medium storing instructions or code that, when executed on a computer, cause the computer to implement the above-described interactive method for playing audiovisual advertisements.

[0111] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0112] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. An audiovisual advertisement play interaction method for performing audiovisual advertisement play interaction in an advertisement indication system, characterized by, Includes the following steps: The system collects human-computer interaction information between smart terminals and audiences during the playback of audiovisual advertisements, extracts audience dwell characteristics in the playback environment of the smart terminal from the human-computer interaction information, and filters out multiple advertising interest points from the audiovisual advertisements through the dwell characteristics. The advertising interest points refer to elements in the advertisement content that are attractive to the audience. By extracting the dwell characteristics of all audiences in each advertising interest point, all visual interest regions of the audience are extracted when the audiovisual advertisement is played on the smart terminal, and then the audience attention of each visual interest region is determined. Based on all audience attention, the visual attention map of the audience in the playback environment of the smart terminal is determined, wherein the audience attention represents the intensity of the audience's attention to the advertising content during the playback of the audiovisual advertisement. The auditory saliency of each advertising interest point is determined based on the self-attention mechanism, and then an auditory saliency map is generated when the audiovisual advertisement is played on the smart terminal by using all the auditory saliency. The visual attention map and the auditory salience map are combined to obtain the fused interactive features of the audiovisual advertisement when it is played on a smart terminal. The playback parameters of the audiovisual advertisement are adjusted using the fused interactive features. Specifically, the selection of multiple advertising interest points from audiovisual advertisements using the aforementioned dwell characteristics includes: Retrieve all significant playback nodes in audiovisual advertisements; The dwell time period in the dwell feature is used to determine the dwell value of each significant playback node; Multiple advertising interest points are selected from audiovisual advertisements based on various dwell times; Specifically, determining the audience attention level for each visual interest area includes: For each visual interest region, the dwell time, dwell frequency, and visual field density of the visual interest region are extracted from the human-computer interaction information. The audience attention of visual interest regions is determined by the dwell time, dwell frequency, and visual field density, thereby obtaining the audience attention of each visual interest region.

2. The method of claim 1, wherein, Extracting audience dwell characteristics from the human-computer interaction information specifically includes: Filter all audience users from all users in the playback environment of the smart terminal; Extract the dwell time and dwell area of ​​each audience user from the human-computer interaction information; The dwell characteristics of the audience in the playback environment of the smart terminal are determined by all dwell time periods and all dwell areas.

3. The method of claim 1, wherein, By extracting the dwell characteristics of all audiences at each advertising point of interest, the specific visual interest areas of the audience when the audiovisual advertisement is played on the smart terminal include: Obtain the dwell area from the dwell characteristics of all audiences in each ad interest point; Identify all areas of visual interest for the audience when audiovisual advertisements are played on smart terminals by identifying all the areas where the advertisements reside.

4. The method as described in claim 1, characterized in that, Determining the auditory saliency of each advertising interest point based on the self-attention mechanism specifically includes: For each ad interest point, obtain the dwell time period of the ad interest point; Extract the audio signal of the audiovisual advertisement within the specified dwell time period; The saliency features of the audio signal are extracted and quantified based on the self-attention mechanism to obtain the auditory saliency of the advertising interest points, and then the auditory saliency of each advertising interest point is obtained.

5. The method as described in claim 1, characterized in that, The smart terminal is a human-computer interaction device.

6. An advertising instruction system, comprising a playback interaction unit, wherein the system employs the method described in any one of claims 1 to 5 to perform playback interaction of audiovisual advertisements, characterized in that, The playback interaction unit includes: The data collection module is used to collect human-computer interaction information between the smart terminal and the audience when the audiovisual advertisement is played, extract the audience's dwell characteristics in the playback environment of the smart terminal from the human-computer interaction information, and filter out multiple advertising interest points from the audiovisual advertisement through the dwell characteristics. The processing module is used to extract all visual interest regions of the audience when the audiovisual advertisement is played on the smart terminal by using the dwell characteristics of all audiences in each advertising interest point, thereby determining the audience attention of each visual interest region, and determining the visual attention map of the audience in the playback environment of the smart terminal based on all audience attention. The processing module is also used to determine the auditory saliency of each advertising interest point based on the self-attention mechanism, and then generate an auditory saliency map of the audiovisual advertisement when it is played on the smart terminal through all the auditory saliency. The execution module is used to combine the visual attention map and the auditory salience map to obtain the fused interaction features when the audiovisual advertisement is played on the smart terminal, and to adjust the playback parameters in the audiovisual advertisement using the fused interaction features.

7. A computer device, characterized in that, The computer device includes a memory and a processor. The memory is used to store computer programs, and the processor is used to call and run the computer programs from the memory, causing the computer device to perform the interactive method for playing audiovisual advertisements according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions or code that, when executed on a computer, cause the computer to implement the interactive method for playing audiovisual advertisements as described in any one of claims 1 to 5.

Citation Information

Patent Citations

  • Precise interactive digital television advertisement putting system and method

    CN103279878A

  • Construction method and system of visual and auditory attention prediction model

    CN117478949A

  • Advertisement precision marketing putting method and system

    CN119168709A