Advertisement indication system and method

Abstract

The invention provides an advertisement indication system and method, and the method comprises the steps: extracting the residence characteristics of audiences in a playing environment where an intelligent terminal is located from human-computer interaction information, and screening out a plurality of advertisement interest points from an audio-visual advertisement through the residence characteristics; extracting all visual interest areas of the audiences when the audio-visual advertisement is played in the intelligent terminal through residence characteristics of all the audiences in each advertisement interest point, determining audience attention degrees of all the visual interest areas, and determining visual attention maps of the audiences in a playing environment where the intelligent terminal is located according to all the audience attention degrees; determining the auditory saliency of each advertisement interest point, and generating an auditory saliency map of the audiovisual advertisement during playing through all the auditory saliency; and carrying out audio-visual combination on the visual attention map and the auditory saliency map to obtain fusion interaction characteristics of the audio-visual advertisement during playing, and adjusting playing parameters in the audio-visual advertisement by using the fusion interaction characteristics. Based on the above scheme, audio-visual joint attention adjustment in audio-visual advertisement playing can be realized.

Description

Technical Field

[0001] The present application relates to the technical field of audio-visual advertisements, and more specifically, to an advertisement indication system and method. Background Art

[0002] Audiovisual advertising uses video and sound as the main body of advertising information, and has the characteristics of highly concentrated and intuitive communication, becoming a modern and eye-catching form of advertising. With the advent of the digital age, the production technology of audiovisual advertising art has become more diversified, and the forms of expression have become richer. The development of digital technology has made audiovisual advertising free from the restrictions of actual shooting space, expanding the imagination of creators, and achieving higher artistry and stronger visibility through animation, special effects, digital music and other technologies.

[0003] In the traditional advertising playback mode, the visual and auditory elements in the advertising playback are usually static and fixed, and cannot be personalized according to the audience's immediate feedback. When watching the advertisement, the audience's visual and auditory attention distribution varies greatly. They may focus on certain specific parts of the advertisement and ignore other content, thereby ignoring the audience's attention differences in different time and space dimensions, resulting in some advertising content failing to effectively attract the audience's attention. As a result, important information in the advertisement may be played when the audience's attention is weaker, resulting in poor information transmission effect and greatly reduced advertising effect. Therefore, how to achieve audio-visual joint attention regulation in audio-visual advertising playback has become a difficult problem faced by the industry. Summary of the invention

[0004] The present application provides an advertisement indication system and method, which can realize audio-visual joint attention adjustment in audio-visual advertisement playback.

[0005] In a first aspect, the present application provides a method for interactively playing audiovisual advertisements, comprising:

[0006] Collecting human-computer interaction information between the smart terminal and the audience when the audio-visual advertisement is played, extracting the audience's residence characteristics in the playback environment where the smart terminal is located from the human-computer interaction information, and screening multiple advertising interest points from the audio-visual advertisement through the residence characteristics;

[0007] By using the resident features of all audiences in each advertisement interest point, all visual interest areas of the audience when the audio-visual advertisement is played in the smart terminal are extracted, and then the audience attention of each visual interest area is determined, and the visual attention map of the audience in the playback environment where the smart terminal is located is determined based on all the audience attention;

[0008] Determine the auditory saliency of each advertisement interest point based on the self-attention mechanism, and then generate an auditory saliency map of the audiovisual advertisement when it is played in the smart terminal through all the auditory saliency;

[0009] The visual attention map and the auditory saliency map are audiovisually combined to obtain fused interactive features when the audiovisual advertisement is played in the smart terminal, and the playback parameters in the audiovisual advertisement are adjusted using the fused interactive features.

[0010] In some embodiments, extracting the audience's residence characteristics in the playback environment where the smart terminal is located from the human-computer interaction information specifically includes:

[0011] Filter out all audience users from all users in the playback environment where the smart terminal is located;

[0012] Extracting the residence time period and residence area of ​​each audience user from the human-computer interaction information;

[0013] The residence characteristics of the audience in the playback environment where the smart terminal is located are determined through all residence time periods and all residence areas.

[0014] In some embodiments, filtering out multiple advertising interest points from the audio-visual advertisement using the resident feature specifically includes:

[0015] Obtain all the significant playback nodes in the audio-visual advertisement;

[0016] Determine the residence value of each playback significant node according to the residence time period in the residence feature;

[0017] A plurality of advertisement interest points are screened out from the audio-visual advertisement based on the respective retention values.

[0018] In some embodiments, extracting all visual interest areas of the audience when the audio-visual advertisement is played in the smart terminal through the resident features of all audiences in each advertisement interest point specifically includes:

[0019] Obtaining the residence area in the residence characteristics of all audiences in each advertising interest point;

[0020] All the resident areas are used to determine all the visual interest areas of the audience when the audio-visual advertisement is played in the smart terminal.

[0021] In some embodiments, determining the audience attention of each visual interest area specifically includes:

[0022] For each visual interest region, extracting the dwelling time, dwelling frequency and visual field density of the visual interest region from the human-computer interaction information;

[0023] The audience attention of the visual interest area is determined by the dwelling time, the dwelling frequency and the visual field density, and then the audience attention of each visual interest area is obtained.

[0024] In some embodiments, determining the auditory prominence of each advertisement interest point based on the self-attention mechanism specifically includes:

[0025] For each advertising interest point, obtain the residence time period of the advertising interest point;

[0026] Extracting the audio signal of the audiovisual advertisement within the dwell time period;

[0027] The significant features of the audio signal are extracted and quantified based on the self-attention mechanism to obtain the auditory saliency of the advertisement interest point, and then the auditory saliency of each advertisement interest point is obtained.

[0028] In some embodiments, the intelligent terminal is a human-computer interaction device.

[0029] In a second aspect, the present application provides an advertisement indication system, including a playback interaction unit, wherein the playback interaction unit includes:

[0030] A collection module, used to collect human-computer interaction information between the smart terminal and the audience when the audio-visual advertisement is played, extract the audience's residence characteristics in the playback environment where the smart terminal is located from the human-computer interaction information, and screen multiple advertising interest points from the audio-visual advertisement through the residence characteristics;

[0031] A processing module is used to extract all visual interest areas of the audience when the audio-visual advertisement is played in the smart terminal through the resident features of all the audience in each advertisement interest point, and then determine the audience attention of each visual interest area, and determine the visual attention map of the audience in the playback environment where the smart terminal is located based on all the audience attention;

[0032] The processing module is also used to determine the auditory saliency of each advertisement interest point based on the self-attention mechanism, and then generate an auditory saliency map when the audio-visual advertisement is played in the smart terminal through all the auditory saliency;

[0033] The execution module is used to perform audio-visual combination of the visual attention map and the auditory saliency map to obtain fused interaction features when the audio-visual advertisement is played in the smart terminal, and use the fused interaction features to adjust the playback parameters in the audio-visual advertisement.

[0034] In a third aspect, the present application provides a computer device, comprising a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that the computer device executes the above-mentioned audio-visual advertisement playback interaction method.

[0035] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions or codes, and when the instructions or codes are executed on a computer, the computer implements the above-mentioned audio-visual advertisement playback interaction method.

[0036] The technical solution provided by the embodiments disclosed in this application has the following beneficial effects:

[0037] In an advertising indication system and method provided by the present application, human-computer interaction information between a smart terminal and an audience is collected when an audio-visual advertisement is played, and the audience's residence characteristics in the playback environment where the smart terminal is located are extracted from the human-computer interaction information. A plurality of advertising interest points are screened out from the audio-visual advertisement through the residence characteristics; all visual interest areas of the audience when the audio-visual advertisement is played in the smart terminal are extracted through the residence characteristics of all audiences in each advertising interest point, and then the audience's attention to each visual interest area is determined, and a visual attention map of the audience in the playback environment where the smart terminal is located is determined based on all the audience's attention; the auditory salience of each advertising interest point is determined based on a self-attention mechanism, and then an auditory salience map when the audio-visual advertisement is played in the smart terminal is generated through all the auditory saliences; the visual attention map and the auditory salience map are audio-visually combined to obtain a fused interaction feature when the audio-visual advertisement is played in the smart terminal, and the playback parameters in the audio-visual advertisement are adjusted using the fused interaction feature.

[0038] It can be seen that in the present application, the visual attention map and the auditory saliency map are combined audiovisually to obtain the fused interactive features when the audiovisual advertisement is played in the smart terminal, and the fused interactive features are used to adjust the playback parameters in the audiovisual advertisement; first, the visual attention map is determined to obtain the saliency distribution graph of the audience's attention area to the visual content during the audiovisual advertisement playback, so as to understand how the audience allocates their attention during the audiovisual advertisement playback and which visual elements need more emphasis. Combined with the advantages of the self-attention mechanism, the visual attention map can be dynamically adjusted according to the specific attention changes of the audience. The dynamic adjustment of the visual content can not only improve the audience's perception of the audiovisual advertisement, but also strengthen the display of the audience's visual focus, thereby optimizing the audiovisual advertising effect. At the same time, in the subsequent audio-visual joint attention adjustment, the visual attention map can guide the adjustment of the audiovisual advertising content according to the audience's attention characteristics, thereby improving the attractiveness of the audiovisual advertisement. , reducing the audience's visual neglect area, thereby improving the communication effect of audio-visual advertising; then, determining the auditory saliency map can obtain the spatial distribution and temporal characteristic graph of the audience's saliency response to the audio content during the audio-visual advertising playback, which is helpful to identify the key points of the audio content in the audio-visual advertising, such as particularly attention-grabbing voice prompts, sound effects or music segments, and at the same time can provide more comprehensive audience attention feedback for audio-visual advertising, which is helpful to synchronously optimize the audio and visual parameters of audio-visual advertising. By integrating the auditory saliency map into the audio-visual joint attention adjustment, the volume, sound effects or voice display method can be adjusted according to the audience's immediate audio attention when the audio-visual advertising is played, thereby ensuring that the key audio information in the audio-visual advertising is best presented, further enhancing the appeal and attractiveness of the audio-visual advertising, and improving the audience's participation and the effect of the audio-visual advertising; in summary, based on the above scheme, the audio-visual joint attention adjustment in the audio-visual advertising playback can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0040] Figure 1 is an exemplary flow chart of a method for interactively playing audiovisual advertisements according to some embodiments of the present application;

[0041] Figure 2 It is a basic flow chart of audio-visual advertisement delivery according to some embodiments of the present application;

[0042] Figure 3 is a schematic diagram of a process for determining fusion interaction features according to some embodiments of the present application;

[0043] Figure 4 is a structural schematic diagram of a playback interaction unit according to some embodiments of the present application;

[0044] Figure 5 It is a structural diagram of a computer device for implementing an interactive method for playing audio-visual advertisements according to some embodiments of the present application. DETAILED DESCRIPTION

[0045] In order to better understand the technical solution of the present application, the technical solution of the present application will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0046] refer to Figure 1 , which is an exemplary flow chart of a method for playing and interacting audiovisual advertisements according to some embodiments of the present application. The method for playing and interacting audiovisual advertisements mainly includes the following steps:

[0047] In step 101, human-computer interaction information between a smart terminal and an audience is collected when an audio-visual advertisement is played, and the audience's residence characteristics in the playback environment where the smart terminal is located are extracted from the human-computer interaction information. Multiple advertising interest points are screened from the audio-visual advertisement based on the residence characteristics.

[0048] It should be noted that in the present application, the audio-visual advertisement is an intelligent advertisement based on a multimodal attention mechanism; the human-computer interaction information is information used to describe the interaction process between the user and the smart terminal, and the smart terminal is a human-computer interaction device; in specific implementation, the monitoring system in the smart terminal can be used to monitor the time period, gaze area and click area of ​​each user operation when the audio-visual advertisement is played in the smart terminal within a specified time period (the default is the last week) as the user's interactive behavior, and the collection of all interactive behaviors can be used as the human-computer interaction information between the smart terminal and the audience when the audio-visual advertisement is played.

[0049] In some embodiments, reference Figure 2 As described above, this figure is a basic flow chart of audio-visual advertising delivery according to some embodiments of the present application. This figure shows the basic structure of the advertising delivery process. First, the advertiser (i.e., the company or individual who wants to promote its products or services) establishes a cooperative relationship with the advertising agency; the advertiser entrusts the advertising agency to act as the agent for the advertisement, and the advertising agency is responsible for publishing the advertising content to the appropriate media; the media plays the role of the advertising space provider in this process, and they have advertising spaces that the audience may notice. These advertising spaces can be television, radio, newspapers, magazines, outdoor billboards or online platforms, etc.

[0050] After the advertisement is released, the audience will "consume" these advertisements through the media. The audience is the recipient of the advertising information. They learn about the advertiser's products or services by watching, listening to or reading the advertisement. The effectiveness of the advertisement depends on many factors, including the attractiveness of the advertising content, the coverage of the media, the audience's preferences and the timing of the advertisement.

[0051] The diagram also reflects the different roles involved in the advertising process and the relationships between them. Advertisers are the initiators of advertising campaigns, advertising agencies are the bridges between advertisers and the media, and the media are the channels through which advertising information is delivered to the audience. The audience's reactions and behaviors are key indicators of the success of advertising. This structured process helps ensure that advertising information can be effectively delivered from advertisers to target audiences, thereby achieving the purpose of advertising.

[0052] In some embodiments, extracting the audience's residence characteristics in the playback environment where the smart terminal is located from the human-computer interaction information can be achieved by using the following steps:

[0053] Filter out all audience users from all users in the playback environment where the smart terminal is located;

[0054] Extracting the residence time period and residence area of ​​each audience user from the human-computer interaction information;

[0055] The residence characteristics of the audience in the playback environment where the smart terminal is located are determined through all residence time periods and all residence areas.

[0056] In specific implementation, first, all audience users are screened out from all users in the playback environment where the smart terminal is located, which can be implemented in the following manner, namely: all users in the playback environment where the smart terminal is located are obtained from the monitoring system of the smart terminal, and all users involved in the human-computer interaction information can be used as audience users, so that all audience users can be obtained; then, the residence time period and residence area of ​​each audience user are extracted from the human-computer interaction information, which can be implemented in the following manner, namely: for each audience user, all interaction behaviors of the audience user are screened out from the human-computer interaction information, so that the set of time periods in each interaction behavior is used as the residence time period of the audience user, and the set of click areas in each interaction behavior is used as the residence area of ​​the audience user, and the residence time period and residence area of ​​each audience user can be obtained by the above method; finally, the residence characteristics of the audience in the playback environment where the smart terminal is located are determined by all residence time periods and all residence areas, which can be implemented in the following manner, namely: the set of all residence time periods is used as the residence time feature, and the set of all residence areas is used as the residence space feature, and the set of residence time feature and residence space feature can be used as the residence feature of the audience in the playback environment where the smart terminal is located.

[0057] It should be noted that, in the present application, the dwell feature refers to the characteristics of the audience's dwelling behavior during the playback of audio-visual advertisements; the audience user refers to the target group that watches and listens to the audio-visual advertisement content; the dwell time period refers to the time interval during which the audience concentrates on a certain advertising interest point during the playback of audio-visual advertisements; the dwell area refers to the specific area that the audience pays attention to during the playback of audio-visual advertisements.

[0058] In some embodiments, filtering out multiple advertising interest points from audio-visual advertisements using the resident features may be achieved by using the following steps:

[0059] Obtain all the significant playback nodes in the audio-visual advertisement;

[0060] Determine the residence value of each playback significant node according to the residence time period in the residence feature;

[0061] A plurality of advertisement interest points are screened out from the audio-visual advertisement based on the respective retention values.

[0062] It should be noted that, in the present application, the advertisement interest point refers to the element in the advertisement content that has strong attraction to the audience; in specific implementation, first, all the playback significant nodes in the audio-visual advertisement are obtained, which can be implemented in the following manner, namely: all the playback significant nodes in the audio-visual advertisement are obtained from the advertisement design instructions of the audio-visual advertisement, and the playback significant nodes refer to the time nodes in the audio-visual advertisement used to highlight the advertisement content; then, the residence value of each playback significant node is determined by the residence time period in the residence feature, which can be implemented in the following manner, namely: for the playback significant nodes, the playback significant nodes are screened from the residence feature. For all the dwell time periods under the playback node, the average of the corresponding durations of all the dwell time periods can be used as the dwell value of the playback significant node. The dwell value of each playback significant node can be obtained by the above method. The dwell value is a quantitative value used to measure the audience's attention intensity on the advertising interest point; finally, based on each dwell value, multiple advertising interest points can be screened out from the audio-visual advertisement. The following method can be used, that is, the average of all the dwell values ​​can be used as the dwell threshold of the audio-visual advertisement, so that the playback significant nodes corresponding to each dwell value higher than the dwell threshold can be used as advertising interest points, and multiple advertising interest points can be obtained.

[0063] In step 102, all visual interest areas of the audience when the audio-visual advertisement is played in the smart terminal are extracted through the resident characteristics of all audiences in each advertisement interest point, and then the audience attention of each visual interest area is determined, and the visual attention map of the audience in the playback environment where the smart terminal is located is determined based on all the audience attention.

[0064] In some embodiments, extracting all visual interest areas of the audience when the audio-visual advertisement is played in the smart terminal through the resident features of all audiences in each advertisement interest point can be achieved by the following steps:

[0065] Obtaining the residence area in the residence characteristics of all audiences in each advertising interest point;

[0066] All the resident areas are used to determine all the visual interest areas of the audience when the audio-visual advertisement is played in the smart terminal.

[0067] It should be noted that in the present application, the visual interest area refers to the screen area that the audience pays attention to when the audio-visual advertisement is played; in the specific implementation, first, the residence area in the residence characteristics of all audiences in each advertising interest point is obtained; then, all the visual interest areas of the audience when the audio-visual advertisement is played in the smart terminal are determined through all the residence areas. This can be achieved in the following way, namely: all the residence areas can be deduplicated and used as the visual interest areas when the audio-visual advertisement is played in the smart terminal, so that all the visual interest areas of the audience when the audio-visual advertisement is played in the smart terminal can be obtained.

[0068] In some embodiments, determining the audience attention of each visual interest area may be achieved by using the following steps:

[0069] For each visual interest region, extracting the dwelling time, dwelling frequency and visual field density of the visual interest region from the human-computer interaction information;

[0070] The audience attention of the visual interest area is determined by the dwelling time, the dwelling frequency and the visual field density, and then the audience attention of each visual interest area is obtained.

[0071] It should be noted that, in the present application, audience attention refers to the interest and attention intensity generated by the audience in a specific advertising content during the playback of audio-visual advertisements; in specific implementation, first, for each visual interest area, the residence time, residence frequency and field of view density of the visual interest area are extracted from the human-computer interaction information, which can be implemented in the following manner, namely: for each visual interest area, all time periods of user operation in the visual interest area are obtained from the human-computer interaction information, the total duration of all time periods can be used as the residence time of the visual interest area, and the residence time represents the cumulative time that the audience stays at the advertising interest point, the number of all time periods can be used as the residence frequency of the visual interest area, and the residence frequency represents the number of times the audience pays attention to the advertising interest point, and at the same time, the self-attention mechanism is used to analyze the visual characteristics of the visual interest area. The density of the eye features is used as the visual field density, which indicates the density of the audience's eye movement trajectories in a specific area. The area with higher density is often the focus of the audience, which helps to accurately locate the focus of the advertisement; then, the audience attention to the visual interest area is determined by the dwell time, the dwell frequency and the visual field density, and then the audience attention to each visual interest area can be obtained. The following method can be used, namely: construct a multivariate regression model, combine the dwell time, the dwell frequency and the visual field density, and calculate the audience attention to the visual interest area, that is, audience attention = a*dwell time+b*dwell frequency+c*visual field density+d, wherein a, b, c, d are parameters of the multivariate regression model, which are obtained by training with historical operation data. The audience attention to each visual interest area can be obtained by the above method.

[0072] In some embodiments, determining the visual attention map of the audience in the playback environment where the smart terminal is located according to the attention of all audiences can be implemented by the following steps:

[0073] Merge all visual interest regions adjacently to obtain multiple merged regions of the audio-visual advertisement;

[0074] Determine the visual attention intensity of each merged area through all audience attention;

[0075] All visual attention intensities are mapped to the full-screen image to obtain the audience's visual attention map in the playback environment of the smart terminal.

[0076] In the specific implementation, first, all the visual interest areas are merged in a neighboring manner to obtain multiple merged areas of the audio-visual advertisement, which can be implemented in the following manner, namely: the visual interest areas with adjacent positions can be merged into one merged area, so as to obtain multiple merged areas of the audio-visual advertisement; then, the visual attention intensity of each merged area is determined by all the audience attention, which can be implemented in the following manner, namely: for each merged area, the audience attention of each visual interest area in the merged area is obtained, so that the average of all the audience attention can be used as the visual attention intensity of the merged area, and the visual attention intensity of each merged area can be obtained by the above method; finally, all the visual attention intensities are mapped to a full-screen image to obtain a visual attention map of the audience in the playback environment where the smart terminal is located, which can be implemented in the following manner, namely: initialize an empty full-screen image, divide the full-screen image according to the position coordinates of the merged area, so as to map each visual attention intensity to the full-screen image, and the mapped full-screen image can be used as the visual attention map of the audience in the playback environment where the smart terminal is located.

[0077] It should be noted that in the present application, the visual attention map represents a saliency distribution graph of the audience's attention area to the visual content during the audio-visual advertisement playback; the merged area is used to describe the comprehensive area of ​​adjacent visual interests in the visual attention map; and the visual attention intensity represents a numerical indicator of the audience's attention level to a specific area in the visual content.

[0078] In step 103, the auditory saliency of each advertisement interest point is determined based on the self-attention mechanism, and then the auditory saliency map of the audio-visual advertisement when it is played in the smart terminal is generated through all the auditory saliency.

[0079] In some embodiments, determining the auditory prominence of each advertisement interest point based on the self-attention mechanism may be implemented by the following steps:

[0080] For each advertising interest point, obtain the residence time period of the advertising interest point;

[0081] Extracting the audio signal of the audiovisual advertisement within the dwell time period;

[0082] The significant features of the audio signal are extracted and quantified based on the self-attention mechanism to obtain the auditory saliency of the advertisement interest point, and then the auditory saliency of each advertisement interest point is obtained.

[0083] It should be noted that, in the present application, auditory saliency indicates the intensity of the audio signal in the audio-visual advertisement playing to attract the audience's attention; in specific implementation, first, for each advertising interest point, the dwelling time period of the advertising interest point is obtained; then, the audio signal of the audio-visual advertisement during the dwelling time period is extracted, which can be implemented in the following manner, namely: a video segment of the audio-visual advertisement played during the dwelling time period is obtained, and the audio signal in the video segment can be used as the audio signal of the audio-visual advertisement during the dwelling time period; finally, the saliency features of the audio signal are extracted and quantified based on the self-attention mechanism to obtain the auditory saliency of the advertising interest point, and then the auditory saliency of each advertising interest point is obtained. The following manner is implemented, namely: a feature extraction algorithm based on the self-attention mechanism (for example: Transformer model) is used to extract the time domain features in the audio signal as the saliency features of the audio signal, and then the saliency features are normalized using a normalization algorithm, and the result after the saliency features are normalized can be used as the auditory saliency of the advertising interest point. The auditory saliency of each advertising interest point can be obtained by the above manner.

[0084] In some embodiments, generating an auditory saliency map of an audiovisual advertisement when it is played in a smart terminal using all auditory saliencies may be implemented by the following steps:

[0085] All auditory saliences are temporally integrated according to the dwelling time period of each advertisement interest point, thereby generating a salience time series of the audiovisual advertisement;

[0086] The saliency map of the audio-visual advertisement is constructed in multiple levels through the saliency time series to obtain an auditory saliency map when the audio-visual advertisement is played in the smart terminal.

[0087] In a specific implementation, first, all auditory prominences are time-sequentially integrated according to the dwelling time period of each advertising interest point, so as to generate a prominence time series of the audiovisual advertisement. This can be implemented in the following manner, namely: the ratio of the dwelling time period of each auditory prominence corresponding to the advertising interest point to the total duration of the audiovisual advertisement is used as a weight, and the weighted value of each auditory prominence is calculated, so that each weighted value is sorted in chronological order according to the middle time point of the dwelling time period of the corresponding advertising interest point as a prominence time series covering the entire playback process; then, the prominence time series is used to construct a saliency map of the audiovisual advertisement in multiple levels, so as to obtain an auditory saliency map when the audiovisual advertisement is played in a smart terminal. This can be implemented in the following manner, namely: a graph theory model based on a saliency neural network is initialized, the prominence time series is used as a time weight node in the graph theory model, and the graph theory model is used to construct a saliency map of the audiovisual advertisement in multiple levels, so that the result of the multi-level construction of the graph theory model can be used as the auditory saliency map when the audiovisual advertisement is played in the smart terminal.

[0088] It should be noted that in the present application, the auditory saliency map represents the spatial distribution and temporal characteristic graph of the audience's saliency response to the audio content during the playback of audio-visual advertisements; the saliency time series represents the dynamic characteristic sequence of the auditory saliency changing over time during the playback of audio-visual advertisements; the graph theory model is a model based on a saliency neural network. The graph theory model constructs a dynamically changing graph structure by using the saliency time series as a time weight node. Each node represents the intensity of the auditory saliency in the playback of audio-visual advertisements within a specific time period. The edges in the graph represent the temporal correlation between nodes. The self-attention mechanism is used to dynamically adjust the weights between nodes, thereby realizing multi-level information transmission and aggregation. Through the multi-level construction of the graph theory model, the temporal evolution of the auditory saliency features in the playback of audio-visual advertisements can be accurately captured, thereby generating a complete auditory saliency map, reflecting the audience's attention changes to the audio content of audio-visual advertisements.

[0089] In step 104, the visual attention map and the auditory saliency map are audiovisually combined to obtain fused interaction features when the audiovisual advertisement is played in the smart terminal, and the playback parameters in the audiovisual advertisement are adjusted using the fused interaction features.

[0090] In some embodiments, the visual attention map and the auditory saliency map are combined to obtain the fusion interaction features when the audio-visual advertisement is played in the smart terminal. Figure 3 The figure is a schematic diagram of a process for determining fusion interaction features in some embodiments of the present application. In this embodiment, determining the fusion interaction features can be implemented by the following steps:

[0091] In step 1041, the visual attention map and the auditory saliency map are weighted fused to obtain a fused feature map;

[0092] In step 1042, the interactive features in the fused feature map are extracted as fused interactive features when the audio-visual advertisement is played in the smart terminal.

[0093] It should be noted that, in the present application, the fused interactive features represent the dynamic distribution of audience attention and response characteristics under the comprehensive influence of multimodal information in audio-visual advertisements; in specific implementation, first, the visual attention map and the auditory saliency map are weightedly fused to obtain a fused feature map, which can be implemented in the following manner, namely: the actual contribution ratio of vision and hearing to audience appeal can be obtained from the advertising design instructions of the audio-visual advertisement as a weight, so that a weighted feature fusion algorithm (for example: a dynamic weight allocation algorithm based on an attention mechanism) can be used to fuse the spatial distribution characteristics of the visual attention map with the temporal change characteristics of the auditory saliency map, and the fused feature map can be used as a fused feature map The fused feature map is a feature map used to describe the comprehensive impact of visual attention and auditory saliency characteristics on the audience's attention when the advertising content is played in the smart terminal; then, the interactive features in the fused feature map are extracted as fused interactive features when the audio-visual advertisement is played in the smart terminal. This can be achieved in the following way, namely: the multimodal attention changes of the audience during the audio-visual advertisement playback can be extracted from the fused feature map using a spatiotemporal correlation analysis algorithm as interactive features. These interactive features characterize the audience's comprehensive response to the audio-visual advertisement content, so that the interactive features can be used as fused interactive features when the audio-visual advertisement is played in the smart terminal to provide support for subsequent playback optimization and advertising strategies.

[0094] In some embodiments, the use of the fused interactive features to adjust the playback parameters in the audio-visual advertisement can be achieved in the following manner, namely: by analyzing the distribution of visual and auditory attention in the fused interactive features, identifying the difference in audience attention to different content, and dynamically adjusting the playback parameters of the advertisement, for example: picture parameter adjustment: adjusting the picture brightness, contrast and color saturation according to the distribution of visual attention, highlighting the content of the high-attention area, and appropriately weakening the low-attention area; audio parameter adjustment: combining the changes in auditory salient features, optimizing the volume, frequency enhancement and sound positioning of the audio, so that the key sound effects or voices are more prominent; playback timing optimization: optimizing the playback speed or the rhythm of content switching according to the temporal salience in the interactive features to ensure that the audience can fully understand the content of the high-salience area; multimodal synchronization adjustment: optimizing the time synchronization of the audio-visual content through the collaborative analysis of visual and auditory features, such as aligning the switching points of the audio and the picture to improve sensory consistency; interactive response enhancement: combining the interactive features to adjust the interactive prompts at specific moments, such as popping up a prompt box or adding click hotspots to enhance user participation.

[0095] In addition, in another aspect of the present application, in some embodiments, the present application provides an advertisement indication system, the advertisement indication system comprising a playback interaction unit, referring to Figure 4, which is a schematic diagram of the structure of a playback interaction unit according to some embodiments of the present application, the playback interaction unit includes: a collection module 201, a processing module 202 and an execution module 203, which are described as follows:

[0096] The acquisition module 201 in the present application is mainly used to collect the human-computer interaction information between the smart terminal and the audience when the audio-visual advertisement is played, extract the audience's residence characteristics in the playback environment where the smart terminal is located from the human-computer interaction information, and screen multiple advertising interest points from the audio-visual advertisement through the residence characteristics;

[0097] Processing module 202, in the present application, is used to extract all visual interest areas of the audience when the audio-visual advertisement is played in the smart terminal through the resident features of all audiences in each advertisement interest point, and then determine the audience attention of each visual interest area, and determine the visual attention map of the audience in the playback environment where the smart terminal is located based on all the audience attentions;

[0098] It should be noted that the processing module 202 is also used to determine the auditory saliency of each advertisement interest point based on the self-attention mechanism, and then generate an auditory saliency map when the audio-visual advertisement is played in the smart terminal through all the auditory saliency;

[0099] Execution module 203, in the present application, is mainly used to perform audio-visual combination of the visual attention map and the auditory saliency map to obtain fused interaction features when the audio-visual advertisement is played in the smart terminal, and use the fused interaction features to adjust the playback parameters in the audio-visual advertisement.

[0100] The above describes in detail the examples of the advertising indication system and method provided by the embodiments of the present application. It can be understood that in order to realize the above functions, the corresponding device includes hardware structures and / or software modules corresponding to the execution of each function. It should be easily appreciated by those skilled in the art that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present application.

[0101] In some embodiments, the present application also provides a computer device, which includes a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that the computer device executes the above-mentioned audio-visual advertisement playback interaction method.

[0102] In some embodiments, reference Figure 5 , the dotted line in the figure indicates that the unit or the module is optional, and the figure is a schematic diagram of the structure of a computer device for implementing the interactive playback method of audio-visual advertisements according to an embodiment of the present application. The interactive playback method of audio-visual advertisements described in the above embodiment can be Figure 5 The computer device shown in the figure is implemented, and the computer device includes at least one processor 301, a memory 302 and at least one communication unit 305. The computer device can be a terminal device, a server or a chip.

[0103] The processor 301 may be a general-purpose processor or a special-purpose processor. For example, the processor 301 may be a central processing unit (CPU), which may be used to control the computer device, execute software programs, and process data of the software programs. The computer device may also include a communication unit 305 to implement signal input (reception) and output (transmission).

[0104] For example, the computer device may be a chip, the communication unit 305 may be an input and / or output circuit of the chip, or the communication unit 305 may be a communication interface of the chip, and the chip may be a component of a terminal device, a network device, or other devices.

[0105] For another example, the computer device may be a terminal device or a server, and the communication unit 305 may be a transceiver of the terminal device or the server, or the communication unit 305 may be a transceiver circuit of the terminal device or the server.

[0106] The computer device may include one or more memories 302, on which a program 304 is stored. The program 304 can be executed by the processor 301 to generate instructions 303, so that the processor 301 performs the method described in the above method embodiment according to the instructions 303. Optionally, data (such as a target audit model) can also be stored in the memory 302. Optionally, the processor 301 can also read the data stored in the memory 302, and the data can be stored at the same storage address as the program 304, or the data can be stored at a different storage address from the program 304.

[0107] The processor 301 and the memory 302 may be provided separately or integrated together, for example, integrated on a system on chip (SOC) of the terminal device.

[0108] It should be understood that each step of the above method embodiment can be completed by a hardware-based logic circuit or software-based instructions in the processor 301. The processor 301 can be a CPU, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, such as discrete gates, transistor logic devices or discrete hardware components.

[0109] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0110] For example, in some embodiments, the present application also provides a computer-readable storage medium, in which instructions or codes are stored. When the instructions or codes are run on a computer, the computer implements the above-mentioned audio-visual advertisement playback interaction method when executed.

[0111] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0112] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for interactively playing audiovisual advertisements, used for interactively playing audiovisual advertisements in an advertisement indication system, characterized in that: The steps include: Collecting human-computer interaction information between the smart terminal and the audience when the audio-visual advertisement is played, extracting the audience's residence characteristics in the playback environment where the smart terminal is located from the human-computer interaction information, and screening multiple advertising interest points from the audio-visual advertisement through the residence characteristics; By using the resident features of all audiences in each advertisement interest point, all visual interest areas of the audience when the audio-visual advertisement is played in the smart terminal are extracted, and then the audience attention of each visual interest area is determined, and the visual attention map of the audience in the playback environment where the smart terminal is located is determined based on all the audience attention; Determine the auditory saliency of each advertisement interest point based on the self-attention mechanism, and then generate an auditory saliency map of the audiovisual advertisement when it is played in the smart terminal through all the auditory saliency; The visual attention map and the auditory saliency map are audiovisually combined to obtain fused interactive features when the audiovisual advertisement is played in the smart terminal, and the playback parameters in the audiovisual advertisement are adjusted using the fused interactive features.

2. The method according to claim 1, characterized in that Extracting the audience's residence characteristics in the playback environment where the smart terminal is located from the human-computer interaction information specifically includes: Filter out all audience users from all users in the playback environment where the smart terminal is located; Extracting the residence time period and residence area of ​​each audience user from the human-computer interaction information; The residence characteristics of the audience in the playback environment where the smart terminal is located are determined through all residence time periods and all residence areas.

3. The method according to claim 1, characterized in that Screening out multiple advertising interest points from the audio-visual advertisements by using the resident features specifically includes: Obtain all the significant playback nodes in the audio-visual advertisement; Determine the residence value of each playback significant node according to the residence time period in the residence feature; A plurality of advertisement interest points are screened out from the audio-visual advertisement based on the respective retention values.

4. The method according to claim 1, characterized in that The audience’s visual interest areas when the audio-visual advertisement is played in the smart terminal are extracted through the resident characteristics of all audiences in each advertisement interest point, including: Obtaining the residence area in the residence characteristics of all audiences in each advertising interest point; All the resident areas are used to determine all the visual interest areas of the audience when the audio-visual advertisement is played in the smart terminal.

5. The method according to claim 1, characterized in that Determining the audience attention of each visual interest area specifically includes: For each visual interest region, extracting the dwelling time, dwelling frequency and visual field density of the visual interest region from the human-computer interaction information; The audience attention of the visual interest area is determined by the dwelling time, the dwelling frequency and the visual field density, and then the audience attention of each visual interest area is obtained.

6. The method according to claim 1, characterized in that The auditory saliency of each advertisement interest point is determined based on the self-attention mechanism, including: For each advertising interest point, obtain the residence time period of the advertising interest point; Extracting the audio signal of the audiovisual advertisement within the dwell time period; The significant features of the audio signal are extracted and quantified based on the self-attention mechanism to obtain the auditory saliency of the advertisement interest point, and then the auditory saliency of each advertisement interest point is obtained.

7. The method according to claim 1, characterized in that The intelligent terminal is a human-computer interaction device.

8. An advertisement indication system, comprising a playback interaction unit, characterized in that: The playback interaction unit comprises: A collection module, used to collect human-computer interaction information between the smart terminal and the audience when the audio-visual advertisement is played, extract the audience's residence characteristics in the playback environment where the smart terminal is located from the human-computer interaction information, and screen multiple advertising interest points from the audio-visual advertisement through the residence characteristics; A processing module is used to extract all visual interest areas of the audience when the audio-visual advertisement is played in the smart terminal through the resident features of all the audience in each advertisement interest point, and then determine the audience attention of each visual interest area, and determine the visual attention map of the audience in the playback environment where the smart terminal is located based on all the audience attention; The processing module is also used to determine the auditory saliency of each advertisement interest point based on the self-attention mechanism, and then generate an auditory saliency map when the audio-visual advertisement is played in the smart terminal through all the auditory saliency; The execution module is used to perform audio-visual combination of the visual attention map and the auditory saliency map to obtain fused interaction features when the audio-visual advertisement is played in the smart terminal, and use the fused interaction features to adjust the playback parameters in the audio-visual advertisement.

9. A computer device, characterized in that: The computer device includes a memory and a processor, the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that the computer device executes the audio-visual advertisement playback interaction method described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions or codes, and when the instructions or codes are executed on a computer, the computer implements the audio-visual advertisement playback interaction method according to any one of claims 1 to 7.

Images