A manifest rewrite based method and system for ad insertion in streaming media without perceptibility

Abstract

The application relates to the technical field of streaming media data processing, in particular to a streaming media advertisement non-sensory insertion method and system based on list rewriting. The method comprises the following steps: acquiring predetermined segment information, performing multi-dimensional sensory context analysis to generate a sensory context descriptor set; performing visual rhythm alignment, auditory feature fusion and narrative flow correlation based on the descriptor set, screening advertisements meeting preset sensory correlation conditions to generate a personalized candidate advertisement set; performing dynamic adaptation of encoding parameters and intelligent synthesis of transition frames based on the candidate advertisement set to generate a non-sensory advertisement recommendation insertion set; embedding the insertion set into an original list to replace an advertisement placeholder, and outputting an updated non-sensory advertisement playing list. The application can realize sensory matching of advertisements and streaming media content in visual, auditory and narrative dimensions, eliminate playing abnormities caused by inconsistent encoding parameters, realize seamless switching of advertisement insertion points through smooth transition frames, and improve user watching experience.

Description

Technical Field

[0001] This application relates to the field of streaming media data processing technology, specifically to a method and system for seamless insertion of streaming media advertisements based on list rewriting. Background Technology

[0002] In the current streaming media service architecture, ad insertion is a crucial aspect of maintaining platform operation, primarily employing two methods: client-side insertion and server-side insertion (SSAI). Server-side insertion technology seamlessly integrates ad content with the main video stream by locating ad placeholders (such as SCTE-35 markers) in the original playlist and replacing the corresponding ad segment URL at that position, thus mitigating client-side ad blocking to some extent. Existing solutions typically select target ads from a candidate ad library based on user profile data or advertiser targeting strategies, integrate the encoded ad segments into the streaming media playlist, and finally distribute them to the terminal player for continuous playback via the content delivery network.

[0003] However, in existing technologies, the insertion of advertisements often leads to perceptual gaps in the user's viewing experience, and can easily cause playback anomalies when switching between content with different encoding parameters. Due to the lack of real-time perceptual matching of the currently playing content at the visual, auditory, and narrative levels, there are often abrupt stylistic differences between advertisements and the main content. At the same time, when the encoding parameters of advertisement segments and content segments are inconsistent, the player needs to reinitialize the decoder, which can cause technical problems such as black screens, stuttering, or audio-visual desynchronization, affecting the continuity and stability of streaming media services. Summary of the Invention

[0004] This application provides a method and system for seamless insertion of streaming media advertisements based on list rewriting to solve the above-mentioned problems.

[0005] Firstly, this application provides a method for seamless insertion of streaming media advertisements based on manifest rewriting, the method comprising: Obtain information about a predetermined segment before the ad insertion point in the streaming media content; perform multi-dimensional perceptual context analysis based on the predetermined segment information; and generate a perceptual context descriptor set. Based on the set of perceptual context descriptors, visual rhythm alignment, auditory feature fusion, and narrative flow association are performed based on personalized perceptual matching of advertisements. Ads that meet the preset perceptual association conditions are selected from the candidate advertisement library to generate a personalized candidate advertisement set. Based on the personalized candidate ad set, context-aware encoding parameter dynamic adaptation and transition frame intelligent synthesis are performed to generate a seamless ad recommendation insertion set; The seamless ad recommendation insertion set is embedded into the original list of streaming media content, the corresponding ad placeholders are replaced, and the updated seamless ad playback list is output.

[0006] Optionally, the process of generating the perceptual context descriptor set includes: The predetermined segment information includes video streams, audio streams, and subtitle / metadata streams; By analyzing the pixel statistical features of consecutive frame sequences in the video stream, the main color scheme, motion rhythm, and illumination distribution of the scene are extracted to generate visual rhythm features. By analyzing the audio spectrum and loudness sequence of the audio stream, the beat and volume change patterns of the background music are extracted to generate auditory rhythm features; By parsing the natural language semantics of the subtitle / metadata stream, the emotional polarity, thematic keywords, and narrative rhythm in the lines or narration are extracted to generate narrative stream features; The visual prosodic features, the auditory rhythm features, and the narrative flow features are structurally associated and integrated into the perceptual context descriptor set.

[0007] Optionally, the process of generating the personalized candidate ad set includes: Based on the visual prosodic features, auditory rhythm features, and narrative flow features in the perceptual context descriptor set, the visual prosodic matching degree, auditory feature matching degree, and narrative flow correlation degree corresponding to each candidate advertisement are analyzed respectively. Candidate ads that exceed a preset candidate threshold in at least one of the visual rhythm matching degree, the auditory feature matching degree, and the narrative flow correlation degree are considered as ads that satisfy the preset perceptual correlation conditions, and thus constitute the personalized candidate ad set.

[0008] Optionally, the analysis process for the visual prosody matching degree includes: Extract the main color sequence, motion vector amplitude sequence, and brightness mean sequence of the last N frames of the predetermined segment from the visual rhythm features to form the source visual rhythm curve group; Extract the corresponding sequence of the first N frames of each candidate advertisement from its visual tone to form a candidate visual rhythm curve group; The cumulative distance between the two sets of curves is calculated using a dynamic time warping algorithm, and its reciprocal is transformed into visual prosodic dynamic similarity. Meanwhile, the difference in shot switching frequency between the last N frames of the predetermined segment and the first N frames of the candidate advertisement is analyzed to obtain the shot rhythm consistency. The visual rhythm dynamic similarity is multiplied by the shot rhythm consistency to obtain the visual rhythm matching degree.

[0009] Optionally, the analysis process for the auditory feature matching degree includes: The frequency band energy distribution, timbre feature vector and loudness envelope curve of the final continuous audio of the predetermined segment are extracted from the auditory rhythm features to form a source auditory feature set. The corresponding features of the starting continuous audio of each candidate advertisement are extracted from the audio features of each candidate advertisement to form a candidate auditory feature set. Calculate the spectral centroid distance, timbre cosine similarity, and loudness Pearson correlation coefficient between the two feature sets, and combine these three values ​​after normalization to obtain the auditory feature fusion degree. Simultaneously, the degree of matching between the background music emotion tags of the predetermined segment's ending audio and the candidate advertisement's starting audio is analyzed to obtain the emotion evolution compatibility. The square root of the product of the auditory feature fusion degree and the emotional evolution compatibility degree is used to obtain the auditory feature matching degree.

[0010] Optionally, the analysis process of the narrative flow correlation includes: The emotional polarity intensity, the set of thematic keywords, and the narrative rhythm type of the last line or narration of the predetermined segment are extracted from the narrative flow features to form a source narrative feature group. Corresponding content is extracted from the advertising words or scene descriptions of each candidate advertisement to form a candidate narrative feature group. Calculate the negative exponential transformation of the difference in emotional polarity intensity between the two feature groups, the overlap of thematic keywords, and the matching degree of narrative rhythm type, and obtain the semantic structure correlation by weighted summation; At the same time, the trend coupling degree is obtained by analyzing whether the narrative arc trend at the end of the predetermined segment is consistent with the emotional evolution direction conveyed by the candidate advertisement; The narrative flow correlation is obtained by weighting and summing the semantic structure correlation degree and the trend coupling degree and then normalizing the sum.

[0011] Optionally, the process of generating the seamless advertising recommendation insertion set includes: For each candidate ad in the personalized candidate ad set, the real-time encoding parameter set of the target streaming media content is extracted, and the candidate ad is transcoded in real time to make the encoding parameters consistent with the target streaming media content, thus obtaining an encoding-adapted ad. The consecutive frames before the advertisement insertion point in the target streaming media content are gradually blended with the starting consecutive frames of the encoded and adapted advertisement to generate a transition frame segment. The encoded and adapted advertisements are timestamped and their order is arranged with the transition frame segments to form the seamless advertisement recommendation insertion set.

[0012] Optionally, the real-time transcoding process includes: By parsing the original list of the target streaming media content, the real-time encoding parameter set of the current segment is obtained. The real-time encoding parameter set includes at least resolution, frame rate, bit rate, and encoding profile / level. Based on the real-time encoding parameter set, the candidate advertisement is decoded and then re-encoded using an adaptive recoding algorithm to make the encoding parameters of the candidate advertisement consistent with the real-time encoding parameter set, thus obtaining the encoded adapted advertisement; During the re-encoding process, the GOP structure, color space, and dynamic range metadata of the candidate advertisement are simultaneously mapped to the same format as the parameters corresponding to the target streaming media content.

[0013] Optionally, the gradient blending includes: By performing a preset ratio of alpha fusion on a pixel-by-pixel basis on the last M frames before the advertisement insertion point in the target streaming media content and the pixel data of the M frames before the advertisement starts, a gradual transition frame sequence with controllable duration is generated. The encoding parameters of the gradual transition frame sequence are set to be consistent with the real-time encoding parameter set, and inserted as an independent segment between the target streaming media content and the encoding adaptation advertisement to obtain the transition frame segment.

[0014] Secondly, this application provides a seamless streaming media advertising insertion system based on manifest rewriting, the system comprising: The context-aware module is used to acquire information about a predetermined segment before the advertisement insertion point in the streaming media content, and to perform multi-dimensional perceptual context analysis based on the predetermined segment information to generate a perceptual context descriptor set. The perception matching module is used to perform visual prosody alignment, auditory feature fusion and narrative flow association based on the perception context descriptor set, and to filter advertisements that meet the preset perception association conditions from the candidate advertisement library to generate a personalized candidate advertisement set. The encoding and synthesis module is used to perform context-aware dynamic adaptation of encoding parameters and intelligent synthesis of transition frames based on the personalized candidate ad set, so as to generate a seamless ad recommendation insertion set. The list update module is used to embed the seamless ad recommendation insertion set into the original list of the streaming media content, replace the corresponding ad placeholders, and output the updated seamless ad playback list.

[0015] This application provides a method and system for seamless insertion of streaming media advertisements based on list rewriting. The scheme obtains information from predetermined segments preceding the advertisement insertion point in the streaming media content and performs multi-dimensional perceptual context analysis to generate a perceptual context descriptor set. Then, based on this descriptor set, it performs personalized perceptual matching based on visual rhythm alignment, auditory feature fusion, and narrative flow association, filtering advertisements from a candidate advertisement library that meet preset perceptual association conditions to generate a personalized candidate advertisement set. On this basis, by dynamically adapting context-aware encoding parameters and intelligently synthesizing transition frames on the personalized candidate advertisement set, a seamless advertisement recommendation insertion set is generated. This insertion set is then embedded into the original list of the streaming media content, replacing the corresponding advertisement placeholders, thereby outputting an updated seamless advertisement playback list. This series of steps ensures that the selected advertisements maintain a high degree of consistency with the currently playing content in terms of visual tone, motion rhythm, audio loudness, and narrative emotion, avoiding user perception interruptions caused by abrupt style changes. Simultaneously, dynamically adapting encoding parameters eliminates decoder restarts, black screens, or stuttering caused by inconsistencies in parameters such as resolution and frame rate, while intelligent synthesis of transition frames achieves smooth and gradual transitions between scenes. This effectively solves the problems of inconsistent perception, poor technical integration, and abrupt transitions in the existing streaming media advertising insertion process, improves the continuity and stability of streaming media playback and the user's immersive viewing experience, and ensures seamless integration of advertising content with the main video stream at both the sensory and technical levels.

[0016] In summary, this application constructs a complete technical closed loop, from multimodal perception analysis to personalized matching, and then to technical parameter adaptation and visual transition synthesis. This solution not only achieves deep integration of advertising content and the streaming media environment in multiple senses, but also ensures the smoothness of the playback process from the underlying encoding and frame-level processing, demonstrating significant systemic advantages and practical application value. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram illustrating an application scenario provided in one embodiment of this application; Figure 2 A flowchart illustrating a method for seamless insertion of streaming media advertisements based on manifest rewriting, provided in one embodiment of this application; Figure 3 This is a schematic diagram of the structure of a streaming media advertising seamless insertion system based on list rewriting, provided as an embodiment of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0020] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article, unless otherwise specified, generally indicates that the preceding and following related objects have an "or" relationship.

[0021] The embodiments of this application will now be described in further detail with reference to the accompanying drawings.

[0022] Figure 1 This application provides an application scenario illustration, illustrating the following process: The streaming media player generates an HLS list containing SCTE-35 placeholders; the terminal player requests the list, which is forwarded to the SSAI center via the CDN network; SSAI retrieves the original list, requests personalized ads from the advertising system, replaces the placeholders, and rewrites the segment sequence; finally, SSAI returns the rewritten list to the terminal player via the CDN network, and the terminal player retrieves the segments in sequence. In this application, Figure 1 The illustrated process provides the execution platform for the entire method, enabling dynamic adaptation and seamless integration of ad content encoding. Specifically, after SSAI obtains the real-time encoding parameter set from the original list, it calls the adaptive recoding algorithm to decode and parameterize the candidate ads. During the recoding process, it simultaneously completes the mapping and correction of the GOP structure, color space, and dynamic range, ensuring that the final replaced segments are consistent with the main content in terms of encoding characteristics. Therefore, Figure 1 The segmented ads retrieved by the mid-terminal player not only enable personalized ad insertion but also avoid visual abrupt changes caused by differences in encoding parameters, thus improving the user experience.

[0023] The specific implementation method can be referred to in the following embodiments, wherein the data mentioned in the embodiments are only for reference and examples, so that relevant personnel can better understand them.

[0024] Figure 2 This is a flowchart illustrating a method for seamless insertion of streaming media advertisements based on manifest rewriting, provided as an embodiment of this application. The method of this embodiment can be applied to servers in the above scenarios. (See also...) Figure 2The process, specifically the implementation steps, are as follows: Example 1: In current streaming services, ad insertion primarily employs two methods: client-side insertion and server-side insertion (SSAI). While SSAI technology achieves ad integration by replacing ad placeholders in the original playlist, it still has significant shortcomings in practical applications. First, existing ad selection mechanisms are mostly based on user profiles or advertiser targeting strategies, lacking real-time perception of the visual, auditory, and narrative aspects of the currently playing content. This results in abrupt ad insertions, disrupting the user's immersion. Second, when the encoding parameters (such as resolution, frame rate, bitrate, etc.) of ad segments and content segments are inconsistent, the player often needs to reinitialize the decoder, leading to technical issues such as black screens, stuttering, or audio-visual desynchronization. Furthermore, the lack of smooth visual transitions between ads and content, with hard cuts or simple flashing white / black frames further reducing the smoothness of the viewing experience.

[0025] To address the aforementioned issues, this application provides a method for seamless insertion of streaming media advertisements based on manifest rewriting, the method comprising: Step 1: Obtain the information of the pre-defined segments before the ad insertion point in the streaming media content; based on the pre-defined segment information, perform multi-dimensional perceptual context analysis to generate a perceptual context descriptor set. The predetermined segment information refers to the content data extracted from the streaming media playlist after locating the advertisement insertion point (e.g., the SCTE-35 marker position) for a predetermined duration (e.g., 3 to 5 seconds) before that point in time. Its sources include video streams, audio streams, and subtitle or metadata streams. Multi-dimensional perceptual context analysis refers to the process of extracting features from three dimensions—visual, auditory, and narrative—using computer vision, audio signal processing, and natural language processing technologies. Specifically, by analyzing the pixel statistical features of consecutive frame sequences in the video stream, the dominant color scheme, motion rhythm, and illumination distribution of the scene are extracted to generate visual prosodic features; by analyzing the spectrum and loudness temporal sequence of the audio stream, the beat and volume change patterns of the background music are extracted to generate auditory rhythm features; and by parsing the natural language semantics of subtitles or metadata streams, the emotional polarity, thematic keywords, and narrative rhythm in the dialogue or narration are extracted to generate narrative flow features. The perceptual context descriptor set is a data set that integrates the above visual prosodic features, auditory rhythm features, and narrative flow features through structured association, used to comprehensively describe the multimodal perceptual environment of the current content segment. For example, the system can call a video analytics interface to calculate the HSV color histogram of the last 5 seconds of content frames to determine the dominant color scheme, use optical flow to calculate motion vector amplitude to characterize the motion rhythm, and simultaneously transcribe narration text using ASR and identify its emotional polarity as positive or negative. Through this multi-dimensional feature extraction and fusion, a digital model that accurately reflects the current content atmosphere can be constructed, providing detailed perceptual basis for subsequent ad matching. This step aims to transform unstructured streaming media content into structured perceptual description information, solving the technical problem of ad selection ignoring content context perception in existing technologies.

[0026] Step 2: Based on the perceptual context descriptor set, perform visual prosody alignment, auditory feature fusion, and narrative flow association based on personalized perceptual matching of advertisements, and select advertisements that meet the preset perceptual association conditions from the candidate advertisement library to generate a personalized candidate advertisement set. Visual rhythm alignment refers to calculating the similarity between candidate ads and current content in terms of primary color scheme, motion rhythm, and illumination, prioritizing ads with similar visual styles to ensure visual continuity during screen transitions. Auditory feature fusion refers to matching candidate ads with current content in terms of audio spectrum, loudness envelope, and background music rhythm to ensure a natural and smooth sound transition. Narrative flow association refers to analyzing the emotional tone and semantic keywords of content segments to select ads that can form a connection or continuation in terms of dialogue, scene atmosphere, or emotional evolution. Preset perceptual association conditions refer to the matching degree thresholds or weighted comprehensive scoring standards set for the above three dimensions. Only when a candidate ad exceeds the standard in at least one dimension or the comprehensive dimension is it considered to meet the conditions. For example, if the current content segment is a tranquil sunset at the beach, the visual rhythm features are displayed as warm colors and low motion rhythm, and the auditory features are the soothing sound of waves, then the system will select travel ads from the ad library that also have warm colors, slow rhythm, and soft background sounds, while excluding fast-paced, cool-toned technology product ads. By synergistically matching visual, auditory, and narrative dimensions, the system can accurately identify ad creatives from a massive pool of candidate ads that highly integrate with the current content in terms of sensory experience, thus creating a personalized candidate ad set. This step expands the ad selection logic from a single user profile to real-time perception of the content environment, reducing user bounce caused by stylistic inconsistencies and improving ad acceptance.

[0027] Step 3: Based on the personalized candidate ad set, perform context-aware dynamic adaptation of encoding parameters and intelligent synthesis of transition frames to generate a seamless ad recommendation insertion set; Dynamic adaptation of encoding parameters refers to extracting the real-time encoding parameter set (at least including resolution, frame rate, bit rate, and encoding profile / level) of the target streaming media content at the current ad insertion point, and performing real-time transcoding on the selected candidate ads to ensure that all encoding parameters are completely consistent with the content stream. Intelligent synthesis of transition frames refers to gradually blending the last consecutive frames of the content before the ad insertion point with the first consecutive consecutive frames of the transcoded ad to generate a transition frame segment with controllable duration. Specifically, the system can parse the metadata of the current content segment to obtain its encoding format as H.264 High Profile, resolution 1920x1080, and frame rate 25fps. It then starts an adaptive recoding algorithm to decode and re-encode the candidate ads, synchronously mapping their GOP structure, color space, and dynamic range metadata. Based on this, the last M frames (e.g., 5 frames) of the content and the M frames before the ad are taken and alpha-fused pixel by pixel at a preset ratio to generate a transition frame sequence with cross-fading effect, and the encoding parameters of this sequence are set to be consistent with the content before and after. For example, when the content ends with a bright image while the ad begins with a dark image, the generated transition frame sequence will present a smooth gradient effect with gradually decreasing brightness, rather than an abrupt change. Through dynamic unification of encoding parameters and intelligent synthesis of transition frames, the decoder restart and black screen phenomenon caused by parameter mismatch is eliminated, and the visual traces of scene transitions are smoothed out. This step ensures that the ad seamlessly integrates with the original content at both the technical and presentation levels, generating a seamless ad recommendation insertion set that includes adapted ad segments and transition frame segments.

[0028] Step 4: Insert the seamless ad recommendations into the original list of embedded streaming media content, replace the corresponding ad placeholders, and output the updated seamless ad playback list.

[0029] The original playlist can be a streaming media playlist file conforming to the HLS (M3U8) or DASH (MPD) standards, containing placeholders (such as SCTE-35 markers) indicating ad insertion locations. The embedding process involves writing the segment URLs (including transition frame segments and encoded ad segments) from the generated seamless ad recommendation insertion set into the original playlist, replacing the original placeholder positions. Outputting the updated seamless ad playlist involves recalculating and calibrating the timestamp sequence (PTS / DTS) in the playlist to ensure the total duration tag (EXTINF) matches the actual segment duration, generating the final executable playlist. For example, the system replaces markers pointing to empty ad slots in the original playlist with URL sequences pointing to transition frame segments and transcoded ad segments in the temporary cache, and adjusts the start timestamps of subsequent content segments to ensure continuity. Finally, the updated playlist is delivered to the CDN network and distributed to the terminal player. This step utilizes a standard manifest rewriting mechanism to achieve seamless injection of advertising content without modifying the player kernel, so that end users cannot perceive any technical traces of ad insertion during playback.

[0030] This application achieves truly seamless insertion of streaming media advertisements through the synergistic effect of the aforementioned technical features. Specifically, a content environment model constructed through multi-dimensional perceptual context analysis, in conjunction with visual rhythm alignment, auditory feature fusion, and narrative flow association strategies, ensures a natural extension of the advertising material into the current content at the sensory level, resolving the issue of perceptual discontinuity. Furthermore, a dynamic adaptation mechanism for encoding parameters adjusts the advertising encoding specifications according to the real-time content stream, eliminating black screens and stuttering caused by parameter differences. Meanwhile, intelligent transition frame synthesis technology further provides a smooth gradient buffer at visual boundaries. These three elements work together to enhance the smoothness and immersion of the user's viewing experience, from perceptual matching to technical adaptation to visual transition, making the advertisement an integral part of the content rather than an interruption.

[0031] Example 2: Another optional embodiment of this application provides a process for generating a perceptual context descriptor set, the method comprising the following steps: Step 1: The pre-defined segment information includes the video stream, audio stream, and subtitle / metadata stream; The predetermined segment information refers to a continuous segment of media data located from the original streaming content list before the ad insertion point. Its source is typically the most recent content segments cached on a CDN or fetched remotely. This information specifically includes three parallel data streams: a video stream carrying the pixel matrix sequence of the image, used to characterize the dynamic changes in the visual scene; an audio stream carrying time-varying sound wave signals, used to characterize the auditory characteristics of ambient sounds and background music; and a subtitle / metadata stream carrying text information or structured descriptive data related to natural language, used to characterize the semantic logic and emotional direction of the content. These three data streams are strictly synchronized on the timeline, collectively forming the basic input source for subsequent multi-dimensional perceptual analysis. For example, when the ad insertion point is at 300 seconds into the video, the predetermined segment information can be extracted as the H.264 encoded video frame sequence, the AAC encoded audio sampling sequence, and the corresponding WebVTT subtitle file or ID3 tag data between 295 seconds and 300 seconds. By acquiring multimodal data in parallel, we can comprehensively cover the visual, auditory, and cognitive dimensions of users when they watch, providing complete data support for generating a high-precision set of perceptual context descriptors.

[0032] Step 2: By analyzing the pixel statistical features of consecutive frame sequences in the video stream, extract the main color scheme, motion rhythm, and illumination distribution of the scene to generate visual rhythm features; Visual rhythm features are a quantitative description of the style and dynamic trends of a video stream, and their generation relies on in-depth analysis of the statistical features of pixels in a continuous frame sequence. The dominant color scheme is obtained by calculating the peak distribution of the HSV color histogram of consecutive frames, reflecting the overall tonal tendency of the scene. Motion rhythm is obtained by calculating the mean and variance of the amplitude of pixel displacement vectors between adjacent frames using optical flow, characterizing the dynamic intensity of the scene content. Illumination distribution is obtained by analyzing the statistical moments (such as mean and standard deviation) of pixel brightness channels, describing the brightness levels of the scene. These features work together to enable the system to capture the visual breath of the video content. For example, if the predetermined segment is a nighttime car chase scene, the analyzed dominant color scheme might be a deep blue tone (with the highest blue component in the RGB ratio), the motion rhythm would be characterized by rapid fluctuations with high amplitude (mean optical flow vector greater than threshold T1), and the illuminance distribution would exhibit low mean and high contrast. The visual rhythm features generated based on the above extraction method can accurately depict the visual atmosphere of the current scene, ensuring that the subsequently selected advertisements maintain a high degree of consistency with the main content in terms of color tone and dynamic rhythm, and avoiding visual abruptness.

[0033] Step 3: By analyzing the audio spectrum and loudness sequence of the audio stream, extract the beat and volume change patterns of the background music to generate auditory rhythm features; Among them, auditory rhythm features are a comprehensive abstraction of the time and frequency domain characteristics of audio streams, aiming to reconstruct the dynamic contours of the sound environment. The beat of background music is obtained by detecting the periodic intervals of frequency band energy peaks after performing a short-time Fourier transform (STFT) on the audio spectrum, usually expressed in BPM (beats per minute). The volume variation pattern is obtained by calculating the first derivative and envelope of the loudness time-series curve (such as the loudness value under the LUFS standard), reflecting the fluctuation pattern of sound intensity. The role of this feature is to identify the emotional tension and rhythm type of the audio, providing a basis for smooth transitions at the auditory level. For example, for an exciting symphonic accompaniment, the extracted beat may be a strong, regular pulse of 140 BPM, and the volume variation pattern is manifested as a large and rapid rise and fall (with a large absolute value of the envelope slope). Through this fine-grained feature extraction, the system can distinguish the auditory differences between quiet dialogue scenes and intense action scenes, and then match advertising audio with similar acoustic characteristics, eliminating auditory discomfort caused by sudden loudness changes or rhythmic breaks.

[0034] Step 4: By parsing the natural language semantics of the subtitle / metadata stream, extract the emotional polarity, thematic keywords, and narrative rhythm from the lines or narration to generate narrative flow features; Narrative flow features are derived from the deep semantic understanding and structured expression of subtitles or metadata text, used to grasp the plot's development. Emotional polarity is obtained by scoring dialogue or narration sentences using a pre-trained sentiment analysis model, resulting in positive, negative, or neutral values ​​and their intensity. Thematic keywords are a set of high-frequency content words or proper nouns extracted from the text using natural language processing techniques (such as TF-IDF algorithms or word vector clustering), representing the core topic of the current discussion. Narrative rhythm is calculated based on sentence length, punctuation density, and the frequency of semantic transition words, reflecting the pace of the plot. This feature allows ad matching to go beyond surface sensory perception and delve into the story's logic. For example, if the subtitle content is "Finally arrived at the long-awaited beach, feeling incredibly happy," the analyzed emotional polarity is high-intensity positive, thematic keywords include "beach" and "vacation," and the narrative rhythm is leisurely. Based on this generated narrative flow feature, the system can guide the selection of ads that also convey a relaxed, travel theme, achieving a natural transition between ad content and the main plot, enhancing user immersion.

[0035] Step 5: Structure and link visual prosodic features, auditory rhythmic features, and narrative flow features to form a set of perceptual context descriptors.

[0036] The perceptual context descriptor set is a standardized fusion product of the aforementioned three types of heterogeneous features. Its integration process achieves structured association by establishing a unified data structure (such as a JSON object or protobuf message). Specifically, the system maps visual prosodic features to the `visual` field of the descriptor, auditory rhythm features to the `audio` field, and narrative flow features to the `narrative` field, attaching a timestamp index to each field to ensure temporal alignment. This structured association not only preserves the independence of each modal feature but also establishes cross-modal reference relationships, enabling subsequent matching algorithms to simultaneously call multi-dimensional data for joint inference. For example, the final generated descriptor set can be represented as: `{visual: {color: #0000FF, motion: 0.8, light: 0.2}, audio: {bpm: 140, volume_pattern: dynamic},narrative: {sentiment: 0.9, keywords: [race, speed], pace: fast}}`. This integration approach constructs a digital twin that comprehensively reflects the current streaming media content perception environment, providing a precise query index and matching benchmark for the next step of personalized advertising filtering based on multi-dimensional perception, and improving the contextual adaptability of advertising recommendations.

[0037] This application achieves a technical leap from single-modal to multi-modal perception through the parallel parsing of video streams, audio streams, and subtitle / metadata streams in the aforementioned steps. Visual prosodic features, auditory rhythm features, and narrative flow features do not exist in isolation but complement each other through structured association: visual features capture visual style, auditory features ensure sound coherence, and narrative features understand semantic logic. These three elements work synergistically to construct a high-fidelity perceptual context model, enabling the subsequently generated personalized candidate ad set to seamlessly integrate with streaming media content across multiple dimensions such as tone, rhythm, and emotion. This solves the technical problems of abrupt ad insertion and perceptual discontinuity in existing technologies.

[0038] Example 3: In another optional embodiment, this application provides a process for generating a personalized candidate advertisement set, which further includes a step of multi-dimensional matching and filtering based on a set of perceptual context descriptors.

[0039] Step 1: Based on the visual prosodic features, auditory rhythm features, and narrative flow features in the perceptual context descriptor set, analyze the visual prosodic matching degree, auditory feature matching degree, and narrative flow correlation degree of each candidate advertisement respectively. This step aims to utilize the structured, perceptual environmental data generated in the previous steps to conduct a multi-dimensional quantitative evaluation of each ad creative in the candidate ad library. Visual rhythm matching refers to the similarity between the candidate ad and the predetermined segment of the streaming media content in terms of color tone, motion rhythm, and illumination distribution. Its value is calculated by dynamically comparing the visual feature sequences of the ad's starting frame and the content's ending frame, representing the visual stylistic coherence between the two. Auditory feature matching refers to the degree of integration between the candidate ad and the predetermined segment of the streaming media content in terms of audio spectrum, loudness envelope, and background music mood. Its value is derived by calculating the spectral centroid distance, timbre similarity, and loudness correlation of audio features, combined with the emotion tag matching results, ensuring a natural and smooth auditory transition. Narrative flow relevance refers to the semantic connection between the candidate ad and the predetermined segment of the streaming media content in terms of emotional polarity, thematic keywords, and narrative rhythm. Its value is obtained by weighted summation of the emotional intensity difference of dialogue or narration, keyword overlap, and the consistency of narrative arc trends, achieving a deep resonance between the ad content and the plot logic. These three dimensions of matching are independent yet complementary. Visual rhythm matching focuses on the immediate connection of visual senses, auditory feature matching focuses on the seamless integration of the sound environment, and narrative flow relevance focuses on the deep resonance of the content logic. For example, when a streaming media content segment is a fast-paced action scene, if the starting screen of a candidate ad has a highly saturated primary color scheme and a large motion vector amplitude, its visual rhythm matching is high; if the beat (BPM) of the background music in the ad is highly consistent with the content segment and the loudness change trend is similar, its auditory feature matching is high; if the ad's dialogue continues the thematic keywords such as striving and victory in the content and the emotional polarity is also positive, its narrative flow relevance is high. Through this multimodal parallel analysis mechanism, the points of convergence between the ad and the content at different perceptual levels can be comprehensively captured, avoiding matching bias caused by single-dimensional evaluation, thus providing accurate data support for subsequent screening.

[0040] Step 2: Select candidate ads that exceed the preset candidate threshold in at least one of the visual rhythm matching degree, auditory feature matching degree, and narrative flow relevance degree, and form a personalized candidate ad set as ads that meet the preset perceptual relevance conditions.

[0041] This step efficiently recalls a subset of ads that meet the requirements for seamless insertion from a massive pool of candidate ads by setting flexible logical judgment conditions. The preset candidate threshold, an empirical value determined based on historical user behavior data and experimental testing, defines the minimum standard for an ad to achieve an acceptable or excellent match with content in a certain dimension. It can be dynamically adjusted according to different streaming media content types (such as movies, news, and short videos) or different advertising strategies. Meeting the preset perceptual relevance condition means that a candidate ad is considered qualified if at least one of the visual, auditory, or narrative dimensions has a match value greater than or equal to the preset candidate threshold; it is not mandatory for all three dimensions to meet the standard simultaneously. This logical filtering mechanism effectively expands the recall scope of candidate ads, preventing situations where no ads are available in specific content scenarios due to overly stringent all-dimensional matching requirements, while also retaining ads that perform well in a strongly correlated dimension to accommodate diverse user perceptual preferences. The personalized candidate ad set is a collection of all candidate ads that meet the above conditions. This set serves as the input source for subsequent encoding adaptation and transition frame synthesis, ensuring that the finally inserted ad has a high degree of perceptual adaptability. For example, setting a preset candidate threshold of 0.75 (normalized score), if a candidate ad has a visual prosodic matching degree of 0.82, even if its auditory and narrative matching degrees are only 0.60, it will still be included in the personalized candidate ad set because its visual dimension exceeds the threshold. Conversely, if another candidate ad has matching degrees of 0.70, 0.65, and 0.68 respectively, none of which exceed the threshold, it will be eliminated. Based on this, by constructing this personalized candidate ad set, the system can maximize the utilization of ad resources while ensuring basic perceptual consistency, and provide a rich selection space for selecting the optimal ad for real-time transcoding and synthesis in subsequent steps, thereby improving the flexibility and success rate of ad insertion.

[0042] This application achieves a leap from single linear matching to multimodal stereoscopic perception through the synergistic effect of visual rhythm matching, auditory feature matching, and narrative flow relevance in the aforementioned steps. By combining the independent evaluation results of the three dimensions with a screening logic that exceeds at least one threshold, it not only solves the problem of abrupt transitions caused by traditional technologies relying solely on user profiles while ignoring the content environment, but also avoids the predicament of low ad recall caused by strict full-dimensional matching. The alignment of visual rhythms ensures the visual continuity of scene transitions, the fusion of auditory features guarantees a smooth transition of the sound environment, and the relevance of the narrative flow deepens the emotional resonance of the content logic. Under the combined effect of these three factors, the selected personalized candidate ad set can form a natural extension with the streaming media content on multiple perceptual levels. Furthermore, the subsequent processing based on this candidate set can eliminate the sense of disconnect experienced by users during viewing, making the ads an organic part of the content experience, improving the user's immersive viewing experience and ad acceptance.

[0043] Example 4: One possible implementation, according to an embodiment of this application, is a visual prosodic matching degree analysis process, which includes the following steps: Step 1: Extract the main color sequence, motion vector amplitude sequence, and brightness mean sequence of the last N frames of the predetermined segment from the visual prosody features to form the source visual prosody curve group; The predetermined segment refers to a continuous video clip before the advertisement insertion point in streaming content, typically 3 to 5 seconds in length, to ensure the complete visual context before the scene transition is captured. The dominant color sequence is time-series data generated by calculating the HSV color histogram of each frame, extracting the most prevalent color component, and converting it to RGB values. The motion vector magnitude sequence is obtained by calculating the displacement vector magnitudes of pixels between adjacent frames using optical flow, representing the dynamic intensity of the scene content. The brightness mean sequence is the arithmetic mean of the grayscale values ​​of all pixels in each frame, reflecting the changing trend of scene illumination. These three sequences together constitute the source visual rhythm curve set, comprehensively describing the visual style tone at the end of the content segment. For example, when N is 10, the system extracts the above three types of data from the last 10 frames before the insertion point. If the segment is a nighttime chase scene, the dominant color sequence will show a deep blue tone, the motion vector magnitude sequence will show high-frequency, large fluctuations, while the brightness mean sequence will remain at a low level. This multi-dimensional feature extraction transforms abstract visual perceptions into quantifiable mathematical sequences, providing a precise data foundation for subsequent similarity calculations.

[0044] Step 2: Extract the corresponding sequence of the first N frames from the visual tone of each candidate advertisement to form a candidate visual rhythm curve group. The candidate ads come from a pre-established ad creative library. During the library entry or real-time request phase, the system performs the same feature extraction operation as described above for each candidate ad, but focuses on the beginning of the ad. This is because the first impression when an ad is inserted is crucial to the continuity of user perception; the visual features of the first N frames directly determine whether the ad's entry point conflicts with the previous scene. The corresponding sequence includes the dominant color scheme, motion vector amplitude, and average brightness data consistent with the dimensions of the source visual rhythm curve group. Specifically, the system iterates through each ad in the personalized candidate ad set, extracts the first N frames of its video stream, and calculates its color distribution, motion intensity, and illumination level to form an independent candidate visual rhythm curve group. Taking a car ad as an example, if its opening is a high-speed driving scene, the motion vector amplitude sequence in its candidate visual rhythm curve group will show high-order values. If the ending of the content segment is also a high-speed movement scene, then the two have potential matching in the motion dimension. This process ensures strict alignment of the comparison objects in terms of time scale and feature dimension.

[0045] Step 3: Calculate the cumulative distance between the two sets of curves using the dynamic time warping algorithm, and transform its reciprocal into visual prosodic dynamic similarity; Dynamic Time Warping (DTW) is a classic algorithm for measuring the similarity between two time series. Its core advantage lies in its ability to handle sequence alignment problems involving inconsistent lengths or non-linear scaling of the time axis. In streaming media scenarios, content segments and advertising segments may have different frame rates, or the rhythm of motion within the scene may vary. Traditional Euclidean distance cannot effectively capture such flexible matching on the time axis. The DTW algorithm constructs a cost matrix to find the optimal alignment path between two curves and calculates the minimum cumulative distortion distance. The smaller the cumulative distance, the closer the shapes of the two curves are. To conform to the intuitive logic that higher matching scores result in higher numerical values, the system performs a reciprocal transformation (or negative exponential transformation) on the calculated cumulative distance to obtain the visual rhythm dynamic similarity. For example, if the end of a content segment is a slowly zooming-in close-up (with the motion vector gradually decreasing), and the beginning of a candidate advertisement is also a similar slowly zooming-in shot, although the durations of the two are slightly different, the DTW algorithm can still identify the high consistency in their motion trends and give a high similarity score. This step effectively solves the problem of misjudging visual rhythm caused by fine-tuning of playback speed or frame rate conversion.

[0046] Step 4: Simultaneously, analyze the difference in shot switching frequency between the last N frames of the predetermined segment and the first N frames of the candidate advertisement to obtain the shot rhythm consistency. Shot switching frequency refers to the number of hard cuts or soft transitions in a scene per unit of time, and is a key macro-indicator for measuring the rhythm of video editing. This step aims to evaluate the coordination of the content and advertisement in terms of narrative rhythm at a coarser level. The system calculates the switching frequency by detecting the number of scene boundaries in the last N frames of a predetermined segment; similarly, it counts the number of scene boundaries in the first N frames of a candidate advertisement. Subsequently, the absolute value of the difference between the two frequencies is calculated and mapped to shot rhythm consistency using a preset decay function (such as a Gaussian function). If the switching frequencies of the two are similar, for example, both are fast-paced action clips (2-3 switches per second) or both are long, lyrical shots (one switch every 5 seconds), the difference is small and the consistency is high; conversely, if the content ends with a gentle long shot while the advertisement begins with a rapid flash cut, the difference is large and the consistency is low. For example, at the end of an intense fight scene in an action movie, the shot switching frequency is extremely high. If a slow-paced landscape advertisement is matched at this time, the shot rhythm consistency will be significantly reduced, indicating that the two are incompatible in terms of editing style. This metric serves as an important supplement to micro-pixel-level similarity, preventing situations where pixels are similar but the rhythm is abrupt.

[0047] Step 5: Multiply the visual rhythm dynamic similarity by the shot rhythm consistency to obtain the visual rhythm matching degree.

[0048] The multiplication operation is a non-linear fusion strategy designed to emphasize the synergistic constraint of two dimensions. A high visual rhythm matching score is achieved only when both the micro-level visual element evolution trend (characterized by visual rhythmic dynamic similarity) and the macro-level editing rhythm (characterized by shot rhythm consistency) simultaneously meet the matching conditions. If the matching score of either dimension is low (close to 0), the product result will significantly decrease, automatically eliminating the candidate ad during the screening process. This mechanism ensures that the selected ads not only smoothly connect with the previous scene in terms of color, brightness, and motion trajectory, but also maintain consistency in the overall viewing rhythm, avoiding user discomfort caused by sudden breaks in visual style. Through the above calculations, the system can generate a quantitative visual rhythm matching score for each candidate ad. This score will serve as one of the core criteria for subsequent screening of personalized candidate ad sets, ensuring a seamless transition of ad insertion points at the visual perception level.

[0049] This application achieves refined calculation of visual rhythm matching degree through the above steps. By introducing a dynamic time warping algorithm to process nonlinear visual feature sequences and combining it with macro-rhythm analysis of shot switching frequency, a multi-level visual matching model is constructed. The product operation of visual rhythm dynamic similarity and shot rhythm consistency creates a strong coupling between micro-pixel changes and macro-editing rhythm. Any mismatch in any single dimension will lead to a decrease in the overall score, thus strictly ensuring a high degree of consistency between the advertisement and the content in visual style. This dual verification mechanism solves the problem of abrupt switching caused by relying solely on single-frame color or simple motion vector matching in existing technologies, allowing advertisements to naturally integrate into the current visual flow and enhancing the user's immersive viewing experience.

[0050] Example 5: Another optional embodiment of this application provides an analysis process for auditory feature matching degree, the method including the following steps: Step 1: Extract the frequency band energy distribution, timbre feature vector and loudness envelope curve of the continuous audio at the end of the predetermined segment from the auditory rhythm features to form the source auditory feature set, and extract the corresponding features of the starting continuous audio from the audio features of each candidate advertisement to form the candidate auditory feature set. The source auditory feature set refers to a structured data set used to characterize the acoustic environment of streaming media content at the instant before the ad insertion point. It originates from the real-time analysis of continuous audio signals at the end of a predetermined segment (e.g., 3 to 5 seconds before the ad insertion point). Specifically, the frequency band energy distribution is obtained by statistically analyzing the energy proportions of different frequency bands (e.g., low frequency 20-250Hz, mid frequency 250-4kHz, high frequency 4kHz-20kHz) after performing a Fast Fourier Transform (FFT) or wavelet transform on the audio signal, reflecting the thickness or sharpness of the sound; the timbre feature vector is a high-dimensional vector extracted based on Mel-frequency cepstral coefficients (MFCCs), used to characterize the texture of the sound and the characteristics of instruments / voices; the loudness envelope curve is a time-varying amplitude sequence obtained by calculating short-time energy and smoothing it, used to describe the dynamics of sound intensity. The candidate auditory feature set consists of similar feature data extracted in advance from the initial continuous audio of each ad in the candidate ad library (e.g., the first 3 seconds of the ad). The construction of the source auditory feature set and the candidate auditory feature set is the foundation for subsequent matching degree calculation. The two sets maintain strict consistency in dimensionality and physical meaning to ensure comparability. For example, when the predetermined segment ends with a stirring symphonic climax, the high-frequency energy proportion in its source auditory feature set is significantly higher, and the loudness envelope curve shows a rapid upward trend. In this case, the system extracts the corresponding features from the starting parts of each advertisement in the candidate advertisement library. If the starting part of a car advertisement also has rich high-frequency and increasing loudness features, then its candidate auditory feature set is highly similar to the source auditory feature set in physical form. Through this multi-dimensional feature extraction, the physical properties of the audio can be comprehensively captured, providing accurate data input for subsequent quantitative matching.

[0051] Step 2: Calculate the spectral centroid distance, timbre cosine similarity, and loudness Pearson correlation coefficient between the two feature sets, and combine these three values ​​after normalization to obtain the auditory feature fusion degree; Auditory feature fusion is a comprehensive quantitative indicator used to measure the physical acoustic similarity between the source and candidate auditory feature sets. This indicator is composed of calculations from three sub-dimensions: the spectral centroid distance reflects the difference in frequency centroids between two audio segments, with smaller distances indicating closer tonal timbre; the timbre cosine similarity is obtained by calculating the cosine of the angle between two timbre feature vectors in multidimensional space, with values ​​closer to 1 indicating more similar timbre texture; and the loudness Pearson correlation coefficient is used to assess the linear correlation between two loudness envelope curves, with higher coefficients indicating more synchronized volume changes. In the implementation, these three values ​​are first calculated separately. Due to their different dimensions and ranges (e.g., distance is a non-negative real number, similarity and correlation coefficients are [-1,1] or [0,1]), normalization (e.g., Min-Max normalization or Z-Score standardization) is required to map them to a unified [0,1] interval. Then, the final auditory feature fusion is obtained by weighted summation or geometric mean. This process represents a shift from single-feature comparison to multi-feature fusion. For example, if the normalized centroid distance between the source and candidate audio frequencies is 0.9 (indicating very close proximity), the timbre cosine similarity is 0.85, and the loudness Pearson correlation coefficient is 0.8, and all three are weighted equally, the auditory feature fusion degree can be calculated as the average of the three, 0.85. This fusion mechanism effectively avoids misjudgments caused by relying solely on loudness or pitch matching, improving the natural auditory connection between advertising audio and content audio.

[0052] Step 3: Simultaneously, analyze the degree of matching between the background music emotion tags of the audio at the end of the predetermined segment and the audio at the beginning of the candidate advertisement to obtain the emotion evolution compatibility. Emotional evolution compatibility refers to a semantic-level metric used to assess the emotional coherence between content audio and advertising audio, derived from a deep analysis of the emotional information carried by the audio. Specifically, the system first uses a pre-trained emotion classification model or metadata parsing module to identify the background music emotion tags (such as tension, sadness, joy, calm, etc.) at the end of a predetermined segment, and similarly identifies the emotion tags of the candidate advertising's starting audio. Subsequently, based on a pre-set emotion evolution matrix or semantic distance model, the matching degree between these two tags is analyzed. This matching degree considers not only the similarity of tags but also the rationality of emotional evolution, i.e., whether it conforms to the natural transition laws of human psychological perception. For example, a transition from tension to release or joy is usually considered high compatibility, while a direct jump from extreme sadness to ecstasy may be considered low compatibility. Emotional evolution compatibility complements the aforementioned physical acoustic features; the former addresses whether it sounds similar, while the latter addresses whether it feels right. For example, if the planned segment ends at the climax of a suspenseful plot, with the emotional label being tension, while the candidate advertisement begins with an ad for an energy drink, with the emotional label being excitement or vitality, there is a logical progression from suppression to explosion in the emotional development between the two, thus resulting in a high emotional compatibility score. Conversely, if the candidate advertisement begins with a warm and soothing home goods ad, the emotional discontinuity may lead to a lower compatibility score. By introducing this dimension, we ensure that the ad insertion is not only smooth in terms of acoustic parameters but also seamlessly connected in terms of narrative emotional flow.

[0053] Step 4: Multiply the auditory feature fusion degree by the emotional evolution compatibility degree and take the square root to obtain the auditory feature matching degree.

[0054] The auditory feature matching score is the final decision-making criterion for filtering advertisements. It integrates the fusion at the physical acoustic level and the compatibility at the emotional semantic level. This value is generated by first multiplying the auditory feature fusion score and the emotional evolution compatibility, and then taking the square root of the product. The product operation reflects the "weakest link" effect: only when both physical and emotional features are high will the final matching score be high. The absence of any dimension will significantly lower the overall score, thus filtering out advertisements that, while having similar voices, have jarring emotions, or whose emotions are appropriate but whose voices are incongruous. The square root operation serves as a non-linear smoothing mechanism, moderately improving the discriminative power of medium matching scores. This prevents the overall score from prematurely approaching zero due to extremely low scores in one dimension, while also avoiding over-saturation in the high-score range, resulting in a more even and stable distribution of matching results. The output of this step directly determines whether candidate advertisements meet the preset perceptual association conditions. For example, if a candidate ad has an auditory feature fusion degree of 0.81 and an emotional evolution compatibility degree of 0.64, their product is 0.5184, and the square root yields an auditory feature matching degree of 0.72. Another candidate ad, while having a fusion degree as high as 0.9, has a compatibility degree of only 0.2, resulting in a product of 0.18, and the square root yields only 0.42. Clearly, the former is superior. Through this calculation method, the system can intelligently filter out ads that highly match the listening experience of the streaming media content, eliminating the audio gaps and emotional disjointness during ad insertion and achieving a truly seamless auditory transition.

[0055] This application organically combines the physical attribute analysis of auditory features with emotional semantic analysis through the aforementioned steps. By extracting frequency band energy, timbre vectors, and loudness envelopes to construct source and candidate feature sets, a data foundation for physical matching is established. Then, the spectral centroid distance, timbre cosine similarity, and loudness correlation coefficient are calculated and normalized, achieving quantitative fusion of multi-dimensional acoustic features. Simultaneously, the matching analysis of background music emotion tags is introduced to construct an emotion evolution compatibility, compensating for the shortcomings of pure physical matching. Finally, using the algorithm logic of the square root of the product, the auditory feature fusion degree and emotion evolution compatibility are synergistically combined to generate an auditory feature matching degree that combines acoustic naturalness and emotional coherence. This multi-level matching mechanism not only solves the abruptness caused by inconsistent audio parameters in traditional technologies but also fundamentally avoids user psychological discomfort caused by conflicting emotional logic, ensuring a smooth and continuous auditory experience during the insertion of streaming media advertisements.

[0056] Example 6: One possible implementation, according to embodiments of this application, is a narrative flow correlation analysis process, which includes the following steps: Step 1: Extract the emotional polarity intensity, the set of thematic keywords, and the narrative rhythm type of the last line or narration of the predetermined segment from the narrative flow features to form the source narrative feature group, and extract the corresponding content from the advertising words or scene descriptions of each candidate advertisement to form the candidate narrative feature group. The narrative flow features are structured data obtained through natural language processing of subtitle text or narration text transcribed by Automatic Speech Recognition (ASR) in the preceding steps. Emotional polarity intensity refers to the quantitative value of the emotional tendency expressed in the text, usually calculated using a pre-trained sentiment analysis model, with a value range of [-1, 1], where -1 represents extremely negative, 0 represents neutral, and 1 represents extremely positive. The set of thematic keywords refers to high-frequency words or entity nouns extracted from the text that represent the core semantics of the current scene; for example, in a live sports broadcast, words like "goal," "last-minute winner," and "champion" might be extracted. Narrative rhythm type refers to the narrative style categorized based on sentence length, punctuation density, and speech rate variations, such as a fast-paced and tense style, a gentle narrative style, or a lyrical and contemplative style. The process of constructing the source narrative feature set and the candidate narrative feature set essentially maps unstructured natural language text into a computable vector space representation. For example, when the predetermined segment is a climax in a movie, and the last line is "We finally won," the system extracts an emotional polarity of 0.9, a keyword set of {victory, team, glory}, and a fast-paced, tense narrative style. Simultaneously, extracting the tagline "Witness the Moment of Glory" from a sports brand advertisement in the candidate ad library yields an emotional polarity of 0.85, a keyword set of {glory, breakthrough, limit}, and an inspiring, uplifting narrative style. This feature extraction method transforms abstract narrative content into concrete mathematical features, laying the foundation for subsequent quantitative matching.

[0057] Step 2: Calculate the negative exponential transformation of the difference in emotional polarity intensity between the two feature groups, the overlap of thematic keywords, and the matching degree of narrative rhythm type, and obtain the semantic structure correlation by weighted summation; The semantic structure relevance is used to quantify the static matching degree between source content and candidate ads in terms of surface semantics and emotional tone. The negative exponential transformation of the emotional polarity intensity difference involves calculating the absolute value of the difference in emotional polarity intensity between the source and candidate narrative feature groups and applying a negative exponential operation. The smaller the emotional difference, the closer the score is to 1, thus avoiding the disjointed experience caused by abrupt emotional shifts. The topic keyword overlap is obtained by calculating the ratio of the number of elements in the intersection to the number of elements in the union of two keyword sets (i.e., the Jaccard similarity coefficient), used to measure the continuity of the topic. The narrative rhythm type matching degree is based on a preset rhythm type label table; if the two types are the same, it is recorded as 1; if they are adjacent types (e.g., rapid and intense), it is recorded as 0.8; otherwise, it is recorded as 0. The semantic structure relevance is obtained by multiplying the above three indicators by preset weight coefficients (e.g., emotional weight 0.4, keyword weight 0.4, rhythm weight 0.2) and then summing them. For example, if the source content sentiment is 0.9 and the ad sentiment is 0.85, the difference is 0.05, and after negative exponent transformation, it is approximately 0.95; the keyword overlap is 0.33 (3 words overlap by 1); and the rhythm type perfect match is 1. Assuming the weights are 0.4 / 0.4 / 0.2 respectively, the semantic structure relevance is... This calculation process ensures that the advertisements are highly consistent with the current content in terms of topic and emotional tone, preventing the abrupt transition from a tragic scene to a celebratory advertisement.

[0058] Step 3: Simultaneously, analyze whether the narrative arc trend at the end of the predetermined segment is consistent with the emotional evolution direction conveyed by the candidate advertisement to obtain the trend coupling degree; The narrative arc trend refers to the slope or direction of the change in emotional intensity of a content segment on the timeline, reflecting whether the story is in an upward phase (e.g., from suppression to explosion), a downward phase (e.g., from excitement to disappointment), or a turning point. The emotional evolution direction refers to the path that the candidate advertisement attempts to guide the audience's emotional changes during its playback period. The trend coupling analysis aims to capture deep, dynamic emotional logic, rather than simply matching static points. Specifically, the system analyzes the sign and magnitude of the derivative of the emotional polarity intensity in the last few seconds of a predetermined segment to determine its trend vector; simultaneously, it analyzes the emotional evolution model in the candidate advertisement script or historical viewing data. If the directions of change are consistent (e.g., the content is on the rising edge of emotional height, and the advertisement is designed to transition from calm to a climax), the trend coupling is high; if the directions are opposite (the content is declining, but the advertisement forcibly elevates the emotion), the coupling is low. For example, at the end of a suspense drama, as the protagonist is about to reveal the truth, the narrative arc shows a steep upward trend. If the candidate ad is a car advertisement with a similar process of suspenseful revelation and emotional release, its trend coupling degree will be judged as high. Conversely, if the ad is a straightforward product introduction, the trend coupling degree will be low. By introducing trend coupling degree, this step ensures that the ad is not only a snapshot of the content but also a continuation of the emotional flow of the content.

[0059] Step 4: After weighted summation of semantic structure correlation and trend coupling, normalize the result to obtain narrative flow correlation.

[0060] Narrative flow relevance is a final evaluation metric that integrates static semantic matching and dynamic sentiment trends, used to determine whether candidate ads meet preset perceptual relevance conditions. The semantic structure relevance and trend coupling obtained above are weighted and summed according to a preset ratio (e.g., 6:4 or 5:5), and then normalized to map them to the standard range of [0,1]. This process integrates micro-level lexical sentiment matching with macro-level narrative momentum alignment, ensuring that the final selected ads both address current hot topics and align with the audience's emotional flow. The narrative flow relevance generated in this way can effectively guide the ad selection system to eliminate ads that, while matching keywords, have conflicting emotional trends, thereby achieving seamless integration of ads and streaming media content at the narrative level and enhancing the user's immersive experience.

[0061] This application achieves a deep narrative connection between advertisements and streaming media content through the aforementioned steps. By extracting feature groups of emotional polarity intensity, thematic keyword set, and narrative rhythm type to form source and candidate features, and using negative exponential transformation of emotional difference, keyword overlap, and rhythm matching degree to calculate semantic structure correlation, it ensures a close connection between the advertisement and the current content at the surface semantic level. On this basis, it further analyzes the consistency between the narrative arc trend and the direction of emotional evolution to obtain trend coupling degree, capturing the dynamic trajectory of the content's emotional development. The weighted fusion of semantic structure correlation degree and trend coupling degree ensures that the final generated narrative flow correlation degree not only focuses on what is said, but also on how it is said and where the emotion is heading. This multi-dimensional narrative matching mechanism makes advertisements no longer a means of interrupting the plot, but a natural extension of the content's emotional logic. For example, automatically inserting a sports brand advertisement with the same exciting emotional evolution at the climax of a sports game victory avoids emotional break and enhances the acceptance of brand information, thereby effectively alleviating the technical problem of stiff matching at the narrative level in existing technologies.

[0062] Example 7: In one optional embodiment, this application provides a process for generating a set of seamless advertising recommendation insertions, the method comprising the following steps: Step 1: For each candidate ad in the personalized candidate ad set, extract the real-time encoding parameter set of the target streaming media content, and perform real-time transcoding on the candidate ad to make the encoding parameters consistent with the target streaming media content, thus obtaining an encoding-adapted ad; The real-time encoding parameter set refers to the set of underlying encoding attributes possessed by the target streaming media content segment that is currently playing or about to be played. This parameter set is obtained by parsing the metadata tags of the corresponding segments in the original manifest file of the target streaming media content (such as an M3U8 file for the HLS protocol or an MPD file for the DASH protocol). Specifically, the real-time encoding parameter set includes at least video resolution, frame rate, video bitrate, and encoding profile and level. For example, when the target streaming media content is a high-definition live signal, its real-time encoding parameter set might be extracted as: resolution 1920×1080, frame rate 25fps, bitrate 8Mbps, encoding format H.264 High Profile Level 4.1. The role of real-time transcoding processing is to eliminate the differences in technical specifications between the candidate advertisement and the target streaming media content. Its execution method is based on the extracted real-time encoding parameter set, starting an adaptive recoding algorithm thread to decode and then re-encode the original video segments of the candidate advertisement. During this process, not only are the resolution and bitrate adjusted to match the target content, but the GOP (Group of Pictures) structure, color space (such as the mapping from BT.709 to BT.2020), and dynamic range metadata (HDR / SDR) of the candidate ads are also simultaneously converted to a format completely consistent with the target streaming media content. Through this real-time transcoding process, candidate ads that might otherwise require the player to reinitialize the decoder due to mismatched encoding parameters, resulting in black screens or stuttering, are transformed into seamlessly encoded and adapted ads.

[0063] Step 2: Gradually blend the consecutive frames before the ad insertion point in the target streaming media content with the starting consecutive frames of the encoded and adapted ad to generate a transition frame segment; Gradient blending is an image processing mechanism used to smooth visual transitions, aiming to create a visually continuous buffer zone between the hard cut point of content ending and advertisement beginning. This transition frame segment is obtained by selecting pixel data from the last M frames (e.g., the last 5 frames) of the target streaming content before the advertisement insertion point, and combining them with pixel data from the first M frames of the advertisement after the aforementioned transcoding process, performing a pixel-by-pixel alpha blending calculation at a preset ratio. Specifically, the blending ratio changes linearly or non-linearly with the frame order. For example, in the first transition frame, the alpha value of the content frame is 1.0 and the advertisement frame is 0.0; in the intermediate frames, the alpha value of the content frame gradually decreases to 0.0 and the advertisement frame gradually increases to 1.0, thus creating a fade-in or cross-fade visual effect. The encoding parameters of the generated gradient transition frame sequence are forcibly set to be completely consistent with the real-time encoding parameter set and exist as an independent media segment. This gradual blending method effectively masks the visual abruptness caused by sudden changes in scene content, making the scene transitions perceived by the audience as smooth as natural camera movement, thus avoiding the destruction of immersion caused by traditional hard cuts or black frames.

[0064] Step 3: Bind the encoded and adapted ads to the transition frame segments with timestamps and arrange their order to integrate them into a seamless ad recommendation insertion set.

[0065] Timestamp binding refers to assigning consecutive and logically correct Presentation Timestamps (PTS) and Decoding Timestamps (DTS) to the generated transition frame segments and encoded adaptation ads to ensure that the player maintains audio-visual synchronization during rendering. This process is based on the end timestamp of the last segment of the target streaming media content. Specifically, the starting PTS of the transition frame segment immediately follows the ending PTS of the last frame of the content, while the starting PTS of the encoded adaptation ad immediately follows the ending PTS of the transition frame segment, thus forming a continuous sequence without breaks on the timeline. Sequential arrangement involves packaging the two into a complete media resource set according to the logical order of transition frame segments first and encoded adaptation ads last. For example, if the content end time is T and the transition frame duration is 0.2 seconds, then the time range of the transition frame is [T, T+0.2s], and the time range of the encoded adaptation ad is [T+0.2s, T+0.2s+Ad_Duration]. By using this timestamp binding and sequential arrangement, the originally discrete transcoded advertising data and synthesized transition data are integrated into a structured, seamless advertising recommendation insertion set. This set can be directly used for subsequent list rewriting operations to replace the advertising placeholders in the original list, ensuring that the terminal player can obtain a data stream with consistent encoding and visual coherence when requesting this part of the content.

[0066] This application achieves a seamless experience in both technology and perception during the insertion of streaming media advertisements through the synergistic effect of the aforementioned technical features. By extracting the real-time encoding parameter set of the target streaming media content and performing real-time transcoding on candidate advertisements, it ensures strict consistency between advertisement segments and the main content in terms of resolution, frame rate, and encoding protocol, eliminating the risk of black screens and stuttering caused by decoder re-initialization from the underlying data structure. Furthermore, it utilizes keyframes from the content and advertisements to generate transition frame segments through gradual blending, and performs precise timestamp binding and sequential arrangement in the time dimension, constructing a smooth visual buffer from content to advertisement. This dynamic adaptation of encoding parameters and intelligent synthesis of transition frames work together; the former solves the technical obstacles of unplayable or unsmooth playback, while the latter solves the perceptual obstacle of abrupt viewing. Together, they enable seamless connection at both the technical transmission layer and the user's audiovisual layer at the advertisement insertion point, improving the continuity of streaming media services and user experience.

[0067] Example 8: Another optional embodiment of this application provides a real-time transcoding processing method, which includes the following steps: Step 1: Obtain the real-time encoding parameter set for the current segment by parsing the original list of the target streaming media content. The real-time encoding parameter set should at least include resolution, frame rate, bit rate, and encoding profile / level. The original manifest can be an index file describing the sequence, duration, and media attributes of streaming content segments, such as an M3U8 file in the HLS protocol or an MPD file in the DASH protocol. The real-time encoding parameter set is a key set of metadata that determines the initialization state and rendering behavior of the video decoder. Specifically, resolution refers to the pixel size of the video frame (e.g., 1920×1080), frame rate refers to the number of frames displayed per second (e.g., 25fps or 60fps), bitrate refers to the amount of data transmitted per unit time (e.g., 8Mbps), and encoding Profile / Level defines the specific configuration level of the encoding algorithm (e.g., H.264 HighProfile Level 4.1). These parameters are directly extracted by parsing specific tags (e.g., EXT-X-STREAM-INF or AdaptationSet attributes) in the original manifest. This step aims to establish a precise target benchmark for subsequent transcoding operations, ensuring that the generated ad segments are fully compatible with the currently playing content segments at the bitstream level. For example, when the system detects that the currently playing segment is marked as 1080p_60fps_High_L4.1, the extraction module will immediately lock the resolution as 1920×1080, the frame rate as 60, and the encoding level as High Profile Level 4.1 as the standard parameters. This list-based real-time parsing method can dynamically adapt to possible bitrate switching (ABR) scenarios in the live stream, avoiding decoder resets or black screen phenomena caused by static parameter presets.

[0068] Step 2: Based on the real-time encoding parameter set, the candidate ads are decoded and then re-encoded using an adaptive recoding algorithm to ensure that the encoding parameters of the candidate ads are consistent with the real-time encoding parameter set, thus obtaining an encoded and adapted ad. The adaptive recoding algorithm is an image processing logic that dynamically adjusts the encoding kernel configuration based on target parameters. The process first fully decodes the original compressed bitstream of the candidate ad, restoring it to an uncompressed baseband video frame sequence (YUV or RGB format). Then, it starts an encoder instance, forcibly injecting the obtained resolution, frame rate, bitrate, and Profile / Level into the encoder configuration items to recompress the baseband frame sequence. This decode-re-encode process eliminates the difference between the original encoding parameters of the candidate ad and the target streaming media content. Specifically, if the original candidate ad is 720p / 30fps / Low Profile, while the target content is 1080p / 60fps / High Profile, the algorithm automatically performs upsampling, frame rate interpolation, and encoding complexity enhancement until the output stream's metrics strictly match the real-time encoding parameter set. For example, when processing an ad creative with an initial bitrate of 4Mbps, if the target stream is currently at a high bitrate (12Mbps), the recoding engine will correspondingly increase the precision of the quantization parameter (QP) and increase the number of reference frames to generate an encoding-adapted ad that meets the 12Mbps requirement. This dynamic mirroring mechanism ensures that the player does not need to reinitialize the decoder context when switching from content to advertisements, thus completely eliminating stuttering or audio-visual desynchronization issues caused by changes in encoding specifications.

[0069] Step 3: During the re-encoding process, the GOP structure, color space, and dynamic range metadata of the candidate ads are simultaneously mapped to the same format as the parameters corresponding to the target streaming media content.

[0070] In this context, the GOP structure refers to the arrangement of keyframes (I-frames) and non-keyframes (P-frames / B-frames) in video encoding; the color space defines the mathematical representation of color (e.g., BT.709, BT.2020); and the dynamic range metadata describes the distribution range of brightness (e.g., SDR, HDR10, Dolby Vision). In the aforementioned re-encoding loop, the system not only replicates the basic resolution and frame rate but also deeply intervenes in the encoder's internal decision-making logic, forcibly aligning these deep structural parameters. Specifically, the system analyzes the GOP length and keyframe interval (e.g., one I-frame every 2 seconds) of the current segment of the target streaming content and strictly adheres to the same interval when inserting I-frames during candidate advertising encoding to prevent screen tearing at the splicing points due to misaligned GOP boundaries. Simultaneously, the color space conversion module transforms the color gamut coordinates of the candidate advertisements to the color gamut standard of the target content and rewrites the dynamic range metadata (e.g., MaxCLL, MaxFALL) to values ​​consistent with the current stream. For example, when the target content is in HDR10 mode and the GOP structure is IBBPBBP..., even if the candidate ad was originally in SDR mode and the GOP structure is IPPPP..., this step will map its color space from BT.709 to BT.2020, inject HDR static metadata, and reconstruct the frame type sequence to perfectly match the IBBPBBP... mode. Through the synchronous mapping of GOP structure, color space, and dynamic range metadata, comprehensive consistency from surface-level encoding parameters to deep bitstream structure is achieved, avoiding color distortion or brightness anomalies caused by implicit compatibility issues.

[0071] This application constructs a comprehensive, seamless encoding mechanism through the synergistic effect of the aforementioned technical features. By parsing the original manifest to obtain a real-time encoding parameter set, it provides a precise dynamic target for transcoding. Based on this, an adaptive recoding algorithm is used to perform decoding followed by re-encoding, ensuring physical consistency of fundamental metrics such as resolution and frame rate. Simultaneously, synchronous mapping of GOP structure, color space, and dynamic range metadata further eliminates potential conflicts between the internal encoding structure and color management. This end-to-end parameter mirroring, from macroscopic parameters to microscopic structures, ensures that candidate ads, when embedded in streaming media content, exhibit characteristics identical to the original content at the bitstream level. This guarantees the continuity of the player's decoding state and improves the smoothness of ad insertion and a seamless visual experience.

[0072] Example 9: In another embodiment, this application provides a gradient blending method, which includes: Step 1: By performing a preset ratio of alpha fusion on each pixel of the last M frames before the ad insertion point in the target streaming media content and the pixel data of the M frames before the ad starts, a gradual transition frame sequence with controllable duration is generated. In this context, the last M frames can refer to the sequence of consecutive video frames immediately preceding the timestamp of the ad insertion point, while the preceding M frames can refer to the sequence of consecutive video frames following the timestamp of the candidate ad's start. M is a positive integer, and its value determines the duration of the transition animation. Alpha blending is an image compositing technique based on the alpha channel, used to superimpose two image sources from different sources according to linear weights. Specifically, for each pair of corresponding pixels, the system dynamically calculates the complementary alpha values ​​of the foreground image (ad start frame) and the background image (content end frame) based on the position index of the current frame in the transition sequence. For example, when the transition duration is set to 0.5 seconds and the frame rate is 25fps, M is set to 12. The alpha blending ratio of the first frame can be set to 8% for the ad and 92% for the original content, and then the proportion of the ad increases linearly frame by frame until it reaches 100% in the 12th frame. Through this pixel-by-pixel weighted blending method, a smooth fade-in and fade-out effect can be visually simulated, effectively masking sudden changes in brightness or color jumps caused by scene transitions. This step aims to utilize the persistence of vision in the human eye to transform abrupt cuts into a continuous visual flow, thereby reducing the sense of discontinuity during viewing.

[0073] Step 2: Set the encoding parameters of the gradient transition frame sequence to be consistent with the real-time encoding parameter set, and insert it as an independent segment between the target streaming media content and the encoding adaptation advertisement to obtain the transition frame segment.

[0074] The real-time encoding parameter set refers to the encoding configuration information that is completely consistent with the current target streaming media content, as determined in the above embodiments and the above embodiments. It includes at least resolution, frame rate, bit rate, encoding profile / level, GOP structure, and color space. Setting the encoding parameters of the transition frame sequence to be consistent means that when the generated original pixel sequence is compressed by the encoder, the various indicators in the aforementioned real-time encoding parameter set are forcibly reused to ensure that the output transition frame segments meet the player's decoding expectations for the main content stream at the binary level. Independent segments can refer to encapsulating the transition frame sequence into a standard media transmission unit (such as a .ts file in the HLS protocol or a .mp4 segment in the DASH protocol) and assigning it an independent timestamp and sequence number. For example, if the target streaming media content is 1080P, H.264 High Profile, 25fps, the generated transition frame segments must also strictly adhere to this specification, even if their actual effective image lasts only a few frames, they must be padded to meet the minimum segment length requirement or short segments must be indicated by special markers. This transition frame segment acts as a connecting bridge, physically inserted between the last segment of the target streaming content and the first segment of the encoded advertisement. By maintaining a high degree of consistency in encoding parameters, the player does not need to reinitialize the decoder or switch the decoding context when decoding this transition frame segment, avoiding black screens, stuttering, or audio-visual desynchronization caused by parameter jumps. Therefore, the transition frame segment not only achieves a smooth transition in visual content but also seamless integration at the data transmission and decoding protocol levels, improving the robustness of streaming playback.

[0075] This application achieves a seamless, all-encompassing experience during ad insertion through the synergistic effect of the aforementioned technical features. By performing pixel-by-pixel alpha fusion between the end frame of the content and the beginning frame of the ad, a visually continuous gradient field is constructed, resolving the visual impact problem caused by traditional hard cuts. Simultaneously, the encoding parameters of the generated transition frame sequence are forced to remain consistent with the real-time content stream, and it is encapsulated as an independent segment insertion list, ensuring the continuity between the transport and decoding layers. This combination of visual soft transition and hard alignment of technical parameters allows the player to handle neither drastic jumps in screen content nor sudden changes in encoding formats when processing ad insertion points, thus completely eliminating the trace of ad insertion at the user's perception level and achieving truly smooth and seamless playback.

[0076] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0077] Figure 3 A schematic diagram of a streaming media advertising seamless insertion system based on manifest rewriting, provided as an embodiment of this application, is shown below. Figure 3 As shown, a streaming media advertising seamless insertion system 300 based on list rewriting in this embodiment includes: a context-aware module 301, a perception matching module 302, an encoding synthesis module 303, and a list update module 304.

[0078] The context awareness module 301 is used to obtain information about a predetermined segment before the advertisement insertion point in the streaming media content, and to perform multi-dimensional perception context analysis based on the predetermined segment information to generate a perception context descriptor set. The perception matching module 302 is used to perform visual rhythm alignment, auditory feature fusion and narrative flow association based on the perception context descriptor set, and to filter advertisements that meet the preset perception association conditions from the candidate advertisement library to generate a personalized candidate advertisement set. The encoding and synthesis module 303 is used to perform context-aware dynamic adaptation of encoding parameters and intelligent synthesis of transition frames based on the personalized candidate ad set, to generate a seamless ad recommendation insertion set; The list update module 304 is used to embed the seamless ad recommendation insertion set into the original list of the streaming media content, replace the corresponding ad placeholders, and output the updated seamless ad playback list.

[0079] The system in this embodiment can be used to execute the methods of any of the above embodiments, and its implementation principle and technical effect are similar, so they will not be described again here.

Claims

1. A method for seamless insertion of streaming media ads based on manifest rewriting, characterized in that, include: Obtain information about a predetermined segment before the ad insertion point in the streaming media content; perform multi-dimensional perceptual context analysis based on the predetermined segment information; and generate a perceptual context descriptor set. Based on the set of perceptual context descriptors, visual rhythm alignment, auditory feature fusion, and narrative flow association are performed based on personalized perceptual matching of advertisements. Ads that meet the preset perceptual association conditions are selected from the candidate advertisement library to generate a personalized candidate advertisement set. Based on the personalized candidate ad set, context-aware encoding parameter dynamic adaptation and transition frame intelligent synthesis are performed to generate a seamless ad recommendation insertion set; The seamless ad recommendation insertion set is embedded into the original list of streaming media content, the corresponding ad placeholders are replaced, and the updated seamless ad playback list is output.

2. The method according to claim 1, characterized in that, The generation process of the perception context descriptor set includes: The predetermined segment information includes video streams, audio streams, and subtitle / metadata streams; By analyzing the pixel statistical features of consecutive frame sequences in the video stream, the main color scheme, motion rhythm, and illumination distribution of the scene are extracted to generate visual rhythm features. By analyzing the audio spectrum and loudness sequence of the audio stream, the beat and volume change patterns of the background music are extracted to generate auditory rhythm features; By parsing the natural language semantics of the subtitle / metadata stream, the emotional polarity, thematic keywords, and narrative rhythm in the lines or narration are extracted to generate narrative stream features; The visual prosodic features, the auditory rhythm features, and the narrative flow features are structurally associated and integrated into the perceptual context descriptor set.

3. The method according to claim 2, characterized in that, The process of generating the personalized candidate ad set includes: Based on the visual prosodic features, auditory rhythm features, and narrative flow features in the perceptual context descriptor set, the visual prosodic matching degree, auditory feature matching degree, and narrative flow correlation degree corresponding to each candidate advertisement are analyzed respectively. Candidate ads that exceed a preset candidate threshold in at least one of the visual rhythm matching degree, the auditory feature matching degree, and the narrative flow correlation degree are considered as ads that satisfy the preset perceptual correlation conditions, and thus constitute the personalized candidate ad set.

4. The method according to claim 3, characterized in that, The analysis process of the visual prosody matching degree includes: Extract the main color sequence, motion vector amplitude sequence, and brightness mean sequence of the last N frames of the predetermined segment from the visual rhythm features to form the source visual rhythm curve group; Extract the corresponding sequence of the first N frames of each candidate advertisement from its visual tone to form a candidate visual rhythm curve group; The cumulative distance between the two sets of curves is calculated using a dynamic time warping algorithm, and its reciprocal is transformed into visual prosodic dynamic similarity. Meanwhile, the difference in shot switching frequency between the last N frames of the predetermined segment and the first N frames of the candidate advertisement is analyzed to obtain the shot rhythm consistency. The visual rhythm dynamic similarity is multiplied by the shot rhythm consistency to obtain the visual rhythm matching degree.

5. The method according to claim 3, characterized in that, The analysis process of the auditory feature matching degree includes: The frequency band energy distribution, timbre feature vector and loudness envelope curve of the final continuous audio of the predetermined segment are extracted from the auditory rhythm features to form a source auditory feature set. The corresponding features of the starting continuous audio of each candidate advertisement are extracted from the audio features of each candidate advertisement to form a candidate auditory feature set. Calculate the spectral centroid distance, timbre cosine similarity, and loudness Pearson correlation coefficient between the two feature sets, and combine these three values ​​after normalization to obtain the auditory feature fusion degree. Simultaneously, the degree of matching between the background music emotion tags of the predetermined segment's ending audio and the candidate advertisement's starting audio is analyzed to obtain the emotion evolution compatibility. The square root of the product of the auditory feature fusion degree and the emotional evolution compatibility degree is used to obtain the auditory feature matching degree.

6. The method according to claim 3, characterized in that, The analysis process of the narrative flow correlation includes: The emotional polarity intensity, the set of thematic keywords, and the narrative rhythm type of the last line or narration of the predetermined segment are extracted from the narrative flow features to form a source narrative feature group. Corresponding content is extracted from the advertising words or scene descriptions of each candidate advertisement to form a candidate narrative feature group. Calculate the negative exponential transformation of the difference in emotional polarity intensity between the two feature groups, the overlap of thematic keywords, and the matching degree of narrative rhythm type, and obtain the semantic structure correlation by weighted summation; At the same time, the trend coupling degree is obtained by analyzing whether the narrative arc trend at the end of the predetermined segment is consistent with the emotional evolution direction conveyed by the candidate advertisement; The narrative flow correlation is obtained by weighting and summing the semantic structure correlation degree and the trend coupling degree and then normalizing the sum.

7. The method according to claim 3, characterized in that, The process of generating the seamless ad recommendation insertion set includes: For each candidate ad in the personalized candidate ad set, the real-time encoding parameter set of the target streaming media content is extracted, and the candidate ad is transcoded in real time to make the encoding parameters consistent with the target streaming media content, thus obtaining an encoding-adapted ad. The consecutive frames before the advertisement insertion point in the target streaming media content are gradually blended with the starting consecutive frames of the encoded and adapted advertisement to generate a transition frame segment. The encoded and adapted advertisements are timestamped and their order is arranged with the transition frame segments to form the seamless advertisement recommendation insertion set.

8. The method according to claim 7, characterized in that, The real-time transcoding process includes: By parsing the original list of the target streaming media content, the real-time encoding parameter set of the current segment is obtained. The real-time encoding parameter set includes at least resolution, frame rate, bit rate, and encoding profile / level. Based on the real-time encoding parameter set, the candidate advertisement is decoded and then re-encoded using an adaptive recoding algorithm to make the encoding parameters of the candidate advertisement consistent with the real-time encoding parameter set, thus obtaining the encoded adapted advertisement; During the re-encoding process, the GOP structure, color space, and dynamic range metadata of the candidate advertisement are simultaneously mapped to the same format as the parameters corresponding to the target streaming media content.

9. The method according to claim 7, characterized in that, The gradual blending includes: By performing a preset ratio of alpha fusion on a pixel-by-pixel basis on the last M frames before the advertisement insertion point in the target streaming media content and the pixel data of the M frames before the advertisement starts, a gradual transition frame sequence with controllable duration is generated. The encoding parameters of the gradual transition frame sequence are set to be consistent with the real-time encoding parameter set, and inserted as an independent segment between the target streaming media content and the encoding adaptation advertisement to obtain the transition frame segment.

10. A seamless streaming media advertising insertion system based on manifest rewriting, characterized in that, The method applied to any one of claims 1-9 includes: The context-aware module is used to acquire information about a predetermined segment before the advertisement insertion point in the streaming media content, and to perform multi-dimensional perceptual context analysis based on the predetermined segment information to generate a perceptual context descriptor set. The perception matching module is used to perform visual prosody alignment, auditory feature fusion and narrative flow association based on the perception context descriptor set, and to filter advertisements that meet the preset perception association conditions from the candidate advertisement library to generate a personalized candidate advertisement set. The encoding and synthesis module is used to perform context-aware dynamic adaptation of encoding parameters and intelligent synthesis of transition frames based on the personalized candidate ad set, so as to generate a seamless ad recommendation insertion set. The list update module is used to embed the seamless ad recommendation insertion set into the original list of the streaming media content, replace the corresponding ad placeholders, and output the updated seamless ad playback list.

Images