Director decision assistance method and system based on material semantic analysis

Abstract

The application discloses a director decision-making auxiliary method and system based on material semantic analysis, and belongs to the technical field of intelligent video production. The method comprises video material acquisition, multi-modal semantic feature extraction, audience feedback prediction and director decision-making assistance. The application realizes the early evaluation of the propagation effect of the unpublished video material by establishing the correlation prediction mechanism between the material content features and the historical audience feedback behaviors, can identify the material segments with high propagation potential in the video editing stage, reduces the experience dependence in the director decision-making process, improves the material screening efficiency and the content production efficiency; meanwhile, the application generates the quantitative material sorting results and the editing optimization suggestions, provides the objective decision-making basis for the director personnel, helps to improve the matching degree between the video content and the target audience interest, and enhances the content propagation effect and the user interaction level.

Description

Technical Field

[0001] This invention belongs to the field of intelligent video production technology, specifically referring to a method and system for assisting directors in making decisions based on semantic analysis of source material. Background Technology

[0002] With the rapid development of short videos, online variety shows, and interactive videos, the requirements for material selection, segment arrangement, and editing in video content production are becoming increasingly stringent. In traditional video production, directors and producers primarily rely on personal experience and subjective judgment during the material selection and editing stages, leading to the following prominent technical problems:

[0003] During video content production, directors and producers often struggle to predict the impact of different clips on audience interaction, engagement, and dissemination during the material selection and editing stages. They also find it difficult to accurately predict the content's dissemination potential and lack a unified prediction and evaluation mechanism. Furthermore, current technologies lack an integrated modeling system that combines semantic analysis of video materials, audience behavior prediction, and editing decision support. This results in material selection, clip arrangement, and editing strategy optimization remaining largely experience-driven, making it difficult to establish data-driven, interpretable decision-making criteria. Consequently, the ability to optimize editing schemes is insufficient, and the uncertainty of content dissemination effects is high.

[0004] Therefore, there is an urgent need for a scientific method and system that can quantitatively assess the dissemination potential, audience interaction, and emotional response of video materials based on multimodal semantic features and historical audience behavior data during the selection and editing stages, and generate a system that can assist directors in making decisions, so as to improve the efficiency of video content production and the ability to predict the dissemination effect of content. Summary of the Invention

[0005] To address the above issues and overcome the shortcomings of existing technologies, this invention provides a method and system for assisting directing decisions based on material semantic analysis. The method includes the following steps:

[0006] Step S1: Obtain video footage;

[0007] Step S2: Multimodal semantic feature extraction;

[0008] Step S3: Audience Feedback Prediction;

[0009] Step S4: Directing and decision support.

[0010] Furthermore, in step S1, the video material acquisition specifically involves acquiring the video material data to be processed, including video frame data, audio data, subtitle text data, and corresponding timeline information, to form the material data to be analyzed.

[0011] Furthermore, in step S2, the multimodal semantic feature extraction is used to realize cross-modal semantic information parsing. Specifically, it extracts semantic features of video images, audio semantic features and text semantic features respectively, and fuses various features based on timeline information to obtain a multimodal semantic feature representation.

[0012] Further, in step S3, the audience feedback prediction is used to predict audience feedback based on multimodal semantic feature representation. Specifically, it associates the audience behavior data corresponding to the historically broadcast videos with the timeline, establishes the prediction relationship between multimodal semantic feature representation and audience behavior, and outputs the audience feedback prediction result of the material data to be analyzed.

[0013] The audience behavior data includes bullet screen data, like data, comment data, forwarding data, dwell time data, and completion rate data;

[0014] The audience feedback prediction results include audience interaction probability, audience dwell contribution value, dissemination tendency value, emotional response distribution results, and comprehensive popularity score.

[0015] The comprehensive popularity score is used to characterize the expected dissemination and interaction triggering capabilities of the target material among the target audience.

[0016] The audience feedback prediction includes the following sub-steps:

[0017] Step S31, audience behavior timeline alignment modeling, is used to establish the temporal correlation between historically played videos and audience behavior. Specifically, it involves acquiring historically played videos and their corresponding audience behavior data, dividing the historically played videos into continuous time segments according to a preset time granularity, constructing a set of behavior events that includes the time of behavior occurrence, behavior type, and behavior intensity weight, and obtaining a time-aligned behavior sequence by mapping each behavior event to the corresponding time segment.

[0018] Step S32 generates propagation behavior feedback tags. Specifically, based on the time-aligned behavior sequence, the behavior intensity weights in each time segment are accumulated and statistically analyzed to calculate the segment behavior heat. Based on the change in behavior heat between adjacent time segments, a propagation growth trend is constructed. Combined with user dwell time, bullet screen and comment sentiment information, corresponding time segment interaction feedback tags, propagation feedback tags, dwell feedback tags and sentiment response tags are generated to obtain a time-series propagation feedback tag set.

[0019] Step S33, propagation potential modeling, is used to assess the potential propagation ability of the target material among the audience. Specifically, it involves extracting the multimodal semantic feature representation of historically broadcast videos through step S2, and constructing a propagation potential model by combining time-aligned behavioral sequences and propagation growth trends. The content conflict degree is obtained by calculating the multimodal semantic differences between adjacent time segments, the emotional activation degree is constructed based on the emotional response tags, and the propagation persistence trend is constructed by combining the propagation growth trend. The content conflict degree, emotional activation degree, and propagation persistence trend are weighted and fused to obtain the propagation potential features of the corresponding time segments.

[0020] Step S34, temporal propagation association modeling, is used to learn the propagation association relationship and hotspot diffusion relationship between time segments. Specifically, it involves constructing a multimodal temporal association network, taking the multimodal semantic feature representation and propagation potential energy feature as the input sequence, performing temporal dependency modeling and cross-segment attention association calculation, and obtaining the propagation association feature representation.

[0021] Step S35 provides feedback on the prediction output, specifically outputting the audience interaction probability, audience dwell contribution value, dissemination tendency value, and emotional response distribution results based on the dissemination association feature representation. By weighting and fusing the emotional activation, audience interaction probability, audience dwell contribution value, and dissemination tendency value, a comprehensive popularity score for each time segment is generated.

[0022] Step S36 generates the prediction result. Specifically, by executing steps S31 to S35, the model is trained to construct an audience feedback prediction model. Then, step S2 extracts the multimodal semantic feature representation of the material data to be analyzed and inputs it into the audience feedback prediction model to obtain the audience feedback prediction result.

[0023] Furthermore, in step S4, the director's decision assistance specifically generates a director's decision assistance scheme based on the audience feedback prediction results, including at least one of the following: recommendation of high-promotion potential segments, material priority ranking, and editing optimization suggestions;

[0024] The director's decision support includes the following sub-steps:

[0025] Step S41, identifying segments with high dissemination potential, is used to filter video segments with high dissemination potential. Specifically, it involves sorting and analyzing the comprehensive popularity scores of each time segment, and combining the segment duration, emotional response distribution results, and segment content similarity to generate recommended results for segments with high dissemination potential.

[0026] Step S42, material priority sorting, is used to determine the priority order of material usage. Specifically, based on the comprehensive popularity score, the priority of the materials to be analyzed is evaluated, and material priority sorting results are generated.

[0027] Step S43 generates editing optimization suggestions, which specifically involves constructing editing optimization evaluation indicators based on the comprehensive popularity score, score change trend, and emotional response distribution structure of each time segment, and generating editing optimization suggestions based on the editing optimization evaluation indicators.

[0028] The video editing and directing decision support system based on semantic analysis of video materials provided by this invention includes a video material acquisition module, a semantic feature extraction module, an audience feedback prediction module, and a directing and directing decision support module.

[0029] The video material acquisition module is used to acquire video materials, obtain material data to be analyzed, and send the material data to be analyzed to the semantic feature extraction module and the audience feedback prediction module.

[0030] The semantic feature extraction module is used for multimodal semantic feature extraction, to obtain a multimodal semantic feature representation, and to send the multimodal semantic feature representation to the audience feedback prediction module;

[0031] The audience feedback prediction module is used to predict audience feedback, obtain audience feedback prediction results, and send the audience feedback prediction results to the director's decision support module.

[0032] The directing decision support module is used to assist in directing decisions and obtain a directing decision support scheme.

[0033] The beneficial effects achieved by the present invention using the above solution are as follows:

[0034] This invention establishes a predictive mechanism linking the characteristics of source material content with historical audience feedback behavior, enabling early assessment of the dissemination effect of unreleased video material. It can identify material segments with high dissemination potential during the video editing stage, reducing reliance on experience in the director's decision-making process and improving the efficiency of material selection and content production. Simultaneously, by generating quantified material ranking results and editing optimization suggestions, it provides directors with objective decision-making basis, helping to improve the matching degree between video content and target audience interests, enhancing content dissemination effectiveness and user interaction levels. Attached Figure Description

[0035] Figure 1 A flowchart illustrating the directing decision support method based on material semantic analysis provided by this invention;

[0036] Figure 2 A schematic diagram of the editing and directing decision support system based on material semantic analysis provided by the present invention;

[0037] Figure 3 This is a flowchart illustrating step S3;

[0038] Figure 4 This is a flowchart illustrating step S4.

[0039] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof. Detailed Implementation

[0040] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0041] In the description of this invention, it should be understood that the terms "upper", "lower", "front", "rear", "left", "right", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0042] Example 1, see Figure 1 The present invention provides a method and system for assisting directors in their production decisions based on semantic analysis of source material. The method includes the following steps:

[0043] Step S1: Obtain video footage;

[0044] Step S2: Multimodal semantic feature extraction;

[0045] Step S3: Audience Feedback Prediction;

[0046] Step S4: Directing and decision support.

[0047] Example 2, see Figure 1 This embodiment is based on the above embodiment. In step S1, the video material acquisition specifically involves acquiring the video material data to be processed, including video frame data, audio data, subtitle text data, and corresponding timeline information, to form the material data to be analyzed.

[0048] In practice, the video data can be presented as a continuous frame sequence or an encoded compressed stream format based on different acquisition sources, such as video streams based on H.264 or H.265 encoding standards, and then decoded to restore a frame-level image sequence. Each video frame corresponds to a unique timestamp to identify its time position in the original video, thus providing a basic time reference for subsequent steps. The audio data is typically in the form of a sampled signal with a sampling rate of 16kHz, 44.1kHz, or 48kHz, and is converted into a time-frequency representation that can be used for feature extraction using a short-time Fourier transform. The audio data is also bound to a unified timeline to ensure synchronization with the video frames.

[0049] In some implementations, the subtitle text data may originate from manually generated subtitle files or automatic speech recognition results. The subtitle file format includes, but is not limited to, SRT, ASS, or VTT standard formats. For video footage without explicit subtitles, audio data can be transcribed using a speech recognition model to generate text sequences corresponding to time segments, establishing an alignment between the text and the video and audio signals. This method allows for a correspondence between video image data, audio data, and text data along the timeline, enabling unified association of multimodal content within the same time segment and providing a data foundation for subsequent multimodal semantic feature extraction.

[0050] The corresponding timeline information is used to construct a unified index structure for multimodal data. It can be represented using a timestamp-based linear index or a segmented window-based discrete index. For example, continuous video can be divided into several time segments according to a preset time granularity. Each time segment corresponds to a set of video frames, an audio signal interval, and a corresponding subtitle text segment, thus forming a structured time-aligned material unit.

[0051] The video materials include, but are not limited to, original footage, unreleased footage, or candidate edits.

[0052] In some implementations, to ensure the stability and consistency of subsequent semantic analysis, the video material data can undergo basic preprocessing operations before entering the material data to be analyzed. These preprocessing operations include resolution unification processing, frame rate standardization processing, audio noise reduction processing, and time axis benchmark unification processing. The time axis benchmark unification processing is used to eliminate time offset errors caused by different acquisition devices, different encoding formats, or different transcription methods, so that video frames, audio signals, and text fragments are all mapped to a unified time coordinate system, thereby improving data consistency in the subsequent multimodal semantic analysis process.

[0053] Example 3, see Figure 1This embodiment is based on the above embodiment. In step S2, the multimodal semantic feature extraction is used to realize cross-modal semantic information parsing. Specifically, it extracts video image semantic features, audio semantic features and text semantic features respectively, and fuses various features based on time axis information to obtain a multimodal semantic feature representation.

[0054] The video image semantic features include at least one of the following: facial expression features, camera switching features, scene change features, and action change features; the audio semantic features include at least one of the following: speech rate features, pitch features, background music rhythm features, and volume change features; the text semantic features include at least one of the following: emotion semantic features, keyword features, and event semantic features.

[0055] The multimodal semantic feature extraction includes the following sub-steps:

[0056] Step S21 involves extracting semantic features from video images. Specifically, the video image data is sampled at the frame level and keyframes are selected. Face detection and expression recognition algorithms are used to extract facial expression features. Shot transition features are extracted through shot boundary detection or inter-frame difference analysis. Image classification networks are used to identify scene types and extract scene change features. Action change features are extracted based on human pose estimation or optical flow analysis, thereby generating semantic features from the video images.

[0057] Step S22 Audio semantic feature extraction: Specifically, the audio data is analyzed in the time domain and frequency domain, speech rate features are extracted to represent the speaking rhythm, pitch features are extracted to represent changes in speech emotion, background music rhythm features are extracted using beat detection and spectrum analysis, volume change features are calculated to represent sound intensity fluctuations, and audio semantic features are generated.

[0058] Step S23 Text semantic feature extraction: Specifically, natural language processing is performed on the subtitle text data, sentiment analysis model is used to extract emotional semantic features, keyword features are obtained through keyword extraction or topic analysis, and event extraction model is used to identify event subjects, actions and objects and generate event semantic features, thereby forming text semantic features;

[0059] Step S24 is feature fusion processing, which specifically maps the semantic features of video images, audio semantic features and text semantic features to a unified time window according to the time axis information, and performs synchronous association processing on various features according to the corresponding time segments. Joint semantic fusion is performed through feature splicing, weighted fusion or cross-modal attention mechanism to generate a semantic feature vector of a unified dimension, and forms a multimodal semantic feature representation according to the preset time window.

[0060] Example 4, see Figure 1 and Figure 3This embodiment is based on the above embodiment. In step S3, the audience feedback prediction is used to predict audience feedback based on multimodal semantic feature representation. Specifically, it associates the audience behavior data corresponding to the historically broadcast videos with the timeline, establishes the prediction relationship between multimodal semantic feature representation and audience behavior, and outputs the audience feedback prediction result of the material data to be analyzed.

[0061] The audience behavior data includes bullet screen data, like data, comment data, forwarding data, dwell time data, and completion rate data;

[0062] The audience feedback prediction results include audience interaction probability, audience dwell contribution value, dissemination tendency value, emotional response distribution results, and comprehensive popularity score.

[0063] The comprehensive popularity score is used to characterize the expected dissemination ability and interaction triggering ability of the target material in the target audience group, and can generate the time distribution of material popularity according to time segments;

[0064] The audience feedback prediction includes the following sub-steps:

[0065] Step S31, audience behavior timeline alignment modeling, is used to establish the temporal correlation between historically played videos and audience behavior. Specifically, it involves acquiring historically played videos and their corresponding audience behavior data, dividing the historically played videos into continuous time segments according to a preset time granularity, constructing a set of behavior events that includes the time of behavior occurrence, behavior type, and behavior intensity weight, and obtaining a time-aligned behavior sequence by mapping each behavior event to the corresponding time segment.

[0066] Step S32 generates propagation behavior feedback tags. Specifically, based on the time-aligned behavior sequence, the behavior intensity weights in each time segment are accumulated and statistically analyzed to calculate the segment behavior heat. Based on the change in behavior heat between adjacent time segments, a propagation growth trend is constructed. Combined with user dwell time, bullet screen and comment sentiment information, corresponding time segment interaction feedback tags, propagation feedback tags, dwell feedback tags and sentiment response tags are generated to obtain a time-series propagation feedback tag set.

[0067] The formula for calculating the popularity of the segment behavior is:

[0068] ;

[0069] In the formula, D i This represents the segment activity popularity of the i-th time segment, used to characterize the overall activity level of audience behavior within that time segment, where i is the time segment index, and b... j It is the j-th behavior event, where j is the index of the behavior event, and v i It is the i-th time segment, w jIt is the behavior intensity weight corresponding to the j-th behavior event;

[0070] The formula for calculating the propagation growth trend is:

[0071] ;

[0072] In the formula, D i-1 It is the heat of the segment behavior in the (i-1)th time segment. It is a minterm, used to avoid the case where the denominator is zero;

[0073] Step S33, propagation potential modeling, is used to assess the potential propagation ability of the target material among the audience. Specifically, it involves extracting the multimodal semantic feature representation of historically broadcast videos through step S2, and constructing a propagation potential model by combining time-aligned behavioral sequences and propagation growth trends. The content conflict degree is obtained by calculating the multimodal semantic differences between adjacent time segments, the emotional activation degree is constructed based on the emotional response tags, and the propagation persistence trend is constructed by combining the propagation growth trend. The content conflict degree, emotional activation degree, and propagation persistence trend are weighted and fused to obtain the propagation potential features of the corresponding time segments.

[0074] The process of constructing emotional activation based on emotional response tags specifically involves weighting and summing the emotional response tags corresponding to each time segment according to preset emotional weights to obtain the emotional activation.

[0075] In some implementations, the propagation potential modeling also includes differentiated weight modeling for different types of propagation behavior, wherein different propagation contribution coefficients are set for likes, comments, reposts and bullet screen behaviors, and the contribution of each behavior is adaptively adjusted based on the attention mechanism to enhance the sensitivity of propagation potential to different audience behavior types, thereby improving the robustness and generalization ability of propagation trend prediction.

[0076] Step S34, temporal propagation association modeling, is used to learn the propagation association relationship and hotspot diffusion relationship between time segments. Specifically, it involves constructing a multimodal temporal association network, taking the multimodal semantic feature representation and propagation potential energy feature as the input sequence, performing temporal dependency modeling and cross-segment attention association calculation, and obtaining the propagation association feature representation.

[0077] In the specific implementation process, the multimodal temporal association network includes a feature encoding module and a temporal attention modeling module; the feature encoding module is used to perform unified vectorization representation of multimodal semantic features and propagation potential features to form a multimodal feature sequence arranged in time segments;

[0078] The temporal attention modeling module specifically performs temporal dependency learning on the multimodal feature sequence based on the TransformerEncoder structure, calculates the association weights between different time segments through a self-attention mechanism, and then performs weighted aggregation of features of each time segment based on the association weights, so that each time segment can integrate the contextual propagation information of historical time segments and future time segments, thereby realizing cross-temporal propagation and diffusion path modeling and generating propagation association feature representations corresponding to each time segment;

[0079] Step S35 provides feedback on the prediction output, specifically outputting the audience interaction probability, audience dwell contribution value, dissemination tendency value, and emotional response distribution results based on the dissemination association feature representation. By weighting and fusing the emotional activation, audience interaction probability, audience dwell contribution value, and dissemination tendency value, a comprehensive popularity score for each time segment is generated.

[0080] In the specific implementation process, the feedback prediction output is achieved by constructing a multi-task regression and classification joint prediction model. The propagation association feature representation is used as a shared input feature. A unified temporal semantic representation is extracted through a shared feature encoding layer and then independently learned by the interaction prediction branch, the dwell contribution prediction branch, the propagation tendency prediction branch, and the emotion classification branch. The audience interaction probability is mapped using the Sigmoid function, the audience dwell contribution value and the propagation tendency value are predicted as continuous values ​​using linear regression, and the emotion response distribution is output as a multi-class probability using the Softmax function.

[0081] Step S36 generates the prediction result. Specifically, by executing steps S31 to S35, the model is trained to build an audience feedback prediction model. Then, by step S2, the multimodal semantic feature representation of the material data to be analyzed is extracted and input into the audience feedback prediction model to obtain the audience feedback prediction result.

[0082] In some implementations, a material popularity time distribution sequence can be constructed based on the comprehensive popularity score of each time segment, and hot segment intervals, emotional climax intervals, and dissemination outbreak intervals can be identified based on the material popularity time distribution sequence to further obtain the material popularity time distribution results.

[0083] By performing the above operations, this solution addresses the technical challenges faced by directors and editors in the video content production process. They struggle to predict the impact of different material segments on audience interaction, engagement, and dissemination during the material selection and editing stages, making it difficult to accurately forecast the content's dissemination potential and lacking a unified prediction and evaluation mechanism. This solution creatively employs an audience feedback prediction method based on dissemination potential modeling and temporal dissemination correlation analysis. By integrating historical audience behavior information and multimodal semantic information, it establishes a predictive relationship between material content and audience feedback, enabling early assessment of the material's dissemination potential, interactive capabilities, and audience emotional responses.

[0084] Example 5, see Figure 1 and Figure 4 This embodiment is based on the above embodiment. In step S4, the director decision assistance specifically generates a director decision assistance scheme based on the audience feedback prediction results, including at least one of the following: high dissemination potential segment recommendation, material priority sorting, and editing optimization suggestions.

[0085] The director's decision support includes the following sub-steps:

[0086] Step S41, identifying segments with high dissemination potential, is used to filter video segments with high dissemination potential. Specifically, it involves sorting and analyzing the comprehensive popularity scores of each time segment, and combining the segment duration, emotional response distribution results, and segment content similarity to generate recommended results for segments with high dissemination potential.

[0087] In the specific implementation process, firstly, the time segments are sorted in descending order based on their comprehensive scores, and the top k time segments with the highest scores are selected to form a candidate set of high dissemination potential. Then, the ability of the time segments to trigger audience emotional fluctuations is evaluated based on the emotional response distribution results. At the same time, the similarity of the content of the segments is combined to remove duplicate time segments with similar or repeated content. Finally, the time segments with high comprehensive popularity scores, strong emotional triggering ability, and good content representativeness are retained to form the recommendation results of high dissemination potential segments.

[0088] The segment content similarity is specifically calculated based on video frame data, audio data, and subtitle text data. The degree of repetition of segment content is assessed by comparing the similarity between multimodal semantic feature representations. This is used to remove time segments with highly similar content, thereby ensuring the diversity and representativeness of the recommendation results.

[0089] Step S42, material priority sorting, is used to determine the priority order of material usage. Specifically, based on the comprehensive popularity score, the materials to be analyzed are evaluated for priority and a material priority sorting result is generated.

[0090] In the specific implementation process, the comprehensive popularity score of all time segments in the material to be analyzed is weighted and averaged according to the preset weight to obtain the overall priority score of the material. Then, all materials are sorted from high to low according to the overall priority score to generate a material priority ranking list, which is used to guide the selection of material editing order, recommend important segments and optimize content allocation strategy.

[0091] Step S43 generates editing optimization suggestions, specifically by constructing editing optimization evaluation indicators based on the comprehensive popularity score, score change trend, and emotional response distribution structure of each time segment, and generating editing optimization suggestions based on the editing optimization evaluation indicators;

[0092] The editing optimization suggestions include suggestions for replacing materials and suggestions for adjusting the editing position;

[0093] In the specific implementation process, the editing optimization evaluation index is compared with the preset threshold. When the editing optimization evaluation index is less than the preset threshold, it is determined that there is room for optimization in the time segment. Combined with the similarity of the segment content, semantically similar time segments are retrieved from the recommended results of segments with high dissemination potential to generate material replacement suggestions. At the same time, based on the changes of the editing optimization index on the timeline, editing position adjustment suggestions are generated to assist the director in optimizing the order and content quality of video materials.

[0094] In some implementations, audience feedback predictions can be performed for at least two candidate editing schemes, and scheme comparison information can be generated based on the prediction results to assist directors in selecting materials.

[0095] By performing the above operations, this solution addresses the technical problem that existing technologies lack an integrated modeling system for video content production that combines semantic analysis of video materials, audience behavior prediction, and editing decision support. This results in material selection, segment arrangement, and editing strategy optimization still being primarily experience-driven, making it difficult to form a data-driven, interpretable decision-making basis. Consequently, this leads to insufficient editing optimization capabilities and high uncertainty in content dissemination effects. This solution creatively constructs an integrated analysis mechanism that combines multimodal semantic analysis of video materials, audience feedback prediction, and director decision support. By correlating the semantic features of materials with historical audience feedback through modeling, it achieves a structured assessment of the dissemination value of materials. This allows for the early identification of material segments with high dissemination potential during the video editing stage, improving video content production efficiency and the ability to predict content dissemination effects.

[0096] Example 6, see Figure 2 This embodiment is based on the above embodiments. The directing decision support system based on material semantic analysis provided by the present invention includes a video material acquisition module, a semantic feature extraction module, an audience feedback prediction module, and a directing decision support module.

[0097] The video material acquisition module is used to acquire video materials, obtain material data to be analyzed, and send the material data to be analyzed to the semantic feature extraction module and the audience feedback prediction module.

[0098] The semantic feature extraction module is used for multimodal semantic feature extraction, to obtain a multimodal semantic feature representation, and to send the multimodal semantic feature representation to the audience feedback prediction module;

[0099] The audience feedback prediction module is used to predict audience feedback, obtain audience feedback prediction results, and send the audience feedback prediction results to the director's decision support module.

[0100] The directing decision support module is used to assist in directing decisions and obtain a directing decision support scheme.

[0101] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0102] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention.

[0103] The present invention and its embodiments have been described above. This description is not restrictive, and the accompanying drawings are only one embodiment of the present invention; the actual structure is not limited thereto. In conclusion, if those skilled in the art are inspired by this description and design similar structures and embodiments without departing from the spirit of the invention, such designs should fall within the protection scope of the present invention.

Claims

1. A decision-making support method for directors based on semantic analysis of source material, characterized by: The method includes the following steps: Step S1: Video material acquisition. Specifically, this involves acquiring the video material data to be processed, including video frame data, audio data, subtitle text data, and corresponding timeline information, to form the material data to be analyzed. Step S2, multimodal semantic feature extraction, is used to realize cross-modal semantic information parsing. Specifically, it extracts semantic features from video images, audio, and text respectively, and fuses these features based on timeline information to obtain a multimodal semantic feature representation. In step S3, the audience feedback prediction is used to predict audience feedback based on multimodal semantic feature representation. Specifically, it associates the audience behavior data corresponding to the historically broadcast videos with the timeline, establishes the prediction relationship between multimodal semantic feature representation and audience behavior, and outputs the audience feedback prediction result of the material data to be analyzed. The audience feedback prediction includes steps S31 (audience behavior time axis alignment modeling), S32 (propagation behavior feedback label generation), S33 (propagation potential modeling), S34 (temporal propagation correlation modeling), S35 (feedback prediction output), and S36 (prediction result generation). In step S4, the director's decision assistance specifically generates a director's decision assistance scheme based on the audience feedback prediction results, including at least one of the following: recommendation of high-potential segments, material priority ranking, and editing optimization suggestions; the director's decision assistance includes step S41 identification of segments with high potential for dissemination, step S42 material priority ranking, and step S43 generation of editing optimization suggestions.

2. The directing decision support method based on material semantic analysis according to claim 1, characterized in that: The audience behavior data includes bullet screen data, like data, comment data, forwarding data, dwell time data, and completion rate data; the audience feedback prediction results include audience interaction probability, audience dwell contribution value, dissemination tendency value, emotional response distribution results, and comprehensive popularity score; the comprehensive popularity score is used to characterize the expected dissemination ability and interaction triggering ability of the target material in the target audience group.

3. The directing decision support method based on material semantic analysis according to claim 2, characterized in that: Step S31, audience behavior timeline alignment modeling, is used to establish the temporal correlation between historically played videos and audience behavior. Specifically, it involves acquiring historically played videos and their corresponding audience behavior data, dividing the historically played videos into continuous time segments according to a preset time granularity, constructing a set of behavior events that includes the time of behavior occurrence, behavior type, and behavior intensity weight, and obtaining a time-aligned behavior sequence by mapping each behavior event to the corresponding time segment. Step S32 generates propagation behavior feedback tags. Specifically, based on the time-aligned behavior sequence, the behavior intensity weights in each time segment are accumulated and statistically analyzed to calculate the behavior popularity of the segment. Based on the changes in behavior popularity between adjacent time segments, a propagation growth trend is constructed. Combined with user dwell time, bullet comments, and comment sentiment information, interactive feedback tags, propagation feedback tags, dwell feedback tags, and sentiment response tags for the corresponding time segments are generated to obtain a time-series propagation feedback tag set.

4. The directing decision support method based on material semantic analysis according to claim 3, characterized in that: Step S33, propagation potential modeling, is used to assess the potential propagation ability of the target material among the audience. Specifically, it involves extracting the multimodal semantic feature representation of historically broadcast videos through step S2, and constructing a propagation potential model by combining time-aligned behavioral sequences and propagation growth trends. The content conflict degree is obtained by calculating the multimodal semantic differences between adjacent time segments, the emotional activation degree is constructed based on the emotional response tags, and the propagation persistence trend is constructed by combining the propagation growth trend. The content conflict degree, emotional activation degree, and propagation persistence trend are weighted and fused to obtain the propagation potential features of the corresponding time segments. Step S34 involves temporal propagation association modeling, which is used to learn the propagation association relationship and hotspot diffusion relationship between time segments. Specifically, it involves constructing a multimodal temporal association network, taking the multimodal semantic feature representation and propagation potential feature as the input sequence, performing temporal dependency modeling and cross-segment attention association calculation, and obtaining the propagation association feature representation.

5. The directing decision support method based on material semantic analysis according to claim 4, characterized in that: Step S35 provides feedback on the prediction output, specifically outputting the audience interaction probability, audience dwell contribution value, dissemination tendency value, and emotional response distribution results based on the dissemination association feature representation. By weighting and fusing the emotional activation, audience interaction probability, audience dwell contribution value, and dissemination tendency value, a comprehensive popularity score for each time segment is generated. Step S36 generates the prediction result. Specifically, by executing steps S31 to S35, the model is trained to construct an audience feedback prediction model. Then, step S2 extracts the multimodal semantic feature representation of the material data to be analyzed and inputs it into the audience feedback prediction model to obtain the audience feedback prediction result.

6. The directing decision support method based on material semantic analysis according to claim 5, characterized in that: Step S41, the segment identification with high dissemination potential, is used to screen video segments with high dissemination potential. Specifically, it involves sorting and analyzing the comprehensive popularity scores of each time segment, and combining the segment duration, emotional response distribution results, and segment content similarity to generate recommended results of high dissemination potential segments.

7. The directing decision support method based on material semantic analysis according to claim 6, characterized in that: Step S42, material priority sorting, is used to determine the priority order of material usage. Specifically, based on the comprehensive popularity score, the materials to be analyzed are evaluated for priority and a material priority sorting result is generated. Step S43 generates editing optimization suggestions, which specifically involves constructing editing optimization evaluation indicators based on the comprehensive popularity score, score change trend, and emotional response distribution structure of each time segment, and generating editing optimization suggestions based on the editing optimization evaluation indicators.

8. A directing decision support system based on material semantic analysis, used to implement the directing decision support method based on material semantic analysis as described in any one of claims 1-7, characterized in that: It includes a video material acquisition module, a semantic feature extraction module, an audience feedback prediction module, and a director's decision support module; The video material acquisition module is used to acquire video materials, obtain material data to be analyzed, and send the material data to be analyzed to the semantic feature extraction module and the audience feedback prediction module. The semantic feature extraction module is used for multimodal semantic feature extraction, to obtain a multimodal semantic feature representation, and to send the multimodal semantic feature representation to the audience feedback prediction module; The audience feedback prediction module is used to predict audience feedback, obtain audience feedback prediction results, and send the audience feedback prediction results to the director's decision support module. The directing decision support module is used to assist in directing decisions and obtain a directing decision support scheme.

Images