Short video automatic live broadcast production system based on big data

Abstract

The application discloses a short video automatic live broadcast production system based on big data, and belongs to the technical field of multimedia information processing. The short video automatic live broadcast production system comprises the following modules: a real-time multi-modal data acquisition layer, which synchronously accesses video streams, audio streams, barrage semantic data and audience behavior data; a big data real-time analysis engine, which calculates the burst potential value of live broadcast content based on a three-modal feature extraction network; an intelligent slicing decision module, which determines slicing start and end points according to an adaptive threshold and a double-modal collaborative boundary detection; an automatic production module, which performs intelligent subtitle generation, emotional BGM matching and multi-platform format adaptation; an instant distribution network and data backflow system, which performs content pushing and effect tracking. Through dynamic weight fusion and cross-modal collaborative gain calculation, the application realizes quantitative evaluation and automatic production of live broadcast content.

Description

Technical Field

[0001] This invention relates to the field of multimedia information processing technology, and in particular to a short video automated live streaming production system and method based on big data. Background Technology

[0002] With the booming development of live-streaming e-commerce and content creation, the secondary dissemination value of live-streaming content is becoming increasingly prominent. A single live stream by a top streamer can generate 200-500 potential video clips, of which approximately 10-20 high-quality clips are selected through intelligent filtering and enter the automated production process. Industry data shows that the secondary dissemination traffic of live-streaming clips can reach 30%-150% of the original live-streaming traffic, contributing 15%-30% of the total GMV. However, traditional manual editing methods have a 2-4 hour lag, missing the golden window for trending topics to spread.

[0003] Existing technologies suffer from systemic deficiencies in real-time performance. Research reveals that current solutions primarily analyze complete data after the live stream ends. The processing flow includes downloading the live stream recording, decoding the complete video, extracting multimodal features, detecting slice points, and exporting segments, typically taking tens of minutes to several hours. However, in short video dissemination scenarios, the golden window for popular content is only within 30 minutes after the live stream ends. Post-event analysis means that the popularity of the sliced ​​content has significantly diminished by the time it is released. Furthermore, existing technologies have the following shortcomings: First, current technologies rely on human experience or simple statistical indicators to assess the value of live stream content, lacking intelligent analytical models; slice decisions depend on single-modal features, potentially leading to significant false positives and false negatives; automated production processes are fragmented, requiring users to manually connect the workflows. Third, existing technologies suffer from data silos and process breakpoints between functional modules, lacking real-time data exchange and synergistic gain mechanisms, and the disseminated data is not effectively fed back to optimize the analytical model.

[0004] Chinese patent CN119996721A discloses a live streaming data processing method that analyzes the features of multimodal data from game live streams and uses a large language model to detect highlight segments. However, this method simply integrates multimodal features and fails to consider the dynamic changes in the information content of each modality. It also does not disclose an automated production and distribution mechanism for short video production. Chinese patent CN121481676A discloses a product recommendation matching method for intelligent live streaming scenarios, focusing on product recommendation rather than short video content production. Chinese patent CN116152711A proposes a multimodal directing method that generates directing scripts through keyword recognition, but it does not involve the quantitative evaluation of the explosive potential of live streaming content or automated segment production.

[0005] In light of the aforementioned technical problems and the shortcomings of existing technologies, this invention proposes a system capable of achieving second-level quantitative evaluation and automated production of live streaming content, in order to solve the technical problems of delayed hotspot capture, high production costs, and low secondary utilization rate of content in existing technologies. Summary of the Invention

[0006] The purpose of this invention is to provide an automated live streaming production system and method for short videos based on big data, so as to solve the technical problems of lagging live streaming hotspot capture, high short video production costs, and low content reuse rate in the existing technology.

[0007] The technical solution of the present invention to solve the above-mentioned technical problems is as follows: In a first aspect, the present invention provides an automated live streaming production system for short videos based on big data, comprising: The real-time multimodal data acquisition layer receives video and audio streams via the RTMP / SRT protocol. The system receives bullet screen semantic data via the WebSocket interface, receives audience behavior data via the front-end event tracking SDK, and achieves time synchronization based on the PTP protocol. The big data real-time analysis engine extracts features from the video stream, bullet screen semantic data, and audience behavior data based on a three-modal feature extraction network. The modal weights are calculated through a dynamic weight fusion mechanism driven by information gain, and the explosive potential value of live content is calculated based on cross-modal collaborative gain. The intelligent slicing decision module dynamically adjusts the trigger threshold based on the potential value distribution within the sliding window. When the potential value exceeds the threshold, it searches for the optimal slice boundary within the neighborhood window based on a dual-modal collaborative strategy of audio keyword recognition and visual scene switching detection. The automated production module performs intelligent subtitle generation, emotional background music matching, and multi-platform format adaptation on the sliced ​​video to generate short video products that can be directly distributed. The instant distribution network and data feedback system achieve global coverage distribution through CDN edge nodes and collect propagation effect data to drive online learning and optimization of the model.

[0008] Furthermore, the big data real-time analysis engine includes a visual feature extraction network, an interaction popularity encoding network, a product click prediction network, and a dynamic weight fusion unit; The visual feature extraction network uses a 3D-CNN architecture to capture the spatiotemporal dynamic features of video frame sequences; the interactive heat encoding network uses a BiLSTM architecture to perform temporal modeling of bullet screen semantic data. The product click prediction network uses a DeepFM architecture to jointly model the interaction between low-order and high-order features; the dynamic weight fusion unit calculates dynamic weights based on the information gain of each modality.

[0009] Furthermore, the formula for calculating the burst potential value is as follows: P(t) = σ(Σw_i(t)·f_i(x_i) + λ·S(t)), Where P(t) is the burst potential value at time t, σ is the Sigmoid activation function, w_i(t) is the dynamic weight of the i-th mode, f_i(x_i) is the feature extraction function output of the i-th mode, λ is the cooperative gain adjustment coefficient, and S(t) is the cross-modal cooperative gain term.

[0010] Furthermore, the intelligent slicing decision module includes an adaptive threshold calculation unit, an audio boundary detection unit, a visual boundary detection unit, and a collaborative boundary refinement unit; The adaptive threshold calculation unit dynamically adjusts the trigger threshold based on the historical potential value distribution and the number of slices already generated. The audio boundary detection unit determines the audio boundary based on CTC keyword recognition and audio energy mutation detection. The visual boundary detection unit determines the visual boundary based on the inter-frame cosine distance and the KL divergence of the category distribution of the scene classifier.

[0011] Furthermore, the automated production module includes an intelligent subtitle generation subsystem, an emotional background music matching subsystem, and a multi-platform format adaptation subsystem; The intelligent subtitle generation subsystem achieves real-time subtitle generation based on a streaming ASR model and visual attention mechanism constraints. The emotional BGM matching subsystem achieves music matching based on an 8-dimensional emotional space model and BPM rhythm detection. The multi-platform format adaptation subsystem achieves multi-platform adaptation based on platform preset templates and semantically aware cropping.

[0012] On the other hand, the present invention also provides a method for automated live streaming production of short videos based on big data, including the following steps: S1: Real-time collection of live streaming data, receiving video streams and audio streams, bullet screen semantic data, receiving audience behavior data, and achieving time synchronization based on communication protocols; S2: Based on a three-modal feature extraction network, feature extraction is performed on the collected data. The weights of each modality are calculated through a dynamic weight fusion mechanism driven by information gain. The explosive potential value of the live content is calculated based on cross-modal collaborative gain. S3: Dynamically adjust the trigger threshold according to the potential value distribution within the sliding window. When the potential value exceeds the threshold, search for the optimal slice boundary within the neighborhood window based on the dual-modal collaborative strategy of audio keyword recognition and visual scene switching detection. S4: Performs intelligent subtitle generation, emotional background music matching, and multi-platform format adaptation on the sliced ​​video to generate a short video product that can be directly distributed; S5: Achieve global coverage distribution through CDN edge nodes and collect propagation effect data to drive online learning optimization of the model.

[0013] Furthermore, the calculation of the burst potential value in step S2 includes: Extracting spatiotemporal dynamic features of video streams based on 3D-CNN architecture; Temporal modeling of bullet screen semantic data based on BiLSTM architecture; Joint modeling of audience behavior data based on DeepFM architecture; Calculate dynamic weights based on the information gain of each modality; Calculate the final burst potential value based on cross-modal cooperative gain terms; The search for the optimal slice boundary in step S3 includes: Calculate the adaptive threshold; Based on CTC keyword recognition, the trigger time of audio keywords is detected, and the energy mutation point is detected by combining the first-order difference of audio energy. Shot switching points are detected based on inter-frame cosine distance, and scene switching points are detected based on the KL divergence of the category distribution of the scene classifier. Within the neighborhood window at the trigger time, the optimal slice boundary is searched based on the boundary loss function.

[0014] The beneficial effects of this invention are: Compared with the prior art, this application has at least the following technical effects: 1. This invention constructs a real-time multimodal data acquisition layer, simultaneously accessing video streams, audio streams, bullet screen semantic data, and audience behavior data, and achieves time synchronization based on the PTP protocol, ensuring the temporal alignment of multi-source heterogeneous data. After the data enters the big data real-time analysis engine, relying on the GPU cluster and model cascading mechanism, it completes trimodal feature extraction and burst potential value calculation in a very short time. Combined with a streaming processing architecture and a circular buffer out-of-order reordering mechanism, the system achieves an end-to-end latency of less than 5 seconds from live data access to potential value output, significantly outperforming the traditional post-analysis mode with a lag of tens of minutes to several hours, breaking through the timeliness bottleneck of capturing hot content.

[0015] 2. The dynamic weight fusion mechanism proposed in this invention dynamically adjusts the weights of each modality based on information gain, allowing modalities with higher information content to receive higher weights in decision-making. Simultaneously, a cross-modal collaborative gain term is introduced to enhance the accuracy of potential value assessment through multi-dimensional collaborative signals such as visual-interaction correlation, product click volatility, and mutual information. In a preferred embodiment, the algorithm for explosive potential value of this invention achieves an AUC of 0.87, an improvement of 11.5% compared to the fixed-weight fusion method, verifying the model's high discriminative ability in complex live streaming scenarios.

[0016] 3. The intelligent slicing decision module of this invention includes an adaptive threshold calculation unit, which dynamically adjusts the trigger threshold based on the potential value distribution of the sliding window and historical statistics to avoid high-frequency false triggering. Audio boundary detection is based on CTC keyword recognition and energy mutation detection, while visual boundary detection is based on inter-frame cosine distance and scene classification KL divergence. The two work together to search for the optimal slice boundary within the neighborhood window using a boundary loss function. In a preferred embodiment, the algorithm controls the boundary positioning error to within 1.2 seconds, which is more than 40% better than the single-modal method, ensuring the semantic integrity of the slice content and natural transitions.

[0017] 4. The automated production module proposed in this invention includes: an intelligent subtitle generation subsystem that integrates streaming ASR and visual attention mechanisms, enhancing keyword recognition accuracy through algorithms with a word error rate of less than 5%; an emotional BGM matching subsystem based on an 8-dimensional emotional space model and BPM rhythm detection, combined with FAISS vector retrieval to achieve a Top-5 recall rate of over 90% in a million-level music library; and a multi-platform format adaptation subsystem that achieves multi-resolution parallel output through semantic-aware cropping (saliency detection, face detection, and text region detection) and NVENC hardware encoding. All three components employ a pipelined parallel architecture, with a total production latency of less than 500ms, meeting the real-time distribution requirements of short videos.

[0018] 5. The real-time distribution network and data feedback system of this invention achieves global distribution through CDN edge nodes and tracks dissemination effect data such as completion rate, like rate, and conversion rate based on a multi-touchpoint attribution model. Data is asynchronously transmitted back to the online optimization unit for model parameters via a message queue, employing a delayed feedback attribution and incremental training strategy to support online model updates and canary releases. Combined with an A / B testing framework, the real-time distribution network and data feedback system can continuously optimize the explosive potential model, slice decision threshold, and background music (BGM) matching strategy based on actual dissemination effects, ensuring continuous evolution and performance improvement of the system in actual business operations. Attached Figure Description

[0019] Figure 1 A schematic diagram of the overall architecture of the automated live streaming production system for short videos based on big data provided in an embodiment of the present invention; Figure 2This is a schematic diagram of the structure of a real-time big data analysis engine provided in an embodiment of the present invention; Figure 3 A flowchart for calculating burst potential value provided in an embodiment of the present invention; Figure 4 A flowchart illustrating the workflow of the intelligent slice decision module provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of the automated production module provided in an embodiment of the present invention; Figure 6 This is a comparison chart of the accuracy of burst potential value prediction in simulation experiments provided by embodiments of the present invention; Figure 7 A comparison diagram of slice boundary positioning errors in a simulation experiment provided for embodiments of the present invention; Figure 8 A comparison chart of the short video propagation effect in a simulation experiment provided for an embodiment of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0022] In the description of this invention, the term "for example" is used to mean "used as an example, illustration, or description." Any embodiment described as "for example" in this invention is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use the invention. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that the invention can be made without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid obscuring the description of the invention with unnecessary detail. Therefore, the invention is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed herein.

[0023] Example 1 like Figure 1 As shown, this embodiment provides an automated live streaming production system for short videos based on big data, including the following modules: The real-time multimodal data acquisition layer handles the synchronous access and preprocessing of heterogeneous data sources.

[0024] The video stream is encoded in H.264 / H.265 and transmitted via RTMP or SRT protocols, with resolutions ranging from 720p to 4K and frame rates from 25 to 60fps. The audio stream is AAC encoded with a sampling rate of 44.1kHz; The semantic data of the bullet comments comes from the WebSocket interface of the live streaming platform and is pushed in JSON format encoded in UTF-8. Audience behavior data includes events such as clicks, swipes, dwell times, product exposure, clicks, adding to cart, and payments.

[0025] The real-time multimodal data acquisition layer uses the PTP protocol for time synchronization and a circular buffer to achieve time alignment and out-of-order rearrangement.

[0026] A real-time data analysis engine, deployed on a GPU cluster, performs streaming data processing and deep learning inference. The real-time data analysis engine runs a dedicated neural network, including a visual feature extraction network that uses a 3D-CNN architecture to capture spatiotemporal dynamics; Interactive heatmap encoding network models temporal evolution using LSTM; The product click prediction network uses DeepFM to jointly model the interaction between low-order and high-order features; The output of the dedicated neural network is dynamically weighted and cross-modal collaborative gain calculated to generate a burst potential value.

[0027] The intelligent slicing decision module receives the time series of potential values ​​and performs threshold judgment and precise boundary positioning. The intelligent slicing decision module employs an adaptive threshold mechanism, dynamically adjusting the trigger threshold based on the distribution of potential values ​​within the sliding window.

[0028] Boundary detection employs a dual-modal collaborative strategy of audio keyword recognition and visual scene switching detection, searching for the optimal start and end points within the neighborhood window at the trigger time.

[0029] The automated production module transforms raw video clips into short, ready-to-distribute short video products. It includes the following parallel subsystems: The intelligent subtitle generation subsystem achieves real-time speech-to-text conversion based on streaming ASR and visual context constraints; The emotional BGM matching subsystem retrieves matching music from the copyright music library through 8-dimensional emotion recognition and BPM rhythm detection. The multi-platform format adaptation subsystem incorporates the technical specifications of various mainstream platforms and automatically performs resolution scaling, safe region cropping, duration constraints, and encoding parameter optimization.

[0030] Real-time distribution networks and data feedback systems enable content delivery and performance tracking. The distribution network achieves global coverage based on CDN edge nodes and automates publishing to channels such as Douyin, Kuaishou, Video Accounts, and Bilibili through the platform's Open API. The data feedback system collects dissemination effect data such as completion rate, like rate, share rate, and conversion rate, and asynchronously sends them back to the analysis engine via a message queue to drive the online learning of the model.

[0031] Example 2 like Figure 2 and Figure 3 As shown, this embodiment provides a specific implementation of the burst potential value calculation model in a big data real-time analysis engine.

[0032] The visual feature extraction network employs a SlowFast-R50 architecture, processing 16 frames in the slow path and 64 frames in the fast path, outputting a 512-dimensional feature vector. This network was pre-trained on the Kinetics-400 dataset and then fine-tuned using 100,000 hours of live streaming data. The single-frame processing latency is approximately 45ms.

[0033] The interactive popularity encoding network adopts a 2-layer BiLSTM architecture with a hidden dimension of 128. It processes bullet screen data within a 30-second time window and outputs a 128-dimensional feature vector. It adopts an incremental calculation strategy, reusing the LSTM state when the window slides, and the single sequence processing latency is about 15ms.

[0034] The product click prediction network adopts the DeepFM architecture, with an FM embedding dimension of 16 and a DNN layer structure of [256, 128, 64]. This network is pre-trained using a product knowledge graph, supports real-time incremental feature updates, and has a single-sample inference latency of approximately 8ms.

[0035] The dynamic weight fusion mechanism calculates the weights of each modality based on information gain.

[0036] Information gain: IG_i(t) = H(Y) - H(Y|X_i), where H(Y) is the entropy of the target variable and H(Y|X_i) is the conditional entropy given the modal features.

[0037] Dynamic weights: w_i(t) = exp(IG_i(t) / τ) / Σexp(IG_j(t) / τ), where the temperature coefficient τ controls the sharpness of the weight distribution. Experiments have shown that setting it to 1.0 can achieve a balance between diversity and discriminability.

[0038] Cross-modal cooperative gain term: S(t) = ρ(V,I)·Var(C)·MI(V,C)·I(I>θ), where ρ(V,I) is the Pearson correlation coefficient between visual features and interaction popularity, Var(C) is the temporal variance of product click features, MI(V,C) is the mutual information between visual features and product clicks, and I(I>θ) is the indicator function for interaction popularity exceeding the threshold θ.

[0039] Burst potential value: P(t) = σ(Σw_i(t)·f_i(x_i) + λ·S(t)), where σ is the Sigmoid activation function and λ is the cooperative gain adjustment coefficient, which is set to 0.5 after optimization on the validation set.

[0040] Real-time inference is optimized using a three-level acceleration strategy, including: Model quantization (FP32→FP16, accuracy loss less than 0.5%), TensorRT graph optimization (throughput increased by 2-3 times), and dynamic batch processing (batch size 8-16). Furthermore, a model cascading and early exit mechanism are employed, and a lightweight filtering network (single-layer fully connected, 0.5ms) quickly filters low-potential periods. Precise network depth calculations are triggered only when the filtering score is greater than 0.3, reducing average computation by 60%-70%.

[0041] Example 3 like Figure 4 As shown, this embodiment provides a specific implementation method for the intelligent slicing decision module, including the following steps: The adaptive threshold is calculated using the following formula: θ(t) = θ_0·[1+α·(μ_w(t)-μ_g) / σ_g]·[1-β·N_c / N_max], Where θ_0 is the base threshold (empirical value 0.6), α is the environmental sensitivity coefficient (set to 0.3 in this embodiment), μ_w(t) is the average potential value within the sliding window (set to a length of 300 seconds in this embodiment), μ_g and σ_g are the global mean and standard deviation of the historical potential values, respectively, β is the slice frequency penalty coefficient (set to 0.2 in this embodiment), N_c is the number of slices generated in the current live session, and N_max is the maximum number of slices in a single session (set to 50 in this embodiment).

[0042] Audio boundary detection employs a Whisper-small model with CTC fine-tuning, a sampling rate of 16kHz, a 2-second processing window, and a character set size of 39. Predefined keyword sets for e-commerce scenarios include {"limited-time flash sale," "last three items," "link up," "countdown," "special offer price"}, etc. Keyword recall is greater than 92%, and false trigger rate is less than 8%. Simultaneously, first-order difference detection of audio energy is used to detect energy abrupt changes, with an energy abrupt change threshold set to 5dB.

[0043] Visual boundary detection utilizes ResNet50 to extract frame features (set to dimension 2048 in this embodiment), detects shot transition points based on inter-frame cosine distance, and detects scene transition points based on the KL divergence of the scene classifier's category distribution. The scene classifier supports multiple scene categories, with a classification accuracy greater than 85% and a single-frame latency of less than 10ms. The shot transition detection F1-score is greater than 0.88.

[0044] Collaborative boundary refinement searches for the optimal boundary within a neighborhood window (set to ±5 seconds in this embodiment) at the trigger time. Boundary loss function: L(b) = -logσ(b) + γ·|b-t_0| / W, where σ(b) is the normalized boundary confidence, γ is the distance penalty coefficient (set to 0.1 in this embodiment), t_0 is the trigger time, and W is the half-length of the window. Conflict resolution strategy: when the distance between the audio boundary and the visual boundary is less than 3 seconds, the midpoint is taken; when the distance is greater than 3 seconds, the higher confidence modality is used; when only one modality exists, the delay is up to 2 seconds.

[0045] Through the above steps, the slice boundary positioning error is less than 1.5 seconds, which is more than 40% better than the single-modal solution.

[0046] Example 4 like Figure 5 As shown, this embodiment provides a specific implementation method for the automated production module.

[0047] The intelligent subtitle generation subsystem is based on the Whisper-small model for domain fine-tuning, adopts a streaming decoding strategy, and sets the beam size to 5.

[0048] The formula for calculating visual attention mechanisms is: a_t = softmax(W_a·[h_a(t);h_v(t)]+b_a), where h_a(t) is the output of the audio encoder and h_v(t) is the output of the visual encoder. This allows the speech decoding process to "focus" on the visual regions related to the current speech content, improving the confidence in recognizing numerical words when a price tag is displayed on the screen. The word error rate is less than 5%, and the first-word delay is less than 200ms.

[0049] The emotional background music matching subsystem utilizes an emotional space model: {excitement, warmth, tension, sadness, anger, fear, neutrality, humor}.

[0050] Audio emotional features are extracted through a music emotion recognition model, which is composed of Liborosa features fused with CRNN; visual emotional features are extracted through a combination of facial expression recognition and scene emotion classification.

[0051] The music library search employed FAISS vector retrieval, with an emotional similarity weight of 0.6, a BPM matching weight of 0.3, and a tonality harmony weight of 0.1. After manual evaluation, the music library, with a size of millions of tracks, achieved a Top-5 recall rate exceeding 90% and a matching satisfaction rate exceeding 85%.

[0052] The dynamic mixing formula is: M(t) = α(t)·S(t) + (1-α(t))·B(t), where α(t) = sigmoid(VAD(t)-0.5), VAD(t) is the speech activity detection output, S(t) is the speech signal, and B(t) is the background music signal.

[0053] The multi-platform format adaptation subsystem includes built-in platform preset templates, including Douyin, Kuaishou, WeChat Video Channel, Bilibili, and others. Semantic-aware cropping is achieved through joint optimization of saliency detection, face detection, and text region detection, with the constraint that the cropped area must meet the target aspect ratio and not conflict with the platform's safe area.

[0054] Using FFmpeg in conjunction with NVENC hardware encoding, the single video encoding latency is less than 300ms, and it supports parallel output across multiple platforms.

[0055] The above subsystems adopt a pipelined parallel architecture, with subtitle generation, BGM matching, and format adaptation executed in parallel. The total production latency is ensured to be less than 500ms through buffer coordination and timeout degradation mechanisms.

[0056] Example 5 To verify the technical effects of this invention, a simulation experiment was conducted. The experimental environment was configured as follows: the GPU server used an NVIDIA A100 (80GB VRAM), the CPU was an Intel Xeon Platinum 8380, the memory was 512GB, and the network bandwidth was 10Gbps; the experimental dataset contained 1000 e-commerce live streams, with a total duration of approximately 3000 hours, covering multiple categories such as clothing, cosmetics, food, and digital products.

[0057] Simulation Experiment 1 (Accuracy of Burst Potential Value Prediction) The burst potential value calculation model of this invention is compared with baseline methods, including: single visual feature method, single interaction feature method, and fixed weight fusion method. The evaluation metric used is AUC (Area Under ROCCurve). Experimental results are as follows: Figure 6 As shown, the AUC of the method of the present invention reaches 0.87, which is 20.8% higher than the single visual feature method (AUC=0.72), 27.9% higher than the single interaction feature method (AUC=0.68), and 11.5% higher than the fixed weight fusion method (AUC=0.78).

[0058] The results show that the dynamic weight fusion mechanism and cross-modal collaborative gain calculation of the present invention significantly improve the prediction accuracy of the burst potential value.

[0059] Simulation Experiment 2 (Slice Boundary Positioning Accuracy) The bimodal collaborative slicing decision algorithm of this invention is compared with baseline methods, including: fixed threshold method, single audio boundary method, and single visual boundary method. The evaluation metric is the boundary localization error (the average time difference between the predicted boundary and the manually labeled boundary). Experimental results are as follows: Figure 7 As shown, the boundary positioning error of the method of the present invention is 1.2 seconds, which is 57.1% better than the fixed threshold method (error 2.8 seconds), 42.9% better than the single audio boundary method (error 2.1 seconds), and 47.8% better than the single visual boundary method (error 2.3 seconds).

[0060] The results show that the adaptive threshold mechanism and dual-modal collaborative boundary detection of the present invention significantly improve the localization accuracy of slice boundaries.

[0061] Simulation Experiment 3 (Short Video Dissemination Effect) A comparative experiment was conducted between the short videos generated by the system of this invention and manually edited short videos.

[0062] For the same live stream session, short videos were produced using the system of this invention and by professional editors, and published on the same platform within the same time period, while keeping other variables consistent. Evaluation metrics included completion rate, like rate, share rate, and conversion rate. Experimental results are as follows: Figure 8 As shown, the short video completion rate generated by the system of this invention reached 68%, which is 9.7% higher than that of manual editing (62%); the like rate reached 12.5%, which is 15.7% higher than that of manual editing (10.8%); the sharing rate reached 3.2%, which is 23.1% higher than that of manual editing (2.6%); and the conversion rate reached 5.8%, which is 18.4% higher than that of manual editing (4.9%).

[0063] The results show that the short videos generated by the system of this invention are superior to those edited manually in terms of dissemination, verifying the practical value of the system.

[0064] In summary, the big data-based automated live streaming production system and method provided by this invention achieves second-level quantitative evaluation and automated production of live streaming content through a complete closed loop of real-time multimodal data acquisition, real-time big data analysis, intelligent slicing decision-making, automated production, and instant distribution feedback. This effectively solves the technical problems of lagging hotspot capture, high production costs, and low secondary utilization rate of content in existing technologies.

[0065] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0066] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0067] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0068] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0069] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0070] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0071] Although preferred embodiments of the invention have been described, those skilled in the art, once they have learned the basic inventive concept, can make other changes and modifications to these embodiments.

[0072] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of this invention and its equivalents, this invention also intends to include these modifications and variations.

Claims

1. A short video automated live streaming production system based on big data, characterized in that: The composition is as follows: The real-time multimodal data acquisition layer receives video and audio streams, bullet screen semantic data, and audience behavior data, and performs time synchronization. The big data real-time analysis engine extracts features from the video stream, bullet screen semantic data, and audience behavior data based on a three-modal feature extraction network. It calculates the modal weights through a dynamic weight fusion mechanism driven by information gain and calculates the explosive potential value of the live content based on cross-modal collaborative gain. The intelligent slicing decision module dynamically adjusts the trigger threshold based on the potential value distribution within the sliding window. When the potential value exceeds the threshold, it searches for the optimal slice boundary within the neighborhood window based on a dual-modal collaborative strategy of audio keyword recognition and visual scene switching detection. The automated production module performs intelligent subtitle generation, emotional background music matching, and multi-platform format adaptation on the sliced ​​video to generate short video products that can be directly distributed. The instant distribution network and data feedback module distribute data globally through edge nodes and collect data on the propagation effect to drive the online learning and optimization of the model.

2. The automated live streaming production system for short videos based on big data as described in claim 1, characterized in that, The real-time big data analysis engine includes: A visual feature extraction network is used to capture the spatiotemporal dynamic features of video frame sequences using a 3D-CNN architecture. An interactive popularity encoding network is developed, which uses a BiLSTM architecture to perform temporal modeling of bullet screen semantic data. The product click prediction network uses the DeepFM architecture to jointly model the interaction between low-order and high-order features. The dynamic weight fusion unit calculates dynamic weights based on the information gain of each modality, and satisfies the formula: w_i(t) ∝ exp(IG_i(t) / τ), Where IG_i(t) is the information gain of the i-th mode at time t, and τ is the temperature coefficient.

3. The automated live streaming production system for short videos based on big data as described in claim 2, characterized in that, The formula for calculating the burst potential value is: P(t) = σ(Σw_i(t)·f_i(x_i) + λ·S(t)), Where P(t) is the burst potential value at time t, σ is the Sigmoid activation function, w_i(t) is the dynamic weight of the i-th mode, f_i(x_i) is the feature extraction function output of the i-th mode, λ is the cooperative gain adjustment coefficient, and S(t) is the cross-modal cooperative gain term.

4. The automated live streaming production system for short videos based on big data according to claim 2, characterized in that, The formula for calculating the cross-modal cooperative gain term S(t) is as follows: S(t) = ρ(V,I)·Var(C)·MI(V,C)·I(I>θ), Where ρ(V,I) is the Pearson correlation coefficient between visual features and interaction popularity, Var(C) is the temporal variance of product click features, MI(V,C) is the mutual information between visual features and product clicks, and I(I>θ) is the indicator function for interaction popularity exceeding the threshold θ.

5. The automated live streaming production system for short videos based on big data according to claim 1, characterized in that, The intelligent slicing decision module includes the following units: The adaptive threshold calculation unit uses the following formula: θ(t) = θ_0·[1+α·(μ_w(t)-μ_g) / σ_g]·[1-β·N_c / N_max] calculates the dynamic threshold, Where θ_0 is the base threshold, α is the environmental sensitivity coefficient, μ_w(t) is the average potential value within the sliding window, μ_g and σ_g are the global mean and standard deviation of the historical potential values, respectively, β is the slice frequency penalty coefficient, N_c is the number of slices generated in the current live session, and N_max is the maximum number of slices in a single session. The audio boundary detection unit detects the trigger time of audio keywords based on CTC keyword recognition and combines the first-order difference of audio energy to detect energy mutation points; The visual boundary detection unit detects shot switching points based on inter-frame cosine distance and scene switching points based on the KL divergence of the category distribution of the scene classifier. The collaborative boundary refinement unit searches for the optimal slice boundary based on the boundary loss function within the neighborhood window at the trigger time.

6. The automated live streaming production system for short videos based on big data according to claim 1, characterized in that, The automated production module includes the following subsystems: The intelligent subtitle generation subsystem converts audio streams into text based on a streaming ASR model and improves subtitle accuracy through a visual attention mechanism. The formula for calculating the visual attention weight is as follows: a_t = softmax(W_a·[h_a(t);h_v(t)]+b_a), Where h_a(t) is the output of the audio encoder and h_v(t) is the output of the visual encoder; The emotional BGM matching subsystem identifies the emotional features of video clips based on an 8-dimensional emotional space model, retrieves matching music from the copyright music library through BPM rhythm detection and tonality harmony analysis, and performs dynamic mixing based on speech activity detection. The multi-platform format adaptation subsystem performs resolution scaling, semantic-aware cropping, and hardware encoding acceleration based on platform preset templates. The semantic-aware cropping is achieved through joint optimization of saliency detection, face detection, and text region detection.

7. The automated live streaming production system for short videos based on big data according to claim 1, characterized in that, The instant delivery network and data feedback system include the following units: The real-time user profile matching unit achieves accurate matching of content and users based on vector retrieval and real-time feature updates. The dissemination effect full-link tracking unit tracks the dissemination effect based on the multi-touchpoint attribution model of exposure-interaction-conversion, and provides a visual dashboard and anomaly alerts; The online model parameter optimization unit enables online updates of model parameters based on delayed feedback attribution and incremental training, and supports canary releases and A / B testing frameworks.

8. The automated live streaming production system for short videos based on big data according to claim 2, characterized in that, The real-time big data analysis engine also includes model cascading and early exit mechanisms, including: A lightweight filtering network for rapid pre-screening of input data; The precise calculation network performs in-depth calculations on data whose scores exceed a preset threshold. Among these measures, a screening score threshold is set, and the average calculation amount is reduced through cascading design.

9. A method for automated live streaming production of short videos based on big data, characterized in that: Includes the following steps: S1 collects live streaming data in real time, receiving video and audio streams, bullet screen semantic data, and audience behavior data, and achieves time synchronization based on communication protocols; S2 extracts features from the collected data based on a three-modal feature extraction network, calculates the modal weights through a dynamic weight fusion mechanism driven by information gain, and calculates the explosive potential value of live content based on cross-modal collaborative gain. S3 dynamically adjusts the trigger threshold based on the potential value distribution within the sliding window. When the potential value exceeds the threshold, the optimal slice boundary is searched within the neighborhood window based on a dual-modal collaborative strategy of audio keyword recognition and visual scene switching detection. S4 performs intelligent subtitle generation, emotional background music matching, and multi-platform format adaptation on the sliced ​​video to generate a short video product that can be directly distributed. S5 achieves global coverage distribution through CDN edge nodes and collects propagation effect data to drive online learning and optimization of the model.

10. The method for automated live streaming production of short videos based on big data according to claim 9, characterized in that, The calculation of the burst potential value in step S2 includes the following steps: S201 extracts spatiotemporal dynamic features of video streams based on a 3D-CNN architecture; S202 performs temporal modeling of bullet screen semantic data based on the BiLSTM architecture; S203 uses the DeepFM architecture to jointly model audience behavior data; S204 Calculates dynamic weights based on the information gain of each modality; S205 calculates the final burst potential value based on cross-modal cooperative gain terms; The search for the optimal slice boundary in step S3 of S206 includes the following steps: S301 calculates the adaptive threshold; The S302 detects the trigger time of audio keywords based on CTC keyword recognition and combines the first-order difference of audio energy to detect energy mutation points; S303 detects camera switching points based on inter-frame cosine distance and scene switching points based on the KL divergence of the scene classifier's category distribution. S304 searches for the optimal slice boundary within the neighborhood window at the trigger time based on the boundary loss function.

Images