Sentiment Analysis and Strategy Optimization System Based on Multimodal Acquisition and Large Language Model

Abstract

This invention relates to the field of artificial intelligence technology, specifically disclosing a sentiment analysis and strategy optimization system based on multimodal data acquisition and a large language model. The system acquires real-time data from bullet comments, user behavior, and live stream content through a multimodal data acquisition module. It then performs cross-modal fusion and semantic parsing via a domain-knowledge-enhanced sentiment analysis engine. Based on a multi-dimensional strategy generation module, it outputs a set of strategies adapted to different optimization objectives. Finally, a strategy dynamic execution and feedback module achieves strategy evaluation, execution, and closed-loop optimization. This invention can improve the accuracy of sentiment analysis and the relevance of strategy generation, enhancing the real-time nature of live stream interaction and user engagement.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, specifically relating to a sentiment analysis and strategy optimization system based on multimodal acquisition and a large language model. Background Technology

[0002] Sentiment analysis is an important branch of artificial intelligence and natural language processing. It aims to identify, extract, and quantify subjective emotional information contained in text through computational models, and is widely used in scenarios such as public opinion monitoring, product feedback, and user experience analysis. Text-based sentiment analysis technology primarily achieves automatic discrimination of emotional polarity by modeling linguistic features.

[0003] Sentiment analysis and strategy optimization for live streaming interactive scenarios is an emerging technological direction in recent years. This direction aims to analyze user comments and interactive content generated during live streaming in real time, and provide data support for optimizing live streaming content and adjusting interactive strategies based on sentiment analysis results, thereby improving user engagement and live streaming effectiveness.

[0004] Existing technologies primarily rely on static text sentiment classification models, which are ill-suited to the dynamic, real-time, and fragmented characteristics of live-streaming comments. Due to a lack of deep integration with specific background knowledge in the live-streaming domain, existing models suffer from insufficient accuracy in sentiment recognition for short texts, internet slang, and non-standard symbols.

[0005] Existing systems typically process multi-dimensional data such as bullet screen text, user behavior, and live stream content in a fragmented manner, failing to establish cross-modal sentiment associations. This makes it difficult to provide targeted and actionable strategy recommendations for real-time optimization of live stream interactions as a whole. Therefore, how to build a sentiment analysis system that can deeply integrate multi-modal data, possess domain knowledge awareness capabilities, and dynamically output optimization strategies has become a pressing technical challenge in this field. Summary of the Invention

[0006] This invention aims to provide a sentiment analysis and strategy optimization system based on multimodal acquisition and a large language model, in order to solve the technical contradictions of existing technologies in live interactive scenarios, such as difficulty in handling dynamic, real-time, and fragmented bullet screen data, lack of deep integration of domain knowledge, and inability to associate multimodal information to generate targeted strategies.

[0007] This invention provides a sentiment analysis and strategy optimization system based on multimodal data acquisition and a large language model. The system includes a multimodal data acquisition module, a domain-knowledge-enhanced sentiment analysis engine, a multi-dimensional strategy generation module, and a strategy dynamic execution and feedback module.

[0008] The multimodal data acquisition module is responsible for capturing user text comments, user interaction data, and audio and video stream data of the live broadcast content in real time.

[0009] The domain-knowledge-enhanced sentiment analysis engine is built on a large language model. By injecting a professional knowledge base from the live streaming domain, it performs deep fusion and sentiment semantic analysis on the collected multimodal data.

[0010] The multi-dimensional strategy generation module receives the output from the sentiment analysis engine and generates a set of corresponding interaction strategies based on multiple preset optimization objectives.

[0011] The strategy dynamic execution and feedback module is responsible for prioritizing and risk-assessing the generated strategies, selecting the optimal strategy for execution, and continuously monitoring the execution effect to form a closed-loop optimization.

[0012] Furthermore, the multimodal data acquisition module specifically includes a text comment acquisition unit, a user behavior acquisition unit, and a live broadcast content parsing unit.

[0013] The text comment acquisition unit obtains the bullet screen text stream in real time through the application programming interface provided by the live streaming platform.

[0014] The user behavior collection unit records user click behavior, gift-giving records, and dwell time data in the live broadcast room.

[0015] The live content parsing unit processes the live audio and video streams in real time, extracts key frame images and performs scene recognition, and converts the anchor's voice into text through speech recognition technology.

[0016] Furthermore, the domain knowledge-enhanced sentiment analysis engine includes a domain knowledge injection layer, a multimodal fusion layer, and a sentiment semantic parsing layer.

[0017] The domain knowledge injection layer embeds a pre-built live streaming domain knowledge base, including a dictionary of popular internet slang, a library of terms used by live streamers, and a glossary of industry terms, into the input representation of the large language model in a vectorized manner.

[0018] The multimodal fusion layer employs a cross-modal attention mechanism to align and fuse text comments, user behavior features, and live stream content features, generating a unified context-aware representation.

[0019] The sentiment semantic parsing layer, based on the fused representation, performs deep semantic reasoning through the multi-head self-attention mechanism of the large language model, and outputs fine-grained sentiment analysis results including sentiment polarity, sentiment intensity and sentiment topic.

[0020] Furthermore, the multi-dimensional strategy generation module includes a strategy objective definition unit, a strategy candidate generation unit, and a strategy preliminary screening unit.

[0021] The strategy goal definition unit pre-sets multiple optimization goals, including improving user interaction rate, increasing user retention rate, and mitigating negative emotions.

[0022] For each optimization objective, the strategy candidate generation unit combines the current sentiment analysis results with the historical strategy library and generates multiple candidate strategy descriptions using the generation capabilities of a large language model.

[0023] The initial strategy screening unit performs preliminary filtering of candidate strategies based on the matching degree between the strategy and the target, as well as the execution complexity of the strategy.

[0024] Furthermore, the strategy dynamic execution and feedback module includes a strategy evaluation and ranking unit, a strategy execution unit, and an effect monitoring and feedback unit.

[0025] The strategy evaluation and ranking unit constructs a strategy utility function, which comprehensively considers the expected benefits, execution costs and potential risks of the strategy, and performs quantitative scoring and priority ranking on the strategies that pass the initial screening.

[0026] The strategy execution unit automatically sends the highest priority strategies to the live streaming interaction system through the control interface, such as adjusting the bullet screen display strategy, triggering specific interactive activities, or modifying the live streaming content recommendation sequence.

[0027] Within a preset time window after strategy execution, the effect monitoring and feedback unit continuously collects key performance indicator data and sends the effect data back to the strategy evaluation and ranking unit to update the parameters in the strategy utility function, thereby enabling online learning and optimization of the strategy generation model.

[0028] Furthermore, the process of building the domain knowledge base includes crawling raw text data from historical live broadcast recordings, industry reports, and user discussion communities, extracting key entities and relationships through a large language model, and forming a structured knowledge graph after review by domain experts.

[0029] The knowledge graph is stored in the form of entity attributes and relationships, and is vectorized using graph neural networks to facilitate its integration into large language models.

[0030] Furthermore, the specific implementation process of the cross-modal attention mechanism is as follows: firstly, the text comment features, user behavior features, and live broadcast content features are projected into the same latent space to obtain the corresponding feature sequences.

[0031] Then, the cross-attention weights between the text feature sequence and the live content feature sequence are calculated to generate a content-enhanced text representation.

[0032] Simultaneously, the cross-attention weights between user behavior feature sequences and content-enhanced text representations are calculated, and the final output is a multimodal unified representation that integrates user behavior context.

[0033] Furthermore, the components of the strategy utility function include the strategy expected return subfunction, the strategy execution cost subfunction, and the strategy risk penalty subfunction.

[0034] The strategy expected return subfunction associates different key performance indicators with the strategy type and predicts the magnitude of their changes.

[0035] The strategy execution cost subfunction quantifies the technical resources and time overhead required to implement the strategy.

[0036] The strategy risk penalty subfunction assesses the probability and severity of negative user feedback that a strategy may trigger.

[0037] The total utility score of a strategy is calculated by subtracting the execution cost score from the expected return score and then subtracting the risk penalty score.

[0038] Furthermore, the system operates within a hierarchical control architecture, which includes a real-time decision-making layer and a periodic optimization layer.

[0039] The real-time decision-making layer processes the bullet screen stream on a second-level timescale and executes high-priority, low-risk instant interaction strategies.

[0040] The cycle optimization layer comprehensively analyzes data over longer time windows on a minute-level timescale, generates and executes comprehensive strategies involving content adjustments or large-scale event planning, and is also responsible for updating the domain knowledge base and strategy model parameters.

[0041] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0042] 1. By constructing a domain-knowledge-enhanced sentiment analysis engine and employing a cross-modal attention mechanism to deeply integrate text, user behavior, and live streaming content information, the system can accurately analyze the sentiment semantics of short texts, internet slang, and non-standard symbols in live streaming scenarios. This improves the accuracy and context relevance of sentiment analysis and overcomes the shortcomings of traditional models in understanding domain-specific expressions.

[0043] 2. By establishing a multi-dimensional strategy generation module and a strategy dynamic execution and feedback module, the system can automatically generate and quantitatively evaluate multiple strategies for different optimization goals based on fine-grained sentiment analysis results. This enables intelligent selection, execution, and closed-loop optimization of strategies, providing real-time, accurate, and operable strategy support for live streaming interaction and improving user engagement and the overall effect of the live stream.

[0044] 3. The hierarchical control architecture and online learning mechanism adopted by the system ensure that the system can respond to real-time interactive needs that change at the second level, and can also perform in-depth strategy optimization and model self-updating at the minute level. It has good scalability and adaptability and can continuously adapt to the dynamic evolution of the live streaming scenario. Attached Figure Description

[0045] Figure 1 This is a schematic diagram of the overall technical architecture of the sentiment analysis and strategy optimization system based on multimodal acquisition and large language model proposed in this invention.

[0046] Figure 2 This is a schematic diagram of the core principle framework of the domain knowledge-enhanced sentiment analysis engine in this invention;

[0047] Figure 3 This is a logical flow diagram of the multi-dimensional strategy generation module in this invention;

[0048] Figure 4 This is a schematic diagram of the multi-level interaction relationship and data flow of the strategy dynamic execution and feedback module in this invention;

[0049] Figure 5 This is a schematic diagram illustrating the interaction between the real-time decision-making layer and the periodic optimization layer in the hierarchical control architecture of this invention. Detailed Implementation

[0050] Please refer to the attached document. Figure 1 This embodiment details the specific technical implementation of a sentiment analysis and strategy optimization system based on multimodal data acquisition and a large language model. The system constructs a complete technical closed loop from multimodal data perception to intelligent strategy generation and execution. Its core lies in leveraging domain knowledge enhancement and the capabilities of a large language model to perform deep semantic understanding and strategy optimization on dynamic and fragmented information in live interactive scenarios.

[0051] The system's infrastructure layer is deployed in a cloud computing environment, including a cluster of high-performance graphics processing units (GPUs) for model inference computation and a distributed stream processing platform for real-time data access and processing. The system connects to the open interfaces of mainstream live streaming platforms via a dedicated network to ensure the real-time performance and stability of data acquisition. The system's main control program is developed using the Python programming language, and the core large language model is based on a pre-trained open-source model with hundreds of billions of parameters, further fine-tuned for domain adaptation.

[0052] The multimodal data acquisition module is the entry point for the system to perceive external information.

[0053] This module contains three highly specialized data collection units: a text comment collection unit, a user behavior collection unit, and a live stream content analysis unit.

[0054] The text comment collection unit continuously retrieves real-time bullet screen text stream data from a specified live stream room by calling the standard application programming interface provided by the live streaming platform at a request frequency of no less than 100 times / second.

[0055] This unit has built-in data deduplication and spam filtering algorithms, which can identify and remove completely duplicate bullet screen content and invalid information that matches the preset spam word pattern.

[0056] All collected raw bullet screen text is appended with a timestamp accurate to the millisecond level and categorized and stored in a distributed cache database according to the live room identifier and the user anonymization identifier.

[0057] The user behavior collection unit focuses on capturing users' non-textual interaction behaviors within the live streaming room.

[0058] This unit listens to event tracking on the front-end page of the live stream and records every mouse click by the user, including but not limited to operations on live stream controls, clicks on product links, and participation in interactive polls.

[0059] Meanwhile, this unit obtains users' gift-giving records by parsing the platform server logs, including the gift type, value, and time of gift-giving.

[0060] In addition, the unit also samples and records the duration of a user's stay in the current live stream at a frequency of 1 time per second. All of this behavioral data is quantified into structured event sequences, with each event containing the behavior type, intensity value, occurrence time, and anonymous user identifier.

[0061] The live content parsing unit is responsible for processing the raw audio and video stream data generated during the live broadcast.

[0062] This unit connects to the real-time audio and video stream push interface of the live streaming platform to continuously decode the video stream and extract key frames.

[0063] The keyframe extraction algorithm adopts an adaptive strategy based on scene change detection. When a change in the video content is detected, such as when the host switches the presentation content or the camera angle changes abruptly, the current frame is immediately captured as a keyframe.

[0064] For each keyframe image extracted, this unit calls a pre-trained deep convolutional neural network model to perform scene recognition and object detection, and outputs a set of semantic labels for the image content, such as indoor environment, outdoor scene, close-up of a specific product, etc.

[0065] For audio streams, this unit integrates a high-performance speech recognition engine to convert the broadcaster's speech into a text transcript in real time.

[0066] This speech recognition engine has been optimized for accents and speaking speed in live streaming scenarios, and can handle the fast-paced, colloquial expressions commonly used by live streamers.

[0067] The final output of the live content parsing unit is a series of semantic tag sequences and speech text sequences with time alignment information.

[0068] The domain-knowledge-enhanced sentiment analysis engine is the core of the entire system's intelligence; please refer to the appendix for its internal structure. Figure 2 .

[0069] The primary component of this engine is the domain knowledge injection layer. The construction of the domain knowledge base is an offline preprocessing process.

[0070] The system crawls massive amounts of text data from three main sources: speech-recognized text from historical live stream recordings, full-text live stream industry analysis reports, and posts and comments from large user discussion communities.

[0071] After data cleaning and noise reduction, the raw text data is input into a dedicated large language model for information extraction.

[0072] The model was trained to identify and extract specific entities in the live streaming field, such as streamer stage names, popular internet memes, popular gaming terms, product brand names, and the relationships between these entities, such as a streamer's expertise in a particular game, or the origin and meaning of a certain internet meme.

[0073] The extracted entities and relationships are manually reviewed and corrected by a team of hired experts in the live streaming field, and finally a structured domain knowledge graph containing hundreds of thousands of nodes and relationship edges is constructed.

[0074] The knowledge graph was then embedded using a graph neural network algorithm, mapping each entity and relation to a high-dimensional dense vector.

[0075] During system operation, the domain knowledge injection layer receives bullet screen text from the text comment collection unit in real time.

[0076] For each bullet comment, the injection layer first performs word segmentation, then queries the domain knowledge graph to identify the domain entities appearing in the text.

[0077] Next, the vector representations corresponding to these entities are concatenated with the word vectors of other common words in the bullet screen text to form an enhanced text input representation, which is then fed into the downstream large language model.

[0078] This injection method is equivalent to equipping the model with a professional dictionary and background knowledge manual for the live streaming field.

[0079] The second core component of the sentiment analysis engine is the multimodal fusion layer.

[0080] This layer is responsible for aligning and deeply integrating data streams from different modalities.

[0081] Specifically, the text comment feature sequence, user behavior feature sequence, and live broadcast content feature sequence are first fed into three independent linear projection layers.

[0082] These projection layers map features from different modalities into the same 512-dimensional latent semantic space, ensuring their semantic comparability. Subsequently, the system initiates a cross-modal attention mechanism.

[0083] The mechanism first calculates the interaction between the text feature sequence and the live content feature sequence. Specifically, the text feature sequence is used as the query vector sequence, and the live content feature sequence is used as the key vector sequence and value vector sequence. Attention weights are calculated using the scaled dot product attention formula.

[0084] This process can be represented as:

[0085] :

[0086] The query matrix Q is composed of a sequence of text features, the key matrix K and the value matrix V are composed of a sequence of live content features, and the scaling factor in the denominator is the square root of the key vector dimension 512.

[0087] The calculated attention weight matrix reflects the correlation strength between each word in the text and each semantic element in the live content.

[0088] By weighted summing of the feature value vectors of live content, a text representation with enhanced content is generated.

[0089] Next, the system further integrates user behavior context.

[0090] The enhanced text representation obtained in the previous step is used as a new query vector sequence, and the user behavior feature sequence is used as a key and value vector sequence. The cross-attention calculation described above is then performed again.

[0091] The final output multimodal unified representation is a context-aware vector sequence that deeply integrates the semantics of the current bullet screen text, the scene information of the real-time live broadcast, and the behavioral tendencies of the user group.

[0092] The final component of a sentiment analysis engine is the sentiment semantic parsing layer.

[0093] This layer takes the unified representation output by the multimodal fusion layer as input and feeds it into the core large language model for deep semantic reasoning.

[0094] Large language models utilize their powerful internal multi-head self-attention mechanism to model the global dependencies of input sequences and capture long-distance semantic associations.

[0095] The model's output header is specifically designed to contain three parallel prediction branches.

[0096] The first branch is the sentiment polarity classifier, which outputs the probability of the positive, negative, or neutral sentiment expressed by the comment.

[0097] The second branch is the sentiment intensity regressor, which outputs a continuous value between 0 and 1, quantifying the intensity of the sentiment. The third branch is the sentiment topic extractor, which, based on the input context, generates one or more phrases describing the core theme to which the sentiment points, such as comments on the streamer's actions, complaints about product prices, or expectations for interactive activities.

[0098] The final output of the sentiment semantic parsing layer is a structured sentiment analysis result object, which contains fine-grained information in three dimensions: sentiment polarity, sentiment intensity, and sentiment theme.

[0099] The multi-dimensional strategy generation module receives fine-grained output from the sentiment analysis engine and generates targeted interaction strategies accordingly.

[0100] Please refer to the appendix for the internal logic flow of this module. Figure 3 .

[0101] The module begins with the strategy objective definition unit. This unit predefines three core optimization objectives, and these objectives and their associated key performance indicators are embedded in the system configuration file.

[0102] The first objective is to increase user interaction rate, with related metrics being the total number of bullet comments sent and the number of clicks on the interaction button per unit of time.

[0103] The second objective is to increase user retention, with the relevant metric being the ratio of average viewing time per user in the live stream to the conversion rate of new users into regular users.

[0104] The third objective is to alleviate negative emotions, with related indicators being the proportion of negative emotion comments and the frequency of user complaints.

[0105] Each objective is assigned a base weight coefficient, which the system administrator can adjust according to the specific operational stage of the live stream.

[0106] Next, the strategy candidate generation unit begins its work.

[0107] This unit maintains a historical strategy library, which stores strategy templates that have been successfully executed in the past and their execution results.

[0108] At any given moment, this unit combines the real-time sentiment analysis results and, for each optimization objective, invokes the text generation capabilities of a large language model to automatically generate natural language descriptions of multiple candidate strategies.

[0109] For example, when negative sentiment themes are detected to be concentrated on excessively long waiting times, a candidate strategy description might be generated to target retention rates: launching a fun quiz activity during the waiting period.

[0110] For the interaction rate target, a possible candidate strategy description is to publish a guessing bullet screen about the upcoming content.

[0111] To address the goal of alleviating negative emotions, a possible strategy is for the broadcaster to provide verbal explanations and commitments.

[0112] Each generated candidate strategy description is automatically parsed and structured, extracting key attributes such as strategy type, expected target, and required resources.

[0113] Subsequently, the initial strategy screening unit performs the first round of filtering on the generated candidate strategy set.

[0114] This unit has a built-in rule engine that evaluates the matching degree between each candidate strategy and the target, and calculates a matching degree score, with a score range of 0-100.

[0115] At the same time, the execution complexity of the strategy is evaluated. Based on the strategy description, it is analyzed whether it needs to call special interfaces, whether it involves complex logical judgments, and the expected computing resources consumed, and a complexity level is given, which is divided into three levels: low, medium and high.

[0116] The initial screening unit will filter out candidate strategies with a matching score below 60 or a high complexity level, ensuring that subsequent evaluation resources are focused on high-potential and feasible solutions.

[0117] The strategy dynamic execution and feedback module is responsible for transforming the selected strategies into actual actions and forming a learning loop.

[0118] For detailed interaction and data flow information for this module, please refer to the appendix. Figure 4 .

[0119] The core of the module is the strategy evaluation and ranking unit.

[0120] This unit constructs a comprehensive policy utility function to quantitatively score each candidate policy that passes the initial screening.

[0121] The policy utility function is a linear combination of three sub-functions.

[0122] The strategy expected return subfunction associates one or more key performance indicators that are most likely to be affected by the strategy type, and predicts the improvement of the relevant indicators after the strategy is implemented based on data from similar historical scenarios, and converts the improvement into a return score between 0 and 100.

[0123] The strategy execution cost subfunction quantifies the technical resources and time costs required to implement the strategy. For example, the cost of calling an interactive activity interface is recorded as 10 points, and the cost of modifying a recommendation sequence is recorded as 5 points. Finally, the total cost is converted into a cost score between 0 and 100.

[0124] The strategy risk penalty sub-function assesses the probability and severity of negative user feedback that a strategy may trigger. For example, an overly aggressive promotion strategy may cause user aversion. This function will give a risk penalty score between 0 and 50 based on the degree to which the strategy content matches the current emotional tone of the user.

[0125] The formula for calculating the total utility score of a strategy is: Total utility score equals expected return score minus execution cost score minus risk penalty score.

[0126] The strategy execution unit receives the most efficient score strategy from the evaluation unit.

[0127] This unit is deeply integrated with the backend management system of the live streaming room, providing a series of standardized control interfaces.

[0128] Depending on the specific type of strategy, the execution unit will automatically call the corresponding interface.

[0129] For example, when adjusting the bullet screen display strategy, the execution unit will call the bullet screen filtering and priority setting interface to display specific types of positive bullet screens at the top.

[0130] When a specific interactive activity is triggered, the execution unit will call the activity publishing interface and pop up a voting, lottery, or Q&A activity window on the live broadcast interface.

[0131] When modifying the recommended sequence of live content, the execution unit will call the intervention interface of the recommendation algorithm engine to temporarily adjust the display order of related products or videos.

[0132] All actions are recorded in detail in the log, including execution time, strategy content, execution interface parameters, etc.

[0133] The effect monitoring and feedback unit starts immediately after the strategy is executed. This unit sets a fixed effect observation time window, typically 5 minutes after the strategy is executed.

[0134] Within this window, the unit continuously collects key performance indicator data related to the strategy objectives. For example, after implementing a strategy to increase interaction rates, it focuses on monitoring changes in the frequency of bullet comments and the click rate of interactive buttons within the following 5 minutes.

[0135] The collected performance data is compared with the baseline data before the strategy is implemented to calculate the actual performance gain of the strategy.

[0136] These performance data are transmitted back to the strategy evaluation and ranking unit in real time.

[0137] The policy evaluation and ranking unit maintains an online learning mechanism for each parameter in the policy utility function.

[0138] It utilizes newly collected performance data, combined with corresponding policy characteristics, and employs the stochastic gradient descent algorithm to fine-tune the model parameters in the utility function related to profit prediction, cost estimation, and risk assessment. This enables the next policy evaluation to more accurately reflect the true value of the policy, thereby achieving closed-loop optimization and self-learning of the system.

[0139] The overall operation of the system is coordinated by a hierarchical control architecture; please refer to the appendix for the interaction relationships. Figure 5 The architecture clearly distinguishes between the real-time decision-making layer and the periodic optimization layer.

[0140] The real-time decision-making layer operates on a second-level timescale, with its data processing cycle set to 1 second.

[0141] This layer focuses on processing high-throughput bullet screen data, performing forward reasoning in a domain-knowledge-enhanced sentiment analysis engine, and quickly generating and executing high-priority, low-risk, and fast-acting real-time interaction strategies, such as simple bullet screen replies, emoji triggers, or lightweight interactive reminders.

[0142] The goal of the real-time decision-making layer is to maintain the live stream's immediate activity and responsiveness.

[0143] The cycle optimization layer operates on a minute-level timescale, with its analysis cycle typically set to 5 minutes.

[0144] This layer receives aggregated data from the real-time decision-making layer, including statistics on sentiment analysis results over the past 5 minutes, a summary of the effects of implemented strategies, and trend analysis of user behavior patterns.

[0145] Based on this more comprehensive data, the cycle optimization layer is responsible for generating and executing more complex medium- and long-term strategies that involve resource allocation or content planning, such as planning a 10-minute themed interactive session, adjusting the schedule of subsequent live broadcast content, or launching a round of wake-up pushes for inactive users.

[0146] Meanwhile, the periodic optimization layer is also responsible for the more demanding task of model updates, including incremental updates to the domain knowledge graph using newly accumulated data, and periodic retraining of the policy generation model and utility function evaluation model to ensure that the system can adapt to the long-term evolution of the live streaming ecosystem.

[0147] The two layers exchange data and synchronize instructions through shared memory and message queues, ensuring that the entire system achieves the best balance between response speed and decision depth.

[0148] This embodiment, through the detailed technical implementation described above, fully demonstrates how a sentiment analysis and strategy optimization system based on multimodal acquisition and a large language model goes through the entire process from bottom-level data acquisition, through the intelligent analysis and decision-making of the intermediate layer, to the final top-level strategy execution and optimization.

[0149] By deeply integrating domain knowledge, adopting advanced cross-modal attention mechanisms, constructing a quantitative strategy evaluation system, and implementing hierarchical control and online learning, this system achieves accurate perception, intelligent decision-making, and continuous optimization of live interactive scenarios, thereby improving the automation and intelligence level of live interactive experiences.

[0150] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0151] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A sentiment analysis and strategy optimization system based on multimodal acquisition and a large language model, characterized in that, include: The multimodal data acquisition module is used to capture user text comments, user interaction behavior data, and audio and video stream data of the live broadcast in real time. A domain-knowledge-enhanced sentiment analysis engine is used to perform deep fusion and sentiment semantic analysis on multimodal data collected by the multimodal data acquisition module by injecting a professional knowledge base in the live streaming domain; The multi-dimensional strategy generation module receives the output from the sentiment analysis engine and generates a corresponding set of interaction strategies based on multiple preset optimization objectives. The strategy dynamic execution and feedback module is used to prioritize and assess the risks of strategies generated by the multi-dimensional strategy generation module, select the optimal strategy for execution, and continuously monitor the execution effect to form a closed-loop optimization. The domain knowledge-enhanced sentiment analysis engine includes a domain knowledge injection layer, a multimodal fusion layer, and a sentiment semantic parsing layer. The domain knowledge injection layer embeds a pre-built live streaming domain knowledge base into the input representation of the large language model in a vectorized manner; The multimodal fusion layer employs a cross-modal attention mechanism to align and fuse text comments, user behavior features, and live stream content features, generating a unified context-aware representation. The sentiment semantic parsing layer, based on the fused representation, performs deep semantic reasoning through the multi-head self-attention mechanism of the large language model, and outputs fine-grained sentiment analysis results including sentiment polarity, sentiment intensity and sentiment topic; The strategy dynamic execution and feedback module includes a strategy evaluation and ranking unit, a strategy execution unit, and an effect monitoring and feedback unit. The strategy evaluation and ranking unit constructs a strategy utility function, which comprehensively considers the expected benefits, execution costs and potential risks of the strategy, and performs quantitative scoring and priority ranking on the strategies that pass the initial screening. The strategy execution unit automatically sends the highest priority strategy to the live interactive system through the control interface; The effect monitoring and feedback unit continuously collects key performance indicator data within a preset time window after the strategy is implemented, and sends the effect data back to the strategy evaluation and ranking unit to update the parameters in the strategy utility function.

2. The sentiment analysis and strategy optimization system based on multimodal acquisition and large language model according to claim 1, characterized in that, The multimodal data acquisition module includes a text comment acquisition unit, a user behavior acquisition unit, and a live content parsing unit; The text comment acquisition unit obtains the bullet screen text stream in real time through the application programming interface provided by the live streaming platform; The user behavior collection unit records users' click behavior, gift-giving records, and dwell time data in the live broadcast room; The live content parsing unit processes the live audio and video streams in real time, extracts key frame images and performs scene recognition, and converts the anchor's voice into text through speech recognition technology.

3. The sentiment analysis and strategy optimization system based on multimodal acquisition and a large language model according to claim 1, characterized in that, The multi-dimensional strategy generation module includes a strategy target definition unit, a strategy candidate generation unit, and a strategy preliminary screening unit. The strategy objective definition unit pre-sets multiple optimization objectives, including improving user interaction rate, increasing user retention rate, and mitigating negative emotions; For each optimization objective, the strategy candidate generation unit combines the current sentiment analysis results with the historical strategy library and generates multiple candidate strategy descriptions using the generation capabilities of the large language model. The initial strategy screening unit performs preliminary filtering of candidate strategies based on the matching degree between the strategy and the target, as well as the execution complexity of the strategy.

4. The sentiment analysis and strategy optimization system based on multimodal acquisition and large language model according to claim 1, characterized in that, The process of constructing the knowledge base in the live streaming domain includes crawling raw text data from historical live streaming recordings, industry reports, and user discussion communities, extracting key entities and relationships through a large language model, and forming a structured knowledge graph after review by domain experts. This knowledge graph is stored in the form of entity attributes and relationships, and is vectorized using a graph neural network.

5. The sentiment analysis and strategy optimization system based on multimodal acquisition and large language model according to claim 1, characterized in that, The specific implementation process of the cross-modal attention mechanism is as follows: First, the text comment features, user behavior features, and live stream content features are projected into the same latent space to obtain the corresponding feature sequences. Then, the cross-attention weights between the text feature sequences and the live stream content feature sequences are calculated to generate content-enhanced text representations. At the same time, the cross-attention weights between the user behavior feature sequences and the content-enhanced text representations are calculated, and finally, a multimodal unified representation that integrates user behavior context is output.

6. The sentiment analysis and strategy optimization system based on multimodal acquisition and a large language model according to claim 1, characterized in that, The components of the strategy utility function include a strategy expected return subfunction, a strategy execution cost subfunction, and a strategy risk penalty subfunction; The strategy expected return sub-function associates different key performance indicators with the strategy type and predicts the magnitude of their changes; The strategy execution cost subfunction quantifies the technical resources and time overhead required to implement the strategy. The strategy risk penalty subfunction assesses the probability and severity of negative user feedback that a strategy may trigger. The total utility score of a strategy is calculated by subtracting the execution cost score from the expected return score and then subtracting the risk penalty score.

7. The sentiment analysis and strategy optimization system based on multimodal acquisition and large language model according to claim 1, characterized in that, The strategy execution unit executes strategies in the following ways: adjusting the bullet screen display strategy, triggering specific interactive activities, or modifying the live stream content recommendation sequence.

8. The sentiment analysis and strategy optimization system based on multimodal acquisition and large language model according to claim 1, characterized in that, The system operates on a hierarchical control architecture, which includes a real-time decision-making layer and a periodic optimization layer. The real-time decision-making layer processes the bullet screen stream on a second-level timescale and executes high-priority, low-risk instant interaction strategies. The cycle optimization layer comprehensively analyzes data over a longer time window at a minute-level time scale to generate and execute comprehensive strategies involving content adjustments or large-scale event planning.

9. A sentiment analysis and strategy optimization system based on multimodal acquisition and a large language model according to claim 8, characterized in that, The periodic optimization layer is also responsible for updating the domain knowledge base and strategy model parameters.

Images