Atmosphere display method and system for text data information analysis based on video stream

Abstract

The invention discloses an atmosphere display method and system based on text data information analysis of video streaming, and the method comprises the following steps: emotional atmosphere configuration: configuring the display effects of emotions and corresponding atmosphere devices, and building a mapping relation between the emotions and the corresponding atmosphere expression effects through the configuration; acquiring and playing video stream data in real time; extracting and analyzing subtitles or bullet screen texts from the video stream data; preprocessing the extracted subtitles or bullet screen texts; performing emotional emotion analysis on the preprocessed subtitles or bullet screen texts, wherein an identified emotional emotion classification result comprises two numerical values, namely an emotional category and an emotional weight; emotion and atmosphere conversion is carried out, and an emotion analysis result is converted into expression data capable of being recognized by atmosphere display; and according to the emotion and atmosphere conversion result, obtaining emotion expression data corresponding to the emotion analysis result of the bullet screen content, and driving an atmosphere device to perform corresponding display.

Description

Technical Field

[0001] The present invention belongs to the technical field of video playback, and in particular relates to an atmosphere display method and system based on text data information analysis of a video stream. Background Art

[0002] With the rapid development of audio-visual technology, atmosphere lighting 、 Atmosphere devices such as fragrances have become a major help in enhancing the sense of technology and comfort in private spaces such as homes and cars. Usually, the atmosphere devices in a specific space will use one or more pre-set effect modes, which can be switched by the user according to the situation. The modes are fixed and the types are limited.

[0003] In addition, when users watch videos, there is still a lack of relevant ways to link with atmosphere devices such as atmosphere lights and fragrances. Some of the existing public methods also require the designers of atmosphere effects to analyze and process the video content to match the atmosphere effects in advance. However, there are a large number of Internet videos and they belong to different copyright holders, which cannot meet the needs of users who want to watch any video on different platforms and experience the atmosphere effects instantly. Summary of the invention

[0004] In view of the above problems, the present invention provides an atmosphere display method and system based on text data information analysis of video streams, which combines video playback with sentiment analysis to achieve real-time sentiment analysis, and displays the sentiment results through an atmosphere device, so that the audience can better fit the sentiment conveyed by the video content, thereby enhancing the video viewing experience.

[0005] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0006] On the one hand, the present invention provides an atmosphere display method based on text data information analysis of a video stream, comprising the following steps:

[0007] Emotional atmosphere configuration, including configuring the display effects of emotions and corresponding atmosphere devices, and establishing a mapping relationship between emotions and corresponding atmosphere expression effects through configuration;

[0008] Acquire and play video stream data in real time, wherein the video stream data includes audio data, video data, and associated subtitles or bullet screen data;

[0009] Extract and parse subtitles or bullet text from the video stream data, wherein the subtitles or bullet text includes a timestamp for specifying a display time, text content, and display effects;

[0010] Preprocess the extracted subtitles or bullet texts, including character encoding conversion, text cleaning, word segmentation, and stop word removal;

[0011] Perform sentiment analysis on the pre-processed subtitles or bullet comments. The identified sentiment classification results include two values: sentiment category and sentiment weight, which are associated with the timestamp of the subtitle or bullet comment analysis text. Among them, the sentiment category refers to the specific sentiment classification result, and the sentiment weight refers to the intensity of the current sentiment result.

[0012] Perform emotion and atmosphere conversion, converting the emotion analysis results into expression data that can be recognized by the atmosphere display;

[0013] According to the results of emotion and atmosphere conversion, the emotional expression data corresponding to the emotional analysis results of the barrage content is obtained, and the atmosphere device is driven to make corresponding displays.

[0014] In a possible implementation, when the atmosphere device is driven to make a corresponding display, the displayed time is the timestamp of the subtitle or barrage data associated with the emotion analysis result.

[0015] In a possible implementation, the audio data, video data, and associated subtitle or bullet screen data have a common and continuous timeline, and are played synchronously based on the timeline.

[0016] In a possible implementation manner, the emotional analysis of the pre-processed subtitles or bullet texts includes:

[0017] Training data preparation: Collect training data from annotated sentiment text datasets, where each text sample is labeled with a corresponding sentiment category label;

[0018] Lightweight model selection and training: Select a lightweight algorithm, train the model using training data, and learn the association pattern between text and sentiment;

[0019] Real-time sentiment analysis: For the pre-processed subtitles or bullet comments, first convert them into bag-of-words vectors, and use the trained model to calculate the sentiment score of the text;

[0020] Sentiment category determination: Determine the sentiment category of the text based on the sentiment score.

[0021] In a possible implementation manner, the atmosphere device is an atmosphere lamp.

[0022] In one possible implementation, emotion and atmosphere conversion is performed, and the results of emotion and mood analysis are converted into recognizable expression data for atmosphere display, including: by reading the configuration information of emotions and corresponding atmosphere expression effects, the emotion category corresponds to the light color of a specific atmosphere light in the configuration form, and the emotion weight represents the brightness or flickering frequency based on the light color, so as to calculate the actual effect that the atmosphere light should present.

[0023] In a possible implementation manner, the atmosphere device is a fragrance device.

[0024] In one possible implementation, emotion and atmosphere conversion is performed to convert the emotion and mood analysis results into recognizable expression data for atmosphere display, including: reading the configuration information of the emotion and the corresponding atmosphere expression effect, the emotion category corresponds to the fragrance type in the fragrance configuration information, and the emotion intensity corresponds to the concentration that should be implemented for the fragrance type.

[0025] Another aspect of the present invention provides an atmosphere display system based on text data information analysis of video streams, characterized in that it includes an atmosphere management device, a video content platform, a content playback device and an atmosphere device, wherein the content playback device further includes a content acquisition module, a content processing module, a content playback module and a sentiment analysis and atmosphere conversion module;

[0026] The atmosphere management device is used to configure the display effects of the emotions and corresponding atmosphere devices, and establish a mapping relationship between the emotions and the corresponding atmosphere expression effects through configuration;

[0027] The video content platform is a service provider that provides video stream data, and the video stream data includes audio data, video data, and associated subtitles or bullet screen data;

[0028] The content acquisition module is used to extract and parse subtitles or barrage text from video stream data, and the subtitles or barrage text includes a timestamp for specifying the display time, text content and display effect; the content playback module is used for audio data playback, video data rendering, text data display, audio data, video data, and time synchronization of associated subtitles or barrage data; the content processing module is used to preprocess the extracted subtitles or barrage text including character encoding conversion, text cleaning, word segmentation and stop word removal; the preprocessed subtitles or barrage text are subjected to emotional analysis, and the classification results of the identified emotional emotions include two numerical values, emotional category and emotional weight, and are associated with the timestamp of the subtitle or barrage analysis text; wherein the emotional category refers to the specific emotional classification result, and the emotional weight refers to the intensity of the current emotional result; the emotional analysis and atmosphere conversion module is used to perform emotional and atmosphere conversion, and convert the emotional analysis results into expression data recognizable by the atmosphere display;

[0029] The atmosphere device is used to obtain the emotional expression data corresponding to the emotional analysis result of the barrage content according to the emotion and atmosphere conversion result, and make corresponding display.

[0030] In a possible implementation, when the atmosphere device is driven to make a corresponding display, the displayed time is the timestamp of the subtitle or barrage data associated with the emotion analysis result.

[0031] In a possible implementation manner, the emotional analysis of the pre-processed subtitles or bullet texts includes:

[0032] Training data preparation: Collect training data from annotated sentiment text datasets, where each text sample is labeled with a corresponding sentiment category label;

[0033] Lightweight model selection and training: Select a lightweight algorithm, train the model using training data, and learn the association pattern between text and sentiment;

[0034] Real-time sentiment analysis: For the pre-processed subtitles or bullet comments, first convert them into bag-of-words vectors, and use the trained model to calculate the sentiment score of the text;

[0035] Sentiment category determination: Determine the sentiment category of the text based on the sentiment score.

[0036] In a possible implementation manner, the atmosphere device is an atmosphere lamp.

[0037] In one possible implementation, emotion and atmosphere conversion is performed, and the results of emotion and mood analysis are converted into recognizable expression data for atmosphere display, including: by reading the configuration information of emotions and corresponding atmosphere expression effects, the emotion category corresponds to the light color of a specific atmosphere light in the configuration form, and the emotion weight represents the brightness or flickering frequency based on the light color, so as to calculate the actual effect that the atmosphere light should present.

[0038] In a possible implementation manner, the atmosphere device is a fragrance device.

[0039] In one possible implementation, emotion and atmosphere conversion is performed to convert the emotion and mood analysis results into recognizable expression data for atmosphere display, including: reading the configuration information of the emotion and the corresponding atmosphere expression effect, the emotion category corresponds to the fragrance type in the fragrance configuration information, and the emotion intensity corresponds to the concentration that should be implemented for the fragrance type.

[0040] The present invention has the following beneficial effects:

[0041] (1) Through the close collaboration of text content extraction, preprocessing, and sentiment analysis, the system can quickly and accurately analyze the preprocessed subtitles or barrage text during real-time video playback, allowing viewers to more deeply feel the emotions conveyed by the video content, enhancing emotional resonance and immersive viewing experience.

[0042] (2) The selected lightweight algorithm and optimized processing steps have the advantages of real-time performance and low performance consumption, thereby ensuring the effective linkage between the emotions conveyed by the video content and the atmosphere devices such as ambient lights and fragrances. There is no need to pre-process the video for large-scale emotional analysis in advance, which also reduces the application threshold of electronic devices as playback devices.

[0043] (3) For platforms that provide video content, karaoke functions, and other applications, real-time sentiment analysis of texts such as subtitles / barrage / lyrics, and simultaneous display through atmosphere devices, not only enhances the attractiveness of the content, but also increases users' enthusiasm for participating in platform applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 A flowchart of the steps of an atmosphere display method based on text data information analysis of a video stream according to an embodiment of the present invention;

[0045] Figure 2 It is a principle block diagram of an atmosphere display system based on video stream text data information analysis according to an embodiment of the present invention;

[0046] Figure 3 It is a principle block diagram of a content playing device in an atmosphere display system based on video stream text data information analysis according to an embodiment of the present invention. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] See also Figure 1 , shown is a flowchart of the steps of the atmosphere display method based on the text data information analysis of the video stream according to an embodiment of the present invention, comprising the following steps:

[0049] S10, emotional atmosphere configuration, including configuring the display effects of emotions and corresponding atmosphere devices, and establishing a mapping relationship between emotions and corresponding atmosphere expression effects through configuration;

[0050] S20, real-time acquisition and playback of video stream data, the video stream data including audio data, video data, and associated subtitles or barrage data; wherein the audio data, video data, and associated subtitles or barrage data have a common and continuous timeline, and are played synchronously based on the timeline; the video stream data to be played can be acquired from an online video platform, a live broadcast platform, or other media sources.

[0051] S30, extracting and parsing subtitles or bullet screen text from the video stream data, where the subtitles or bullet screen text includes a timestamp for specifying a display time, text content, and display effects;

[0052] S40, preprocessing the extracted subtitle or bullet text including character encoding conversion, text cleaning (removing special characters, HTML tags, etc.), word segmentation (splitting the text into words or phrases), and stop word removal (removing meaningless common words);

[0053] S50, performing emotion analysis on the pre-processed subtitles or bullet comment texts, and the identified emotion classification results include two values, emotion category and emotion weight, which are associated with the timestamp of the subtitle or bullet comment analysis text; wherein the emotion category refers to the specific emotion classification result, and the emotion weight refers to the intensity of the current emotion result;

[0054] S60, performing emotion and atmosphere conversion, converting the emotion analysis result into expression data recognizable by the atmosphere display;

[0055] S70, based on the emotion and atmosphere conversion results, obtain the emotion expression data corresponding to the emotion analysis result of the barrage content, drive the atmosphere device to make corresponding display, so that the audience can accurately capture the emotions at different time points and resonate.

[0056] In one embodiment of the present invention, the atmosphere device is an atmosphere light. In S10, the display effect of the emotion and the corresponding atmosphere device is configured, and the mapping relationship between the emotion and the corresponding atmosphere expression effect is established through configuration, including: configuring the emotion and the corresponding atmosphere light color, flashing frequency and other display effects. On the basis of the default emotion-atmosphere configuration provided by the system, the user can modify the relevant configuration information according to his or her own preferences. Generally, an emotion can contain one or more colors that match the emotion. If there are multiple colors corresponding to the emotion, you can periodically change the selection among these colors. In S60, emotion and atmosphere conversion is performed, and the emotion and mood analysis results are converted into expression data that can be recognizable by the atmosphere display, including: by reading the configuration information of the emotion and the corresponding atmosphere expression effect, the emotion category corresponds to the specific atmosphere light color in the configuration form, and the emotion weight represents the brightness or flashing frequency based on the light color, so as to convert the effect that the atmosphere light should actually present.

[0057] In another embodiment of the present invention, the atmosphere device is an aroma device. In S10, the emotions and the display effects of the corresponding atmosphere devices are configured. Establishing the mapping relationship between emotions and the corresponding atmosphere expression effects through configuration includes: configuring the aroma type and the intensity of the aroma. On the basis of the default emotion-atmosphere configuration provided by the system, the user can modify the relevant configuration information according to their own preferences. For the aroma device, one emotion type corresponds to an aroma cartridge included in a given aroma device, and the user can configure the aroma cartridge type and the concentration information of the aroma. In S60, for the emotion and atmosphere conversion, converting the emotional analysis result into the expression data recognizable by the atmosphere display includes: by reading the configuration information of the emotions and the corresponding atmosphere expression effects, the emotion category corresponds to the aroma type in the aroma configuration information, and the emotional intensity corresponds to the concentration to be implemented for this aroma type.

[0058] In one embodiment of the present invention, in S40, word segmentation is the process of splitting a continuous text sequence into individual words or vocabulary items. In languages such as Chinese, there is no clear separator between words. Therefore, word segmentation helps to split the text into meaningful units, facilitating subsequent processing and analysis. For example, splitting a sentence into words can better understand the meaning and context of each word. Stop words refer to words that frequently appear in the text but usually have no practical meaning in the analysis, such as "de" (的), "shi" (是), "zai" (在), etc. These words usually do not have an important impact on the sentiment analysis of the text, but instead increase the complexity of calculation and processing. Therefore, removing stop words means removing these words without practical meaning from the text to reduce the noise and interference in the analysis. In sentiment analysis, word segmentation and removing stop words can help the system more accurately capture the emotional sentiment of the text. Word segmentation enables each word to be analyzed independently, and removing stop words can reduce the irrelevant factors during analysis. These two preprocessing steps help to improve the accuracy and efficiency of sentiment analysis.

[0059] In one embodiment of the present invention, in S50, the sentiment analysis of the preprocessed subtitle or bullet screen text includes:

[0060] Training data preparation: Collect training data from the labeled sentiment text dataset, where each text sample has a corresponding sentiment category label, such as joy, anger, sorrow, fear, etc.;

[0061] Lightweight model selection and training: Select lightweight algorithms, such as the Naive Bayes classifier, fasttext, etc., and use the training data to train the model to learn the association patterns between the text and the sentiment;

[0062] Real-time sentiment analysis: For the preprocessed subtitle or bullet screen text, first convert it into a bag-of-words vector, and use the trained model to calculate the sentiment score of the text;

[0063] Sentiment category determination: Determine the sentiment category of the text based on the sentiment score.

[0064] Taking the four emotions of joy, anger, sadness and fear as an example, a set of labeled emotional text data is used in the training stage:

[0065] Barrage text: "This movie is so funny!", corresponding sentiment classification label: "joy".

[0066] Barrage text: "The plot of this movie is so rubbish...", corresponding sentiment classification label: "angry".

[0067] The barrage text: "This scene is so touching that I can't help but cry", the corresponding emotion classification label: "sadness".

[0068] The barrage text: "This horror movie is a bit scary, I'm scared...", the corresponding sentiment classification label is: "fear".

[0069] Subsequent processing is performed based on these labeled emotional text data, such as training a naive Bayes classifier. In the real-time analysis stage, when new bullet comment data "I'm so happy, I'm laughing to death" enters the system, the naive Bayes classifier will classify it as "joy" emotion. In the real-time analysis process, the bullet comment text is classified according to the learned emotional model to further optimize the emotional analysis results.

[0070] In one embodiment of the present invention, in S60, the color display mechanism of the ambient light is usually divided into two types: fixed and dimmable. Fixed colors can include 3 colors, 8 colors, 16 colors, and more than 128 colors. Assuming that red, blue, green, yellow, orange, purple, white, and pink are 8 basic colors, more colors can be derived according to the degree of brightness. For example, from 8 colors to 16 colors, light red and dark red are derived from red; 32 colors can be derived from light red, bright red, dark red, and dark red, and so on.

[0071] When the number of expressible colors is small under a fixed color mechanism, the color can be expressed by adjusting the flickering frequency. If the emotional weight of the emotional analysis result is higher, the flickering frequency is faster.

[0072] If the light is dimmable, rules can be formulated based on the light colors paired with emotions to convert the values ​​of different emotional weights into corresponding brightness and saturation, thereby obtaining the color that should be displayed as a result of this emotional analysis.

[0073] The above are several simple examples of emotion-atmosphere conversion rules. In actual applications, due to the different parameters and capabilities of the atmosphere light display devices, there may be other simpler or more complex display schemes.

[0074] During the actual processing, the time-atmosphere expression data obtained by S70 can be synchronously cached to the memory or written to a file. When the user jumps to the content that has been watched, there is no need to repeat the analysis, thereby improving the processing efficiency.

[0075] Through the above-mentioned atmosphere display method based on video stream text data information analysis, through the close collaboration of text content extraction, preprocessing and sentiment analysis, the system can quickly and accurately analyze the preprocessed subtitles or bullet texts during real-time video playback, so that the audience can feel the emotions conveyed by the video content more deeply, enhance emotional resonance and immersive viewing experience. The selected lightweight algorithm and optimized processing steps have the advantages of real-time and low performance consumption, so as to ensure the effective linkage between the emotions conveyed by the video playback content and the atmosphere light. There is no need to preprocess the playback video with a large amount of calculation for sentiment analysis in advance, which also reduces the application threshold of electronic devices as playback devices. For platforms that provide video content, karaoke functions and other applications, real-time sentiment analysis of texts such as subtitles / bullet / lyrics, and then synchronous display through the atmosphere light, enhances the attractiveness of the content while also increasing the enthusiasm of users to participate in platform applications.

[0076] For method embodiments, see Figure 2 Another embodiment of the present invention provides an atmosphere display system based on text data information analysis of video streams, including an atmosphere management device, a video content platform, a content playback device and an atmosphere device, see Figure 3The content playback device further includes a content acquisition module, a content processing module, a content playback module and an emotion analysis and atmosphere conversion module; wherein the atmosphere management device is used to configure the display effect of the emotion and the corresponding atmosphere device, and establishes a mapping relationship between the emotion and the corresponding atmosphere expression effect through configuration; the video content platform is a service provider that provides video stream data, and the video stream data includes audio data, video data, and associated subtitles or barrage data; the content acquisition module is used to extract and parse subtitles or barrage texts from the video stream data, and the subtitles or barrage texts include a timestamp for specifying the display time, text content and display effects; the content playback module is used for audio data playback, video data rendering, text data display, audio data, video data, and time synchronization of associated subtitles or barrage data; the content processing module is used to pre-process the extracted subtitles or barrage texts, including character encoding conversion, text cleaning, word segmentation and stop word removal; the pre-processed subtitles or barrage texts are subjected to emotion analysis, and the identified emotion classification results include two numerical values, emotion category and emotion weight, and are associated with the timestamp of the subtitle or barrage analysis text; wherein the emotion category refers to the specific emotion classification result. Emotional weight refers to the intensity of the current emotional result; the emotional analysis and atmosphere conversion module is used to convert emotions and atmosphere, converting the emotional analysis results into expression data that can be recognizable by the atmosphere display. The emotional category corresponds to the light color of the specific atmosphere light in the configuration form, and the emotional weight represents the brightness or flashing frequency based on the light color, so as to calculate the actual effect of the atmosphere light; the atmosphere device is used to obtain the emotional expression data corresponding to the emotional analysis results of the barrage content based on the emotion and atmosphere conversion results, and make corresponding displays.

[0077] In the embodiment of the present invention, the video content platform is a service provider that provides video content, including online video platforms, live broadcast platforms, karaoke application platforms, etc. The content playback device is an electronic device that has the ability to acquire, parse, decode, present / render network audio and video and the ability to process and render associated bullet screen or subtitle text data, including but not limited to mobile phones, tablets, computers, TV set-top boxes, car-mounted devices, etc. The content playback device can realize the synchronous playback of audio and video and associated text data, such as bullet screen or subtitles; perform real-time emotional analysis on text data, convert the analysis results into atmosphere light control signals, and drive the atmosphere light to perform corresponding presentation. The content acquisition module obtains video stream data from the video content platform by accessing streaming media services, API calls, etc. In order to meet the requirements of real-time and low performance consumption, a lightweight emotional analysis method is used to perform specific emotional analysis on the text data associated with the video to determine the emotional emotions. In the implementation of the present invention, the atmosphere presentation device is an atmosphere light display device, including an atmosphere light controller and a light-emitting device. The atmosphere light display device receives a control signal to achieve the display effect of the atmosphere expression data corresponding to the aforementioned emotional analysis. The configuration of the atmosphere management device is provided by the system by default, and the user can also manually choose to modify it.

[0078] In one embodiment of the present invention, the atmosphere device is an atmosphere light. In the atmosphere management device, the display effect of the emotion and the corresponding atmosphere device is configured, and the mapping relationship between the emotion and the corresponding atmosphere expression effect is established through configuration, including: configuring the emotion and the corresponding atmosphere light color, flashing frequency and other display effects. On the basis of the default emotion-atmosphere configuration provided by the system, the user can modify the relevant configuration information according to his own preferences. Generally, an emotion can contain one or more colors that match the emotion. If there are multiple colors corresponding to the emotion, you can periodically change the selection among these colors. In the content processing module, emotion and atmosphere conversion is performed, and the emotion and mood analysis results are converted into expression data that can be recognizable by the atmosphere display, including: by reading the configuration information of the emotion and the corresponding atmosphere expression effect, the emotion category corresponds to the specific atmosphere light color in the configuration form, and the emotion weight represents the brightness or flashing frequency based on the light color, so as to calculate the effect that the atmosphere light should actually present.

[0079] In another embodiment of the present invention, the atmosphere device is an aroma device. In the atmosphere management device, the display effects of emotions and corresponding atmosphere devices are configured. The mapping relationship between emotions and corresponding atmosphere expression effects established through configuration includes: configuring the aroma type and the intensity of the aroma. On the basis of the default emotion-atmosphere configuration provided by the system, users can modify the relevant configuration information according to their own preferences. For the aroma device, one emotion type corresponds to an aroma cartridge included in a given aroma device. Users can configure the aroma cartridge type and the concentration information of the aroma. In the content processing module, emotion and atmosphere conversion are performed, and the emotional analysis result is converted into expression data recognizable by the atmosphere display, including: by reading the configuration information of emotions and corresponding atmosphere expression effects, the emotion category corresponds to the aroma type in the aroma configuration information, and the emotional intensity corresponds to the concentration to be implemented for this aroma type.

[0080] In one embodiment of the present invention, in the content processing module, word segmentation is the process of splitting a continuous text sequence into individual words or vocabulary items. In languages such as Chinese, there are no clear separators between words. Therefore, word segmentation helps to split the text into meaningful units, facilitating subsequent processing and analysis. For example, splitting a sentence into words can better understand the meaning and context of each word. Stop words refer to words that frequently appear in the text but usually have no practical meaning in the analysis, such as "de", "shi", "zai", etc. These words generally do not have an important impact on the sentiment analysis of the text, but instead increase the complexity of calculation and processing. Therefore, removing stop words means removing these words without practical meaning from the text to reduce the noise and interference in the analysis. In sentiment analysis, word segmentation and removing stop words can help the system more accurately capture the emotional sentiment of the text. Word segmentation enables each word to be analyzed independently, while removing stop words can reduce irrelevant factors during analysis. These two preprocessing steps help to improve the accuracy and efficiency of sentiment analysis.

[0081] In one embodiment of the present invention, in the content processing module, the sentiment analysis of the preprocessed subtitle or bullet screen text includes:

[0082] Training data preparation: Collect training data from a labeled emotional text dataset, where each text sample is accompanied by a corresponding emotional category label, such as joy, anger, sorrow, fear, etc.;

[0083] Lightweight model selection and training: Select lightweight algorithms, such as the Naive Bayes classifier, fasttext, etc., and use the training data to train the model to learn the association patterns between the text and emotions;

[0084] Real-time sentiment analysis: For the preprocessed subtitle or bullet screen text, first convert it into a bag-of-words vector, and use the trained model to calculate the sentiment score of the text;

[0085] Sentiment category determination: Determine the sentiment category of the text based on the sentiment score.

[0086] Taking the four emotions of joy, anger, sadness and fear as an example, a set of labeled emotional text data is used in the training stage:

[0087] Barrage text: "This movie is so funny!", corresponding sentiment classification label: "joy".

[0088] Barrage text: "The plot of this movie is so rubbish...", corresponding sentiment classification label: "angry".

[0089] The barrage text: "This scene is so touching that I can't help but cry", the corresponding emotion classification label: "sadness".

[0090] The barrage text: "This horror movie is a bit scary, I'm scared...", the corresponding sentiment classification label is: "fear".

[0091] Subsequent processing is performed based on these labeled emotional text data, such as training a naive Bayes classifier. In the real-time analysis stage, when new bullet comment data "I'm so happy, I'm laughing to death" enters the system, the naive Bayes classifier will classify it as "joy" emotion. In the real-time analysis process, the bullet comment text is classified according to the learned emotional model to further optimize the emotional analysis results.

[0092] In one embodiment of the present invention, when the atmosphere device is an atmosphere lamp, the color display mechanism of the atmosphere lamp is usually divided into two types: fixed and dimmable. Fixed colors may include 3 colors, 8 colors, 16 colors, or even 128 colors. Assuming that red, blue, green, yellow, orange, purple, white, and pink are 8 basic colors, more colors can be derived according to the degree of brightness. For example, from 8 colors to 16 colors, light red and dark red are derived from red; 32 colors can be derived from light red, bright red, dark red, and dark red, and so on.

[0093] When the number of expressible colors is small under a fixed color mechanism, the color can be expressed by adjusting the flickering frequency. If the emotional weight of the emotional analysis result is higher, the flickering frequency is faster.

[0094] If the light is dimmable, rules can be formulated based on the light colors paired with emotions to convert the values ​​of different emotional weights into corresponding brightness and saturation, thereby obtaining the color that should be displayed as a result of this emotional analysis.

[0095] In one embodiment of the present invention, when the atmosphere device is a fragrance device, according to the default configuration of the system, the emotion category corresponds to the fragrance type in the fragrance configuration information, that is, which fragrance box it belongs to, and the emotion intensity corresponds to the concentration that should be implemented for the fragrance type. The concentration can be adjusted by the air volume, for example, setting three different wind speeds of low, high, and medium to drive the blower to operate at the corresponding gear.

[0096] The above are several simple examples of emotion-atmosphere conversion rules. In actual applications, due to the different parameters and capabilities of the atmosphere light display devices, there may be other simpler or more complex display schemes.

[0097] In the actual processing process, the time-atmosphere expression data obtained in the atmosphere device can be synchronously cached to the memory or written to a file. When the user jumps to the content that has been watched, there is no need to repeat the analysis, thereby improving the processing efficiency.

[0098] In a specific application embodiment, the method of the present invention is used to implement an atmosphere display system based on text data information analysis of a video stream, which is an in-vehicle atmosphere light and fragrance presentation system based on real-time video barrage analysis. The car is equipped with an atmosphere management device in the central control system, which allows users to configure the emotion-atmosphere mapping relationship by themselves.

[0099] Assume that the car's ambient light system can display 16 fixed colors, of which red, blue, green, and gray are the four basic colors. According to the degree of brightness, they can be derived into light red, bright red, dark red, dark red, light green, bright green, dark green, dark green, and so on. Assume that the configuration relationship selected by the user is: red represents anger; green represents joy; blue represents sadness; and gray represents fear.

[0100] Assume that the car's fragrance system is equipped with four fragrance boxes, including floral, wood, spice, and herbal. The default configuration of the system is: spice-anger; floral-joy; wood-sadness; herbal-fear. The emotional intensity corresponds to the concentration of the fragrance, which is reflected by the blower speed level. The emotional intensity covers the range of 0-100 points, and it is divided into three equal parts corresponding to low, medium, and high levels. The user has not manually modified it, and the default configuration is used.

[0101] When a user selects a movie to watch, the system obtains the corresponding video stream for playback. During playback, it obtains the barrage stream data and uses the Bayesian algorithm to perform real-time sentiment analysis.

[0102] When the protagonist in the movie encounters a major setback, the real-time video barrage emotional analysis system extracts the audience's barrage text. After emotional analysis, the system identifies the audience's emotional tendency as "sad" and "sad", with an emotional intensity of 90 points. Through the atmosphere light configuration information, the system converts the analysis result to dark blue. Through the fragrance configuration information, the fragrance type corresponding to the analysis result is wood, and the system selects a wooden fragrance box, and the emotional intensity of 90 points corresponds to a high wind speed gear.

[0103] When the video plays to the timestamp associated with the barrage text analysis, the system drives the atmosphere device, and the atmosphere light is displayed in dark blue. Based on the analysis results of the fragrance, the system drives the fragrance device to release the fragrance based on the configuration corresponding to the analysis.

[0104] In a specific application embodiment, the method of the present invention is used to implement an atmosphere display system based on text data information analysis of a video stream, which is a karaoke atmosphere light display system based on real-time lyrics analysis. It is assumed that an atmosphere light device is configured in the user's home environment, and the user configures his or her preferred emotion-atmosphere mapping relationship through the atmosphere management device.

[0105] The set-top box in the user's home has a karaoke platform equipped with a karaoke atmosphere light display system with real-time lyrics analysis.

[0106] When the user selects a song to be sung on the karaoke platform, the real-time emotion analysis system obtains the corresponding song video and lyrics of the song from the karaoke platform, obtains the time-emotion analysis results corresponding to the lyrics through the emotion analysis algorithm, and converts the atmosphere light configuration information into corresponding color / frequency / animation presentation information.

[0107] While the user is singing and the lyrics are displayed, the system drives the atmosphere lighting device to synchronously display the lighting effects corresponding to the emotional analysis results.

[0108] Through the above-mentioned atmosphere display system based on video stream text data information analysis, through the close collaboration of text content extraction, preprocessing and sentiment analysis, the system can quickly and accurately analyze the preprocessed subtitles or bullet texts during real-time video playback, so that the audience can feel the emotions conveyed by the video content more deeply, enhance emotional resonance and immersive viewing experience. The selected lightweight algorithm and optimized processing steps have the advantages of real-time and low performance consumption, so as to ensure the effective linkage between the emotions conveyed by the video playback content and the atmosphere light. There is no need to preprocess the playback video with a large amount of calculations in advance, which also reduces the application threshold of electronic devices as playback devices. For platforms that provide video content, karaoke functions and other applications, real-time sentiment analysis of texts such as subtitles / bullet / lyrics, and then synchronously display them through atmosphere lights, not only enhances the attractiveness of the content, but also increases the enthusiasm of users to participate in platform applications.

[0109] It should be understood that the exemplary embodiments described herein are illustrative rather than restrictive. Although one or more embodiments of the present invention have been described in conjunction with the accompanying drawings, it should be understood by those skilled in the art that various changes in form and detail may be made without departing from the spirit and scope of the present invention as defined by the appended claims.

Claims

1. A method for displaying atmosphere based on text data information analysis of video stream, characterized in that: The following steps are involved: Emotional atmosphere configuration, including configuring the display effects of emotions and corresponding atmosphere devices, and establishing a mapping relationship between emotions and corresponding atmosphere expression effects through configuration; Acquire and play video stream data in real time, wherein the video stream data includes audio data, video data, and associated subtitles or bullet screen data; Extract and parse subtitles or bullet text from the video stream data, wherein the subtitles or bullet text includes a timestamp for specifying a display time, text content, and display effects; Preprocess the extracted subtitles or bullet texts, including character encoding conversion, text cleaning, word segmentation, and stop word removal; Perform sentiment analysis on the pre-processed subtitles or bullet comments. The identified sentiment classification results include two values: sentiment category and sentiment weight, which are associated with the timestamp of the subtitle or bullet comment analysis text. Among them, the sentiment category refers to the specific sentiment classification result, and the sentiment weight refers to the intensity of the current sentiment result. Perform emotion and atmosphere conversion, converting the emotion analysis results into expression data that can be recognized by the atmosphere display; According to the results of emotion and atmosphere conversion, the emotional expression data corresponding to the emotional analysis results of the barrage content is obtained, and the atmosphere device is driven to make corresponding displays.

2. The atmosphere display method based on text data information analysis of video stream as claimed in claim 1, characterized in that: When the atmosphere device is driven to make a corresponding display, the displayed time is the timestamp of the subtitle or barrage data associated with the emotion analysis result.

3. The atmosphere display method based on text data information analysis of video stream as claimed in claim 1, characterized in that: The audio data, video data, and associated subtitle or bullet screen data have a common and continuous timeline, and are played synchronously based on the timeline.

4. The atmosphere display method based on text data information analysis of video stream as claimed in claim 1, characterized in that: The emotional analysis of the pre-processed subtitles or bullet texts includes: Training data preparation: Collect training data from annotated sentiment text datasets, where each text sample is labeled with a corresponding sentiment category label; Lightweight model selection and training: Select a lightweight algorithm, train the model using training data, and learn the association pattern between text and sentiment; Real-time sentiment analysis: For the pre-processed subtitles or bullet comments, first convert them into bag-of-words vectors, and use the trained model to calculate the sentiment score of the text; Sentiment category determination: Determine the sentiment category of the text based on the sentiment score.

5. The atmosphere display method based on text data information analysis of video stream according to any one of claims 1 to 4, characterized in that: The atmosphere device is an atmosphere lamp.

6. The atmosphere display method based on text data information analysis of video stream as claimed in claim 5, characterized in that: Emotion and atmosphere conversion is performed, and the results of emotion analysis are converted into recognizable expression data for atmosphere display, including: by reading the configuration information of emotions and corresponding atmosphere expression effects, the emotion category corresponds to the light color of the specific atmosphere light in the configuration form, and the emotion weight represents the brightness or flickering frequency based on the light color, so as to calculate the actual effect that the atmosphere light should present.

7. The atmosphere display method based on text data information analysis of video stream according to any one of claims 1 to 4, characterized in that: The atmosphere device is a fragrance device.

8. The atmosphere display method based on text data information analysis of video stream as claimed in claim 7, characterized in that: Emotion and atmosphere conversion is performed to convert the results of emotion analysis into recognizable expression data for atmosphere display, including: by reading the configuration information of emotions and corresponding atmosphere expression effects, the emotion category corresponds to the fragrance type in the fragrance configuration information, and the emotion intensity corresponds to the concentration that should be implemented for the fragrance type.

9. An atmosphere display system based on text data information analysis of video streams, characterized in that: It includes an atmosphere management device, a video content platform, a content playback device and an atmosphere device, wherein the content playback device further includes a content acquisition module, a content processing module, a content playback module and a sentiment analysis and atmosphere conversion module; The atmosphere management device is used to configure the display effects of the emotions and corresponding atmosphere devices, and establish a mapping relationship between the emotions and the corresponding atmosphere expression effects through configuration; The video content platform is a service provider that provides video stream data, and the video stream data includes audio data, video data, and associated subtitles or bullet screen data; The content acquisition module is used to extract and parse subtitles or barrage text from video stream data, and the subtitles or barrage text includes a timestamp for specifying the display time, text content and display effect; the content playback module is used for audio data playback, video data rendering, text data display, audio data, video data, and time synchronization of associated subtitles or barrage data; the content processing module is used to preprocess the extracted subtitles or barrage text including character encoding conversion, text cleaning, word segmentation and stop word removal; the preprocessed subtitles or barrage text are subjected to emotional analysis, and the classification results of the identified emotional emotions include two numerical values, emotional category and emotional weight, and are associated with the timestamp of the subtitle or barrage analysis text; wherein the emotional category refers to the specific emotional classification result, and the emotional weight refers to the intensity of the current emotional result; the emotional analysis and atmosphere conversion module is used to perform emotional and atmosphere conversion, and convert the emotional analysis results into expression data recognizable by the atmosphere display; The atmosphere device is used to obtain the emotional expression data corresponding to the emotional analysis result of the barrage content according to the emotion and atmosphere conversion result, and make corresponding display.

10. The atmosphere display system based on video stream text data information analysis as claimed in claim 9, characterized in that: When the atmosphere device is driven to make a corresponding display, the displayed time is the timestamp of the subtitle or barrage data associated with the emotion analysis result.

11. The atmosphere display system based on video stream text data information analysis as claimed in claim 9, characterized in that: The emotional analysis of the pre-processed subtitles or bullet texts includes: Training data preparation: Collect training data from annotated sentiment text datasets, where each text sample is labeled with a corresponding sentiment category label; Lightweight model selection and training: Select a lightweight algorithm, train the model using training data, and learn the association pattern between text and sentiment; Real-time sentiment analysis: For the pre-processed subtitles or bullet comments, first convert them into bag-of-words vectors, and use the trained model to calculate the sentiment score of the text; Sentiment category determination: Determine the sentiment category of the text based on the sentiment score.

12. The atmosphere display system based on video stream text data information analysis according to any one of claims 9 to 11, characterized in that: The atmosphere device is an atmosphere lamp.

13. The atmosphere display system based on video stream text data information analysis as claimed in claim 12, characterized in that: Emotion and atmosphere conversion is performed, and the results of emotion analysis are converted into recognizable expression data for atmosphere display, including: by reading the configuration information of emotions and corresponding atmosphere expression effects, the emotion category corresponds to the light color of the specific atmosphere light in the configuration form, and the emotion weight represents the brightness or flickering frequency based on the light color, so as to calculate the actual effect that the atmosphere light should present.

14. The atmosphere display system based on video stream text data information analysis according to any one of claims 9 to 11, characterized in that: The atmosphere device is a fragrance device.

15. The atmosphere display system based on video stream text data information analysis as claimed in claim 14, characterized in that: Emotion and atmosphere conversion is performed to convert the results of emotion analysis into recognizable expression data for atmosphere display, including: by reading the configuration information of emotions and corresponding atmosphere expression effects, the emotion category corresponds to the fragrance type in the fragrance configuration information, and the emotion intensity corresponds to the concentration that should be implemented for the fragrance type.

Images