Method, apparatus, device and storage medium for video playing

Abstract

The application relates to the technical field of intelligent devices, and discloses a method, device and equipment for video playing and a storage medium. The method comprises the following steps: acquiring current multi-modal physiological information of a user watching a current video, and determining a current emotional intensity value corresponding to the current multi-modal physiological information; wherein the multi-modal physiological information comprises one or more of galvanic skin response (GSR), heart rate (HR), skin temperature and respiratory rate; in the case that the current emotional intensity value satisfies a current trigger condition corresponding to a current narrative branch of the current video, determining a current playing parameter corresponding to the current narrative branch of the current video according to the current emotional intensity value, wherein the current trigger condition is obtained by analyzing a narrative branch mark embedded in the current video; and playing the current video according to the current playing parameter. In this way, the emotional intensity value is used to drive the video playing adjustment in real time, and the individualization and intelligentization degree of the video playing is greatly improved.

Description

Technical Field

[0001] This application relates to the field of smart device technology, such as methods, apparatus, devices, and storage media for video playback. Background Technology

[0002] With the rapid development of smart terminals and multimedia technologies, interactive video and immersive audio-visual playback have become mainstream forms of audiovisual entertainment. Users are increasingly demanding personalized, contextualized, and atmospheric video playback experiences. Currently, most mainstream video playback solutions use fixed playback parameters and fixed video content branches, failing to adapt to the user's real-time status, resulting in increasingly prominent limitations in the user experience. Among related technologies, solutions for adjusting video interaction based on user emotions include: pre-setting multiple plot branches in the video and relying on user manual operation to select the branch direction; the playback parameters such as audio, subtitles, and picture are also factory-fixed configurations, and the playback effect cannot be dynamically adjusted according to the user's emotional changes during viewing; the interaction is mechanical and it is difficult to create an immersive viewing atmosphere. Alternatively, it may include: relying on single visual or voice modalities such as facial expressions and voice to recognize emotions; the recognition results are easily affected by lighting, occlusion, and environmental noise, resulting in low recognition accuracy; even if a few solutions introduce physiological signal collection, they lack personalization, cannot achieve smooth and gradual parameter adjustment, and are prone to abrupt changes in picture and sound effects during the viewing process, resulting in a disjointed viewing experience. In summary, existing video playback technologies suffer from problems such as insufficient accuracy in emotion recognition, rigid adjustment of playback parameters, and poor versatility in video branch design, making it difficult to meet the current demand for high-quality, adaptive, and all-scenario intelligent viewing.

[0003] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0004] To provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. This summary is not intended as a general commentary, nor is it intended to identify key / important components or describe the scope of protection of these embodiments, but rather as a prelude to the detailed description that follows.

[0005] This disclosure provides a method, apparatus, device, and storage medium for video playback to solve the technical problem that traditional video playback modes are fixed and cannot adapt to the dynamic emotional changes of users.

[0006] In some embodiments, the method includes: The system acquires the current multimodal physiological information of the user watching the current video and determines the current emotional intensity value corresponding to the current multimodal physiological information; wherein, the multimodal physiological information includes one or more of the following: skin conductance response (GSR), heart rate (HR), skin temperature, and respiratory rate; If the current emotion intensity value satisfies the current triggering condition corresponding to the current video narrative branch, the current playback parameters corresponding to the current video narrative branch are determined based on the current emotion intensity value. The current triggering condition is obtained by parsing the narrative branch markers embedded in the current video. Play the current video according to the current playback parameters.

[0007] In some embodiments, determining the current emotion intensity value corresponding to the current multimodal physiological information includes: The current multimodal physiological information is preprocessed, and the normalized value corresponding to each physiological information in the current multimodal physiological information is obtained based on the baseline calibration range. The baseline calibration range is the range of user physiological signals recorded when guiding users to complete four standard emotion induction tasks of calm, excitement, tension and relaxation. Based on each normalized value, a weighted fusion is performed according to formula (1) to obtain the current emotion intensity value E; E = 100×(w1×GSR_norm + w2×HR_norm + w3×Temp_norm + w4×RR_norm)(1)where: GSR_norm, HR_norm, Temp_norm, and RR_norm are the normalized values ​​corresponding to skin conductance response (GSR), heart rate (HR), skin temperature, and respiratory rate, respectively, and w1, w2, w3, and w4 are dynamic weighting coefficients, and the sum of the weighting coefficients is 1.

[0008] In some embodiments, it also includes: Based on the Emotion Branching Protocol (EBMP), one or more narrative branch markers corresponding to video narrative branches are embedded in the encapsulated metadata layer of the current video. The narrative branch markers include one or more of the following: triggering conditions, switching actions, and switching transition durations.

[0009] In some embodiments, determining the current playback parameters corresponding to the current video narrative branch includes: If the current emotional intensity value E is greater than the first threshold, increase the gain value of the first range of audio; if the current emotional intensity value E is less than the second threshold, increase the gain value of the second range of audio. Here, the first threshold is greater than the second threshold, and the frequency value corresponding to the first range is greater than the frequency value corresponding to the second range. If the current emotional intensity value E is greater than the third threshold, the subtitles will be switched to a bright color scheme and the font size will be enlarged. If the current emotional intensity value E is less than the fourth threshold, the subtitles will be switched to a soft color scheme and the default font size will be restored. The third threshold is greater than the fourth threshold. If the current emotional intensity value E is greater than the fifth threshold, increase the volume of the background music; if the current emotional intensity value E is less than the sixth threshold, decrease the volume of the background music. The fifth threshold is greater than the sixth threshold.

[0010] In some embodiments, playing the current video according to the current playback parameters includes: Within the transition duration corresponding to the current narrative branch marker, switch to a related narrative branch of the current narrative branch, keeping the main storyline of the current video unchanged; Within the transition duration corresponding to the current narrative branch marker, the audio, subtitle, and video parameters are continuously and gradually adjusted according to the current playback parameters.

[0011] In some embodiments, obtaining the current multimodal physiological information of the user watching the current video includes: In cases where multiple users are watching the current video, the user with the longest screen gaze duration is identified as the current user, and their current multimodal physiological information is obtained; or, In the case of multiple users watching the current video, the multimodal physiological information of each user is obtained, so that the current emotional intensity value is obtained by averaging and fusing the emotional intensity values ​​corresponding to each multimodal physiological information.

[0012] In some embodiments, it also includes: Based on the current emotional intensity value E, control is exercised over home environment equipment, which includes one or more of the following: lighting fixtures, curtains, air conditioners, and fresh air systems.

[0013] In some embodiments, the apparatus for video playback includes a processor and a memory storing program instructions, the processor being configured to execute the above-described method for video playback when the program instructions are executed.

[0014] In some embodiments, the device includes: a device body; and means for video playback as described above, which are mounted on the device body.

[0015] In some embodiments, the storage medium stores program instructions that, when executed, perform the above-described method for video playback.

[0016] The method, apparatus, device, and storage medium for video playback provided in this disclosure can achieve the following technical effects: Based on the acquired multimodal physiological signals of video viewers, corresponding emotion intensity values ​​are generated through personalized baseline calibration and weighted fusion. Playback parameters can be determined on demand according to real-time emotion values ​​to play corresponding videos. This achieves a non-perceptible, passive, and real-time dynamic video adaptive control mechanism, eliminating dependence on user-initiated operation and removing the disconnect in the viewing experience caused by manual interaction. By driving video playback adjustment in real time through emotion intensity values, the video playback effect can accurately match the user's real-time emotional state, greatly improving the personalization and intelligence of video playback.

[0017] The above general description and the description below are exemplary and illustrative only and are not intended to limit this application. Attached Figure Description

[0018] One or more embodiments are illustrated by way of example with reference to the accompanying drawings. These illustrations and drawings do not constitute a limitation on the embodiments. Elements having the same reference numerals in the drawings are shown as similar elements. The drawings are not to be scaled. And wherein: Figure 1 This is a schematic diagram of an architecture for a video playback system provided in an embodiment of this disclosure; Figure 2 This is a flowchart illustrating a video playback method provided in an embodiment of this disclosure; Figure 3 This disclosure provides a flowchart illustrating a video playback method. Figure 4 This is a schematic diagram of a video playback device provided in an embodiment of this disclosure; Figure 5 This is a schematic diagram of a video playback device provided in an embodiment of this disclosure; Figure 6 This is a schematic diagram of a video playback device provided in an embodiment of this disclosure; Figure 7 This is a schematic diagram of a device provided in an embodiment of this disclosure. Detailed Implementation

[0019] To provide a more detailed understanding of the features and technical content of the embodiments of this disclosure, the implementation of the embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. The accompanying drawings are for illustrative purposes only and are not intended to limit the embodiments of this disclosure. In the following technical description, for ease of explanation, several details are used to provide a full understanding of the disclosed embodiments. However, one or more embodiments may still be implemented without these details. In other cases, well-known structures and devices may be simplified in their depiction to simplify the drawings.

[0020] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this disclosure described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion.

[0021] Unless otherwise stated, the term "multiple" means two or more.

[0022] In this embodiment of the disclosure, the character " / " indicates that the objects before and after it are in an "or" relationship. For example, A / B means: A or B.

[0023] The term "and / or" describes an association between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or A and B.

[0024] In this embodiment, multimodal physiological signal acquisition is introduced and combined with user emotional baseline calibration to calculate continuous emotional intensity values, achieving imperceptible emotion perception. Based on the Emotion Branch Marker Protocol (EBMP) narrative branch marking mechanism, trigger conditions and switching strategies are stored only in video metadata, without presetting any audio or video adjustment parameters in the markings. All parameters are dynamically generated by the playback device based on real-time emotional intensity values. The narrative branches are limited to style branches with the same source and main plot. The switching process does not change the story direction, but only gradually adapts and adjusts the picture, audio, and subtitles. A two-emotion fusion judgment mechanism is also added for multi-user viewing, and the same emotional intensity output is reused to synchronously drive video playback adjustment and linkage control of smart home devices throughout the house. This not only accurately captures the dynamic changes in users' emotions during the viewing process, but also flexibly adapts the atmosphere of the video content without human intervention, reducing the cost of video content production and distribution, ensuring the integrity of the film's narrative and the smoothness of playback. It is also compatible with various viewing scenarios for single and multiple people, opens up the data linkage link between the content end and the home control end, and builds a fully closed-loop immersive viewing system of emotion perception, content adaptation, and synchronous environmental control. This effectively overcomes various pain points of existing technologies and has stronger practicality and scenario expansion capabilities.

[0025] Figure 1 This is a schematic diagram of an architecture for a video playback system provided in an embodiment of this disclosure. Figure 1 As shown, the video playback system includes a video playback device 100 and a physiological signal acquisition device 200. The video playback device 100 and the physiological signal acquisition device 200 establish a wireless communication connection to realize real-time interaction of physiological data.

[0026] The physiological signal acquisition device 200 is a smart wearable device that supports multimodal physiological data acquisition. Specifically, it can be a professional smart bracelet or smartwatch that supports Galvanic Skin Response (GSR) acquisition, or a consumer-grade smartwatch that integrates Photoplethysmography (PPG) pulse wave acquisition. In some embodiments, the sampling rate of the physiological signal acquisition device 200 is not less than 5.0 Hz. This allows for real-time and continuous acquisition of multimodal physiological information such as GSR, Heart Rate (HR), Skin Temperature, and Respiratory Rate during the user's movie-watching process. The preprocessed physiological data is then transmitted to the video playback device 100 in real time via a built-in Bluetooth 5.0 or Wi-Fi 6 communication module, providing raw data support for subsequent emotion intensity calculation and video adaptive control. The video playback device 100 can be a smart terminal with video decoding and playback functions, such as a smart TV, network set-top box, projector, or high-definition display. The device has multiple built-in functional modules, such as a main processing SoC, an NPU computing module, a DSP preprocessing module, a storage module, and an IoT communication module. In some embodiments, the main processing SoC is equipped with an NPU with a computing power of ≥2 TOPS, which can efficiently run the biofeedback narrative mapping engine and the AI ​​real-time content modulation engine to meet the computing power requirements for real-time video processing, physiological data calculation, and content adaptive adjustment. The built-in DSP module is used to perform low-power filtering and noise reduction preprocessing of multimodal physiological signals to improve data accuracy. The device has a storage capacity of not less than 5.0MB for temporarily caching user physiological signal data, video EBMP tag metadata, and narrative branch configuration information. Furthermore, the video playback device 100 is equipped with WiFi 6 and Bluetooth 5.0 communication modules, thereby enabling low-latency data transmission with the physiological signal acquisition device 200. In some implementations, such as Figure 1 As shown, the system also includes: smart home devices 300. In this way, the video playback device 100 can also be linked with external smart home devices 300 via Zigbee or WiFi IoT protocols, including smart lights with adjustable color temperature and brightness, temperature-controlled air conditioners, smart curtains, fresh air equipment, etc. The video playback device 100 can also be equipped with an HDMI 2.1 interface, which can realize low-latency picture output and ensure smooth and lag-free switching of narrative branches and gradual adjustment of audio and video parameters. In some implementations, such as Figure 1 As shown, the system also includes a video content providing device 400, which can provide content to cloud servers or content distribution servers, serving as the content pre-production and distribution end of the entire system, responsible for the standardized encapsulation and rule pre-configuration of video resources. In this way, the video content providing device 400 can embed one or more narrative branch tags corresponding to video narrative branches in the encapsulation metadata layer of the current video during the video production and encapsulation stage, based on the Emotion Branch Marking Protocol EBMP. The narrative branch tags include one or more of the following: trigger conditions, switching actions, and switching transition duration, providing a standardized rule basis for the terminal device's emotion analysis, branch determination, and adaptive playback control. Based on the complete four-terminal collaborative architecture of the content provider, data acquisition, intelligent playback, and environmental execution terminals built by this system, the physiological signal acquisition device 200 collects real-time physiological data of users imperceptibly, the video content provider 400 pre-embeds video EBMP narrative markers and distributes resources, and the video playback device 100 uses built-in algorithms to complete user-personalized baseline calibration and multimodal signal weighted fusion to generate continuous emotion intensity values. Combined with the pre-embedded EBMP narrative branch markers, it dynamically matches and calculates playback parameters in real time, achieving smooth switching of the same narrative branches and continuous gradual adjustment of audio and video subtitle parameters without changing the original plot of the video. At the same time, it is compatible with both single-person and multi-person collaborative viewing scenarios, adapts to the multi-user emotion fusion and control logic, and can also link with smart home devices to complete the synchronous adaptation of the whole-house environment atmosphere. Ultimately, it achieves a seamless, personalized, and fully immersive adaptive video playback effect based on the user's real-time emotions.

[0027] Built as Figure 1 As shown, after being used in a video playback system, video playback devices, such as televisions and set-top boxes, can rely on the user's physiological data collected imperceptibly by wearable devices, combined with the built-in biofeedback mapping algorithm and EBMP narrative branch parsing mechanism, to achieve imperceptible, personalized, smooth and adaptive intelligent video playback control. Figure 2 This is a flowchart illustrating a video playback method provided in an embodiment of this disclosure. Figure 2 As shown, the process for video playback includes: Step 201: Obtain the current multimodal physiological information of the user watching the current video, and determine the current emotional intensity value corresponding to the current multimodal physiological information; wherein, the multimodal physiological information includes one or more of the following: skin conductance response (GSR), heart rate (HR), skin temperature, and respiratory rate.

[0028] Here, it can be accessed via Figure 1 The physiological signal acquisition device shown collects multimodal physiological information of the user in real time and continuously during the movie viewing process, and transmits it to the video playback device through low-latency wireless communication. The video playback device can then filter and denoise the acquired multimodal physiological information to remove abnormal data caused by environmental interference and motion artifacts, thus ensuring the accuracy and continuity of the original physiological data.

[0029] In some embodiments, determining the current emotion intensity value corresponding to the current multimodal physiological information includes: preprocessing the current multimodal physiological information and obtaining the normalized value corresponding to each physiological information in the current multimodal physiological information based on the baseline calibration range, wherein the baseline calibration range is the range of user physiological signals recorded when guiding the user to complete four standard emotion induction tasks of calm, excitement, tension and relaxation; and performing weighted fusion based on formula (1) according to each normalized value to obtain the current emotion intensity value E. E = 100×(w1×GSR_norm + w2×HR_norm + w3×Temp_norm + w4×RR_norm)(1)where: GSR_norm, HR_norm, Temp_norm, and RR_norm are the normalized values ​​corresponding to skin conductance response (GSR), heart rate (HR), skin temperature, and respiratory rate, respectively, and w1, w2, w3, and w4 are dynamic weighting coefficients, and the sum of the weighting coefficients is 1.

[0030] Specifically, when a video playback device establishes a wireless connection and activates the function for the first time, the device can automatically trigger a personalized baseline calibration process, guiding the user to complete four standardized emotion-inducing tasks in sequence: sitting quietly and maintaining a calm state for two minutes, watching a preset excitement segment, watching a preset tension segment, and watching a preset relaxation segment. Throughout the process, the device continuously collects and records raw data of various dimensions of physiological signals of the user under different standard emotional states, and fits the baseline values ​​and dynamic change ranges of physiological signals specific to the user. This can accurately adapt to the individual differences in physiological characteristics of different users, avoid the emotion recognition bias caused by a uniform judgment standard, and achieve accurate emotion perception for different users. After baseline calibration is completed, the physiological signal acquisition device can continuously acquire real-time physiological data of the user during the movie viewing process at a sampling rate of no less than 5.0Hz. The data is then uploaded to the video playback device in real time via Bluetooth 5.0 or WiFi 6 wireless link. The video playback device can then perform filtering and noise reduction preprocessing of multimodal physiological signals to remove abnormal data caused by motion artifacts and environmental interference, ensuring the stability and validity of the input data. Then, the physiological signals are normalized in combination with the user's exclusive baseline range, and the original physiological data of different dimensions and amplitudes are uniformly mapped to the standard normalized value in the [0,1] interval. This gives the normalized value corresponding to each physiological information in the current multimodal physiological information.

[0031] Then, based on each normalized value, a weighted fusion is performed according to formula (1) to obtain the current emotion intensity value E. Among them, the weight coefficients can be adaptively fine-tuned according to the individual user baseline calibration results. The system defaults to a fixed benchmark weight. In some embodiments, w1=0.4, w2=0.3, w3=0.2, w4=0.1, and the sum of all weight coefficients is 1.0, ensuring the rationality and stability of the multimodal physiological signal fusion results. Furthermore, through differentiated weight allocation, physiological indicators such as skin conductance response and heart rate, which are more sensitive to emotional fluctuations during movie watching, can be highlighted, while low-correlation signal interference is weakened, further improving the accuracy of emotion intensity value calculation. This disclosure uses the 100x linear amplification coefficient built into formula (1) to directly map and scale the normalized weighted fusion result in the [0, 1] interval, and finally outputs a continuous quantitative emotion intensity value E in the [0, 100] interval, which can accurately characterize the real-time and subtle emotional fluctuations of the user during movie watching.

[0032] Step 202: If the current emotion intensity value satisfies the current triggering condition corresponding to the current video narrative branch, determine the current playback parameters corresponding to the current video narrative branch based on the current emotion intensity value. The current triggering condition is obtained by parsing the narrative branch markers embedded in the current video.

[0033] In this embodiment of the disclosure, one or more narrative branch markers corresponding to video narrative branches can also be embedded in the encapsulated metadata layer of the current video based on the Emotion Branch Marker Protocol (EBMP). The narrative branch markers include one or more of the following: triggering conditions, switching actions, and switching transition durations.

[0034] In some embodiments, during the offline video production stage, the video content providing device may embed invisible narrative branch markers in the video encapsulation metadata layer according to the EBMP standard. These markers do not occupy video frame layers or affect the original video playback effect; they serve only as rule configuration information that the video playback device can parse. For example, the standardized format of a single EBMP narrative branch marker is: <ebmp marker="分支ID" trigger="E>The `75" action="Switch to branch A" duration="500ms">` directive is used to uniquely identify the corresponding video narrative branch. The `trigger` is used to configure a threshold trigger condition based on the continuous emotional intensity value `e`. The `action` is used to bind a preset narrative branch switching action, and the `duration` is used to limit the transition duration for smooth branch switching. In some embodiments, the default transition duration is configured as 500ms to ensure smooth branch switching without abrupt jumps. Furthermore, in this embodiment, the `ebmp` marker only stores trigger rules and switching configurations, without pre-setting any fixed playback adjustment parameters for audio, subtitles, or video. All playback parameters can be dynamically calculated and generated by the video playback device based on the user's real-time emotional intensity value, thereby effectively reducing video content production costs and achieving personalized adjustment effects for the same video content to suit different users' emotional states. In some embodiments, the video playback device runs an AI real-time content modulation engine to compare and determine the EBMP trigger conditions obtained from parsing with the real-time emotion intensity value. If the current emotion intensity value meets the trigger conditions of the corresponding narrative branch, the corresponding video playback parameters can be dynamically calculated and matched based on the quantified value of the current emotion intensity value to achieve adaptive modulation of multi-dimensional audio-visual content. In some embodiments, determining the current playback parameters corresponding to the current video narrative branch includes: increasing the gain value of the first range of audio when the current emotional intensity value E is greater than a first threshold, and increasing the gain value of the second range of audio when the current emotional intensity value E is less than a second threshold, wherein the first threshold is greater than the second threshold, and the frequency value corresponding to the first range is greater than the frequency value corresponding to the second range; switching the subtitles to a bright color scheme and enlarging the subtitle font when the current emotional intensity value E is greater than a third threshold, and switching the subtitles to a soft color scheme and restoring the default subtitle font size when the current emotional intensity value E is less than a fourth threshold, wherein the third threshold is greater than the fourth threshold; increasing the volume of the background music when the current emotional intensity value E is greater than a fifth threshold, and decreasing the volume of the background music when the current emotional intensity value E is less than a sixth threshold, wherein the fifth threshold is greater than the sixth threshold.

[0035] In these embodiments, the specific playback parameters are determined in three dimensions: audio gain adjustment, subtitle style adjustment, and background music volume adjustment. Specifically, regarding audio gain adjustment: a first threshold and a second threshold are preset, with the first threshold being greater than the second threshold. If the current emotional intensity value E is greater than the first threshold, it can be determined that the user is in a tense or excited emotional state, and the audio gain of the high-frequency band (2kHz-8kHz) is increased, meaning that the first range of audio is high-frequency audio, thus enhancing the clarity and atmosphere of the picture and sound effects. If the current emotional intensity value E is less than the second threshold, it can be determined that the user is in a relaxed or calm emotional state, and the audio gain of the low-frequency band (60Hz-250Hz) is increased, meaning that the second range of audio can be low-frequency audio, thus enhancing the soft and warm texture of the audio and video, and achieving differentiated adaptation of audio frequency bands under different emotional states. Regarding subtitle style adjustment: a third and fourth threshold are preset, with the third threshold being greater than the fourth threshold. If the current emotional intensity value E is greater than the third threshold, the video subtitles will be switched to a bright color scheme and the font size will be enlarged to enhance the visual effect of tense and suspenseful plot. If the current emotional intensity value E is less than the fourth threshold, the subtitles will be switched to a soft color scheme and the default font size will be restored to match a relaxed and soothing viewing atmosphere. Regarding background music adjustment: a fifth and a sixth threshold are preset, with the fifth threshold being greater than the sixth threshold. If the current emotional intensity value E is greater than the fifth threshold, the background music volume is increased to enhance the emotional atmosphere; if the current emotional intensity value E is less than the sixth threshold, the background music volume is decreased to create a comfortable and calm viewing experience. For example, in some implementation scenarios, a high-emotion trigger condition can be configured as E>75. When the current emotion intensity value E>75, a tense narrative branch is triggered, simultaneously executing modulation logic such as high-frequency gain boost, increased BGM volume, and prominently enlarged subtitles. A low-emotion trigger condition can be configured as E<30. If the current emotion intensity value E<30, a relaxed narrative branch is triggered, simultaneously executing modulation logic such as low-frequency gain boost, decreased BGM volume, and softer subtitle display. Furthermore, in this embodiment, the AI ​​real-time content modulation engine relies on continuous emotion intensity values ​​to achieve fine-grained parameter matching, rather than a fixed adjustment at a single level. It can dynamically adapt the adjustment range according to the E value, achieving a flexible and gradual adaptation of audio-visual parameters to the user's emotional fluctuations. In some embodiments, when the EBMP flag triggering condition is met, the terminal can replace the screen elements of the corresponding narrative branch according to the switching action bound to the flag. While keeping the original main plot of the video unchanged, the terminal can adaptively replace the background screen, auxiliary materials, adapted content branches and other audiovisual elements to ensure narrative continuity and achieve accurate matching between the viewing atmosphere and the user's real-time emotions.

[0036] Step 203: Play the current video according to the current playback parameters.

[0037] In some embodiments, playing the current video includes: switching to a source narrative branch of the current narrative branch within the transition duration corresponding to the current narrative branch marker, while keeping the main plot of the current video unchanged; and continuously and gradually adjusting the audio, subtitle, and video parameters according to the current playback parameters within the transition duration corresponding to the current narrative branch marker.

[0038] Specifically, the AI ​​real-time content modulation engine of the video playback device can perform real-time audio-visual modulation and branch synchronization switching processes after obtaining the real-time emotion intensity value E and the corresponding current playback parameters. First, based on the current playback parameters, including audio EQ parameters and BGM volume parameters, it can generate fine-grained audio modulation instructions. Then, relying on the device's built-in NPU computing power, it runs audio processing algorithms to perform real-time equalization gain adjustment and dynamic modulation of the background music volume on the original video audio track. The overall algorithm processing latency is controlled within the switching transition time, such as within 200ms, ensuring that the audio adjustment is real-time, seamless, and without lag. While the audio is modulated, the subtitle visual parameters in the current playback parameters determined by the current emotion intensity value E can be updated in real time to update the subtitle color and font size, realizing the dynamic adaptive switching of subtitle style according to the user's emotional state, and completing the fine-grained gradual adaptation of the subtitle dimension. Meanwhile, if the current emotion intensity value E meets the triggering condition of the EBMP narrative branch marker, then according to the preset transition duration of the marker, such as 500ms, a smooth transition between the same narrative branch and replaceable visual elements can be completed without changing the original main plot of the video or altering the narrative content and story logic. Only the video background, atmosphere materials, style-matching elements, and other audiovisual content are changed, ensuring the coherence and natural rhythm of the film's narrative. In this way, this embodiment can complete branch switching, audio modulation, subtitle adjustment, and visual style updates all synchronously within the preset transition duration, relying on the continuous emotion value to drive the stepless gradual adjustment of parameters, avoiding the disjointed experience problems such as abrupt changes in visuals, sudden changes in sound effects, and abrupt changes in subtitles caused by traditional fixed-level switching. After all audio, video, subtitles, and picture modulation are completed, the device will output the fully adapted audio and video picture and audio to the display device for playback in real time. It realizes low-latency adaptive control of the entire link of emotion perception, parameter calculation, content modulation, branch switching, and picture output, and truly achieves real-time, seamless, and accurate matching between user emotions and video viewing atmosphere. Correspondingly, it realizes smooth switching of the same source branch and continuous gradual adjustment of parameters, which greatly improves the smoothness, immersion and personalization of intelligent viewing.

[0039] As can be seen, in this embodiment, a personalized continuous emotion intensity value is constructed based on the user's multimodal physiological signals. Combined with the EBMP emotion branching tagging protocol, the video playback parameters are dynamically matched. Without changing the main plot of the video, the continuous gradual adjustment of audio, video, subtitles, and picture and the smooth switching of the same narrative branches are completed. This effectively solves the problems of fixed video playback parameters, stiff interaction, and poor emotion adaptability in traditional video playback, and realizes low latency, imperceptible, and personalized intelligent video adaptive playback.

[0040] In some scenarios, there may be one, two, or more users watching the video. Therefore, in some embodiments, obtaining the current multimodal physiological information of the user watching the current video includes: if there are multiple users watching the current video, determining the user with the longest screen gaze duration as the current user and obtaining the current user's current multimodal physiological information; or, if there are multiple users watching the current video, obtaining the multimodal physiological information of each user, such that the current emotional intensity value is obtained by averaging the emotional intensity values ​​corresponding to each multimodal physiological information.

[0041] In multi-person movie-watching scenarios, all viewers can wear physiological signal acquisition devices. Through these devices, the video playback device can simultaneously collect multimodal physiological information from each user and calculate independent emotional intensity values ​​E1, E2, E3…En for each user. In some embodiments, the screen-gazing duration of each user can be monitored in real time. The user with the longest screen-gazing duration and highest viewing engagement is identified as the primary viewer. The emotional intensity value of this primary viewer is used as the sole core driving signal to adjust subsequent video playback parameters and atmosphere, aligning with the core viewer's actual experience and ensuring an immersive viewing experience. Alternatively, the device can aggregate the independent emotional intensity values ​​of all viewers and calculate a global average emotional intensity value using the mean fusion formula E_avg=(E1+E2+E3+…+En) / N. This balanced emotional value is used as the unified driving basis, taking into account the overall emotional state of all viewers and avoiding frequent abrupt changes in the video atmosphere caused by extreme fluctuations in the emotions of a single user. Furthermore, the device can uniformly drive the adaptive adjustment of video audio, subtitles, and related narrative branches based on the final emotional intensity value obtained through filtering or fusion. This effectively fills the technical gap in traditional adaptive video solutions that lack multi-person scene adaptation, and significantly improves the scenario compatibility and viewing adaptation balance of the solution.

[0042] like Figure 1 As shown, the system for video playback also includes smart home devices, such as smart lights with adjustable color temperature and brightness, temperature-controlled air conditioners, smart curtains, and fresh air systems. Therefore, whether it is a single-person viewing scenario or a multi-person viewing scenario, the video playback device can also be linked with smart home devices. That is, in some embodiments, it also includes: controlling home environment devices according to the current emotional intensity value E, wherein the home environment devices include one or more of the following: lights, curtains, air conditioners, and fresh air systems.

[0043] Specifically, video playback devices can simultaneously match and calculate home environment parameters that adapt to the current user's viewing mood based on the calculated real-time continuous emotional intensity value E, achieving integrated and coordinated control of the video content atmosphere and the physical environment atmosphere. For example, in terms of intelligent lighting control, the device can adaptively adjust color temperature and brightness according to the emotional threshold: if the current emotional intensity value E > 60, it can be determined that the user is in an excited or tense viewing state, and the lighting can be switched to a 2700K warm color tone to create an immersive atmosphere; while if the current emotional intensity value E < 40, it can be determined that the user is in a calm or relaxed viewing state, and the lighting can be switched to a 6500K cool color tone to create a refreshing and comfortable viewing environment; at the same time, the light brightness is dynamically adapted to the emotional intensity value, maintaining 50% basic brightness when the E value is in the range of [40, 60], and increasing to 80% high brightness when the E value is greater than 80% in a high emotional state, achieving flexible adaptation of brightness to emotional fluctuations. Regarding air conditioning temperature control, if the current emotional intensity value E > 70, indicating high tension and excitement, the air conditioner can be lowered by 2°C to help cool down and soothe the user, relieving tension. Conversely, if the current emotional intensity value E < 30, indicating a relaxed state, the air conditioner can be raised by 1°C to further enhance the comfortable and relaxing movie-watching experience. Similarly, smart curtains and fresh air systems can be synchronized with the overall movie-watching atmosphere. For example, in a state of high emotional tension, the curtains can automatically close and the fresh air level can be lowered to block out external interference and focus on the movie-watching atmosphere. In a state of low emotional relaxation, the curtain opening and closing can be slightly adjusted to maintain a stable fresh air output, adapting to a relaxed movie-watching experience, achieving coordinated and adaptive control of all types of home appliances. Specific details are not listed here.

[0044] In actual operation, the video playback device can calculate the corresponding control command based on the current emotion intensity value E, and then send precise control commands to all smart devices in the house via IoT protocols such as Zigbee or WiFi. Smart lights, air conditioners, fresh air systems, and curtains can respond quickly within 1 to 2 seconds and adjust their parameters, achieving synchronous adaptation between the home environment and the video audiovisual atmosphere. Therefore, this embodiment breaks down the traditional technical barriers between independent audiovisual playback and the home environment, using the same emotion intensity value as a unified driving source to achieve bidirectional linkage between video content modulation and physical environment control. This allows for a high degree of unity between the content atmosphere and the environmental atmosphere, comprehensively enhancing the immersive and integrated smart viewing experience.

[0045] The following describes the operation process in a specific embodiment, illustrating the video playback process provided by the embodiments of the present invention.

[0046] In one embodiment of this disclosure, the system for video playback can be as follows: Figure 1 As shown, the video playback device uses a smart TV as the core processing and playback device. Viewers wear professional smart bracelets that support GSR (Geophysical Skin Resonance) response data acquisition. The entire system constructs a closed-loop linkage architecture of "wearable acquisition end - TV processing end - smart home execution end." The smart TV can establish low-latency data communication with the smart bracelet via Bluetooth 5.0 or WiFi wireless links, and simultaneously network with smart home devices such as smart lights, air conditioners, smart curtains, and fresh air systems via the Zigbee / WiFi IoT protocol. During the video content production stage, the content creator has pre-embedded implicit narrative branch markers corresponding to the video narrative branches in the video encapsulation metadata layer based on the Emotional Branch Marking Protocol (EBMP) of this invention through the video content providing device. These markers do not occupy video frame layers or affect the original video playback effect; they are only used for TV rule parsing and adaptive adjustment. The standardized format of a single EBMP narrative branch marker is: <ebmp marker="分支ID" trigger="E>After the TV is initially paired with the user's smart bracelet, it automatically triggers a personalized baseline calibration process, guiding the user to complete four standardized emotion induction tasks: calm, excitement, tension, and relaxation. It continuously collects and records the baseline values ​​and dynamic change ranges of multimodal physiological signals under different emotional states, establishing a user-specific baseline calibration range. This provides benchmark support for subsequent physiological signal normalization processing and personalized emotion calculation. After completing the pre-initialization configuration, the TV can enter real-time adaptive video playback mode. Figure 3 This is a schematic diagram of signaling interaction for a video playback method provided in an embodiment of this disclosure. Figure 3 As shown, the entire system achieves seamless emotion adaptation and intelligent control through real-time multi-terminal signaling interaction. The specific video playback process includes: Step 301: The TV acquires the current multimodal physiological information collected by the user's smart bracelet. The multimodal physiological information includes one or more of the following: skin conductance response (GSR), heart rate (HR), skin temperature, and respiratory rate.

[0047] The smart bracelet collects dynamic physiological data of the user in real time during the movie viewing process, and after preliminary noise reduction preprocessing, it is wirelessly transmitted to the TV. The TV uses its built-in DSP module to perform secondary filtering and outlier removal on the received multimodal physiological signals, eliminating data deviations caused by motion artifacts and environmental interference, and ensuring the continuity and accuracy of the input data. Step 302: Based on the baseline calibration range corresponding to the current user, the TV performs normalization processing on the current multimodal physiological information to obtain the normalized value corresponding to each physiological information in the current multimodal physiological information.

[0048] Among them, the baseline calibration range is the user's personalized physiological benchmark range established through four standard emotion-inducing tasks of calm, excitement, tension and relaxation when the user first binds the smart bracelet device, which can eliminate the influence of individual differences in the physiological characteristics of different users.

[0049] Step 303: The TV performs weighted fusion based on each normalized value and formula (1) to obtain the current emotional intensity value E.

[0050] The original physiological data of different dimensions and amplitudes are uniformly mapped to the interval [0, 1]. Then, the continuous real-time emotional intensity value E of the user in the interval [0, 100] is calculated by using a preset weighted fusion algorithm combined with dynamic weight coefficients. In formula (1), w1=0.4, w2=0.3, w3=0.2, w4=0.1, and the total weight is 1.0. In this way, it can be adaptively fine-tuned according to the user's baseline characteristics to accurately represent the subtle fluctuations in the user's real-time movie-watching emotions. Step 304: The TV parses the EBMP narrative branch markers embedded in the currently playing video to obtain the emotional triggering conditions, branch switching actions, and preset switching transition durations corresponding to each narrative branch.

[0051] This embodiment adopts the EBMP emotion branching tagging protocol, storing trigger rules and switching logic only at the video metadata layer, without pre-setting any fixed audio and video playback parameters in the video file. This achieves a decoupled design of rules and parameters, significantly reducing the cost of video content production and iteration, and significantly improving the universality, adaptability, and scalability of the same video content.

[0052] Step 305: Determine if the emotion intensity value meets the triggering condition of the corresponding narrative branch. If yes, proceed to step 306; otherwise, proceed to step 309.

[0053] Step 306: The TV uses the AI ​​real-time content modulation engine to dynamically calculate and match the current playback parameters corresponding to the current video narrative branch based on the current E value. The playback parameters include: audio EQ parameters, BGM volume parameters, subtitle style parameters, and picture branch parameters.

[0054] In this way, by relying on the computing power of the edge NPU to realize real-time parameter calculation, differentiated and personalized audio-visual adjustment parameters can be output according to the user's real-time emotional state, so as to achieve a "one person, a thousand faces" intelligent viewing adaptation effect.

[0055] Step 307: Within the transition time preset by the EBMP marker, the TV completes the smooth switching of the same narrative branches and the continuous gradual adjustment of all-dimensional audio-visual parameters, and continuously outputs the modulated audio and video images.

[0056] During the transition, the original storyline and narrative logic of the video remain unchanged, with only the audiovisual content such as video atmosphere materials and background images being replaced. At the same time, the audio equalization gain, background music volume, subtitle color and font size are gradually modulated within a low latency of 200ms, outputting video and audio that adapt to the user's real-time emotional state. This completely avoids the problems of abrupt transitions and disjointed experience in traditional interactive videos, ensuring smooth viewing and immersion. Step 308: The TV activates all smart home devices in the house based on the current emotional intensity value E.

[0057] In this way, based on the current emotional intensity value E, the system can adaptively match environmental control parameters such as smart light color temperature, light brightness, and air conditioning temperature. Control commands can be sent through Zigbee or WiFi IoT protocols, and various home devices can respond quickly within 1 to 2 seconds and complete parameter adaptation, achieving a high degree of synchronization and collaborative adaptation between the video audiovisual atmosphere and the home physical environment atmosphere. Step 309: The TV remains in the current video playback state.

[0058] If the user's real-time emotional intensity value does not reach the trigger threshold of any narrative branch, the system maintains the current audio and video playback parameters and screen branches unchanged, continuously collecting physiological signals, updating emotional intensity values, and monitoring trigger conditions to ensure stable video playback and no frequent erroneous switching.

[0059] As can be seen, in this embodiment, the TV relies on a multi-terminal collaborative system architecture and solves the technical pain points of traditional video playback modes, such as fixed modes, reliance on manual interaction, poor accuracy of emotion recognition, abrupt branch switching, inability to adapt to multi-person scenarios and lack of environmental linkage, through a complete set of technical means, including personalized baseline calibration, multimodal signal weighted fusion quantification of continuous emotion values, EBMP decoupled narrative labeling rules, AI real-time stepless audio-visual modulation, seamless switching of same-source branches, and synchronous linkage with home environment. It achieves the technical effects of imperceptible triggering, low-latency modulation, personalized adaptation, full-scene compatibility, and software and hardware collaborative linkage. Without changing the main narrative line of the video, it allows the audio-visual atmosphere, picture style, and home environment to be flexibly adapted to the user's real-time emotional dynamics, significantly improving the personalization, smoothness, and immersive experience of smart viewing.

[0060] Based on the above process for video playback, a device for video playback can be constructed.

[0061] Figure 4 This is a schematic diagram of a video playback device provided in an embodiment of this disclosure. Figure 4 As shown, the video playback device includes: an acquisition and determination module 410, an analysis and determination module 420, and an adjustment playback module 430.

[0062] The acquisition and determination module 410 is configured to acquire the current multimodal physiological information of the user watching the current video and determine the current emotional intensity value corresponding to the current multimodal physiological information. The multimodal physiological information includes one or more of the following: skin conductance response (GSR), heart rate (HR), skin temperature, and respiratory rate.

[0063] The analysis and determination module 420 is configured to determine the current playback parameters corresponding to the current video narrative branch based on the current emotion intensity value, provided that the current emotion intensity value meets the current triggering condition corresponding to the current video narrative branch. The current triggering condition is obtained by parsing the narrative branch markers embedded in the current video.

[0064] The playback module 430 is adjusted and configured to play the current video according to the current playback parameters.

[0065] In some embodiments, the determination module 410 includes: The normalization unit is configured to preprocess the current multimodal physiological information and obtain the normalized value corresponding to each physiological information in the current multimodal physiological information based on the baseline calibration range. The baseline calibration range is the range of user physiological signals recorded when guiding the user to complete four standard emotion induction tasks: calm, excitement, tension, and relaxation.

[0066] The fusion unit is configured to perform weighted fusion based on formula (1) for each normalized value to obtain the current emotion intensity value E.

[0067] E = 100×(w1×GSR_norm + w2×HR_norm + w3×Temp_norm + w4×RR_norm)(1)where: GSR_norm, HR_norm, Temp_norm, and RR_norm are the normalized values ​​corresponding to skin conductance response (GSR), heart rate (HR), skin temperature, and respiratory rate, respectively, and w1, w2, w3, and w4 are dynamic weighting coefficients, and the sum of the weighting coefficients is 1.

[0068] In some embodiments, it also includes: The embedding module is configured to embed one or more narrative branch markers corresponding to video narrative branches in the encapsulation metadata layer of the current video based on the Emotion Branch Marker Protocol (EBMP). The narrative branch markers include one or more of the following: trigger conditions, switching actions, and switching transition durations.

[0069] In some embodiments, the analysis and determination module 420 includes: The first adjustment unit is configured to increase the gain value of the first range of audio when the current emotion intensity value E is greater than a first threshold, and to increase the gain value of the second range of audio when the current emotion intensity value E is less than a second threshold, wherein the first threshold is greater than the second threshold, and the frequency value corresponding to the first range is greater than the frequency value corresponding to the second range.

[0070] The second adjustment unit is configured to switch the subtitles to a bright color scheme and enlarge the subtitle font when the current emotion intensity value E is greater than the third threshold, and to switch the subtitles to a soft color scheme and restore the default subtitle font size when the current emotion intensity value E is less than the fourth threshold. The third threshold is greater than the fourth threshold.

[0071] The third adjustment unit is configured to increase the volume of the background music when the current emotional intensity value E is greater than the fifth threshold, and to decrease the volume of the background music when the current emotional intensity value E is less than the sixth threshold, wherein the fifth threshold is greater than the sixth threshold.

[0072] In some embodiments, the playback module 430 is specifically configured to switch the current narrative branch to a source narrative branch within the transition duration corresponding to the current narrative branch marker, while keeping the main storyline of the current video unchanged; and to perform continuous and gradual adjustments to the audio, subtitle, and video parameters according to the current playback parameters within the transition duration corresponding to the current narrative branch marker.

[0073] In some embodiments, the determination module 410 is further configured to, in the case of multiple users watching the current video, determine the user with the longest screen gaze duration as the current user and obtain the current multimodal physiological information of the current user; or, in the case of multiple users watching the current video, obtain the multimodal physiological information of each user, such that the current emotional intensity value is obtained by averaging the emotional intensity values ​​corresponding to each multimodal physiological information.

[0074] In some embodiments, it also includes: The linkage control module is configured to control home environment devices based on the current emotional intensity value E. These home environment devices include one or more of the following: lighting fixtures, curtains, air conditioners, and fresh air systems.

[0075] The following example illustrates the video playback process of the device for video playback provided in the embodiments of the present invention.

[0076] In one embodiment of this disclosure, the device for video playback can be applied to a television, such as... Figure 1 In the system architecture, the video content providing device includes an embedding module, which, based on the Emotional Branch Marking Protocol (EBMP) of this invention, pre-embeds implicit narrative branch markers corresponding to the video narrative branches in the video encapsulation metadata layer. These markers do not occupy video frame layers or affect the original video playback effect; they are only used for television rule parsing and adaptive adjustment. The standardized format of a single EBMP narrative branch marker is as follows: <ebmp marker="分支ID" trigger="E>75" action="Switch to branch A" duration="500ms" >. And the devices in a television used for video playback, such as As shown in Figure 5, the system includes: an acquisition and determination module 410, an analysis and determination module 420, an adjustment playback module 430, and a linkage control module 440. The acquisition and determination module 410 includes: a normalization unit 411 and a fusion unit 412, while the analysis and determination unit 420 includes: a first adjustment unit 421, a second adjustment unit 422, and a third adjustment unit.

[0077] After the TV is initially paired with the user's smart bracelet, it automatically triggers a personalized baseline calibration process, guiding the user to complete four standardized emotion induction tasks: calm, excitement, tension, and relaxation. It continuously collects and records the baseline values ​​and dynamic change ranges of multimodal physiological signals under different emotional states, establishing a user-specific baseline calibration range to provide personalized benchmark support for subsequent physiological signal normalization processing and accurate emotion quantification.

[0078] In this way, when there are multiple users watching the current video, the TV can identify the user with the longest screen viewing time as the current user. The acquisition module 410 then acquires the current multimodal physiological information collected by the current user's smart bracelet. This multimodal physiological information includes one or more of the following: skin conductance response (GSR), heart rate (HR), skin temperature, and respiratory rate. Furthermore, the normalization unit 411 in the acquisition module 410 normalizes the current multimodal physiological information based on the baseline calibration range corresponding to the current user, mapping the original physiological data of different dimensions and amplitudes to the [0, 1] interval, thus obtaining the normalized value corresponding to each physiological information in the current multimodal physiological information. The fusion unit 412 then performs weighted fusion based on formula (1) according to each normalized value and amplifies the range by a factor of 100, ultimately obtaining a continuously quantized current emotional intensity value E in the [0, 100] interval, achieving a refined quantitative representation of the subtle fluctuations in the user's viewing emotions.

[0079] The analysis and determination module 420 can parse the EBMP narrative branch markers embedded in the currently playing video in real time, accurately obtaining the emotion trigger threshold conditions, branch switching actions, and preset switching transition durations corresponding to each narrative branch. Thus, when the emotion intensity value meets the trigger conditions of the corresponding narrative branch, the analysis and determination module 420 dynamically calculates and matches the refined playback parameters corresponding to the current video narrative branch based on the continuously changing E value, achieving seamless adaptation and adjustment of emotion and audio-visual parameters.

[0080] ​Specifically, if the current emotional intensity value E > the first threshold 70, the first adjustment unit 421 increases the audio gain of the high-frequency band (2kHz-8kHz) by +6dB to enhance the clarity of the sound effects and the sense of tension. At the same time, the second adjustment unit 422 switches the video subtitles to a bright red (#FF0000) and enlarges the subtitle font by 20%, while the third adjustment unit 423 increases the BGM volume by 15% to match the user's excited and tense emotional state. If the current emotional intensity value E < the second threshold 30, the first adjustment unit 421 increases the audio gain of the low-frequency band (60Hz-250Hz) by +3dB to enhance the warm and soft texture of the audio. At the same time, the second adjustment unit 422 switches the subtitles to a soft blue (#0000FF) and restores the standard font size of the subtitles. The third adjustment unit 423 reduces the BGM volume by 10% to match the user's relaxed and soothing viewing mood. In this embodiment, the first, third, and fifth thresholds are equal to the high-emotion trigger thresholds for controlling audio, subtitles, and background music, and can be 70; while the second, fourth, and sixth thresholds are equal to the low-emotion trigger thresholds, and can be 30.

[0081] Therefore, the playback module 430 can invoke corresponding branch switching actions, smoothly switching between narrative branches within a preset 500ms transition time. This maintains the original main storyline and narrative logic of the video throughout, only replacing background visuals, atmosphere materials, and style-appropriate elements, effectively avoiding the abrupt transitions and disjointed storylines of traditional interactive videos. Furthermore, within a 200ms low-latency edge-side processing time, the playback module 430 simultaneously performs continuous and gradual modulation of audio EQ gain, BGM volume, subtitle style, and visual branches, continuously outputting the adapted audio and video to the display device. This achieves seamless, low-latency adaptation across the entire process, from emotion-driven processing and parameter calculation to content modulation and visual output. Meanwhile, the linkage control module 440, based on a unified emotion intensity value E, links all smart home devices in the house to achieve coordinated and synchronous control of the audio-visual atmosphere and the physical environment atmosphere. Specifically, when E > 60, the lighting switches to a warm 2700K tone, and when E < 40, it switches to a cool 6500K tone, dynamically adjusting the brightness of the lights according to the E value. When E > 70, the air conditioner is lowered by 2°C to alleviate the user's tense and excited state, and when E < 30, the air conditioner is raised by 1°C to enhance the comfortable and relaxing experience. All home devices can quickly respond and complete parameter adaptation within 1 to 2 seconds.

[0082] As can be seen, in this embodiment, the device for video playback, through modular division of labor and cooperation, relies on the complete device logic of pre-configuration of EBMP rules, personalized baseline calibration, multimodal emotion fusion quantification, multi-dimensional audio-visual adaptive modulation, smooth branch switching of the same source, and linkage of the whole house environment. It gets rid of the defects of traditional video playback parameters being fixed, requiring manual interaction adjustment, low emotion adaptation accuracy, and single scene adaptation, and achieves a smart video adaptive playback effect that is imperceptible, low-latency, personalized, and immersive in all scenarios for single or multiple people.

[0083] This disclosure provides an apparatus 600 for video playback, the structure of which is as follows: Figure 6 As shown, it includes: The processor 1000 and memory 1001 may further include a communication interface 1002 and a bus 1003. The processor 1000, communication interface 1002, and memory 1001 can communicate with each other via the bus 1003. The communication interface 1002 can be used for information transmission. The processor 1000 can call logical instructions stored in the memory 1001 to execute the video playback method described in the above embodiment.

[0084] Furthermore, the logic instructions in the aforementioned memory 1001 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium.

[0085] The memory 1001, as a computer-readable storage medium, can be used to store software programs and computer-executable programs, such as program instructions / modules corresponding to the methods in the embodiments of this disclosure. The processor 1000 executes functional applications and data processing by running the program instructions / modules stored in the memory 1001, that is, it implements the method for video playback in the above method embodiments.

[0086] The memory 1001 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the terminal device. Furthermore, the memory 1001 may include high-speed random access memory and may also include non-volatile memory.

[0087] This disclosure provides a video playback device, including: a processor and a memory storing program instructions, wherein the processor is configured to execute a video playback method when executing the program instructions.

[0088] This disclosure provides a device, specifically a television, set-top box, projector, or other device with video playback functionality, such as the video playback device described above. Figure 7 As shown, the device includes a device body 700, in which the aforementioned video playback device 400 (600) is installed. The installation relationship described herein is not limited to placement within the product, but also includes installation connections with other components of the product, including but not limited to physical connections, electrical connections, or signal transmission connections. Those skilled in the art will understand that the video playback device 400 (600) can be adapted to suitable corresponding device bodies to achieve other feasible embodiments.

[0089] This disclosure provides a storage medium storing program instructions that, when executed, perform the video playback method described above.

[0090] This disclosure provides a computer program product, which includes a computer program stored on a storage medium. The computer program includes program instructions that, when executed by a computer, cause the computer to perform the above-described video playback method.

[0091] The aforementioned storage medium can be a transient computer-readable storage medium or a non-transitory computer-readable storage medium.

[0092] The technical solutions of this disclosure can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes one or more instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in this disclosure. The aforementioned storage medium can be a non-transitory storage medium, including: a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and other media capable of storing program code; it can also be a transient storage medium.

[0093] The foregoing description and accompanying drawings fully illustrate embodiments of the present disclosure to enable those skilled in the art to practice them. Other embodiments may include structural, logical, electrical, procedural, and other changes. The embodiments represent only possible variations. Individual components and functions are optional unless explicitly required, and the order of operation may vary. Parts and features of some embodiments may be included or replace parts and features of other embodiments. The scope of the embodiments of this disclosure includes the entire scope of the claims and all available equivalents of the claims. While the terms "first," "second," etc., may be used in this application to describe elements, these elements should not be limited by these terms. These terms are used only to distinguish one element from another. For example, a first element may be called a second element without changing the meaning of the description, and similarly, a second element may be called a first element, provided that all occurrences of "first element" are consistently renamed and all occurrences of "second element" are consistently renamed. First and second elements are both elements, but may not be the same element. Moreover, the terminology used in this application is only for describing embodiments and is not intended to limit the claims. As used in the description of the embodiments and claims, unless the context clearly indicates otherwise, the singular forms "a," "an," and "the" are intended to equally include the plural forms. Similarly, the term "and / or" as used herein means including one or more of the associated listed items and all possible combinations thereof. Additionally, when used herein, the terms "comprise" and its variations "comprises" and / or "comprising" refer to the presence of stated features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof. Without further limitations, an element defined by the phrase "comprising an..." does not exclude the presence of additional identical elements in the process, method, or apparatus that includes said element. In this document, each embodiment may focus on the differences from other embodiments, and similar or identical parts between embodiments can be referred to mutually. For methods, products, etc., disclosed in the embodiments, if they correspond to the method section disclosed in the embodiments, the relevant parts can be referred to the description of the method section.

[0094] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this disclosure. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0095] The methods and products disclosed in the embodiments herein (including but not limited to devices and equipment) can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units may be merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to implement this embodiment according to actual needs. In addition, the functional units in the embodiments of this disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0096] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than that shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. In the descriptions corresponding to the flowcharts and block diagrams in the accompanying drawings, the operations or steps corresponding to different blocks may also occur in a different order than disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. Each block in a block diagram and / or flowchart, and combinations of blocks in a block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.< / ebmp> < / ebmp> < / ebmp>

Claims

1. A method for video playback, characterized in that, include: The system acquires the current multimodal physiological information of the user watching the current video and determines the current emotional intensity value corresponding to the current multimodal physiological information. The multimodal physiological information includes one or more of the following: skin conductance response (GSR), heart rate (HR), skin temperature, and respiratory rate. If the current emotion intensity value satisfies the current triggering condition corresponding to the current video narrative branch, the current playback parameters corresponding to the current video narrative branch are determined based on the current emotion intensity value. The current triggering condition is obtained by parsing the narrative branch markers embedded in the current video. Play the current video according to the current playback parameters.

2. The method according to claim 1, characterized in that, The determination of the current emotion intensity value corresponding to the current multimodal physiological information includes: The current multimodal physiological information is preprocessed, and the normalized value corresponding to each physiological information in the current multimodal physiological information is obtained based on the baseline calibration range. The baseline calibration range is the range of user physiological signals recorded when guiding users to complete four standard emotion induction tasks of calm, excitement, tension and relaxation. Based on each normalized value, a weighted fusion is performed according to formula (1) to obtain the current emotion intensity value E; E = 100×(w1×GSR_norm + w2×HR_norm + w3×Temp_norm + w4×RR_norm)(1)where: GSR_norm, HR_norm, Temp_norm, and RR_norm are the normalized values ​​corresponding to skin conductance response (GSR), heart rate (HR), skin temperature, and respiratory rate, respectively, and w1, w2, w3, and w4 are dynamic weighting coefficients, and the sum of the weighting coefficients is 1.

3. The method according to claim 1, characterized in that, Also includes: Based on the Emotion Branching Protocol (EBMP), one or more narrative branch markers corresponding to video narrative branches are embedded in the encapsulated metadata layer of the current video. The narrative branch markers include one or more of the following: triggering conditions, switching actions, and switching transition durations.

4. The method according to claim 1, characterized in that, The steps for determining the current playback parameters corresponding to the current video narrative branch include: If the current emotional intensity value E is greater than the first threshold, increase the gain value of the first range of audio; if the current emotional intensity value E is less than the second threshold, increase the gain value of the second range of audio. Here, the first threshold is greater than the second threshold, and the frequency value corresponding to the first range is greater than the frequency value corresponding to the second range. If the current emotional intensity value E is greater than the third threshold, the subtitles will be switched to a bright color scheme and the font size will be enlarged. If the current emotional intensity value E is less than the fourth threshold, the subtitles will be switched to a soft color scheme and the default font size will be restored. The third threshold is greater than the fourth threshold. If the current emotional intensity value E is greater than the fifth threshold, increase the volume of the background music; if the current emotional intensity value E is less than the sixth threshold, decrease the volume of the background music. The fifth threshold is greater than the sixth threshold.

5. The method according to claim 1, characterized in that, The step of playing the current video according to the current playback parameters includes: Within the transition duration corresponding to the current narrative branch marker, switch to a related narrative branch of the current narrative branch, keeping the main storyline of the current video unchanged; Within the transition duration corresponding to the current narrative branch marker, the audio, subtitle, and video parameters are continuously and gradually adjusted according to the current playback parameters.

6. The method according to claim 1, characterized in that, The acquisition of the current multimodal physiological information of the user watching the current video includes: In cases where multiple users are watching the current video, the user with the longest screen gaze duration is identified as the current user, and their current multimodal physiological information is obtained; or, In the case of multiple users watching the current video, the multimodal physiological information of each user is obtained, so that the current emotional intensity value is obtained by averaging and fusing the emotional intensity values ​​corresponding to each multimodal physiological information.

7. The method according to claims 1-6, characterized in that, Also includes: Based on the current emotional intensity value E, control is exercised over home environment equipment, which includes one or more of the following: lighting fixtures, curtains, air conditioners, and fresh air systems.

8. An apparatus for video playback, the apparatus comprising a processor and a memory storing program instructions, characterized in that, The processor is configured to perform the method for video playback as described in any one of claims 1 to 7 when executing the program instructions.

9. A device, characterized in that, include: Equipment body; The device for video playback as described in claim 8 is installed on the device body.

10. A storage medium storing program instructions, characterized in that, When the program instructions are executed, they perform the method for video playback as described in any one of claims 1 to 7.

Images