Multimedia playback system and multimedia playback query system

Abstract

This invention relates to the field of multimedia playback control technology, specifically to a multimedia playback system and a multimedia playback query system. A control input unit collects and stores user input data and environmental awareness data; a playback strategy decision unit, based on user input and environmental awareness data, and combined with an adaptive contextual playback optimization algorithm, formulates the optimal playback strategy in real time; a playback adjustment unit, based on the optimal playback strategy, dynamically adjusts the specific parameters of audio and video playback; and an interactive feedback unit provides real-time feedback to the user on playback status, playback progress, system performance, and environmental adaptability, and also feeds back user actions to the control input unit. This system intelligently adjusts the playback strategy based on the real-time environment, user behavior, and device status using an adaptive contextual playback optimization algorithm.

Description

Technical Field

[0001] This invention relates to the field of multimedia playback control technology, and more specifically, to a multimedia playback system and a multimedia playback query system. Background Technology

[0002] The multimedia playback system and multimedia playback query system aim to improve the smoothness, clarity and efficiency of the user's viewing experience by accurately capturing user input, sensing environmental changes in real time and analyzing device resource status, and combining intelligent decision-making algorithms to achieve personalized adjustment of playback strategies and resource optimization. Through adaptive contextual playback optimization algorithms, the system adjusts playback strategies and dynamically adjusts audio and video quality, playback mode and volume parameters under different network bandwidth, device resources and user activity states.

[0003] Existing multimedia playback systems often struggle to dynamically adjust playback strategies based on real-time environments and user needs. Furthermore, due to limited device resources, low network bandwidth, low battery power, and a lack of personalized user behavior modeling, user experience can fluctuate. Network fluctuations or device performance bottlenecks can cause playback interruptions, image quality degradation, and sound blurring. Therefore, a multimedia playback system and a multimedia playback query system are provided. Summary of the Invention

[0004] The purpose of this invention is to provide a multimedia playback system and a multimedia playback query system to solve the problems mentioned in the background art, which are caused by limited device resources, low network bandwidth, low battery power and lack of personalized user behavior modeling, resulting in fluctuating user experience, network fluctuations or device performance bottlenecks causing playback interruption, degraded picture quality and blurry sound.

[0005] To achieve the above objectives, in one aspect, the present invention aims to provide a multimedia playback system, comprising:

[0006] The control input unit is used to collect and store user input data sets and environmental perception data sets in real time;

[0007] The playback strategy decision unit is communicatively connected to the control input unit and is used to generate a dynamic playback strategy based on the user input data set and the environmental perception data set through an adaptive contextual playback optimization algorithm.

[0008] A playback adjustment unit is connected to the output of the playback strategy decision unit and is used to adjust the set of audio and video playback parameters in real time according to the dynamic playback strategy.

[0009] The interactive feedback unit is interactively connected to the control input unit and the playback adjustment unit respectively, generates feedback information including playback status, environmental adaptability indicators and system performance parameters, and feeds back the user interaction data to the control input unit in a closed loop.

[0010] As a further improvement to this technical solution, the control input unit includes:

[0011] The user input module is used to collect input including pause / play commands. Volume adjustment commands Mode switching command and audio preference commands User operation data flow;

[0012] The environment sensing module is used to collect data including network bandwidth. Network latency CPU load changes Device battery power User activity status and user location Environmental status data stream;

[0013] User operation data streams and environment status data streams are synchronously stored in the system cache using timestamps.

[0014] As a further improvement to this technical solution, the adaptive contextual playback optimization algorithm integrates a reinforcement learning framework, multimodal perception fusion technology, and a user behavior prediction model, specifically including:

[0015] Construct a decision network based on Q-learning, whose state space includes user input feature vectors and environment-aware feature vectors;

[0016] Design a reward mechanism that includes a playback quality scoring function and a resource consumption penalty function;

[0017] The user behavior pattern database is updated in real time through the online learning module.

[0018] As a further improvement to this technical solution, the playback strategy decision unit includes:

[0019] The data fusion module is used to perform feature-level fusion of user input data sets and environmental perception data sets, and to calculate a multi-dimensional data weight matrix.

[0020] The context analysis module, connected to the data fusion module, is used to extract the features of the current playback scene and construct an optimization problem model that includes network constraints, user activity status, and device resource limitations.

[0021] The strategy optimization module is used to solve a multi-objective optimization function to generate an optimal playback strategy that includes quality priority and resource allocation schemes.

[0022] As a further improvement to this technical solution, the data fusion module performs the following operations:

[0023] Calculate the weight of pause playback ,in To pause the frequency, The interval between adjacent pause operations;

[0024] Generate volume adjustment weights ,in This represents the change in volume. To adjust the time interval, For volume variance;

[0025] Determine mode switching weights ,in Select the number of times for the current mode. Total number of mode switches For frequency switching;

[0026] Constructing network quality factors , This represents the time delay impact coefficient.

[0027] As a further improvement to this technical solution, the scenario analysis module performs the following calculations:

[0028] Network Adaptability Indicators , This is the fill rate of the network buffer;

[0029] User activity adaptability indicators , For screen frame rate of change, Visual impact factor;

[0030] Spatial volume adaptability index , This refers to the user's location, specifically the encoded value of the user's location.

[0031] As a further improvement to this technical solution, the strategy optimization module generates a playback strategy through the following steps:

[0032] Constructing the user experience objective function , The impact of pause playback weight on user experience;

[0033] Define resource consumption function ;

[0034] Establish multi-objective optimization equations , , These are adjustable weight parameters;

[0035] A constrained stochastic gradient descent algorithm is used to solve for the Pareto optimal solution set, and the output includes decision parameters including resolution, bitrate, and playback mode.

[0036] As a further improvement to this technical solution, the playback adjustment unit includes:

[0037] The video processing module is used to dynamically adjust the video resolution level and H.264 / HEVC encoding parameters based on network adaptability indicators;

[0038] The audio processing module is used to implement multi-channel equalization adjustment and dynamic range compression based on spatial volume adaptability indicators;

[0039] The resource scheduling module is used to implement power consumption-performance balancing strategies based on the device's resource consumption, including adaptive frame rate adjustment and background process priority control.

[0040] As a further improvement to this technical solution, the interactive feedback unit includes:

[0041] An augmented reality display interface is used to show playback status, system performance, and current environmental adaptability;

[0042] The multimodal feedback channel integrates voice command recognition, gesture control, and eye-tracking modules to form a closed-loop control circuit.

[0043] The feedback interface includes configurable transparency levels and supports non-intrusive information overlay display. On the other hand, this invention provides a multimedia playback query system, including:

[0044] Memory, which stores the control program of the multimedia playback system;

[0045] The processor executes the control program to achieve:

[0046] Receive user query requests and parse semantic content;

[0047] Use environmental awareness data to generate a contextual model;

[0048] Simulations of multiple playback strategies are performed in parallel using distributed computing nodes;

[0049] The output is a visual decision tree containing the optimal playback scheme and its expected effect.

[0050] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0051] 1. The multimedia playback system and multimedia playback query system are based on an adaptive contextual playback optimization algorithm. Based on user input, environmental perception data and device status, the playback strategy is dynamically adjusted to ensure the best audio and video playback experience under various network environments, device performance status and user behavior patterns. Based on online training of reinforcement learning and multimodal perception technology, the system can perceive and respond to user operations, network bandwidth changes, device load and user activity status in real time to optimize playback mode and audio and video quality.

[0052] 2. In this multimedia playback system and multimedia playback query system, through the close collaboration of the data fusion module, the context analysis module and the strategy optimization module, the intelligent fusion of user behavior and environmental data is achieved, thereby optimizing the playback strategy and ensuring that a high-quality playback experience can still be provided even under conditions of low bandwidth or limited device resources. Attached Figure Description

[0053] Figure 1 This is an overall flowchart of the present invention;

[0054] The meanings of the labels in the diagram are as follows:

[0055] 1. Control input unit; 2. Playback strategy decision unit; 3. Playback adjustment unit; 4. Interactive feedback unit. Detailed Implementation

[0056] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention. Example 1:

[0057] Please see Figure 1 As shown, a multimedia playback system is provided, including:

[0058] Control input unit 1, the control input unit 1 is used to collect and store user input data set and environmental perception data set;

[0059] The control input unit 1 includes a user input module and an environment perception module;

[0060] The user input module is used to collect user input data, including pause playback commands. Volume adjustment commands Mode switching command and audio preference commands ;

[0061] In this embodiment, the pause playback command , Indicates playback; Indicates a pause;

[0062] Volume adjustment commands , Indicates the minimum volume; Indicates the maximum volume;

[0063] Mode switching command , Indicates a single track loop; Indicates random loop; Indicates a custom sequence;

[0064] Audio preference commands , Representation standard; Indicates surrounding; Indicates customization;

[0065] The environmental perception module collects environmental perception data, including network bandwidth. Network latency CPU load changes Device battery power User activity status and user location .

[0066] In this embodiment, network bandwidth Network latency User activity status , Indicates stillness; Indicates movement; user location , Indicates indoors; Indicates outdoors.

[0067] It also includes a playback strategy decision unit 2, which is communicatively connected to the control input unit 1. The playback strategy decision unit 2 generates a dynamic playback strategy based on the user input data set and the environmental perception data set through an adaptive context playback optimization algorithm.

[0068] The adaptive contextual playback optimization algorithm integrates a reinforcement learning framework, multimodal perception fusion technology, and user behavior prediction model. The adaptive contextual playback optimization algorithm adjusts the playback strategy in real time based on the user input data set and the environmental perception data set.

[0069] In this embodiment, reinforcement learning enables the system to continuously learn and optimize during user interaction through reward mechanisms and exploration strategies, and automatically adjust decision rules based on user actions and environmental changes.

[0070] Multimodal sensing technology integrates various sensing data to comprehensively understand user behavior, environmental changes, and device status information, avoiding the lack of personalized and differentiated playback strategies that may result from a single sensor and input method.

[0071] User behavior modeling technology builds personalized user behavior models by analyzing users' historical behavior data and preferences, providing customized playback strategies for each user, taking into account users' long-term preferences and immediate needs;

[0072] Adaptive optimization algorithms can find the optimal playback strategy settings under constraints by optimizing the search method based on the current state of the device.

[0073] By combining reinforcement learning, multimodal perception technology, user behavior modeling technology, and adaptive optimization algorithms, the adaptive contextual playback optimization algorithm can adjust playback strategies in real time in a changing and complex environment to provide the best playback experience. The adaptive contextual playback optimization algorithm finds a balance between real-time perception, intelligent decision-making, personalized customization, and resource optimization, providing users with a smooth, personalized, and efficient multimedia playback strategy.

[0074] The playback strategy decision unit 2 includes a data fusion module, a context analysis module, and a strategy optimization module;

[0075] The data fusion module described therein performs feature-level fusion of user input data set and environmental perception data set, quantifies and calculates the influence weight between data, and calculates a multi-dimensional data weight matrix;

[0076] The context analysis module is connected to the data fusion module. Based on the quantification results from the data fusion module, it extracts context features and constructs the current playback requirements and environmental constraints.

[0077] The strategy optimization module is used to combine the context features of the context analysis module to construct a playback strategy optimization objective function, maximize user experience and minimize resource consumption, and solve the multi-objective optimization function to generate the optimal playback strategy that includes quality priority and resource allocation scheme.

[0078] The data fusion module performs feature-level fusion of the user input data set and the environmental perception data set, quantifies and calculates the influence weights between the data, and calculates a multi-dimensional data weight matrix. The specific steps are as follows:

[0079] S2.1.1 Pause playback command The pause playback weight is determined by the frequency of pause playback commands and the switching time interval:

[0080] ;

[0081] in, Weight for pausing playback; The frequency of pause playback commands; The time interval between two pause playback commands;

[0082] S2.1.2, Volume Adjustment Command The volume control weights are determined by the volume change amount, the volume adjustment time interval, and the variance of the volume change:

[0083] ;

[0084] in, Weights are assigned to control volume. This represents the change in volume. This is the time interval for volume adjustment; To prevent small values ​​from being divided by zero; The variance of the volume change;

[0085] S2.1.3 Mode Switching Command The playback mode switching weight is determined by the number of times the current mode is selected, the frequency of mode switching, and the sum of all mode selections:

[0086] ;

[0087] in, Switch weights for playback modes; This represents the number of times the current mode has been selected. This refers to the frequency of mode switching; The sum of all playback mode selections; For the first One playback mode; This is the sequence number of the playback mode;

[0088] S2.1.4, network bandwidth Combined with network latency Calculate the network quality factor:

[0089] ;

[0090] in, This is the network quality factor.

[0091] The context analysis module extracts and calculates context features based on the quantification results from the data fusion module, and constructs the current playback requirements and environmental constraints. The specific steps are as follows:

[0092] S2.2.1, Based on network quality factors Calculate network condition adaptability:

[0093] ;

[0094] in, For network condition adaptability; This is the fill rate of the network buffer;

[0095] S2.2.2, Based on user activity status Calculate user activity adaptability based on screen display:

[0096] ;

[0097] in, Adaptability to user activities; The frame rate of change of the screen content; Weights for frame changes in activity adaptability; The specific value is determined through user experiments to test the impact of user activity on the playback experience at different frame rate changes, and then adjusted accordingly. The value;

[0098] S2.2.3, Combine volume control weights With user location Calculate volume adaptability:

[0099] ;

[0100] in, For volume adaptation.

[0101] The strategy optimization module is used to combine the context features of the context analysis module to construct a playback strategy optimization objective function, maximize user experience and minimize resource consumption, and optimize the final playback strategy. The specific steps are as follows:

[0102] S2.3.1, Adaptability based on network conditions User activity adaptability And pause playback weight Calculate user experience score:

[0103] ;

[0104] in, Rate the user experience; The impact of pause playback weight on user experience;

[0105] In this embodiment, if If so, the pause playback weight will have a moderate non-linear impact on the user experience score;

[0106] like The impact of playback will be relatively smooth and will not significantly affect the rating.

[0107] like Frequent pauses in playback will significantly lower the user experience score;

[0108] S2.3.2, Based on network bandwidth CPU load changes and device battery power Computational resource consumption:

[0109] ;

[0110] in, For resource consumption;

[0111] S2.3.3, Based on user experience rating and resource consumption Construct a playback strategy optimization objective function, maximize user experience score and minimize resource consumption, and optimize the final playback strategy:

[0112] ;

[0113] in, Weighting coefficients for user experience rating; This represents the weighting coefficient for resource consumption.

[0114] In this embodiment, Controlling user experience scores, the system needs to prioritize user experience when network and device resources are strained. The value will be relatively large; Resource consumption is constrained under resource-limited conditions, such as low bandwidth, low battery power, or high CPU load. The value needs to be increased;

[0115] S2.3.4 Solve the objective function for optimizing the playback strategy by gradient descent, update the playback strategy parameters, and obtain the optimal parameters and the optimal playback strategy.

[0116] It also includes a playback adjustment unit 3, which dynamically adjusts the specific parameters of audio and video playback based on the optimal playback strategy;

[0117] The playback adjustment unit 3 includes a video processing module, an audio processing module, and a resource scheduling module;

[0118] The video processing module adapts to network conditions based on the following indicators. and resource consumption Dynamically adjust video resolution levels and H.264 / HEVC encoding parameters to adapt to network conditions and reduce buffering; high bandwidth: increase video resolution; low bandwidth: decrease resolution.

[0119] The audio processing module adjusts the volume level according to the user's environment and activity status, and adapts to the user's activity. and pause playback command Dynamically adjust playback mode;

[0120] If user activity is low, the volume can be increased appropriately to enhance audio clarity; if user activity is high, the volume should be automatically reduced to avoid audio interference.

[0121] If a user frequently pauses playback, the system can automatically pause or provide a low-bandwidth playback mode; if the system detects that a user has been inactive for an extended period, it can automatically pause playback.

[0122] The resource scheduling module is based on the resource consumption of the equipment. Dynamically adjust playback settings.

[0123] When the device's battery is low or the CPU load is high, the system will automatically reduce the resolution and volume; if the device has sufficient resources, the system will enhance the playback quality and provide the best resolution and volume.

[0124] It also includes an interactive feedback unit 4, which provides real-time feedback to the user on playback status, playback progress, system performance, and environmental adaptability, and feeds back the user's actions to the control input unit 1;

[0125] The interactive feedback unit 4 includes an augmented reality display interface and a multimodal feedback channel;

[0126] The augmented reality display interface is used to provide feedback to the user on the playback status, system performance, and current environmental adaptability through the interactive interface.

[0127] Display playback status: Show the current status of video playback, including playback progress, buffering status, and playback time. Display the current playback time, remaining time, and playback progress bar on the video playback interface.

[0128] System resource monitoring information: Displays network bandwidth, CPU load, device battery level, and video resolution information. If the user's network bandwidth is low, the system will remind the user of poor network conditions and display the current network bandwidth and network latency.

[0129] Dynamic quality feedback: Displays changes in video quality when there are network fluctuations or device resource constraints. When the device's CPU load is too high or the device's battery power is low, the user interface will display data such as remaining battery time and CPU utilization.

[0130] The multimodal feedback channel is used to receive real-time user feedback commands and feed them back to the control input unit 1. It integrates voice command recognition, gesture control, and eye-tracking modules to form a closed-loop control circuit; the feedback interface includes configurable transparency levels and supports non-intrusive information overlay display. Example 2:

[0131] This embodiment provides a multimedia playback query system, including: a memory for storing a control program for the multimedia playback system; and a processor for executing the control program to achieve the following:

[0132] Receive user query requests and parse semantic content;

[0133] Use environmental awareness data to generate a contextual model;

[0134] Simulations of multiple playback strategies are performed in parallel using distributed computing nodes;

[0135] The output is a visual decision tree containing the optimal playback scheme and its expected effect.

[0136] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention.

Claims

1. A multimedia playback system, characterized in that: include: The control input unit (1) is used to collect and store user input data sets and environmental perception data sets in real time; The playback strategy decision unit (2) is communicatively connected to the control input unit (1) and is used to generate a dynamic playback strategy based on the user input data set and the environmental perception data set through an adaptive contextual playback optimization algorithm. The playback strategy decision unit (2) further includes: The data fusion module is used to perform feature-level fusion of user input data sets and environmental perception data sets, and to calculate a multi-dimensional data weight matrix. The context analysis module, connected to the data fusion module, is used to extract the features of the current playback scene and construct an optimization problem model that includes network constraints, user activity status, and device resource limitations. The strategy optimization module is used to solve a multi-objective optimization function to generate an optimal playback strategy that includes quality priority and resource allocation schemes; The context analysis module performs the following calculations: Network Adaptability Indicators , This is the fill rate of the network buffer. Network quality factor; User activity adaptability indicators , For screen frame rate of change, As a visual impact factor, Indicates the user's activity status; Spatial volume adaptability index , For user location, Weight the volume adjustment; The strategy optimization module generates a playback strategy through the following steps: Constructing the user experience objective function , The impact of pause playback weight on user experience Weight for pausing playback; Define resource consumption function , Indicates network bandwidth. This indicates changes in CPU load. Indicates the device's battery level; Establish multi-objective optimization equations , , These are adjustable weight parameters; The Pareto optimal solution set is solved using a constrained stochastic gradient descent algorithm, and the output includes decision parameters including resolution, bit rate, and playback mode. The adaptive contextual playback optimization algorithm integrates a reinforcement learning framework, multimodal perception fusion technology, and a user behavior prediction model, specifically including: Construct a decision network based on Q-learning, whose state space includes user input feature vectors and environment-aware feature vectors; Design a reward mechanism that includes a playback quality scoring function and a resource consumption penalty function; The user behavior pattern database is updated in real time through the online learning module; The playback adjustment unit (3) is connected to the output of the playback strategy decision unit and is used to adjust the audio and video playback parameter set in real time according to the dynamic playback strategy. The interactive feedback unit (4) is interactively connected to the control input unit (1) and the playback adjustment unit (3) respectively, generates feedback information including playback status, environmental adaptability indicators and system performance parameters, and feeds back the user interaction data to the control input unit (1) in a closed loop.

2. The multimedia playback system according to claim 1, characterized in that: The control input unit (1) includes: The user input module is used to collect input including pause / play commands. Volume adjustment commands Mode switching command and audio preference commands User operation data flow; The environment sensing module is used to collect data including network bandwidth. Network latency CPU load changes Device battery power User activity status and user location Environmental status data stream; User operation data streams and environment status data streams are synchronously stored in the system cache using timestamps.

3. The multimedia playback system according to claim 1, characterized in that: The data fusion module performs the following operations: Calculate the weight of pause playback ,in To pause the frequency, The interval between adjacent pause operations; Generate volume adjustment weights ,in This represents the change in volume. To adjust the time interval, For volume variance; Determine mode switching weights ,in Select the number of times for the current mode. Total number of mode switches For frequency switching; Constructing network quality factors , This is the time delay impact coefficient. Indicates network bandwidth. This indicates network latency.

4. The multimedia playback system according to claim 1, characterized in that: The playback adjustment unit (3) includes: The video processing module is used to dynamically adjust the video resolution level and H.264 / HEVC encoding parameters based on network adaptability indicators; The audio processing module is used to implement multi-channel equalization and dynamic range compression based on spatial volume adaptability indicators; The resource scheduling module is used to implement power consumption-performance balancing strategies based on the device's resource consumption, including adaptive frame rate adjustment and background process priority control.

5. The multimedia playback system according to claim 1, characterized in that: The interactive feedback unit (4) includes: An augmented reality display interface is used to show playback status, system performance, and current environmental adaptability; The multimodal feedback channel integrates voice command recognition, gesture control, and eye-tracking modules to form a closed-loop control circuit. The feedback interface includes configurable transparency levels and supports non-intrusive information overlay display.

6. A multimedia playback query system, applied to the multimedia playback system according to any one of claims 1-5, characterized in that, include: Memory, used to store control programs; The processor executes the control program to achieve: Receive user query requests and parse semantic content; Use environmental awareness data to generate a contextual model; Simulations of multiple playback strategies are performed in parallel using distributed computing nodes; The output is a visual decision tree containing the optimal playback scheme and its expected effect.

Images