Shared decoding capability performance scheduling system and method suitable for low-performance terminal

Abstract

The invention discloses a sharing decoding capability performance scheduling system and method suitable for a low-performance terminal, relates to the technical field of monitoring, and solves the problems of multi-channel video playing, multi-picture projection screen playing, multi-screen video polling, performance bottleneck of a sharing capability terminal, and difficulty in effective processing of audio control and synchronization and operation interaction experience in a low-performance PC (Personal Computer) environment. The system comprises a sharing terminal, a service platform, a sharing platform, a stability guarantee module, a client and a local video operation optimization module. And the sharing terminal receives the configuration information to execute a decoding task, and provides a decoding capability for the client through the sharing platform. And the stability guarantee module monitors the state of the decoding process and restarts the decoding task when the decoding process is abnormal. And the client communicates with the shared terminal to transmit an operation instruction, and the local video operation optimization module acquires window split screen information to realize mouse position identification and service information display. The method has a good application prospect in the fields of mobile video playing, remote monitoring and the like.

Description

Technical Field

[0001] The present invention relates to the field of monitoring technology, and in particular to a shared decoding capability performance scheduling system suitable for low-performance terminals. Background Art

[0002] In the field of live video and multi-screen presentation, with the increase of network bandwidth and the development of video encoding technology, more and more users want to watch multiple video streams at the same time. However, in a low-performance PC environment, the need to process multiple video streams may cause system resource overload, decoding difficulties and other problems, affecting the user's viewing experience.

[0003] At present, the patent with application number CN202211324213.0 uses a confluence server to synthesize multiple video streams. Although it can reduce the decoding burden of the client, when the confluence server handles multiple confluence tasks, performance may become a bottleneck, resulting in delays, freezes or frame losses; at the same time, it lacks independent control functions for each video audio and real-time interactive operation support. The patent with application number 1CN202311179972.7 provides a method for high-performance hardware decoding and rendering, but its scope of application is relatively limited and may not be implemented in a low-performance PC environment. The patent with application number CN202310028674.1 provides a method for presenting multiple screens, which reduces server resource usage, but there are still limitations on audio control issues and the inability to be applied to the monitoring field in complex scenarios.

[0004] In summary, the current technology still has shortcomings in dealing with multiple video streams, audio control, real-time interaction, etc. in a low-performance PC environment. Further improvements are needed to enhance the stability of the system and user experience. Summary of the invention

[0005] In order to solve the technical problems that multi-channel video playback, multi-screen projection playback, multi-screen video patrol, shared capability terminal performance bottleneck, audio control and synchronization and operation interaction experience are difficult to effectively handle in a low-performance PC environment, the present invention provides the following solutions: a shared decoding capability performance scheduling system suitable for low-performance terminals, the system comprising: a shared terminal, a business platform, a shared platform, a stability guarantee module, a client and a local video operation optimization module;

[0006] The shared terminal is used to receive the configuration information provided by the service platform and perform the decoding task according to the configuration information;

[0007] The shared terminal is further used to provide decoding capability to the client through the shared platform and transmit a shared code stream;

[0008] The stability guarantee module is used to receive the decoding process status information sent by the shared terminal and restart the decoding task in an abnormal situation;

[0009] The client is used to communicate with the shared terminal through the service platform to convey operation instructions;

[0010] The local video operation optimization module is used to obtain the window split screen information through the shared terminal to identify the mouse position and display related business information.

[0011] Furthermore, the client is also used to enable a multi-channel sharing mode and apply for multiple sharing channels from the service platform;

[0012] The service platform is further used to provide multiple shared channel numbers according to the application and synchronize them to the shared terminal;

[0013] The sharing terminal is further used to initiate a request to join a sharing channel to the sharing platform according to the sharing channel number;

[0014] The sharing platform is also used to provide a sharing channel to the sharing terminal.

[0015] Furthermore, the service platform is also used to reorganize the original patrol group into multiple new video groups, and establish a corresponding shared channel for each new video group;

[0016] The client is also used to create a playback window equal to the number of shared channels.

[0017] Furthermore, the shared terminal is also used to adjust the number of split screens or to turn on and off a specific video stream;

[0018] The sharing platform is also used to capture the process window screen and audio, and display them on the client.

[0019] Furthermore, the sharing terminal is also used to accept screenshot or video recording instructions initiated by the client through the sharing platform, create a background player for stream decoding according to the screenshot or video recording instructions, and enable the client to display real-time stream data through the business platform.

[0020] A method for scheduling shared decoding capability performance for low-performance terminals, using a system for scheduling shared decoding capability performance for low-performance terminals as described above, the method comprising:

[0021] S1. Set up a binding relationship between client users and shared terminals on the business platform, allowing multiple users to bind to the same shared terminal; this many-to-one relationship ensures effective use and management of resources;

[0022] S2. After the user logs in to the client, he obtains the list of associated playback devices and starts the sharing mode. The business platform records the opening status of the sharing mode to provide accurate information for subsequent operations.

[0023] S3. When the sharing mode is turned on, the client requests the sharing platform to create a shared channel and obtain the channel number. The business platform synchronizes the channel number to the shared terminal to ensure smooth communication.

[0024] S4. After receiving the channel number notification from the service platform, the shared terminal automatically starts the corresponding decoding process and joins the shared channel. The client receives the notification that the shared terminal has successfully joined the channel, confirming that the connection has been successfully established.

[0025] S5. Once the shared channel is successfully joined, the shared terminal starts to capture the picture and audio of the decoding process and pushes it as a shared code stream to the sharing platform. The client can view the corresponding video picture and audio content in real time.

[0026] S5. To ensure the stable operation of the decoding process, the stability assurance module runs the watchdog service and monitors and recovers abnormally exited decoding tasks in a timely manner according to the channel number to ensure the continuous push of stable shared code streams;

[0027] S7. During the decoding process, the client needs to synchronize the screenshot and recording operations with the real-time playback screen. The client periodically requests the business platform for the timestamp and playback status corresponding to the video screen of the current playback device. After receiving the corresponding data, it links with the business platform and ensures smooth data transmission to achieve synchronous interaction between operations and playback screen.

[0028] The beneficial effects of the present invention are:

[0029] 1. By sharing decoding capabilities, it is possible to play multiple videos on low-performance terminals, significantly improving the user viewing experience.

[0030] 2. The stability assurance module ensures the stable operation of decoding tasks and automatic restart under abnormal conditions, enhancing the reliability of the system.

[0031] 3. Reorganize the video patrol groups and create independent shared channels through the business platform, effectively optimizing the performance and smoothness of multi-screen video patrol tasks.

[0032] 4. The audio control and synchronization issues have been resolved, so that each audio stream can adjust the volume independently, improving the audio playback quality.

[0033] 5. The interactive experience optimization module identifies the mouse position and displays relevant business information, improving the user's interactive experience and convenience for multi-channel video operations.

[0034] 6. The process window screen and audio are captured through the sharing platform and displayed on the client, so that users can efficiently obtain real-time streaming data. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 is a schematic diagram of the system of the present invention;

[0036] Figure 2 A logical timing diagram of the system of the present invention when sharing decoding capability;

[0037] Figure 3 It is a schematic diagram of the combination of multiple shared code streams;

[0038] Figure 4 This is a schematic diagram of video playback in multi-window synchronous playback mode;

[0039] Figure 5 This is a schematic diagram of video playback in single-window rotation playback mode;

[0040] Figure 6 It is a logical timing diagram of the system of the present invention when the shared decoding terminal interacts with each other;

[0041] Figure 7 It is a schematic diagram of local operation interaction;

[0042] Figure 8 This is a flow chart of the video screenshot and recording functions;

[0043] Fig. 9 It is a logical timing diagram of the system of the present invention when multiple shared terminal performances are scheduled;

[0044] Fig.10 Flowchart for task prioritization and resource allocation;

[0045] Fig.11 Flowchart for task scheduling and resource allocation;

[0046] Fig.12 Adjust the diagram for task resource allocation. DETAILED DESCRIPTION

[0047] In order to make the technical solutions and advantages of the embodiments of the present invention more clearly understood, the exemplary embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than an exhaustive list of all the embodiments. It should be noted that the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict.

[0048] Example: Combination Figure 1 To illustrate this embodiment,

[0049] A shared decoding capability performance scheduling system applicable to low-performance terminals, the system comprising: a shared terminal, a service platform, a shared platform, a stability guarantee module, a client, and a local video operation optimization module;

[0050] The shared terminal is used to receive the configuration information provided by the service platform and perform the decoding task according to the configuration information;

[0051] The shared terminal is further used to provide decoding capability to the client through the shared platform and transmit a shared code stream;

[0052] The stability guarantee module is used to receive the decoding process status information sent by the shared terminal and restart the decoding task in an abnormal situation;

[0053] The client is used to communicate with the shared terminal through the service platform to convey operation instructions;

[0054] The local video operation optimization module is used to obtain the window split screen information through the shared terminal to identify the mouse position and display related business information.

[0055] Specifically, terminals with high decoding capabilities are selected as shared terminals, such as high-configuration computers, cloud computers or servers, and their decoding capabilities can be evaluated individually; if the shared terminal is composed of multiple terminals, one of the hosts is responsible for performance scheduling, and the remaining terminals act as slaves to ensure reasonable allocation of resources; at the same time, the terminal must have independent decoding, shared decoding capabilities and synchronous sharing instruction functions, supporting up to 64 videos being decoded simultaneously and up to 16 tasks being executed.

[0056] The client is also used to enable a multi-channel sharing mode and apply for multiple sharing channels from the service platform;

[0057] The service platform is further used to provide multiple shared channel numbers according to the application and synchronize them to the shared terminal;

[0058] The sharing terminal is further used to initiate a request to join a sharing channel to the sharing platform according to the sharing channel number;

[0059] The sharing platform is also used to provide a sharing channel to the sharing terminal.

[0060] Specifically, through Figure 3It can be seen that this embodiment proposes a sharing mechanism based on a combination of multiple shared code streams, allowing the client to connect to multiple shared channels, each channel provided by a shared terminal. Through this mechanism, the client can simultaneously display the video streams provided by multiple shared terminals with lower decoding resource consumption. For example, through a 2*2 split-screen layout, video streams from 4 shared terminals can be displayed, achieving the effect of displaying 256 videos. The overall process includes the client turning on the multi-channel sharing mode, the business platform allocating multiple shared channels, the shared terminal pulling up the decoding process and pushing the shared code stream, and the client rendering the code streams of different channels on the split-screen window to achieve multi-channel shared code stream combination playback.

[0061] The service platform is also used to reorganize the original patrol group into multiple new video groups and establish a corresponding shared channel for each new video group;

[0062] The client is also used to create a playback window equal to the number of shared channels.

[0063] Specifically, this embodiment implements multi-screen video patrol based on a combination of multiple shared code streams. By reorganizing the patrol groups and setting up a dedicated shared channel for each new group, two playback modes are implemented on the playback terminal: multi-window synchronous playback mode and single-window rotation playback mode. The former creates multiple playback windows corresponding to the shared channels, realizing instant screen switching to enhance the user's viewing experience; the latter uses the same playback window to play code streams of different channels in turn, effectively saving decoding resource consumption.

[0064] Figure 4 The operation of rearranging a patrol group containing 256 videos into 4 groups is demonstrated. Each group corresponds to an independent playback window, and 4 shared channels are created. Each channel is responsible for playing the video content in the corresponding group device list and outputting the shared streams of these videos to the corresponding playback window. When it is necessary to switch to the next screen video, the client only needs to display the next playback window in sequence, thus achieving a fast and smooth video patrol experience.

[0065] and Figure 5 The video playback principle in the single-window rotation playback mode is demonstrated. In this mode, the system creates a unique playback window and plays the shared code streams corresponding to the video groups from different shared channels in turn through this window. In this mode, although there is a loading process when switching to the next screen, the consumption of decoding resources is reduced from multiple to single, which helps save system resources. The shared terminal is also used to adjust the number of split screens or turn on and off specific video streams;

[0066] The shared terminal is also used to adjust the number of split screens or to turn on and off specific video streams;

[0067] The sharing platform is also used to capture the process window screen and audio, and display them on the client.

[0068] Specifically, in the context of shared decoding capabilities, this embodiment designs a real-time signaling transmission strategy to meet the high-frequency interaction requirements of users who need to operate devices in the split screen. Figure 6 It can be seen that the specific plan includes: users interact with the video screen, such as adjusting the number of split screens or turning on / off a specific video stream; these operations are transmitted to the corresponding shared terminal through the business platform; the shared terminal then performs the corresponding operations, such as adjusting the number of split screens or turning on / off a specific video stream; after the operation is completed, the view window information is immediately updated, and the picture in the shared code stream is automatically synchronized with the decoding process; the interactive operation also includes independently adjusting the volume of each audio stream to achieve precise control and synchronization; finally, the latest shared code stream data displayed by the client is consistent with the user interaction effect.

[0069] The sharing terminal is also used to accept the screenshot or video recording instructions initiated by the client through the sharing platform, create a background player for stream decoding according to the screenshot or video recording instructions, and enable the client to display real-time stream data through the business platform.

[0070] Specifically, this embodiment implements local operation interaction of multiple videos, aiming to handle the needs of users to operate devices in the split screen in a shared decoding environment. It includes two parts:

[0071] First, for operations that need to be completed locally, such as viewing device details, popping up right-click menus, settings, etc., users can achieve them by clicking on a specific location in the playback window. The system saves the split-screen configuration data and records the rectangular area coordinates and bound device ID corresponding to each split-screen in memory to determine the device selected by the user. These local operations are controlled by sending network requests through the platform, including device PTZ control, intercom switch, focus adjustment, etc.

[0072] pass Figure 7 It can be seen that in the playback window scenario under the four-split screen, users can view the corresponding device details by clicking a specific location, and send a request to the platform through the client to control and set operations on the device.

[0073] Another type of operation involves the screenshot and recording function during video playback. The specific process is as follows: Figure 8 As shown:

[0074] When the user takes a screenshot or records a video, first determine whether the shared mode function is used. If so, call the player directly to perform the operation; if the shared mode is not used, the subsequent steps are determined based on whether the selected window is bound to the device. The system will create a background player that only pulls the stream and decodes it without rendering. When the user triggers a screenshot or record, the corresponding operation is performed and saved locally. To maintain synchronization, the system will request the timestamp and status of the video screen of the current playback device from the business platform, and respond according to different timestamps. This ensures the accuracy and timeliness of screenshots and recordings.

[0075] like Figure 2 As shown, a method for scheduling shared decoding capability performance for low-performance terminals adopts a system for scheduling shared decoding capability performance for low-performance terminals as described above, and the method includes:

[0076] S1. Set up a binding relationship between client users and shared terminals on the business platform, allowing multiple users to bind to the same shared terminal; this many-to-one relationship ensures effective use and management of resources;

[0077] S2. After the user logs in to the client, he obtains the list of associated playback devices and starts the sharing mode. The business platform records the opening status of the sharing mode to provide accurate information for subsequent operations.

[0078] S3. When the sharing mode is turned on, the client requests the sharing platform to create a shared channel and obtain the channel number. The business platform synchronizes the channel number to the shared terminal to ensure smooth communication.

[0079] S4. After receiving the channel number notification from the service platform, the shared terminal automatically starts the corresponding decoding process and joins the shared channel. The client receives the notification that the shared terminal has successfully joined the channel, confirming that the connection has been successfully established.

[0080] S5. Once the shared channel is successfully joined, the shared terminal starts to capture the picture and audio of the decoding process and pushes it as a shared code stream to the sharing platform. The client can view the corresponding video picture and audio content in real time.

[0081] S5. To ensure the stable operation of the decoding process, the stability assurance module runs the watchdog service and monitors and recovers abnormally exited decoding tasks in a timely manner according to the channel number to ensure the continuous push of stable shared code streams;

[0082] S7. During the decoding process, the client needs to synchronize the screenshot and recording operations with the real-time playback screen. The client periodically requests the business platform for the timestamp and playback status corresponding to the video screen of the current playback device. After receiving the corresponding data, it links with the business platform and ensures smooth data transmission to achieve synchronous interaction between operations and playback screen.

[0083] Example 2: Combination Figure 9-12 In order to effectively utilize limited terminal resources, improve the utilization rate of high-performance terminal equipment, reduce resource waste, and ensure efficient execution of tasks, this embodiment proposes a performance scheduling scheme for multiple shared terminals, which is executed by the system described in Example 1. Fig. 9 It can be seen that the overall timing logic of the scheme is:

[0084] S1. Each shared terminal configured and bound to the business platform, including the master and slave, needs to upload performance indicator information to the business platform in real time. This information includes the overall performance indicator and the resource usage of each shared process, so as to analyze the user's usage;

[0085] S2. The host obtains the performance indicator information of all slaves from the business platform, including the performance indicator of each shared task on each slave, which helps the host to allocate resources and schedule tasks;

[0086] S3: When there is only a single host and no slaves, resources are allocated directly based on the overall performance of the host without the need for slave scheduling.

[0087] S4. If multiple slaves are deployed, dynamic scheduling and task priority strategies are required. The host will select a slave with a lower load and send the information of the slave to the business platform. Subsequently, the business platform will push a message to the selected slave to join the shared channel. Once the slave joins the shared channel, decoding sharing can begin;

[0088] S5. Subsequently, the service platform sends a message to the corresponding slave to join the shared channel again, so that the slave can join the shared channel and start decoding sharing.

[0089] The specific scheduling process is as follows: Fig.10 As shown, through Fig.10 It can be seen that:

[0090] S1. When the business platform receives a shared task, it first notifies the host of the task. The host is responsible for managing all slave information and shared task information.

[0091] S2. After determining whether the task issued is a high-priority task, the host will assign a corresponding weight to it and enter it into the scheduling queue according to the weight.

[0092] S3. For those tasks that do not occupy much CPU and have a short usage time after each use by the user, a weight of 2 will be assigned and they will be added to the scheduling queue according to the weight.

[0093] S3. For other tasks, a standard weight of 3 is uniformly assigned and they are added to the scheduling queue according to the weight.

[0094] S4. When scheduling, sort the tasks according to their weights. Each task has a basic resource allocation value P, and the resources allocated to each task are P plus additional resources (such as P1).

[0095] How to select the appropriate slave for task allocation and scheduling based on task weight and resource conditions during task scheduling Fig.11 ,pass Fig.11 It can be seen that:

[0096] S1. The system allocates corresponding reserved resources to each task according to its weight;

[0097] S2. The host considers the performance indicators of the slave and the performance upper limit of the task being executed, and traverses all slaves to find a slave with matching idle resources;

[0098] S3. Once a qualified slave is found, the host will schedule tasks to the slave and send the slave information to the business platform so that the shared tasks can be distributed to the slave;

[0099] S4. If no idle resources matching the reserved resource size are found, it is necessary to identify the slave machine with the largest idle resources and adjust the tasks on the slave machine according to its allocation weight.

[0100] If certain circumstances arise, the allocation weight of each task needs to be adjusted. However, if the currently executed task has occupied more resources than the basic weight, it means that the resource demand of the task is high, and there is no need to adjust the allocation weight of the task. Fig.12 In the example, the current resource usage of task 2 has exceeded the basic weight, so there is no need to adjust its allocation weight 2. On the contrary, reasonable resource allocation can only be achieved by adjusting the allocation weight 1 of task 1.

[0101] In addition, when facing problems such as limited remote resources or users are unwilling to open shared services, you can consider adopting a solution of locally deployed shared services. Users can independently configure local high-performance terminals as decoding service centers to provide services for other terminals and implement management and control functions. This solution can reduce network latency, improve service quality and response speed, meet users' needs for high-performance decoding services, and enhance the overall service experience.

Claims

1. A shared decoding capability performance scheduling system suitable for low-performance terminals, characterized in that: The system includes: a shared terminal, a business platform, a shared platform, a stability guarantee module, a client, and a local video operation optimization module; The shared terminal is used to receive the configuration information provided by the service platform and perform the decoding task according to the configuration information; The shared terminal is further used to provide decoding capability to the client through the shared platform and transmit a shared code stream; The stability guarantee module is used to receive the decoding process status information sent by the shared terminal and restart the decoding task in an abnormal situation; The client is used to communicate with the shared terminal through the service platform to convey operation instructions; The local video operation optimization module is used to obtain the window split screen information through the shared terminal to identify the mouse position and display related business information.

2. A shared decoding capability performance scheduling system applicable to low-performance terminals according to claim 1, characterized in that: The client is also used to enable a multi-channel sharing mode and apply for multiple sharing channels from the service platform; The service platform is further used to provide multiple shared channel numbers according to the application and synchronize them to the shared terminal; The sharing terminal is further used to initiate a request to join a sharing channel to the sharing platform according to the sharing channel number; The sharing platform is also used to provide a sharing channel to the sharing terminal.

3. A shared decoding capability performance scheduling system applicable to low-performance terminals according to claim 1, characterized in that: The service platform is also used to reorganize the original patrol group into multiple new video groups and establish a corresponding shared channel for each new video group; The client is also used to create a playback window equal to the number of shared channels.

4. The shared decoding capability performance scheduling system applicable to low-performance terminals according to claim 1, characterized in that: The shared terminal is also used to adjust the number of split screens or to turn on and off specific video streams; The sharing platform is also used to capture the process window screen and audio, and display them on the client.

5. The shared decoding capability performance scheduling system applicable to low-performance terminals according to claim 1, characterized in that: The sharing terminal is also used to accept the screenshot or video recording instructions initiated by the client through the sharing platform, create a background player for stream decoding according to the screenshot or video recording instructions, and enable the client to display real-time stream data through the business platform.

6. A method for scheduling shared decoding capability performance for low-performance terminals, using a system for scheduling shared decoding capability performance for low-performance terminals as claimed in any one of claims 1 to 5, characterized in that: The method comprises: S1. Set up a binding relationship between client users and shared terminals on the business platform, allowing multiple users to bind to the same shared terminal; this many-to-one relationship ensures effective use and management of resources; S2. After the user logs in to the client, he obtains the list of associated playback devices and starts the sharing mode. The business platform records the opening status of the sharing mode to provide accurate information for subsequent operations. S3. When the sharing mode is turned on, the client requests the sharing platform to create a shared channel and obtain the channel number. The business platform synchronizes the channel number to the shared terminal to ensure smooth communication. S4. After receiving the channel number notification from the service platform, the shared terminal automatically starts the corresponding decoding process and joins the shared channel. The client receives the notification that the shared terminal has successfully joined the channel, confirming that the connection has been successfully established. S5. Once the shared channel is successfully joined, the shared terminal starts to capture the picture and audio of the decoding process and pushes it as a shared code stream to the sharing platform. The client can view the corresponding video picture and audio content in real time. S5. To ensure the stable operation of the decoding process, the stability assurance module runs the watchdog service and monitors and recovers abnormally exited decoding tasks in a timely manner according to the channel number to ensure the continuous push of stable shared code streams; S7. During the decoding process, the client needs to synchronize the screenshot and recording operations with the real-time playback screen. The client periodically requests the business platform for the timestamp and playback status corresponding to the video screen of the current playback device. After receiving the corresponding data, it links with the business platform and ensures smooth data transmission to achieve synchronous interaction between operations and playback screen.

Images