Performance scheduling method, system and equipment based on shared decoding capability and medium

Abstract

The invention provides a performance scheduling method, system and equipment based on shared decoding capability, and a medium. The method comprises the following steps: receiving a shared task sent by a service platform; sending an index information request to a service platform; the index information request is used for indicating the service platform to return real-time performance index information of all decoding nodes; according to the priority of the shared task and / or the historical resource occupancy rate, distributing a weight for the shared task; determining a target node matched with the weight according to the real-time performance index information of all decoding nodes; sending the target node identifier to a service platform, so that the service platform sends the sharing task to the target node; and the target node is used for creating a decoding process according to the sharing task, capturing a window picture and an audio of the decoding process, and pushing the window picture and the audio to the sharing platform as a sharing code stream, so that the sharing platform sends the sharing code stream to the client. Important tasks can be ensured to be responded preferentially through task priority ranking, and efficient execution of the tasks is ensured.

Description

Technical Field

[0001] This invention relates to the field of streaming media technology, and in particular to a performance scheduling method, system, device, and medium based on shared decoding capabilities. Background Technology

[0002] Split-screen playback is a function that allows users to simultaneously display and operate multiple video interfaces on the screen of an electronic device (such as a computer, mobile phone, or tablet). This requires the hardware of the electronic device to decode and play each video stream separately. However, most users currently use terminals with relatively poor CPU and GPU performance, such as set-top boxes, which can connect to large screens but have poor performance themselves. These devices are difficult to play multiple high-definition videos simultaneously.

[0003] In existing technologies, high-performance terminals are used as shared decoding service centers. When users need to play multiple video streams simultaneously, the shared decoding service center acquires multiple video streams and performs multi-channel decoding to obtain multiple bitstreams. These multiple bitstreams are then combined into a single bitstream for the user to view, which can solve the problems of insufficient memory and bandwidth in low-performance terminals.

[0004] However, as the scale of business expands, for example in the field of video surveillance, there are often scenarios where dozens or even hundreds of video feeds are played simultaneously. This requires configuring multiple high-performance terminals as decoding service centers. How to effectively utilize the equipment resources of the decoding service centers, reduce resource waste, and ensure efficient task execution has become a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0005] This invention provides a performance scheduling method, system, device, and medium based on shared decoding capabilities to solve the problem of how to effectively utilize device resources when facing large-scale decoding tasks in the prior art, and to achieve efficient resource utilization.

[0006] This invention provides a performance scheduling method based on shared decoding capabilities, applied to a master node, comprising the following steps.

[0007] Receive a shared task sent by the business platform; the shared task is a shared task sent by the client to the business platform. Send an indicator information request to the business platform; the indicator information request is used to instruct the business platform to return real-time performance indicator information of all decoding nodes; Based on the priority of the shared task and / or the historical resource utilization rate, a weight is assigned to the shared task; Based on the real-time performance metrics of all the decoding nodes, determine the target node that matches the weight; The target node sends its target node identifier to the business platform, so that the business platform sends the shared task to the target node based on the target node identifier; the target node is used to create a decoding process according to the shared task, capture the window screen and audio of the decoding process, and push the window screen and audio of the decoding process as a shared bitstream to the sharing platform, so that the sharing platform sends the shared bitstream to the client.

[0008] According to a performance scheduling method based on shared decoding capability provided by the present invention, the step of assigning weights to the shared tasks based on their priority and / or historical resource occupancy rates includes: If the priority of the shared task is high priority, then the first weight is assigned to the shared task; If the priority of the shared task is not high priority and the historical resource occupancy rate is less than a preset threshold, then a second weight is assigned to the shared task. If the priority of the shared task is not high priority, and the historical resource occupancy rate is above a preset threshold, then a third weight is assigned to the shared task. Wherein, the first weight is greater than the second weight, and the second weight is greater than the third weight; The step of determining the target node matching the weight based on the real-time performance index information of all decoding nodes includes: The resource allocation value of the shared task is determined based on the preset base weight and the allocation weight; wherein the allocation weight is one of the first weight, the second weight, and the third weight. Based on the available resources of each of the decoding nodes, the target node that matches the resource allocation value of the shared task is determined.

[0009] According to a performance scheduling method based on shared decoding capability provided by the present invention, the method further includes: If no target node matching the resource allocation value of the shared task is found, the decoding node with the largest idle resources is determined and used as the target node.

[0010] According to a performance scheduling method based on shared decoding capability provided by the present invention, the method further includes: If the current free resources on the decoding node with the largest free resources are less than the resource allocation value of the shared task, then the allocation weight of the currently executing shared task is adjusted according to the relationship between the current occupied resources of the shared task and the basic weight of the shared task, so that the current free resources on the decoding node with the largest free resources are greater than or equal to the resource allocation value of the shared task.

[0011] According to a performance scheduling method based on shared decoding capability provided by the present invention, the shared task sent by the receiving service platform includes: Receive multiple shared tasks sent by the service platform; the multiple shared tasks are multiple shared tasks created by the same client; each shared task corresponds one-to-one with the channel number of each shared channel; The client is used to receive the shared bitstream of each shared channel, parse the shared bitstream of each shared channel, and render it in the preset split screen corresponding to the channel number.

[0012] According to a performance scheduling method based on shared decoding capability provided by the present invention, the client is configured to receive an image capture command from a user for a specified window; in response to the image capture command, a player is created; the player is configured to pull and parse the bitstream, but not render it; The client is also used to compare the timestamp of the decoding node corresponding to the specified window with the timestamp of the player, and obtain the image corresponding to the latest timestamp based on the comparison result.

[0013] The present invention also provides a performance scheduling system based on shared decoding capabilities, including a client, a service platform, a shared platform, and decoding nodes; wherein, the decoding nodes include master nodes and slave nodes; The client is used to send shared tasks to the business platform; The business platform is used to send the shared task to the master node; The master node sends an indicator information request to the business platform; The business platform is also used to respond to the indicator information request by returning real-time performance indicator information of all decoding nodes to the master node; The master node is also configured to assign weights to the shared tasks based on their priority and / or historical resource utilization; determine target nodes that match the weights based on the real-time performance metrics of all decoding nodes; and send the target node identifier of the target node to the business platform. The business platform is also used to send the shared task to the target node according to the target node identifier; The target node is used to create a decoding process according to the shared task, capture the window screen and audio of the decoding process, and push the window screen and audio of the decoding process to the sharing platform. The sharing platform is used to send the shared bitstream to the client; The client is also used to parse the shared bitstream and render it on the display window.

[0014] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the performance scheduling method based on shared decoding capability as described above.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the performance scheduling method based on shared decoding capability as described above.

[0016] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the performance scheduling method based on shared decoding capability as described above.

[0017] This invention provides a performance scheduling method, system, device, and medium based on shared decoding capabilities. The method involves receiving shared tasks sent by a service platform (the shared tasks are those sent by the client to the service platform); sending an indicator information request to the service platform; the indicator information request instructing the service platform to return real-time performance indicator information for all decoding nodes; assigning weights to the shared tasks based on their priority and / or historical resource utilization; determining target nodes matching the weights based on the real-time performance indicator information of all decoding nodes; sending the target node's identifier to the service platform, enabling the service platform to send shared tasks to the target node based on the identifier; and the target node creating a decoding process based on the shared task, capturing the window screen and audio of the decoding process, and pushing the window screen and audio of the decoding process as a shared bitstream to the sharing platform, so that the sharing platform can send the shared bitstream to the client. This method ensures that important tasks receive priority response through task priority sorting, guaranteeing efficient task execution; rationally allocating resources according to task requirements; monitoring load conditions and allocating resources to terminals with low loads, reducing resource waste and improving the device utilization rate of decoding nodes. Especially when multiple videos are viewed simultaneously, it can ensure smooth playback for low-performance clients, while saving resources on decoding nodes and reducing the cost of building a decoding center. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of the application environment of the performance scheduling method based on shared decoding capability provided by the present invention.

[0020] Figure 2 This is a flowchart illustrating the performance scheduling method based on shared decoding capabilities provided by the present invention.

[0021] Figure 3 This is a timing diagram of the decoding node scheduling method provided by the present invention.

[0022] Figure 4 This is a flowchart illustrating the weight allocation method for shared tasks provided by the present invention.

[0023] Figure 5 This is a flowchart illustrating the method for dynamically adjusting the weights of shared tasks provided by the present invention.

[0024] Figure 6 This is a schematic diagram of resource allocation on slave nodes with the maximum available resources provided by the present invention.

[0025] Figure 7 This is a schematic diagram illustrating the effect of the multi-channel shared bitstream combination provided by the present invention.

[0026] Figure 8 This is a schematic diagram of the playback window under the four-screen split-screen configuration provided by the present invention.

[0027] Figure 9 This is a flowchart illustrating the local operation and interaction method for multiple video streams provided by the present invention.

[0028] Figure 10 This is a timing diagram of the multi-channel video sharing operation and interaction scheme provided by the present invention.

[0029] Figure 11 This is a schematic diagram of the polling window in the multi-window synchronous playback mode provided by the present invention.

[0030] Figure 12 This is a schematic diagram of the polling window in the single-window rotating playback mode provided by the present invention.

[0031] Figure 13 This is a schematic diagram of the performance scheduling device based on shared decoding capability provided by the present invention.

[0032] Figure 14 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0034] The following is combined Figures 1-14 Specific embodiments of the present invention are described below.

[0035] Figure 1 This diagram illustrates the application environment of the performance scheduling method based on shared decoding capabilities provided in this application. This application primarily addresses the problem of simultaneous playback of multiple video streams on various low-performance terminals, such as in the security monitoring field. To address this issue, this application proposes a decoding performance scheduling scheme suitable for low-performance terminals. This method aims to improve the user experience when operating various clients, meeting the demand for simultaneous viewing of multiple surveillance videos on low-performance hardware devices. Low-performance terminals refer to devices with relatively poor processing capabilities in hardware resources such as CPU, GPU, and memory. For example, their CPU clock speed may be low (around 1-1.5GHz), resulting in slower data processing and instruction execution. Also, the number of processor cores may be limited, with only 2 or 4 cores, restricting performance during multitasking. Furthermore, some devices have small RAM capacities, only a few hundred megabytes or less, limiting their ability to run very simple programs. Additionally, some devices have poor GPU performance; for example, some set-top boxes may experience stuttering, color distortion, or inability to play high-definition videos. This application solves the technical challenge of playing multiple high-definition videos on low-performance devices by utilizing a high-performance terminal as a decoding service center and sharing its decoding capabilities with lower-performance terminals. It also provides a solution for combining multiple shared bitstreams, enabling simultaneous playback of hundreds of videos. Figure 1 As shown, Figure 1 The document showcases low-performance terminals, such as low-performance PCs, set-top boxes, and TVs. The shared platform and business platform can be implemented using servers or server clusters, while high-performance servers can be implemented using cloud computers, servers, or high-performance PCs. Specifically, a high-performance server can be a server cluster, including one master node and multiple slave nodes.

[0036] Figure 2 This is a flowchart illustrating the performance scheduling method based on shared decoding capabilities provided by the present invention, as shown below. Figure 2As shown below, the method is applied to the master node in the aforementioned high-performance server cluster as the execution subject, and the method includes the following steps.

[0037] Step 201: Receive a shared task sent by the business platform; the shared task is a shared task sent by the client to the business platform.

[0038] The business platform can be understood as the control center of the entire system, mainly used for user management, device management, split-screen information management, stream address management, and shared signaling interaction. Shared tasks refer to tasks sent by low-performance terminals that utilize the decoding capabilities of shared decoding nodes; these decoding nodes are the nodes in the aforementioned high-performance server cluster. The high-performance server cluster can include only one master node, or it can include one master node and multiple slave nodes. The client refers to the aforementioned low-performance terminal. The following section combines... Figure 3 The timing diagram of the decoding node scheduling method is shown below.

[0039] Specifically, firstly, when a user needs to view multiple video streams simultaneously on the client, the client needs to acquire and simultaneously parse and render these streams. Low-performance terminals, due to slow loading, are prone to lag. Therefore, in this embodiment, upon receiving a user's viewing request, the client creates a shared task and sends it to the business platform. The business platform then forwards this shared task to the master node in the high-performance server cluster (also known as the decoding node). The master node is used for performance scheduling of the slave nodes to ensure reasonable resource allocation.

[0040] Step 202: Send an indicator information request to the service platform; the indicator information request is used to instruct the service platform to return real-time performance indicator information of all decoding nodes.

[0041] Among all the decoding nodes, there can be only one master node, or it can consist of one master node and multiple slave nodes.

[0042] Specifically, both the master and slave nodes are pre-bound to the business platform, and each master and slave node must upload its performance metrics information to the business platform in real time. This information covers overall performance metrics as well as the resource usage of each running shared process, in order to facilitate the analysis of resource utilization.

[0043] The master node sends a metric information request to the business platform in order to obtain the performance metric information of all slave machines from the business platform. This information includes the performance metrics of each shared task on each slave machine, so that the master can allocate resources and schedule tasks.

[0044] Step 203: Assign weights to the shared tasks based on their priority and / or historical resource utilization rate.

[0045] It should be noted that in the case of only one master node and no slave nodes, there is no need for slave node scheduling. Resource allocation will be based directly on the full performance of the master node.

[0046] If multiple slave nodes are deployed, a dynamic scheduling strategy and a task priority strategy are required. All shared tasks are pre-assigned priorities, such as high, medium, and low. When a shared task has a high priority, it is assigned a weight of P1. If it does not have a high priority, and the corresponding client has consistently used less CPU resources or had a shorter usage time than a pre-assigned threshold for each shared task created in its historical execution history, then a weight is assigned to the current shared task. 2; If the current shared task is neither high priority nor has a historical resource utilization rate above a preset threshold, then assign a weight P3; where 3. Place the shared task into the scheduling queue according to its assigned weight, and let it wait for scheduling in order of weight.

[0047] Step 204: Based on the real-time performance index information of all decoding nodes, determine the target node that matches the weight.

[0048] Specifically, the master node determines the target node that matches the weight based on the real-time performance index information of all decoding nodes. For example, when the allocation weight of the shared task is high, the decoding node with more idle resources is selected as the target node based on the aforementioned real-time performance index information.

[0049] Step 205: Send the target node identifier of the target node to the business platform, so that the business platform sends the shared task to the target node according to the target node identifier; The target node is used to create a decoding process according to the shared task, capture the window screen and audio of the decoding process, and push the window screen and audio of the decoding process as a shared bitstream to the sharing platform, so that the sharing platform sends the shared bitstream to the client.

[0050] Each decoding node is identified using a unique node identifier.

[0051] Specifically, once the master node identifies the target node, it sends the target node's identifier to the business platform. The business platform then sends the shared task to the target node corresponding to that identifier. Upon receiving the shared task, the target node automatically creates a decoding process, executes the shared task using this process, and begins capturing the window display and audio of the decoding process. The captured information is then pushed to the sharing platform as a shared stream. Each shared task has an independent decoding process, and a single decoding process can simultaneously decode up to 64 video streams. Each decoding node can execute a maximum of 16 shared tasks.

[0052] Optionally, to ensure the stability of the decoding process, a watchdog service is configured on the decoding node so that the decoding task can be quickly restarted and restored in case of abnormalities.

[0053] It's important to note that a shared channel is also created between the sharing platform and the client, identified by a unique channel number. In detail, when a user logs in on the client, they can choose to activate sharing mode. The business platform then initiates sharing mode: the client requests the creation of a shared channel from the sharing platform. Once the shared channel is successfully created, it returns a channel number to the client. The client then synchronizes this channel number to the selected target node through the business platform. The target node then pushes the shared stream to the sharing platform through the shared channel corresponding to that channel number.

[0054] The sharing platform returns the shared bitstream to the client through this sharing channel, and the client can then see the corresponding shared bitstream video and play the corresponding audio.

[0055] The above embodiment involves receiving a shared task sent by a service platform; the shared task is a shared task sent by the client to the service platform; sending an indicator information request to the service platform; the indicator information request instructs the service platform to return real-time performance indicator information of all decoding nodes; assigning weights to the shared task based on its priority and / or historical resource utilization; determining a target node matching the weights based on the real-time performance indicator information of all decoding nodes; sending the target node's identifier to the service platform so that the service platform can send the shared task to the target node based on the target node identifier; and the target node, which creates a decoding process based on the shared task, captures the window screen and audio of the decoding process, and pushes the window screen and audio of the decoding process as a shared bitstream to the sharing platform so that the sharing platform can send the shared bitstream to the client. This method ensures that important tasks receive priority response through task priority sorting, guaranteeing efficient task execution; rationally allocating resources according to task requirements; monitoring load conditions and allocating resources to terminals with low load, reducing resource waste, and improving the device utilization rate of decoding nodes. Especially when multiple videos are viewed simultaneously, it can ensure smooth playback for low-performance clients, while saving resources on decoding nodes and reducing the cost of building a decoding center.

[0056] In one embodiment, such as Figure 4 As shown, Figure 4 A schematic diagram illustrating the weight allocation method for shared tasks is provided. Step 203 includes: if the priority of the shared task is high priority, then a first weight P1 is assigned to the shared task; if the priority of the shared task is not high priority and the historical resource occupancy rate is less than a preset threshold, then a second weight P2 is assigned to the shared task; if the priority of the shared task is not high priority and the historical resource occupancy rate is above the preset threshold, then a third weight P3 is assigned to the shared task; wherein, the first weight is greater than the second weight, and the second weight is greater than the third weight.

[0057] Specifically, 1) When the business platform receives a shared task, it will first notify the master node of the task. The master node will manage all slave information and shared task information.

[0058] 2) If the master node determines that the task sent is a high-priority task, it will assign a corresponding weight. And enter the scheduling queue according to the weight.

[0059] 3) If the CPU usage and usage duration of the task are both low each time it is used by the triggering user, then assign the corresponding weight. 2. And enter the scheduling queue according to the weight.

[0060] 4) All other tasks are uniformly assigned standard weights. 3. And enter the scheduling queue according to the weight.

[0061] 5) Among them 3. The scheduler will sort the corresponding tasks according to their weight.

[0062] Accordingly, step 204 includes: determining the resource allocation value of the shared task based on a preset base weight P and an allocation weight; wherein the allocation weight is one of the first weight, the second weight, and the third weight; and determining the target node that matches the resource allocation value of the shared task based on the idle resources of each of the decoding nodes.

[0063] Specifically, each shared task has a base resource allocation value P, and the final resources allocated to each task are... .For example, Figure 4 The resources corresponding to high-priority tasks are P+P1.

[0064] Furthermore, if no target node matching the resource allocation value of the shared task is found, the decoding node with the largest idle resources is determined and used as the target node.

[0065] In the above embodiments, weights are assigned to shared tasks based on their priority and / or historical resource utilization, thereby providing a basis for orderly scheduling of shared tasks. In addition, resources are allocated to each shared task based on the matching relationship between the real-time performance indicators (including idle resource information) of all decoding nodes and the weights assigned to shared tasks, ensuring efficient utilization of the computing resources of decoding nodes, which is beneficial for saving resources. At the same time, it can also ensure the efficient execution of shared tasks, providing the client with smooth and uninterrupted audio and video playback.

[0066] In one embodiment, such as Figure 5 As shown, a flowchart of a method for dynamically adjusting the weight of a shared task is also provided. The method further includes: if the current idle resources on the decoding node with the largest idle resources are less than the resource allocation value of the shared task, then the allocation weight of the currently executing shared task is adjusted according to the relationship between the current occupied resources of the currently executing shared task and the basic weight of the shared task, so that the current idle resources on the decoding node with the largest idle resources are greater than or equal to the resource allocation value of the shared task.

[0067] Specifically, if no available resource matching the resource allocation value of the shared task can be found, the slave node with the largest available resource must first be identified, and the resource allocation value of the shared task existing on the slave node must be adjusted accordingly based on the relationship between the largest available resource and the resource allocation value of the shared task.

[0068] like Figure 6 As shown, Figure 6 This diagram illustrates the resource allocation on the slave node with the largest idle resource. On this slave node, if... In such cases, it is necessary to adjust the allocation weights of each existing shared task on that node, for example... Figure 6 In and 2. However, if the resources actually used by the currently executing shared task have exceeded its base weight (e.g., ... Figure 6 Task 2 in the example indicates that the task has high resource requirements, so there is no need to adjust the allocation weight of the task. For example... Figure 6 If Task 2's current resource usage has exceeded its base weight, then no adjustment to the allocation weight is needed. 2. This can only be achieved by adjusting the allocation weights of Task 1. 1. Implement the adjustment of idle resources on the slave node.

[0069] The above embodiments, by adopting a dynamic scheduling strategy, that is, dynamically adjusting the allocation weights and adjusting the task allocation in real time, can maximize resource utilization.

[0070] In one embodiment, a method for combining multiple shared bitstreams is also provided, such as... Figure 7 The above, Figure 7 A schematic diagram illustrating the effect of combining multiple shared bitstreams is shown. Step 201 above includes: receiving multiple shared tasks sent by the service platform; the multiple shared tasks are multiple shared tasks created by the same client; each shared task corresponds one-to-one with the channel number of each shared channel; the client is used to receive the shared bitstream of each shared channel, parse the shared bitstream of each shared channel, and render it in the preset split screen corresponding to the channel number.

[0071] Specifically, 1) If the client needs to play more than 100 video streams, then enable the multi-stream sharing mode.

[0072] 2) The client requests multiple shared channels from the business platform, and the business platform synchronizes the channel number of each shared channel to the decoding node.

[0073] 3) Each decoding node that receives a shared task automatically creates a decoding process that corresponds one-to-one with the received shared task and adds it to the corresponding shared channel.

[0074] 4) Once the decoding node successfully joins the shared channel, it begins to capture the window screen and audio of the decoding process. The captured information is then pushed to the sharing platform as a shared bitstream, and the client can then see the corresponding shared bitstream screen and audio.

[0075] 5) The client renders the shared bitstreams from different shared channels onto the corresponding split-screen windows, ultimately achieving a solution for combined playback of multiple shared bitstreams.

[0076] Renderings are shown below. Figure 7 As shown, in Figure 7 In the example shown, a client can connect to multiple shared channels simultaneously, each provided by a single decoding node. For instance, to view a total of 4*64 video streams on the client, each decoding node would provide a shared stream containing 64 video streams. By using a 2*2 split-screen layout on the client, video streams from four decoding nodes can be displayed separately, thus achieving the effect of displaying 256 video streams while consuming only four decoding resources (referring to four shared channels).

[0077] The above embodiments, by splitting the multi-channel video playback task on a single client into a mechanism that combines multiple shared bitstreams, can achieve the effect of simultaneous playback of hundreds of videos.

[0078] In one embodiment, the client is further configured to receive an image capture command from a user for a specified window and create a player; the player is only used to pull and parse the bitstream, and does not render it; The client is also used to compare the timestamp of the decoding node corresponding to the specified window with the timestamp of the player, and obtain the image corresponding to the latest timestamp based on the comparison result.

[0079] Specifically, this embodiment provides a solution for screenshotting (or screen recording) multiple video streams.

[0080] It should be noted that some operations can be completed locally on the client side without needing to be synchronized to the decoding node via the business platform signaling. Examples include: selecting a window to view device details, popping up a right-click menu, and performing settings operations on the device corresponding to the video stream.

[0081] For example, Figure 8 This diagram illustrates the playback windows in a four-screen split-screen configuration on the client side. Using this diagram as a coordinate system, when a user wants to view the device details of the decoding node corresponding to the video played in split-screen number 4, they can click on the point in the diagram. The position, i.e., the corresponding coordinate position. .

[0082] The client's local memory stores all the split-screen configuration data, including the current number of split screens, the coordinates of the rectangular area corresponding to each split screen, and the device ID of the decoding node bound to each split screen. Figure 8 The fourth split-screen in the four-screen format is determined by the coordinates of the top left corner. and the coordinates of the bottom right corner It consists of a rectangular region. The coordinates of the screen are exactly covered by the rectangular area corresponding to the screen with serial number 4.

[0083] Therefore, it can be determined that the currently selected device is the decoding node device bound to Split Screen 4. To perform settings or other operations on this device, a network request can be sent directly from the client to the service platform. Since control is directly through the service platform, these operations also include: pan / tilt control of the decoding node device, intercom on / off switch, sleep mode, focus adjustment, etc.

[0084] In addition, there is another type of operation that cannot be achieved through the above methods, such as screenshotting and screen recording during video playback, which requires storing the video footage locally on the client as an image or video file. Therefore, this embodiment proposes the following solution, such as... Figure 9 As shown, Figure 9 A flowchart illustrating the local operation and interaction method for multiple video streams is provided, which includes the following steps.

[0085] 1) When a user initiates a screenshot or screen recording command on the client, the client will first determine whether the screenshot / screen recording function in shared mode is used. If shared mode is used, the client's local player will be directly invoked to perform the screenshot or screen recording operation. Since the shared stream contains all playback frames under the entire split screen, the saved image and video files will contain multiple frames.

[0086] 2) If the screenshot / screen recording function in sharing mode is not used, the next step will depend on whether the currently selected window is bound to the decoding node device. If the window is not bound to the decoding node device, the user will be prompted to select a new window.

[0087] 3) If the window is already bound to a decoding node device, a player will be created in the background of the client that only pulls the stream and decodes it without rendering.

[0088] 4) When the user triggers the screenshot and screen recording operation, the player will then render the video and perform the corresponding operation to save the file locally, thereby realizing the screenshot / screen recording function that only includes the video screen corresponding to the decoding node device.

[0089] 5) Because the video played by the player created in the background may be out of sync with the video being played by the decoding node, the sharing mode will periodically request the timestamp and playback status of the video from the current playback device from the business platform.

[0090] 6) When the timestamp of the playback screen on the decoding node is less than the timestamp of the playback screen on the local player created in the background, it means that the playback screen of the local player has been updated. If it is screen recording, the local player can directly perform the screen recording operation; if it is screenshotting, in order to ensure the quality of the screenshot and to synchronize the screenshot with the screen actually seen by the user, it is necessary to notify the decoding node to perform a screenshot operation on the device and send the screenshot to the business platform. The client can then obtain the corresponding screenshot data through the business platform.

[0091] 7) When the timestamp of the playback screen of the decoding node is greater than the timestamp of the playback screen created by the local background player, it means that the playback screen of the decoding node has been updated. Therefore, the local player needs to perform frame tracking to synchronize to the position corresponding to the timestamp before taking a screenshot / screen recording.

[0092] The above embodiments enable screenshot / screen recording of a specific split screen in split-screen mode by creating a local player.

[0093] In one embodiment, a shared operation and interaction scheme for multiple video streams is also provided, such as... Figure 10 As shown, Figure 10 A sequence diagram illustrating the shared operation and interaction scheme for multiple video streams is provided, including the following steps.

[0094] In a shared decoding capability solution, users frequently need to operate the decoding node devices within the split screen. These high-frequency interactions include, but are not limited to: adjusting video position, closing and reopening specific video streams, adjusting video volume, and changing the number of split screens. To ensure that these operations are reflected on the decoding nodes in real time, we adopted the following strategy: the signaling corresponding to these operations is transmitted to the relevant decoding nodes in real time through the service platform, as detailed below.

[0095] 1) When the sharing mode is enabled, users can interact with the client screen, such as adjusting the number of split screens or turning on / off specific video streams.

[0096] 2) These user operations will be sent to the business platform via signaling. The business platform will then transmit the corresponding operation notifications to the decoding node based on the binding relationship between the client and the decoding node.

[0097] 3) After receiving these operation signals, the decoding process will perform corresponding operations, such as adjusting the number of split screens or turning on / off a specific video stream.

[0098] 4) After the decoding process completes its operation, the relevant view window information will be updated immediately. Since the sharing scheme uses process window capture, no additional operation is required; the screen in the shared bitstream will automatically synchronize with the decoding process's screen. Capturing the process window screen refers to obtaining the visual content of a specified application window within the operating system through specific technical means and converting it into processable image data. This technology involves multi-dimensional system-level operations.

[0099] 5) Interactive operations also include adjusting the audio volume of individual videos. Since the solution achieves this by capturing the audio of the process, the volume of the audio played in that process can be adjusted independently, thereby enabling precise control and synchronization of the volume of each audio stream in multiple video sources.

[0100] 6) Ultimately, the latest shared bitstream data displayed on the client will be consistent with the actual interactive effect of the user's operation.

[0101] The above embodiments achieve synchronization between client user operations and decoding nodes through a multi-channel video sharing and interaction scheme.

[0102] In one embodiment, a multi-screen video rotation scheme based on multiple shared bitstream combinations is also provided. Multi-screen video rotation is a common service in the security field and is frequently used by users. For example, in a rotation group containing hundreds of monitoring devices, all video feeds are played in a loop using a split-screen mode of 64 screens or more per group. On the one hand, playing 64 video feeds presents a performance bottleneck. Even if a computer has good performance and can achieve 64-screen playback, slow loading and a poor user experience often occur when switching to the next video group.

[0103] To address these issues, this embodiment proposes a multi-screen video rotation scheme. Specifically, the rotation group is reorganized into multiple new video groups, and a dedicated shared channel is established for each new group. These shared channels are responsible for playing their respective video content, thereby resolving the performance bottleneck and slow loading issues when switching videos in traditional rotation groups.

[0104] On the client side, this solution offers two playback modes for users to choose from: 1. Multi-window synchronous playback mode: In this mode, the client creates a number of playback windows equal to the number of shared channels, with each window bound to a stream received from the corresponding shared channel. When switching to the next screen of video is needed, simply switch to the corresponding shared channel and display the window, thus achieving near-instantaneous screen switching and significantly improving the user's viewing experience.

[0105] like Figure 11 , Figure 11This diagram illustrates a polling window in a multi-window synchronous playback mode. A polling group containing 256 video feeds is reorganized into four smaller groups, with an independent playback window created for each group. Simultaneously, four shared channels are created, each responsible for playing the video content from the monitoring device list of its corresponding group and outputting the shared bitstream of these videos to the appropriate client playback window. When switching to the next screen of video is required, the client simply displays the next playback window sequentially, achieving a fast and smooth video polling experience. This method simplifies the video switching process, improves operational efficiency, and enhances the continuity of the user's viewing experience.

[0106] II. Single-Window Alternating Playback Mode: In this mode, although the same number of shared channels are created, the same playback window (i.e., the player) is used to play the streams from different shared channels in turn. This mode involves a loading process when switching to the next screen, but the consumption of decoding resources (referring to the player created in the client background) is reduced from multiple to a single resource, helping to save system resources. For example... Figure 12 As shown, Figure 12 This is a schematic diagram of the polling window in the single-window polling mode. In this mode, the client creates only one playback window, on which the shared bitstream corresponding to each video group is played in turn.

[0107] Both modes are based on a multi-channel shared bitstream solution, each with its own advantages and disadvantages. Users can choose freely according to their needs and preferences to achieve the best user experience.

[0108] In one embodiment, a performance scheduling system based on shared decoding capabilities is also provided, including a client, a service platform, a sharing platform, and decoding nodes; wherein, the decoding nodes include master nodes and slave nodes; The client is used to send shared tasks to the business platform; The business platform is used to send the shared task to the master node; The master node sends an indicator information request to the business platform; The business platform is also used to respond to the indicator information request by returning real-time performance indicator information of all decoding nodes to the master node; The master node is also configured to assign weights to the shared tasks based on their priority and / or historical resource utilization; determine target nodes that match the weights based on the real-time performance metrics of all decoding nodes; and send the target node identifier of the target node to the business platform. The business platform is also used to send the shared task to the target node according to the target node identifier; The target node is used to create a decoding process according to the shared task, capture the window screen and audio of the decoding process, and push the window screen and audio of the decoding process to the sharing platform. The sharing platform is used to send the shared bitstream to the client; The client is also used to parse the shared bitstream and render it on the display window.

[0109] Specifically, the execution steps of each component in the above system embodiment are as described in the above method embodiment, and will not be repeated here.

[0110] The performance scheduling device based on shared decoding capability provided by the present invention is described below. The performance scheduling device based on shared decoding capability described below and the performance scheduling method based on shared decoding capability described above can be referred to in correspondence.

[0111] like Figure 13 As shown, this application provides a performance scheduling device based on shared decoding capabilities, applied to the master node in the decoding node. The performance scheduling device based on shared decoding capabilities includes the following modules.

[0112] The shared task receiving module 1301 is used to receive shared tasks sent by the business platform; the shared task is a shared task sent by the client to the business platform. The indicator information request sending module 1302 is used to send an indicator information request to the business platform; the indicator information request is used to instruct the business platform to return the real-time performance indicator information of all decoding nodes. The weight allocation module 1303 is used to allocate weights to the shared tasks based on the priority of the shared tasks and / or the historical resource occupancy rate. The target node determination module 1304 is used to determine the target node that matches the weight based on the real-time performance index information of all the decoding nodes. The target node identifier sending module 1305 is used to send the target node identifier of the target node to the business platform, so that the business platform sends the shared task to the target node according to the target node identifier; the target node is used to create a decoding process according to the shared task, capture the window screen and audio of the decoding process, and push the window screen and audio of the decoding process as a shared bitstream to the sharing platform, so that the sharing platform sends the shared bitstream to the client.

[0113] In one embodiment, the weight allocation module 1303 is further configured to: If the priority of the shared task is high priority, then the first weight is assigned to the shared task; If the priority of the shared task is not high priority and the historical resource occupancy rate is less than a preset threshold, then a second weight is assigned to the shared task. If the priority of the shared task is not high priority, and the historical resource occupancy rate is above a preset threshold, then a third weight is assigned to the shared task. Wherein, the first weight is greater than the second weight, and the second weight is greater than the third weight; The aforementioned target node determination module 1304 is further used for: The resource allocation value of the shared task is determined based on the preset base weight and the allocation weight; wherein the allocation weight is one of the first weight, the second weight, and the third weight. Based on the available resources of each of the decoding nodes, the target node that matches the resource allocation value of the shared task is determined.

[0114] In one embodiment, the target node determination module 1304 is further configured to: If no target node matching the resource allocation value of the shared task is found, the decoding node with the largest idle resources is determined and used as the target node.

[0115] In one embodiment, the target node determination module 1304 is further configured to: If the current free resources on the decoding node with the largest free resources are less than the resource allocation value of the shared task, then the allocation weight of the currently executing shared task is adjusted according to the relationship between the current occupied resources of the shared task and the basic weight of the shared task, so that the current free resources on the decoding node with the largest free resources are greater than or equal to the resource allocation value of the shared task.

[0116] In one embodiment, the shared task receiving module 1301 described above is further configured to: Receive multiple shared tasks sent by the service platform; the multiple shared tasks are multiple shared tasks created by the same client; each shared task corresponds one-to-one with the channel number of each shared channel; The client is used to receive the shared bitstream of each shared channel, parse the shared bitstream of each shared channel, and render it in the preset split screen corresponding to the channel number.

[0117] In one embodiment, the client is configured to receive an image capture command from a user for a specified window; in response to the image capture command, create a player; the player is configured to pull and parse the bitstream without rendering; the client is further configured to compare the timestamp of the decoding node corresponding to the specified window with the timestamp of the player, and obtain the image corresponding to the latest timestamp based on the comparison result.

[0118] Figure 14 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 14 As shown, the electronic device may include: a processor 1410, a communications interface 1420, a memory 1430, and a communications bus 1440, wherein the processor 1410, the communications interface 1420, and the memory 1430 communicate with each other through the communications bus 1440. The processor 1410 can invoke logical instructions in the memory 1430 to execute a performance scheduling method based on shared decoding capabilities. This method includes: receiving a shared task sent by a service platform; the shared task being a shared task sent by a client to the service platform; sending an indicator information request to the service platform; the indicator information request instructing the service platform to return real-time performance indicator information for all decoding nodes; assigning weights to the shared task based on its priority and / or historical resource utilization; determining a target node matching the weights based on the real-time performance indicator information for all decoding nodes; sending a target node identifier to the service platform so that the service platform sends the shared task to the target node based on the target node identifier; and the target node creating a decoding process based on the shared task, capturing the window screen and audio of the decoding process, and pushing the window screen and audio of the decoding process as a shared bitstream to the sharing platform so that the sharing platform sends the shared bitstream to the client.

[0119] Furthermore, the logical instructions in the aforementioned memory 1430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0120] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the performance scheduling method based on shared decoding capability provided by the above methods. The method includes: receiving a shared task sent by a service platform; the shared task being a shared task sent by a client to the service platform; sending an indicator information request to the service platform; the indicator information request being used to instruct the service platform to return real-time performance indicator information of all decoding nodes; assigning weights to the shared task according to the priority of the shared task and / or historical resource utilization rate; determining a target node matching the weights according to the real-time performance indicator information of all decoding nodes; sending a target node identifier of the target node to the service platform, so that the service platform sends the shared task to the target node according to the target node identifier; the target node being used to create a decoding process according to the shared task, capture the window screen and audio of the decoding process, and push the window screen and audio of the decoding process as a shared bitstream to the sharing platform, so that the sharing platform sends the shared bitstream to the client.

[0121] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements a performance scheduling method based on shared decoding capability provided by the above methods. The method includes: receiving a shared task sent by a service platform; the shared task being a shared task sent by a client to the service platform; sending an indicator information request to the service platform; the indicator information request instructing the service platform to return real-time performance indicator information of all decoding nodes; assigning weights to the shared task according to its priority and / or historical resource utilization rate; determining a target node matching the weights based on the real-time performance indicator information of all decoding nodes; sending a target node identifier to the service platform so that the service platform sends the shared task to the target node according to the target node identifier; and the target node being used to create a decoding process according to the shared task, capture the window screen and audio of the decoding process, and push the window screen and audio of the decoding process as a shared bitstream to the sharing platform so that the sharing platform sends the shared bitstream to the client.

[0122] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0123] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0124] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A performance scheduling method based on shared decoding capabilities, characterized in that, Applied to the master node, including: Receive a shared task sent by the business platform; the shared task is a shared task sent by the client to the business platform. Send an indicator information request to the business platform; the indicator information request is used to instruct the business platform to return real-time performance indicator information of all decoding nodes; Based on the priority of the shared task and / or the historical resource utilization rate, a weight is assigned to the shared task; Based on the real-time performance metrics of all the decoding nodes, determine the target node that matches the weight; The target node sends its target node identifier to the business platform, so that the business platform sends the shared task to the target node based on the target node identifier; the target node is used to create a decoding process according to the shared task, capture the window screen and audio of the decoding process, and push the window screen and audio of the decoding process as a shared bitstream to the sharing platform, so that the sharing platform sends the shared bitstream to the client.

2. The performance scheduling method based on shared decoding capability according to claim 1, characterized in that, The step of assigning weights to the shared tasks based on their priority and / or historical resource utilization includes: If the priority of the shared task is high priority, then the first weight is assigned to the shared task; If the priority of the shared task is not high priority and the historical resource occupancy rate is less than a preset threshold, then a second weight is assigned to the shared task. If the priority of the shared task is not high priority, and the historical resource occupancy rate is above a preset threshold, then a third weight is assigned to the shared task. Wherein, the first weight is greater than the second weight, and the second weight is greater than the third weight; The step of determining the target node matching the weight based on the real-time performance index information of all decoding nodes includes: The resource allocation value of the shared task is determined based on the preset base weight and the allocation weight; wherein the allocation weight is one of the first weight, the second weight, and the third weight. Based on the available resources of each of the decoding nodes, the target node that matches the resource allocation value of the shared task is determined.

3. The performance scheduling method based on shared decoding capability according to claim 2, characterized in that, The method further includes: If no target node matching the resource allocation value of the shared task is found, the decoding node with the largest idle resources is determined and used as the target node.

4. The performance scheduling method based on shared decoding capability according to claim 3, characterized in that, The method further includes: If the current free resources on the decoding node with the largest free resources are less than the resource allocation value of the shared task, then the allocation weight of the currently executing shared task is adjusted according to the relationship between the current occupied resources of the shared task and the basic weight of the shared task, so that the current free resources on the decoding node with the largest free resources are greater than or equal to the resource allocation value of the shared task.

5. The performance scheduling method based on shared decoding capability according to claim 1, characterized in that, The shared task sent by the receiving service platform includes: Receive multiple shared tasks sent by the service platform; the multiple shared tasks are multiple shared tasks created by the same client; each shared task corresponds one-to-one with the channel number of each shared channel; The client is used to receive the shared bitstream of each shared channel, parse the shared bitstream of each shared channel, and render it in the preset split screen corresponding to the channel number.

6. The performance scheduling method based on shared decoding capability according to claim 1, characterized in that, The client is configured to receive an image capture command from a user for a specified window; in response to the image capture command, a player is created; the player is configured to pull and parse the bitstream, but not render it. The client is also used to compare the timestamp of the decoding node corresponding to the specified window with the timestamp of the player, and obtain the image corresponding to the latest timestamp based on the comparison result.

7. A performance scheduling system based on shared decoding capabilities, characterized in that, It includes a client, a business platform, a sharing platform, and decoding nodes; wherein, the decoding nodes include master nodes and slave nodes; The client is used to send shared tasks to the business platform; The business platform is used to send the shared task to the master node; The master node sends an indicator information request to the business platform; The business platform is also used to respond to the indicator information request by returning real-time performance indicator information of all decoding nodes to the master node; The master node is also configured to assign weights to the shared tasks based on their priority and / or historical resource utilization; determine target nodes that match the weights based on the real-time performance metrics of all decoding nodes; and send the target node identifier of the target node to the business platform. The business platform is also used to send the shared task to the target node according to the target node identifier; The target node is used to create a decoding process according to the shared task, capture the window screen and audio of the decoding process, and push the window screen and audio of the decoding process to the sharing platform. The sharing platform is used to send the shared bitstream to the client; The client is also used to parse the shared bitstream and render it on the display window.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the performance scheduling method based on shared decoding capability as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the performance scheduling method based on shared decoding capability as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the performance scheduling method based on shared decoding capability as described in any one of claims 1 to 6.

Images