Video encoding, decoding and rendering method with middleware cooperating with local resources

Abstract

The invention discloses a video encoding and decoding and rendering method based on middleware cooperating with local resources, and belongs to the technical field of internet communication. Encoded video data is processed through the plug middleware, data label information is added, the data label information is sent to a switchboard terminal through a remote transmission protocol, and after the switchboard terminal receives the video data, the video data is transmitted to the local resources through the plug middleware. And the signaling interaction result is forwarded to the corresponding sub-machine virtual machine terminal, and then is forwarded to the corresponding sub-machine terminal through a data transmission protocol by using local network resources. Through the above mode, a cooperative working mode of the plug middleware and the local resources of the switchboard terminal is introduced to optimize the video encoding and decoding and rendering process in the cloud desktop environment, thereby reducing the occupation of network resources and the performance consumption of a virtual machine, effectively improving the efficiency of screen broadcasting and the user experience, and improving the user experience. And more efficient technical support is provided for application of the cloud desktop in scenes such as teaching and office.

Description

Technical Field

[0001] The present invention relates to the field of Internet communication technologies, and particularly relates to a method for video encoding, decoding, and rendering by a middleware collaborating with local resources. Background Art

[0002] Video encoding and decoding in a cloud desktop is a process of compressing and encoding or decompressing and decoding the original video data. In a cloud desktop environment, the main purpose is to reduce the amount of video data transmitted, lower the network bandwidth requirements, and at the same time ensure that the user terminal can play the video smoothly. In a cloud desktop architecture, the server side encodes the video content generated by the user's operations and sends it to the client through the network. The client receives the encoded video stream and decodes it to restore the original video image for display.

[0003] Rendering is a process of converting data such as models and scenes stored in a computer into a final displayable image or video frame through a series of calculations.

[0004] In a traditional cloud desktop environment, for example, using a mainframe and multiple sub - machines, the screen broadcasting function of the mainframe often directly sends video data to each terminal through a virtual machine. However, this method has many drawbacks:

[0005] First, the video data needs to be transmitted from the virtual machine to each terminal separately, resulting in extremely high consumption of network bandwidth. In a multi - terminal scenario, such as in a teaching scenario with many sub - machines, this large amount of repeated data transmission will quickly exhaust the network bandwidth resources, leading to network congestion and affecting the data transmission efficiency.

[0006] Second, the virtual machine not only has to be responsible for capturing video data, but also for encoding and transmitting it to each terminal, which greatly increases its computational load. As the number of terminals increases, the performance burden on the virtual machine becomes heavier, which may lead to a slower processing speed and even lags.

[0007] Based on this, the present invention designs a method for video encoding, decoding, and rendering by a middleware collaborating with local resources to solve the above problems. Summary of the Invention

[0008] In view of the above - mentioned drawbacks of the prior art, the present invention provides a method for video encoding, decoding, and rendering by a middleware collaborating with local resources.

[0009] To achieve the above objectives, the present invention is realized through the following technical solutions:

[0010] A method for video encoding, decoding, and rendering by a middleware collaborating with local resources, comprising the following steps:

[0011] Step 1: The cloud desktop software and the slave terminal perform signaling interaction through the plugin middleware to determine the slave virtual machine terminal information corresponding to each slave terminal;

[0012] Step 2: The cloud desktop software starts the content capture module to capture the video data of the main desktop, and then encodes the video data through the encoding module;

[0013] Step 3: The encoded video data is processed by the plugin middleware and data label information is added, and then sent to the main terminal;

[0014] Step 4: After receiving the video data, the main terminal forwards the video data to the corresponding slave virtual machine terminal according to the signaling interaction result, and then uses the network resources of the main machine locally to forward the data to the corresponding slave terminal through the data transmission protocol;

[0015] Step 5: The slave terminal decodes the video data through the decoding module and then transmits it to the rendering module, and the rendering module displays the rendered picture on the screen of the slave terminal.

[0016] Furthermore, the specific implementation steps of Step 1 are as follows:

[0017] A1. The slave terminal sends a registration request containing its own device identifier and network configuration to the plugin middleware. After receiving these requests, the plugin middleware forwards them to the cloud desktop software;

[0018] A2. The cloud desktop software verifies the identity of the slave terminal. By querying the predefined list of legal terminals or using an encryption authentication algorithm, it ensures that the accessed slave terminal is authorized;

[0019] A3. The plugin middleware detects the network status information of the slave terminal in real time, and based on the network status information, the cloud desktop software negotiates with the slave terminal to select an appropriate data transmission protocol;

[0020] A4. After signaling interaction, the corresponding relationship between each slave virtual machine and the corresponding slave terminal is determined, and the corresponding relationship information is stored in the cache of the plugin middleware for use during subsequent data transmission.

[0021] Furthermore, the encryption authentication algorithm uses a symmetric encryption algorithm or a new generation symmetric encryption standard algorithm.

[0022] Furthermore, the network status information includes network bandwidth, latency, and packet loss rate.

[0023] Furthermore, the data transmission protocol uses the TCP protocol or the IP protocol.

[0024] Further, the specific implementation steps of step two are as follows:

[0025] B1. The cloud desktop software starts the content capture module, and the content capture module periodically captures the entire screen image of the PBX to generate video data;

[0026] B2. The content capture module transmits the video data to the encoding module in the form of raw image data;

[0027] B3. According to the network status information and transmission protocol determined by the previous signaling interaction, the encoding module selects an appropriate encoding format to encode the video data during the encoding process.

[0028] Further, the specific implementation steps of step three are as follows:

[0029] C1. The encoded video data is sent to the plugin middleware;

[0030] C2. The plugin middleware encapsulates the video data and adds data tag information;

[0031] C3. The plugin middleware sends the video data to the PBX terminal through the network. During the transmission process, the plugin middleware will monitor the network status information in real time to obtain the network congestion situation, and then dynamically adjust the data sending rate according to the network congestion situation.

[0032] Further, the data tag information includes the target terminal identifier, data type, and timestamp.

[0033] Further, the specific implementation steps of step four are as follows:

[0034] D1. After receiving the video data from the plugin middleware, the PBX terminal first parses the data tag information;

[0035] D2. The PBX terminal queries the relationship between each sub-machine virtual machine terminal and the corresponding sub-machine terminal stored during the previous signaling interaction according to the target terminal identifier therein, determines the sub-machine virtual machine terminal corresponding to each video data, and sends it to the sub-machine virtual machine terminal;

[0036] D3. The PBX terminal uses the local network resources to forward the video data to the corresponding sub-machine terminal of the sub-machine virtual machine terminal through the data transmission protocol. During the forwarding process, the PBX terminal will cache and schedule the video data to ensure the orderly transmission of the data. If the network of a certain sub-machine terminal has a short-term interruption or congestion, the PBX terminal will temporarily store the video data in the local cache and wait until the network returns to normal before continuing to forward, avoiding frame freezes caused by data loss.

[0037] Further, the specific implementation steps of Step 5 are as follows:

[0038] E1. After the slave machine terminal receives the forwarded video data, it transmits it to the local decoding module.

[0039] E2. The decoding module calls the corresponding decoding algorithm for decoding according to the encoding format of the video data.

[0040] E3. The decoded video data is transmitted to the rendering module in the form of original image frames.

[0041] E4. The rendering module arranges the original image frames in chronological order, and adjusts and optimizes the images according to the display device parameters of the slave machine terminal.

[0042] E5. The rendering module displays the rendered picture on the screen of the slave machine terminal, realizing the same screen broadcast effect as the slave machine virtual machine, and providing a clear and smooth screen broadcast experience; at the same time, the slave machine virtual machine terminal will monitor the performance indicators of the decoding and rendering processes in real time. If performance anomalies are found, it will promptly feedback to the cloud desktop software for adjustment and optimization.

[0043] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. By introducing the collaborative working mode of the plugin middleware and the local resources of the master machine terminal, the present invention optimizes the video encoding, decoding, and rendering processes in the cloud desktop environment, reduces the network resource occupation and virtual machine performance consumption, effectively improves the efficiency of screen broadcasting and the user experience, and provides more efficient technical support for the application of cloud desktops in teaching, office, and other scenarios;

[0044] 2. The present invention captures and encodes the video data of the master machine desktop through the cloud desktop software. The encoded video data is processed by the plugin middleware and data tag information is added, and then sent to the master machine terminal through the remote transmission protocol, reducing the computing load of the virtual machine and ensuring the data processing speed; by adding data tags, it can provide multiple information for subsequent video data processing, distribution, and management, which helps to achieve more accurate video stream control, quality adjustment, and data statistics functions. Compared with directly transmitting video data through the virtual machine, it can better meet the diverse requirements for video data in complex application scenarios;

[0045] 3. After the master machine terminal receives the video data, it first forwards it to the corresponding slave machine virtual machine terminal according to the signaling interaction result, and then forwards it to the corresponding slave machine terminal through the local network resources via the data transmission protocol. By combining the signaling interaction result for two - stage forwarding, the present invention can more precisely control the data flow direction, improve the accuracy and efficiency of data transmission, and is especially suitable for complex cloud desktop environments with multiple terminals and multiple virtual machines. Description of the Drawings

[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0047] Figure 1 Connection of a video encoding / decoding and rendering system for middleware to cooperate with local resources according to the present invention Figure 1 ;

[0048] Figure 2 Connection of a video encoding / decoding and rendering system for middleware to cooperate with local resources according to the present invention Figure 2 ;

[0049] Figure 3 Connection of a video encoding / decoding and rendering system for middleware to cooperate with local resources according to the present invention Figure 3 ;

[0050] Figure 4 Flowchart of a method for middleware to cooperate with local resources for video encoding / decoding and rendering according to the present invention. Specific embodiments

[0051] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of the present invention.

[0052] Embodiment 1: In some embodiments, please refer to Figures 1 - 4 in the accompanying drawings of the specification. A method for middleware to cooperate with local resources for video encoding / decoding and rendering includes the following steps:

[0053] Step 1: The cloud desktop software and the sub-machine terminal perform signaling interaction through the plugin middleware to determine the sub-machine virtual machine terminal information corresponding to each sub-machine terminal;

[0054] Plugin is a computer application program that interacts with the main application program to provide specific functions;

[0055] The plugin middleware refers to a plugin, which is a software component that plays the role of a bridge and expands functions.

[0056] The specific implementation steps of Step 1 are as follows:

[0057] A1. The sub - machine terminal sends a registration request containing basic information such as its own device identifier and network configuration to the plugin middleware. After receiving these requests, the plugin middleware forwards them to the cloud desktop software.

[0058] A2. The cloud desktop software verifies the identity of the sub - machine terminal. By querying a predefined list of legitimate terminals or using an encryption authentication algorithm, it ensures that the connected sub - machine terminal is authorized.

[0059] Preferably, the predefined list of legitimate terminals refers to a pre - set list containing information related to terminals that are authorized to access the system.

[0060] The content of the legitimate terminal list includes device identifier information, username, user ID, and other characteristic information.

[0061] Preferably, the encryption authentication algorithm uses the DES algorithm (Data Encryption Standard, a symmetric encryption algorithm) or the AES algorithm (Advanced Encryption Standard, a new generation symmetric encryption standard algorithm) or other algorithms.

[0062] The present invention determines the sub - machine terminal and sub - machine virtual machine terminal information through signaling interaction via the plugin middleware. In the access verification link, by combining the predefined list of legitimate terminals and the encryption authentication algorithm, especially the mentioned new generation symmetric encryption standard algorithm, it improves security compared to traditional simple identity verification methods.

[0063] A3. The plugin middleware real - time detects the network status information of the main - machine terminal, and based on the network status information, the cloud desktop software negotiates with the main - machine terminal to select an appropriate data transmission protocol.

[0064] Preferably, the network status information includes indicators such as network bandwidth, latency, and packet loss rate.

[0065] Preferably, the data transmission protocol uses TCP (Transmission Control Protocol) or IP (Internet Protocol).

[0066] By real - time detecting the network status information and negotiating to select a data transmission protocol accordingly, this dynamic adaptation to the network situation can significantly improve data transmission efficiency and stability.

[0067] A4. Through signaling interaction, the corresponding relationship between each sub - machine virtual machine and the corresponding sub - machine terminal is determined, and the corresponding relationship information is stored in the cache of the plugin middleware for use during subsequent data transmission.

[0068] Step 2: The content capture module of the cloud desktop software captures the video data of the console desktop, and then encodes the video data through the encoding module;

[0069] The specific implementation steps of Step 2 are as follows:

[0070] B1. The content capture module of the cloud desktop software is started. The content capture module periodically captures the entire screen image of the console to generate video data;

[0071] The capture frequency can be adjusted according to actual needs. For example, when performing a dynamic demonstration, the capture frequency can be appropriately increased to ensure the smoothness of the video;

[0072] B2. The content capture module transmits the video data to the encoding module in the form of raw image data;

[0073] Preferably, the encoding module can select to use a hardware encoder or a software encoder according to the system configuration;

[0074] Preferably, if the console is equipped with a dedicated graphics processing unit (GPU), the hardware encoder is preferentially used. It can utilize the parallel computing power of the GPU to achieve efficient video encoding, reduce the encoding time and CPU load; the software encoder is used when the hardware resources are limited or more flexible configuration of encoding parameters is required.

[0075] B3. According to the network status information and transmission protocol determined by the previous signaling interaction, the encoding module selects an appropriate encoding format to encode the video data during the encoding process.

[0076] Preferably, the encoding format adopts the H.265 encoding format or other encoding formats such as the H.264 encoding format.

[0077] For example, when the network bandwidth is sufficient, the H.265 encoding format can be adopted to obtain higher video quality; while when the network bandwidth is narrow, the H.264 encoding format is selected and the bit rate is appropriately reduced to ensure the stable transmission of video data.

[0078] In the video encoding and decoding process, the encoding module selects the encoding format based on the network status information and transmission protocol obtained from the signaling interaction, changing the limitation of the previous fixed encoding method, being able to better adapt to different network environments and improve the video transmission quality.

[0079] Step 3: The encoded video data is processed by the plugin middleware and data tag information is added, and then sent to the console terminal;

[0080] The specific implementation steps of Step 3 are as follows:

[0081] C1. The encoded video data is sent to the plugin middleware;

[0082] C2. The plugin middleware encapsulates the video data and adds data tag information;

[0083] Preferably, the data tag information includes the target terminal identifier, data type, timestamp, etc., so that the terminal can correctly identify and process this data, facilitating the accurate identification and processing of video data in a complex system. And the middleware dynamically adjusts the sending rate according to network congestion. These measures ensure the orderly and efficient transmission of data.

[0084] There are two benefits to processing video data through the plugin middleware:

[0085] First, reduce the network resource occupancy of the cloud desktop;

[0086] Second, reduce the performance consumption of the sub-machine virtual machine because the cloud desktop of the sub-machine no longer receives video data.

[0087] C3. The plugin middleware sends the video data to the main machine terminal through the network. During the transmission process, the plugin middleware will monitor the network status information in real time to obtain the network congestion situation, and then dynamically adjust the data sending rate according to the network congestion situation. If the plugin middleware detects network congestion, it will appropriately reduce the sending rate to avoid data loss; when the network condition improves, it will increase the sending rate to ensure the timely transmission of video data.

[0088] Step Four. After the main machine terminal receives the video data, according to the signaling interaction result, it forwards the video data to the corresponding sub-machine virtual machine terminal, and then uses the network resources of the main machine locally to forward the video data to the corresponding sub-machine terminal through the data transmission protocol;

[0089] The specific implementation steps of Step Four are as follows:

[0090] D1. After the main machine terminal receives the video data from the plugin middleware, it first parses the data tag information;

[0091] D2. The main machine terminal queries the relationship between each sub-machine virtual machine terminal and the corresponding sub-machine terminal stored during the signaling interaction process according to the target terminal identifier therein, determines the sub-machine virtual machine terminal corresponding to each video data, and sends it to the sub-machine virtual machine terminal;

[0092] D3. The main machine terminal utilizes local network resources, such as a local area network connection or a wireless access point, to forward the video data to the corresponding sub - machine terminal of the sub - machine virtual machine terminal through a data transmission protocol. During the forwarding process, the main machine terminal will cache and schedule the video data to ensure the orderly transmission of the data. If the network of a certain sub - machine terminal experiences a short - term interruption or congestion, the main machine terminal will temporarily store the video data in the local cache and wait until the network returns to normal before continuing to forward it, avoiding frame freezes caused by data loss.

[0093] Step Five: The sub - machine terminal decodes the video data using the decoding module and then transmits it to the rendering module. The rendering module displays the rendered image on the screen of the sub - machine terminal.

[0094] The specific implementation steps of Step Five are as follows:

[0095] E1. After receiving the forwarded video data, the sub - machine terminal transmits it to the local decoding module.

[0096] E2. According to the encoding format of the video data, the decoding module calls the corresponding decoding algorithm for decoding. If a hardware encoder is used for encoding, the decoding module of the sub - machine terminal can utilize the decoding ability of the local GPU to achieve a fast and efficient decoding process.

[0097] E3. The decoded video data is transmitted to the rendering module in the form of original image frames.

[0098] E4. The rendering module arranges the original image frames in chronological order and adjusts and optimizes the images according to the display device parameters of the sub - machine terminal, such as resolution, refresh rate, etc.

[0099] E5. The rendering module displays the rendered image on the screen of the sub - machine terminal, achieving the same screen broadcasting effect as the sub - machine virtual machine and providing a clear and smooth screen broadcasting experience. At the same time, the sub - machine virtual machine terminal will real - time monitor the performance metrics of the decoding and rendering processes, such as decoding frame rate, rendering latency, etc. If performance anomalies are found, it will promptly feedback to the cloud desktop software for adjustment and optimization.

[0100] Embodiment Two: In some embodiments, as Figure 3 shown, as a preferred embodiment of the present invention, a video encoding, decoding, and rendering system for middleware to cooperate with local resources includes: a main machine and multiple sub - machines;

[0101] The main machine includes cloud desktop software, a content capture module, an encoding module, a plugin middleware, a main machine terminal, and multiple sub - machine virtual machine terminals;

[0102] The cloud desktop software is used to implement signaling interaction of the cloud desktop main console terminal, process the content of the main console screen, and coordinate data transmission between different devices. It is the core control software of the entire system, responsible for managing and scheduling various resources and operations in the system;

[0103] The content capture module is mainly used to periodically capture snapshots of the main console screen, obtain the original image data of the screen, and send it to the encoding module;

[0104] The encoding module is responsible for encoding the original image data obtained by the content capture module. The encoding module can select to use a hardware encoder or a software encoder according to the system configuration, select a suitable encoding format to encode the video data during the encoding process, and then send the encoded video file to the plugin middleware;

[0105] The plugin middleware is a bridge connecting the cloud desktop software and the main console terminal, playing the role of an intermediary for signaling interaction and data transmission. It receives the registration requests sent by the sub-terminal and forwards them to the cloud desktop software, assists the cloud desktop software and the sub-terminal in negotiating and selecting a data transmission protocol, and also processes the encoded video data, adds necessary data tag information, and adjusts the data sending rate according to the network status information to ensure that the data can be accurately and efficiently transmitted between the cloud desktop software and the sub-terminal;

[0106] The main console terminal is the terminal where the main console is located. It is the initiating end and control end of the entire screen broadcast. Users perform operations such as teaching content display through the main console terminal, and the screen content will be captured by the content capture module of the cloud desktop software and then transmitted to the sub-terminal after a series of processes such as encoding;

[0107] The sub-virtual machine terminal is a complete computer system with complete hardware system functions simulated by software and running in a completely isolated environment. It uses virtualization technology to create multiple mutually isolated virtual computer environments on a physical computer. Each virtual machine can install and run its own operating system and applications, just like an independent physical computer. It is mainly used to receive instructions and data input by users and present the results processed by the computer system to the corresponding sub-terminal in a visual or other perceivable form.

[0108] The sub-terminal includes a sub-terminal, a decoding module, and a rendering module;

[0109] The sub-terminal is a device in a subordinate position in the entire system, relying on the resources and services of the main console to run. It is mainly used to receive and process the data sent by the main console and feedback the operations and requests of users to the main console;

[0110] The decoding module is responsible for restoring the video data encoded and compressed from the content on the main console screen to the original image data and sending it to the rendering module;

[0111] The rendering module processes and optimizes the decoded original image data and finally displays it on the slave console screen. It is responsible for arranging the original image data in the form of original image frames in chronological order and adjusting the display effect of the image to provide a good visual experience;

[0112] If the resolution of the display device is 1920×1080, the rendering module will adjust the resolution of the image to match it; at the same time, it will also perform optimization processing on the color, contrast, brightness, etc. of the image to make the picture clearer and more vivid.

[0113] The above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A video encoding, decoding and rendering method using middleware in collaboration with local resources, characterized in that: The following steps are involved: Step 1: The cloud desktop software and the slave terminal interact with each other through the plugin middleware to determine the slave virtual machine terminal information corresponding to each slave terminal; Step 2: The cloud desktop software starts the content capture module to capture the video data of the switchboard desktop, and then encodes the video data through the encoding module; Step 3: The encoded video data is processed by the plugin middleware and data tag information is added before being sent to the switchboard terminal; Step 4: After receiving the video data, the switchboard terminal forwards the video data to the corresponding slave virtual machine terminal according to the signaling interaction result, and then uses the local network resources of the switchboard to forward the data to the corresponding slave terminal through the data transmission protocol; Step 5: The slave terminal decodes the video data using the decoding module and transmits the decoded video data to the rendering module, and the rendering module displays the rendered image on the screen of the slave terminal.

2. The video encoding, decoding and rendering method of middleware in collaboration with local resources according to claim 1 is characterized in that: The specific implementation steps of step one are: A1. The slave terminal sends a registration request containing its own device identification and network configuration to the plugin middleware. After receiving these requests, the plugin middleware forwards them to the cloud desktop software. A2. The cloud desktop software verifies the identity of the slave terminal by querying a predefined list of legal terminals or using an encryption authentication algorithm to ensure that the connected slave terminal is authorized. A3. The plugin middleware detects the network status information of the slave terminal in real time, and based on the network status information, the cloud desktop software negotiates with the slave terminal to select the appropriate data transmission protocol; A4. Determine the correspondence between each slave virtual machine and the corresponding slave terminal through signaling interaction, and store the correspondence information in the cache of the plugin middleware.

3. The video encoding, decoding and rendering method of middleware in collaboration with local resources according to claim 2 is characterized in that: The encryption authentication algorithm adopts a symmetric encryption algorithm or a new generation of symmetric encryption standard algorithm.

4. The video encoding, decoding and rendering method of middleware in collaboration with local resources according to claim 2 is characterized in that: Network status information includes network bandwidth, latency, and packet loss rate.

5. The video encoding, decoding and rendering method of middleware in collaboration with local resources according to claim 2 is characterized in that: The data transmission protocol adopts TCP protocol or IP protocol.

6. The video encoding, decoding and rendering method of middleware in collaboration with local resources according to claim 1, characterized in that: The specific implementation steps of step 2 are: B1. The cloud desktop software starts the content capture module, which periodically collects the entire screen image of the switchboard to generate video data; B2, the content capture module transmits the video data to the encoding module in the form of original image data; B3. According to the network status information and transmission protocol determined by the previous signaling interaction, the encoding module selects an encoding format to encode the video data during the encoding process.

7. The video encoding, decoding and rendering method of middleware in collaboration with local resources according to claim 1, characterized in that: The specific implementation steps of step three are: C1. The encoded video data is sent to the plugin middleware; C2, plugin middleware encapsulates the video data and adds data tag information; C3. The plugin middleware sends the video data to the switchboard terminal through the network. During the transmission process, the plugin middleware monitors the network status information in real time to obtain the network congestion situation, and then dynamically adjusts the data transmission rate according to the network congestion situation.

8. The video encoding, decoding and rendering method of middleware in collaboration with local resources according to claim 7 is characterized in that: The data tag information includes a target terminal identifier, a data type and a timestamp.

9. The video encoding, decoding and rendering method of middleware-coordinated local resources according to claim 1, characterized in that: The specific implementation steps of step 4 are: D1. After receiving the video data from the plugin middleware, the switchboard terminal first parses the data tag information; D2. The switchboard terminal queries the relationship between each sub-machine virtual machine terminal and the corresponding sub-machine terminal stored in the signaling interaction process according to the target terminal identifier, determines the sub-machine virtual machine terminal corresponding to each video data, and sends it to the sub-machine virtual machine terminal; D3. The PBX terminal uses local network resources to forward the video data to the corresponding sub-machine terminal of the sub-machine virtual machine terminal through the data transmission protocol. During the forwarding process, the PBX terminal will cache and schedule the video data to ensure the orderly transmission of the data. If the network of a sub-machine terminal is temporarily interrupted or congested, the PBX terminal will temporarily store the video data in the local cache and wait for the network to return to normal before continuing to forward it.

10. The video encoding, decoding and rendering method of middleware in collaboration with local resources according to claim 1, characterized in that: The specific implementation steps of step five are: After receiving the forwarded video data, E1, the slave terminal transmits it to the local decoding module; E2, the decoding module calls the corresponding decoding algorithm for decoding according to the encoding format of the video data; E3, the decoded video data is transmitted to the rendering module in the form of original image frames; E4, the rendering module arranges the original image frames in chronological order and adjusts the image resolution to match the display device resolution of the slave terminal; E5. The rendering module displays the rendered image on the screen of the slave terminal. At the same time, the slave virtual machine terminal monitors the performance indicators of the decoding and rendering process in real time. If any performance abnormality is found, it will feedback to the cloud desktop software.

Images