Video processing method and system, device, and storage medium
By performing video processing on cloud devices and achieving the decoding capabilities of target video compression standards on local devices, the network bandwidth consumption problem of cloud devices when transmitting video data to local devices is solved, low-latency playback and efficient video processing are achieved, and users are enhanced Experience.
Patent Information
- Application Number
- PCT/CN2024/088049
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-09
- Filing Date
- 2024-04-16
- Publication Date
- 2024-11-14
AI Technical Summary
In the prior art, when cloud devices transmit video data to local devices, the network bandwidth consumes a large amount, resulting in high streaming delay of the browser and low streaming efficiency, making it difficult to achieve low-latency playback, affecting the user experience.
By performing video processing on cloud devices and encoding based on target video compression standards, network bandwidth consumption is reduced, and the decoding capability of target video compression standards is achieved on the playback end of local devices, reducing network bandwidth requirements, and improving video processing efficiency and user experience.
It realizes low-latency playback, reduces network bandwidth consumption and browser streaming delay, improves video processing efficiency and user experience, and is suitable for local devices in various hardware configurations.
Smart Images

Figure CN2024088049_14112024_PF_FP_ABST
Abstract
Description
Video processing method, device, system and storage medium
[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on May 9, 2023, with application number 202310530786.7 and invention name “Video Processing Method, Device, System and Storage Medium”, the entire contents of which are incorporated by reference into this application. Technical Field
[0002] The present application relates to the field of Internet technology, and in particular to a video processing method, device, system and storage medium. Background Art
[0003] With the development of the internet, terminal technologies, and communication technologies, massive amounts of video data are being uploaded from users' local devices to cloud devices for storage. In practical applications, if a user requires video processing, they can request the cloud device to process the stored video data through the browser on their local device. Video processing includes, but is not limited to, cropping, splicing, transitioning, filtering, adding special effects, watermarks, and stickers. The cloud device then pushes the processed video data to the local device's browser, which then plays it for the user.
[0004] Currently, transmitting video data of the supported video compression standards from cloud devices to the browser of local devices may consume a lot of network bandwidth, resulting in high browser streaming delay and low browser streaming efficiency, making it difficult to achieve low-latency playback, which affects the user experience.
[0005] Summary of the Invention
[0006] Multiple aspects of the present application provide a video processing method, device, system and storage medium to support the decoding capability of the playback end compatible with the target video compression standard, so as to better achieve low-latency playback.
[0007] An embodiment of the present application provides a video processing method, which is applied to a playback end of a local device, the method comprising: sending a video processing request to a cloud device, the video processing request including an identifier of first video data; receiving second video data returned by the cloud device, the second video data being obtained by encoding the first video data after video processing by the cloud device based on a target video compression standard, wherein the second video data of the target video compression standard satisfies a preset condition for network bandwidth resources when transmitted between the cloud device and the local device; decoding the second video data based on a software decoding program written in a preset programming language to obtain third video data, the software decoding program supporting decoding of video data encoded according to the target video compression standard.
[0008] An embodiment of the present application also provides a video processing method, which is applied to a cloud device, and the method includes: receiving a video processing request sent by a playback end in a local device, the video processing request including an identifier of first video data; decoding the first video data according to the video processing request to obtain multiple initial video frames; performing video processing on the multiple initial video frames to obtain multiple intermediate video frames; encoding the multiple intermediate video frames based on a target video compression standard to obtain multiple video frames; synthesizing the multiple video frames to obtain second video data; and returning the second video data to the playback end, wherein the second video data of the target video compression standard meets preset conditions for network bandwidth resources when transmitted between the cloud device and the local device.
[0009] An embodiment of the present application also provides a local device, including: a communication component, a memory and a processor; the memory is used to store a computer program; the processor is coupled to the memory, and is used to execute the computer program to perform the steps in the video processing method.
[0010] An embodiment of the present application also provides a cloud device, including: a communication component, a memory and a processor; the memory is used to store a computer program; the processor is coupled to the memory, and is used to execute the computer program to execute the steps in the video processing method.
[0011] An embodiment of the present application also provides a video processing system, including: the above-mentioned local device and the above-mentioned cloud device.
[0012] An embodiment of the present application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor is enabled to implement the steps in the video processing method. BRIEF DESCRIPTION OF THE DRAWINGS
[0013] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0014] FIG1 is a system architecture diagram of an exemplary video processing system provided in an embodiment of the present application;
[0015] FIG2 is a signaling interaction diagram of a video processing method provided in an embodiment of the present application;
[0016] FIG3 is a signaling interaction diagram of another video processing method provided in an embodiment of the present application;
[0017] FIG4 is a diagram of an application scenario provided by an embodiment of the present application;
[0018] FIG5 is a flow chart of a video processing method provided in an embodiment of the present application;
[0019] FIG6 is a flow chart of another video processing method provided in an embodiment of the present application;
[0020] FIG7 is a schematic diagram of the structure of a local device provided in an embodiment of the present application;
[0021] FIG8 is a schematic structural diagram of a cloud device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0022] To make the purpose, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the specific embodiments of this application and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.
[0023] In the embodiments of the present application, "at least one" refers to one or more, and "more" refers to two or more. "And / or" describes the access relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent three situations: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. In the text description of the present application, the character " / " generally indicates that the previous and next associated objects are in an "or" relationship. In addition, in the embodiments of the present application, "first", "second", "third", etc. are only used to distinguish the contents of different objects and have no other special meanings.
[0024] First, some terms involved in the embodiments of this application are introduced:
[0025] Cloud devices refer to devices that provide cloud services, including, but not limited to, cloud phones, cloud computers, cloud hosts, cloud servers, and cloud gaming consoles. Cloud device operating systems include, but are not limited to, Android and Linux. Linux is a multi-user, multi-tasking operating system based on the POSIX (Portable Operating System Interface of UNIX) that supports multi-threading and multiple CPUs (Central Processing Units).
[0026] Local device: refers to the user's local terminal device, including but not limited to mobile phones, computers, in-vehicle devices, and wearable devices. To distinguish them from cloud devices, mobile phones that are local devices can be referred to as local phones, computers that are local devices can be referred to as local computers, in-vehicle devices that are local devices can be referred to as in-vehicle devices, and wearable devices that are local devices can be referred to as local wearable devices. The same applies to other local devices.
[0027] H.263 is a digital video coding standard developed by ITU-T (International Telecommunication Union Telecommunication Standardization Sector). H.263 is known for its low-bitrate compression format and revolutionary performance improvements. H.263 is mainly used in Internet applications, multimedia messaging, and video conferencing.
[0028] H.264: A new video compression standard that elevates motion image compression technology to a higher level, providing high-quality image transmission over lower bandwidths. H.264 encoding saves bitrate and is highly error-resistant, making it suitable for video transmission over wireless channels with high packet loss rates and severe interference, ensuring smooth image quality. H.264 offers superior compression performance to H.263.
[0029] H.265 (HEVC, High Efficiency Video Coding): is a new video compression coding standard. H.265 is based on the video coding standard H.264, retaining some of the original technologies and using new technologies to improve and optimize certain aspects such as bit rate, encoding quality, and latency. It improves compression efficiency, enhances robustness and error recovery capabilities, reduces real-time latency, and reduces complexity.
[0030] H.266: Compared with H.265, H.266 further optimizes compression performance. While ensuring the clarity of the video remains unchanged, the storage volume of H.266 is reduced by half, the broadband traffic is saved by half, and the picture quality is more delicate.
[0031] WebSocket protocol: It is a network transmission protocol. The player and the server only need to complete a handshake to create a persistent connection between them and perform two-way data transmission.
[0032] Currently, transmitting video data of the video compression standard supported by the browser from the cloud device to the browser of the local device may consume a lot of network bandwidth, the browser's streaming delay is high, the browser's streaming efficiency is low, and it is difficult to achieve low-latency playback, which affects the user experience.
[0033] To this end, the embodiments of the present application provide a video processing method, device, system and storage medium. In this embodiment, the cloud device has the cloud editing capability to undertake video processing tasks, and the way in which the playback end of the local device requests the cloud device to perform video processing can greatly reduce the consumption of network bandwidth caused by video processing, greatly improve the overall video processing efficiency, and can also meet the video processing needs of various local devices with different hardware configurations. Furthermore, the playback end of the local device, with the support of a software decoding program written in a preset programming language, expands the decoding capability of the target video compression standard, and reduces the demand for network bandwidth resources when the video data of the target video compression standard is transmitted between the cloud device and the local device. In this way, the playback end of the local device can decode and play the video data of the target video compression standard with relatively superior performance transmitted by the cloud device. The cloud device transmitting the video data of the target video compression standard to the playback end of the local device can greatly reduce the consumption of network bandwidth, reduce the streaming delay of the playback end, improve the streaming efficiency of the playback end, better achieve low-latency playback, and improve user experience.
[0034] Figure 1 is a system architecture diagram of an exemplary video processing system provided in an embodiment of the present application. Referring to Figure 1, the system may include a local device 10 and a cloud device 20. The local device 10 is installed with a browser, and the browser has been functionally expanded and has the decoding capability of the target video compression standard. The target video compression standard has been iteratively upgraded at least once relative to the H.264 video compression standard. The target video compression standard includes, for example, but is not limited to: H.265 video compression standard, H.266 video compression standard. Of course, the browser may also have the decoding capability of various video encoding formats such as H.264, and there is no restriction on this.
[0035] In actual applications, users may have diverse video processing needs, such as but not limited to: cutting processing needs, splicing processing needs, transition processing needs, filter processing needs, adding special effects needs, adding watermarks needs, and adding stickers needs. Since the cloud device 20 has cloud editing capabilities to undertake video processing tasks, as shown in ① in Figure 1, the user can trigger the browser of the local device 10 to send a video processing request to the cloud device 20 to request the cloud device 20 to perform video processing. The cloud device 20 provides a rich variety of video data (also known as video materials) that can be processed. After receiving the video processing request, the cloud device 20 selects the video material required by the user from the rich video materials; and performs various video processing such as cutting processing, splicing processing, transition processing, filter processing, adding special effects, adding watermarks and adding stickers on the selected video material to obtain the processed video data; the cloud device 20 encodes the processed video data according to the target video compression standard to obtain video data of the target video compression standard. As shown in ② in Figure 1, the cloud device 20 sends the video data of the target video compression standard to the browser of the local device 10. The browser of the local device 10 decodes the video data of the target video compression standard with the support of the decoding capability of the target video compression standard, and plays the decoded video data.
[0036] In this embodiment, on the one hand, the cloud device 20 has the cloud editing capability to undertake video processing tasks, and the way in which the browser of the local device 10 requests the cloud device 20 to perform video processing can greatly reduce the consumption of network bandwidth caused by video processing, greatly improve the overall video processing efficiency, and can also meet the video processing needs of local devices 10 with different hardware configurations. On the other hand, the browser of the local device 10 has expanded the decoding capability of the target video compression standard, and the target video compression standard has been upgraded at least once relative to the H.264 video compression standard. In this way, the browser of the local device 10 can decode and play video data of more optimized and improved video compression standards such as H.265, H.266, etc. transmitted by the cloud device 20. The cloud device 20 transmitting video data of the target video compression standard to the browser of the local device 10 can greatly reduce the consumption of network bandwidth, reduce the browser's streaming delay, improve the browser's streaming efficiency, better achieve low-latency playback, and improve user experience.
[0037] The local device 10 can interact with the cloud device 20 via a wired network or a wireless network. For example, a wired network can include a coaxial cable, a twisted pair, an optical fiber, etc., and a wireless network can be a 2G (2nd Generation) network, a 3G (3rd Generation) network, a 4G (4th Generation) network, a 5G (5th Generation) network, a Wireless Fidelity (WIFI) network, etc. This application does not limit the specific type or form of interaction, as long as it can realize the function of interaction between the local device 10 and the cloud device 20.
[0038] The local device 10 can be hardware or software. When the local device 10 is hardware, the local device 10 is, for example, a mobile phone, a tablet computer, a desktop computer, a wearable smart device, a smart home device, etc. When the local device 10 is software, it can be installed in the hardware devices listed above. In this case, the local device 10 is, for example, multiple software modules or a single software module, etc., which is not limited by the embodiments of the present application. The cloud device 20 can be hardware or software. When the cloud device 20 is hardware, the cloud device 20 includes, for example, but is not limited to: a cloud phone, a cloud computer, a cloud host, a cloud server, or a cloud game console, etc. When the cloud device 20 is software, it can be installed in the hardware devices listed above. In this case, the cloud device 20 is, for example, multiple software modules or a single software module, etc., which is not limited by the embodiments of the present application.
[0039] It should be noted that the application scenario shown in Figure 1 is only an exemplary application scenario, and the present embodiment does not limit the application scenario. The present embodiment does not limit the devices included in Figure 1, nor does it limit the positional relationship between the devices in Figure 1.
[0040] The following describes in detail the technical solutions provided by various embodiments of the present application in conjunction with the accompanying drawings.
[0041] FIG2 is a signaling interaction diagram of a video processing method provided by an embodiment of the present application. Referring to FIG2 , the method may include the following steps:
[0042] 201. A playback terminal in a local device sends a video processing request to a cloud device, where the video processing request includes an identifier of first video data.
[0043] In this embodiment, the playback end refers to any device with a video data playback function, which may be composed of software and / or hardware. The playback end includes, for example, but is not limited to: a browser, PaaS (Platform as a Service) application service, SaaS (Infrastructure as a Service) application service, and IaaS (Software as a Service) application service.
[0044] In actual applications, before the playback end of the local device executes step 201, a network transmission protocol (Communications Protocol) connection is established between the playback end of the local device and the cloud device. Network transmission protocols include, but are not limited to, TCP (Transmission Control Protocol) and UDP (User Datagram Protocol). For example, an HTTP (Hyper Text Transfer Protocol) connection can be established between the playback end of the local device and the cloud device. The HTTP protocol is a type of TCP protocol.
[0045] Alternatively, in order to achieve data exchange between the local device's playback terminal and the cloud device with lower latency, a WebSocket connection based on the WebSocket protocol can be established between the local device's playback terminal and the cloud device. The WebSocket protocol is also a TCP protocol.
[0046] In actual applications, the local device's player may succeed or fail in establishing a WebSocket connection with the cloud device. Optionally, to improve the reliability of the connection between the local device's player and the cloud device, after the local device's player establishes a WebSocket connection with the cloud device, the local device's player determines whether the WebSocket connection with the cloud device has been successfully established. If the WebSocket connection with the cloud device has not been successfully established, the local device's player establishes an HTTP connection with the cloud device.
[0047] Specifically, the playback end of the local device will first try to establish a WebSocket connection with the cloud device. If the playback end of the local device fails to successfully establish a WebSocket connection with the cloud device, the playback end of the local device can also establish an HTTP connection with the cloud device to ensure the reliability of the connection.
[0048] Further optionally, in order to increase the possibility of the local device's playback end establishing a WebSocket connection with the cloud device, while taking into account connection efficiency and connection reliability, the local device's playback end is allowed to try to establish a WebSocket connection with the cloud device multiple times within a preset time period that can be flexibly set on demand. If the local device's playback end still fails to successfully establish a WebSocket connection with the cloud device after the preset time period, the local device's playback end can also establish an HTTP connection with the cloud device to ensure connection reliability.
[0049] Based on this, if a WebSocket connection is not successfully established with the cloud device, an optional implementation method for establishing an HTTP connection with the cloud device is: if a WebSocket connection is not successfully established with the cloud device, determine whether the preset time has arrived; if the preset time has not arrived, repeat the steps of establishing a WebSocket connection with the cloud device until a WebSocket connection is successfully established with the cloud device or the preset time has arrived; if a WebSocket connection is not successfully established with the cloud device after the preset time has arrived, establish an HTTP connection with the cloud device.
[0050] In practice, when the local device's player attempts to establish a WebSocket connection with the cloud device for the first time, a preset timer begins. If the previous attempt to establish a WebSocket connection between the local device's player and the cloud device fails before the preset timer expires, the next attempt to establish a WebSocket connection between the local device's player and the cloud device will continue until the local device's player successfully establishes a WebSocket connection with the cloud device or the preset timer expires.
[0051] In this embodiment, after a network transmission protocol connection is established between the playback end of the local device and the cloud device, if the user has a video processing requirement, the user can trigger the playback end of the local device to send a video processing request to the cloud device to request the cloud device to perform video processing. The video processing request can carry an identifier of the first video data to instruct the cloud device to perform video processing on the first video data. The first video data can be one or more video data. For example, if the video processing is a splicing operation on multiple video data, then the first video data is multiple video data. For example, if the video processing is a cutting or filtering operation on a certain video data, then the first video data can be one video data.
[0052] Further optionally, in order to better instruct the cloud device to perform video processing, the video processing request may also include video processing parameters required for video processing of the first video data. The video processing parameters are parameters that satisfy the video processing requirements and indicate video processing. For example, if the video processing is filter processing, the video processing parameters include filter parameters, the duration of the video that needs to be filtered, or which video segment needs to be filtered. For example, if the video processing is watermark processing, the video processing parameters include the font, font size, rotation angle, transparency, watermark position, the duration of the video that needs to be watermarked, or which video segment needs to be watermarked. For example, if the video processing is cutting processing, the video processing parameters include the duration of the required cut video segment, the cutting start point and the cutting end point, etc. For example, if the video processing is splicing processing, the video processing parameters include the duration of the required spliced video segment, the splicing order of each video data, etc. It is worth noting that the video processing parameters are flexibly set according to the user's video processing requirements and there is no restriction on this.
[0053] In actual applications, the video quality of the first video data to be processed may be high or low. Video quality can be characterized by indicators such as clarity, resolution, bit rate, or frame rate. Further, optionally, in order to better perform video processing, before the cloud device performs video processing, the playback end of the local device can request the cloud device to adjust the video quality of the video data to be processed, thereby ensuring that the video quality of the video data after video processing is at a high level.
[0054] Based on the above, exemplarily, before the playback end of the local device sends a video processing request to the cloud device, it can also send a video quality adjustment request to the cloud device, and the video quality adjustment request includes the identifier of the first video data and the target video quality parameter. The cloud device performs video quality detection on the first video data according to the video quality adjustment request to obtain the current video quality parameter of the first video data; if the current video quality parameter and the target video quality parameter do not meet the matching conditions, the first video data is adjusted according to the target video quality parameter, so that the current video quality parameter of the adjusted first video data is adjusted to the target video quality parameter; the cloud device returns video quality adjustment response information to the playback end of the local device, and the video quality adjustment response information is used to indicate that the current video quality parameter of the first video data has been adjusted to the target video quality parameter.
[0055] Specifically, the target video quality parameters refer to video quality parameters that meet user needs, for example, the clarity, resolution, bit rate or frame rate of the first video data that meet user needs. After receiving the video quality adjustment request sent by the playback end of the local device, the cloud device first performs video quality detection on the first video data to obtain the current video quality parameters of the first video data. The current video quality parameters include, but are not limited to: the current clarity, current resolution, current bit rate or current frame rate of the first video data, etc. Further optionally, in order to improve the accuracy of video quality detection, a machine learning model can be used to perform video quality detection on the first video data. The machine learning model is trained using massive training data and its annotated clarity, resolution, bit rate or frame rate, etc. The trained machine learning model can detect the clarity, resolution, bit rate or frame rate, etc. of the video data.
[0056] Next, the cloud device matches the target video quality parameter and the current video quality parameter of the first video data. If the target video quality parameter and the current video quality parameter meet the matching conditions, it means that the target video quality parameter and the current video quality parameter are the same or similar. In this case, there is no need to adjust the first video data according to the target video quality parameter. The cloud device can return video quality adjustment response information to the playback end of the local device. The video quality adjustment response information is used to indicate that the current video quality parameter of the first video data is already the target video quality parameter. If the target video quality parameter and the current video quality parameter do not meet the matching conditions, it means that there is a difference between the target video quality parameter and the current video quality parameter. In this case, it is necessary to adjust the first video data according to the target video quality parameter. After the adjustment, the cloud device can return video quality adjustment response information to the playback end of the local device. The video quality adjustment response information is used to indicate that the current video quality parameter of the first video data has been adjusted to the target video quality parameter.
[0057] In actual applications, matching conditions are set as needed. For example, if the difference between the target video quality parameter and the current video quality parameter falls within the numerical range set as needed, it is considered that the target video quality parameter and the current video quality parameter meet the matching conditions. If the difference between the target video quality parameter and the current video quality parameter does not fall within the numerical range set as needed, it is considered that the target video quality parameter and the current video quality parameter do not meet the matching conditions. For another example, if the difference between the target video quality parameter and the current video quality parameter is less than a preset threshold, it is considered that the target video quality parameter and the current video quality parameter meet the matching conditions. If the difference between the target video quality parameter and the current video quality parameter is greater than or equal to the preset threshold, it is considered that the target video quality parameter and the current video quality parameter do not meet the matching conditions.
[0058] 202. The cloud device decodes the first video data according to the video processing request to obtain multiple initial video frames.
[0059] 203. The cloud device processes the multiple initial video frames to obtain multiple intermediate video frames.
[0060] In this embodiment, for ease of understanding, the video frame obtained by decoding the first video data is referred to as an initial video frame, and the video frame obtained by processing the initial video frame is referred to as an intermediate video frame.
[0061] In actual applications, processes such as cutting or splicing do not involve rendering tasks, but processes such as filtering, adding special effects, and adding stickers do. Optionally, to better process video, the cloud device also provides rendering capabilities. Based on this, when the cloud device processes multiple initial video frames to obtain multiple intermediate video frames, if the video processing involves rendering tasks, the rendering program deployed in the cloud device is called to process the multiple initial video frames to obtain multiple intermediate video frames.
[0062] 204. The cloud device encodes the multiple intermediate video frames based on the target video compression standard to obtain multiple video frames.
[0063] In actual applications, the cloud device encodes multiple intermediate video frames based on the target video compression standard to obtain multiple video frames. In different application scenarios, the target video compression standard varies. It is only necessary that the network bandwidth requirement for video data using the target video compression standard when transmitted between the cloud device and the local device meets preset conditions. The preset conditions are flexibly set based on actual application needs. When the network bandwidth requirement for video data using the target video compression standard when transmitted between the cloud device and the local device meets the preset conditions, the network bandwidth requirement for video data using the target video compression standard when transmitted between the cloud device and the local device is considered to be reduced. Of course, this reduced demand for network bandwidth is relative to the network bandwidth requirement for video data using the video compression standard that the playback end originally supports decoding when transmitted between the cloud device and the local device. In other words, in comparison, the network bandwidth requirement for video data using the video compression standard that the playback end originally supports decoding is higher when transmitted between the cloud device and the local device, while the network bandwidth requirement for video data using the target video compression standard when transmitted between the cloud device and the local device is lower.
[0064] For example, if the browser originally only supports decoding video data of the H.263 video compression standard, the target video compression standard may be any of the following video compression standards, such as H.264, H.265, or H.266. In comparison, when transmitting video data of the H.263 video compression standard between the cloud device and the local device, the network bandwidth resource requirements are higher, while when transmitting video data of the H.264, H.265, or H.266 video compression standards between the cloud device and the local device, the network bandwidth resource requirements are lower.
[0065] For example, if the browser originally only supports decoding video data of the H.264 video compression standard, the target video compression standard may be any of the following: H.265 or H.266 video compression standards. In comparison, H.264 video compression standard video data requires higher network bandwidth resources when transmitted between cloud devices and local devices, while H.265 or H.266 video compression standards require lower network bandwidth resources when transmitted between cloud devices and local devices.
[0066] In actual applications, the cloud device can also encode multiple intermediate video frames using a video encoding format that the playback end originally supports decoding to obtain multiple video frames, without any restrictions. For example, the browser originally supports decoding video data of the H.264 video compression standard. The cloud device can also encode multiple intermediate video frames based on various video encoding formats such as the H.264 video encoding format to obtain multiple video frames, without any restrictions.
[0067] 205. The cloud device synthesizes the multiple video frames to obtain second video data.
[0068] In actual applications, the cloud device can directly synthesize multiple video frames to obtain the second video data. Further optionally, in order to better perform video processing and better meet the user's video processing needs, the cloud device sends a preview video frame selected from the multiple video frames to the playback end before synthesizing the multiple video frames. The playback end plays the preview video frame; in response to the preview video frame meeting the video processing requirements, the playback end sends a video synthesis instruction to the cloud device. The cloud device synthesizes the multiple video frames according to the video synthesis instruction to obtain the second video data.
[0069] Specifically, a preview video frame is a partial or complete video frame extracted from the multiple video frames to be synthesized. Before the cloud device synthesizes the multiple video frames, the playback end plays the preview video frame to the user, allowing the user to review whether the preview video frame meets the video processing requirements. For example, whether the position or effect of stickers, watermarks, or special effects added to the preview video frame meets the video processing requirements. Another example is whether the filter effects or transition effects meet the video processing requirements. Another example is whether the start and end time points of the cut meet the video processing requirements. If the preview video frame meets the video processing requirements, the user triggers the playback end of the local device to request the cloud device to perform the synthesis operation. If the preview video frame does not meet the video processing requirements, the user triggers the playback end of the local device to send a new video processing request to the cloud device again until the new preview video frame meets the video processing requirements. It is understandable that the video processing parameters in the new video processing request are adjusted based on the video processing parameters in the previous video processing request. The preview video frame obtained by the cloud device based on the new video processing request is closer to the video processing requirements than the preview video frame obtained by the previous video processing request. By continuously adjusting the video processing parameters, the video processing effect is improved and the user's satisfaction with the video data after video processing is improved.
[0070] 206. The cloud device returns the second video data to the playback end.
[0071] 207. The playback end decodes the second video data based on a software decoding program written in a preset programming language to obtain third video data.
[0072] In this embodiment, the playback end has software decoding capabilities. In addition, the playback end supports a preset programming language. The software decoding program written based on the preset programming language has the decoding capability to decode video data encoded by the target video compression standard, that is, the software decoding program supports decoding video data encoded by the target video compression standard. The preset programming language includes, for example, but is not limited to: WebAssembly (WASM). WebAssembly is a binary instruction format for a stack-based virtual machine, and is an efficient, low-level programming language with a compact binary format that can run in the playback end with performance close to that of a desktop device. It is worth noting that the developer writes a software decoding program based on the preset programming language according to the technical logic of decoding video data of the target video compression standard. For relevant content about code development, please refer to the prior art.
[0073] In this embodiment, the playback end decodes the second video data in a software decoding manner based on a software decoding program written in a preset programming language to obtain the third video data. At this point, the user has realized video processing by utilizing the editing capabilities provided by the cloud device, and the playback end is compatible with the decoding capabilities of video compression standards with relatively superior performance, which can save more network bandwidth and does not require high hardware configurations for local devices. For example, if the browser originally supports decoding capabilities for the H.264 video compression standard, the software decoding program written in a preset programming language enables the browser to be compatible with decoding capabilities for more optimized and improved video compression standards such as H.265 and H.266. Compared with the H.264 video compression standard, it can save more than 40% of network bandwidth and does not require high hardware configurations for local devices.
[0074] The technical solution provided by this embodiment is that the cloud device has the cloud editing capability to undertake video processing tasks, and the way in which the playback end of the local device requests the cloud device to perform video processing can greatly reduce the consumption of network bandwidth caused by video processing, greatly improve the overall video processing efficiency, and can also meet the video processing needs of various local devices with different hardware configurations. Furthermore, the playback end of the local device, with the support of a software decoding program written in a preset programming language, expands the decoding capability of the target video compression standard, and reduces the demand for network bandwidth resources when the video data of the target video compression standard is transmitted between the cloud device and the local device. In this way, the playback end of the local device can decode and play the video data of the target video compression standard with relatively superior performance transmitted by the cloud device. The cloud device transmitting the video data of the target video compression standard to the playback end of the local device can greatly reduce the consumption of network bandwidth, reduce the streaming delay of the playback end, improve the streaming efficiency of the playback end, better achieve low-latency playback, and improve user experience.
[0075] FIG3 is a signaling interaction diagram of another video processing method provided by an embodiment of the present application. Referring to FIG3 , the method may include the following steps:
[0076] 301. The player in the local device establishes a network nested WebSocket connection with the cloud device.
[0077] 302. The playback terminal in the local device sends a video processing request to the cloud device, where the video processing request includes an identifier of the first video data.
[0078] 303. The cloud device decodes the first video data according to the video processing request to obtain multiple initial video frames.
[0079] 304. The cloud device processes the multiple initial video frames to obtain multiple intermediate video frames.
[0080] 305. The cloud device encodes the multiple intermediate video frames based on the target video compression standard to obtain multiple video frames.
[0081] 306. The cloud device sends a preview video frame selected from the multiple video frames to the playback end.
[0082] 307. The playback end plays the preview video frame.
[0083] 308. In response to the preview video frame meeting the video processing requirement, the playback end sends a video synthesis instruction to the cloud device.
[0084] 309. The cloud device synthesizes the multiple video frames according to the video synthesis instruction to obtain second video data.
[0085] 310. The cloud device returns the second video data to the playback end.
[0086] 311. The playback end decodes the second video data based on a software decoding program written in a preset programming language to obtain third video data.
[0087] For the implementation manner and technical effects of each step in the method embodiment shown in FIG3 , reference can be made to the relevant introduction of the aforementioned embodiment, which will not be repeated here.
[0088] To better understand the technical solution provided by the embodiments of the present application, a specific scenario embodiment is described below in conjunction with Figure 4. Referring to Figure 4, the editing kernel runs on the cloud device and starts the channel service for controlling stream transmission and the push stream service for media stream transmission.
[0089] In response to the user's editing control operation, the browser sends editing control instructions to the cloud device via a WebSocket connection. The cloud device receives the editing control instructions through the channel service and sends them to the editing kernel. The editing kernel processes the original video in response to the editing control instructions. If the video processing involves rendering, it calls the cloud rendering program for cloud rendering. The streaming service captures the H.265 video frames output by the editing kernel and pushes them to the browser via a WebSocket connection. The browser previews the H.265 video frames pulled from the cloud device. If the previewed video frames meet the user's editing requirements, the user triggers the browser to send a synthesis instruction to the cloud device via a WebSocket connection. In response to the synthesis instruction, the editing kernel synthesizes the H.265 video frames obtained from the video processing based on the original video to generate a new video. This new video is then sent to the browser via the streaming service's WebSocket connection. The browser decodes and plays the new video using a WebAssembly software decoder. If the previewed video frame does not meet the user's editing needs, the browser responds to the user's new editing control operation and sends new editing control instructions to the cloud device based on the Websoket protocol connection, so that the editing kernel can respond to the new editing control instructions and re-process the new original video until the new preview video frame meets the user's editing needs.
[0090] In this embodiment, a push-pull streaming solution using WebAssembly, WebSocket, and H.265 formats is used between the cloud device and the browser. The cloud device performs video processing, rendering, and synthesis tasks, eliminating the need to run the editing kernel in the browser, perform synthesis tasks in the browser, or download and play the original video. This reduces the hardware requirements for the local device where the browser resides, and enables real-time preview and playback on the browser front end. Furthermore, the browser offers good compatibility and expands the browser's video format. Compared to H.264, H.265 codecs can save over 40% of network bandwidth, saving even more bandwidth and enabling better low-latency playback, improving the user experience.
[0091] FIG5 is a flowchart of a video processing method provided by an embodiment of the present application, which is applied to a playback terminal of a local device that supports a preset programming language. Referring to FIG5 , the method may include the following steps:
[0092] 501. Send a video processing request to a cloud device, where the video processing request includes an identifier of first video data.
[0093] 502. Receive second video data returned by the cloud device, where the second video data is obtained by encoding the first video data after video processing by the cloud device based on a target video compression standard.
[0094] The demand for network bandwidth resources when the second video data of the target video compression standard is transmitted between the cloud device and the local device meets a preset condition.
[0095] 503. Decode the second video data based on a software decoding program written in a preset programming language to obtain third video data.
[0096] The software decoding program supports decoding video data encoded by a target video compression standard.
[0097] Further optionally, before sending the video processing request to the cloud device, the above method also includes: establishing a network nested word WebSocket connection with the cloud device.
[0098] Further optionally, after establishing a network nested WebSocket connection with the cloud device, the above method also includes: determining whether a WebSocket connection is successfully established with the cloud device; if a WebSocket connection is not successfully established with the cloud device, establishing an HTTP connection with the cloud device.
[0099] Further optionally, if a WebSocket connection is not successfully established with the cloud device, an HTTP connection is established with the cloud device, including: if a WebSocket connection is not successfully established with the cloud device, determining whether a preset time has been reached; if the preset time has not been reached, repeating the steps of establishing a WebSocket connection with the cloud device until a WebSocket connection is successfully established with the cloud device or the preset time has been reached; if a WebSocket connection is not successfully established with the cloud device after the preset time has been reached, establishing an HTTP connection with the cloud device.
[0100] Further optionally, before receiving the second video data returned by the cloud device, the above method also includes: receiving a preview video frame returned by the cloud device, the preview video frame being at least part of a plurality of video frames, and the plurality of video frames being obtained by the cloud device encoding the first video data after video processing based on a target video compression standard; playing the preview video frame; and in response to the preview video frame meeting the video processing requirements, sending a video synthesis instruction to the cloud device, so that the cloud device synthesizes the plurality of video frames to obtain the second video data.
[0101] Further optionally, the above method also includes: in response to the preview video frame not meeting the video processing requirement, sending a new video processing request to the cloud device again until the new preview video frame meets the video processing requirement.
[0102] Further optionally, before sending the video processing request to the cloud device, the above method also includes: sending a video quality adjustment request to the cloud device, the video quality adjustment request including the identifier of the first video data and the target video quality parameters; receiving video quality adjustment response information returned by the cloud device, the video quality adjustment response information is used to indicate that the current video quality parameters of the first video data have been adjusted to the target video quality parameters.
[0103] For the implementation manner and technical effects of each step in the method embodiment shown in FIG5 , reference can be made to the relevant introduction of the aforementioned embodiment, which will not be repeated here.
[0104] FIG6 is a flowchart of another video processing method provided in an embodiment of the present application, which is applied to a cloud device.
[0105] Referring to FIG6 , the method may include the following steps:
[0106] 601. Receive a video processing request sent by a player in a local device, where the video processing request includes an identifier of first video data.
[0107] 602. Decode the first video data according to the video processing request to obtain a plurality of initial video frames.
[0108] 603. Perform video processing on the multiple initial video frames to obtain multiple intermediate video frames.
[0109] 604. Encode the multiple intermediate video frames based on the target video compression standard to obtain multiple video frames.
[0110] 605. Synthesize multiple video frames to obtain second video data.
[0111] 606. Return the second video data to the playback end.
[0112] Further optionally, synthesizing multiple video frames to obtain second video data includes: sending a preview video frame selected from multiple video frames to the playback end, so that the playback end can play the preview video frame; receiving a video synthesis instruction sent by the playback end in response to the preview video frame meeting the video processing requirements; and synthesizing multiple video frames according to the video synthesis instruction to obtain second video data.
[0113] Further optionally, multiple initial video frames are processed to obtain multiple intermediate video frames, including: if the video processing involves a rendering task, calling a rendering program deployed in the cloud device to process the multiple initial video frames to obtain multiple intermediate video frames.
[0114] Further optionally, before receiving the video processing request sent by the playback end, the method further includes: receiving a video quality adjustment request sent by the playback end, the video quality adjustment request including an identifier of the first video data and a target video quality parameter; performing a video quality detection on the first video data according to the video quality adjustment request to obtain a current video quality parameter of the first video data;
[0115] If the current video quality parameter and the target video quality parameter do not meet the matching conditions, the first video data is adjusted according to the target video quality parameter so that the current video quality parameter of the adjusted first video data is adjusted to the target video quality parameter; and the video quality adjustment response information is returned to the cloud device.
[0116] For the implementation manner and technical effects of each step in the method embodiment shown in FIG6 , reference can be made to the relevant introduction of the aforementioned embodiment, which will not be repeated here.
[0117] It should be noted that the execution entity of each step of the method provided in the above embodiment can be the same device, or the method can be executed by different devices. For example, the execution entity of steps 501 to 503 can be device A; for another example, the execution entity of steps 501 and 502 can be device A, and the execution entity of step 503 can be device B; and so on.
[0118] In addition, in some of the processes described in the above embodiments and the accompanying drawings, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or may be executed in parallel. The sequence numbers of the operations, such as 501, 502, etc., are only used to distinguish between different operations, and the sequence numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., and do not represent the order of precedence, nor do they limit "first" and "second" to be different types.
[0119] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.
[0120] FIG7 is a schematic diagram of the structure of a local device provided in an embodiment of the present application. As shown in FIG7 , the local device includes: a memory 71, a processor 72, and a communication component 73;
[0121] The memory 71 is used to store computer programs and can be configured to store various other data to support operations on the computing platform. Examples of such data include instructions for any application or method operating on the computing platform, contact data, phone book data, messages, pictures, videos, etc.
[0122] The memory 71 can be implemented by any type of volatile or non-volatile memory device or a combination thereof, such as static random-access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable programmable read only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.
[0123] The processor 72 is coupled to the memory 71 and is configured to execute the computer program in the memory 71 to perform the steps in the video processing method.
[0124] Furthermore, as shown in Figure 7 , the local device also includes other components, such as a display 74 , a power supply 75 , and an audio component 76 . Figure 7 only schematically illustrates some of the components, and does not imply that the local device only includes the components shown in Figure 7 . Furthermore, the components within the dashed box in Figure 7 are optional, not required, and their specific requirements may depend on the product form factor of the local device.
[0125] The detailed implementation process of the processor executing the steps in the video processing method can be found in the relevant description in the aforementioned embodiments, which will not be repeated here.
[0126] FIG8 is a schematic diagram of the structure of a cloud device provided in an embodiment of the present application. As shown in FIG8 , the cloud device includes: a memory 81, a processor 82, and a communication component 83;
[0127] The memory 81 is used to store computer programs and can be configured to store various other data to support operations on the computing platform. Examples of such data include instructions for any application or method operating on the computing platform, contact data, phone book data, messages, pictures, videos, etc.
[0128] The memory 81 can be implemented by any type of volatile or non-volatile memory device or a combination thereof, such as static random-access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable programmable read only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.
[0129] The processor 82 is coupled to the memory 81 and is configured to execute the computer program in the memory 81 to perform the steps in the video processing method.
[0130] The detailed implementation process of the processor executing the steps in the video processing method can be found in the relevant description in the aforementioned embodiments, which will not be repeated here.
[0131] Furthermore, as shown in Figure 8 , the cloud device also includes other components, such as a display 84 , a power supply 85 , and an audio component 86 . Figure 8 only schematically illustrates some of the components, and does not imply that the cloud device only includes the components shown in Figure 8 . Furthermore, the components within the dashed box in Figure 8 are optional, not mandatory, and their specific requirements depend on the product form factor of the cloud device.
[0132] The detailed implementation process of the processor executing the steps in the video processing method can be found in the relevant description in the aforementioned embodiments, which will not be repeated here.
[0133] Accordingly, an embodiment of the present application also provides a computer-readable storage medium storing a computer program, which, when executed, can implement the steps in the above method embodiment that can be executed by a cloud device or a local device.
[0134] Accordingly, an embodiment of the present application also provides a computer program product, including a computer program / instruction. When the computer program / instruction is executed by a processor, the processor is enabled to implement the steps in the above method embodiment that can be executed by a cloud device or a local device.
[0135] The above-mentioned communication component is configured to facilitate wired or wireless communication between the device where the communication component is located and other devices. The device where the communication component is located can access a wireless network based on a communication standard, such as WiFi (Wireless Fidelity), 2G (2Generation, 2nd generation), 3G (3Generation, 3rd generation), 4G (4Generation, 4th generation) / LTE (long Term Evolution, long term evolution), 5G (5Generation, 5th generation) and other mobile communication networks, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wide band (UWB) technology, Bluetooth (BT) technology and other technologies.
[0136] The above-mentioned display includes a screen, which may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor can not only sense the boundary of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.
[0137] The power supply assembly provides power to various components of the device in which the power supply assembly is located. The power supply assembly may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device in which the power supply assembly is located.
[0138] The above-mentioned audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC), and when the device where the audio component is located is in an operating mode, such as call mode, recording mode, and voice recognition mode, the microphone is configured to receive external audio signals. The received audio signal can be further stored in a memory or sent via a communication component. In some embodiments, the audio component also includes a speaker for outputting audio signals.
[0139] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.
[0140] The present application is described with reference to the flow chart and / or block diagram of the method, device (system), and computer program product according to the embodiment of the present application. It should be understood that each flow process and / or box in the flow chart and / or block diagram and the combination of the flow process and / or box in the flow chart and / or block diagram can be realized by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processing machine or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for realizing the function specified in one flow chart flow or multiple flows and / or one box or multiple boxes of the block diagram.
[0141] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.
[0142] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.
[0143] In a typical configuration, a computing device includes one or more processors (Central Processing Unit, CPU), input / output interfaces, network interfaces, and memory.
[0144] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.
[0145] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, Phase Change RAM (PRAM), Static Random-Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically-Erasable Programmable Read-Only Memory (EEPROM), flash memory or other memory technology, CD-ROM, Digital Versatile Disc (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device. According to the definition in this article, computer-readable media does not include transitory media such as modulated data signals and carrier waves.
[0146] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.
[0147] The above are merely embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.
Claims
1. A video processing method, characterized in that: A playback end applied to a local device, the playback end supporting a preset programming language, the method comprising: Sending a video processing request to a cloud device, wherein the video processing request includes an identifier of the first video data; Receiving second video data returned by the cloud device, where the second video data is obtained by encoding the first video data after video processing by the cloud device based on a target video compression standard, wherein the demand for network bandwidth resources of the second video data of the target video compression standard when transmitted between the cloud device and the local device meets a preset condition; The second video data is decoded based on a software decoding program written in the preset programming language to obtain third video data, and the software decoding program supports decoding of video data encoded by the target video compression standard.
2. The method according to claim 1, characterized in that: Before sending a video processing request to the cloud device, it also includes: Establish a network nested WebSocket connection with the cloud device.
3. The method according to claim 2, characterized in that After establishing a network nested WebSocket connection with the cloud device, it also includes: Determine whether a WebSocket connection is successfully established with the cloud device; If a WebSocket connection is not successfully established with the cloud device, an HTTP connection is established with the cloud device.
4. The method according to claim 3, characterized in that If a WebSocket connection is not successfully established with the cloud device, an HTTP connection is established with the cloud device, including: If the WebSocket connection with the cloud device is not successfully established, determine whether the preset time has expired; If the preset time has not yet arrived, repeat the step of establishing a WebSocket connection with the cloud device until a WebSocket connection is successfully established with the cloud device or the preset time has arrived; If a WebSocket connection is not successfully established with the cloud device after the preset time has expired, an HTTP connection is established with the cloud device.
5. The method according to any one of claims 1 to 4, characterized in that: Before receiving the second video data returned by the cloud device, the method further includes: Receiving a preview video frame returned by the cloud device, where the preview video frame is at least a portion of a plurality of video frames, and the plurality of video frames are obtained by encoding the first video data after video processing by the cloud device based on the target video compression standard; Play the preview video frame; In response to the preview video frame satisfying the video processing requirement, a video synthesis instruction is sent to the cloud device, so that the cloud device synthesizes the multiple video frames to obtain the second video data.
6. The method according to claim 5, characterized in that Also includes: In response to the preview video frame not meeting the video processing requirement, a new video processing request is sent to the cloud device again until the new preview video frame meets the video processing requirement.
7. The method according to any one of claims 1 to 6, characterized in that: Before sending a video processing request to the cloud device, it also includes: Sending a video quality adjustment request to the cloud device, wherein the video quality adjustment request includes an identifier of the first video data and a target video quality parameter; Video quality adjustment response information returned by the cloud device is received, where the video quality adjustment response information is used to indicate that a current video quality parameter of the first video data has been adjusted to the target video quality parameter.
8. A video processing method, characterized in that: Applied to a cloud device, the method includes: Receiving a video processing request sent by a playback end in a local device, wherein the video processing request includes an identifier of the first video data; Decoding the first video data according to the video processing request to obtain a plurality of initial video frames; Performing video processing on a plurality of initial video frames to obtain a plurality of intermediate video frames; Encoding the plurality of intermediate video frames based on a target video compression standard to obtain a plurality of video frames; synthesizing the multiple video frames to obtain second video data; The second video data is returned to the playback end, wherein the demand for network bandwidth resources of the second video data of the target video compression standard when transmitted between the cloud device and the local device meets a preset condition.
9. The method according to claim 8, characterized in that The step of synthesizing the plurality of video frames to obtain second video data includes: Sending a preview video frame selected from the plurality of video frames to the playback end, so that the playback end plays the preview video frame; Receiving a video synthesis instruction sent by the playback end in response to the preview video frame meeting the video processing requirement; The multiple video frames are synthesized according to the video synthesis instruction to obtain the second video data.
10. The method according to claim 8, characterized in that The multiple initial video frames are processed to obtain multiple intermediate video frames, including: If the video processing involves a rendering task, a rendering program deployed in the cloud device is called to process the multiple initial video frames to obtain multiple intermediate video frames.
11. The method according to any one of claims 8 to 10, characterized in that: Before receiving the video processing request sent by the player, it also includes: receiving a video quality adjustment request sent by the playback end, wherein the video quality adjustment request includes an identifier of the first video data and a target video quality parameter; Performing video quality detection on the first video data according to the video quality adjustment request to obtain current video quality parameters of the first video data; If the current video quality parameter and the target video quality parameter do not meet the matching condition, the first video data is adjusted according to the target video quality parameter so that the current video quality parameter of the adjusted first video data is The quality parameter is adjusted to the target video quality parameter; The video quality adjustment response information is returned to the cloud device.
12. A local device, characterized in that: include: communications components, memory, and processors; The memory is used to store a computer program; the processor is coupled to the memory and is used to execute the computer program to perform the steps in the method according to any one of claims 1 to 7.
13. A cloud device, characterized in that: include: communications components, memory, and processors; The memory is used to store a computer program; the processor is coupled to the memory and is used to execute the computer program to perform the steps in the method according to any one of claims 8 to 11.
14. A video processing system, characterized in that: include: The local device as described in claim 12 and the cloud device as described in claim 13.
15. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the processor is enabled to implement the steps of the method according to any one of claims 1 to 7 or the steps of the method according to any one of claims 8 to 11.
Citation Information
Patent Citations
Cross-browser H.265 video playing method
CN111083167A
Multi-picture video data processing method and device, computer equipment and storage medium
CN113301295A
Method and system for playing H.265 video stream on browser
CN113556612A
Video data processing method and system of coding technology
CN114466223A
Video processing method, device and system and storage medium
CN116567300A
Cited By
Efficient and collaborative multimedia equipment intelligent management method and system
CN119883655A
