End-cloud resource collaboration method, electronic device, and readable storage medium

Abstract

The present application discloses a method for end-cloud resource coordination, an electronic device, and a readable storage medium, including: receiving an end-cloud resource coordination request sent by a target terminal for an original video, wherein the end-cloud resource coordination request carries the video type information of the original video and the video playback capability information of the target terminal; determining the codec parameters and the corresponding video preprocessing coordination information corresponding to the original video based on the video type information and the video playback capability information; converting the original video into the corresponding target video bitstream based on the codec parameters and the video preprocessing coordination information; sending the codec parameters, the video preprocessing coordination information, and the target video bitstream to the target terminal together, so that the target terminal can convert the target video bitstream into the corresponding video to be played based on the codec parameters and the video preprocessing coordination information. The present application solves the technical problem of poor cloud game picture effects due to poor service resource management of cloud servers.

Description

Technical Field

[0001] The present application relates to the field of resource management technology, and in particular to a terminal-cloud resource collaboration method, an electronic device, and a readable storage medium. Background Art

[0002] With the continuous development of 5G technology, cloud gaming, a new type of gaming service structure, has been increasingly widely used. It uses the computing resources of cloud servers to provide users of gaming terminals with a better gaming experience. That is, the operation and rendering logic of cloud gaming are migrated to cloud servers, and gaming terminals only need to have video decoding and video playback capabilities. At present, the service resources of cloud servers are shared by various gaming terminals. In order to ensure the quality of cloud gaming, cloud servers usually perform a certain degree of content analysis and video pre-processing on the game screen frames before video encoding. However, since the resource information between the cloud server and the gaming terminals cannot be communicated with each other, the cloud server cannot customize resource configuration for each gaming terminal, and the service resources of the cloud server are limited. That is, the limited service resources of the cloud server are not fully and effectively utilized. As a result, due to the poor management of the service resources of the cloud server, the quality experience of cloud gaming cannot be further improved. That is, the current cloud gaming picture effect is poor. Summary of the invention

[0003] The main purpose of this application is to provide a terminal-cloud resource collaboration method, an electronic device and a readable storage medium, aiming to solve the technical problem in the prior art of poor cloud game picture effects caused by poor service resource management of cloud servers.

[0004] To achieve the above objectives, the present application provides a method for coordinating end-cloud resources, which is applied to a cloud server. The method for coordinating end-cloud resources includes:

[0005] Receiving a terminal-cloud resource coordination request sent by a target terminal for an original video, wherein the terminal-cloud resource coordination request carries video type information of the original video and video playback capability information of the target terminal;

[0006] Determine, according to the video type information and the video playback capability information, encoding and decoding parameters corresponding to the original video and corresponding video preprocessing coordination information;

[0007] According to the codec parameters and the video preprocessing collaborative information, the original video is converted into a corresponding target video code stream, wherein the codec parameters are used by the cloud server to encode the original video, and the video preprocessing collaborative information is used by the cloud server to determine the video preprocessing strategy of the original video before encoding the original video;

[0008] The codec parameters, the video preprocessing coordination information and the target video code stream are sent together to the target terminal, so that the target terminal can convert the target video code stream into the corresponding video to be played according to the codec parameters and the video preprocessing coordination information, wherein the codec parameters are used by the target terminal to decode the target video code stream, and the video preprocessing coordination information is used by the target terminal to determine the video preprocessing strategy of the video to be played after decoding the target video code stream.

[0009] To achieve the above objectives, the present application also provides a terminal-cloud resource collaboration method, which is applied to a target terminal. The terminal-cloud resource collaboration method includes:

[0010] Sending an end-cloud resource coordination request to a cloud server, wherein the end-cloud resource coordination request carries the video type information of the target video and the video playback capability information of the target terminal, so that the cloud server can determine the codec parameters and the corresponding video preprocessing coordination information corresponding to the original video according to the video type information and the video playback capability information, and convert the original video into a corresponding target video bitstream according to the codec parameters and the video preprocessing coordination information, wherein the codec parameters are used by the cloud server to encode the original video, and the video preprocessing coordination information is used by the cloud server to determine the video preprocessing strategy of the original video before encoding the original video;

[0011] Receiving the encoding and decoding parameters, the video preprocessing coordination information and the target video bitstream sent by the cloud server;

[0012] According to the codec parameters and the video preprocessing collaborative information, the target video stream is converted into the corresponding video to be played, wherein the codec parameters are used by the target terminal to decode the target video stream, and the video preprocessing collaborative information is used by the target terminal to determine the video preprocessing strategy of the video to be played after decoding the target video stream.

[0013] To achieve the above objectives, the present application also provides a device for coordinating end-cloud resources, which is applied to a cloud server. The device for coordinating end-cloud resources includes:

[0014] A first receiving module, configured to receive a terminal-cloud resource coordination request sent by a target terminal for an original video, wherein the terminal-cloud resource coordination request carries video type information of the original video and video playback capability information of the target terminal;

[0015] A determination module, configured to determine the encoding and decoding parameters corresponding to the original video and the corresponding video preprocessing coordination information according to the video type information and the video playback capability information;

[0016] A first conversion module, configured to convert the original video into a corresponding target video bitstream according to the codec parameters and the video preprocessing coordination information, wherein the codec parameters are used by the cloud server to encode the original video, and the video preprocessing coordination information is used by the cloud server to determine a video preprocessing strategy for the original video before encoding the original video;

[0017] The first sending module is used to send the codec parameters, the video preprocessing coordination information and the target video code stream to the target terminal, so that the target terminal can convert the target video code stream into the corresponding video to be played according to the codec parameters and the video preprocessing coordination information, wherein the codec parameters are used by the target terminal to decode the target video code stream, and the video preprocessing coordination information is used by the target terminal to determine the video preprocessing strategy of the video to be played after decoding the target video code stream.

[0018] To achieve the above objectives, the present application further provides a device for coordinating end-cloud resources, which is applied to a target terminal. The device for coordinating end-cloud resources includes:

[0019] A second sending module is used to send an end-cloud resource coordination request to the cloud server, wherein the end-cloud resource coordination request carries the video type information of the target video and the video playback capability information of the target terminal, so that the cloud server can determine the codec parameters and the corresponding video preprocessing coordination information corresponding to the original video according to the video type information and the video playback capability information, and convert the original video into the corresponding target video bitstream according to the codec parameters and the video preprocessing coordination information, wherein the codec parameters are used by the cloud server to encode the original video, and the video preprocessing coordination information is used by the cloud server to determine the video preprocessing strategy of the original video before encoding the original video;

[0020] A second receiving module is used to receive the encoding and decoding parameters, the video preprocessing coordination information and the target video code stream sent by the cloud server;

[0021] The second conversion module is used to convert the target video code stream into the corresponding video to be played based on the codec parameters and the video preprocessing collaborative information, wherein the codec parameters are used by the target terminal to decode the target video code stream, and the video preprocessing collaborative information is used by the target terminal to determine the video preprocessing strategy of the video to be played after decoding the target video code stream.

[0022] The present application also provides an electronic device, which includes: a memory, a processor, and a program of the end-cloud resource collaboration method stored in the memory and executable on the processor. When the program of the end-cloud resource collaboration method is executed by the processor, the steps of the end-cloud resource collaboration method as described above can be implemented.

[0023] The present application also provides a computer-readable storage medium, on which is stored a program for implementing the end-cloud resource collaboration method. When the program of the end-cloud resource collaboration method is executed by a processor, the steps of the end-cloud resource collaboration method as described above are implemented.

[0024] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned end-cloud resource collaboration method when executed by a processor.

[0025] The present application provides an end-cloud resource collaboration method, an electronic device and a readable storage medium, which are applied to a cloud server, that is, receiving an end-cloud resource collaboration request sent by a target terminal for an original video, wherein the end-cloud resource collaboration request carries the video type information of the original video and the video playback capability information of the target terminal; then, based on the video type information and the video playback capability information, determining the encoding and decoding parameters and the corresponding video preprocessing collaboration information corresponding to the original video, thereby achieving the purpose of interactively determining the encoding and decoding parameters and the corresponding video preprocessing collaboration information corresponding to the target terminal through information interaction between the cloud server and the target video for the target video; based on the encoding and decoding parameters and the video preprocessing collaboration information, converting the original video into the corresponding target video The video bitstream, wherein the codec parameters are used by the cloud server to encode the original video, and the video preprocessing collaborative information is used by the cloud server to determine the video preprocessing strategy of the original video before encoding the original video; the codec parameters, the video preprocessing collaborative information and the target video bitstream are sent to the target terminal together, so that the target terminal can convert the target video bitstream into a corresponding video to be played according to the codec parameters and the video preprocessing collaborative information, wherein the codec parameters are used by the target terminal to decode the target video bitstream, and the video preprocessing collaborative information is used by the target terminal to determine the video preprocessing strategy of the video to be played after decoding the target video bitstream;Since the video preprocessing collaborative information is used to determine the task allocation of the video preprocessing task of the target video in the target terminal and the cloud server, and then the cloud server and the target terminal determine whether to preprocess the video preprocessing task according to the task allocation, and since the codec parameters are used to encode the target video into the target video code stream after the cloud server determines whether to collaboratively process the target video according to the video preprocessing task, and decode the target video code stream into the target video code before the target terminal determines whether to collaboratively process the target video according to the video preprocessing task, that is, even if there is a video preprocessing task in the cloud server or the target terminal, the video preprocessing of the target video will be completed before encoding or after decoding the target video, and will not cause coding distortion to affect the image quality. At the same time, the allocation of video preprocessing tasks depends on the video playback capability information, that is, when the service resources of the target terminal are sufficient to perform a certain degree of content analysis and video processing on the target video, the video preprocessing tasks allocated according to the video playback capability will use the service resources of the target terminal to perform video preprocessing on the target video, rather than only performing video content analysis and video preprocessing before video encoding through the cloud server, that is, the purpose of effectively utilizing the idle service resources of the target terminal is achieved, and the limited service resources of the cloud server are also fully utilized, so the technical defect that the video quality experience of the target video cannot be further improved due to the poor service resource management effect of the cloud server is overcome, that is, the video picture effect of the target video is improved, so the picture effect of the cloud game picture is also improved. ; BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0027] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0028] Figure 1 This is a flowchart of the first embodiment of the end-cloud resource collaboration method of the present application;

[0029] Figure 2 A schematic diagram of a cloud gaming scenario for the end-cloud resource collaboration method of this application;

[0030] Figure 3 A flowchart of the video content analysis performed by the cloud server in the end-cloud resource collaboration method of this application;

[0031] Figure 4 A flowchart of video content analysis performed by a target terminal of the end-cloud resource collaboration method of this application;

[0032] Figure 5 A schematic diagram of the second video enhancement task of the end-cloud resource collaboration method of this application;

[0033] Figure 6 This is a flow chart of the second embodiment of the end-cloud resource collaboration method of the present application;

[0034] Figure 7 A schematic diagram of a third embodiment of the end-cloud resource collaboration device of the present application;

[0035] Figure 8 This is a schematic diagram of a fourth embodiment of the end-cloud resource collaboration device of the present application;

[0036] Fig. 9 Schematic diagram of the device structure of the hardware operating environment involved in the end-cloud resource collaboration method in the embodiment of the present application.

[0037] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0038] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the scope of protection of the present invention.

[0039] Embodiment 1

[0040] First of all, it should be understood that in the current video or image processing process in the cloud scenario, the video is usually generated on the cloud server side, and then the video image of the video is collected and then the corresponding video image processing is performed, and then the processed video image is encoded, and then decoded by the user terminal, and finally the video image is displayed. That is, the current requirements for user terminals are simplified to video decoding capabilities and video playback capabilities. For some specific user terminals, only post-processing is performed on the decoded video image, such as sharpening, blurring or noise reduction. However, since the video pre-processing work is completed by the cloud server, and the information of the cloud server and the user terminal cannot be communicated, when the cloud server performs video pre-processing, it can only preset processing parameters according to the number of user terminals carried by the cloud server or pre-process the video image according to the established processing parameters. However, due to the service performance of different user terminals There are differences in the cloud server's service resource management, which leads to differences in video quality on different user terminals. Therefore, the video quality of the user terminal is not compatible with the video playback effect expected by the user. In other words, it is easy to have poor video picture effects due to poor service resource management of the cloud server. At the same time, the idle computing resources of the user terminal are not fully utilized. Taking cloud games as an example, in order to reduce the impact of video encoding distortion on game quality, the cloud server needs to combine the visual characteristics of the human eye and perform video content analysis and video pre-processing on the cloud game image before video encoding. This will undoubtedly further increase the computing resource overhead of the cloud server. Under the limited cloud server computing resources and the increase and decrease of computing resources, the cloud server cannot provide users with a better cloud game quality experience. Therefore, there is an urgent need for a method to solve the problem of poor cloud game picture effects due to poor service resource management of the cloud server.

[0041] The present application embodiment provides a method for coordinating end-cloud resources, which is applied to a cloud server. In the first embodiment of the method for coordinating end-cloud resources of the present application, referring to Figure 1 , the end-cloud resource collaboration method includes:

[0042] Step S10, receiving a terminal-cloud resource coordination request sent by a target terminal for an original video, wherein the terminal-cloud resource coordination request carries video type information of the original video and video playback capability information of the target terminal;

[0043] Step S20, determining the encoding and decoding parameters corresponding to the original video and the corresponding video preprocessing coordination information according to the video type information and the video playback capability information;

[0044] In this embodiment, it should be noted that the target terminal and the cloud server are connected in communication, wherein the communication connection can be connected in a TCP (Transmission Control Protocol) communication connection mode, the target terminal is used to represent an intelligent terminal that needs to perform end-cloud resource collaboration, and can be an intelligent terminal with idle computing resources or an intelligent terminal without idle computing resources, specifically a smart phone, a player game terminal, and a tablet computer, etc. The cloud server is used to represent a cloud server that performs end-cloud resource collaboration, and the end-cloud resource collaboration method is applied to the video or image processing process in the cloud scene, and is used for the same cloud server and target terminal interaction protocol and data format. The cloud scene can specifically be a real-time communication (RTC) scene, and the real-time communication scene includes but is not limited to remote calls, remote offices, interactive live broadcasts, cloud games, video conferences and many other scenes. In an implementable manner, assuming that the real-time communication scene is a cloud game scene, even for user terminals with relatively limited graphics processing and data computing capabilities, the cloud server's cloud game technology can enable them to run high-quality games, refer to Figure 2 , Figure 2 To illustrate a cloud gaming scenario, user 11 and user terminal 12 interact. As cloud server 12 and user terminal 13 are connected in communication and the game runs in cloud server 12, cloud server 12 collects game video images, encodes the game video images, and transmits the code stream to user terminal 13 via a network. User terminal 13 decodes the code stream and displays the video image according to the decoding result.

[0045] In addition, it should be noted that the original video is composed of original video frames, specifically, videos that have not undergone video preprocessing and are waiting to be sent to the target terminal. The original video frames are used to represent video key frames that reflect the video content. The end-cloud resource coordination request is used to coordinate the service resources of the target terminal and the service resources of the cloud server. That is, the target terminal sends the end-cloud resource coordination request, and the cloud server coordinates the service resources of the target terminal and the service resources of the cloud server, and then through the cooperation and collaboration of the cloud server and the smart terminal, the video processing parameters that are most suitable for the characteristics of the target terminal are deployed. Therefore, when the target terminal device sends a request for end-cloud resource coordination, the end-cloud resource coordination request will carry the video type of the target video. The information and the video playback capability information of the target terminal, wherein the video playback capability information is used to characterize the video playback capability of the target video, and the video type information is used to characterize the video type of the target video. For example, in one implementable method, assuming that the target video is a cloud game video, for cloud games corresponding to different cloud game videos, the requirements for game delay are different, that is, the delay in the encoding and decoding process of the cloud game video is different. Therefore, for cloud games with high game requirements, when the cloud game screen meets the user's expected playback effect, the delay of the encoding and decoding process is minimized. For cloud games with low game requirements, when the delay of the encoding and decoding process is appropriate, the clarity of the cloud game screen is fully ensured.

[0046] In addition, it should be noted that the video playback capability information includes video codec capability information and video processing capability information, wherein the video codec capability information is used to characterize the video codec capability of the target terminal, and the video codec capability is used to determine the codec parameters of the target video during network transmission between the cloud server and the target terminal, and the video codec capability can be obtained through H265 (High Efficiency Video Coding), AV1 (AO Media Video 1, video coding format), multi-channel decoding, stream SVC (Scalable Video Coding, layered video coding), stream SCC (HEVC SCC Profile, screen content coding), YUV444, YUV422, HDR10 (High Dynamic Range 10. Whether high dynamic range imaging), high performance mode and low latency mode are supported for embodiment. Among them, YUV444 and YUV422 are two data storage formats for uncompressed video. "Y" is used to represent the grayscale value, and "U" and "V" are used to describe the image color and saturation, that is, the color of the specified pixel. Among them, the three components of YUV444 are sampled as Y:U:V=4:4:4, and the three components of YUV422 are sampled as Y:U:V=4:1:1. YUV444 has more detailed and more accurate colors. The high performance mode is a mode that can manually control the operating frequency of the chip to achieve higher performance. The video processing capability information is used to characterize the video processing capability of the target terminal. The video processing capability includes video image processing capability, video post-processing capability and resource management capability. Among them, the image processing capability can be processed by processing frame rate, processing smoothing control and image The specific information of the mode is reflected, for example, the processing frame rate can be 60fps or 120fps, the processing smoothness control can be whether to support the frame loss processing strategy, the image mode can be a cinema mode or a vivid mode, etc., and any image mode is composed of specific parameter information such as contrast, brightness and saturation. The video post-processing capability can be reflected by whether black screen detection, video sharpening, HDR display and video super-resolution are supported. Among them, video super-resolution is reflected by whether it has the corresponding specified video super-algorithm parameters, and video sharpening is reflected by whether it has the corresponding specified sharpening area and the corresponding specified sharpening algorithm parameters. The resource management capability can be reflected by specific information such as the number of decoders, the number of decoding Buffers, the number of SurfaceView instances, resource occupancy detection, CPU frequency changes, decoding chip capability changes, and device temperature changes.

[0047] In addition, it should be noted that the video processing capability is used to determine the video preprocessing coordination information, and the video preprocessing coordination information is used by the cloud server to determine the video preprocessing strategy of the original video before encoding the original video, and is used by the target terminal to determine the video preprocessing strategy of the video to be played after decoding the target video code stream. The video preprocessing strategy is used to allocate video preprocessing tasks, and the video preprocessing tasks include all the video preprocessing tasks allocated to the cloud server, all the video preprocessing tasks allocated to the target terminal, and part of the video preprocessing tasks allocated to the cloud server and part of the target terminal. That is, the video preprocessing coordination information is used to indicate the most reasonable allocation of video preprocessing tasks to the cloud server and the target terminal according to the video processing capability of the target terminal, so as to achieve the cloud server and the target terminal are respectively in the optimal video processing configuration for preprocessing the target video, which can achieve more reasonable cooperation and collaboration between the cloud server and the smart terminal, and thus provide a better video picture experience.

[0048] In addition, it should be noted that determining that the target terminal does not have the video processing capability means that the target terminal only has basic video decoding and video playback functions, and does not have the video image processing capability realized by hardware configuration. Determining that the target terminal has the video processing capability means that the target terminal has not only the basic video playback capability, but also the video image processing capability realized by hardware. According to the different video image processing capabilities, it can be divided into two situations. The first situation is that at a given video frame rate, only local area video image processing can be completed, and full image processing cannot be completed. The second situation is that at a given video frame rate, only some sub-processes of the entire video image processing task can be completed, and the entire process cannot be completed. For example, in a 1080p@60fps video application scenario, smart device A has a specific video sharpening algorithm realized by hardware, but due to limited device performance, at a video frame rate of 60fps, only a video size of 540p can be processed. At this time, it is determined that device A has partial video image processing capabilities. Having full video processing capabilities means that in addition to having basic video playback functions, it also has specific video image processing capabilities realized by hardware, and at a given video frame rate, video image processing of all areas and all processes can be completed.

[0049] As an example, steps S10 to S20 include: receiving an end-cloud resource collaboration request sent by a target terminal for a target video, and obtaining the original video corresponding to the end-cloud resource collaboration request, wherein the end-cloud resource collaboration request includes video type information of the target video and video playback capability information of the target terminal; determining the encoding and decoding parameters corresponding to the original video and the corresponding video preprocessing collaboration information based on the video type information and the video playback capability information.

[0050] The step of determining the encoding and decoding parameters and the corresponding video preprocessing coordination information corresponding to the original video according to the video type information and the video playback capability information includes:

[0051] The encoding and decoding parameters corresponding to the original video are determined according to the video type information and the video encoding and decoding capability information, and the video preprocessing coordination information corresponding to the original video is determined according to the video processing capability information.

[0052] Step S30, converting the original video into a corresponding target video bitstream according to the codec parameters and the video preprocessing collaborative information, wherein the codec parameters are used by the cloud server to encode the original video, and the video preprocessing collaborative information is used by the cloud server to determine a video preprocessing strategy for the original video before encoding the original video;

[0053] Step S40, the codec parameters, the video preprocessing coordination information and the target video code stream are sent together to the target terminal, so that the target terminal can convert the target video code stream into the corresponding video to be played according to the codec parameters and the video preprocessing coordination information, wherein the codec parameters are used by the target terminal to decode the target video code stream, and the video preprocessing coordination information is used by the target terminal to determine the video preprocessing strategy of the video to be played after decoding the target video code stream.

[0054] In this embodiment, it should be noted that, depending on the different device capabilities of the target terminal, the video preprocessing collaborative information allocates different video preprocessing tasks to the target terminal and the cloud server. Taking a certain video image of the target video as an example, the target terminal may have full video image processing capabilities for the video image, may have partial video image processing capabilities for the video image, or may not have video image processing capabilities for the video image. When the target terminal has full video image processing capabilities, the cloud server is only responsible for sampling and encoding the video image for the video image. After the cloud server sends the encoded video image to the target terminal, the target terminal decodes and processes the encoded video image. When the target terminal does not have video image processing capabilities, the cloud server continues to be responsible for related logic such as operation and rendering, while the target terminal is responsible for decoding and playback. When the target terminal has partial video images, the processing tasks corresponding to the video image are reasonably divided for the cloud server and the target terminal. The division of processing tasks can be based on the difference in video image processing capabilities between the cloud server and the target terminal, and the video image is processed in different regions. For example, it is assumed that the video image is divided into two major areas, A and B, wherein area A is processed by the cloud server through its own video image processing capability, and area B is processed by the target terminal through its own video image processing. Then, for the target video, if the video preprocessing task is used to instruct the cloud server not to perform collaborative processing on all target video frames constituting the target video, it can be determined that the target terminal has full video processing capability. If the video preprocessing task is used to instruct the cloud server to perform collaborative processing on all target video frames constituting the target video, it can be determined whether the target terminal does not have video processing capability according to the processing method of each target video frame by the cloud server. If any area of ​​any target frame in each of the target video frames is processed by the video processing capability of the target terminal, it can be determined that the target terminal has partial video processing capability. Otherwise, it can be determined that the target terminal does not have video processing capability. Among them, the step of the cloud server performing video image processing is completed before the cloud server performs video encoding, that is, pre-video preprocessing, and the step of the target terminal performing video image processing is completed after the target terminal performs video decoding, that is, post-video preprocessing.

[0055] As an example, steps S30 to S40 include: if the video preprocessing collaborative information is first video preprocessing collaborative information that determines that there is a video preprocessing task for the target video assigned to the cloud server, then according to the video preprocessing task corresponding to the first video preprocessing collaborative information, the target video is preprocessed, and according to the codec parameters, the target video after the video preprocessing is encoded to obtain the target video code stream; if the video preprocessing collaborative information is second video preprocessing collaborative information that determines that there is no video preprocessing task for the target video assigned to the cloud server, then according to the codec parameters, the target video after the video preprocessing is encoded to obtain the target video code stream. stream, wherein the video codec parameters may specifically be resolution, bit rate, frame rate, etc.; the codec parameters, the video preprocessing collaborative information, and the target video code stream are sent together to the target terminal, so that the target terminal can convert the target video code stream into the corresponding video to be played according to the codec parameters and the video preprocessing collaborative information, wherein the codec parameters are used by the target terminal to decode the target video code stream, and the video preprocessing collaborative information is used by the target terminal to determine the video preprocessing strategy of the video to be played after decoding the target video code stream, wherein the collaborative processing method is to perform complete video image processing on all target video frames of the target video.

[0056] The step of converting the original video into a corresponding target video bitstream according to the encoding and decoding parameters and the video preprocessing collaborative information includes:

[0057] Step A10, determining whether to collaboratively preprocess the original video according to the video preprocessing collaborative information;

[0058] Step A20: If yes, perform video preprocessing on the original video according to the video preprocessing collaborative information to obtain a preprocessed original video;

[0059] Step A30, encoding the pre-processed original video according to the encoding and decoding parameters to obtain the target video code stream;

[0060] Step A40: If not, encode the preprocessed original video according to the encoding and decoding parameters to obtain the target video code stream.

[0061] As an example, step A10 to step A40 include: determining whether to collaboratively preprocess the original video based on the video preprocessing collaborative information; if it is determined to collaboratively preprocess the original video, then performing video preprocessing on the original video based on the video preprocessing collaborative information to obtain the preprocessed original video; encoding the preprocessed original video based on the codec parameters to obtain the target video bitstream; if it is determined not to collaboratively preprocess the original video, then encoding the preprocessed original video based on the codec parameters to obtain the target video bitstream. When the cloud server does not collaboratively preprocess the original video, it means that the video playback capability information of the target terminal is sufficient to indicate that the target terminal has full video processing capabilities, and then the video preprocessing task of the original video is completely handed over to the target terminal to complete, that is, at this time, both the idle service resources of the target terminal are utilized and the service resources of the cloud server are saved, and the purpose of reasonable collaboration of the service resources between the cloud server and the target terminal for the original video based on simple information interaction is achieved.

[0062] The preprocessed original video includes a single collaborative original video and a multi-collaborative original video, and the step of performing video preprocessing on the original video according to the video preprocessing collaborative information to obtain the preprocessed original video includes:

[0063] Step B10, determining a video preprocessing task corresponding to the video preprocessing collaborative information, wherein the video preprocessing task includes a video analysis task and a video enhancement task;

[0064] Step B20: if the collaborative processing task is a multi-collaborative pre-processing task, collaborative analysis is performed on the original video according to the video analysis task to obtain collaborative analysis information;

[0065] Step B30, according to the video enhancement task, collaboratively enhance the target video to obtain the multi-collaborative original video;

[0066] Step B40: If the collaborative processing task is a single collaborative pre-processing task, the target video is collaboratively enhanced according to the video enhancement task to obtain the single collaborative original video.

[0067] In the present embodiment, it should be noted that the video preprocessing task includes a video enhancement task and a video analysis task, that is, the collaborative processing task includes a single collaborative preprocessing task and a multiple collaborative preprocessing task, the single collaborative preprocessing task is a collaborative preprocessing task that only performs video image enhancement on the target video, the multiple collaborative preprocessing tasks are collaborative preprocessing tasks that perform video content analysis and video image enhancement on the target video, the single collaborative original video is the target video after the cloud server only performs video image enhancement on the target video, and the multiple collaborative original videos are the target video after the cloud server performs video image enhancement and video image analysis on the target video, wherein, if the collaborative processing task is a multiple collaborative processing task, it can be determined whether the terminal performing the video content analysis is the cloud server or the target terminal according to the computing power of the target terminal, wherein the video content analysis is used to guide video processing collaboration and video encoding and decoding collaboration. For example, in an implementable manner, refer to Figure 3 , Figure 3 is a flowchart showing the cloud server performing video content analysis. When the cloud server performs video content analysis, that is, performs video image analysis on the target video, the steps are before the step of performing video image enhancement on the target video. Figure 4 , Figure 4 The flowchart represents the target terminal performing video content analysis. When the target terminal performs video content analysis, that is, performs video image analysis on the target video, the step is before the step of performing video image enhancement on the target video.

[0068] As an example, step B10 to step B40 include: determining the video preprocessing task corresponding to the video preprocessing collaborative information based on the computing power of the target terminal; if the computing power is not greater than the preset computing power threshold, determining the collaborative processing task to be a multi-collaborative preprocessing task; if the computing power is greater than the preset computing power threshold, determining the collaborative processing task to be a single collaborative preprocessing task; if the collaborative processing task is a multi-collaborative preprocessing task, then based on the video analysis task, performing collaborative analysis on the original video to obtain collaborative analysis information, wherein the collaborative analysis information is used to guide the cloud server to collaboratively enhance the target video; based on the video enhancement task, collaboratively enhance the target video to obtain the multi-collaborative original video; if the collaborative preprocessing task is a single collaborative preprocessing task, then based on the video enhancement task, collaboratively enhance the target video to obtain the single collaborative original video.

[0069] The single collaborative original video includes an overall collaboratively enhanced original video and a partially collaboratively enhanced original video, and the step of collaboratively enhancing the target video according to the video enhancement task to obtain the single collaborative original video includes:

[0070] Step C10: if the video enhancement task is the first video enhancement task that is completely assigned to the cloud server, then based on the first video enhancement task, the original video is overall enhanced to obtain the overall collaboratively enhanced target video;

[0071] Step C20: If the video enhancement task is the first video enhancement task that is completely assigned to the cloud server, the original video is overall enhanced according to the first video enhancement task to obtain the overall collaboratively enhanced target video.

[0072] In this embodiment, it should be noted that the first video enhancement task is used to represent that the cloud server performs overall video processing on all video images of the target video through relevant algorithms and collaborative preprocessing tasks, and the second video enhancement task is used to represent that partial video processing is performed on the video images of the target video through partial algorithms and collaborative preprocessing tasks, wherein the partial video processing can be that the target video frames of the target video exist and are processed by the cloud server.

[0073] As an example, steps C10 to C20 include: if the video enhancement task is the first video enhancement task that is completely assigned to the cloud server, then based on the first video enhancement task, the original video is overall enhanced to obtain the overall collaboratively enhanced target video, wherein the first video enhancement task is used to characterize a video enhancement task in which there is no video enhancement subtask; if the video enhancement task is the first video enhancement task that is completely assigned to the cloud server, then based on the first video enhancement task, the original video is overall enhanced to obtain the overall collaboratively enhanced target video.

[0074] The step of partially enhancing the original video according to the second video enhancement task to obtain the partially collaboratively enhanced original video includes:

[0075] Step D10, obtaining a video enhancement subtask corresponding to the second video enhancement task;

[0076] Step D20, based on the video enhancement subtask, partially enhance the original video to obtain the partially collaboratively enhanced target video.

[0077] In this embodiment, it should be noted that the second video enhancement task is used to characterize a video enhancement task with a video enhancement subtask, and the video enhancement subtask can be independently deployed on a cloud server or a target terminal for a separate enhancement task according to an algorithm. For example, in an implementable manner, refer to Figure 5 , Figure 5: is a schematic diagram representing a second video enhancement task, wherein the second video enhancement task B is composed of a video enhancement subtask b, a video enhancement subtask c and a video enhancement subtask d. The video enhancement subtask b processes the video image of the target video by processing algorithm x, the collaborative processing subtask c processes the video image of the target video by processing algorithm y, and the collaborative processing task d processes the video image of the target video by processing algorithm z. If some video enhancement subtasks of the second enhanced video task are run on a cloud server and some are run on a target terminal, then it is possible to distinguish whether the processing algorithm corresponding to the video enhancement subtask is run on the cloud server or on the target terminal according to the computing power of the target terminal. When all the second video enhancement tasks are deployed on the cloud server, the target terminal does not have video processing capabilities. When there is a video enhancement subtask deployed on the cloud server, the target terminal has video processing capabilities.

[0078] As an example, step D10 to step D40 include: obtaining a video enhancement subtask corresponding to the video preprocessing collaborative information; based on the video enhancement subtask, partially enhancing the original video to obtain the partially collaboratively enhanced target video.

[0079] Wherein, after the step of sending the encoding and decoding parameters, the video preprocessing coordination information and the target video code stream to the target terminal, the terminal-cloud resource coordination method further includes:

[0080] Step E10: if the current operation state parameter sent by the target terminal is received, predict whether to adjust the encoding and decoding parameters and the video preprocessing coordination information within a preset operation time period according to the current operation state parameter;

[0081] Step E20: If the current operating state parameter sent by the target terminal is received, predict whether to adjust the encoding and decoding parameters and the video preprocessing coordination information within a preset operating time period based on the current operating state parameter.

[0082] In this embodiment, it should be noted that when the target terminal plays the target video, the target terminal will report its own operating status to the cloud server in real time. Therefore, the current operating status parameters are used to characterize the current operating status. The current operating status parameters include CPU / GPU frequency information, hardware temperature information, power consumption information, etc. After receiving the current operating status parameters, the cloud server will intelligently predict the probability of freezes in the future time period. When the hardware resources cannot meet the performance requirements, it can interact with the cloud server to dynamically adjust the encoding and decoding parameters and video preprocessing collaborative information to ensure the video quality experience.

[0083] As an example, steps E10 to E20 include: if the current operating status parameter sent by the target terminal is received, determining whether the current operating status parameter is greater than a preset current operating status parameter threshold, and if so, predicting that the video playback collaborative strategy will be adjusted within a preset operating time period, wherein it can be set that when any operating parameter is greater than the preset operating parameter threshold, the operating status parameter is deemed to be greater than the preset current operating status parameter threshold, and the preset operating time period is set by the user; if it is predicted that the video playback collaborative strategy will be adjusted within the preset operating time period, adjusting the codec parameters to obtain the adjusted codec parameters, and adjusting the video preprocessing collaborative information to obtain the adjusted video preprocessing collaborative information, and returning to the execution step: based on the codec parameters and the video preprocessing collaborative information, converting the original video into a corresponding target video code stream, wherein the adjustment method can be a method of adjusting according to the idle computing resources of the target terminal.

[0084] The embodiment of the present application provides a method for end-cloud resource collaboration, that is, receiving an end-cloud resource collaboration request sent by a target terminal for an original video, wherein the end-cloud resource collaboration request carries the video type information of the original video and the video playback capability information of the target terminal; then determining the codec parameters and the corresponding video preprocessing collaboration information corresponding to the original video based on the video type information and the video playback capability information, thereby achieving the purpose of interactively determining the codec parameters and the corresponding video preprocessing collaboration information corresponding to the target terminal through information interaction between the cloud server and the target video for the target video; converting the original video into the corresponding target video bitstream based on the codec parameters and the video preprocessing collaboration information, wherein, The codec parameters are used by the cloud server to encode the original video, and the video preprocessing collaborative information is used by the cloud server to determine the video preprocessing strategy of the original video before encoding the original video; the codec parameters, the video preprocessing collaborative information and the target video code stream are sent together to the target terminal, so that the target terminal can convert the target video code stream into a corresponding video to be played according to the codec parameters and the video preprocessing collaborative information, wherein the codec parameters are used by the target terminal to decode the target video code stream, and the video preprocessing collaborative information is used by the target terminal to determine the video preprocessing strategy of the video to be played after decoding the target video code stream;Since the video preprocessing collaborative information is used to determine the task allocation of the video preprocessing task of the target video in the target terminal and the cloud server, and then the cloud server and the target terminal determine whether to preprocess the video preprocessing task according to the task allocation, and since the codec parameters are used to encode the target video into the target video code stream after the cloud server determines whether to collaboratively process the target video according to the video preprocessing task, and decode the target video code stream into the target video code before the target terminal determines whether to collaboratively process the target video according to the video preprocessing task, that is, even if there is a video preprocessing task in the cloud server or the target terminal, the video preprocessing of the target video will be completed before encoding or after decoding the target video, and will not cause coding distortion to affect the image quality. At the same time, the allocation of video preprocessing tasks depends on the video playback capability information, that is, when the service resources of the target terminal are sufficient to perform a certain degree of content analysis and video processing on the target video, the video preprocessing tasks allocated according to the video playback capability will use the service resources of the target terminal to perform video preprocessing on the target video, rather than only performing video content analysis and video preprocessing before video encoding through the cloud server, that is, the purpose of effectively utilizing the idle service resources of the target terminal is achieved, and the limited service resources of the cloud server are also fully utilized, so the technical defect that the video quality experience of the target video cannot be further improved due to the poor service resource management effect of the cloud server is overcome, that is, the video picture effect of the target video is improved, so the picture effect of the cloud game picture is also improved. ;

[0085] Embodiment 2

[0086] Further, refer to Figure 6 In another embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above description and will not be described in detail later. On this basis, the end-cloud resource collaboration method is applied to the target terminal and further includes:

[0087] Step F10, sending an end-cloud resource coordination request to the cloud server, wherein the end-cloud resource coordination request carries the video type information of the target video and the video playback capability information of the target terminal, so that the cloud server can determine the codec parameters and the corresponding video preprocessing coordination information corresponding to the original video according to the video type information and the video playback capability information, and convert the original video into the corresponding target video bitstream according to the codec parameters and the video preprocessing coordination information, wherein the codec parameters are used by the cloud server to encode the original video, and the video preprocessing coordination information is used by the cloud server to determine the video preprocessing strategy of the original video before encoding the original video;

[0088] Step F20, receiving the encoding and decoding parameters, the video preprocessing coordination information and the target video code stream sent by the cloud server;

[0089] Step F30, converting the target video stream into the corresponding video to be played based on the codec parameters and the video preprocessing collaborative information, wherein the codec parameters are used by the target terminal to decode the target video stream, and the video preprocessing collaborative information is used by the target terminal to determine the video preprocessing strategy of the video to be played after decoding the target video stream.

[0090] In this embodiment, it should be noted that the video display instruction is used to trigger the target terminal to automatically collect the video type information of the target video and the video playback capability information of the target terminal, and then generate an end-cloud resource collaboration request, which can be specifically triggered by the user through voice or buttons, etc. For the target video after collaborative processing, it can be transmitted to the display of the target terminal through the I / O interface, and then displayed and played based on the display. The target video decoded by the target terminal can be a target video that the cloud server has not performed a video analysis task and a video enhancement task, or a target video that has not performed a video analysis task but has performed a video enhancement task, or a target video that has performed a video analysis task but has not performed a video enhancement task. The video enhancement tasks can be a first video enhancement task and a second video enhancement task. The video to be played is a video waiting to be played on a preset display interface of the target terminal. After obtaining the video to be played, the video to be played can be played through a video playback instruction.

[0091] As an example, steps F10 to F40 include: if a video display instruction input by a user for a target video is detected, a terminal-cloud resource coordination request is sent to the cloud server, wherein the terminal-cloud resource coordination request carries the video type information of the target video and the video playback capability information of the target terminal, so that the cloud server can determine the codec parameters and the corresponding video preprocessing coordination information corresponding to the original video according to the video type information and the video playback capability information, and convert the original video into the corresponding target video bitstream according to the codec parameters and the video preprocessing coordination information, wherein the codec parameters are used by the cloud server to The original video is encoded, and the video preprocessing collaborative information is used by the cloud server to determine the video preprocessing strategy of the original video before encoding the original video; the encoding and decoding parameters, the video preprocessing collaborative information and the target video stream sent by the cloud server are received; according to the encoding and decoding parameters and the video preprocessing collaborative information, the target video stream is converted into a corresponding video to be played, wherein the encoding and decoding parameters are used by the target terminal to decode the target video stream, and the video preprocessing collaborative information is used by the target terminal to determine the video preprocessing strategy of the video to be played after decoding the target video stream.

[0092] In one implementable manner, the task allocation of the video preprocessing tasks determined by the video preprocessing collaborative information includes that all the video preprocessing tasks are allocated to the cloud server, all the video preprocessing tasks are allocated to the target terminal, and part of the video preprocessing tasks are allocated to the cloud server and part of them are allocated to the target terminal. The specific steps of performing video preprocessing on the target video according to the video preprocessing tasks can refer to the specific steps of performing video preprocessing on the target video according to the video preprocessing tasks by the cloud server, and will not be repeated here.

[0093] Wherein, after the step of converting the target video code stream into a corresponding video to be played according to the encoding and decoding parameters and the video preprocessing collaborative information, the end-cloud resource collaboration method further includes:

[0094] Step G10, obtaining current operating status parameters, and sending the current operating status parameters to the cloud server, so that the cloud server can predict whether to adjust the encoding and decoding parameters and the video preprocessing coordination information within a preset operating time period based on the current operating status parameters.

[0095] As an example, step G10 includes: obtaining current operating status parameters, and sending the current operating status parameters to the cloud server, so that the cloud server predicts whether to adjust the encoding and decoding parameters and the video preprocessing collaborative information within a preset operating time period based on the current operating status parameters, wherein the cloud server predicts whether to adjust the encoding and decoding parameters and the video preprocessing collaborative information within a preset operating time period based on the current operating status parameters. The specific steps of step E10 to step E20 can be referred to, and will not be repeated here.

[0096] The embodiment of the present application provides a method for end-cloud resource collaboration, which is applied to a target terminal, that is, sending an end-cloud resource collaboration request to a cloud server, wherein the end-cloud resource collaboration request carries the video type information of the target video and the video playback capability information of the target terminal, so that the cloud server can determine the codec parameters and the corresponding video preprocessing collaboration information corresponding to the original video according to the video type information and the video playback capability information, and convert the original video into the corresponding target video bitstream according to the codec parameters and the video preprocessing collaboration information, wherein the codec parameters are used by the cloud server to process the original video. The cloud server comprises a plurality of coding sections, wherein the coding section comprises a plurality of coding sections, and the plurality of coding sections are encoded by the cloud server. The plurality of coding sections are encoded by the cloud server. The plurality of coding sections are encoded by the cloud server. The plurality of coding sections are encoded by the cloud server. The plurality of coding sections are encoded by the cloud server. The plurality of coding sections are encoded by the cloud server. The plurality of coding sections are encoded by the cloud server. Since the video preprocessing collaborative information is used to determine the task allocation of the video preprocessing task of the target video in the target terminal and the cloud server, and then the cloud server and the target terminal determine whether to preprocess the video preprocessing task according to the task allocation, and since the codec parameters are used to encode the target video into the target video code stream after the cloud server determines whether to collaboratively process the target video according to the video preprocessing task, and decode the target video code stream into the target video code before the target terminal determines whether to collaboratively process the target video according to the video preprocessing task, that is, even if there is a video preprocessing task in the cloud server or the target terminal, the video preprocessing of the target video will be completed before encoding or after decoding the target video, and will not cause coding distortion to affect the image quality. At the same time, the allocation of video preprocessing tasks depends on the video playback capability information, that is, when the service resources of the target terminal are sufficient to perform a certain degree of content analysis and video processing on the target video, the video preprocessing tasks allocated according to the video playback capability will use the target terminal service resources to perform video preprocessing on the target video, rather than only using the cloud server to perform video content analysis and video preprocessing before video encoding. That is, the purpose of effectively utilizing the idle service resources of the target terminal is achieved, and the limited service resources of the cloud server are also fully utilized. Therefore, the technical defect that the video quality experience of the target video cannot be further improved due to the poor service resource management of the cloud server is overcome. That is, the video picture effect of the target video is improved, so the picture effect of the cloud game picture is also improved.

[0097] Embodiment 3

[0098] The present application also provides a device for coordinating end-cloud resources, which is applied to a cloud server. Figure 7 , the end-cloud resource collaboration device comprises:

[0099] The first receiving module 101 is configured to receive a terminal-cloud resource coordination request sent by a target terminal for an original video, wherein the terminal-cloud resource coordination request carries video type information of the original video and video playback capability information of the target terminal;

[0100] A determination module 102, configured to determine the encoding and decoding parameters corresponding to the original video and the corresponding video preprocessing coordination information according to the video type information and the video playback capability information;

[0101] A first conversion module 103, configured to convert the original video into a corresponding target video bitstream according to the codec parameters and the video preprocessing collaborative information, wherein the codec parameters are used by the cloud server to encode the original video, and the video preprocessing collaborative information is used by the cloud server to determine a video preprocessing strategy for the original video before encoding the original video;

[0102] The first sending module 104 is used to send the codec parameters, the video preprocessing coordination information and the target video code stream to the target terminal, so that the target terminal can convert the target video code stream into the corresponding video to be played according to the codec parameters and the video preprocessing coordination information, wherein the codec parameters are used by the target terminal to decode the target video code stream, and the video preprocessing coordination information is used by the target terminal to determine the video preprocessing strategy of the video to be played after decoding the target video code stream.

[0103] Optionally, the first conversion module 103 is further configured to:

[0104] Determining whether to collaboratively preprocess the original video according to the video preprocessing collaborative information;

[0105] If yes, performing video preprocessing on the original video according to the video preprocessing collaborative information to obtain a preprocessed original video;

[0106] Encoding the preprocessed original video according to the encoding and decoding parameters to obtain the target video bit stream;

[0107] If not, the preprocessed original video is encoded according to the encoding and decoding parameters to obtain the target video code stream.

[0108] Optionally, the pre-processed original video includes a single-cooperative original video and a multi-cooperative original video, and the conversion module 103 is further used for:

[0109] Determine a video preprocessing task corresponding to the video preprocessing collaborative information, wherein the video preprocessing task includes a video analysis task and a video enhancement task;

[0110] If the collaborative processing task is a multi-collaborative pre-processing task, collaborative analysis is performed on the original video according to the video analysis task to obtain collaborative analysis information;

[0111] According to the video enhancement task, collaboratively enhance the target video to obtain the multi-collaborative original video;

[0112] If the collaborative processing task is a single collaborative preprocessing task, the target video is collaboratively enhanced according to the video enhancement task to obtain the single collaborative original video.

[0113] Optionally, the first conversion module 103 is further configured to:

[0114] If the video enhancement task is the first video enhancement task that is completely assigned to the cloud server, then according to the first video enhancement task, the original video is overall enhanced to obtain the overall collaboratively enhanced target video;

[0115] If the video enhancement task is a second video preprocessing task that is partially assigned to the cloud server, then according to the second video enhancement task, the original video is partially enhanced to obtain the partially collaboratively enhanced original video.

[0116] Optionally, the first conversion module 103 is further configured to:

[0117] Obtaining a video enhancement subtask corresponding to the second video enhancement task;

[0118] According to the video enhancement subtask, the original video is partially enhanced to obtain the partially collaboratively enhanced target video.

[0119] Optionally, the end-cloud resource collaboration device is further used for:

[0120] If the current operation state parameter sent by the target terminal is received, predicting whether to adjust the encoding and decoding parameters and the video preprocessing coordination information within a preset operation time period according to the current operation state parameter;

[0121] If so, the adjusted codec parameters are used as the codec parameters, and the adjusted video preprocessing collaborative information is used as the video preprocessing collaborative information, and the execution step is returned: based on the codec parameters and the video preprocessing collaborative information, the original video is converted into the corresponding target video code stream.

[0122] The end-cloud resource collaboration device provided by the present invention adopts the end-cloud resource collaboration method in the above embodiment to solve the technical problem of poor cloud game screen effects due to poor service resource management of the cloud server. Compared with the prior art, the beneficial effects of the end-cloud resource collaboration device provided by the embodiment of the present invention are the same as the beneficial effects of the end-cloud resource collaboration method provided by the above embodiment, and other technical features in the end-cloud resource collaboration device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0123] Embodiment 4

[0124] The present application also provides a device for coordinating end-cloud resources, which is applied to a target terminal. Figure 8 , the end-cloud resource collaboration device comprises:

[0125] The second sending module 201 is used to send an end-cloud resource coordination request to the cloud server, wherein the end-cloud resource coordination request carries the video type information of the target video and the video playback capability information of the target terminal, so that the cloud server can determine the codec parameters and the corresponding video preprocessing coordination information corresponding to the original video according to the video type information and the video playback capability information, and convert the original video into the corresponding target video bitstream according to the codec parameters and the video preprocessing coordination information, wherein the codec parameters are used by the cloud server to encode the original video, and the video preprocessing coordination information is used by the cloud server to determine the video preprocessing strategy of the original video before encoding the original video;

[0126] A second receiving module 202 is used to receive the encoding and decoding parameters, the video preprocessing coordination information and the target video stream sent by the cloud server;

[0127] The second conversion module 203 is used to convert the target video code stream into the corresponding video to be played based on the codec parameters and the video preprocessing collaborative information, wherein the codec parameters are used by the target terminal to decode the target video code stream, and the video preprocessing collaborative information is used by the target terminal to determine the video preprocessing strategy of the video to be played after decoding the target video code stream.

[0128] Optionally, the end-cloud resource collaboration device is further used for:

[0129] The current operating state parameters are obtained and sent to the cloud server, so that the cloud server can predict whether to adjust the encoding and decoding parameters and the video preprocessing coordination information within a preset operating time period according to the current operating state parameters.

[0130] The end-cloud resource collaboration device provided by the present invention adopts the end-cloud resource collaboration method in the above embodiment to solve the technical problem of poor cloud game screen effects due to poor service resource management of the cloud server. Compared with the prior art, the beneficial effects of the end-cloud resource collaboration device provided by the embodiment of the present invention are the same as the beneficial effects of the end-cloud resource collaboration method provided by the above embodiment, and other technical features in the end-cloud resource collaboration device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0131] Embodiment 5

[0132] An embodiment of the present invention provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the end-cloud resource collaboration method in the above-mentioned embodiment one.

[0133] Reference below Fig. 9 , which shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present disclosure. The electronic device in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Fig. 9 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0134] like Fig. 9 As shown, the electronic device may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM) 1004. Various programs and data required for the operation of the electronic device are also stored in the RAM 1004. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus.

[0135] Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1009. The communication device can allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows an electronic device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or have alternatively.

[0136] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 1009, or installed from the storage device 1003, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0137] The electronic device provided by the present invention adopts the end-cloud resource coordination method in the above embodiment to solve the technical problem of poor cloud game screen effects due to poor service resource management of the cloud server. Compared with the prior art, the beneficial effects of the electronic device provided by the embodiment of the present invention are the same as the beneficial effects of the end-cloud resource coordination method provided by the above embodiment, and other technical features in the electronic device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0138] It should be understood that the various parts of the present disclosure can be implemented with hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0139] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

[0140] Embodiment 6

[0141] This embodiment provides a computer-readable storage medium having computer-readable program instructions stored thereon, and the computer-readable program instructions are used to execute the end-cloud resource collaboration method in the above embodiment.

[0142] The computer-readable storage medium provided in the embodiment of the present invention may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0143] The computer-readable storage medium may be included in the electronic device, or may exist independently without being installed in the electronic device.

[0144] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by an electronic device, the electronic device: receives a terminal-cloud resource coordination request sent by a target terminal for an original video, wherein the terminal-cloud resource coordination request carries the video type information of the original video and the video playback capability information of the target terminal; determines the codec parameters and the corresponding video preprocessing coordination information corresponding to the original video based on the video type information and the video playback capability information; converts the original video into a corresponding target video bitstream based on the codec parameters and the video preprocessing coordination information, wherein the codec parameters are used by the cloud server to The original video is encoded, and the video preprocessing collaborative information is used by the cloud server to determine the video preprocessing strategy of the original video before encoding the original video; the encoding and decoding parameters, the video preprocessing collaborative information and the target video code stream are sent to the target terminal together, so that the target terminal can convert the target video code stream into a corresponding video to be played according to the encoding and decoding parameters and the video preprocessing collaborative information, wherein the encoding and decoding parameters are used by the target terminal to decode the target video code stream, and the video preprocessing collaborative information is used by the target terminal to determine the video preprocessing strategy of the video to be played after decoding the target video code stream.

[0145] Or sending an end-cloud resource coordination request to the cloud server, wherein the end-cloud resource coordination request carries the video type information of the target video and the video playback capability information of the target terminal, so that the cloud server can determine the codec parameters and the corresponding video preprocessing coordination information corresponding to the original video according to the video type information and the video playback capability information, and convert the original video into a corresponding target video bitstream according to the codec parameters and the video preprocessing coordination information, wherein the codec parameters are used by the cloud server to encode the original video, and the video preprocessing coordination information is used by the cloud server to determine the video preprocessing strategy of the original video before encoding the original video; receiving the codec parameters, the video preprocessing coordination information and the target video bitstream sent by the cloud server; converting the target video bitstream into a corresponding to-be-played video according to the codec parameters and the video preprocessing coordination information, wherein the codec parameters are used by the target terminal to decode the target video bitstream, and the video preprocessing coordination information is used by the target terminal to determine the video preprocessing strategy of the to-be-played video after decoding the target video bitstream.

[0146] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0147] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0148] The modules involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of the module does not limit the unit itself in some cases.

[0149] The computer-readable storage medium provided by the present invention stores computer-readable program instructions for executing the above-mentioned end-cloud resource collaboration method, which solves the technical problem of poor cloud game graphics due to poor service resource management of the cloud server. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the embodiment of the present invention are the same as the beneficial effects of the end-cloud resource collaboration method provided by the above-mentioned embodiment, and will not be elaborated here.

[0150] Embodiment 7

[0151] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned end-cloud resource collaboration method when executed by a processor.

[0152] The computer program product provided by this application solves the technical problem of poor cloud game graphics due to poor service resource management of cloud servers. Compared with the prior art, the beneficial effects of the computer program product provided by the embodiment of the present invention are the same as the beneficial effects of the end-cloud resource collaboration method provided by the above embodiment, which will not be repeated here.

[0153] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent processing scope of the present application.

Claims

1. A method for coordinating end-cloud resources, characterized in that: Applied to a cloud server, the end-cloud resource collaboration method includes: Receiving a terminal-cloud resource coordination request sent by a target terminal for an original video, wherein the terminal-cloud resource coordination request carries video type information of the original video and video playback capability information of the target terminal; Determine, according to the video type information and the video playback capability information, encoding and decoding parameters corresponding to the original video and corresponding video preprocessing coordination information; According to the codec parameters and the video preprocessing collaborative information, the original video is converted into a corresponding target video code stream, wherein the codec parameters are used by the cloud server to encode the original video, and the video preprocessing collaborative information is used by the cloud server to determine the video preprocessing strategy of the original video before encoding the original video; The codec parameters, the video preprocessing coordination information, and the target video code stream are sent together to the target terminal, so that the target terminal can convert the target video code stream into a corresponding video to be played according to the codec parameters and the video preprocessing coordination information, wherein the codec parameters are used by the target terminal to decode the target video code stream, and the video preprocessing coordination information is used by the target terminal to determine the video preprocessing strategy of the video to be played after decoding the target video code stream; The step of converting the original video into a corresponding target video bitstream according to the encoding and decoding parameters and the video preprocessing collaborative information comprises: Determining whether to collaboratively preprocess the original video according to the video preprocessing collaborative information; If yes, performing video preprocessing on the original video according to the video preprocessing collaborative information to obtain a preprocessed original video; Encoding the preprocessed original video according to the encoding and decoding parameters to obtain the target video bit stream; If not, encoding the original video according to the encoding and decoding parameters to obtain the target video bit stream; The preprocessed original video includes a single collaborative original video and a multi-collaborative original video. The step of performing video preprocessing on the original video according to the video preprocessing collaborative information to obtain the preprocessed original video includes: Determine a video preprocessing task corresponding to the video preprocessing collaborative information, wherein the video preprocessing task includes a video analysis task and a video enhancement task; If the collaborative processing task is a multi-collaborative pre-processing task, collaborative analysis is performed on the original video according to the video analysis task to obtain collaborative analysis information; According to the video enhancement task, collaboratively enhance the target video to obtain the multi-collaborative original video; If the collaborative processing task is a single collaborative preprocessing task, the target video is collaboratively enhanced according to the video enhancement task to obtain the single collaborative original video.

2. The end-cloud resource collaboration method according to claim 1, characterized in that: The single collaborative original video includes the overall collaboratively enhanced original video and the partially collaboratively enhanced original video. The step of collaboratively enhancing the target video according to the video enhancement task to obtain the single collaborative original video comprises: If the video enhancement task is the first video enhancement task that is completely assigned to the cloud server, then according to the first video enhancement task, the original video is overall enhanced to obtain the overall collaboratively enhanced target video; If the video enhancement task is a second video preprocessing task that is partially assigned to the cloud server, then according to the second video enhancement task, the original video is partially enhanced to obtain the partially collaboratively enhanced original video.

3. The end-cloud resource collaboration method according to claim 2, characterized in that: The step of partially enhancing the original video according to the second video enhancement task to obtain the partially collaboratively enhanced original video comprises: Obtaining a video enhancement subtask corresponding to the second video enhancement task; According to the video enhancement subtask, the original video is partially enhanced to obtain the partially collaboratively enhanced target video.

4. The end-cloud resource collaboration method according to claim 1, characterized in that: After the step of sending the encoding and decoding parameters, the video preprocessing coordination information, and the target video code stream to the target terminal, the terminal-cloud resource coordination method further includes: If the current operation state parameter sent by the target terminal is received, predicting whether to adjust the encoding and decoding parameters and the video preprocessing coordination information within a preset operation time period according to the current operation state parameter; If so, the adjusted codec parameters are used as the codec parameters, and the adjusted video preprocessing collaborative information is used as the video preprocessing collaborative information, and the execution step is returned: based on the codec parameters and the video preprocessing collaborative information, the original video is converted into the corresponding target video code stream.

5. A method for coordinating end-cloud resources, characterized in that: Applied to the target terminal, the end-cloud resource collaboration method includes: Send an end-cloud resource collaboration request to the cloud server, wherein the end-cloud resource collaboration request carries the video type information of the original video and the video playback capability information of the target terminal, so that the cloud server can determine the encoding and decoding parameters and the corresponding video preprocessing collaboration information corresponding to the original video based on the video type information and the video playback capability information, and determine whether to collaboratively preprocess the original video based on the video preprocessing collaboration information; if the original video is collaboratively preprocessed, determine the video preprocessing task corresponding to the video preprocessing collaboration information, wherein the video preprocessing task includes a video analysis task and a video enhancement task; if the collaborative processing task is a multi-collaborative preprocessing task, then based on the video analysis task, perform collaborative analysis on the original video to obtain Collaborative analysis information; according to the video enhancement task, collaboratively enhance the target video to obtain multiple collaborative original videos; if the collaborative processing task is a single collaborative preprocessing task, then according to the video enhancement task, collaboratively enhance the target video to obtain a single collaborative original video; according to the codec parameters, encode the single collaborative original video or the multiple collaborative original videos to obtain the target video bitstream; if the original video is not collaboratively preprocessed, then encode the original video according to the codec parameters to obtain the target video bitstream, wherein the codec parameters are used by the cloud server to encode the original video, and the video preprocessing collaborative information is used by the cloud server to determine the video preprocessing strategy of the original video before encoding the original video; Receiving the encoding and decoding parameters, the video preprocessing coordination information and the target video bitstream sent by the cloud server; According to the codec parameters and the video preprocessing collaborative information, the target video stream is converted into the corresponding video to be played, wherein the codec parameters are used by the target terminal to decode the target video stream, and the video preprocessing collaborative information is used by the target terminal to determine the video preprocessing strategy of the video to be played after decoding the target video stream.

6. The end-cloud resource collaboration method according to claim 5, characterized in that: After the step of converting the target video code stream into a corresponding video to be played according to the encoding and decoding parameters and the video preprocessing coordination information, the end-cloud resource coordination method further includes: The current operating state parameters are obtained and sent to the cloud server, so that the cloud server can predict whether to adjust the encoding and decoding parameters and the video preprocessing coordination information within a preset operating time period according to the current operating state parameters.

7. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the steps of the end-cloud resource collaboration method described in any one of claims 1 to 4 or 5 to 6.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program for implementing the end-cloud resource collaboration method, and the program for implementing the end-cloud resource collaboration method is executed by a processor to implement the steps of the end-cloud resource collaboration method as described in any one of claims 1 to 4 or 5 to 6.

Images