Video file processing method, device and system

[0086] Example one

[0087] See figure 1 , Which shows a video file processing method provided by an embodiment of the present invention, which can be applied to the cloud desktop server; the method can include:

[0088] S101: intercept a video file stream to be played;

[0089] S102: Separate the video file stream according to the data type to obtain a separated data stream;

[0090] S103: Send the separated data stream through the channel corresponding to the separated data stream;

[0091] for figure 1 The technical solution shown, it should be noted that after the cloud desktop server intercepts the video file stream, it does not decode the video file stream, but only separates it according to the data type, and sends the separated data stream to Cloud terminal, so that the cloud terminal decodes and plays the separated data stream according to the local hardware decoding ability, reduces the operating pressure of the cloud desktop server and reduces the traffic between the cloud terminal and the cloud desktop server, and can adapt to different cloud desktops Protocol video playback.

[0092] Exemplarily, the interception of the video file stream to be played specifically includes:

[0093] When the play instruction is acquired, the video file to be played indicated by the play instruction is intercepted.

[0094] It should be noted that in the specific implementation process of this example, taking the windows system as an example, a system hook can be injected into the cloud desktop server through a system plug-in. When the cloud desktop server receives an instruction from the user to play a video, the system hook is triggered , Intercept the video file to be played. Here, the system hook Hook is a very important system interface, which can monitor various event messages in the system or process, intercept and process messages sent to the target window. Whenever a specific message is sent, the system hook can capture the specific message before it reaches the destination address, that is, the hook function first gets the control right. At this time, the system hook can either process or change the specific message message, or continue to deliver the message without processing, or force the end of the message delivery. Simply put, the system hook can monitor the messages sent by all processes, intercept the messages and perform subsequent processing;

[0095] Take the Android system as an example. Since the Android system is based on the Linux open source system, the pstrace mechanism can be used to replace the existing function interface by injecting the pre-written dynamic link library to enter the processing process of the video file, and then intercept the waiting to be played Video files.

[0096] Exemplarily, the separating the video file stream according to the data type to obtain the separated data stream specifically includes:

[0097] Construct the player;

[0098] And, separating the video file stream into an audio stream, a video stream, and a subtitle stream by the player.

[0099] Correspondingly, further, the sending the separated data stream through the channel corresponding to the separated data stream specifically includes:

[0100] Respectively establishing transmission channels corresponding to the audio stream, the video stream, and the subtitle stream;

[0101] And, sending the identification of the transmission channel corresponding to the audio stream, the video stream, and the subtitle stream;

[0102] And, the audio stream, the video stream, and the subtitle stream are respectively sent through a transmission channel corresponding to the audio stream, the video stream, and the subtitle stream.

[0103] For the above example, it should be noted that after intercepting the video file to be played, a simple player can be constructed, and the role of the player is mainly to separate and transmit the video file stream. For example, on the Windows platform, a player can be built through the DirectShow framework. The player mainly includes a splitter and a renderer. The renderer does not perform actual display but is used to transmit the separated video stream.

[0104] On the Android platform or the IOS platform, the player can be built through the gstreamer framework. The player also mainly includes a splitter and a renderer. The renderer is a modified gstreamer component, which is mainly used to transmit the separated video stream.

[0105] Take separation as an example. The video file stream usually includes audio stream and video stream, and for non-native language video files, it can also include subtitle stream. Therefore, the constructed player can separate the video file stream into audio stream, video stream and subtitle stream. . After the separation of the video file stream is completed, the constructed player may need to transmit the separated audio stream, video stream, and subtitle stream to the cloud terminal. In the specific implementation process, the cloud desktop server and the cloud terminal establish a TCP channel corresponding to the separated audio stream, video stream, and subtitle stream, and notify the cloud terminal of the identification of each TCP channel, such as the port number, so that the cloud desktop server can The audio stream, video stream, and subtitle stream are transmitted through the established TCP channel; and the cloud terminal can also receive the separated audio stream, video stream, and subtitle stream from the corresponding TCP channel according to the identification of each TCP channel, such as the port number.

[0106] Understandably, the data stream transmitted in the TCP channel is only the separated data, that is, the data before the video decoding. The amount of data is less than the video display screen, so the data transmission between the cloud terminal and the cloud desktop server is reduced. Of traffic.

[0107] Exemplarily, when the cloud desktop server sends the separated data stream, it can also send play instruction information to the cloud terminal; wherein the play instruction information is used to instruct the cloud terminal to play the video when the video is played. Play parameters such as position and window size. Therefore, after receiving the play instruction, the cloud terminal can play the video according to the corresponding position and size of the local current main window of the play parameter. It is understandable that the cloud desktop server can send the playback instruction information along with each video frame of the video stream, so that the cloud terminal can control the playback parameters of the cloud terminal during video playback in real time.

[0108] Exemplarily, before the implementation of the above technical solution, it is also necessary to include a preprocessing process after the cloud terminal logs in to the cloud desktop server; this process may include: acquiring the video decoding capability of the cloud terminal. Specifically, after logging in to the cloud desktop server, the cloud terminal starts the client corresponding to the cloud desktop server in the cloud terminal, and establishes a TCP connection with the cloud desktop server through the client, so that the cloud terminal can report its own video through the TCP connection Decoding ability.

[0109] This embodiment provides a method for processing video files applied to a cloud desktop server; after the cloud desktop server intercepts the video file stream, it does not decode the video file stream, but only separates it according to data types, and Send the separated data stream to the cloud terminal, so that the cloud terminal decodes and plays the separated data stream according to the local hardware decoding capability, reducing the operating pressure of the cloud desktop server and reducing the gap between the cloud terminal and the cloud desktop server Traffic can be adapted to video playback of different cloud desktop protocols.

[0110] Example two

[0111] Based on the same technical idea as the foregoing embodiment, see figure 2 , Which shows a method for processing a video file provided by an embodiment of the present invention. The method is applied to a cloud terminal, and the method may include:

[0112] S201: Establish a transmission channel corresponding to the separated data stream;

[0113] S202: Receive a corresponding separated data stream through the transmission channel;

[0114] S203: Decode and play the separated data stream.

[0115] It should be noted that the separated data stream may include an audio stream, a video stream, and a subtitle stream; therefore, correspondingly, the establishment of the transmission channel corresponding to the separated data stream specifically includes:

[0116] Establish a transmission channel for transmitting the audio stream, the video stream, and the subtitle stream.

[0117] It should be noted that see image 3 , The specific implementation process of the above solution may include:

[0118] S301: The cloud terminal receives the identifier of the TCP channel sent by the cloud desktop server; for example, the port number;

[0119] S302: The cloud terminal establishes a TCP channel corresponding to the separated audio stream, video stream, and subtitle stream according to the TCP channel identifier and the cloud desktop server;

[0120] S303: The cloud terminal receives the audio stream, video stream, and subtitle stream transmitted by the cloud desktop server through the established TCP channel;

[0121] Understandably, since the audio stream, video stream, and subtitle stream received by the cloud terminal are not decoded, they are only separated from the video file stream. Therefore, the amount of data is less than the video display screen, thereby reducing the cloud terminal and cloud desktop server The amount of data that needs to be spent on data transmission.

[0122] S304: After receiving the separated audio stream, video stream, and subtitle stream, the cloud terminal can decode these data streams according to the local decoding capability;

[0123] Specifically, taking the video stream as an example, FFMPEG (Fast Forward Moving Picture Expert Group) can be used for decoding, VAAPI (Video Acceleration API) hardware decoding can be used for Intel chips, and VDPAU (Video Decode and Presentation API for Unix) can be used for AMD chips. ) Hardware decoding, for ARM chips, the hardware decoding library provided by the manufacturer can be used for decoding, or the GStreamer framework can be used for decoding; for windows cloud terminals, the DirectShow framework can be used for hardware decoding; the specific implementation of hardware decoding for video streams is in the field Technical common sense, this embodiment will not repeat it.

[0124] Further, in addition to the cloud terminal receiving the separated data stream, the method further includes:

[0125] Receive play instruction information;

[0126] A window for playing is established according to the playing instruction information.

[0127] Specifically, the play instruction information is used to indicate the play parameters such as the window position and window size when the cloud terminal is playing the video when the video is playing. Therefore, after receiving the play instruction, the cloud terminal can play the video according to the corresponding position and size of the local current main window of the play parameter.

[0128] It is understandable that the cloud desktop server can send the playback instruction information along with each video frame of the video stream, so that the cloud terminal can control the playback parameters of the cloud terminal during video playback in real time.

[0129] Exemplarily, before the implementation of the above technical solution, it is also necessary to include a pre-processing process after the cloud terminal logs in to the cloud desktop server; this process may include:

[0130] Establishing a connection with a cloud desktop server and reporting the cloud terminal's own video decoding capability to the cloud desktop server;

[0131] Acquire the main window of the current interface of the cloud desktop server.

[0132] It should be noted that after obtaining the main window of the current interface of the cloud desktop server, the video playback window can be a child window under the main window.

[0133] This embodiment provides a method for processing video files applied to the cloud terminal side; the cloud terminal receives the separated data stream sent by the cloud desktop server, and decodes and plays the separated data stream according to the local hardware decoding capability, Reduce the operating pressure of the cloud desktop server and reduce the traffic between the cloud terminal and the cloud desktop server, and can adapt to video playback of different cloud desktop protocols.

[0134] Example three

[0135] Based on the same technical idea as the foregoing embodiment, see Figure 4 , Which shows the detailed process of a video file processing method provided by the embodiment of the present invention; the process is applied to Figure 5 As shown in the cloud desktop system, the cloud desktop system includes: a cloud desktop server and a cloud terminal; the method may include:

[0136] S401: The cloud terminal initiates a TCP connection establishment request to the cloud desktop server;

[0137] Understandably, a client corresponding to the cloud desktop server may be installed in the cloud terminal, so that the client interacts with the cloud desktop server through the client, which is not specifically limited in this embodiment.

[0138] S402: The cloud desktop server feeds back the request response to the cloud terminal;

[0139] S403: The cloud terminal obtains the main window of the current desktop protocol;

[0140] After obtaining the main window of the current interface of the cloud desktop server, the video playback window can be a child window under the main window.

[0141] S404: The cloud terminal reports the local video decoding capability;

[0142] S405: When the cloud desktop server obtains the play instruction, trigger the system hook to intercept the video file stream to be played indicated by the play instruction;

[0143] S406: The cloud desktop server separates the video file stream to be played into an audio stream, a video stream, and a subtitle stream;

[0144] S407: The cloud terminal and the cloud desktop server respectively establish corresponding TCP transmission channels for audio streams, video streams, and subtitle streams;

[0145] Specifically, the cloud desktop server and the cloud terminal establish a TCP channel corresponding to the separated audio stream, video stream, and subtitle stream, and notify the cloud terminal of the identification of each TCP channel, such as the port number, so that the cloud desktop server can establish The obtained TCP channel transmits audio stream, video stream and subtitle stream; and the cloud terminal can also receive the separated audio stream, video stream and subtitle stream from the corresponding TCP channel according to the identification of each TCP channel, such as the port number.

[0146] S408: The cloud desktop server transmits the corresponding audio stream, video stream, and subtitle stream to the cloud terminal through the TCP transmission channel, and sends the play instruction information to the cloud terminal;

[0147] It should be noted that the playback instruction information is used to instruct the cloud terminal to play video playback parameters such as window position and window size during video playback; understandably, the cloud desktop server can follow the playback instruction information to the video stream. Each video frame is sent, so that the cloud terminal's playback parameters during video playback can be controlled in real time.

[0148] Since the audio stream, video stream, and subtitle stream received by the cloud terminal are not decoded, they are only separated from the video file stream. Therefore, the amount of data is less than the video display screen, thereby reducing the data between the cloud terminal and the cloud desktop server The amount of traffic required to transmit.

[0149] S409: The cloud terminal performs decoding operations on the audio stream, video stream, and subtitle stream, and adjusts the position and size of the local window in real time according to the playback instruction information;

[0150] S410: The cloud terminal decodes the video stream into display data and displays it in a local window established by the cloud terminal.

[0151] This embodiment provides a detailed flow of a video file processing method; after the cloud desktop server intercepts the video file stream, it does not decode the video file stream, but only separates it according to the data type, and separates the separated The data stream is sent to the cloud terminal, so that the cloud terminal decodes and plays the separated data stream according to the local hardware decoding ability, reduces the operating pressure of the cloud desktop server and reduces the traffic between the cloud terminal and the cloud desktop server, and can be adapted Video playback with different cloud desktop protocols.