Video transmission method, system, electronic device, storage medium and chip system
By encoding and decoding video frames with different resolutions during video transmission, the problem of video data transmission consuming network bandwidth is solved, achieving higher compression rates and storage space utilization.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HUAWEI TECH CO LTD
- Filing Date
- 2021-08-31
- Publication Date
- 2026-07-03
Smart Images
Figure CN115733980B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of video technology, and in particular to a video transmission method, system, electronic device, storage medium, and chip system. Background Technology
[0002] With the continuous development of streaming media, the storage space occupied by video data is constantly increasing. Correspondingly, transmitting video data over the internet is causing increasingly strained network bandwidth resources. Therefore, it is necessary to compress video data.
[0003] To alleviate the network bandwidth consumption issue during video data transmission, the sending device can currently compress the video data according to video encoding rules before sending it to the receiving device. However, even after compression, the video data still consumes a significant amount of network bandwidth during transmission. Summary of the Invention
[0004] This application provides a video transmission method, system, electronic device, storage medium, and chip system, which solves the problem that the prior art still occupies a lot of network bandwidth resources during the transmission of video data.
[0005] To achieve the above objectives, this application adopts the following technical solution:
[0006] Firstly, a video transmission method is provided, the method being applied to a video transmission system consisting of a transmitting device and a receiving device, the method comprising:
[0007] The transmitting device determines two sets of video frames based on the video data of the target video. Each set of video frames has the same resolution, and the resolution of one set of video frames is greater than that of the other set of video frames. The frame number of any video frame in one set of video frames is different from that of any video frame in the other set of video frames. Each set of video frames includes multiple video frames with discontinuous frame numbers.
[0008] The transmitting device encodes each group of video frames to obtain the encoded data corresponding to each group of video frames.
[0009] The transmitting device sends first encoded data and second encoded data corresponding to the two sets of video frames, respectively, wherein the resolution corresponding to the first encoded data is greater than the resolution corresponding to the second encoded data.
[0010] The receiving device receives the first encoded data and the second encoded data;
[0011] The receiving device decodes the first encoded data and the second encoded data respectively to obtain first decoded data corresponding to the first encoded data and second decoded data corresponding to the second encoded data, wherein the resolution of the first decoded data is greater than the resolution of the second decoded data.
[0012] The sending device determines two sets of video frames based on the video data of the target video, and encodes and compresses each set of video frames to obtain the encoded data corresponding to each set of video frames. Then, it sends the encoded data to the receiving device. By transmitting encoded data at different resolutions, the compression rate can be improved by 30%-40%, thereby effectively improving the compression rate of the target video data, reducing the network bandwidth resources occupied when transmitting video data, avoiding waste of network bandwidth resources, and reducing the cost of transmitting video data.
[0013] The receiving device decodes the first encoded data and the second encoded data to obtain the first decoded data and the second decoded data. Then, it synthesizes the first decoded data and the second decoded data to obtain the synthesized video data. The receiving device only needs to store the first encoded data and the second encoded data, which occupy less storage space, to play the video data of the target video. There is no need to store the high-resolution video data of the target video, which can reduce the storage space occupied and improve the utilization rate of storage space.
[0014] In a first possible implementation of the first aspect, the video data of the target video includes: first video data and second video data;
[0015] The transmitting device determines two sets of video frames based on the video data of the target video, including:
[0016] The transmitting device downsamples the first video data to obtain the second video data, wherein the resolution of each video frame in the first video data is higher than the resolution of each video frame in the second video data.
[0017] The transmitting device selects multiple video frames from the first video data and the second video data respectively to obtain the two sets of video frames.
[0018] In a second possible implementation of the first aspect, the resolution of each video frame in the video data of the target video is the same;
[0019] The transmitting device determines two sets of video frames based on the video data of the target video, including:
[0020] The transmitting device selects a portion of video frames from the video data of the target video to obtain a set of video frames;
[0021] The transmitting device downsamples another portion of the video frames in the target video's video data to obtain another set of video frames.
[0022] The sending device acquires the video data of the target video in different ways and encodes the video data of the target video in a way that corresponds to the acquisition method. This can improve the flexibility of acquiring video data and the flexibility of encoding the video data of the target video.
[0023] Based on the first or second possible implementation of the first aspect, in the third possible implementation of the first aspect, in the set of video frames with the larger resolution of the two sets of video frames, the difference between the frame numbers of every two adjacent video frames is m, where m is a positive integer greater than 1.
[0024] By arranging non-contiguous video frames within a set of high-resolution video frames, the storage space occupied by the set of video frames can be reduced, and the compression rate of encoding the set of video frames can be improved.
[0025] Based on any one of the first to third possible implementations of the first aspect, in the fourth possible implementation of the first aspect, the video frame with the smaller resolution among the two sets of video frames includes multiple sets of video frames with consecutive frame numbers, and the number of video frames included in each set of video frames with consecutive frame numbers is n, where n is a positive integer greater than or equal to 1.
[0026] By arranging video frames with consecutive or non-consecutive frame numbers in a group of video frames with lower resolution, the storage space occupied by the group of video frames can be reduced, and the compression rate of encoding the group of video frames can be improved.
[0027] Based on any of the above possible implementations of the first aspect, in the fifth possible implementation of the first aspect, the frame numbers of the video frames obtained after combining the two sets of video frames are consecutive.
[0028] The fact that the frame numbers of the video frames in the two sets of video frames are consecutive indicates that the two sets of video frames are complementary, which can improve the accuracy of the recovered video frames and thus improve the playback effect of the synthesized video data.
[0029] Based on any of the possible implementations of the first aspect described above, in a sixth possible implementation of the first aspect, before the transmitting device determines the two sets of video frames based on the video data of the target video, the method further includes:
[0030] The transmitting device obtains the video data of the target video from the storage space;
[0031] Alternatively, the sending device may collect video data of the target video in real time.
[0032] By using different methods to acquire video data, the flexibility of acquiring video data can be improved. Alternatively, an encoding method corresponding to the acquisition method can be used to encode the acquired video data.
[0033] Based on any of the possible implementations of the first aspect described above, in the seventh possible implementation of the first aspect, the transmitting device transmits the first encoded data and the second encoded data corresponding to the two sets of video frames respectively, including:
[0034] The transmitting device sends the first encoded data and the second encoded data corresponding to the two sets of video frames to the receiving device;
[0035] Alternatively, the sending device sends the first encoded data and the second encoded data corresponding to the two sets of video frames to the server, and the server forwards the first encoded data and the second encoded data corresponding to the two sets of video frames to the receiving device.
[0036] By using different methods to send encoded data, the flexibility of sending encoded data can be improved.
[0037] In an eighth possible implementation of the first aspect, the receiving device processes the second decoded data based on the first decoded data to obtain a restored frame with the same resolution as the first decoded data, including:
[0038] The receiving device uses a pre-set artificial intelligence (AI) super-resolution model to process the video frames of the second decoded data based on the video frames of the first decoded data, thereby obtaining the restored frame.
[0039] The receiving device uses an AI super-resolution model to process low-resolution video frames based on high-resolution video frames with adjacent frame numbers. By utilizing the high-frequency spatial information of high-resolution video frames, the complexity of the super-resolution network can be reduced, thereby reducing the operating conditions of the AI super-resolution model. This enables the AI super-resolution model to be run through portable terminals, improving the versatility of the AI super-resolution model.
[0040] Based on the eighth possible implementation of the first aspect, in the ninth possible implementation of the first aspect, the receiving device processes the video frame of the second decoded data according to the video frame of the first decoded data using a pre-set artificial intelligence (AI) super-resolution model to obtain the restored frame, including:
[0041] The receiving device inputs the first video frame of the second decoded data and at least one video frame with a frame number consecutive to the first video frame into the AI super-resolution model to obtain the restored frame corresponding to the first video frame.
[0042] During the process of processing video frames using the AI super-resolution model, the receiving device can obtain information about missing frames based on low-resolution video frames. This saves network bandwidth resources, reduces the complexity of processing video frames, and improves the accuracy of frame reconstruction.
[0043] Secondly, a video transmission method is provided, the method comprising:
[0044] Two sets of video frames are determined based on the video data of the target video. Each set of video frames has the same resolution, and the resolution of one set of video frames is greater than that of the other set. Any video frame in one set of video frames has a different frame number than any video frame in the other set. Each set of video frames includes multiple video frames with discontinuous frame numbers.
[0045] Each group of video frames is encoded separately to obtain the encoded data corresponding to each group of video frames;
[0046] Send the encoded data as described.
[0047] Frame number gaps are used to indicate that the frame numbers of two video frames are not consecutive. Accordingly, each group of video frames includes adjacent video frames whose frame numbers differ by a positive integer greater than 1.
[0048] In a first possible implementation of the second aspect, the video data of the target video includes: first video data and second video data;
[0049] The step of determining two sets of video frames based on the video data of the target video includes:
[0050] The first video data is downsampled to obtain the second video data, wherein the resolution of each video frame in the first video data is higher than the resolution of each video frame in the second video data;
[0051] Multiple video frames are selected from the first video data and the second video data respectively to obtain the two sets of video frames.
[0052] In a second possible implementation of the second aspect, the resolution of each video frame in the video data of the target video is the same;
[0053] The step of determining two sets of video frames based on the video data of the target video includes:
[0054] A subset of video frames is selected from the video data of the target video to obtain a set of video frames;
[0055] Another portion of the video frames in the target video's video data is downsampled to obtain another set of video frames.
[0056] Based on the first or second possible implementation of the second aspect, in the third possible implementation of the second aspect, in the set of video frames with the larger resolution of the two sets of video frames, the difference between the frame numbers of any two adjacent video frames is m, where m is a positive integer greater than 1.
[0057] Based on any one of the first to third possible implementations of the second aspect, in the fourth possible implementation of the second aspect, the video frame with the smaller resolution among the two sets of video frames includes multiple sets of video frames with consecutive frame numbers, and the number of video frames included in each set of video frames with consecutive frame numbers is n, where n is a positive integer greater than or equal to 1.
[0058] Based on any of the above possible implementations of the second aspect, in the fifth possible implementation of the second aspect, the frame numbers of the video frames obtained after combining the two sets of video frames are consecutive.
[0059] Based on any of the possible implementations of the second aspect described above, in a sixth possible implementation of the second aspect, before determining the two sets of video frames based on the video data of the target video, the method further includes:
[0060] Retrieve the video data of the target video from the storage space;
[0061] Alternatively, video data of the target video can be collected in real time.
[0062] Based on any of the possible implementations of the second aspect described above, in the seventh possible implementation of the second aspect, the sending of each of the encoded data includes:
[0063] Send the encoded data to the receiving device;
[0064] Alternatively, the encoded data can be sent to a server, which is used to forward the encoded data to the receiving device.
[0065] Thirdly, a video transmission method is provided, the method comprising:
[0066] Receive first encoded data and second encoded data of the target video, wherein the first encoded data and the second encoded data correspond to different resolutions;
[0067] The first encoded data and the second encoded data are decoded respectively to obtain first decoded data corresponding to the first encoded data and second decoded data corresponding to the second encoded data, wherein the resolution of the first decoded data is greater than the resolution of the second decoded data.
[0068] Based on the first decoded data, the second decoded data is processed to obtain a restored frame with the same resolution as the first decoded data;
[0069] The first decoded data and the restored frame are combined to obtain the video data of the target video.
[0070] In a first possible implementation of the third aspect, the step of processing the second decoded data according to the first decoded data to obtain a restored frame with the same resolution as the first decoded data includes:
[0071] Using a pre-set AI super-resolution model, the video frames of the second decoded data are processed based on the video frames of the first decoded data to obtain the restored frames.
[0072] Based on the first possible implementation of the third aspect described above, in the second possible implementation of the third aspect, the step of processing the video frame of the second decoded data according to the video frame of the first decoded data using a pre-set AI super-resolution model to obtain the restored frame includes:
[0073] The first video frame of the second decoded data, and at least one video frame whose frame number is consecutive to the first video frame, are input into the AI super-resolution model to obtain the restored frame corresponding to the first video frame.
[0074] Fourthly, a video transmission device is provided, the device comprising:
[0075] The encoding module is used to determine two sets of video frames based on the video data of the target video. Each set of video frames has the same resolution, and the resolution of one set of video frames is greater than that of the other set of video frames. The frame number of any video frame in one set of video frames is different from that of any video frame in the other set of video frames. Each set of video frames includes multiple video frames with discontinuous frame numbers.
[0076] The encoding module is also used to encode each group of video frames separately to obtain the encoded data corresponding to each group of video frames;
[0077] The sending module is used to send the encoded data.
[0078] In a first possible implementation of the fourth aspect, the video data of the target video includes: first video data and second video data;
[0079] The encoding module is specifically used to downsample the first video data to obtain the second video data, wherein the resolution of each video frame in the first video data is higher than the resolution of each video frame in the second video data; and to select multiple video frames from the first video data and the second video data respectively to obtain the two sets of video frames.
[0080] In a second possible implementation of the fourth aspect, the resolution of each video frame in the video data of the target video is the same;
[0081] The encoding module is specifically used to select a portion of video frames from the video data of the target video to obtain a set of video frames; and to downsample another portion of the video frames from the video data of the target video to obtain another set of video frames.
[0082] Based on the first or second possible implementation of the fourth aspect, in the third possible implementation of the fourth aspect, in the set of video frames with the larger resolution of the two sets of video frames, the difference between the frame numbers of any two adjacent video frames is m, where m is a positive integer greater than 1.
[0083] Based on any one of the first to third possible implementations of the fourth aspect, in the fourth possible implementation of the fourth aspect, the video frame with the smaller resolution of the two sets of video frames includes multiple sets of video frames with consecutive frame numbers, and the number of video frames included in each set of video frames with consecutive frame numbers is n, where n is a positive integer greater than or equal to 1.
[0084] Based on any of the above possible implementations of the fourth aspect, in the fifth possible implementation of the fourth aspect, the frame numbers of the video frames obtained after combining the two sets of video frames are consecutive.
[0085] Based on any of the possible implementations of the fourth aspect described above, in a sixth possible implementation of the fourth aspect, the apparatus further includes:
[0086] The acquisition module is used to acquire video data of the target video from the storage space;
[0087] Alternatively, an acquisition module may be used to collect video data of the target video in real time.
[0088] Based on any of the above possible implementations of the fourth aspect, in the seventh possible implementation of the fourth aspect, the sending module is specifically used to send each of the encoded data to the receiving device.
[0089] Alternatively, the sending module is specifically used to send each of the encoded data to the server, and the server is used to forward each of the encoded data to the receiving device.
[0090] Fifthly, a video transmission device is provided, the device comprising:
[0091] The receiving module is used to receive first encoded data and second encoded data of the target video, wherein the first encoded data and the second encoded data correspond to different resolutions.
[0092] A decoding module is used to decode the first encoded data and the second encoded data respectively to obtain first decoded data corresponding to the first encoded data and second decoded data corresponding to the second encoded data, wherein the resolution of the first decoded data is greater than the resolution of the second decoded data.
[0093] The processing module is used to process the second decoded data according to the first decoded data to obtain a restored frame with the same resolution as the first decoded data;
[0094] The processing module is also used to combine the first decoded data and the restored frame to obtain the video data of the target video.
[0095] In a first possible implementation of the fifth aspect, the processing module is specifically used to process the video frame of the second decoded data according to the video frame of the first decoded data through a pre-set AI super-resolution model to obtain the restored frame.
[0096] Based on the first possible implementation of the fifth aspect, in the second possible implementation of the fifth aspect, the processing module is specifically used to input the first video frame of the second decoded data and at least one video frame whose frame number is consecutive to the first video frame into the AI super-resolution model to obtain the restored frame corresponding to the first video frame.
[0097] Sixthly, a video transmission system is provided, the video transmission system comprising: a transmitting end device and a receiving end device;
[0098] The transmitting device is used to perform the video transmission method as described in any one of the second aspects;
[0099] The receiving device is used to perform the video transmission method as described in any one of the third aspects.
[0100] A seventh aspect provides an electronic device comprising: a processor for running a computer program stored in a memory to implement the video transmission method as described in any one of the second or third aspects.
[0101] Eighthly, a computer-readable storage medium is provided, characterized in that the computer-readable storage medium stores a computer program, which, when executed by a processor, implements the video transmission method as described in any one of the second or third aspects.
[0102] Ninth aspect, a chip system is provided, the chip system including a memory and a processor, the processor executing a computer program stored in the memory to implement the video transmission method as described in any one of the second or third aspects.
[0103] It is understood that the beneficial effects of the second to ninth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. Attached Figure Description
[0104] Figure 1 A system architecture diagram of a video transmission system provided in this application embodiment;
[0105] Figure 2 A structural block diagram of a transmitting end device and a receiving end device transmitting video data is provided in an embodiment of this application;
[0106] Figure 3 A schematic flowchart illustrating a video transmission method provided in an embodiment of this application;
[0107] Figure 4A This is a schematic diagram illustrating the selection of video frames for encoding based on video data, provided as an embodiment of this application.
[0108] Figure 4B This is a schematic diagram illustrating another method for selecting video frames for encoding based on video data, provided as an embodiment of this application.
[0109] Figure 4C This is a schematic diagram illustrating another method for selecting video frames for encoding based on video data, provided as an embodiment of this application.
[0110] Figure 5A This is a schematic diagram illustrating the grouping of video frames in video data, as provided in an embodiment of this application.
[0111] Figure 5B This is a schematic diagram illustrating another method of grouping video frames in video data, provided as an embodiment of this application.
[0112] Figure 5C This is a schematic diagram illustrating another method of grouping video frames in video data, provided as an embodiment of this application.
[0113] Figure 6 This is a schematic diagram illustrating the decoding of encoded data according to an embodiment of this application.
[0114] Figure 7 A schematic diagram illustrating a process for generating a restored frame using an AI super-resolution model, provided as an embodiment of this application;
[0115] Figure 8 A framework architecture diagram of an AI super-resolution model provided in this application embodiment;
[0116] Figure 9 A schematic flowchart illustrating another video transmission method provided in an embodiment of this application;
[0117] Figure 10 This is a schematic diagram illustrating another method of grouping video frames in video data, provided as an embodiment of this application.
[0118] Figure 11 A structural frame of a video transmission device provided in an embodiment of this application;
[0119] Figure 12 This application provides a structural frame for another video transmission device according to an embodiment of the present application;
[0120] Figure 13 This application provides a structural frame for yet another video transmission device according to an embodiment of the present application;
[0121] Figure 14 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application. Detailed Implementation
[0122] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known video compression methods, video compression standards, video transmission methods, and electronic devices are omitted so as not to obscure the description of this application with unnecessary detail.
[0123] The terminology used in the following embodiments is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “the,” “the,” and “the” are intended to also include expressions such as “one or more,” unless the context clearly indicates otherwise.
[0124] First, the system framework of the video transmission system involved in the embodiments of this application is introduced. See [link to relevant documentation]. Figure 1 , Figure 1 The system framework of the video transmission system shown includes: a transmitting device 110, a receiving device 120, and a server 130.
[0125] In this embodiment, server 130 can establish communication connections with both sending device 110 and receiving device 120, and sending device 110 can also establish a communication connection with receiving device 120. Correspondingly, during the transmission of video data, sending device 110 can send video data to receiving device 120 or send video data to server 130, and then forward the video data to receiving device 120 through server 130. This application embodiment does not limit the method by which sending device 110 sends video data.
[0126] The following describes the process of transmitting video data of a target video from the transmitting device 110 to the receiving device 120 as an example.
[0127] See Figure 2 , Figure 2 A block diagram illustrating the structure of a transmitting and receiving device for transmitting video data is shown, such as... Figure 2 As shown, the transmitting device 110 may include a data acquisition module 1101 and an encoding module 1102.
[0128] The data acquisition module 1101 is connected to the encoding module 1102. The data acquisition module 1101 may be a camera of the transmitting device 110, and the encoding module 1102 may include an encoder.
[0129] Before transmitting the target video data, the sending device 110 can first acquire the first video data of the target video through the data acquisition module 1101, and then downsample the first video data to obtain the second video data, wherein the resolution of the first video data is higher than that of the second video data. Afterwards, the encoder in the encoding module 1102 can use a dual-thread approach to encode a portion of the video frames in the first video data and a portion of the video frames in the second video data in parallel, obtaining first encoded data corresponding to the first video data and second encoded data corresponding to the second video data, so that the first encoded data and the second encoded data can be sent to the receiving device 120 or the server 130.
[0130] The above describes encoding and compressing the video data of the target video obtained in real time. The sending device 110 may also include the video data of the target video that has been stored in advance. The sending device 110 may also compress the video data of the target video that has been stored in advance before sending the first encoded data and the second encoded data to the receiving device 120 or the server 130.
[0131] Accordingly, see Figure 2The transmitting device 110 may further include a storage module 1103, which may include pre-stored video data of the target video.
[0132] During the encoding process of the pre-stored target video data, the transmitting device 110 can first process the pre-stored target video data in the storage module 1103 to obtain multiple high-resolution continuous video frames. Then, it can group the multiple continuous video frames according to a preset rule to obtain two groups of video frames. Next, it can downsample one group of video frames to obtain low-resolution video frames, thus obtaining one set of high-resolution video frames and one set of low-resolution video frames. The encoder in the encoding module 1102 can then encode the two sets of video frames with different resolutions to obtain first encoded data and second encoded data, which can then be sent to the receiving device 120 or the server 130.
[0133] See Figure 2 The receiving device 120 may include: a decoding module 1201, a synthesis module 1202, a playback module 1203, and a storage module 1204.
[0134] The decoding module 1201 is connected to the synthesis module 1202 and the storage module 1204, respectively. The synthesis module 1202 is also connected to the playback module 1203. Furthermore, the decoding module 1201 may include a decoder.
[0135] After receiving the first and second encoded data corresponding to different resolutions, the receiving device 120 can use the decoder of the decoding module 1201 to decode the first and second encoded data in parallel using a dual-thread approach, obtaining the first decoded data and the second decoded data, which are two sets of video frames. The compositing module 1202 can then use a pre-set artificial intelligence (AI) super-resolution model to reconstruct the low-resolution set of video frames based on the high-resolution set of video frames, obtaining the reconstructed frames. Then, the compositing module 1202 can combine the high-resolution set of video frames and the reconstructed frames to obtain the composited video data. Finally, the playback module 1203 can play the composited video data.
[0136] After decoding two sets of video frames, the storage module 1204 can store the first decoded data and the second decoded data, which are respectively composed of the two sets of video frames. Furthermore, the receiving device 120 can play the first decoded data and the second decoded data respectively.
[0137] It should be noted that, Figure 2The illustration shows the process by which the sending device 110 sends first encoded data and second encoded data to the receiving device 120 using a point-to-point transmission method. In practical applications, the sending device 110 may also first send the first encoded data and second encoded data to the server 130. Then, the server 130 may forward the received first encoded data and second encoded data to the receiving device 120. This embodiment does not limit the method of sending the first encoded data and second encoded data to the receiving device 120.
[0138] Furthermore, in the video transmission system provided in this application embodiment, the sending device 110 can transmit the currently collected video data in real time. For example, the video transmission system can be applied in video call scenarios, remote conferencing scenarios, and live web streaming scenarios.
[0139] Of course, in the video transmission system provided in this application embodiment, the sending device 110 can also transmit pre-stored video data. This application embodiment does not limit the application time and application scenario of the video transmission system.
[0140] Furthermore, the above description is based on the example of the target video containing two video data of different resolutions. In actual applications, the sending device 110 can obtain video data corresponding to multiple resolutions of the target video. This application embodiment does not limit the number of video data corresponding to the target video.
[0141] Furthermore, the transmitting device 110 can also downsample the video data of the target video to obtain multiple sets of video frames corresponding to various resolutions. In this embodiment of the application, the multiple resolutions obtained by the transmitting device 110 through downsampling are not limited, nor is the number of multiple sets of video frames obtained by encoding and compression limited.
[0142] For simplicity, the following explanation uses the example of the encoded data corresponding to two resolutions obtained by the transmitting device 110 through encoding and compression, that is, the example of the encoded data sent by the transmitting device 110 to the receiving device 120 including the first encoded data and the second encoded data.
[0143] Figure 3 This is a schematic flowchart illustrating a video transmission method provided in an embodiment of this application. It is intended as an example and not a limitation. This method can be applied to the aforementioned sending and receiving devices. See also... Figure 3 The method includes:
[0144] Step 301: The sending device acquires the video data of the target video.
[0145] The transmitting device can acquire video data in various ways. Specifically, the transmitting device can acquire the video data of the target video in real time, or it can acquire the video data of the target video that has been stored in a pre-set storage space. This application embodiment does not limit the method of acquiring the video data of the target video.
[0146] The sending device can acquire the video data of the target video using any of the following methods, see Method 1 and Method 2 below:
[0147] Method 1: The sending device collects video data of the target video.
[0148] During video calls, remote conferences, or live video streaming between the sending and receiving devices, the sending device can capture video data of the target video in real time using a pre-configured camera. Furthermore, to facilitate subsequent encoding and compression of the video data, the sending device can simultaneously acquire video data at multiple resolutions for the target video using a modified application programming interface (API).
[0149] For example, during a video call, the sending device can capture images from the camera in real time and simultaneously output video data at two resolutions: high-resolution first video data and low-resolution second video data. The second video data is obtained by downsampling the first video data.
[0150] It should be noted that the camera used to capture video data of the target video can be a built-in camera of the sending device or an external camera connected to the sending device; this application embodiment does not limit this. For example, the camera can be a built-in camera of a mobile phone or an external camera connected to a computer.
[0151] Method 2: The sending device obtains the video data of the pre-stored target video.
[0152] The transmitting device can send not only real-time captured target video data to the receiving device, but also pre-stored target video data. If the transmitting device needs to send pre-stored target video data to the receiving device, it can retrieve the target video data from a pre-defined storage space.
[0153] For example, the sending device can detect user-triggered operations, determine the storage path of the target video data to be transmitted based on the user-triggered operations, and then retrieve the target video data from the corresponding storage space according to the storage path.
[0154] It should be noted that in practical applications, the storage space for the video data acquired by the sending device can be the storage space built into the sending device, an external storage device connected to the sending device, or a server connected to the sending device, i.e., cloud storage space. This application embodiment does not limit the storage space.
[0155] Step 302: The sending device encodes and compresses the video data of the acquired target video to obtain the first encoded data and the second encoded data.
[0156] The first encoded data is high-resolution video data after encoding and compression, and the second encoded data is low-resolution video data after encoding and compression.
[0157] In step 301, the sending device can acquire the video data of the target video in different ways. Corresponding to step 301, in step 302, the sending device can encode and compress the target video data in different ways according to the different acquisition methods of the video data to obtain the first encoded data and the second encoded data.
[0158] Similar to step 301, the sending device may also compress the video data of the acquired target video using any of the following methods, see Method 1 and Method 2 below, wherein Method 1 of step 302 corresponds to Method 1 of step 301, and Method 2 of step 302 corresponds to Method 2 of step 301.
[0159] Method 1: The sending device encodes a portion of the video frames in the first video data and the second video data according to preset rules to obtain the first encoded data and the second encoded data.
[0160] After acquiring the first video data and the second video data of the target video through the camera, the transmitting device can encode a portion of the video frames in the first video data and the second video data respectively through a pre-set encoder, thereby obtaining the first encoded data generated from the first video data and the second encoded data generated from the second video data.
[0161] During the encoding process of partial video frames in the first video data and the second video data, the transmitting device can select partial video frames from the first video data according to the video frame number used to represent the order of each video frame in the first video data, and obtain a set of video frames. Then, the selected set of video frames is encoded by the encoder to obtain the first encoded data.
[0162] The transmitting device can select a portion of the video frames in the second video data to obtain another set of video frames. Then, the encoder can encode the other set of video frames in a similar manner to the above encoding process to obtain the second encoded data.
[0163] For example, see Figure 4A , Figure 4B and Figure 4C , Figure 4A , Figure 4B and Figure 4C Both the first and second video data are shown in the diagram. Each set of first and second video data includes 10 video frames, with the resolution of the first video data frames being higher than that of the second video data frames. The transmitting device selects video frames using the encoder, as described above. Figure 4A The transmitting device can select frames 1, 3, 5, 7, and 9 from the first video data, and frames 2, 4, 6, 8, and 10 from the second video data; or, see [link to relevant documentation]. Figure 4B The transmitting device can select frames 1, 4, 7, and 10 from the first video data, and frames 2, 3, 5, 6, 8, and 9 from the second video data; or, see [link to relevant documentation]. Figure 4C The transmitting device can select frames 1, 5, and 9 from the first video data, and frames 2, 3, 4, 6, 7, 8, and 10 from the second video data. Then, the transmitting device can encode the video frames selected from the first video data using an encoder to obtain first encoded data, and simultaneously encode the video frames selected from the second video data using an encoder to obtain second encoded data.
[0164] It should be noted that the sending device can also select video frames in other ways according to preset rules. This application embodiment does not limit the way of selecting video frames.
[0165] Method 2: The transmitting device downsamples a portion of the video frames in the acquired target video data according to preset rules to obtain two sets of video frames with different resolutions. The two sets of video frames are then encoded to obtain the first encoded data and the second encoded data.
[0166] After obtaining the pre-stored target video data from the storage space, the transmitting device can downsample some video frames in the pre-stored target video data to obtain two sets of video frames with different resolutions. Then, the two sets of video frames can be compressed by a pre-set encoder to obtain the first encoded data and the second encoded data.
[0167] Specifically, the transmitting device can first group the video frames according to preset rules, combining the video frame numbers of each video frame in the pre-stored video data, to obtain two groups of video frames with the same resolution. Then, the transmitting device can downsample one group of video frames according to preset rules to obtain a group of low-resolution video frames. Combining this with the un-downsampled group of video frames, two groups of video frames with different resolutions can be obtained.
[0168] Then, the transmitting device can use a method similar to Method 1 to encode and compress the two sets of video frames respectively through an encoder to obtain the first encoded data and the second encoded data.
[0169] It should be noted that in both Method 1 and Method 2, the video frames obtained from the two sets of video frames have the same resolution, and the resolution of one set of video frames is greater than that of the other set. However, the frame number of any video frame in one set of video frames is different from the frame number of any video frame in the other set of encoded data. Furthermore, each set of video frames can include multiple video frames with discontinuous frame numbers.
[0170] Furthermore, in a high-resolution set of video frames, the frame numbers of the individual video frames are not consecutive, while in a low-resolution set of video frames, the frame numbers of the individual video frames may or may not be consecutive. To improve the compression rate of video data and reduce the storage space and network bandwidth resources occupied by compressed data, in a high-resolution set of video frames, the difference between the frame numbers of any two adjacent video frames is m, where m is a positive integer greater than 1. In a low-resolution set of video frames, multiple sets of consecutively numbered video frames can be included, with each set containing n video frames, where n is a positive integer greater than or equal to 1. For example, n can be greater than or equal to 1 and less than or equal to 3. Adjacent to each other within a set of video frames means that no other video frames exist between them. For example, if a set of video frames includes frames with frame numbers 1, 3, and 5, then frames with frame numbers 1 and 3 are adjacent, and frames with frame numbers 3 and 5 are adjacent.
[0171] Furthermore, the video frames corresponding to the first encoded data are complementary to the video frames corresponding to the second encoded data. That is, after combining the video frames corresponding to the first encoded data and the video frames corresponding to the second encoded data, a series of consecutive video frames with consecutive frame numbers can be obtained, and each consecutive video frame number corresponds one-to-one with the video frame number of the target video.
[0172] For example, see Figure 5A , Figure 5B and Figure 5C , Figure 5A , Figure 5B and Figure 5C The image shows video frames from pre-stored video data, which includes 10 video frames. The transmitting device can group these 10 video frames according to a preset rule, see [link to documentation]. Figure 5A For example, frames 1, 3, 5, 7, and 9 can be used as the first group of video frames, and frames 2, 4, 6, 8, and 10 as the second group of video frames; or, see [link to relevant documentation]. Figure 5B The transmitting device can use frames 1, 4, 7, and 10 as the first group of video frames, and frames 2, 3, 5, 6, 8, and 9 as the second group of video frames; or, see [link to relevant documentation]. Figure 5C The transmitting device can use frames 1, 5, and 9 as the first group of video frames, and frames 2, 3, 4, 6, 7, 8, and 10 as the second group of video frames. Then, the transmitting device can downsample the second group of video frames to obtain a set of low-resolution video frames. Finally, the transmitting device can encode the first group of video frames using an encoder to obtain first encoded data, and then encode the downsampled second group of video frames using the same encoder to obtain second encoded data.
[0173] It should be noted that in practical applications, the transmitting device in step 301 (method one) may also only collect high-resolution video data, and then encode and compress it through step 302 (method two). This application embodiment does not limit the method of real-time acquisition of video data, nor does it limit the method of encoding and compressing video data.
[0174] Step 303: The sending device sends the first encoded data and the second encoded data to the receiving device.
[0175] After receiving the first and second encoded data, the sending device can use a dual-channel approach to send the first and second encoded data to the receiving device while consuming less network bandwidth resources.
[0176] Step 304: The receiving device receives the first encoded data and the second encoded data.
[0177] Step 305: The receiving device decodes the first encoded data and the second encoded data respectively to obtain the first decoded data and the second decoded data.
[0178] After receiving the first encoded data and the second encoded data, the receiving device can decode the first encoded data using a pre-set decoder to obtain the first decoded data corresponding to the first encoded data, and decode the second encoded data using the decoder to obtain the second decoded data corresponding to the second encoded data, that is, obtain the video frames corresponding to the first encoded data and the second encoded data respectively.
[0179] Meanwhile, the receiving device can store the first decoded data and the second decoded data, thereby saving storage space. Accordingly, if the receiving device needs to play high-resolution video data corresponding to the first decoded data and the second decoded data, the receiving device can execute steps 306 and 307 to play the video based on the first decoded data and the second decoded data.
[0180] It should be noted that during the reception of encoded data, the receiving device can decode the received encoded data in real time to obtain decoded video frames, and then assemble decoded data from multiple decoded video frames. For example, see... Figure 6 After receiving a portion of the first encoded data and a portion of the second encoded data, the receiving device can decode the received encoded data, thereby obtaining the first frame and the third frame based on the portion of the first encoded data, and obtaining the second frame and the fourth frame based on the portion of the second encoded data. At the same time, the receiving device can also continue to receive the remaining first encoded data and second encoded data sent by the sending device.
[0181] Step 306: The receiving device synthesizes the first decoded data and the second decoded data to obtain the synthesized video data.
[0182] After decoding the first encoded data and the second encoded data to obtain the first decoded data and the second decoded data respectively, the receiving device can use a pre-set AI super-resolution model to reconstruct the video frames included in the second decoded data based on the high-resolution video frames included in the first decoded data, thus obtaining high-resolution reconstructed frames. Then, the receiving device can combine the high-resolution video frames included in the first decoded data and the high-resolution reconstructed frames to obtain video data of the target video synthesized from multiple high-resolution consecutive video frames.
[0183] Specifically, see Figure 7 , Figure 7 The diagram illustrates the process by which a receiving device generates a high-resolution reconstructed frame using an AI super-resolution model, which includes multiple neural network convolutional layers. If the receiving device needs to reconstruct a first video frame included in the second decoded data, it can select a high-resolution video frame with a consecutive frame number from the multiple high-resolution video frames included in the first decoded data, based on the frame number of the first video frame. Then, the receiving device can input the selected first video frame and the corresponding high-resolution video frame into a pre-set AI super-resolution model. The AI super-resolution model, combined with the corresponding high-resolution video frame, reconstructs the first video frame, obtaining the high-resolution reconstructed frame corresponding to the first video frame.
[0184] The first video frame can be any video frame in the second decoded data. A high-resolution video frame with a consecutive frame number to the first video frame can be a video frame with consecutive frame numbers preceding or following the low-resolution video frame. For example, if the first video frame has a frame number of 2, then the consecutive video frames can have frame numbers of 1 or 3; this embodiment does not limit this.
[0185] After recovering the low-resolution video frames corresponding to the second decoded data, the receiving device can sort and combine the high-resolution video frames and the restored frames according to the video frame numbers of the high-resolution video frames corresponding to the first decoded data and the video frame numbers of the low-resolution video frames corresponding to the restored frames, to obtain high-resolution synthesized video data.
[0186] Further, see Figure 8 , Figure 8 The diagram shows the framework architecture of the AI super-resolution model. The AI super-resolution model can obtain a high-resolution video frame with parameters of 1*540*960*6 through the first input terminal (input_1). Here, 1 represents the number of video frames, 540*960 represents the resolution of the video frame, and 6 represents the number of channels of the video frame obtained by the AI super-resolution model. The meaning of the parameters of each video frame in the following text is similar to that of the parameters of this high-resolution video frame, and will not be repeated here. Similarly, the AI super-resolution model can obtain a low-resolution video frame with parameters 1*270*480*6 through the second input terminal (input_2). The AI super-resolution model can first perform a spatial-to-depth operation on the high-resolution video frame to obtain a video frame with parameters 1*270*480*24. Then, it can perform a concatenation operation on the converted high-resolution video frame and the low-resolution video frame, concatenating the converted high-resolution video frame and the low-resolution video frame and superimposing the dimensions to obtain a concatenated video frame with parameters 1*270*480*30.
[0187] Subsequently, the AI super-resolution model performs multiple convolutions (Conv2D) on the associated video frames and corrects them using an activation function (ReLU). Then, it overlays the convolutional video frames with the associated video frames (Add), performs a depth-to-space operation (DepthToSpace) on the overlaid video frames, and finally outputs (Identity) the restored video frames with parameters of 1*540*960*6.
[0188] In this process, when convolutional layers are used to process each video frame, the parameters of the convolutional kernel in each convolutional layer are different. See also Figure 8When performing the first convolution on a video frame with parameters of 1*270*480*30, the parameters of the convolution kernel (filter) are 6*3*3*30, and the error (bias) parameter is 6, resulting in a video frame with parameters of 1*270*480*6. Then, this video frame with parameters of 1*270*480*6 can be convolved again, with the parameters of the convolution kernel being 12*3*3*6 and the error parameter being 12, resulting in a video frame with parameters of 1*270*480*12. Finally, this video frame with parameters of 1*270*480*12 is convolved again, with the parameters of the convolution kernel being 24*1*1*12 and the error parameter being 24, resulting in a video frame with parameters of 1*270*480*24.
[0189] In addition, the AI super-resolution model can also perform convolution again on the video frame with parameters of 1*270*480*30 (i.e., the associated video frame) in another way. The parameters of this convolution kernel are 24*1*1*30 and the error parameter is 24, resulting in another video frame with parameters of 1*270*480*24. The two video frames with parameters of 1*270*480*24 can then be superimposed to obtain the superimposed video frame.
[0190] Moreover, the AI super-resolution model mentioned above can include four convolutional layers, and the number of channels in the convolutional kernel is very small, requiring very low computational cost. When the receiving device is a mobile phone, tablet, or other portable mobile device, it meets the conditions for the AI super-resolution model to recover video frames, and the restored frame can be obtained.
[0191] It should be noted that the pixels in the aforementioned video frames are represented in YUV format. Correspondingly, before inputting the video frames into the AI super-resolution model, the YUV format data of each video frame can be converted to obtain 6-channel YUV format data. Among them, the first 4 channels can be Y data representing brightness, and the last 2 channels can be U data and V data representing color and saturation, respectively.
[0192] Furthermore, the above explanation only uses the reconstruction of a low-resolution video frame from a high-resolution video frame as an example. In practical applications, the receiving device can input multiple high-resolution video frames into the AI super-resolution model to reconstruct a low-resolution video frame. For example, if the video frame number of the low-resolution video frame to be reconstructed is 2, then the video frame numbers of the high-resolution video frames input into the AI super-resolution model can be 1 and 3. This application does not limit this aspect.
[0193] Furthermore, if the video frames in the second decoded data include consecutive video frames, then during the process of restoring these consecutive video frames, the receiving device can restore them using video frames adjacent to the consecutive video frames in the video frames included in the first decoded data, or it can restore the remaining video frames in the consecutive video frames using adjacent restored frames. The process of restoring consecutive video frames is similar to the process of restoring low-resolution video frames described above, and will not be repeated here.
[0194] For example, the first decoded data includes a video frame with frame number 1, and the second decoded data includes three consecutive video frames with frame numbers 2, 3, and 4. The receiving device can input each video frame with frame numbers 1, 2, 3, and 4 into the AI super-resolution model, and the AI super-resolution model can process the video frames with frame numbers 2, 3, and 4 to obtain the restored frames corresponding to the video frames with frame numbers 2, 3, and 4 respectively.
[0195] Alternatively, the receiving device can input both video frames numbered 1 and 2 into the AI super-resolution model. The AI super-resolution model can then process video frame number 2 based on video frame number 1 to obtain the restored frame corresponding to video frame number 2. Then, it can input video frame number 3, and use the restored frame corresponding to video frame number 2 to process video frame number 3 to obtain the restored frame corresponding to video frame number 3. The receiving device can continue the recovery processing of video frame number 4 in a similar manner to that used for video frame number 3, thus completing the recovery processing of consecutive video frames.
[0196] Step 307: The receiving device plays the synthesized video data.
[0197] After obtaining the synthesized video data based on the first and second decoded data, the receiving device can play the synthesized video data, thereby obtaining high-resolution video data with less network bandwidth and less storage space.
[0198] In practical applications, the receiving device can not only play the video data synthesized from the first and second decoded data, but also play the stored first and second decoded data separately. This application embodiment does not limit this.
[0199] Additionally, see Figure 9 The sending device can also send first encoded data and second encoded data to the server, and the server forwards the first encoded data and second encoded data to the receiving device.
[0200] Furthermore, the sending device may not encode and compress the video data of the target video, but instead send the high-resolution video data of the target video to the server. The server then encodes and compresses the video data of the target video to obtain first encoded data and second encoded data. The server then sends the encoded first encoded data and second encoded data to the receiving device. In this embodiment, the format of the video data sent by the sending device to the server is not limited.
[0201] It should be noted that the above embodiment only involves the sending device sending compressed data corresponding to two different resolutions to the receiving device. In actual applications, the sending device can also first obtain three or more sets of video frames with different resolutions, then compress each set of video frames, and then send the compressed data obtained from each compression to the receiving device.
[0202] For example, see Figure 10 , Figure 10 The image shows video frames from pre-stored video data, which includes 10 video frames. The transmitting device can divide these 10 frames into three groups according to a preset rule. For example, frames 1, 5, and 9 can be grouped as the first group; frames 2, 4, 6, 8, and 10 as the second group; and frames 3 and 7 as the third group. The transmitting device can then downsample the second and third groups of video frames, respectively, resulting in two groups of low-resolution video frames, with the third group having a lower resolution than the second group. Finally, the transmitting device can encode the first group, the downsampled second group, and the downsampled third group of video frames using an encoder, respectively, to obtain first encoded data, second encoded data, and third encoded data.
[0203] Correspondingly, the receiving device can receive multiple encoded data sent by the sending device and decode them to obtain multiple sets of video frames with different resolutions. Then, using a pre-set AI super-resolution model, the video frames of different resolutions can be restored to obtain reconstructed frames, thereby improving the accuracy of the restored frames.
[0204] In summary, the video transmission method provided in this application involves the sending device determining two sets of video frames based on the video data of the target video, encoding and compressing each set of video frames to obtain encoded data corresponding to each set of video frames, and then sending the encoded data to the receiving device. By transmitting encoded data at different resolutions, the compression rate can be improved by 30%-40%, thereby effectively improving the compression rate of the target video data, reducing the network bandwidth resources occupied when transmitting video data, avoiding waste of network bandwidth resources, and reducing the cost of transmitting video data.
[0205] Furthermore, the receiving device decodes the first encoded data and the second encoded data to obtain the first decoded data and the second decoded data, and then synthesizes them to obtain the synthesized video data. The receiving device only needs to store the first encoded data and the second encoded data, which occupy less storage space, to play the video data of the target video. There is no need to store the high-resolution video data of the target video, which can reduce the storage space occupied and improve the utilization rate of storage space.
[0206] Similarly, the server does not need to store video data of multiple resolutions of the target video, which can reduce the storage space occupied by the target video data on the server, improve the utilization of server storage space, and reduce server maintenance costs.
[0207] In addition, the sending device can acquire the video data of the target video in different ways and encode the video data of the target video in a way that corresponds to the acquisition method. This can improve the flexibility of acquiring video data and the flexibility of encoding the video data of the target video.
[0208] Furthermore, the process of encoding to obtain the first encoded data and the second encoded data, as well as the process of decoding to obtain the first decoded data and the second decoded data, are independent of the standard encoding and decoding stages, and can be compatible with any standard encoding and decoding technology, thus improving the compatibility of transmitted video data.
[0209] Furthermore, the receiving device uses a pre-set AI super-resolution model to synthesize complementary first and second decoded data from video frames, which can improve the accuracy of the recovered video frames and thus improve the playback effect of the synthesized video data.
[0210] Moreover, the receiving device uses the AI super-resolution model to process low-resolution video frames based on high-resolution video frames with adjacent frame numbers. By utilizing the high-frequency spatial information of high-resolution video frames, the complexity of the super-resolution network can be reduced, thereby reducing the operating conditions of the AI super-resolution model. This enables the AI super-resolution model to be run through portable terminals, improving the versatility of the AI super-resolution model.
[0211] In addition, during the process of processing video frames using the AI super-resolution model, the receiving device can obtain information about missing frames based on low-resolution video frames. This saves network bandwidth resources, reduces the complexity of processing video frames, and improves the accuracy of frame reconstruction.
[0212] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.
[0213] Corresponding to the video transmission method described in the above embodiments, Figure 11 This is a structural block diagram of a video transmission device provided in an embodiment of this application. For ease of explanation, only the parts related to the embodiment of this application are shown.
[0214] Reference Figure 11 The device includes:
[0215] The encoding module 1101 is used to determine two sets of video frames based on the video data of the target video. Each set of video frames has the same resolution, and the resolution of one set of video frames is greater than that of the other set of video frames. The frame number of any video frame in one set of video frames is different from that of any video frame in the other set of video frames. Each set of video frames includes multiple video frames with discontinuous frame numbers.
[0216] The encoding module 1101 is also used to encode each group of video frames separately to obtain the encoded data corresponding to each group of video frames;
[0217] The sending module 1102 is used to send each encoded data.
[0218] Optionally, the video data of the target video includes: first video data and second video data;
[0219] The encoding module 1101 is specifically used to downsample the first video data to obtain the second video data, wherein the resolution of each video frame in the first video data is higher than the resolution of each video frame in the second video data; and to select multiple video frames from the first video data and the second video data respectively to obtain the two sets of video frames.
[0220] Optionally, the resolution of each video frame in the target video's video data is the same;
[0221] The encoding module 1101 is specifically used to select a portion of video frames from the video data of the target video to obtain a set of video frames; and to downsample another portion of the video frames from the video data of the target video to obtain another set of video frames.
[0222] Optionally, in the set of video frames with the larger resolution, the difference between the frame numbers of any two adjacent video frames is m, where m is a positive integer greater than 1.
[0223] Optionally, the set of video frames with the smaller resolution among the two sets of video frames may include multiple sets of video frames with consecutive frame numbers. Each set of video frames with consecutive frame numbers may include n video frames, where n is a positive integer greater than or equal to 1.
[0224] Optionally, the frame numbers of the video frames obtained by combining the two sets of video frames are consecutive.
[0225] Optional, see Figure 12 The device also includes:
[0226] The acquisition module 1103 is used to acquire the video data of the target video from the storage space;
[0227] Alternatively, module 1103 can be used to acquire video data of the target video in real time.
[0228] Optionally, the transmitting module 1102 is specifically used to transmit each of the encoded data to the receiving device;
[0229] Alternatively, the sending module is specifically used to send each of the encoded data to the server, which in turn forwards the encoded data to the receiving device.
[0230] Figure 13 This is a structural block diagram of another video transmission device provided in the embodiments of this application. For ease of explanation, only the parts related to the embodiments of this application are shown.
[0231] Reference Figure 13 The device includes:
[0232] The receiving module 1301 is used to receive first encoded data and second encoded data of the target video, wherein the first encoded data and the second encoded data correspond to different resolutions.
[0233] Decoding module 1302 is used to decode the first encoded data and the second encoded data respectively to obtain first decoded data corresponding to the first encoded data and second decoded data corresponding to the second encoded data, wherein the resolution of the first decoded data is greater than the resolution of the second decoded data.
[0234] Processing module 1303 is used to process the second decoded data according to the first decoded data to obtain a restored frame with the same resolution as the first decoded data;
[0235] The processing module 1303 is also used to combine the first decoded data and the restored frame to obtain the video data of the target video.
[0236] Optionally, the processing module 1303 is specifically used to process the video frame of the second decoded data according to the video frame of the first decoded data through a pre-set AI super-resolution model to obtain the restored frame.
[0237] Optionally, the processing module 1303 is specifically used to input the first video frame of the second decoded data and at least one video frame whose frame number is consecutive to the first video frame into the AI super-resolution model to obtain the restored frame corresponding to the first video frame.
[0238] In summary, the video transmission apparatus provided in this application embodiment encodes and compresses the video data of the target video to obtain and send high-resolution first encoded data and low-resolution second encoded data to the receiving device. By compressing and transmitting first encoded data and second encoded data of different resolutions, the compression rate can be improved by 30%-40%, thereby effectively improving the compression rate of the target video data, reducing the network bandwidth resources occupied when transmitting video data, avoiding waste of network bandwidth resources, and reducing the cost of transmitting video data.
[0239] The following description uses an electronic device as an example to illustrate the transmitting and receiving devices involved in the embodiments of this application. Please refer to [link / reference]. Figure 14 , Figure 14 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.
[0240] The electronic device may include a processor 1410, an external memory interface 1420, an internal memory 1421, a universal serial bus (USB) interface 1430, a charging management module 1440, a power management module 1441, a battery 1442, an antenna 1, an antenna 2, a mobile communication module 1450, a wireless communication module 1460, an audio module 1470, a speaker 1470A, a receiver 1470B, a microphone 1470C, a headphone jack 1470D, a sensor module 1480, buttons 1490, a motor 1491, an indicator 1492, a camera 1493, a display screen 1494, and a subscriber identification module (SIM) card interface 1495, etc. The sensor module 1480 may include a pressure sensor 1480A, a gyroscope sensor 1480B, a barometric pressure sensor 1480C, a magnetic sensor 1480D, an accelerometer sensor 1480E, a distance sensor 1480F, a proximity sensor 1480G, a fingerprint sensor 1480H, a temperature sensor 1480J, a touch sensor 1480K, an ambient light sensor 1480L, a bone conduction sensor 1480M, etc.
[0241] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device. In other embodiments of this application, the electronic device may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.
[0242] Processor 1410 may include one or more processing units, such as an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). These different processing units may be independent devices or integrated into one or more processors.
[0243] The controller can serve as the nerve center and command center of an electronic device. Based on the instruction opcode and timing signals, the controller generates operation control signals to control the fetching and execution of instructions.
[0244] The processor 1410 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 1410 is a cache memory. This memory can store instructions or data that the processor 1410 has just used or that are used repeatedly. If the processor 1410 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 1410, and thus improves the efficiency of the system.
[0245] In some embodiments, the processor 1410 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.
[0246] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 1410 may include multiple I2C buses. The processor 1410 can couple to the touch sensor 1480K, charger, flash, camera 1493, etc., through different I2C bus interfaces. For example, the processor 1410 can couple to the touch sensor 1480K through the I2C interface, enabling the processor 1410 and the touch sensor 1480K to communicate through the I2C bus interface, thus realizing the touch function of the electronic device.
[0247] The UART interface is a universal serial data bus used for asynchronous communication. This bus can be a bidirectional communication bus. It converts the data to be transmitted between serial and parallel communication. In some embodiments, the UART interface is typically used to connect the processor 1410 and the wireless communication module 1460. For example, the processor 1410 communicates with the Bluetooth module in the wireless communication module 1460 via the UART interface to implement Bluetooth functionality. In some embodiments, the audio module 1470 can transmit audio signals to the wireless communication module 1460 via the UART interface to enable music playback through Bluetooth headphones.
[0248] The MIPI interface can be used to connect the processor 1410 to peripheral devices such as the display screen 1494 and the camera 1493. The MIPI interface includes a camera serial interface (CSI) and a display serial interface (DSI). In some embodiments, the processor 1410 and the camera 1493 communicate via the CSI interface to enable the electronic device's shooting function. The processor 1410 and the display screen 1494 communicate via the DSI interface to enable the electronic device's display function.
[0249] The GPIO interface is configurable via software. It can be configured as a control signal or a data signal. In some embodiments, the GPIO interface can be used to connect the processor 1410 to a camera 1493, a display screen 1494, a wireless communication module 1460, an audio module 1470, a sensor module 1480, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.
[0250] The USB 1430 interface conforms to the USB standard specification, specifically including Mini USB, Micro USB, and USB Type-C interfaces. The USB 1430 interface can be used to connect chargers to charge electronic devices, and also for data transfer between electronic devices and peripherals. It can also be used to connect headphones for audio playback. Furthermore, this interface can be used to connect other electronic devices, such as AR devices.
[0251] It is understood that the interface connection relationships between the modules illustrated in the embodiments of the present invention are merely illustrative and do not constitute a limitation on the structure of the electronic device. In other embodiments of this application, the electronic device may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.
[0252] The power management module 1441 connects the battery 1442, the charging management module 1440, and the processor 1410. The power management module 1441 receives input from the battery 1442 and / or the charging management module 1440, providing power to the processor 1410, internal memory 1421, external memory, display 1494, camera 1493, and wireless communication module 1460. The power management module 1441 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 1441 may also be located within the processor 1410. In other embodiments, the power management module 1441 and the charging management module 1440 may be housed in the same device.
[0253] The wireless communication function of electronic devices can be implemented through antenna 1, antenna 2, mobile communication module 1450, wireless communication module 1460, modem processor, and baseband processor.
[0254] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device can be used to cover one or more communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with a tuning switch.
[0255] The mobile communication module 1450 can provide solutions for wireless communication applications including 2G / 3G / 4G / 5G in electronic devices. The mobile communication module 1450 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 1450 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 1450 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 1450 may be housed in processor 1410. In some embodiments, at least some functional modules of the mobile communication module 1450 and at least some modules of the processor 1410 may be housed in the same device.
[0256] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through an audio device (not limited to speaker 1470A, receiver 1470B, etc.) or displays images or videos through the display screen 1494. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 1410 and may be housed in the same device as the mobile communication module 1450 or other functional modules.
[0257] The wireless communication module 1460 can provide solutions for wireless communication applications in electronic devices, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 1460 can be one or more devices integrating at least one communication processing module. The wireless communication module 1460 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 1410. The wireless communication module 1460 can also receive signals to be transmitted from processor 1410, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.
[0258] In some embodiments, antenna 1 of the electronic device is coupled to mobile communication module 1450, and antenna 2 is coupled to wireless communication module 1460, enabling the electronic device to communicate with networks and other devices via wireless communication technology. The wireless communication technology may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time-Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology, etc. The GNSS may include Global Positioning System (GPS), Global Navigation Satellite System (GLONASS), BeiDou Navigation Satellite System (BDS), Quasi-Zenith Satellite System (QZSS), and / or Satellite Based Augmentation Systems (SBAS).
[0259] Electronic devices implement display functions through a GPU, a display screen 1494, and an application processor. The GPU is a microprocessor for image processing, connecting the display screen 1494 and the application processor. The GPU performs mathematical and geometric calculations and is used for graphics rendering. The processor 1410 may include one or more GPUs, which execute program instructions to generate or modify display information.
[0260] Display screen 1494 is used to display images, videos, etc. Display screen 1494 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Mini LED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device may include one or N displays 1494, where N is a positive integer greater than 1.
[0261] Electronic devices can achieve shooting functions through ISP, camera 1493, video codec, GPU, display 1494 and application processor.
[0262] The ISP (Image Signal Processor) is used to process data fed back from the camera 1493. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimization of image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 1493.
[0263] Camera 1493 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the electronic device may include one or N cameras 1493, where N is a positive integer greater than 1.
[0264] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when an electronic device is selecting a frequency, a DSP can perform a Fourier transform on the frequency energy.
[0265] Video codecs are used to compress or decompress digital video. Electronic devices can support one or more video codecs. This allows the electronic device to play or record video in various encoded formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.
[0266] An NPU (Neural Processing Unit) is a computational processor for neural networks (NNs). By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.
[0267] The external storage interface 1420 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device. The external memory card communicates with the processor 1410 through the external storage interface 1420 to perform data storage functions. For example, music, video, and other files can be saved on the external memory card.
[0268] Internal memory 1421 can be used to store computer executable program code, which includes instructions. Processor 1410 executes various functional applications and data processing of the electronic device by running the instructions stored in internal memory 1421. Internal memory 1421 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of the electronic device (such as audio data, phonebook, etc.). Furthermore, internal memory 1421 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.
[0269] Electronic devices can implement audio functions, such as music playback and recording, through audio modules 1470, speakers 1470A, receivers 1470B, microphones 1470C, headphone jacks 1470D, and application processors.
[0270] The audio module 1470 is used to convert digital audio information into analog audio signal output, and also to convert analog audio input into digital audio signal. The audio module 1470 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 1470 may be located in the processor 1410, or some functional modules of the audio module 1470 may be located in the processor 1410.
[0271] The 1470A speaker, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. Electronic devices can listen to music or make hands-free calls through the 1470A speaker.
[0272] The receiver 1470B, also known as a "handpiece," is used to convert audio electrical signals into sound signals. When an electronic device answers a phone call or voice message, the receiver 1470B can be brought close to the ear to hear the voice.
[0273] Microphone 1470C, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can speak by bringing their mouth close to microphone 1470C, inputting the sound signal into microphone 1470C. Electronic devices can have at least one microphone 1470C. In some embodiments, electronic devices can have two microphones 1470C, which, in addition to collecting sound signals, can also perform noise reduction. In other embodiments, electronic devices can have three, four, or more microphones 1470C, enabling sound signal collection, noise reduction, sound source identification, and directional recording, among other functions.
[0274] The 1470D headphone jack is used to connect wired headphones. The 1470D headphone jack can be a USB 1430 interface or a 3.5mm Open Mobile Terminal Platform (OMTP) standard interface, a CTIA (Cellular Telecommunications Industry Association of the USA) standard interface.
[0275] Pressure sensor 1480A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 1480A can be disposed on display screen 1494. There are many types of pressure sensors 1480A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may include at least two parallel plates with conductive material. When force is applied to pressure sensor 1480A, the capacitance between the electrodes changes. The electronic device determines the pressure intensity based on the change in capacitance. When a touch operation is applied to display screen 1494, the electronic device detects the intensity of the touch operation based on pressure sensor 1480A. The electronic device can also calculate the touch position based on the detection signal from pressure sensor 1480A. In some embodiments, touch operations applied to the same touch position but with different touch operation intensities can correspond to different operation commands. For example: when a touch operation with an intensity less than a first pressure threshold is applied to the SMS application icon, a command to view an SMS is executed. When a touch operation with an intensity greater than or equal to the first pressure threshold is applied to the SMS application icon, a command to create a new SMS is executed.
[0276] The 1480E accelerometer can detect the magnitude of acceleration in various directions (typically three axes) of an electronic device. When the electronic device is stationary, it can detect the magnitude and direction of gravity. It can also be used to identify the posture of electronic devices, and is applicable to screen orientation switching, pedometers, and other applications.
[0277] Touch sensor 1480K, also known as a "touch panel," can be located on display screen 1494. The touch sensor 1480K and display screen 1494 together form a touchscreen, also known as a "touchscreen." Touch sensor 1480K detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 1494. In other embodiments, touch sensor 1480K may also be located on the surface of the electronic device, in a different position than display screen 1494.
[0278] Buttons 1490 include a power button, volume buttons, etc. Buttons 1490 can be mechanical buttons or touch-sensitive buttons. The electronic device can receive button input and generate key signal inputs related to user settings and function control of the electronic device.
[0279] Indicator 1492 can be an indicator light, which can be used to indicate charging status, power changes, messages, missed calls, notifications, etc.
[0280] The SIM card interface 1495 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 1495 to achieve contact and separation with the electronic device. The electronic device can support one or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 1495 can support Nano SIM cards, Micro SIM cards, and other SIM cards. Multiple cards can be inserted into the same SIM card interface 1495 simultaneously. The multiple cards can be of the same or different types. The SIM card interface 1495 is also compatible with different types of SIM cards. The SIM card interface 1495 is also compatible with external memory cards. The electronic device interacts with the network through the SIM card to achieve functions such as calls and data communication. In some embodiments, the electronic device uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device and cannot be separated from it.
[0281] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.
[0282] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0283] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0284] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between devices or units through some interfaces, and may be electrical, mechanical, or other forms.
[0285] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0286] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0287] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to an electronic device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.
[0288] Finally, it should be noted that the above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A method of video transmission, characterized by, The method is applied to a video transmission system consisting of a transmitting device and a receiving device, and the method includes: The transmitting device determines two sets of video frames based on the video data of the target video. Each set of video frames has the same resolution, and the resolution of one set of video frames is greater than that of the other set. The frame number of any video frame in one set is different from that of any video frame in the other set. Each set of video frames includes multiple video frames with discontinuous frame numbers. The combined video frames from the two sets of video frames have continuous frame numbers, and each continuous video frame has a one-to-one correspondence with the frame numbers of the target video frames. However, the frame numbers of the video frames in the set with the higher resolution are not continuous. The transmitting device encodes each group of video frames to obtain the encoded data corresponding to each group of video frames. The transmitting device sends first encoded data and second encoded data corresponding to the two sets of video frames, respectively, wherein the resolution corresponding to the first encoded data is greater than the resolution corresponding to the second encoded data. The receiving device receives the first encoded data and the second encoded data; The receiving device decodes the first encoded data and the second encoded data respectively to obtain first decoded data corresponding to the first encoded data and second decoded data corresponding to the second encoded data, wherein the resolution of the first decoded data is greater than the resolution of the second decoded data. The receiving device processes the second decoded data according to the first decoded data to obtain a restored frame with the same resolution as the first decoded data, including: the receiving device inputs the first video frame of the second decoded data and at least one video frame with a frame number consecutive to the first video frame into the AI super-resolution model to obtain the restored frame corresponding to the first video frame. The receiving device combines the first decoded data and the restored frame to obtain the video data of the target video.
2. The method of claim 1, wherein, The video data of the target video includes: first video data and second video data; The transmitting device determines two sets of video frames based on the video data of the target video, including: The transmitting device downsamples the first video data to obtain the second video data, wherein the resolution of each video frame in the first video data is higher than the resolution of each video frame in the second video data. The transmitting device selects multiple video frames from the first video data and the second video data respectively to obtain the two sets of video frames.
3. The method of claim 1, wherein, The target video has the same resolution for each video frame in its video data; The transmitting device determines two sets of video frames based on the video data of the target video, including: The transmitting device selects a portion of video frames from the video data of the target video to obtain a set of video frames; The transmitting device downsamples another portion of the video frames in the target video's video data to obtain another set of video frames.
4. The method according to claim 2 or 3, characterized in that, In the set of video frames with the larger resolution, the difference between the frame numbers of any two adjacent video frames is m, where m is a positive integer greater than 1.
5. The method according to any one of claims 2 to 4, characterized in that, The set of video frames with the lower resolution includes multiple sets of consecutively numbered video frames. Each set of consecutively numbered video frames includes n video frames, where n is a positive integer greater than or equal to 1.
6. The method according to any one of claims 1 to 5, characterized in that, Before the transmitting device determines two sets of video frames based on the video data of the target video, the method further includes: The transmitting device obtains the video data of the target video from the storage space; Alternatively, the sending device may collect video data of the target video in real time.
7. The method according to any one of claims 1 to 6, characterized in that, The transmitting device transmits the first encoded data and the second encoded data corresponding to the two sets of video frames, respectively, including: The transmitting device sends the first encoded data and the second encoded data corresponding to the two sets of video frames to the receiving device; Alternatively, the sending device sends the first encoded data and the second encoded data corresponding to the two sets of video frames to the server, and the server forwards the first encoded data and the second encoded data corresponding to the two sets of video frames to the receiving device.
8. A video transmission method characterized by comprising: The method includes: Two sets of video frames are determined based on the video data of the target video. Each set of video frames has the same resolution, and the resolution of one set of video frames is greater than that of the other set. The frame number of any video frame in one set is different from that of any video frame in the other set. Each set of video frames includes multiple video frames with discontinuous frame numbers. The combined video frames from the two sets of video frames have continuous frame numbers, and each continuous video frame has a one-to-one correspondence with the frame numbers of the target video frames. However, the frame numbers of the video frames in the set with the higher resolution are not continuous. Each group of video frames is encoded separately to obtain the encoded data corresponding to each group of video frames; Send the encoded data as described.
9. The method of claim 8, wherein, The video data of the target video includes: first video data and second video data; The step of determining two sets of video frames based on the video data of the target video includes: The first video data is downsampled to obtain the second video data, wherein the resolution of each video frame in the first video data is higher than the resolution of each video frame in the second video data; Multiple video frames are selected from the first video data and the second video data respectively to obtain the two sets of video frames.
10. The method of claim 8, wherein, The target video has the same resolution for each video frame in its video data; The step of determining two sets of video frames based on the video data of the target video includes: A subset of video frames is selected from the video data of the target video to obtain a set of video frames; Another portion of the video frames in the target video's video data is downsampled to obtain another set of video frames.
11. The method according to claim 9 or 10, characterized in that, In the set of video frames with the larger resolution, the difference between the frame numbers of any two adjacent video frames is m, where m is a positive integer greater than 1.
12. The method according to any one of claims 9 to 11, characterized in that, The set of video frames with the lower resolution includes multiple sets of consecutively numbered video frames. Each set of consecutively numbered video frames includes n video frames, where n is a positive integer greater than or equal to 1.
13. A method of video transmission, the method comprising: The method includes: The system receives first and second encoded data of the target video, wherein the first and second encoded data correspond to different resolutions. The first and second encoded data are obtained by encoding two sets of video frames separately. These two sets of video frames are determined based on the video data of the target video. Each set of video frames has the same resolution, with one set having a higher resolution than the other. The frame number of any frame in one set differs from that in the other set. Each set of video frames includes multiple frames with discontinuous frame numbers. The combined video frames from the two sets have consecutive frame numbers, and each consecutive frame number corresponds one-to-one with the frame numbers of the target video. The first encoded data and the second encoded data are decoded respectively to obtain first decoded data corresponding to the first encoded data and second decoded data corresponding to the second encoded data, wherein the resolution of the first decoded data is greater than the resolution of the second decoded data. Based on the first decoded data, the second decoded data is processed to obtain a restored frame with the same resolution as the first decoded data. This includes: the receiving device inputs the first video frame of the second decoded data and at least one video frame with a frame number consecutive to the first video frame into the AI super-resolution model to obtain the restored frame corresponding to the first video frame. The first decoded data and the restored frame are combined to obtain the video data of the target video.
14. A video transmission system characterized by comprising: The video transmission system includes: a transmitting device and a receiving device; The transmitting device is used to perform the video transmission method as described in any one of claims 8 to 12; The receiving device is used to perform the video transmission method as described in claim 13.
15. An electronic device, comprising: include: A processor for running a computer program stored in a memory to implement the video transmission method as described in any one of claims 8 to 12 or claim 13.
16. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the video transmission method as described in any one of claims 8 to 12 or claim 13.
17. A chip system, characterized by The chip system includes a memory and a processor, the processor executing a computer program stored in the memory to implement the video transmission method as described in any one of claims 8 to 12 or claim 13.