Video decoding method and related equipment

Abstract

The present application provides a video decoding method, including: receiving a video stream, the video stream carrying a video level, a video resolution, and a number of reference frames actually used in video encoding; obtaining the video level, video resolution, and the number of reference frames actually used in video encoding from the video stream; calculating the optimal image cache size according to the video resolution and the number of reference frames actually used in video encoding; obtaining the optimal video level corresponding to the optimal image cache size according to the correspondence between the maximum decoded image cache size and the video level; if the video level in the video stream is greater than the optimal video level, modifying the video level in the video stream to the optimal video level; inputting the video stream after the video level modification into a video decoder for decoding. The present application also provides a computer-readable storage medium, an electronic device, and a chip system. The present application can reduce the delay of video decoding.

Description

Technical Field

[0001] The present application relates to the field of video encoding and decoding technology, and in particular to a video decoding method and related equipment. Background Art

[0002] Existing video encoding often refers to the previous image frames for encoding. Accordingly, when decoding a video, it is necessary to refer to the previous image frames (i.e., forward reference frames), so it is necessary to set up a certain video buffer to cache the decoded image frames. The current video decoding, such as H.264-based video decoding, only specifies how to calculate the maximum number of cache frames. In order to ensure compatibility, many video decoders cache according to the calculated maximum number of cache frames, and the image frames in the video buffer will not be output in advance, but will wait until the video buffer is full before outputting the first stored frame.

[0003] The current video decoding method calculates a large maximum number of cache frames, but for video streams with a small number of reference frames actually used during video encoding, there will be over-caching, resulting in increased video delay during decoding. Summary of the invention

[0004] The embodiments of the present application disclose a video decoding method and related devices, which can reduce the delay of video decoding.

[0005] In a first aspect, the present application discloses a video decoding method, which is applied to an electronic device, and the method comprises:

[0006] receiving a video stream, wherein the video stream carries a video level, a video resolution, and a number of reference frames actually used for video encoding;

[0007] Acquire the video level, video resolution, and number of reference frames actually used for video encoding from the video stream;

[0008] Calculating an optimal image cache size according to the video resolution and the number of reference frames actually used in video encoding;

[0009] According to the correspondence between the maximum decoded image cache size and the video level, obtaining the optimal video level corresponding to the optimal image cache size;

[0010] If the video level in the video stream is greater than the optimal video level, modifying the video level in the video stream to the optimal video level;

[0011] The video stream after the video level modification is input to the video decoder for decoding.

[0012] The video decoding method of the present application does not directly decode the video stream by the video decoder after the electronic device receives the video stream, but optimizes and modifies the video level of the video stream before passing it to the video decoder. The video decoder caches the video according to the modified video level, thereby reducing the delay of video decoding of the electronic device.

[0013] In some optional implementations, obtaining the video level, video resolution, and number of reference frames actually used for video encoding from the video stream includes:

[0014] Obtain a sequence parameter set SPS frame from the video stream;

[0015] The video level, video resolution, and number of reference frames actually used for video encoding are obtained from the SPS frame.

[0016] In some optional implementations, the correspondence between the maximum decoded image cache size and the video level is pre-stored in the electronic device.

[0017] In some optional implementations, the video stream is encoded according to the H.264 standard.

[0018] In some optional implementations, the optimal image cache size is calculated according to the following formula:

[0019] Optimal image cache size = (PicWidthInMbs*FrameHeightInMbs*384) / 1024*number of reference frames actually used in video encoding;

[0020] Among them, PicWidthInMbs=round up (video width / m), FrameHeightInMbs=round up (video height / n), the macroblock size of the video stream is m*n, and the video resolution=video width*video height.

[0021] In some optional implementations, the video stream includes a real-time network video stream.

[0022] In some optional implementations, obtaining the optimal video level corresponding to the optimal image cache size according to the correspondence between the maximum decoded image cache size and the video level includes:

[0023] Obtaining an actual macroblock processing rate and an actual video bit rate of the video stream;

[0024] According to the correspondence between the maximum decoded image cache size, the maximum macroblock processing rate, the maximum video bit rate and the video level, the optimal video level corresponding to the optimal image cache size, the actual macroblock processing rate and the actual video bit rate is obtained.

[0025] In some optional embodiments, the method further comprises:

[0026] Transmit the video stream with the modified video level to other devices.

[0027] By transmitting the video stream with the modified video level to other devices, the other devices can cache according to the modified video level, thereby reducing the delay of video decoding of other devices.

[0028] In some optional implementations, before obtaining the video level, video resolution, and number of reference frames actually used for video encoding from the video stream, the method further includes:

[0029] If the video stream is an encrypted code stream, the video stream is decrypted.

[0030] Encrypting and transmitting the video stream can improve the security of the video data.

[0031] In some optional embodiments, the method further comprises:

[0032] The decoded video stream is sent to the display for playback.

[0033] The second aspect of the present application discloses a computer-readable storage medium, comprising computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the video decoding method as described in the first aspect.

[0034] The third aspect of the present application discloses an electronic device, which includes a processor and a memory, wherein the memory is used to store instructions, and the processor is used to call the instructions in the memory so that the electronic device executes the video decoding method described in the first aspect.

[0035] The fourth aspect of the present application discloses a chip system, which is applied to an electronic device; the chip system includes an interface circuit and a processor; the interface circuit and the processor are interconnected through lines; the interface circuit is used to receive signals from the memory of the electronic device and send signals to the processor, the signals including computer instructions stored in the memory; when the processor executes the computer instructions, the chip system executes the video decoding method described in the first aspect.

[0036] It should be understood that the computer-readable storage medium described in the second aspect, the electronic device described in the third aspect, and the chip system described in the fourth aspect all correspond to the method of the first aspect. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a schematic diagram of an application scenario of the video decoding method disclosed in an embodiment of the present application.

[0038] Figure 2 It is a flowchart of the video decoding method disclosed in the embodiment of the present application.

[0039] Figure 3 It is a schematic diagram of the correspondence between the maximum decoded image cache size and the video level.

[0040] Figure 4 It is a schematic diagram of the structure of an electronic device disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0041] In order to better understand the video decoding method and related devices disclosed in the embodiments of the present application, the application scenario of the video decoding method of the present application is first described below.

[0042] Figure 1 Schematic diagram of the application scenario of the video decoding method disclosed in the embodiment of the present application. Figure 1 As shown, the video decoding method is applied to an electronic device 10. The electronic device 10 includes a video decoder 100 and a display 101. The electronic device 10 receives a video stream from an external device 11, modifies the video level in the video stream to an optimal video level (from a first video level, such as 5.1, to a second video level, such as 3 in the figure), the video decoder 100 sets a video buffer according to the video level in the video stream, uses the video buffer to decode image frames in the video stream, transmits the decoded image frames to the display 101, and the display 101 displays the video according to the decoded image frames.

[0043] In one embodiment of the present application, the video stream may be a real-time network video stream, such as a live network video stream.

[0044] The electronic device 10 may include a smart phone, a tablet computer, a desktop computer, a laptop computer, a car computer, etc.

[0045] The external device 11 may include a network camera, a video server, etc.

[0046] The electronic device 10 and the external device 11 may be connected in a wired manner, such as through a USB (Universal Serial Bus) or wirelessly, such as through infrared, Bluetooth, or WiFi.

[0047] Figure 2: is a flow chart of a video decoding method disclosed in an embodiment of the present application. The video decoding method is applied in an electronic device, and specifically includes:

[0048] 201, an electronic device receives a video stream from an external device, where the video stream carries a video level, a video resolution, and a number of reference frames actually used for video encoding.

[0049] The video stream may include a code stream encoded according to the H.264 standard.

[0050] 202, the electronic device obtains the video level, video resolution, and the number of reference frames actually used for video encoding from the video stream.

[0051] In one embodiment of the present application, the video stream is encoded according to the H.264 standard, and obtaining the video level, video resolution, and number of reference frames actually used for video encoding from the video stream includes:

[0052] Obtaining a sequence parameter set (SPS) frame from the video stream;

[0053] The video level, video resolution, and number of reference frames actually used for video encoding are obtained from the SPS frame.

[0054] The SPS frame may be the first frame of a video stream, and obtaining the first frame of a video stream is to obtain the SPS frame.

[0055] It should be noted that if the video level, video resolution, and video encoding actually used are contained in other data units of the video stream (such as the header information of the video stream), the video level, video resolution, and reference frame number actually used for the video encoding are obtained from the other data units of the video stream accordingly.

[0056] 203. The electronic device calculates an optimal image cache size according to the video resolution and the number of reference frames actually used for video encoding.

[0057] In one embodiment of the present application, the video stream is encoded according to the H.264 standard, and the optimal image cache size is calculated according to the following formula:

[0058] Optimal image cache size = (PicWidthInMbs*FrameHeightInMbs*384) / 1024*number of reference frames actually used in video encoding;

[0059] Among them, PicWidthInMbs = round up (video width / m), FrameHeightInMbs = round up (video height / n), the macroblock size of the video stream is m*n, and the video resolution = video width*video height.

[0060] For example, the number of reference frames actually used in video encoding is 1, the video resolution is 1920*1080, that is, the video width is 1920 (pixels), the video height is 1080 (pixels), and the macroblock size is 16*16, then PicWidthInMbs=round up (video width / 16)=round up (1920 / 16)=120, FrameHeightInMbs=round up (video height / 16)=round up (1080 / 16)=round up (67.5)=68, the optimal image cache size=(PicWidthInMbs*FrameHeightInMbs*384) / 1024*the number of reference frames actually used in video encoding=(120*68*384) / 1024*1=3060.

[0061] 204. The electronic device obtains an optimal video level corresponding to the optimal image cache size according to a correspondence between the maximum decoded image cache size and the video level.

[0062] In one embodiment of the present application, the correspondence between the maximum decoded image cache size and the video level is pre-stored in the electronic device.

[0063] Figure 3 This is a schematic diagram of the correspondence between the maximum decoded image cache size and the video level. Figure 3 , the maximum decoded image cache size 1782.0 corresponds to a video level of 2.1, the maximum decoded image cache size 3037.5 corresponds to a video level of 3, and the maximum decoded image cache size 6750 corresponds to a video level of 3.1. The maximum decoded image cache size closest to the optimal image cache size 3060 is 3037.5, so it can be determined that the optimal video level corresponding to the optimal image cache size 3060 is 3.

[0064] In other embodiments, the video level is not only related to the decoded image cache size, but also to the macroblock processing rate and the video bit rate. The actual macroblock processing rate and the actual video bit rate of the video stream can be obtained, and the optimal video level corresponding to the optimal image cache size, the actual macroblock processing rate and the actual video bit rate can be obtained according to the correspondence between the maximum decoded image cache size, the maximum macroblock processing rate, the maximum video bit rate and the video level.

[0065] 205 , if the video level in the video stream is greater than the optimal video level, the electronic device modifies the video level in the video stream to the optimal video level.

[0066] For example, the video level in the video stream is 5.1, and the optimal video level corresponding to the optimal image cache size 3060 is 3. The video level 5.1 in the video stream is greater than the optimal video level 3 corresponding to the optimal image cache size 3060. Therefore, the video level in the video stream is modified to the optimal video level 3.

[0067] It should be understood that if the video level in the video stream is less than or equal to the optimal video level, the electronic device directly inputs the video stream into the video decoder for decoding.

[0068] 206 , the electronic device inputs the video stream with the modified video level into a video decoder for decoding.

[0069] For example, a video stream whose video level is modified from 5.1 to 3 is input to a video decoder for decoding. The video encoder sets a video buffer according to the modified video level, and uses the video buffer to decode image frames in the video stream.

[0070] For example, the video level in the modified video stream is changed to 3, the corresponding maximum video bit rate (MaxDPB) is 3037.5, and the maximum number of cached frames is 1024*MaxDPB / (PicWidthInMbs*FrameHeightInMbs*384)=1024*3037.5 / (120*68*384)=0.99≈1. Therefore, the video decoder sets the video buffer to 1 frame, and after receiving the second image frame, decodes the second image frame with reference to the decoded first image frame, writes the decoded second image frame into the video buffer, and outputs the decoded first image frame. If the frame rate of the video stream is 30 frames per second (i.e., one frame is received every 33.3ms), the delay of video decoding is 1*33.3=33.3ms.

[0071] After an electronic device receives a video stream, the present application does not directly decode it by a video decoder, but optimizes and modifies the video level of the video stream before passing it to the video decoder. The video decoder caches the video according to the modified video level, thereby reducing the delay of video decoding of the electronic device.

[0072] For example, for a video stream with a video resolution of 1920*1080, a frame rate of 30FPS (frames per second, one frame received every 33.3ms), and a video level of 5.1, the number of reference frames actually used in video encoding is 1. After entering the video decoder, it will be recognized that 16 forward reference frames are required. After the 16th frame image is sent to the video buffer, the video decoder outputs the first frame image from the video decoder. Therefore, the time required for the video decoder to output the first frame image is 15*33.3=499.5ms. And for each subsequent frame image, there is a delay of nearly 500ms from the time the video decoder receives the image to the time the image is output from the video buffer. If only one frame image is cached, the video decoder can output the first frame image when the second frame image is received, and the delay is 1*33.3=33.3ms. Compared with the actual number of reference frames used in video encoding, the delay of video decoding using the calculated maximum number of cached frames is increased by 466.2ms.

[0073] In other embodiments, the method further comprises:

[0074] Transmit the video stream with the modified video level to other devices.

[0075] By transmitting the video stream with the modified video level to other devices, the other devices can cache according to the modified video level, thereby reducing the delay of video decoding of other devices.

[0076] In other embodiments, before obtaining the video level, video resolution, and number of reference frames actually used for video encoding from the video stream, the method further includes:

[0077] If the video stream is an encrypted code stream, the video stream is decrypted.

[0078] The external device can send the encrypted video stream to the electronic device. If the received video stream is an encrypted code stream, the electronic device decrypts the video stream. For example, the external device encrypts the video stream using the Advanced Encryption Standard (AES) encryption algorithm, and correspondingly, the electronic device decrypts the video stream using the AES decryption algorithm. Encrypted video transmission between the external device and the electronic device can improve the security of the video data.

[0079] In other embodiments, the method further comprises:

[0080] The decoded video stream is sent to the display for playback.

[0081] The decrypted video stream can be played through a video player, for example, through Youku, iQiyi or other video applications.

[0082] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. For ordinary technicians in this field, improvements can be made without departing from the creative concept of the present application, but these all fall within the protection scope of the present application.

[0083] See also Figure 4 , Figure 4 Schematic diagram of the structure of the electronic device (electronic device or second electronic device) disclosed in this application. Figure 4 As shown, the electronic device 40 includes one or more processors 401 and a memory 402. The processor 401 and the memory 402 are connected via one or more communication buses 403. The memory 402 is used to store one or more computer programs 404. The one or more computer programs 404 are configured to be executed by the processor 401. The one or more computer programs 404 include instructions, which can be used to execute the various steps performed by the electronic device (such as a mobile phone) in the above embodiment to implement the video decoding function in the electronic device.

[0084] This embodiment further provides a computer storage medium, in which computer instructions are stored. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the video decoding method in the above-mentioned embodiment.

[0085] This embodiment further provides a computer program product. When the computer program product is run on an electronic device, the electronic device executes the above-mentioned related steps to implement the video decoding method in the above-mentioned embodiment.

[0086] In addition, an embodiment of the present application also provides a device, which can specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor can execute the computer-executable instructions stored in the memory so that the chip executes the video decoding method in the above-mentioned method embodiments.

[0087] Among them, the electronic device, computer storage medium, computer program product or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above and will not be repeated here.

[0088] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0089] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0090] The unit described as a separate component may or may not be physically separated, and the component shown as a unit may be one physical unit or multiple physical units, that is, it may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiment.

[0091] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0092] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program code.

[0093] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A video decoding method, applied to an electronic device, characterized in that: The method comprises: receiving a video stream, wherein the video stream carries a video level, a video resolution, and a number of reference frames actually used for video encoding; Acquire the video level, video resolution, and number of reference frames actually used for video encoding from the video stream; Calculating an optimal image cache size according to the video resolution and the number of reference frames actually used in video encoding; According to the correspondence between the maximum decoded image cache size and the video level, obtaining the optimal video level corresponding to the optimal image cache size; If the video level in the video stream is greater than the optimal video level, modifying the video level in the video stream to the optimal video level; The video stream after the video level modification is input to the video decoder for decoding.

2. The video decoding method according to claim 1, wherein: The step of obtaining the video level, video resolution, and number of reference frames actually used for video encoding from the video stream includes: Obtain a sequence parameter set SPS frame from the video stream; The video level, video resolution, and number of reference frames actually used for video encoding are obtained from the SPS frame.

3. The video decoding method according to claim 1, wherein: The correspondence between the maximum decoded image cache size and the video level is pre-stored in the electronic device.

4. The video decoding method according to claim 1, wherein: The video stream is encoded according to the H.264 standard.

5. The video decoding method according to claim 4, characterized in that: The optimal image cache size is calculated according to the following formula: Optimal image cache size = (PicWidthInMbs*FrameHeightInMbs*384) / 1024*number of reference frames actually used in video encoding; Among them, PicWidthInMbs=round up (video width / m), FrameHeightInMbs=round up (video height / n), the macroblock size of the video stream is m*n, and the video resolution=video width*video height.

6. The video decoding method according to claim 1, wherein: The video stream includes a real-time network video stream.

7. The video decoding method according to claim 1, wherein: The obtaining the optimal video level corresponding to the optimal image cache size according to the correspondence between the maximum decoded image cache size and the video level comprises: Obtaining an actual macroblock processing rate and an actual video bit rate of the video stream; According to the correspondence between the maximum decoded image cache size, the maximum macroblock processing rate, the maximum video bit rate and the video level, the optimal video level corresponding to the optimal image cache size, the actual macroblock processing rate and the actual video bit rate is obtained.

8. The video decoding method according to any one of claims 1 to 7, characterized in that: The method further comprises: Transmit the video stream with the modified video level to other devices.

9. The video decoding method according to any one of claims 1 to 7, characterized in that: Before obtaining the video level, video resolution, and number of reference frames actually used for video encoding from the video stream, the method further includes: If the video stream is an encrypted code stream, the video stream is decrypted.

10. The video decoding method according to any one of claims 1 to 7, characterized in that: The method further comprises: The decoded video stream is sent to the display for playback.

11. A computer-readable storage medium, characterized in that: The method comprises computer instructions, and when the computer instructions are executed on an electronic device, the electronic device executes the video decoding method according to any one of claims 1 to 10.

12. An electronic device, characterized in that: The electronic device comprises a processor and a memory, wherein the memory is used to store instructions, and the processor is used to call the instructions in the memory, so that the electronic device executes the video decoding method according to any one of claims 1 to 10.

13. A chip system, which is applied to an electronic device; the chip system includes an interface circuit and a processor; the interface circuit and the processor are interconnected through lines; the interface circuit is used to receive signals from a memory of the electronic device and send signals to the processor, the signals including computer instructions stored in the memory; when the processor executes the computer instructions, the chip system executes the video decoding method as described in any one of claims 1 to 10.

Images