Video encoding and decoding method, code stream processing method and related device

Abstract

A video coding method is applied to a video coding process. In the method, for a video comprising a plurality of video clips, only a first image in a first video clip is completely coded. Moreover, the non-first image in each video clip is coded by taking the previous image as the reference frame, and the first image in other video clips except the first video clip is coded by taking the first image in the first video clip as the reference frame, so that the number of the images needing to be completely coded is effectively reduced, and the coding efficiency is improved. The data size of the code stream obtained after video coding is reduced, and fast decoding of the code stream is not affected.

Description

Technical Field

[0001] The present application relates to the field of video processing technology, and in particular to a video encoding and decoding method, a bit stream processing method and related devices. Background Art

[0002] With the popularization of the Internet, video transmission is becoming more and more common in various fields of society. In some specific scenarios (such as security scenarios and multimedia scenarios), video transmission has become an indispensable role.

[0003] Generally, in the process of video transmission, in order to reduce the amount of video data that needs to be transmitted, the video is usually encoded, thereby achieving compression of the video data. In the related art, when encoding a video, a long video is usually divided into multiple short video segments, and the first image in each video segment is encoded independently (i.e., encoded based on all the contents of the image), and the other images after the first image in each video segment are encoded with reference to the previous image (i.e., encoded based on the difference between the current image and the previous image). Since the difference between adjacent images in a video is usually relatively small, for most images in a video, only the slight difference between this part of the image and other images can be encoded, thereby effectively reducing the amount of encoded data.

[0004] However, in the related art, the amount of data after video encoding is still large, and it is difficult to promote and apply in some scenarios where storage resources or bandwidth resources are limited. Summary of the invention

[0005] The present application provides a video encoding method, which can reduce the data volume of a code stream obtained after video encoding without affecting the fast decoding of the code stream.

[0006] In a first aspect, the present application provides a video encoding method, which is applied to a video encoding end. The method includes: the video encoding end first obtains a video, and segments the obtained video in a pre-specified manner. That is, the video encoding end divides the obtained video into a plurality of continuous and non-overlapping video segments. In this way, the video obtained by the video encoding end actually includes a plurality of video segments, and each video segment includes a plurality of images.

[0007] Then, the target image in the video is used as a key frame, and the video encoding end encodes the target image to obtain encoding data of the target image, where the target image is the first image in the first video segment of the multiple video segments. When the target image is encoded using the target image as a key frame, the picture content representing the target image needs to be completely encoded without referring to other images to complete the encoding of the target image.

[0008] Finally, in the process of encoding any image located after the target image in the video, the image is encoded based on the reference frame corresponding to the image to obtain the encoded data of the image. The encoded data of the target image and the encoded data of the image after the target image are used to form the code stream of the video. In addition, the first image in each video segment after the first video segment in the multiple video segments is a first-category image, the reference frame corresponding to the first-category image is the target image, the image other than the first image in each video segment is a second-category image, and the reference frame corresponding to the second-category image is the previous image of the second-category image in the video.

[0009] That is to say, each image after the target image in the video has a corresponding reference frame during the encoding process, and the reference frames corresponding to images at different positions are different. That is, the reference frame corresponding to the first image in the video clip is the target image, and the reference frame corresponding to the non-first image in the video clip is the previous image.

[0010] In this solution, for a video including multiple video clips, only the first image in the first video clip is fully encoded. In addition, the non-first image in each video clip is encoded with the previous image as the reference frame, and the first image in other video clips other than the first video clip is encoded with the first image in the first video clip as the reference frame, thereby effectively reducing the number of images that need to be fully encoded and the amount of data in the bitstream obtained after video encoding without affecting the fast decoding of the bitstream.

[0011] In a possible implementation, when encoding an image after a target image, the difference between the image and its corresponding reference frame can be determined based on the reference frame corresponding to the image. That is, for any image after the target image, the difference in picture content between the image and the corresponding reference frame can be determined based on the reference frame corresponding to the image. The original image can be restored based on the picture content of the reference frame and the obtained difference. In this way, encoding can be performed based on the obtained difference between the images to obtain the encoded data of the image, wherein the encoded data of the image is used to indicate the difference between the image and its corresponding reference frame.

[0012] In this scheme, for images with reference frames, the encoding process no longer encodes the entire image, but encodes the difference between the image and the reference frame. This makes it necessary to only transmit the slight difference between the image and the reference frame when transmitting the bitstream, without the need to transmit the complete content of the image, thus effectively reducing the amount of data transmission.

[0013] In a possible implementation, each of the multiple video segments includes the same number of images, for example, each video segment includes 50 images or 100 images.

[0014] In this solution, by setting each video segment to include the same number of images, it is possible to facilitate segmentation of the video and ensure that the encoded data corresponding to the image at each position in each video segment can be quickly determined in the bit stream obtained after encoding, which is beneficial to the rapid decoding of the video.

[0015] In a possible implementation, the number of images included in the video does not exceed a preset threshold. The preset threshold can be set according to actual conditions, for example, the preset threshold is 1000, 2000 or 10000. In a scenario where the image changes little, the preset threshold can be set to a larger value; in a scenario where the image changes little, the preset threshold can be set to a smaller value.

[0016] Therefore, in this solution, by limiting the number of images included in the video, the number of images between key frames in the bitstream can be specified to ensure that the distance between key frames is within a controllable range, thereby ensuring that the data volume of the final encoded bitstream can be as small as possible.

[0017] In a possible implementation, the video acquisition scenario is a security scenario.

[0018] In security scenarios, cameras used to capture video are usually fixedly deployed at specific locations, so most of the image content in the images captured by the camera is fixed, and the image content between adjacent images does not change much. Therefore, when the method provided in this embodiment is used for encoding in security scenarios, the difference between the encoded image and its corresponding reference frame is not large, and ultimately the amount of data in the encoded code stream can be significantly reduced.

[0019] The second aspect of the present application provides a video decoding method, which is applied to a video decoding end. The method includes: obtaining a code stream of a video, the code stream includes coded data corresponding to an image in the video, the video includes multiple continuous video clips, and each video clip includes multiple images; decoding the coded data corresponding to a target image in the code stream to obtain a target image, the target image is the first image in the first video clip of the multiple video clips, and the target image is a key frame; in the process of decoding an image located after the target image in the video, based on a reference frame corresponding to the image, decoding the coded data corresponding to the image in the code stream to obtain an image; wherein the first image in each video clip after the first video clip in the multiple video clips is a first-type image, the reference frame corresponding to each first-type image is a target image, the image other than the first image in each video clip is a second-type image, and the reference frame corresponding to the second-type image is the previous image of the second-type image in the video.

[0020] In a possible implementation, when decoding each image after the target image, the coded data corresponding to the image in the code stream may be parsed first to obtain the difference between the image and its corresponding reference frame. That is, for any image after the target image, the difference in picture content between the image and the corresponding reference frame may be determined based on the coded data corresponding to the image. Then, the image is generated based on the picture content of the reference frame corresponding to the image and the difference between the image and the reference frame.

[0021] In a possible implementation manner, each video segment in the multiple video segments includes the same number of images.

[0022] In a possible implementation manner, the number of images included in the video does not exceed a preset threshold.

[0023] In a possible implementation, the video acquisition scenario is a security scenario.

[0024] In a third aspect, the present application provides a code stream processing method, comprising: obtaining a first code stream corresponding to a video, the first code stream comprising coded data corresponding to an image in the video, the video comprising a plurality of continuous video segments, and each video segment comprising a plurality of images; decoding the coded data of a target image in the first code stream to obtain a target image, the target image being the first image in a first video segment of the plurality of video segments, and the target image being a key frame; taking the target image as a reference frame, decoding the coded data corresponding to a first category of images to obtain a first category of images, wherein the first image in each video segment after the first video segment in the plurality of video segments belongs to the first category of images; taking the first category of images as key frames, encoding the first category of images to obtain new coded data corresponding to the first category of images; replacing the coded data of the first category of images in the first code stream with the new coded data corresponding to the first category of images to obtain a second code stream corresponding to the video.

[0025] That is to say, in the first code stream, the coded data corresponding to the first type of image is obtained by encoding the target image as a reference frame, that is, the coded data of the first type of image actually indicates the difference between the first type of image and the target image; and in the second code stream, the coded data corresponding to the first type of image is obtained by encoding the first type of image itself.

[0026] In this solution, by decoding the encoded data corresponding to the first image of the video clip in the bitstream and re-encoding it into the encoded data of the key frame, it is possible to convert a bitstream with a smaller data volume into a regular bitstream, so that devices that do not support specific decoding methods can still achieve normal decoding of the bitstream, ensuring the applicability of the solution in various scenarios.

[0027] In a possible implementation, in the process of decoding the encoded data of the first type of image with the target image as a reference frame, the encoded data corresponding to the first type of image in the first code stream can be parsed to obtain the difference between the first type of image and the target image; based on the content and the difference of the target image, the first type of image is generated.

[0028] In a possible implementation manner, each video segment in the multiple video segments includes the same number of images.

[0029] In a possible implementation manner, the number of images included in the video does not exceed a preset threshold.

[0030] In a possible implementation, the video acquisition scenario is a security scenario.

[0031] The fourth aspect of the present application provides a video encoding device, including: an acquisition module, used to acquire a video to be encoded, the video includes multiple continuous video clips, and each video clip includes multiple images; an encoding module, used to encode the target image in the video as a key frame, and obtain encoding data of the target image, the target image is the first image in the first video clip of the multiple video clips; the encoding module is also used to encode the image based on the reference frame corresponding to the image in the process of encoding the image located after the target image in the video, and obtain the encoding data of the image, the encoding data of the target image and the encoding data of the image are used to constitute a code stream of the video; wherein the first image in each video clip after the first video clip in the multiple video clips is a first type of image, the reference frame corresponding to the first type of image is the target image, the image other than the first image in each video clip is a second type of image, and the reference frame corresponding to the second type of image is the previous image of the second type of image in the video.

[0032] In one possible implementation, the encoding module is further used to: determine the difference between the image and the reference frame corresponding to the image based on the reference frame corresponding to the image; and perform encoding based on the difference to obtain encoding data of the image, wherein the encoding data of the image is used to indicate the difference between the image and the reference frame corresponding to the image.

[0033] In a possible implementation manner, each video segment in the multiple video segments includes the same number of images.

[0034] In a possible implementation manner, the number of images included in the video does not exceed a preset threshold.

[0035] In a possible implementation, the video acquisition scenario is a security scenario.

[0036] According to a fifth aspect of the present application, there is provided a video decoding device, comprising: an acquisition module, for acquiring a code stream of a video, the code stream comprising encoded data corresponding to an image in the video, the video comprising a plurality of continuous video segments, and each video segment comprising a plurality of images; a decoding module, for decoding the encoded data corresponding to a target image in the code stream to obtain a target image, the target image being the first image in a first video segment of the plurality of video segments, and the target image being a key frame; the decoding module, further for decoding the encoded data corresponding to the image in the code stream based on a reference frame corresponding to the image in a process of decoding an image located after the target image in the video to obtain an image; wherein the first image in each video segment after the first video segment in the plurality of video segments is a first type of image, the reference frame corresponding to the first type of image is the target image, the image in each video segment other than the first image is a second type of image, and the reference frame corresponding to the second type of image is the previous image of the second type of image in the video.

[0037] In a possible implementation, the decoding module is further used to: parse the coded data corresponding to the image in the code stream to obtain the difference between the image and the reference frame corresponding to the image; and generate the image based on the content and difference of the reference frame corresponding to the image.

[0038] In a possible implementation manner, each video segment in the multiple video segments includes the same number of images.

[0039] In a possible implementation manner, the number of images included in the video does not exceed a preset threshold.

[0040] In a possible implementation, the video acquisition scenario is a security scenario.

[0041] In a sixth aspect, the present application provides a bitstream processing device, comprising: an acquisition module, used to acquire a first bitstream corresponding to a video, the first bitstream including encoded data corresponding to an image in the video, the video including a plurality of continuous video segments, and each video segment including a plurality of images; a decoding module, used to decode the encoded data of a target image in the first bitstream to obtain a target image, the target image being the first image in a first video segment of the plurality of video segments, and the target image being a key frame; the decoding module, further used to decode the encoded data corresponding to a first category of images using the target image as a reference frame to obtain a first category of images, wherein the first image in each video segment after the first video segment in the plurality of video segments belongs to the first category of images; the encoding module, used to encode the first category of images using the first category of images as key frames to obtain new encoded data corresponding to the first category of images; and a processing module, used to replace the encoded data of the first category of images in the first bitstream with the new encoded data corresponding to the first category of images to obtain a second bitstream corresponding to the video.

[0042] In a possible implementation, the decoding module is further used to: parse the coded data corresponding to the first type of image in the first code stream to obtain the difference between the first type of image and the target image; and generate the first type of image based on the content and the difference of the target image.

[0043] In a possible implementation manner, each video segment in the multiple video segments includes the same number of images.

[0044] In a possible implementation manner, the number of images included in the video does not exceed a preset threshold.

[0045] In a possible implementation, the video acquisition scenario is a security scenario.

[0046] The seventh aspect of the present application provides a video encoding device, including: a processor and a memory; the memory is used to store computer instructions, and when the processor executes the instruction, the video encoding device executes the method of any implementation method of the above-mentioned first aspect.

[0047] In an eighth aspect, the present application provides a video decoding device, comprising: a processor and a memory; the memory is used to store computer instructions, and when the processor executes the instruction, the video decoding device executes the method of any implementation method of the above-mentioned second aspect.

[0048] The ninth aspect of the present application provides a bit stream processing device, including: a processor and a memory; the memory is used to store computer instructions, and when the processor executes the instruction, the video decoding device executes the method of any implementation method of the third aspect above.

[0049] The tenth aspect of the present application provides a computer-readable storage medium, which stores instructions. When the instructions are executed on an electronic device, the electronic device can execute any of the methods described above.

[0050] In an eleventh aspect, the present application provides a computer program product comprising instructions, which, when executed on an electronic device, enables the electronic device to execute any of the methods described above.

[0051] The twelfth aspect of the present application provides a chip, which includes a processor and a communication interface, the communication interface is used to communicate with modules outside the chip shown, and the processor is used to run computer programs or instructions so that the device installed with the chip can execute any of the methods described above.

[0052] Among them, the technical effects brought about by any design method in the second aspect to the twelfth aspect can refer to the technical effects brought about by different implementation methods in the above-mentioned first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 A schematic diagram of decoding when performing random access to a bit stream provided in an embodiment of the present application;

[0054] Figure 2 A schematic diagram of a security scenario provided in an embodiment of the present application;

[0055] Figure 3 A schematic diagram of a system architecture in a security scenario provided by an embodiment of the present application;

[0056] Figure 4 A schematic diagram of a video encoding method provided in an embodiment of the present application;

[0057] Figure 5 A schematic diagram of dividing a video into multiple video segments provided in an embodiment of the present application;

[0058] Figure 6 A schematic diagram of determining a reference frame corresponding to an image in a video provided in an embodiment of the present application;

[0059] Figure 7 A schematic diagram of a before-and-after comparison of video encoding provided in an embodiment of the present application;

[0060] Figure 8 A schematic diagram of a video decoding method provided in an embodiment of the present application;

[0061] Fig. 9 A schematic diagram of a flow chart of a code stream processing method provided in an embodiment of the present application;

[0062] Fig.10 A schematic diagram of converting a first bitstream into a second bitstream provided in an embodiment of the present application;

[0063] Fig.11 A schematic diagram of the structure of a video encoding device provided in an embodiment of the present application;

[0064] Fig.12 A schematic diagram of the structure of a video decoding device provided in an embodiment of the present application;

[0065] Fig.13 A schematic diagram of the structure of a bit stream processing device provided in an embodiment of the present application;

[0066] Fig.14 A schematic diagram of the structure of a chip according to an embodiment of the present application. DETAILED DESCRIPTION

[0067] In order to make the purpose, technical solutions and advantages of the present application clearer, the embodiments of the present application are described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only embodiments of a part of the present application, rather than all embodiments. It is known to those of ordinary skill in the art that with the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0068] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the descriptions used in this way can be interchanged where appropriate, so that the embodiments can be implemented in a sequence other than that illustrated or described in the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or modules is not necessarily limited to those steps or modules that are clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, methods, products or devices. The naming or numbering of the steps that appear in the present application does not mean that the steps in the method flow must be executed in the time / logical sequence indicated by the naming or numbering. The process steps that have been named or numbered can change the execution order according to the technical purpose to be achieved, as long as the same or similar technical effects can be achieved.

[0069] The division of units in this application is a logical division. There may be other division methods when it is implemented in actual applications. For example, multiple units can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection between units can be electrical or other similar forms, which are not limited in this application. In addition, the units or sub-units described as separate components may or may not be physically separated, may or may not be physical units, or may be distributed in multiple circuit units, and some or all of the units may be selected according to actual needs to achieve the purpose of the present application.

[0070] To facilitate understanding, some technical terms involved in the embodiments of the present application are first introduced below.

[0071] (1) Video Coding

[0072] Video encoding refers to the process of converting an original video format file into another video format file through compression technology.

[0073] Specifically, a video is a continuous sequence of images, which is essentially composed of continuous frames, one frame being one image. Due to the persistence of vision of the human eye, when an image sequence is played at a certain rate, what people see is a video with continuous action. Since the similarity between continuous frames is extremely high, in order to facilitate video transmission, the original video is usually encoded and compressed to remove redundancy in the spatial and temporal dimensions.

[0074] At present, commonly used video coding standards are H.264 and H.265. Taking the H.264 coding standard as an example, the video coding process includes the following steps: image segmentation, intra-frame prediction, inter-frame prediction, transformation, quantization and entropy coding.

[0075] In addition, when encoding each image in a video, each image can be divided into multiple image coding slices, and each image coding slice can be divided into multiple macroblocks (i.e., multiple image blocks). Among them, macroblock is the smallest processing unit of video coding and carries specific information of the video.

[0076] (2) Image segmentation

[0077] Image segmentation refers to dividing a large image in the original video into multiple small image blocks.

[0078] (3) Intra-frame prediction

[0079] Intra-frame prediction refers to using the boundary pixels of adjacent image blocks as reference pixels in the current frame to predict the pixels of the current image block, with the aim of removing spatial redundancy of the video signal.

[0080] For a single-frame image, the grayscale change of the current pixel of the image is very small compared with the adjacent pixels at the top, bottom, left, and right positions in the frame, and the correlation is extremely strong. There is spatial redundancy in the image, and intra-frame prediction can reduce the spatial redundancy in the above image.

[0081] (4) Inter-frame prediction

[0082] Inter-frame prediction uses the correlation between video image frames, that is, temporal correlation, to achieve the purpose of image compression. It is widely used in compression coding of ordinary television, conference television, video telephone, and high-definition television.

[0083] Specifically, inter-frame prediction uses adjacent encoded images to find the best matching block for the coding block in the current image, and uses this best matching block as the prediction value of the current coding block. The prediction value is then subtracted from the original pixel value of the current coding block to obtain the residual value of the current coding block. Subsequent transformation, quantization and other operations are all based on the residual value.

[0084] In the process of inter-frame prediction, the best matching block is found for the coding block in the current image by motion estimation, and a motion vector (MV) is calculated. The motion vector is used to represent the relative displacement between the current coding block and the best matching block in the reference image.

[0085] In general, since the video is composed of multiple frames and contains multiple frames per second, the pixels of the current image are very similar to the pixels at corresponding positions in the previous and next frames, and are highly correlated, and there is temporal redundancy. The use of inter-frame prediction can reduce the temporal redundancy in the above images.

[0086] (5) Transformation

[0087] Transformation refers to converting an image described in the form of pixels in the spatial domain to a transform domain (usually the frequency domain) and expressing it in the form of transform coefficients. Generally speaking, most areas in an image are flat areas and areas where the content changes slowly, that is, most of the content in the image is low-frequency information (such as the overall contour information of the image), and the high-frequency information in the image (such as the edge or detail information of the image) is often relatively small, so appropriate transformation can convert the dispersed distribution of image energy in the spatial domain into a relatively concentrated distribution in the transform domain, so as to achieve the purpose of removing redundancy. In general, transformation refers to the conversion of an image from the time domain to the frequency domain, thereby removing the correlation between adjacent data, that is, removing spatial redundancy.

[0088] Specifically, the transformation does not directly encode the spatial image signal, but first maps the spatial image signal to another orthogonal vector space (transform domain or frequency domain), generates a batch of transform coefficients, and then encodes these transform coefficients. Transformation is actually an indirect coding method. The key problem is that when describing in the time domain or spatial domain, the data has a large correlation and data redundancy. After the transformation is described in the transform domain, the data correlation is greatly reduced, the data redundancy is reduced, the parameters are independent, and the amount of data is small. In this way, after the video data is transformed and then quantized, the encoding can obtain a larger compression ratio. Typical transformation methods include discrete cosine transform (DCT), discrete Fourier transform (DFT), etc.

[0089] (6) Quantification

[0090] Quantization refers to the mapping of transform coefficients to a finite number of quantization coefficient values, which can realize the mapping of a data set containing a large amount of data to a data set containing a small amount of data. Simply put, quantization is to reduce the amount of encoded data by representing fine data with coarser data, or by removing information that is not sensitive to the human eye.

[0091] In the video encoding process, the quantization process is the only lossy process, which mainly determines the compression rate of the entire video. The higher the compression rate of the video, the worse the video quality and the lower the bit rate; conversely, the lower the video compression rate, the better the video quality and the higher the bit rate. Specifically, quantization is the process of mapping the continuous value of the signal into multiple discrete amplitudes, realizing a many-to-one mapping of signal values. In the transformation step, after the residual data is transformed, the transform coefficient has a larger value range. Quantization can effectively reduce the value range of the signal, thereby obtaining a better compression effect.

[0092] Generally speaking, the quantization parameter (QP) in the quantization process corresponds to the serial number of the quantization step. The size of the quantization parameter can reflect the compression of spatial details. The smaller the quantization parameter, the finer the quantization, the more details retained in the image, the higher the image quality, and the larger the storage space occupied by the image; the larger the quantization parameter, the coarser the quantization, the more details lost in the image, the lower the image quality, and the smaller the storage space occupied by the image.

[0093] (7) Entropy Coding

[0094] Entropy coding is a type of coding that uses the statistical characteristics of the source to perform bit rate compression. The basic idea of ​​entropy coding is to make the code words before and after as random as possible, reduce the correlation between the before and after, and get closer to the Shannon entropy of the source. In general, entropy coding has the function of eliminating statistical redundancy between data. It is usually used as the last step at the encoding end to write syntax elements into the output bit stream.

[0095] (8)YUV

[0096] YUV is a color encoding method, where Y represents brightness, and U and V represent chrominance. Compared with RGB, YUV is more suitable for encoding and decoding.

[0097] (9) Keyframe

[0098] In video coding, a key frame refers to an image frame whose content needs to be fully encoded. It is an independent frame that carries all the information and can be decoded independently without referring to other image frames. When decoding a key frame, you only need to decode the key frame itself, because the encoded data of the key frame already includes all the content in the key frame. Generally, a key frame is also called an I frame.

[0099] (10) Non-keyframe

[0100] In video coding, non-key frames refer to image frames other than key frames. Generally speaking, when encoding non-key frames, the difference between non-key frames and other frames is actually encoded. In this way, when decoding non-key frames, it is necessary to combine the picture content of other frames and the decoded content of non-key frames to generate the actual picture of non-key frames.

[0101] A common non-key frame is a forward predicted frame (P frame). When encoding a P frame, it is usually encoded based on the difference between the current P frame and the previous frame, thereby reducing the amount of encoded data of the P frame.

[0102] (11) Bitstream

[0103] A bitstream is a coded data stream obtained by encoding the images in a video. In each video coding standard, the data structure in the bitstream is defined to indicate the content information carried by each part of the data in the bitstream.

[0104] At present, in the video encoding process, the video is divided into multiple video segments, and different encoding methods are used to encode different images in the video segments. Among them, the first image in the video segment is encoded independently, and other images after the first image in the video segment are encoded with reference to the previous image.

[0105] However, in consideration of fast decoding of the video, the length of the video segments divided when encoding the video is usually not too long, so that there are still many images in the video that need to encode the complete content, and it is difficult to further reduce the amount of data after the video is encoded. Specifically, for the video bitstream, when decoding the bitstream, if any position in the bitstream is randomly accessed, the decoder usually needs to find the nearest key frame in front of the position, and then decode it frame by frame from the found key frame to the currently accessed position, so as to decode and obtain the image of the currently accessed position. In this way, if the video segments divided during encoding are too long, the intervals between the key frames will be too long, which will easily cause the waiting time required for decoding to be too long, affecting the user experience.

[0106] For example, see Figure 1 , Figure 1 A schematic diagram of decoding when performing random access to a code stream is provided in an embodiment of the present application. Figure 1It can be seen from the code stream shown that when encoding the video, each video segment includes 10 images, and the first image in the video segment is an I frame, and the following 9 images in the video segment are P frames, and each P frame is encoded with reference to its previous image. In this way, when accessing the code stream, if the access position 1 is the position in the code stream corresponding to the 8th image (i.e., the 7th P frame) in the first video segment, it is necessary to decode from the I frame in the first video segment in sequence, and a total of 1 I frame and 7 P frames need to be decoded to obtain the image content corresponding to the access position 1. If the access position 2 is the position in the code stream corresponding to the 10th image (i.e., the 9th P frame) in the second video segment, it is necessary to decode from the I frame in the second video segment in sequence, and a total of 1 I frame and 9 P frames need to be decoded to obtain the image content corresponding to the access position 2. Obviously, if a video segment is too long, it will include a large number of P frames, which may easily result in more images actually needing to be decoded at a certain position in the random access code stream, resulting in a long waiting time for decoding, affecting the normal execution of the decoding service.

[0107] In view of this, the present embodiment provides a video encoding method, in which after dividing the video to be encoded into multiple video segments, only the first image in the first video segment is completely encoded. Moreover, the non-first image in each video segment is encoded with the previous image as a reference frame, and the first image in other video segments other than the first video segment is encoded with the first image in the first video segment as a reference frame, thereby effectively reducing the number of images to be completely encoded, reducing the data volume of the bitstream obtained after video encoding, and not affecting the fast decoding of the bitstream.

[0108] For example, the video encoding method provided in this embodiment may be applied to a security scene. The video encoding method provided in this embodiment may also be applied to other scenes, which are not specifically limited in this embodiment.

[0109] See also Figure 2 , Figure 2 A schematic diagram of a security scenario provided in an embodiment of the present application. Figure 2As shown, in the security scenario, the intelligent security system can usually be composed of an acquisition module, a storage module and an analysis module. For example, the acquisition module may include a security camera, an internet protocol camera (IPC) or a software defined camera (SDC), etc., the storage module may include a video cloud node (VCN), etc., and the analysis module may include a video content management system (VCM), etc. The acquisition module at the front end acquires the video, compresses and encodes the video, and transmits the encoded video data to the storage module at the back end through the network. The storage module stores the acquired video data. The analysis module extracts the video data stored in the storage module, performs intelligent identification and analysis on the video data, and obtains the analysis results.

[0110] The video encoding method provided in this embodiment can be applied to the front-end acquisition module in the security scene, for example, to a security camera with video encoding capability. In this way, after the security camera acquires the video, it can encode the acquired video based on the video encoding method provided in this embodiment, thereby reducing the data volume of the encoded code stream.

[0111] The video decoding method provided in this embodiment can be applied to a storage module or an analysis module in a security scenario, and the video captured by the acquisition module is obtained by decoding the code stream received from the acquisition module.

[0112] See also Figure 3 , Figure 3 This is a schematic diagram of a system architecture in a security scenario provided by an embodiment of the present application. Figure 3 As shown, the system architecture includes one or more front-end cameras, back-end storage devices, video processing platforms and user terminals.

[0113] The front-end camera is used to capture video, and the video encoding method provided in this embodiment is used to encode the captured video to obtain a code stream and send the code stream to a back-end storage device or a video processing platform. The back-end storage device is used to store the code stream sent by the front-end camera.

[0114] The video processing platform is used to process the bit stream encoded by the front-end camera, and the video processing platform can obtain the bit stream encoded by the front-end camera from the front-end camera or the back-end storage device. Specifically, the video processing platform can use the video decoding method provided in this embodiment to decode the bit stream to obtain a video, so that the user can play the video decoded by the video processing platform online.

[0115] Alternatively, the video processing platform may use the code stream processing method provided in this embodiment to transcode the code stream, thereby converting the code stream encoded by the front-end camera into a common code stream with a larger code rate, and transmitting the common code stream to the user terminal, so that the user terminal can download the common code stream corresponding to the video. In this way, the user terminal can decode the downloaded common code stream based on a conventional video decoding method.

[0116] In addition, if the user terminal has the ability to execute the video decoding method provided in this embodiment, the user terminal can also download the code stream encoded by the front-end camera from the video processing platform, and use the video decoding method provided in this embodiment to decode the downloaded code stream to obtain the video.

[0117] For ease of understanding, the video encoding method, video decoding method and bitstream processing method provided by this embodiment will be respectively introduced below.

[0118] See also Figure 4 , Figure 4 A schematic diagram of a video encoding method provided in an embodiment of the present application. Figure 4 As shown, the video encoding method provided in this embodiment is applied to a video encoding end, and specifically includes the following steps 401-403.

[0119] Step 401: Acquire a video, where the video includes a plurality of continuous video segments, and each video segment includes a plurality of images.

[0120] In this embodiment, the video encoding end first obtains the video to be encoded, and segments the obtained video in a pre-specified manner. That is, the video encoding end divides the obtained video into a plurality of continuous and non-overlapping video segments. In this way, the video obtained by the video encoding end actually includes a plurality of video segments, and each video segment includes a plurality of images.

[0121] In simple terms, a video is an image sequence consisting of multiple continuous images. Therefore, dividing a video into multiple video clips is actually dividing the image sequence into multiple image sets, each of which includes multiple continuous images, and the images included in different image sets are not repeated.

[0122] For example, see Figure 5 , Figure 5 A schematic diagram of dividing a video into multiple video segments provided in an embodiment of the present application. Figure 5As shown, the video includes multiple images arranged in sequence. After the video is divided into a video segment in a manner of every 10 images, it can be obtained that the 1st to 10th images in the video belong to video segment 1, the 11th to 20th images in the video belong to video segment 2, and so on, every 10 consecutive images constitute a video segment. In this way, each image in the video can be divided into a unique video segment, and different video segments will not include repeated images.

[0123] Optionally, each of the multiple video segments includes the same number of images, for example, each video segment includes 50 images or 100 images. By setting each video segment to include the same number of images, it is convenient to segment the video and ensure that the encoding data corresponding to the image at each position in each video segment can be quickly determined in the bitstream obtained after encoding, which is conducive to fast decoding of the video.

[0124] Optionally, the acquisition scenario of the above-mentioned video to be encoded is a security scenario or other scenarios, and this embodiment does not limit the video acquisition scenario. In the security scenario, the camera used to collect video is usually fixedly deployed in a specific position, so most of the picture content in the image collected by the camera is fixed, and the picture content between adjacent images does not change much. Therefore, when the method provided by this embodiment is applied to encode in the security scenario, the difference between the encoded image and its corresponding reference frame is not large, and the data amount of the encoded code stream can be significantly reduced in the end, that is, the optimization effect is very obvious. In other scenarios, for the collected video, the picture content between adjacent images usually does not change much, so when encoding based on the method provided by this embodiment, the data amount of the encoded code stream can also be effectively reduced.

[0125] Step 402 , taking a target image in the video as a key frame, encoding the target image to obtain encoding data of the target image, wherein the target image is the first image in the first video segment of the multiple video segments.

[0126] In this embodiment, after the video is segmented, it can be determined that the first image in the first video segment of the multiple video segments is the target image, that is, the target image is actually the first image in the video. Since the video encoding process is usually a process of encoding the images in the video in sequence, the target image is also the first image to be encoded in the video.

[0127] When encoding the target image with the target image as the key frame, the picture content representing the target image needs to be encoded completely without referring to other images to complete the encoding of the target image. In this way, after encoding the target image, the encoded data of the image target obtained actually includes all the picture content in the target image. When decoding, the encoded data of the target image is decoded independently to restore the target image.

[0128] Among them, the process of encoding the target image can refer to the encoding process introduced above, that is, it includes the steps of image segmentation, intra-frame prediction, inter-frame prediction, transformation, quantization and entropy coding, which will not be repeated here.

[0129] Step 403 , in the process of encoding the image located after the target image in the video, the image is encoded based on the reference frame corresponding to the image to obtain the encoded data of the image.

[0130] After encoding the target image, the images following the target image in the video can be encoded in sequence to obtain the encoding data corresponding to each subsequent image. In this way, the encoding data of the target image and the encoding data of each image following the target image can constitute the code stream of the video.

[0131] It should be noted that in a video, when encoding any image after a target image, the current image needs to be encoded based on the reference frame corresponding to the current image. In other words, each image after the target image has a corresponding reference frame, and encoding needs to be implemented based on the picture content of the reference frame.

[0132] In this embodiment, the images after the target image in the video can be divided into two types. The reference frames corresponding to the different types of images are determined in different ways.

[0133] Among them, the first image in each video segment after the first video segment in the multiple video segments is a first-category image, and the reference frame corresponding to each first-category image is a target image. That is, among the multiple video segments, the first image in all other video segments except the first video segment belongs to the first-category image. Moreover, the reference frames of all the first-category images are the same, and are all target images.

[0134] The images other than the first image in each video clip are second-category images, and the reference frame corresponding to each second-category image is the previous image of each second-category image in the video. That is, in each video clip, all images except the first image in the video clip are second-category images. And, for any second-category image, the reference frame corresponding to the second-category image is the previous image of the second-category image in the video.

[0135] In this solution, after the video to be encoded is divided into multiple video segments, only the first image in the first video segment is fully encoded. In addition, the non-first image in each video segment is encoded with the previous image as the reference frame, and the first image in other video segments other than the first video segment is encoded with the first image in the first video segment as the reference frame, thereby effectively reducing the number of images that need to be fully encoded and the amount of data in the bitstream obtained after video encoding without affecting the fast decoding of the bitstream.

[0136] Through tests in some static scenes (such as security scenes), it was found that the data amount of the code stream obtained after encoding by the video encoding method provided in this embodiment can be reduced to 1 / 3 of the data amount of the code stream encoded by the conventional video encoding method, so that the data amount of the code stream that needs to be transmitted in the end is greatly reduced.

[0137] Specifically, when encoding each image after the target image, the difference between each image and its corresponding reference frame can be determined based on the reference frame corresponding to each image. That is, for any image after the target image, the difference in picture content between the image and the corresponding reference frame can be determined based on the reference frame corresponding to the image. Based on the picture content of the reference frame and the obtained difference, the original image can be restored. In this way, encoding can be performed based on the difference between the obtained images to obtain the encoded data of each image, wherein the encoded data of each image is used to indicate the difference between each image and its corresponding reference frame. In other words, in the encoding process, the image is no longer completely encoded, but the difference between the image and the reference frame is encoded, so that when transmitting the code stream, only the slight difference between the image and the reference frame is required, and the complete content of the image is no longer transmitted, which effectively reduces the amount of data transmission.

[0138] For example, see Figure 6 , Figure 6 A schematic diagram of determining a reference frame corresponding to an image in a video provided by an embodiment of the present application. Figure 6As shown, the video includes 15 images, which are arranged in sequence from image 1 to image 15; among them, image 1 to image 5 are divided into video segment 1, image 6 to image 10 are divided into video segment 2, and image 11 to image 15 are divided into video segment 3. Since image 1 is the first image in the video, image 1 can be determined as the above-mentioned target image, that is, image 1 is encoded independently. Figure 2 - Each of the images 15 has a corresponding reference frame.

[0139] For image 6 in video segment 2 and image 11 in video segment 3, since image 6 and image 11 do not belong to the first video segment, and both image 6 and image 11 are the first images in the video segments, image 6 and image 11 are Fig.11 It can be determined as the first type of image mentioned above. And, image 6 and Fig.11 The corresponding reference frames are all image 1.

[0140] In addition, images 2-5, 7-10, and 12-15 are not the first images in the video clip, so it can be determined that images 2-5, 7-10, and 12-15 are the second type of images mentioned above. Moreover, the reference frame corresponding to each image in this part of images is the previous image of the current image. For example, the reference frame of image 2 is the previous image of image 2 (i.e., image 1), the reference frame of image 3 is the previous image of image 3 (i.e., image 2)... The reference frame of image 7 is the previous image of image 7 (i.e., image 6), and so on.

[0141] For example, see Figure 7 , Figure 7 The following is a schematic diagram of a before-and-after comparison of video encoding provided in an embodiment of the present application. Figure 7 As shown, it is assumed that the video to be encoded includes 15 images, and the frame numbers of these 15 images are 0-14. Then, when encoding these images in the video, the image with frame number 0 is encoded as an I frame, and each image with frame number 1-14 is encoded as a P frame. In addition, the reference frames of the images with frame number 5 and frame number 10 are both the images with frame number 0, that is, the image with frame number 0 can be input as a reference frame for encoding during encoding, such as the I frame indicated by the dotted line before the images with frame number 5 and frame number 10. In addition, the reference frames of images with other frame numbers are the previous images of the image.

[0142] Specifically, when encoding an image, the encoder usually needs to cache the reference frame corresponding to the currently encoded image. For example, when encoding the first frame of the image, the encoder needs to cache the 0th frame of the image as a reference frame; when encoding the second frame of the image, the encoder needs to cache the 1st frame of the image as a reference frame. Based on this, when encoding the fifth frame of the image, the 0th frame of the image can be sent back to the encoder as a reference frame for the fifth frame of the image (that is, overwriting the reference frame of the fourth frame of the image stored in the cache), but the encoding result of the 0th frame of the image is no longer written into the bitstream, that is, the 0th frame of the image is only used as a reference frame for the fifth frame of the image to assist the encoding process of the fifth frame of the image. Alternatively, after encoding the 0th frame of the image, the encoder can use a storage area to store the 0th frame of the image for a long time, so that the 0th frame of the image can be obtained from the storage area as a reference frame when encoding other images later.

[0143] Optionally, the number of images included in the video divided into multiple video segments does not exceed a preset threshold. The preset threshold can be set according to actual conditions, for example, the preset threshold is 1000, 2000 or 10000. In a scene where the image changes little, the preset threshold can be set to a larger value; in a scene where the image changes little, the preset threshold can be set to a smaller value.

[0144] In most scenarios of actual applications, the video encoding end continuously acquires images, that is, the number of images actually acquired by the video encoding end is greater than the above-mentioned preset threshold. In this case, the video encoding end can use the above-mentioned preset threshold as the threshold value for repeatedly executing the above-mentioned method, that is, each time the video encoding end acquires the above-mentioned preset threshold number of images, the acquired images are encoded based on the method introduced in the above-mentioned embodiment. In this way, during the encoding process, a key frame is set for each image with a preset threshold number of intervals. For example, assuming that the preset threshold is 1000, the video encoding end can use the 1st to 1000th images as the above-mentioned video, that is, only the 1st image is encoded as a key frame, and the 2nd to 1000th images are encoded based on the reference frame; then, the video encoding end can continue to use the 1001st to 2000th images as the above-mentioned video, that is, only the 1001st image is encoded as a key frame, and the 1002nd to 2000th images are encoded based on the reference frame.

[0145] It is understandable that in the video divided into multiple video clips, there is actually only one key frame (i.e., the first image in the video), and the other images in the video are encoded with reference to the key frame or the previous image. In this way, when there are too many images in the video, the image at the end of the video may be quite different from the image at the beginning of the video, resulting in the amount of calculation and the amount of data not being optimized much when the image at the end of the video is encoded with the first image in the video as the reference frame. Then, if the image at the end of the video is still encoded with the first image as the reference frame, the amount of data of the code stream obtained in the end may be larger than the amount of data of the code stream obtained by resetting a reference frame.

[0146] Therefore, in this solution, by limiting the number of images included in the video, the number of images between key frames in the bitstream can be specified to ensure that the distance between key frames is within a controllable range, thereby ensuring that the data volume of the final encoded bitstream can be as small as possible.

[0147] A video encoding method provided by the present embodiment is introduced above. A video decoding method provided by the present embodiment will be introduced below, that is, how to decode a bit stream encoded based on the above video encoding method.

[0148] See also Figure 8 , Figure 8 A schematic diagram of a video decoding method provided in an embodiment of the present application. Figure 8 As shown, the video decoding method provided in this embodiment is applied to a video decoding end, and specifically includes the following steps 801-803.

[0149] Step 801, obtaining a bitstream of a video, where the bitstream includes coded data corresponding to an image in the video, the video includes a plurality of continuous video segments, and each video segment includes a plurality of images.

[0150] In this embodiment, the bit stream obtained by the video decoding end is based on the above Figure 4 The bitstream obtained by encoding in the corresponding embodiment. That is, the bitstream includes the encoding data corresponding to each image in the video, and the encoding data corresponding to the first image in the video is encoded with the first image as the key frame, while the encoding data corresponding to other images in the video is obtained based on the difference encoding between the current image and the reference frame.

[0151] Step 802: decode the coded data of the target image in the code stream to obtain the target image, where the target image is the first image in the first video segment of the multiple video segments, and the target image is a key frame.

[0152] When decoding a bitstream, the coded data corresponding to the target image may be decoded first, that is, the coded data of the first image in the video corresponding to the bitstream may be decoded to obtain the target image. Generally speaking, the coded data in the bitstream will carry a corresponding frame number. Based on the frame number corresponding to the coded data, the coded data corresponding to the target image in the bitstream may be determined, thereby achieving decoding to obtain the target image.

[0153] Step 803 , in the process of decoding the image located after the target image in the video, based on the reference frame corresponding to the image, the coded data corresponding to the image in the code stream is decoded to obtain the image.

[0154] Among them, the first image in each video clip after the first video clip in multiple video clips is a first-category image, the reference frame corresponding to each first-category image is a target image, the image other than the first image in each video clip is a second-category image, and the reference frame corresponding to each second-category image is the previous image of each second-category image in the video.

[0155] Similarly, based on the frame number corresponding to the coded data in the bitstream, the image actually corresponding to each coded data in the bitstream can be determined. In this way, after determining the image actually corresponding to each coded data, the reference frame corresponding to the current image to be decoded can be determined based on which category of images the current image to be decoded belongs to, and then the image is decoded based on the picture content of the reference frame and the coded data corresponding to the image. For example, assuming that the current image to be decoded belongs to the first category of images, the image is decoded based on the decoded target image and the coded data corresponding to the current image to be decoded; assuming that the current image to be decoded belongs to the second category of images, the image is decoded based on the picture content of the previous image that has been decoded and the coded data corresponding to the current image to be decoded.

[0156] Compared with conventional video decoding methods, since the encoded data of the images in the decoded code stream except the encoded data of the first image are all obtained based on the reference frame encoding (that is, there is only one encoded data corresponding to the key frame in the code stream), when decoding the other encoded data after the first encoded data in the code stream, it is necessary to first determine the reference frame corresponding to the encoded data before the image decoding can be realized based on the reference frame.

[0157] Specifically, when decoding each image after the target image, the coded data corresponding to each image in the code stream may be parsed first to obtain the difference between each image and its corresponding reference frame. That is, for any image after the target image, based on the coded data corresponding to the image, the difference in picture content between the image and the corresponding reference frame may be determined. Then, based on the picture content of the reference frame corresponding to each image and the difference between the image and the reference frame, each image is generated. That is to say, for any image, the picture content of the image may be restored based on the picture content of the reference frame and the difference between the image and the reference frame.

[0158] Optionally, for a video corresponding to the bitstream, each of the multiple video segments of the video includes the same number of images.

[0159] Optionally, the number of images included in the video does not exceed a preset threshold.

[0160] Optionally, the video acquisition scenario is a security scenario.

[0161] The video encoding method and video decoding method provided by the present embodiment are introduced above. Based on the video encoding method provided by the present embodiment, a bitstream with a smaller data volume can be obtained, and based on the video decoding method provided by the present embodiment, a bitstream with a smaller data volume can be decoded. However, in some scenarios, it may be necessary to convert the bitstream with a smaller data volume into a conventional bitstream so that a device that does not support the above decoding method can still achieve normal decoding of the bitstream. For example, in a security scenario, the bitstream encoded based on the above video encoding method is stored on a video processing platform, and the user downloads the bitstream in the security scenario through a user terminal, and the user terminal does not support the above video encoding method. At this time, the video processing platform needs to convert the bitstream encoded based on the video encoding method provided by the present embodiment into a normal bitstream, and then transmit the converted normal bitstream to the user terminal to ensure that the user terminal can smoothly decode the downloaded bitstream.

[0162] See also Fig. 9 , Fig. 9 A schematic diagram of a code stream processing method provided in an embodiment of the present application. Fig. 9 As shown, the bit stream processing method provided in this embodiment is applied to a video processing end, and specifically includes the following steps 901-905.

[0163] Step 901: Obtain a first bitstream corresponding to a video, wherein the first bitstream includes coded data corresponding to an image in the video, and the video includes a plurality of continuous video segments, and each video segment includes a plurality of images.

[0164] Step 902: decode the coded data of the target image in the first code stream to obtain the target image, where the target image is the first image in the first video segment of the multiple video segments, and the target image is a key frame.

[0165] In this embodiment, steps 901-902 are similar to the above-mentioned steps 801-802. Please refer to the above-mentioned steps 801-802 for details, which will not be repeated here.

[0166] Step 903 , using the target image as a reference frame, decoding the coded data corresponding to the first category of images to obtain the first category of images, wherein the first image in each video segment after the first video segment in the multiple video segments belongs to the first category of images.

[0167] In this embodiment, after decoding to obtain the target image, each coded data in the first bitstream is no longer decoded, but only the coded data corresponding to the first type of image in the video is decoded, so as to obtain the first type of image in the video corresponding to the first bitstream. The video corresponding to the first bitstream is divided into multiple video segments, and except for the first video segment, the first image in other video segments in the multiple video segments belongs to the first type of image. In this way, in the first bitstream, based on the frame number corresponding to each coded data and the length of the video segment, the coded data corresponding to the first type of image in the first bitstream can be determined, so that the decoding of these coded data can be realized with the target image as the reference frame.

[0168] Specifically, in the process of decoding the encoded data corresponding to the first category image, the encoded data corresponding to each first category image in the first code stream may be parsed to obtain the difference between each first category image and the target image; then, based on the content of the target image and the difference between each first category image and the target image, each first category image is generated.

[0169] Step 904: Encode the first category of images using the first category of images as key frames to obtain new encoded data corresponding to the first category of images.

[0170] After decoding and obtaining the first category images in the video, each first category image may be used as a key frame to completely encode the screen content of the first category image, thereby obtaining new encoding data corresponding to each first category image.

[0171] Step 905: Replace the coded data of the first type of images in the first code stream with new coded data corresponding to the first type of images to obtain a second code stream corresponding to the video.

[0172] After re-encoding each first-category image as a key frame to obtain new encoded data corresponding to the first-category image, the re-encoded new encoded data can be used to replace the encoded data corresponding to the first-category image in the first bitstream. In this way, in the second bitstream, the first image in each video clip is encoded using the image as a key frame, thereby ensuring that the format of the second bitstream is the same as that of the common bitstream.

[0173] That is to say, in the first code stream, the coded data corresponding to the first type of image is obtained by encoding the target image as a reference frame, that is, the coded data of the first type of image actually indicates the difference between the first type of image and the target image; and in the second code stream, the coded data corresponding to the first type of image is obtained by encoding the first type of image itself.

[0174] For example, see Fig.10 , Fig.10 A schematic diagram of converting a first bitstream into a second bitstream is provided in an embodiment of the present application. Fig.10 As shown, it is assumed that the first code stream is obtained by encoding a video including 15 images using the video encoding method provided in this embodiment. The video is divided into 3 video segments, the frame numbers corresponding to the images included in the first video segment are 0-4, the frame numbers corresponding to the images included in the first video segment are 5-9, and the frame numbers corresponding to the images included in the first video segment are 10-14. In the first code stream, the 0th frame image is an I frame, that is, the encoding data of the 0th frame image is encoded based on the complete content of the 0th frame image; the 5th frame image and the 10th frame image are both P frames, and the encoding data of the 5th frame image and the encoding data of the 10th frame image are both encoded with the 0th frame image as the reference frame. In the process of processing the first code stream, it can be determined that the encoding data corresponding to the 5th frame image and the encoding data corresponding to the 10th frame image are both encoding data of the first image in the video segment based on the frame numbers corresponding to the encoding data in the first code stream. Therefore, based on the picture content of the 0th frame image, the coded data corresponding to the 5th frame image can be decoded to obtain the 5th frame image; then, the 5th frame image can be re-encoded with the 5th frame image as the I frame, thereby obtaining new coded data for the 5th frame image. Similarly, for the 10th frame image, based on the picture content of the 0th frame image, the coded data corresponding to the 10th frame image can be decoded to obtain the 10th frame image; then, the 10th frame image can be re-encoded with the 10th frame image as the I frame, thereby obtaining new coded data for the 10th frame image. Finally, the coded data corresponding to the 5th frame image and the 10th frame image in the first code stream are replaced with the re-encoded new coded data, thereby realizing code stream conversion, so that the second code stream includes coded data corresponding to multiple different key frames.

[0175] Optionally, for a video corresponding to the bitstream, each of the multiple video segments of the video includes the same number of images.

[0176] Optionally, the number of images included in the video does not exceed a preset threshold.

[0177] Optionally, the video acquisition scenario is a security scenario.

[0178] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of the method. It is understandable that, in order to realize the above functions, the electronic device includes a hardware structure and / or software module corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the modules and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0179] The embodiment of the present application can divide the functional modules of the electronic device according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0180] The following is a detailed description of the device used to execute the above method in the embodiment of the present application.

[0181] See also Fig.11 , Fig.11 FIG. 1 is a schematic diagram of the structure of a video encoding device provided in an embodiment of the present application. Fig.11As shown, the video encoding device includes: an acquisition module 1101, used for acquiring a video to be encoded, the video includes a plurality of continuous video segments, and each video segment includes a plurality of images; an encoding module 1102, used for encoding a target image in the video as a key frame, and obtaining encoding data of the target image, wherein the target image is the first image in the first video segment of the plurality of video segments; the encoding module 1102 is also used for encoding the image in the video after the target image based on a reference frame corresponding to the image, and obtaining encoding data of the image, wherein the encoding data of the target image and the encoding data of the image are used to constitute a code stream of the video; wherein the first image in each video segment after the first video segment in the plurality of video segments is a first type of image, the reference frame corresponding to the first type of image is the target image, the image in each video segment other than the first image is a second type of image, and the reference frame corresponding to the second type of image is the previous image of the second type of image in the video.

[0182] In one possible implementation, the encoding module 1102 is further used to: determine the difference between the image and the reference frame corresponding to the image based on the reference frame corresponding to the image; and perform encoding based on the difference to obtain encoding data of the image, wherein the encoding data of the image is used to indicate the difference between the image and the reference frame corresponding to the image.

[0183] In a possible implementation manner, each video segment in the multiple video segments includes the same number of images.

[0184] In a possible implementation manner, the number of images included in the video does not exceed a preset threshold.

[0185] In a possible implementation, the video acquisition scenario is a security scenario.

[0186] See also Fig.12 , Fig.12 FIG. 1 is a schematic diagram of the structure of a video decoding device provided in an embodiment of the present application. Fig.12As shown, the video decoding device includes: an acquisition module 1201, which is used to acquire a code stream of a video, the code stream includes coded data corresponding to an image in the video, the video includes a plurality of continuous video segments, and each video segment includes a plurality of images; a decoding module 1202, which is used to decode the coded data corresponding to a target image in the code stream to obtain a target image, the target image is the first image in the first video segment of the plurality of video segments, and the target image is a key frame; the decoding module 1202 is also used to decode the coded data corresponding to the image in the code stream based on a reference frame corresponding to the image in the process of decoding an image located after the target image in the video to obtain an image; wherein the first image in each video segment after the first video segment in the plurality of video segments is a first-category image, the reference frame corresponding to the first-category image is the target image, the image other than the first image in each video segment is a second-category image, and the reference frame corresponding to the second-category image is a previous image of the second-category image in the video.

[0187] In a possible implementation, the decoding module 1202 is further used to: parse the coded data corresponding to the image in the code stream to obtain the difference between the image and the reference frame corresponding to the image; and generate the image based on the content and difference of the reference frame corresponding to the image.

[0188] In a possible implementation manner, each video segment in the multiple video segments includes the same number of images.

[0189] In a possible implementation manner, the number of images included in the video does not exceed a preset threshold.

[0190] In a possible implementation, the video acquisition scenario is a security scenario.

[0191] See also Fig.13 , Fig.13 This is a schematic diagram of the structure of a bit stream processing device provided in an embodiment of the present application. Fig.13As shown, the bitstream processing device includes: an acquisition module 1301, which is used to acquire a first bitstream corresponding to a video, the first bitstream includes coded data corresponding to an image in the video, the video includes a plurality of continuous video segments, and each video segment includes a plurality of images; a decoding module 1302, which is used to decode the coded data of a target image in the first bitstream to obtain a target image, the target image is the first image in a first video segment of the plurality of video segments, and the target image is a key frame; the decoding module 1302 is also used to decode the coded data corresponding to a first type of image with the target image as a reference frame to obtain a first type of image, wherein the first image in each video segment after the first video segment in the plurality of video segments belongs to the first type of image; the encoding module 1303, which is used to encode the first type of image with the first type of image as a key frame to obtain new coded data corresponding to the first type of image; and a processing module 1304, which is used to replace the coded data of the first type of image in the first bitstream with the new coded data corresponding to the first type of image to obtain a second bitstream corresponding to the video.

[0192] In a possible implementation, the decoding module 1302 is further configured to: parse the coded data corresponding to the first type of image in the first code stream to obtain the difference between the first type of image and the target image; and generate the first type of image based on the content and the difference of the target image.

[0193] In a possible implementation manner, each video segment in the multiple video segments includes the same number of images.

[0194] In a possible implementation manner, the number of images included in the video does not exceed a preset threshold.

[0195] In a possible implementation, the video acquisition scenario is a security scenario.

[0196] See also Fig.14 , Fig.14 FIG. 1 is a schematic diagram of the structure of a chip according to an embodiment of the present application. Fig.14 As shown, the chip may include hardware modules such as a hardware decoder, a hardware image processor, a neural network processor (neural-network processing unit, NPU), a central processing unit (central processing unit, CPU) and a hardware encoder, which can support video encoding and decoding. Wherein, when the chip is deployed at the video encoding end, the hardware encoder in the chip can be used to execute the video encoding method provided in the embodiment of the present application. When the chip is deployed at the video decoding end, the hardware decoder in the chip can be used to execute the video decoding method provided in the embodiment of the present application. The hardware encoder and hardware decoder in the chip can cooperate to execute the code stream processing method provided in the embodiment of the present application.

[0197] It should be noted that the information interaction, execution process, etc. between the modules / or components in the above device are the same as those in this application. Figure 3-Figure 10 The corresponding method embodiments are based on the same concept. For specific contents, please refer to the description in the method embodiments shown above in this application, which will not be repeated here.

[0198] It should be noted that for the specific implementation of the device and the beneficial effects brought about, please refer to Figure 3-Figure 10 The descriptions in the corresponding method embodiments will not be repeated here one by one.

[0199] An embodiment of the present application also provides a computer-readable storage medium, including instructions, which, when executed on a computer, enables the computer to control an electronic device to execute any one of the implementation methods shown in the aforementioned method embodiments.

[0200] An embodiment of the present application also provides a computer program product, which includes a computer program code. When the computer program code runs on a computer, the computer executes any one of the implementation methods shown in the aforementioned method embodiments.

[0201] An embodiment of the present application also provides a chip system, including a memory and a processor, the memory is used to store computer programs, and the processor is used to call and run computer programs from the memory, so that the chip executes any one of the implementation methods shown in the aforementioned method embodiments.

[0202] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0203] Through the description of the above implementation methods, the technicians in the relevant field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented with corresponding hardware, and the specific hardware structures used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation method in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, U disk, mobile hard disk, ROM, RAM, disk or optical disk, etc., including a number of instructions to enable a computer device to execute the methods described in each embodiment of the present application.

[0204] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0205] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a communication device, a computing device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, communication device, computing device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a communication device, a data center, or other data storage device that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

[0206] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0207] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

Claims

1. A video encoding method, characterized in that: include: Acquire a video, wherein the video includes a plurality of continuous video clips, and each video clip includes a plurality of images; Taking a target image in the video as a key frame, encoding the target image to obtain encoded data of the target image, wherein the target image is the first image in the first video segment of the multiple video segments; In the process of encoding an image located after the target image in the video, encoding the image based on a reference frame corresponding to the image to obtain encoded data of the image, and the encoded data of the target image and the encoded data of the image are used to form a code stream of the video; Among them, the first image in each video clip after the first video clip in the multiple video clips is a first type of image, the reference frame corresponding to the first type of image is the target image, and the image other than the first image in each video clip is a second type of image, and the reference frame corresponding to the second type of image is the previous image of the second type of image in the video.

2. The method according to claim 1, characterized in that The step of encoding the image based on a reference frame corresponding to the image to obtain encoded data of the image includes: determining, based on a reference frame corresponding to the image, a difference between the image and the reference frame corresponding to the image; Encoding is performed based on the difference to obtain encoded data of the image, wherein the encoded data of the image is used to indicate the difference between the image and a reference frame corresponding to the image.

3. The method according to claim 1 or 2, characterized in that: Each video segment of the plurality of video segments includes a same number of images.

4. The method according to any one of claims 1 to 3, characterized in that: The number of images included in the video does not exceed a preset threshold.

5. The method according to any one of claims 1 to 4, characterized in that: The video is acquired in a security scenario.

6. A video decoding method, characterized in that: include: Acquire a code stream of a video, the code stream comprising coded data corresponding to an image in the video, the video comprising a plurality of continuous video segments, and each video segment comprising a plurality of images; Decoding the coded data corresponding to the target image in the bitstream to obtain the target image, where the target image is the first image in the first video segment of the multiple video segments, and the target image is a key frame; In a process of decoding an image located after the target image in the video, based on a reference frame corresponding to the image, the coded data corresponding to the image in the bitstream is decoded to obtain the image; Among them, the first image in each video clip after the first video clip in the multiple video clips is a first type of image, the reference frame corresponding to the first type of image is the target image, and the image other than the first image in each video clip is a second type of image, and the reference frame corresponding to the second type of image is the previous image of the second type of image in the video.

7. The method according to claim 6, characterized in that The decoding of the coded data corresponding to the image in the code stream based on the reference frame corresponding to the image includes: Parsing the coded data corresponding to the image in the code stream to obtain a difference between the image and a reference frame corresponding to the image; The image is generated based on the content of a reference frame corresponding to the image and the difference.

8. The method according to claim 6 or 7, characterized in that: Each video segment of the plurality of video segments includes a same number of images.

9. The method according to any one of claims 6 to 8, characterized in that: The number of images included in the video does not exceed a preset threshold.

10. The method according to any one of claims 6 to 9, characterized in that: The video is acquired in a security scenario.

11. A code stream processing method, characterized in that: include: Acquire a first bitstream corresponding to a video, the first bitstream including encoded data corresponding to an image in the video, the video including a plurality of continuous video segments, and each video segment including a plurality of images; Decoding the coded data of the target image in the first bitstream to obtain the target image, where the target image is the first image in the first video segment of the multiple video segments, and the target image is a key frame; Taking the target image as a reference frame, decoding the coded data corresponding to the first type of image to obtain the first type of image, wherein the first image in each video segment after the first video segment in the multiple video segments belongs to the first type of image; Taking the first type of images as key frames, encoding the first type of images to obtain new encoded data corresponding to the first type of images; The coded data of the first type of images in the first code stream is replaced with new coded data corresponding to the first type of images to obtain a second code stream corresponding to the video.

12. The method according to claim 11, characterized in that The step of decoding the coded data of the first type of image by taking the target image as a reference frame comprises: Parsing the coded data corresponding to the first type of image in the first code stream to obtain a difference between the first type of image and the target image; The first type of image is generated based on the content of the target image and the difference.

13. The method according to claim 11 or 12, characterized in that: Each video segment of the plurality of video segments includes a same number of images.

14. The method according to any one of claims 11 to 13, characterized in that: The number of images included in the video does not exceed a preset threshold.

15. The method according to any one of claims 11 to 14, characterized in that: The video is acquired in a security scenario.

16. A video encoding device, characterized in that: include: An acquisition module, used for acquiring a video, wherein the video includes a plurality of continuous video segments, and each video segment includes a plurality of images; an encoding module, configured to encode a target image in the video as a key frame to obtain encoding data of the target image, wherein the target image is the first image in the first video segment of the multiple video segments; The encoding module is further used to encode the image located after the target image in the video based on the reference frame corresponding to the image to obtain the encoded data of the image, and the encoded data of the target image and the encoded data of the image are used to form a code stream of the video; Among them, the first image in each video clip after the first video clip in the multiple video clips is a first type of image, the reference frame corresponding to the first type of image is the target image, and the image other than the first image in each video clip is a second type of image, and the reference frame corresponding to the second type of image is the previous image of the second type of image in the video.

17. A video decoding device, characterized in that: include: An acquisition module, used to acquire a code stream of a video, wherein the code stream includes coded data corresponding to an image in the video, wherein the video includes a plurality of continuous video segments, and each video segment includes a plurality of images; A decoding module, configured to decode the coded data corresponding to the target image in the bitstream to obtain the target image, wherein the target image is the first image in the first video segment of the multiple video segments, and the target image is a key frame; The decoding module is further configured to decode the coded data corresponding to the image in the bitstream based on the reference frame corresponding to the image in the decoding process of the image located after the target image in the video to obtain the image; Among them, the first image in each video clip after the first video clip in the multiple video clips is a first type of image, the reference frame corresponding to the first type of image is the target image, and the image other than the first image in each video clip is a second type of image, and the reference frame corresponding to the second type of image is the previous image of the second type of image in the video.

18. A code stream processing device, characterized in that: include: An acquisition module, configured to acquire a first bitstream corresponding to a video, wherein the first bitstream includes coded data corresponding to an image in the video, wherein the video includes a plurality of continuous video segments, and each video segment includes a plurality of images; a decoding module, configured to decode the coded data of the target image in the first bitstream to obtain the target image, wherein the target image is the first image in the first video segment of the multiple video segments, and the target image is a key frame; The decoding module is further configured to decode the coded data corresponding to the first type of image by taking the target image as a reference frame to obtain the first type of image, wherein the first image in each video segment after the first video segment in the multiple video segments belongs to the first type of image; An encoding module, configured to encode the first category of images by taking the first category of images as key frames to obtain new encoding data corresponding to the first category of images; The processing module is used to replace the coded data of the first type of images in the first code stream with new coded data corresponding to the first type of images, so as to obtain a second code stream corresponding to the video.

19. A computer storage medium, characterized in that: The computer storage medium stores instructions, which, when executed by a computer, cause the computer to implement the method of any one of claims 1 to 15.

20. A computer program product, characterized in that The computer program product stores instructions, which, when executed by a computer, cause the computer to implement the method according to any one of claims 1 to 15.

Images