Video coding method and related device

Abstract

The invention discloses a video coding method which can be applied to a video coding end in a security scene. In the method, when an image block in an image of a video is coded, under the condition that the motion vector of the image block needing to be coded relative to a reference image block is determined to be smaller, the image block is coded in a block skip mode, namely, the prediction residual error and the motion vector difference value of the image block relative to the reference image block are both 0; therefore, the prediction residual and the motion vector difference value of the image block are not coded and transmitted. The condition that the motion vector of the image block is relatively small relative to that of the reference image block is actually slight disturbance in the video, so that the image block is coded in the block skip mode, and some unimportant disturbance contents can be skipped in the video coding process to reduce the data volume after video coding; and the quality of the coded video can be ensured by keeping a normal motion part.

Description

Technical Field

[0001] The present application relates to the field of video processing technology, and in particular to a video encoding method and related devices. Background Art

[0002] With the popularization of the Internet, video transmission is becoming more and more common in various fields of society. In some specific scenarios (such as security scenarios and multimedia scenarios), video transmission has become an indispensable role.

[0003] Generally, in the process of video transmission, in order to reduce the amount of video data that needs to be transmitted, the video is usually encoded, thereby achieving compression of the video data. In the related art, when encoding a video, a long video is usually divided into multiple short video segments, and the first image in each video segment is encoded independently (i.e., encoded based on all the contents of the image), and the other images after the first image in each video segment are encoded with reference to the previous image (i.e., encoded based on the difference between the current image and the previous image). Since the difference between adjacent images in a video is usually relatively small, for most images in a video, only the slight difference between this part of the image and other images can be encoded, thereby effectively reducing the amount of encoded data.

[0004] However, in the related art, the amount of data after video encoding is still large, and it is difficult to promote and apply in some scenarios where storage resources or bandwidth resources are limited. Summary of the invention

[0005] The present application provides a video encoding method, which can reduce the data volume of a code stream obtained after video encoding without affecting the video quality as much as possible.

[0006] The first aspect of the present application provides a video encoding method, which is applied to a video encoding end. The method includes: the video encoding end obtains a first image, and the first image is an image in a video to be encoded. Then, the video encoding end determines a reference image block corresponding to a first image block in the first image in a second image, and the second image is a reference frame of the first image. Among them, the reference image block is an image block in the second image that best matches the first image, and the first image block has the same size as the reference image block. Finally, in response to the first image block and the reference image block satisfying a preset condition, the video encoding end encodes the first image block through a block skip mode to obtain an encoding result.

[0007] The preset conditions include that the displacement of the first image block relative to the reference image block is not greater than a first threshold, and the block skip mode is a mode in which the prediction residual and the motion vector difference of the first image block are not encoded. The first threshold may be, for example, N pixels, that is, when the displacement of the first image block relative to the reference image block is not greater than N pixels, it is considered that the first image block meets the preset conditions, and thus the first image block is encoded through the block skip mode to obtain an encoding result.

[0008] In this solution, when encoding an image block in an image of a video, if it is confirmed that the motion vector of the image block to be encoded is smaller than that of the reference image block, the image block is encoded using the block skip mode, that is, the prediction residual and motion vector difference of the image block compared to the reference image block are considered to be 0, and the prediction residual and motion vector difference of the image block are not encoded and transmitted. Since the situation that the motion vector of the image block is smaller than that of the reference image block is actually some slight disturbance in the video, encoding the image block using the block skip mode can skip some unimportant disturbance content during the video encoding process to reduce the amount of data after video encoding, and can maintain the normal motion part to ensure the quality of the encoded video.

[0009] In a possible implementation, the preset condition also includes that the first image block does not belong to a flickering area in the video. The flickering area in the video refers to an area whose brightness changes greatly over time (ie, an area that flickers).

[0010] In this solution, the condition that the first image block does not belong to the flickering area is added to the preset conditions. Only when the first image block meets both the displacement requirements and the requirements of the non-flickering area, the encoding of the motion and detail content of the first image block will be skipped, ensuring that the encoded video can retain the detail information of the flickering area, thereby ensuring the quality of the encoded video.

[0011] In a possible implementation, the first image block does not belong to the flickering area in the video, specifically the first image block does not meet the flickering condition, the flickering condition is that the sum of the differences between the gradient of the pixels in the first image block and the gradient of the corresponding pixels in the reference image block is less than a second threshold, and the sum of the differences between the brightness of the pixels in the first image block and the brightness of the corresponding pixels in the reference image block is greater than a third threshold, and the gradient of the pixels in the first image block is the difference between the pixels in the first image block and the adjacent pixels.

[0012] When the sum of the differences between the gradients of the pixels in the first image block and the gradients of the corresponding pixels in the reference image block is less than the second threshold, and the sum of the differences between the brightness of the pixels in the first image block and the brightness of the corresponding pixels in the reference image block is greater than the third threshold, it can be considered that the pixel distribution of the first image block relative to the reference image block has not changed much, but the brightness of the overall pixels in the first image block has changed significantly. Therefore, it can be confirmed that the brightness of the first image block has changed suddenly relative to the reference image block, that is, the first image block and the reference image block actually belong to the flickering area.

[0013] In this solution, based on the pixel gradient difference and pixel brightness difference between the first image block and the reference image block, it can be effectively determined whether the first image block belongs to the flickering area, thereby ensuring effective identification of the flickering area in the image and improving the feasibility of the solution.

[0014] In a possible implementation manner, the preset condition also includes that a sum of differences between pixels in the first image block and corresponding pixels in the reference image block is less than a fourth threshold.

[0015] In simple terms, each pixel in an image block has a pixel value to indicate the content represented by the pixel. Thus, for each pixel in the first image block, the pixel value difference between the pixel and the corresponding pixel in the reference image block can be calculated; by calculating the pixel value difference between each pair of pixels in the first image block and the reference image block that have a corresponding relationship, the sum of the differences between the pixels in the first image block and the corresponding pixels in the reference image block can be obtained.

[0016] In this solution, by adding the condition that the difference between the pixels in the image block to be encoded and the corresponding pixels in the reference image block is less than a specific threshold in the requirement of using the block skip mode, it can be ensured that the encoding process skips the encoding of some detailed information with little content change, and ensures that the detailed information with large content change can be presented in the video, thereby ensuring the quality of the encoded video.

[0017] In a possible implementation manner, the second image is an image frame previous to the first image in the video.

[0018] In a possible implementation manner, the second image is an encoded image obtained by encoding an image in the video using a block skip mode.

[0019] In this scheme, by selecting the encoded image as the reference frame of the first image, it can be ensured that the decoding process can be based on the decoded image to achieve decoding of other subsequent images, and the motion skipped when using block skip mode encoding will not be accumulated and skipped, avoiding the phenomenon of video picture motion distortion.

[0020] In a possible implementation, the magnitude of the first threshold is related to the resolution of the first image, and the first threshold and the resolution of the first image have a positive correlation, that is, the higher the resolution of the first image, the larger the value of the first threshold; and the lower the resolution of the first image, the smaller the value of the first threshold.

[0021] In this solution, by setting the size of the first threshold to be related to the resolution of the first image, the first threshold can be adaptively adjusted based on the resolution of the first image, ensuring that only some unimportant disturbance content is skipped during the video encoding process without affecting normal motion content, thereby ensuring the quality of the encoded video.

[0022] In a possible implementation manner, the reference image block is an image block in the second image that has the smallest difference with the first image block.

[0023] In a possible implementation, the video acquisition scenario is a security scenario.

[0024] A second aspect of the present application provides a video encoding device, including:

[0025] An acquisition module, used to acquire a first image, where the first image is an image in the video;

[0026] A processing module, used to determine a reference image block corresponding to a first image block in a first image in a second image, the second image being a reference frame of the first image;

[0027] The processing module is further configured to encode the first image block in a block skip mode to obtain an encoding result in response to the first image block and the reference image block satisfying a preset condition;

[0028] The preset conditions include that the displacement of the first image block relative to the reference image block is not greater than a first threshold, and the block skip mode is a mode in which the prediction residual and the motion vector difference of the first image block are not encoded.

[0029] In a possible implementation manner, the preset condition also includes that the first image block does not belong to a flickering area in the video.

[0030] In a possible implementation, the first image block does not belong to the flickering area in the video, specifically the first image block does not meet the flickering condition, the flickering condition is that the sum of the differences between the gradient of the pixels in the first image block and the gradient of the corresponding pixels in the reference image block is less than a second threshold, and the sum of the differences between the brightness of the pixels in the first image block and the brightness of the corresponding pixels in the reference image block is greater than a third threshold, and the gradient of the pixels in the first image block is the difference between the pixels in the first image block and the adjacent pixels.

[0031] In a possible implementation manner, the preset condition also includes that a sum of differences between pixels in the first image block and corresponding pixels in the reference image block is less than a fourth threshold.

[0032] In a possible implementation manner, the second image is an image frame previous to the first image in the video.

[0033] In a possible implementation manner, the second image is an encoded image obtained by encoding an image in the video using a block skip mode.

[0034] In a possible implementation manner, the size of the first threshold is related to the resolution of the first image.

[0035] In a possible implementation manner, the reference image block is an image block in the second image that has the smallest difference with the first image block.

[0036] In a possible implementation, the video acquisition scenario is a security scenario.

[0037] The third aspect of the present application provides a video encoding device, which includes: a memory and a processor; the memory stores codes, and the processor is configured to execute the codes. When the codes are executed, the video encoding device executes a method as in any one of the implementation modes of the first aspect.

[0038] A fourth aspect of the present application provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on an electronic device, the electronic device can execute any of the methods described above.

[0039] A fifth aspect of the present application provides a computer program product comprising instructions, which, when executed on an electronic device, enables the electronic device to execute any of the methods described above.

[0040] In a sixth aspect, the present application provides a chip, which includes a processor and a communication interface, wherein the communication interface is used to communicate with modules outside the chip shown, and the processor is used to run computer programs or instructions so that a device equipped with the chip can execute any of the methods described above.

[0041] Among them, the technical effects brought about by any design method in the second to sixth aspects can refer to the technical effects brought about by different implementation methods in the above-mentioned first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A schematic diagram of determining a reference image block in an encoding process provided by an embodiment of the present application;

[0043] Figure 2 A schematic diagram of a security scenario provided in an embodiment of the present application;

[0044] Figure 3 A schematic diagram of a system architecture in a security scenario provided by an embodiment of the present application;

[0045] Figure 4 A schematic diagram of a video encoding method provided in an embodiment of the present application;

[0046] Figure 5 A schematic diagram of encoding a portion of image blocks in an image using a block skip mode provided in an embodiment of the present application;

[0047] Figure 6 A schematic diagram of comparing a first image block belonging to a flickering area with a reference image block provided in an embodiment of the present application;

[0048] Figure 7 A schematic diagram of encoding an image using a block skip mode in a security scenario provided by an embodiment of the present application;

[0049] Figure 8 A schematic diagram of an image obtained after encoding using a block skip mode in a security scenario provided by an embodiment of the present application;

[0050] Fig. 9 A schematic diagram of comparison of multiple images in a video before and after encoding provided by an embodiment of the present application;

[0051] Fig.10 A schematic diagram of the structure of a video encoding device provided in an embodiment of the present application;

[0052] Fig.11 A schematic diagram of the structure of a chip according to an embodiment of the present application. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solutions and advantages of the present application clearer, the embodiments of the present application are described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only embodiments of a part of the present application, rather than all embodiments. It is known to those of ordinary skill in the art that with the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0054] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the descriptions used in this way can be interchanged where appropriate, so that the embodiments can be implemented in a sequence other than that illustrated or described in the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or modules is not necessarily limited to those steps or modules that are clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, methods, products or devices. The naming or numbering of the steps that appear in the present application does not mean that the steps in the method flow must be executed in the time / logical sequence indicated by the naming or numbering. The process steps that have been named or numbered can change the execution order according to the technical purpose to be achieved, as long as the same or similar technical effects can be achieved.

[0055] The division of units in this application is a logical division. There may be other division methods when it is implemented in actual applications. For example, multiple units can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection between units can be electrical or other similar forms, which are not limited in this application. In addition, the units or sub-units described as separate components may or may not be physically separated, may or may not be physical units, or may be distributed in multiple circuit units, and some or all of the units may be selected according to actual needs to achieve the purpose of the present application.

[0056] To facilitate understanding, some technical terms involved in the embodiments of the present application are first introduced below.

[0057] (1) Video Coding

[0058] Video encoding refers to the process of converting an original video format file into another video format file through compression technology.

[0059] Specifically, a video is a continuous sequence of images, which is essentially composed of continuous frames, one frame being one image. Due to the persistence of vision of the human eye, when an image sequence is played at a certain rate, what people see is a video with continuous action. Since the similarity between continuous frames is extremely high, in order to facilitate video transmission, the original video is usually encoded and compressed to remove redundancy in the spatial and temporal dimensions.

[0060] At present, commonly used video coding standards are H.264 and H.265. Taking the H.264 coding standard as an example, the video coding process includes the following steps: image segmentation, intra-frame prediction, inter-frame prediction, transformation, quantization and entropy coding.

[0061] In addition, when encoding each image in a video, each image can be divided into multiple image coding slices, and each image coding slice can be divided into multiple macroblocks (i.e., multiple image blocks). Among them, macroblock is the smallest processing unit of video coding and carries specific information of the video.

[0062] (2) Image segmentation

[0063] Image segmentation refers to dividing a large image in the original video into multiple small image blocks.

[0064] (3) Intra-frame prediction

[0065] Intra-frame prediction refers to using the boundary pixels of adjacent image blocks as reference pixels in the current frame to predict the pixels of the current image block, with the aim of removing spatial redundancy of the video signal.

[0066] For a single-frame image, the grayscale change of the current pixel of the image is very small compared with the adjacent pixels at the top, bottom, left, and right positions in the frame, and the correlation is extremely strong. There is spatial redundancy in the image, and intra-frame prediction can reduce the spatial redundancy in the above image.

[0067] (4) Inter-frame prediction

[0068] Inter-frame prediction uses the correlation between video image frames, that is, temporal correlation, to achieve the purpose of image compression. It is widely used in compression coding of ordinary television, conference television, video telephone, and high-definition television.

[0069] Specifically, inter-frame prediction uses adjacent encoded images (i.e., reference frames) to find the best matching block for the coding block in the current image, and uses this best matching block (also called reference image block) as the prediction value of the current coding block. The prediction value is then subtracted from the original pixel value of the current coding block to obtain the prediction residual of the current coding block. Subsequent transformation, quantization and other operations are all based on the prediction residual.

[0070] In the process of inter-frame prediction, the best matching block is found for the coding block in the current image by motion estimation, and a motion vector (MV) is calculated. The motion vector is used to represent the relative displacement between the current coding block and the best matching block in the reference image.

[0071] For example, see Figure 1 , Figure 1 A schematic diagram of determining a reference image block in an encoding process provided by an embodiment of the present application. If shown in 1, Figure 1 The image on the right is the image to be encoded (hereinafter referred to as the current image), and the image on the left is the reference frame. During the encoding process, it is necessary to refer to the reference frame on the left to realize the encoding of the current image on the right. Specifically, the current image is first divided into blocks to obtain multiple image blocks. Then, for an image block in the current image to be encoded (hereinafter referred to as the current encoded image block), an image block that best matches the current encoded image block is found in the reference frame as the reference image block of the current encoded image block. Figure 1 As shown in the figure, for the two image blocks in the current image on the right, the corresponding reference image blocks can be found in the reference frame on the left. In this way, based on the position of the current coded image block in the current image and the position of the reference image block in the reference frame, the relative displacement between the current coded image block and the reference image block can be determined, that is, the motion vector corresponding to the current coded image block can be obtained.

[0072] In general, since the video is composed of multiple frames and contains multiple frames per second, the pixels of the current image are very similar to the pixels at corresponding positions in the previous and next frames, and are highly correlated, and there is temporal redundancy. The use of inter-frame prediction can reduce the temporal redundancy in the above images.

[0073] (5) Reference frame

[0074] A reference frame is an image that is used as a reference for the image currently being encoded during video encoding. In the actual encoding process, all the contents of the current image are not directly encoded, but the difference between the current image and the reference frame is encoded.

[0075] (6) Prediction residual

[0076] The prediction residual refers to the difference between the current image block to be encoded and the reference image block during the video encoding process.

[0077] (7) Motion vector

[0078] A motion vector refers to the relative displacement between a current image block to be encoded and a reference image block during video encoding.

[0079] (8) Motion Vector Difference (MVD)

[0080] The motion vector difference refers to the difference between the motion vector corresponding to the current image block to be encoded and the motion vector corresponding to the adjacent image block. Since the motion vectors corresponding to adjacent image blocks in the same image of the video are correlated, the difference between the motion vector corresponding to an image block and the motion vector corresponding to the image library adjacent to the image block can be calculated, and only the motion vector difference is encoded during the encoding process, thereby reducing the amount of video encoding data.

[0081] For example, assuming that the motion vector corresponding to an image block is (1, 2), and the motion vectors corresponding to other image blocks adjacent to the image block are also (1, 2), then the motion vector difference corresponding to the image block is 0.

[0082] (9) Transformation

[0083] Transformation refers to converting an image described in the form of pixels in the spatial domain to a transform domain (usually the frequency domain) and expressing it in the form of transform coefficients. Generally speaking, most areas in an image are flat areas and areas where the content changes slowly, that is, most of the content in the image is low-frequency information (such as the overall contour information of the image), and the high-frequency information in the image (such as the edge or detail information of the image) is often relatively small, so appropriate transformation can convert the dispersed distribution of image energy in the spatial domain into a relatively concentrated distribution in the transform domain, so as to achieve the purpose of removing redundancy. In general, transformation refers to the conversion of an image from the time domain to the frequency domain, thereby removing the correlation between adjacent data, that is, removing spatial redundancy.

[0084] Specifically, the transformation does not directly encode the spatial image signal, but first maps the spatial image signal to another orthogonal vector space (transform domain or frequency domain), generates a batch of transform coefficients, and then encodes these transform coefficients. Transformation is actually an indirect coding method. The key problem is that when describing in the time domain or spatial domain, the data has a large correlation and data redundancy. After the transformation is described in the transform domain, the data correlation is greatly reduced, the data redundancy is reduced, the parameters are independent, and the amount of data is small. In this way, after the video data is transformed and then quantized, the encoding can obtain a larger compression ratio. Typical transformation methods include discrete cosine transform (DCT), discrete Fourier transform (DFT), etc.

[0085] (10) Quantification

[0086] Quantization refers to the mapping of transform coefficients to a finite number of quantization coefficient values, which can realize the mapping of a data set containing a large amount of data to a data set containing a small amount of data. Simply put, quantization is to reduce the amount of encoded data by representing fine data with coarser data, or by removing information that is not sensitive to the human eye.

[0087] In the video encoding process, the quantization process is the only lossy process, which mainly determines the compression rate of the entire video. The higher the compression rate of the video, the worse the video quality and the lower the bit rate; conversely, the lower the video compression rate, the better the video quality and the higher the bit rate. Specifically, quantization is the process of mapping the continuous value of the signal into multiple discrete amplitudes, realizing a many-to-one mapping of signal values. In the transformation step, after the residual data is transformed, the transform coefficient has a larger value range. Quantization can effectively reduce the value range of the signal, thereby obtaining a better compression effect.

[0088] Generally speaking, the quantization parameter (QP) in the quantization process corresponds to the serial number of the quantization step. The size of the quantization parameter can reflect the compression of spatial details. The smaller the quantization parameter, the finer the quantization, the more details retained in the image, the higher the image quality, and the larger the storage space occupied by the image; the larger the quantization parameter, the coarser the quantization, the more details lost in the image, the lower the image quality, and the smaller the storage space occupied by the image.

[0089] (11) Entropy Coding

[0090] Entropy coding is a type of coding that uses the statistical characteristics of the source to perform bit rate compression. The basic idea of ​​entropy coding is to make the code words before and after as random as possible, reduce the correlation between the before and after, and get closer to the Shannon entropy of the source. In general, entropy coding has the function of eliminating statistical redundancy between data. It is usually used as the last step at the encoding end to write syntax elements into the output bit stream.

[0091] (12)YUV

[0092] YUV is a color encoding method, where Y represents brightness, and U and V represent chrominance. Compared with RGB, YUV is more suitable for encoding and decoding.

[0093] (13) Bitstream

[0094] A bitstream is a coded data stream obtained by encoding the images in a video. In each video coding standard, the data structure in the bitstream is defined to indicate the content information carried by each part of the data in the bitstream.

[0095] (14) Block skip mode

[0096] Block skip mode is a coding mode in the video coding process that can reduce. In block skip mode, by defining an image block to be encoded as a skip block, it means that the prediction residual and motion vector difference of the image block are both 0. Therefore, during the encoding process, the prediction residual and motion vector difference of the image block can no longer be encoded, thereby reducing the content to be encoded and reducing the amount of data after video encoding. In other words, only image blocks with prediction residual and motion vector difference of 0 will be encoded using block skip mode. Among them, a prediction residual of 0 means that there is no difference between the current image block and the reference image block in the reference frame, and a motion vector difference of 0 means that the motion of the current image block and the adjacent image block is the same.

[0097] The inventors have found that, since a video is essentially composed of a group of continuous images, adjacent images in a video are usually obtained by shooting the same content at adjacent moments, so the contents between adjacent images often have a high correlation. In addition, when the video acquisition device (such as a camera) is fixed, when the object captured by the video moves, the position of the same object in different images will change. For example, a vehicle will appear in different positions in different images during movement. In most scenarios, people usually only need to pay attention to the significant movement of objects in the video, and do not need to pay attention to the slight disturbances of some objects in the video. For example, in a security scenario, you usually only need to pay attention to the movement of pedestrians, vehicles, and other movements, and do not need to pay attention to the slight disturbances of leaves or flowers caused by wind.

[0098] In view of this, an embodiment of the present application provides a video encoding method, when encoding an image block in an image of a video, if it is confirmed that the motion vector of the image block to be encoded is smaller than that of the reference image block, the image block is encoded through a block skip mode, that is, it is considered that the prediction residual and motion vector difference of the image block compared with the reference image block are both 0, so the prediction residual and motion vector difference of the image block are not encoded and transmitted. Since the situation that the motion vector of the image block is smaller than that of the reference image block is actually some slight disturbance in the video, encoding the image block through the block skip mode can skip some unimportant disturbance content in the video encoding process to reduce the amount of data after video encoding, and can maintain the normal motion part to ensure the quality of the encoded video.

[0099] For example, the video encoding method provided in this embodiment may be applied to a security scene. The video encoding method provided in this embodiment may also be applied to other scenes, which are not specifically limited in this embodiment.

[0100] See also Figure 2 , Figure 2A schematic diagram of a security scenario provided in an embodiment of the present application. Figure 2 As shown, in the security scenario, the intelligent security system can usually be composed of an acquisition module, a storage module and an analysis module. For example, the acquisition module may include a security camera, an internet protocol camera (IPC) or a software defined camera (SDC), etc., the storage module may include a video cloud node (VCN), etc., and the analysis module may include a video content management system (VCM), etc. The acquisition module at the front end acquires the video, compresses and encodes the video, and transmits the encoded video data to the storage module at the back end through the network. The storage module stores the acquired video data. The analysis module extracts the video data stored in the storage module, performs intelligent identification and analysis on the video data, and obtains the analysis results.

[0101] The video encoding method provided in this embodiment can be applied to the front-end acquisition module in the security scene, for example, to a security camera with video encoding capability. In this way, after the security camera acquires the video, it can encode the acquired video based on the video encoding method provided in this embodiment, thereby reducing the data volume of the encoded code stream.

[0102] See also Figure 3 , Figure 3 This is a schematic diagram of a system architecture in a security scenario provided by an embodiment of the present application. Figure 3 As shown, the system architecture includes one or more front-end cameras, back-end storage devices, video processing platforms and user terminals.

[0103] The front-end camera is used to capture video, and the video encoding method provided in this embodiment is used to encode the captured video to obtain a code stream and send the code stream to a back-end storage device or a video processing platform. The back-end storage device is used to store the code stream sent by the front-end camera.

[0104] The video processing platform is used to process the bit stream encoded by the front-end camera, and the video processing platform can obtain the bit stream encoded by the front-end camera from the front-end camera or the back-end storage device. Specifically, the video processing platform can use the video decoding method provided in this embodiment to decode the bit stream to obtain a video, so that the user can play the video decoded by the video processing platform online.

[0105] Alternatively, the video processing platform may retrieve the code stream requested to be downloaded by the user terminal from the backend storage device according to the download request sent by the user terminal, and transmit the code stream to the user terminal, so that the user terminal can download the video.

[0106] See also Figure 4 , Figure 4 A schematic diagram of a video encoding method provided in an embodiment of the present application. Figure 4 As shown, the video encoding method provided in this embodiment is applied to a video encoding end, and specifically includes the following steps 401-403.

[0107] Step 401, obtaining a first image, where the first image is an image in a video.

[0108] In this embodiment, a video to be encoded is first obtained, and then multiple images in the video are encoded one by one. The first image is an image in the video, and the first image needs to refer to other encoded images for encoding during the encoding process.

[0109] Step 402: Determine a reference image block in a second image corresponding to a first image block in a first image, where the second image is a reference frame of the first image.

[0110] In the process of encoding the first image, the reference frame of the first image may be determined first, then the first image may be divided into blocks to obtain a plurality of image blocks in the first image, and then a corresponding reference image block may be determined in the reference frame for each image block in the first image.

[0111] In this embodiment, the first image block may be any image block in the first image, and the size of the first image block is determined by the actual block division of the first image. For example, the size of the first image block may be 8*8, that is, the first image block is composed of 8*8 pixels.

[0112] In addition, the reference image block has the same size as the first image block, and the reference image block is the image block in the second image that has the highest similarity with the first image block. Similarly, when determining the reference image block corresponding to the first image block in the second image, the second image can be firstly divided into image blocks in the same manner as the first image is divided into image blocks, thereby obtaining a plurality of image blocks in the second image. Then, the image block that best matches the first image block (i.e., the image block with the highest similarity to the first image block) is found from the plurality of image blocks in the second image, thereby obtaining the reference image block corresponding to the first image block.

[0113] Step 403 , in response to the first image block and the reference image block satisfying a preset condition, encoding the first image block in a block skip mode to obtain an encoding result.

[0114] In this embodiment, the preset condition includes that the displacement of the first image block relative to the reference image block (i.e., the motion vector of the first image block) is not greater than a first threshold value. The first threshold value may be, for example, N pixels, that is, when the displacement of the first image block relative to the reference image block is not greater than N pixels, it is considered that the first image block meets the preset condition, and thus the first image block is encoded by the block skip mode to obtain an encoding result.

[0115] When the block skip mode is used to encode the first image block, it can be considered that the prediction residual and motion vector difference of the first image block are both 0, so the prediction residual and motion vector difference of the first image block are not encoded, thereby reducing the data amount of the encoding result obtained after encoding.

[0116] That is to say, when the displacement of the first image block relative to the reference image block is small, it can be considered that the movement of the first image block is actually a slight disturbance. Therefore, the disturbance of encoding the first image block is skipped during the video encoding process to reduce the amount of data after video encoding.

[0117] It can be understood that the following steps 401-403 introduce the process of encoding an image block (i.e., the first image block) in the first image. In the actual encoding process of the image, multiple image blocks in the image also need to be encoded. For the image blocks in the image that meet the above preset conditions, the block skip mode can also be used to encode the image blocks; for the image blocks in the image that do not meet the above preset conditions, the skip mode is not used to encode the image blocks, but the prediction residual and motion vector difference corresponding to the image block are normally encoded and transmitted.

[0118] For example, see Figure 5 , Figure 5 A schematic diagram of using a block skip mode to encode a portion of image blocks in an image provided by an embodiment of the present application. Figure 5 As shown, Figure 5 The reference frame, the original image and the encoded image are included. The original image is the original image that needs to be encoded in the video (such as the first image mentioned above), the reference frame is the image that the original image needs to refer to during the encoding process (such as the second image mentioned above), and the encoded image refers to the image obtained after encoding the original image (that is, the image obtained after decoding the encoding result corresponding to the original image). When the method provided in the embodiment of the present application is used to encode the original image, the block skip mode may be used to encode some image blocks in the original image, resulting in a certain difference between the encoded image and the original image.

[0119] exist Figure 5In the original image, the image block 1, the image block 2 and the image block 3 are included, and the image block 1, the image block 2 and the image block 3 all find the corresponding reference image block in the reference frame (as indicated by the arrow direction). Moreover, the displacement of the image block 1 relative to the corresponding reference image block is not greater than the first threshold value, and the displacement of the image block 2 relative to the corresponding reference image block and the displacement of the image block 3 relative to the corresponding reference image block are greater than the first threshold value. Therefore, when encoding the original image, the block skip mode is used to encode the image block 1, while the block skip mode is not used to encode the image block 2 and the image block 3. In this way, after the original image is encoded, in the obtained encoded image, the position and content of the image block 2 and the image block 3 in the original image do not change, that is, the content of the position where the image block 2 and the image block 3 in the original image are located is the same as the content at the same position in the encoded image. However, since the image block 1 in the original image is encoded using the block skip mode, the content of the position where the image block 1 in the original image is located is different from the content at the same position in the encoded image. In simple terms, the difference between the image block 1 in the original image and the reference image block in the reference frame is not encoded, resulting in that the content corresponding to the image block 1 in the original image in the encoded image is actually the reference image block in the reference frame.

[0120] In this embodiment, when determining the reference frame of the first image (ie, the second image), it is necessary to select an already encoded image as the reference frame of the first image. Therefore, the reference frame of the first image may actually have multiple situations.

[0121] In a possible case, the previous frame of the first image in the video is not encoded using the block skip mode, and the encoded image corresponding to the previous frame of the first image in the video is the same as the previous frame of the first image in the video, so the second image is the previous frame of the first image in the video. That is, an image in the same video that is located before the first image can be used as a reference frame of the first image, thereby performing encoding of the first image.

[0122] In another possible case, the previous frame image of the first image in the video is encoded using a block skip mode, and the encoded image corresponding to the previous frame image of the first image in the video is likely to be different from the previous frame image of the first image in the video. In this case, the second image, which is a reference frame of the first image, can be an encoded image obtained by encoding the image in the video using a block skip mode, for example, the second image is an encoded image obtained by encoding the previous frame image of the first image in the video using a block skip mode.

[0123] In this scheme, by selecting the encoded image as the reference frame of the first image, it can be ensured that the decoding process can be based on the decoded image to achieve decoding of other subsequent images, and the motion skipped when using block skip mode encoding will not be accumulated and skipped, avoiding the phenomenon of video picture motion distortion.

[0124] In this embodiment, the specific value of the first threshold value can be adjusted according to the encoding situation, and this embodiment does not make specific limitations. Generally speaking, the larger the value of the first threshold value, the more image blocks will be encoded in the video through the block skip mode, and the lower the bit rate of the bit stream obtained after encoding will be. However, at the same time, the larger the value of the first threshold value, the more image blocks will be encoded in the video through the block skip mode, and the greater the quality difference between the video obtained after encoding and the original video will be, that is, there is a certain difference in picture details between the video obtained after encoding and the original video. Therefore, in practical applications, the value of the first threshold value can be determined according to actual needs. When it is expected to obtain a lower bit rate without paying attention to the quality of picture details, a larger value can be taken for the first threshold value; when it is expected to have better picture detail quality, a smaller value can be taken for the first threshold value.

[0125] In one possible example, the size of the first threshold is related to the resolution of the first image, and the first threshold is positively correlated with the resolution of the first image. That is, the higher the resolution of the first image, the larger the value of the first threshold; the lower the resolution of the first image, the smaller the value of the first threshold. For example, when the resolution of the first image is 1920*1080, the value of the first threshold can be 4 pixels. It can be understood that when the resolution of the first image is low, the number of pixels used to form an object will also be relatively small. When the first threshold is set relatively high, it is easy to ignore the normal movement of the object during the video encoding process, thereby affecting the quality of the video.

[0126] Therefore, by setting the size of the first threshold to be related to the resolution of the first image, the first threshold can be adaptively adjusted based on the resolution of the first image, ensuring that only some unimportant disturbance content is skipped during the video encoding process without affecting normal motion content, thereby ensuring the quality of the encoded video.

[0127] Optionally, the acquisition scene of the above video is a security scene or other scenes, and this embodiment does not limit the acquisition scene of the video. In the security scene, the camera used to collect the video is usually fixedly deployed at a specific position, and the movement of the same object in the actual scene can be reflected by the position of the object between different images of the video. Generally speaking, the more violent the movement of the object, the greater the distance between the positions of the object in different images; the slighter the movement of the object, the smaller the distance between the positions of the object in different images. Moreover, in the security scene, the movement that needs to be paid attention to is often the movement of pedestrians, motor vehicles and non-motor vehicles with large amplitudes, and there is no need to pay attention to the slight disturbance of leaves or flowers caused by wind. Therefore, the video encoding method provided by this embodiment is applied in the video encoding process of the security scene, so that slight disturbances such as wind and grass can be ignored as much as possible during the video encoding process, thereby reducing the amount of data after video encoding, and can maintain the normal motion part to ensure the quality of the encoded video.

[0128] In some embodiments, the preset condition may include other conditions in addition to the above-mentioned displacement of the first image block relative to the reference image block being no greater than the first threshold.

[0129] In a possible implementation, the preset condition also includes that the first image block does not belong to a flickering area in the video. The flickering area in the video refers to an area whose brightness changes greatly over time (i.e., an area that flickers). For example, assuming that the video includes images 1 to 4, the position belonging to the flickering area in image 1 has a higher brightness, the position belonging to the same flickering area in image 2 has a lower brightness, the position belonging to the same flickering area in image 3 has a higher brightness, and the position belonging to the same flickering area in image 4 has a lower brightness, that is, the brightness of the same flickering area in different images fluctuates.

[0130] It is understandable that in some scenarios, the flickering area in the video is usually what the user needs to pay attention to (so the flickering often has a warning nature, such as the flashing lights of an ambulance, the flashing high beams on a car, the flashing lights of a roadside warning sign, and the flashing lights of a flashlight held by a pedestrian), so it is often necessary to save the detailed information of the flickering area in the video, and the flickering part in the encoded video cannot be skipped. Based on this, in this embodiment, the condition that the first image block does not belong to the flickering area is added to the preset conditions. Only when the first image block meets both the displacement requirements and the requirements of the non-flickering area, the encoding of the motion and detailed content of the first image block will be skipped, ensuring that the encoded video can retain the detailed information of the flickering area and ensure the quality of the encoded video.

[0131] Optionally, the first image block does not belong to the flickering area in the video specifically means that the first image block does not meet the flickering condition. The flickering condition is that the sum of the differences between the gradient of the pixels in the first image block and the gradient of the corresponding pixels in the reference image block is less than the second threshold, and the sum of the differences between the brightness of the pixels in the first image block and the brightness of the corresponding pixels in the reference image block is greater than the third threshold, and the gradient of the pixels in the first image block is the difference between the pixels in the first image block and the adjacent pixels.

[0132] Specifically, for each pixel in the first image block, the gradient of the pixel can be calculated. That is, the gradient of any pixel in the first image block is the difference between the pixel and the adjacent pixel in the first image block, such as the difference between the pixel and an adjacent pixel, or the average value of the difference between the pixel and multiple adjacent pixels (such as adjacent pixels in four directions of up, down, left, and right). The sum of the differences between the gradient of the pixel in the first image block and the gradient of the corresponding pixel in the reference image block refers to the sum of the gradient differences of each pair of pixels having a corresponding relationship in the first image block and the reference image block. That is, the first image block and the reference image block both include multiple pixels, and each pixel in the first image block has a corresponding pixel in the reference image block. By calculating the gradient difference between each pair of pixels having a corresponding relationship, and then adding the gradient differences of multiple pairs of pixels, the above-mentioned sum of differences can be obtained. Since the gradient of a pixel represents the difference between a certain pixel and its adjacent pixels, if the sum of the differences between the gradients of the pixels in the first image block and the gradients of the corresponding pixels in the reference image block is less than the second threshold, it can be considered that the difference between the pixels in the first image block and the adjacent pixels is close to the difference between the pixels in the reference image block and the adjacent pixels, that is, the distribution of the pixels is basically the same.

[0133] Similarly, since each pixel in the first image block and the reference image block has a corresponding brightness value (e.g., Y value in YUV), the brightness of any pixel in any image block can be determined. The difference between the brightness of a pixel in the first image block and the brightness of a corresponding pixel in the reference image block refers to the sum of the brightness differences of each pair of pixels in the first image block and the reference image block that have a corresponding relationship. In this way, when the sum of the differences between the brightness of the pixels in the first image block and the brightness of the corresponding pixels in the reference image block is greater than the third threshold, it can be considered that the brightness of the entire pixel in the first image block is significantly different from the brightness of the entire pixel in the reference image block.

[0134] Therefore, when the sum of the differences between the gradients of the pixels in the first image block and the gradients of the corresponding pixels in the reference image block is less than the second threshold, and the sum of the differences between the brightness of the pixels in the first image block and the brightness of the corresponding pixels in the reference image block is greater than the third threshold, it can be considered that the pixel distribution of the first image block relative to the reference image block has not changed much, but the brightness of the pixels as a whole in the first image block has changed significantly. Therefore, it can be confirmed that the brightness of the first image block has changed suddenly relative to the reference image block, that is, the first image block and the reference image block actually belong to the flickering area.

[0135] The values ​​of the second threshold and the third threshold may be determined or adjusted according to actual conditions, such as based on requirements of bit rate and detail information richness, and this embodiment does not specifically limit this.

[0136] For example, see Figure 6 , Figure 6 A schematic diagram of comparing a first image block belonging to a flickering area with a reference image block provided in an embodiment of the present application. Figure 6 As shown, the first image block and the reference image block both include 8*8 pixels, and the brightness value of the 4*4 pixels in the center of the reference image block is 10, and the brightness value of the 48 pixels in the periphery of the reference image block is 50, the brightness value of the 4*4 pixels in the center of the first image block is 210, and the brightness value of the 48 pixels in the periphery of the reference image block is 250. Obviously, since the difference between the pixels in the center and the pixels in the periphery of the reference image block and the first image block is the same, the sum of the gradient difference values ​​of the pixels in the first image block and the gradient difference values ​​of the pixels in the reference image block is smaller; and the brightness value of each pixel in the first image block is increased by 200 compared with the brightness value of the corresponding pixel in the reference image block, so the sum of the brightness difference values ​​of the pixels in the first image block and the pixels in the reference image block is larger. In this case, it can be determined that the first image block belongs to the flickering area.

[0137] In this solution, based on the pixel gradient difference and pixel brightness difference between the first image block and the reference image block, it can be effectively determined whether the first image block belongs to the flickering area, thereby ensuring effective identification of the flickering area in the image and improving the feasibility of the solution.

[0138] Generally speaking, since the reference image block determined in the reference frame for a current image block in the image is the most similar image block, and under the restriction that the displacement of the current image block relative to the reference image block is not greater than the first threshold as described above, the difference between the current image block and the reference image block is usually relatively small, it can often be considered that the current image block is formed due to a slight disturbance of the object corresponding to the reference image block. However, in some extreme cases, the difference between the current image block and the reference image block may also be large, such as when the current image block belongs to the above-mentioned flickering area. Therefore, for image blocks that are significantly different from the reference image blocks, it is also possible not to use the block skip mode for encoding.

[0139] In another possible implementation, the preset condition further includes that the sum of the differences between the pixels in the first image block and the corresponding pixels in the reference image block is less than a fourth threshold. The value of the fourth threshold may be determined according to actual conditions, such as based on requirements of bit rate and detail information richness, and this embodiment does not specifically limit this.

[0140] In simple terms, each pixel in the image block has a pixel value to represent the content represented by the pixel. In this way, for each pixel in the first image block, the pixel value difference between the pixel and the corresponding pixel in the reference image block can be calculated; by calculating the pixel value difference between each pair of pixels in the first image block and the reference image block that have a corresponding relationship, the sum of the differences between the pixels in the first image block and the corresponding pixels in the reference image block can be obtained. The greater the pixel value difference between the two pixels, the greater the difference in the content represented by the pixel. Therefore, if the sum of the differences between the pixels in the first image block and the corresponding pixels in the reference image block is less than the fourth threshold, it can be considered that the content in the first image block is not much different from the content in the reference image block, and the block skip mode can be used to encode the first image block. When the content in the first image block is greatly different from the content in the reference image block, the first image block will not meet the preset conditions, so the block skip mode is not used to encode the first image block.

[0141] In this solution, by adding the condition that the difference between the pixels in the image block to be encoded and the corresponding pixels in the reference image block is less than a specific threshold in the requirement of using the block skip mode, it can be ensured that the encoding process skips the encoding of some detailed information with little content change, and ensures that the detailed information with large content change can be presented in the video, thereby ensuring the quality of the encoded video.

[0142] See also Figure 7 , Figure 7 The present invention provides a schematic diagram of encoding an image using a block skip mode in a security scenario. Figure 7As shown, after obtaining an image in a video, the image is first divided into blocks. For example, the image is divided into 8*8 units, so that the size of each image block obtained by division is 8*8.

[0143] Then, the motion vector of each image block is calculated in combination with the reference frame corresponding to the current image, and it is calculated whether each image block belongs to the flickering area.

[0144] Secondly, for each image block, determine whether the image block meets the preset conditions. That is, whether the motion vector of the image block is not greater than N pixels, and the sum of the differences between the gradients of the pixels in the image block and the gradients of the corresponding pixels in the reference image block is less than 8*8*a, and the sum of the differences between the brightness of the pixels in the image block and the brightness of the corresponding pixels in the reference image block is greater than 8*8*b. Where a and b are preset values, and both a and b are greater than 0.

[0145] Finally, for an image block that meets the preset conditions, the image block can be defined as a skip block, so that the image block is encoded using the block skip mode. If all image blocks in the current image meet the above preset conditions, the current image can be defined as a skip frame, so that the entire image is encoded using the skip mode.

[0146] See also Figure 8 , Figure 8 The following is a schematic diagram of an image obtained by encoding in a block skip mode in a security scenario provided by an embodiment of the present application. Figure 8 As shown, in the security scene, the nearby vehicles occupy more pixels in the image, so the motion vector of the image block corresponding to the nearby vehicles is larger, and the block skip mode will not be used to encode the image block corresponding to the nearby vehicles during encoding. However, for the farthest vehicle, since the farthest vehicle occupies fewer pixels in the image and the movement speed of the farthest vehicle itself is lower, the motion vector of the image block corresponding to the farthest vehicle is smaller, and the block skip mode may be used to encode the image block corresponding to the near vehicle during encoding. In short, in the video encoding process, since the motion vector corresponding to the distant vehicle is smaller, the motion of this part of the vehicle will be skipped intermittently during the encoding process; the position of the distant vehicle will be updated only when the movement distance of the distant vehicle is greater than a certain value, otherwise the distant vehicle will be regarded as stationary, that is, the position of the distant vehicle will not be updated in real time in each image. Since the clarity of the distant vehicle itself in the image is low, it does not need to be focused on in the security scene, so encoding based on the encoding method provided in this embodiment can effectively reduce the amount of encoded data without affecting the overall quality of the video.

[0147] For example, see Fig. 9 , Fig. 9 The following is a schematic diagram of the comparison of multiple images in a video before and after encoding provided by an embodiment of the present application. Fig. 9 As shown, image 1, image 2 and image 3 are three consecutive images in the video, and the encoded image 1, the encoded image 2 and the encoded image 3 are the encoded images corresponding to image 1, image 2 and image 3, respectively. In addition, the reference frame of the encoded image 2 is the encoded image 1, and the reference frame of the encoded image 3 is the encoded image 2.

[0148] In image 1, image 2, and image 3, there are image blocks corresponding to the same object (for example, object 1), and the positions of the image blocks in each image are different. Specifically, in image 1, the image block corresponding to object 1 is in the upper left corner of image 1; in image 2, the image block corresponding to object 1 is moved to the right by a certain distance compared to the position in image 1; in image 3, the image block corresponding to object 1 is moved to the right by a certain distance compared to the position in image 2.

[0149] When encoding image 1, the block skip mode is not used to encode the image block corresponding to object 1, so the encoded image 1 is the same as image 1. When encoding image 2, since the displacement of the image block corresponding to object 1 relative to the reference image block is not greater than the first threshold, the block skip mode is used to encode the image block corresponding to object 1, resulting in that in the encoded image 2, the image block corresponding to object 1 is still located at the upper left corner and has not moved. When encoding image 3, since the encoded image 2 is selected as the reference frame, the image block corresponding to object 1 in image 3 is displaced by a long distance relative to the reference image block located in the upper left corner of the reference frame, that is, the displacement of the image block corresponding to object 1 in image 3 relative to the reference image block located in the upper left corner of the reference frame is greater than the first threshold, and thus the block skip mode is not used to encode the image block corresponding to object 1 in image 3.

[0150] Depend on Fig. 9 It can be seen that for objects that move continuously in the video, after encoding using the video encoding method provided in this embodiment, the movement of the object in some intermediate images may be lost, but the overall movement of the object will still be retained (only part of the movement trajectory of the object will be skipped), which will not have much impact on the quality of the video in practical applications.

[0151] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of the method. It is understandable that, in order to realize the above functions, the electronic device includes a hardware structure and / or software module corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the modules and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0152] The embodiment of the present application can divide the functional modules of the electronic device according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0153] The following is a detailed description of the device used to execute the above method in the embodiment of the present application.

[0154] See also Fig.10 , Fig.10 FIG. 1 is a schematic diagram of the structure of a video encoding device provided in an embodiment of the present application. Fig.10 As shown, the video encoding device includes: an acquisition module 1001, used to acquire a first image, the first image is an image in a video; a processing module 1002, used to determine a reference image block corresponding to a first image block in the first image in a second image, the second image being a reference frame of the first image; the processing module 1002 is also used to, in response to the first image block and the reference image block satisfying a preset condition, encode the first image block through a block skip mode to obtain an encoding result; wherein the preset condition includes that the displacement of the first image block relative to the reference image block is not greater than a first threshold value, and the block skip mode is a mode in which the prediction residual and motion vector difference of the first image block are not encoded.

[0155] In a possible implementation manner, the preset condition also includes that the first image block does not belong to a flickering area in the video.

[0156] In a possible implementation, the first image block does not belong to the flickering area in the video, specifically the first image block does not meet the flickering condition, the flickering condition is that the sum of the differences between the gradient of the pixels in the first image block and the gradient of the corresponding pixels in the reference image block is less than a second threshold, and the sum of the differences between the brightness of the pixels in the first image block and the brightness of the corresponding pixels in the reference image block is greater than a third threshold, and the gradient of the pixels in the first image block is the difference between the pixels in the first image block and the adjacent pixels.

[0157] In a possible implementation manner, the preset condition also includes that a sum of differences between pixels in the first image block and corresponding pixels in the reference image block is less than a fourth threshold.

[0158] In a possible implementation manner, the second image is an image frame previous to the first image in the video.

[0159] In a possible implementation manner, the second image is an encoded image obtained by encoding an image in the video using a block skip mode.

[0160] In a possible implementation manner, the size of the first threshold is related to the resolution of the first image.

[0161] In a possible implementation manner, the reference image block is an image block in the second image that has the smallest difference with the first image block.

[0162] In a possible implementation, the video acquisition scenario is a security scenario.

[0163] See also Fig.11 , Fig.11 FIG. 1 is a schematic diagram of the structure of a chip according to an embodiment of the present application. Fig.11 As shown, the chip may include hardware modules such as a hardware decoder, a hardware image processor, a neural network processor (neural-network processing unit, NPU), a central processing unit (central processing unit, CPU) and a hardware encoder, which can support video encoding and decoding. Wherein, when the chip is deployed at the video encoding end, the hardware encoder in the chip can be used to execute the video encoding method provided in the embodiment of the present application. When the chip is deployed at the video decoding end, the hardware decoder in the chip can be used to execute the video decoding method provided in the embodiment of the present application. The hardware encoder and hardware decoder in the chip can cooperate to execute the code stream processing method provided in the embodiment of the present application.

[0164] It should be noted that the information interaction, execution process, etc. between the modules / or components in the above device are the same as those in this application. Figure 3-Figure 10The corresponding method embodiments are based on the same concept. For specific contents, please refer to the description in the method embodiments shown above in this application, which will not be repeated here.

[0165] It should be noted that for the specific implementation of the device and the beneficial effects brought about, please refer to Figure 4-Figure 9 The descriptions in the corresponding method embodiments will not be repeated here one by one.

[0166] An embodiment of the present application also provides a computer-readable storage medium, including instructions, which, when executed on a computer, enables the computer to control an electronic device to execute any one of the implementation methods shown in the aforementioned method embodiments.

[0167] An embodiment of the present application also provides a computer program product, which includes a computer program code. When the computer program code runs on a computer, the computer executes any one of the implementation methods shown in the aforementioned method embodiments.

[0168] An embodiment of the present application also provides a chip system, including a memory and a processor, the memory is used to store computer programs, and the processor is used to call and run computer programs from the memory, so that the chip executes any one of the implementation methods shown in the aforementioned method embodiments.

[0169] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0170] Through the description of the above implementation methods, the technicians in the relevant field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented with corresponding hardware, and the specific hardware structures used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation method in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, U disk, mobile hard disk, ROM, RAM, disk or optical disk, etc., including a number of instructions to enable a computer device to execute the methods described in each embodiment of the present application.

[0171] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0172] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a communication device, a computing device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, communication device, computing device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a communication device, a data center, or other data storage device that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

[0173] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0174] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

Claims

1. A video encoding method, characterized in that: include: Acquire a first image, where the first image is an image in a video; Determine a reference image block in a second image corresponding to a first image block in a first image, where the second image is a reference frame of the first image; In response to the first image block and the reference image block satisfying a preset condition, encoding the first image block in a block skip mode to obtain an encoding result; The preset condition includes that the displacement of the first image block relative to the reference image block is not greater than a first threshold, and the block skip mode is a mode for not encoding the prediction residual and motion vector difference of the first image block.

2. The method according to claim 1, characterized in that The preset condition also includes that the first image block does not belong to a flickering area in the video.

3. The method according to claim 2, characterized in that The first image block does not belong to the flickering area in the video specifically because the first image block does not meet the flickering condition, the flickering condition being that the sum of the differences between the gradient of the pixels in the first image block and the gradient of the corresponding pixels in the reference image block is less than a second threshold, and the sum of the differences between the brightness of the pixels in the first image block and the brightness of the corresponding pixels in the reference image block is greater than a third threshold, and the gradient of the pixels in the first image block is the difference between the pixels in the first image block and the adjacent pixels.

4. The method according to claim 1, characterized in that The preset condition also includes that the sum of differences between pixels in the first image block and corresponding pixels in the reference image block is less than a fourth threshold.

5. The method according to any one of claims 1 to 4, characterized in that: The second image is an image frame preceding the first image in the video.

6. The method according to any one of claims 1 to 4, characterized in that: The second image is an encoded image obtained by encoding the image in the video through a block skip mode.

7. The method according to any one of claims 1 to 6, characterized in that: The size of the first threshold is related to the resolution of the first image.

8. The method according to any one of claims 1 to 7, characterized in that: The reference image block is an image block in the second image that has the smallest difference with the first image block.

9. The method according to any one of claims 1 to 8, characterized in that: The video is acquired in a security scenario.

10. A video encoding device, characterized in that: include: An acquisition module, used to acquire a first image, where the first image is an image in a video; A processing module, configured to determine a reference image block corresponding to a first image block in a first image in a second image, wherein the second image is a reference frame of the first image; The processing module is further configured to encode the first image block in a block skip mode to obtain an encoding result in response to the first image block and the reference image block satisfying a preset condition; The preset condition includes that the displacement of the first image block relative to the reference image block is not greater than a first threshold, and the block skip mode is a mode for not encoding the prediction residual and motion vector difference of the first image block.

11. The device according to claim 10, characterized in that The preset condition also includes that the first image block does not belong to a flickering area in the video.

12. The device according to claim 11, characterized in that The first image block does not belong to the flickering area in the video specifically because the first image block does not meet the flickering condition, the flickering condition being that the sum of the differences between the gradient of the pixels in the first image block and the gradient of the corresponding pixels in the reference image block is less than a second threshold, and the sum of the differences between the brightness of the pixels in the first image block and the brightness of the corresponding pixels in the reference image block is greater than a third threshold, and the gradient of the pixels in the first image block is the difference between the pixels in the first image block and the adjacent pixels.

13. The device according to claim 10, characterized in that The preset condition also includes that the sum of differences between pixels in the first image block and corresponding pixels in the reference image block is less than a fourth threshold.

14. The device according to any one of claims 10 to 13, characterized in that: The second image is an image frame preceding the first image in the video.

15. The device according to any one of claims 10 to 13, characterized in that: The second image is an encoded image obtained by encoding the image in the video through a block skip mode.

16. The device according to any one of claims 10 to 15, characterized in that: The size of the first threshold is related to the resolution of the first image.

17. The device according to any one of claims 10 to 16, characterized in that: The reference image block is an image block in the second image that has the smallest difference with the first image block.

18. The device according to any one of claims 10 to 17, characterized in that: The video is acquired in a security scenario.

19. A video encoding device, characterized in that: The device comprises a memory and a processor; the memory stores codes, the processor is configured to execute the codes, and when the codes are executed, the device executes the method according to any one of claims 1 to 9.

20. A computer storage medium, characterized in that The computer storage medium stores instructions, which, when executed by a computer, cause the computer to implement the method of any one of claims 1 to 9.

21. A computer program product, characterized in that The computer program product stores instructions, which, when executed by a computer, cause the computer to implement the method according to any one of claims 1 to 9.

Images