Video coding method, video decoding method and device

Abstract

The embodiment of the invention provides a video coding method and device and a video decoding method and device, and relates to the technical field of video coding and decoding. Comprising the following steps: acquiring a target image block which is a rectangular image block obtained by performing block division on a video frame; performing texture edge detection on the target image block to obtain at least one texture edge of the target image block; segmenting the target image block into two image areas based on the at least one texture edge, and obtaining a boundary of the two image areas; performing curve fitting on the boundary to obtain a segmentation curve of the target image block; segmenting the target image block into two sub-image blocks based on the segmentation curve of the target image block; and obtaining code stream data corresponding to the target image block according to the segmentation curve of the target image block and the two sub-image blocks. Some embodiments of the invention are used for enabling the segmentation line of block division to be more fit with a real texture edge, thereby improving the performance of video coding.

Description

Technical Field

[0001] Some embodiments of the present application relate to the technical field of video encoding and decoding, and more specifically, to a video encoding method, a video decoding method and a device. Background Art

[0002] A complete image in a video is usually called a video frame. The content within a video frame and between video frames is often highly correlated, so there is a lot of redundant information in the video. The purpose of video encoding is to remove as much redundant information as possible in the video to reduce the amount of data representing the video content.

[0003] At present, when encoding video frames, mainstream video coding standards first divide video frames into coding tree units (CTUs) of different sizes, and use the divided CTUs as the minimum unit for encoding and decoding. The encoding and decoding tasks for CTUs can be further refined to divide the CTUs into multiple coding units (CUs). This process is called block partitioning. The coding performance of the video is closely related to the block partitioning. Taking inter-frame prediction as an example, inter-frame prediction records the difference between adjacent frames, specifically the difference between the current coding block and the coding block in the coded frame, which is described by motion vectors and prediction residuals. If the block partitioning is unreasonable, the motion compensation overhead will increase and may cause serious blocking effects. Therefore, in the video encoding process, it is necessary to make the boundary of the block partition as close to the real texture edge of the image as possible. However, the block partitioning methods of the current video coding standards are all linear partitioning, that is, the image blocks are further divided by straight lines. But the real image content does not always have straight edges, and even if it has straight edges, the partitioning line of the block partition may not be able to fit the texture edge of the object in the image. Therefore, the current block division method will increase the boundary texture compensation overhead, thereby affecting the performance of video coding. Summary of the invention

[0004] The exemplary embodiments of the present application provide a video encoding method, a video decoding method and an apparatus for making the segmentation lines of block divisions more closely fit the real texture edges, thereby improving the performance of video encoding.

[0005] Some embodiments of the present application provide the following technical solutions:

[0006] In a first aspect, some embodiments of the present application provide a video encoding method, including:

[0007] Obtaining a target image block, wherein the target image block is a rectangular image block obtained by dividing the video frame into blocks;

[0008] Performing texture edge detection on the target image block to obtain at least one texture edge of the target image block;

[0009] Dividing the target image block into two image areas based on the at least one texture edge, and acquiring a boundary line between the two image areas;

[0010] Performing curve fitting on the dividing line to obtain a segmentation curve of the target image block;

[0011] Splitting the target image block into two sub-image blocks based on a segmentation curve of the target image block;

[0012] According to the segmentation curve of the target image block and the two sub-image blocks, the code stream data corresponding to the target image block is acquired.

[0013] In a second aspect, some embodiments of the present application provide a video decoding method, including:

[0014] Acquire bitstream data corresponding to a target image block, wherein the target image block is a rectangular image block obtained by dividing a video frame into blocks; the bitstream data corresponding to the target image block includes: first bitstream data, second bitstream data, and third bitstream data; the first bitstream data is bitstream data obtained by encoding information of a segmentation curve used to segment the target image block into two sub-image blocks; the second bitstream data and the third bitstream data are bitstream data obtained by encoding the two sub-image blocks respectively;

[0015] Acquire the segmentation curve according to the first bitstream data;

[0016] Segmenting the target image block based on the segmentation curve to obtain the two sub-image blocks;

[0017] Reconstructing the two sub-image blocks according to the second code stream data and the third code stream data;

[0018] The reconstructed target image block is obtained according to the two reconstructed sub-image blocks.

[0019] In a third aspect, some embodiments of the present application provide a video encoding device, including:

[0020] a memory configured to store a computer program;

[0021] The processor is configured to, when calling a computer program, enable the video encoding device to implement the video encoding method described in the first aspect.

[0022] In a fourth aspect, some embodiments of the present application provide a video decoding device, including:

[0023] a memory configured to store a computer program;

[0024] The processor is configured to, when calling a computer program, enable the video decoding device to implement the video decoding method described in the second aspect.

[0025] In a fifth aspect, some embodiments of the present application provide a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a computing device, the computing device implements the video encoding method described in the first aspect or the video decoding method described in the second aspect.

[0026] In a sixth aspect, some embodiments of the present application provide a computer program product, which, when executed on a computer, enables the computer to implement the video encoding method described in the first aspect or the video decoding method described in the second aspect.

[0027] It can be seen from the above technical scheme that the video encoding method provided in the embodiment of the present application, when encoding the target image block obtained by block division of the video frame, first performs texture edge detection on the target image block to obtain at least one texture edge of the target image block, and then divides the target image block into two image areas based on the at least one texture edge, and obtains the dividing line of the two image areas, and then performs curve fitting on the dividing line to obtain the segmentation curve of the target image block, and divides the target image block into two sub-image blocks based on the segmentation curve of the target image block, and finally obtains the code stream data corresponding to the target image block according to the segmentation curve of the target image block and the two sub-image blocks. Since the embodiment of the present application can obtain the texture edge of the rectangular image block obtained by block division of the video frame, divide the rectangular image block into two image areas based on the texture edge, perform curve fitting on the boundary line of the two image areas, and divide the rectangular image block into two sub-image blocks according to the segmentation curve obtained by curve fitting, the segmentation curve used to divide the rectangular image block into two sub-image blocks in the embodiment of the present application is obtained by fitting the texture edge of the rectangular image block. Therefore, the embodiment of the present application can make the segmentation line of the block division more closely fit the real texture edge, thereby improving the performance of video encoding. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the implementation methods of some embodiments of the present application or related technologies, the following is a brief introduction to the drawings required for use in the description of the embodiments or related technologies. Obviously, the drawings described below are some embodiments of the present application, and a person skilled in the art can also obtain other drawings based on these drawings.

[0029] Figure 1A block diagram of a video decoding system in some embodiments of the present application is shown;

[0030] Figure 2 A schematic diagram showing the structure of a video encoder in some embodiments of the present application is shown;

[0031] Figure 3 A schematic diagram showing the structure of a video decoder in some embodiments of the present application is shown;

[0032] Figure 4 One of the flow charts of the steps of the video encoding method in some embodiments of the present application is shown;

[0033] Figure 5 The second flowchart of the video encoding method in some embodiments of the present application is shown;

[0034] Figure 6 A schematic diagram showing non-maximum suppression in some embodiments of the present application is shown;

[0035] Figure 7 A schematic diagram showing a set of image regions in some embodiments of the present application;

[0036] Figure 8 A schematic diagram showing a merging scheme in some embodiments of the present application;

[0037] Fig. 9 A schematic diagram showing the dividing lines in some embodiments of the present application;

[0038] Fig.10 A schematic diagram showing a second-order Bezier curve in some embodiments of the present application is shown;

[0039] Fig.11 A schematic diagram showing a third-order Bezier curve in some embodiments of the present application is shown;

[0040] Fig.12 Schematic diagrams of sub-curves in other embodiments of the present application are shown;

[0041] Fig.13 A schematic diagram showing two sub-image blocks in some embodiments of the present application;

[0042] Fig.14 A flowchart showing the steps of a video decoding method in some embodiments of the present application is shown. DETAILED DESCRIPTION

[0043] In order to make the purpose and implementation method of the present application clearer, the exemplary implementation method of the present application will be clearly and completely described below in conjunction with the drawings in the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only part of the embodiments of the present application, rather than all the embodiments.

[0044] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0045] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0046] References to "some implementations", "some embodiments", etc. in the specification indicate that the described implementations or embodiments may include specific features, structures, or characteristics, but not every embodiment may include the specific features, structures, or characteristics. In addition, such phrases do not necessarily refer to the same implementation. In addition, when a specific feature, structure, or characteristic is described in connection with an embodiment, it is considered to be within the knowledge of those skilled in the art to implement such feature, structure, or characteristic in connection with other implementations (whether or not explicitly described herein).

[0047] The embodiments of the present application relate to the field of video coding and decoding technology. The following first describes a video coding and decoding framework for executing the video encoding method and video decoding method provided by the embodiments of the present application.

[0048] Video can be regarded as a sequence of multiple video frames (images). Video playback can be regarded as the display of video frames at a preset speed (for example, 24 frames per second, 30 frames per second, and 60 frames per second) in the order of the sequence. In theory, the amount of video data is positively correlated with the resolution of the video frame. The higher the resolution of the video frame, the larger the amount of video data. If the pixel data of each pixel of all video frames is directly saved in the video file, the amount of video data will be very large, which will make the video difficult to store and transmit. Video encoding and decoding is proposed to solve this problem to a certain extent. Video decoding mainly includes: video encoding and video decoding. Among them, video encoding can be understood as the process of compressing the original video frame, and video decoding can be understood as the process of reconstructing the video frame based on the compressed video data.

[0049] Reference Figure 1 The block diagram of the video decoding system in some embodiments of the present application is shown in FIG. Figure 1As shown, the video decoding system 100 includes: a source device 10 and a destination device 20. The source device 10 can obtain original video data through a video source 101, and encode the original video frame through a video encoder 102 to obtain video encoding data, and provide the video encoder 102 output video encoding data to the destination device 20 through an output interface 103. The destination device 20 can obtain the video encoding data provided by the source device 10 through an input interface 201, and decode the video encoding data through a video decoder 202 to obtain video decoding data, and input the video decoding data into a player 203 to play the video. The source device 10 and the destination device 20 may include any of a wide range of devices, such as: a personal computer (Program Counter), a notebook computer, a tablet computer, a set-top box, a mobile phone, a television, a camera, a display, a digital media player, a video game console, a video streaming device, etc.

[0050] In some embodiments, the video source 101 of the source device 10 may be a video shooting device, such as a camera. In other embodiments, the video source 101 may be a component capable of generating video based on computer graphics, such as a screen recording component, an animation generation component, etc.

[0051] In some embodiments, the destination device 20 may receive the video encoding data provided by the source device 10 via a computer-readable medium. The computer-readable medium may include any type of medium or device capable of moving the video encoding data from the source device 10 to the destination device 20. In one example, the computer-readable medium may include a communication medium. The communication medium may modulate the video encoding data according to a communication standard (e.g., a wireless communication protocol) and transmit it to the destination device 20. The communication medium may include any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet network (e.g., a local area network, a wide area network, or a global network, such as the Internet). The communication medium may include a router, a switch, a base station, or any other device that can be used to facilitate communication from the source device 10 to the destination device 20.

[0052] In some examples, the video encoding data may be output from the output interface 103 of the source device 10 to a storage device. Accordingly, the video encoding data may be accessed from the storage device by the input interface 203 of the destination device 20. The storage device may include any of a variety of distributed or locally accessed data storage media, such as a hard drive, a Blu-ray disc, a DVD, a CD-ROM, a flash memory, a volatile or non-volatile memory, or any other suitable digital storage medium for storing video encoding data. In another example, the storage device may be a server or an intermediate storage device for storing the video encoding data generated by the source device 10. The destination device 20 may obtain the stored video encoding data from the storage device via streaming or downloading. The file server may be any type of server capable of storing encoded video data and transmitting the encoded video data to the destination device 20. In some embodiments. The file server includes a network server (e.g., for a website), an FTP server, a network attached storage device, or a local disk drive. The destination device 20 may access the encoded video data through any standard data connection (including an Internet connection). This may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., DSL, cable modem, etc.), or a combination of both suitable for accessing encoded video data stored on a file server. The transmission of the encoded video data from the storage device may be a streaming transmission, a download transmission, or a combination thereof.

[0053] At present, the video coding standard has gradually evolved from ISO / IECMPEG-1 (International StandardizationOrganization / International Electrotechnical CommissionMoving Picture ExpertsGroup-1), through ISO / IECMPEG-2, ISO / IECMPEG-4, Advanced Video Coding (AVC), High Efficiency Video Coding (HEVC), etc. to Versatile Video Coding (VVC). The video encoding method and video decoding method provided in the embodiments of the present application can be applied to any suitable video decoding standard. For example: HEVC standard, VVC standard, etc.

[0054] Reference Figure 2 As shown, Figure 2Schematic diagram of the structure of a video encoder in some embodiments of the present application. The video encoder 200 provided in the embodiment of the present application includes: a segmentation module 21, a motion estimation module 22, a motion compensation module 23, a residual calculation module 24, a quantization transformation module 25 and an entropy coding module 26. The process of video encoding by the video encoder 200 includes:

[0055] First, the video frame to be encoded is divided into a plurality of rectangular image blocks by the segmentation module 21. The segmentation module 21 can decide whether to further segment the rectangular image blocks according to the actual encoding situation. Specifically, since selecting a larger encoding block is more conducive to improving the encoding efficiency, for rectangular image blocks with relatively flat pixels, fewer segmentations can be selected, and the segmentation depth is shallower. On the contrary, for rectangular image blocks with more complex textures or located at the edge of the image, more segmentation times can be selected, and the segmentation depth is deeper. The segmentation module 21 will also segment at least one rectangular image block into two irregular sub-image blocks based on a segmentation curve that fits the real texture edge. Then, the motion estimation module 22 performs motion estimation (ME) on each coding block (including: rectangular image blocks and irregular sub-image blocks that are not further divided) to obtain a reference block (RB) corresponding to each coding block, and then the motion compensation module 23 performs motion compensation (MC) on the reference block to obtain a prediction block and a motion vector (MV) corresponding to each coding block, and the residual calculation module 24 calculates the residual between the prediction block and the corresponding coding block to obtain a prediction residual, and finally the quantization transformation module 25 transforms and quantizes the prediction residual to obtain a transformation coefficient, and the entropy coding module 26 performs entropy coding on the segmentation curve, motion vector, transformation coefficient and other information to obtain the bit stream data of the video frame to be encoded.

[0056] It should be noted that the above encoding process only describes some steps in the video encoding process. In addition to the above steps, the video encoding process also includes other steps. For example, inputting information such as the prediction mode and the transformation mode into the entropy encoding module 26, and adding the entropy encoding result of the prediction mode, the transformation mode and the like information to the encoded data of the video frame to be encoded.

[0057] Reference Figure 3 As shown, Figure 3 Schematic diagram of the structure of the video decoder in some embodiments of the present application. The video encoder 300 provided in the embodiment of the present application includes: an entropy decoding module 31, a motion estimation module 32, a motion compensation module 33, an inverse quantization transformation module 34, a fusion module 35 and a reconstruction module 36. The process of video decoding by the video encoder 300 includes:

[0058] First, the code stream data of the video frame to be encoded is entropy decoded by the entropy decoding module 31 to obtain at least one of the information such as the segmentation curve for segmenting the rectangular image block of the video frame to be reconstructed into two irregular sub-image blocks, the motion vectors of the two sub-image blocks, the transformation coefficients of the two sub-image blocks, the motion vectors of the rectangular image blocks, and the transformation coefficients of the rectangular image blocks. Then, on the one hand, the motion estimation module 32 determines the reference block corresponding to each block to be reconstructed (including: rectangular image blocks and irregular sub-image blocks that are not further segmented), and the motion compensation module 33 performs motion compensation on the reference block according to the motion vector to obtain the prediction block corresponding to the block to be reconstructed; on the other hand, the inverse quantization transformation module 34 performs inverse quantization and inverse transformation operations on the transformation coefficient to obtain the prediction residual, and the fusion module 35 adds and fuses the prediction residual with the prediction block to obtain the reconstruction data of the block to be reconstructed, and finally the reconstruction module 36 reconstructs each block to be reconstructed according to the reconstruction data of the block to be reconstructed to obtain the reconstructed video frame.

[0059] The present application embodiment provides a video encoding method, referring to Figure 4 As shown, the video encoding method includes the following steps S41 to S46:

[0060] S41, obtaining a target image block.

[0061] The target image block is a rectangular image block obtained by dividing the video frame into blocks.

[0062] Under video coding standards such as HEVC and VVC, after receiving the video frame to be encoded, the encoder will first divide the video frame to be encoded into multiple image blocks, and encode the image blocks obtained by dividing the video frame to be encoded as the minimum coding unit. In addition, the encoding task for the image block can be further refined. For example: In the HEVC standard, the image block obtained by dividing the video frame to be encoded is called a coding tree unit (Coding Tree Unit, CTU), and the coding tree unit can be further divided into smaller square coding units (Coding unit, CU) by quadtree division. The maximum size of the coding tree unit can be supported to 64×64 and the minimum size can be supported to 16×16. In order to meet the needs of ultra-high-definition video encoding such as 4K and 8K, the VVC standard expands the maximum size of CTU to 128×128, and further division methods are no longer limited to quadtree division, but also support binary tree and ternary tree division. The target image block in the embodiment of the present application can be a coding tree unit, or it can be a coding unit obtained by further dividing the coding tree unit, and the embodiment of the present application does not limit this.

[0063] S42: Perform texture edge detection on the target image block to obtain at least one texture edge of the target image block.

[0064] In the embodiment of the present application, the texture edge of the target image block refers to an image region in the target image block where the brightness changes significantly.

[0065] In the embodiment of the present application, the edge detection algorithm used for texture edge detection of the target image block is not limited, and the edge detection algorithm used is subject to the ability to obtain the texture edge of the target image block. Exemplarily, in the embodiment of the present application, the texture edge detection of the target image block can be performed using edge detection algorithms such as Sobel, Prewitt, Roberts, Canny, and Marr-Hildreth.

[0066] S43: Segment the target image block into two image areas based on the at least one texture edge, and obtain a boundary line between the two image areas.

[0067] It should be noted that in some cases, the brightness of the pixels in the image block is relatively flat, and texture edge detection on the image block cannot obtain a texture edge that can divide the image block into two image regions. For such image blocks, the embodiment of the present application may not further divide the sub-image blocks, but directly encode the image block as a coding unit. In other cases, the texture edge obtained by texture edge detection on the image block will divide the image block into image regions greater than 2. For such image blocks, multiple image regions can be first merged into two image regions, and then the subsequent video encoding steps can be performed.

[0068] S44: Perform curve fitting on the boundary line to obtain a segmentation curve of the target image block.

[0069] Generally, the boundary line between two image areas has no function expression or the function expression is very complex. Directly representing the boundary line between the two image areas in the bitstream data will greatly increase the overhead of the bitstream data. Therefore, the embodiment of the present application performs curve fitting on the boundary line between the two image areas to simplify the function expression of the boundary line, thereby reducing the overhead caused by the boundary line.

[0070] In some embodiments, the above step S44 (performing curve fitting on the dividing line to obtain the segmentation curve of the target image block) includes: performing Bezier curve fitting on the dividing line, and determining the curve obtained by performing Bezier curve fitting on the dividing line as the segmentation curve of the target image block.

[0071] S45 . Divide the target image block into two sub-image blocks based on the segmentation curve of the target image block.

[0072] It should be noted that the segmentation curve is a continuous curve. The continuous segmentation curve may pass through the smallest unit (pixel point) of the digital image, but the smallest unit of the digital image cannot be segmented. Therefore, the segmentation curve cannot be directly applied to the segmentation of the digital image. Therefore, when the target image block is segmented into two sub-image blocks based on the segmentation curve of the target image block, it is necessary to sample the segmentation curve into a discrete curve, and then segment the target image block into two sub-image blocks through the sampled curve.

[0073] S46: Acquire code stream data corresponding to the target image block according to the segmentation curve of the target image block and the two sub-image blocks.

[0074] The video encoding method provided by the embodiment of the present application, when encoding a target image block obtained by block division of a video frame, first performs texture edge detection on the target image block to obtain at least one texture edge of the target image block, then divides the target image block into two image areas based on the at least one texture edge, and obtains a boundary line between the two image areas, then performs curve fitting on the boundary line to obtain a segmentation curve of the target image block, and divides the target image block into two sub-image blocks based on the segmentation curve of the target image block, and finally obtains code stream data corresponding to the target image block according to the segmentation curve of the target image block and the two sub-image blocks. Since the embodiment of the present application can obtain the texture edge of the rectangular image block obtained by block division of the video frame, divide the rectangular image block into two image areas based on the texture edge, perform curve fitting on the boundary line of the two image areas, and divide the rectangular image block into two sub-image blocks according to the segmentation curve obtained by curve fitting, the segmentation curve used to divide the rectangular image block into two sub-image blocks in the embodiment of the present application is obtained by fitting the texture edge of the rectangular image block. Therefore, the embodiment of the present application can make the segmentation line of the block division more closely fit the real texture edge, thereby improving the performance of video encoding.

[0075] As an extension and refinement of the above embodiment, the present application embodiment provides another video encoding method, referring to Figure 5 As shown, the video encoding method includes the following steps S501 to S514:

[0076] S501: Obtain a target image block.

[0077] The target image block is a rectangular image block obtained by dividing the video frame into blocks.

[0078] S502: Convert the target image block into a grayscale image to obtain a grayscale image block corresponding to the target image block.

[0079] In some embodiments, the color space of the target image block is a YUV color space, and the pixel value of each pixel of the target image block is composed of a brightness signal Y representing a brightness value and two chrominance signals U and V representing color values. The implementation method of converting the target image block into a grayscale image may include: deleting the chrominance signals U and V in the pixel value of each pixel and retaining only the brightness signal Y, thereby converting the target image block into a grayscale image.

[0080] In some embodiments, the color space of the target image block is an RGB color space, and the implementation method of converting the target image block into a grayscale image may include: converting the color space of the target image block from the RGB color space to the YUV color space, and then deleting the chrominance signals U and V in the pixel value of each pixel point, and only retaining the brightness signal Y, thereby converting the target image block into a grayscale image.

[0081] Since the human eye is much more sensitive to brightness than chromaticity, brightness is often used as important information in image processing. Therefore, the embodiment of the present application first converts the target image block into a grayscale image and filters out the chromaticity signal in the target image block, thereby reducing the amount of data processing in the subsequent edge detection process.

[0082] S503 : Perform filtering processing on the grayscale image block based on a Gaussian kernel of a preset size.

[0083] In some embodiments, the Gaussian kernel of the preset size may be a 5*5 Gaussian kernel.

[0084] In some embodiments, the Gaussian kernel G σ The definition of can be shown as follows:

[0085]

[0086] Where σ is the standard deviation, x and y are the distance values ​​from the center of the Gaussian kernel in the X-axis direction and the Y-axis direction respectively.

[0087] The grayscale image block is represented as I, and the grayscale image block after filtering is represented as I σ , the Gaussian kernel of the preset size is represented by G σ , then the above step S503 can be expressed as the following formula (2):

[0088]

[0089] in, Represents a convolution operation.

[0090] It should be noted that in the embodiment of the present application, two-dimensional Gaussian kernels of other sizes may be selected to filter the grayscale image block, for example: a 7*7 Gaussian kernel is selected to filter the grayscale image block, and another example: a 3*3 Gaussian kernel is selected to filter the grayscale image block. The embodiment of the present application does not limit the size of the Gaussian kernel, and in actual use, the Gaussian kernel can be selected according to the resolution of the grayscale image block. That is, when the resolution of the grayscale image block is large, a larger Gaussian kernel can be selected, and when the resolution of the grayscale image block is small, a smaller Gaussian kernel can be selected.

[0091] Since the embodiment of the present application also performs filtering processing on the target image block through the above-mentioned step S503, the embodiment of the present application can filter out the noise in the target image block, thereby improving the accuracy of subsequent texture edge detection.

[0092] S504: Perform texture edge detection on the grayscale image block to obtain at least one texture edge of the target image block.

[0093] In some embodiments, the above step S504 (performing texture edge detection on the grayscale image block to obtain at least one texture edge of the target image block) includes the following steps a to c:

[0094] Step a: Calculate the gradient value and gradient direction of the grayscale image block after filtering.

[0095] In the embodiment of the present application, the gradient value and gradient direction of the grayscale image block refer to the global gradient value and global gradient direction of the grayscale image block. The global gradient value includes the gradient value of each pixel in the grayscale image block, and the global gradient direction includes the gradient direction of each pixel in the grayscale image block.

[0096] Exemplarily, calculating the gradient value and gradient direction of the grayscale image block after filtering may include: calculating the gradient value and gradient direction of the grayscale image block after filtering based on a Sobel operator, a Roberts operator, a Prewitt operator, or a Laplacian operator.

[0097] Exemplarily, the above step a is described below by taking the calculation of the gradient value and gradient direction of the grayscale image block after filtering based on the Sobel operator as an example.

[0098] The templates of the Sobel operator in the X-axis direction and the Y-axis direction are shown in the following formulas (3) and (4) respectively:

[0099]

[0100]

[0101] If the gradient values ​​of the grayscale image block after filtering in the X-axis direction and the Y-axis direction are represented as G x and G y , then the calculation formulas for the gradient values ​​of the grayscale image block in the X-axis direction and the Y-axis direction after filtering can be respectively as follows:

[0102] G x =I σ *S x (5)

[0103] G y =I σ *S y (6)

[0104] The gradient value and gradient direction of the grayscale image block after filtering are calculated according to the gradient values ​​of the grayscale image block after filtering in the X-axis direction and the Y-axis direction and the following formulas (7) and (8):

[0105]

[0106]

[0107] Wherein, G is the gradient value of the grayscale image block after filtering, and θ is the gradient direction of the grayscale image block after filtering.

[0108] Step b: performing non-maximum suppression on the gradient value of the grayscale image block after filtering according to the gradient value and gradient direction of the grayscale image block after filtering.

[0109] Although the global gradient value of the grayscale image block after filtering is obtained through the above step a, the texture edge directly obtained based on the gradient value is too rough, so it is necessary to perform non-maximum suppression on the gradient value to refine the edge.

[0110] In some embodiments, the above step b (performing non-maximum suppression on the gradient value of the grayscale image block after filtering according to the gradient value and gradient direction of the grayscale image block after filtering) includes: traversing the pixel points of the grayscale image block after filtering, and determining whether the gradient value of the current pixel point is the maximum value among the surrounding pixel points with similar gradient direction; if so, retaining the current pixel point as a candidate edge point, if not, deleting the current pixel point from the candidate edge points.

[0111] In some embodiments, the implementation method for determining whether the gradient value of the current pixel is the maximum value of the surrounding pixels with similar gradient directions may be: discretizing the gradient direction θ into eight directions of 0°, 45°, 90°, 135°, 180°, 225°, 270°, and 315°, traversing each pixel belonging to the texture edge, obtaining the gradient value of the pixel closest to the corresponding first direction and the gradient value of the pixel closest to the corresponding second direction, if the gradient value of the current pixel is greater than the gradient value of the pixel in the first direction, and the gradient value of the current pixel is greater than the gradient value of the pixel in the second direction, then retain the current pixel as a pixel belonging to the texture edge, otherwise suppress the current pixel and remove it from the texture edge. Among them, the first direction corresponding to the pixel is the direction with the smallest angle with the gradient direction of the pixel among the eight directions, and the second direction corresponding to the pixel is the opposite direction of the first direction corresponding to the pixel.

[0112] For example, refer to Figure 6 As shown in the figure, the gradient direction of pixel a is θ, and the direction with the smallest angle with the gradient direction θ of pixel a among the eight directions obtained by discrete gradient direction is the 45° direction. The pixel closest to pixel a in the 45° direction is pixel b. The opposite direction of the 45° direction is the 225° direction. The pixel closest to pixel a in the 225° direction is pixel c. Therefore, the gradient value G of pixel b is obtained. b and the gradient value G of pixel c c , if the gradient value G of pixel a a Greater than the gradient value G of pixel b b , and the gradient value G of pixel a a Greater than the gradient value G of pixel c c , then pixel a is retained as a pixel belonging to the texture edge, and if the gradient value G of pixel a a Less than the gradient value G of pixel b b and / or the gradient value G of pixel a a Less than the gradient value G of pixel c c , then the pixel a is suppressed and removed from the texture edge.

[0113] Step c: determining the texture edge of the target image block based on the first threshold and the second threshold.

[0114] The first threshold is smaller than the second threshold.

[0115] The above step c (determining the texture edge of the target image block based on the first threshold and the second threshold) includes: determining that the pixel point with a gradient value greater than the second threshold is a strong edge point, and the strong edge point belongs to the texture edge, determining that the pixel point with a gradient value less than the first threshold does not belong to the texture edge, determining that the pixel point with a gradient value between the second threshold and the first threshold belongs to a weak edge point, and further determining whether the weak edge point belongs to the texture edge based on whether the weak edge point is connected to the strong edge. Exemplarily, along the contour line of the strong edge until the end point s(x,y) of the strong edge, it can be determined whether there is a weak edge point in the 8-neighborhood of the end point s(x,y), if so, the weak edge point is classified as a texture edge, recorded as r(x,y), and the above operation is repeated with r(x,y) as the starting point, so as to obtain the texture edge of the current image block.

[0116] S505: Perform image region segmentation on the target image block based on the at least one texture edge to obtain an image region set corresponding to the target image block.

[0117] For example, refer to Figure 7 As shown, the texture edge of the target image block includes: texture edge 71 and texture edge 72. Therefore, the image area set corresponding to the target image block obtained based on the texture edge of the target image block includes a total of 3 image areas, and the 3 image areas are image area A, image area B and image area C.

[0118] S506: Obtain the number of image regions in the image region set.

[0119] That is, the number of image regions obtained by dividing the image region of the target image block by the texture edge of the target image block (the at least one texture edge) is determined.

[0120] As above Figure 7 In the example shown, the target image block is likely to have more than one texture edge, which means that the target image block will be divided into more than 2 image areas. Taking into account the bit rate cost, too many sub-image areas cannot bring matching performance improvements. Therefore, when the number of image areas is greater than 2, the embodiment of the present application will first merge the image areas into two image areas. If the number of image areas is equal to 2, no merging process is required. That is, if the number of image areas in the image area set is less than 2, the target image block is directly encoded as a whole. If the number of image areas in the image area set is equal to 2, jump to step S508. If the number of image areas in the image area set is greater than 2, execute the following step S507:

[0121] S507: Merge the image regions in the image region set into two image regions, so as to divide the target image block into two image regions.

[0122] In some embodiments, the above step S507 (merging the image regions in the image region set into two image regions to divide the target image block into two image regions) includes the following steps 1 to 3:

[0123] Step 1: Obtain at least two merging schemes for the image region set.

[0124] The at least two merging schemes include various schemes for merging the image regions in the image region set into two image regions.

[0125] When obtaining the merging solution of the image region set, it may be based on the connectivity of each image region in the image region set. If two image regions have a common boundary, the two image regions may be merged into one image region.

[0126] For example, refer to Figure 8 As shown, the image region set includes: image region A, image region B and image region C, so there are 3 merging schemes. Merging scheme 1 is: merging image region A and image region B to obtain image region D, thereby merging the image regions in the image region set into two image regions, image region D and image region C; merging scheme 2 is: merging image region A and image region C to obtain image region E, thereby merging the image regions in the image region set into two image regions, image region E and image region B; merging scheme 3 is: merging image region B and image region C to obtain image region F, thereby merging the image regions in the image region set into two image regions, image region F and image region A.

[0127] Step 2: respectively obtain the difference between the at least two merging schemes.

[0128] Among them, the difference of any merging scheme is the absolute difference between the first similarity and the second similarity of the merging scheme; the first similarity and the second similarity of any merging scheme are respectively the sum of the absolute differences in the grayscale values ​​of each pixel point of the two image areas and the corresponding co-located blocks under the merging scheme.

[0129] That is, the difference of the merging scheme can be obtained through the following steps 21 to 23:

[0130] Step 21: respectively obtain the same-position blocks corresponding to the two image regions under the merging scheme.

[0131] Continuing with the above example, when obtaining the difference of merging scheme 1, the same-position blocks of image area D and image area C are obtained respectively; when obtaining the difference of merging scheme 2, the same-position blocks of image area E and image area B are obtained respectively; when obtaining the difference of merging scheme 3, the same-position blocks of image area F and image area A are obtained respectively.

[0132] There is a strong temporal correlation between video frames that are close in time domain, so the adjacent video coded frames can be used to estimate the current video frame, and the adjacent coded video frames of the current video frame are different in different coding structures. Coding structures include: All Intra (AI), Low Delay (LD), Low Latency (LP), and Random Access (RA). The co-located block corresponding to any image area refers to the image area in the adjacent coded video frame of the current video frame that has the same position as the image area.

[0133] Step 22: respectively calculate the sum of the absolute differences of the grayscale values ​​of each pixel of the two image regions under the merging scheme and the corresponding co-located blocks to obtain the first similarity and the second similarity of the merging scheme.

[0134] The first similarity of the merge scheme j is expressed as Δ j1 , then the first similarity Δ of the merge scheme j can be calculated by the following formula (9): j1 :

[0135]

[0136] Wherein, n1 is the number of pixels in the first image region under the merging scheme j, and n1 is also the number of pixels in the same-position block corresponding to the first image region under the merging scheme j; is the brightness value of pixel i in the first image area; is the brightness value of pixel i in the same-position block corresponding to the first image area.

[0137] The second similarity of the merge scheme j is expressed as Δ j2 , the second similarity Δ of the merge scheme j can be calculated by the following formula (10): j2 :

[0138]

[0139] Wherein, n2 is the number of pixels in the second image area under the merging scheme j, and n2 is also the number of pixels in the same-position block corresponding to the second image area under the merging scheme j; is the brightness value of pixel i in the second image area; is the brightness value of pixel i in the same-position block corresponding to the second image area.

[0140] Step 23: Calculate the absolute difference between the first similarity and the second similarity of the merged solution to obtain the difference of the merged solution.

[0141] The difference of the merge scheme j is expressed as D j , then the difference D of the merge scheme j can be calculated by the following formula (11): j :

[0142] D j =|Δ j1 -Δ j2 | (11)

[0143] Step 3: Merge the image regions in the image region set into two image regions by using a merging scheme with the largest difference among the at least two merging schemes.

[0144] As shown in the above example, the differences of merging scheme 1, merging scheme 2, and merging scheme 3 are D1, D2, and D3 respectively. If D1>D2 and D1>D3, the image areas in the image area set are merged into two image areas, image area D and image area C, through merging scheme 1. If D2>D1 and D2>D3, the image areas in the image area set are merged into two image areas, image area B and image area E, through merging scheme 2. If D3>D1 and D3>D2, the image areas in the image area set are merged into two image areas, image area A and image area F, through merging scheme 3.

[0145] Because each merging scheme merges the image regions in the image region set into two image regions, the difference D of the merging scheme j The larger the value, the greater the difference in grayscale values ​​of the two image partitions obtained by the merging scheme, and the greater the possibility that the two image partitions obtained by the merging scheme belong to different image content objects. Therefore, by merging the image areas in the image area set into two image areas through the merging scheme with the largest difference among the at least two merging schemes, the target image block can be divided into two image areas more reasonably.

[0146] It should be noted that the purpose of step S507 is to find and construct an optimal edge line among multiple edges. In the above implementation, each image area of ​​the target image block is merged to achieve this purpose, but the embodiment of the present application is not limited to this. Other solutions that can solve the problem of selecting multiple edge lines can also be used as variant solutions, such as: using a path-finding algorithm to find the longest path. For example, the Astar image search algorithm can be used to find the optimal dividing line to obtain a discrete curve whose starting point and end point are both on the boundaries of the coding block.

[0147] S508: Obtain a boundary line between the two image regions.

[0148] like Fig. 9As shown, the target image block is divided into two image regions, namely, image region D and image region E, and thus a boundary line 90 between image region D and image region E can be obtained.

[0149] S509: Perform Bezier curve fitting on the boundary line, and determine a curve obtained by performing Bezier curve fitting on the boundary line as a segmentation curve of the target image block.

[0150] In some embodiments, the above step S509 (performing Bezier curve fitting on the boundary line, and determining the curve obtained by performing Bezier curve fitting on the boundary line as the segmentation curve of the target image block) includes the following steps ① to ④:

[0151] Step ①, performing second-order Bezier curve fitting on the boundary line to obtain a first fitting result.

[0152] The second-order Bezier curve is a curve defined by a starting point s, an end point e and a control point p. Therefore, the second-order Bezier curve fitting is performed on the dividing line, that is, the starting point of the dividing line is used as the starting point s of the second-order Bezier curve, the end point of the dividing line is used as the end point e of the second-order Bezier curve, and the control point p is obtained.

[0153] The formula of the second-order Bezier curve is shown in the following formula (12):

[0154] B(t)=(1-t) 2 ·s+2·(1-t)·t·p+t 2 ·e (12)

[0155] Among them, t∈[0,1] is the endpoint interpolation variable of the Bezier curve function.

[0156] The dividing line is fitted with a second-order Bezier curve, that is, the coordinates of a set of points on the dividing line (x0, y0),…, (x m ,y m ) is substituted into the following formula (13), and the coordinates of the control point p are solved according to the coefficients of the Bezier curve obtained when the value of formula (13) is minimum:

[0157]

[0158] Among them, B ix and B iy is the x-coordinate and y-coordinate of the i-th point after fitting, X i and Y i are the x- and y-coordinates of the ith point on the dividing line.

[0159] For example, refer to Fig.10 As shown, Fig.10 The dividing line is Fig.10 The line 101 in the figure is fitted with a second-order Bezier curve to obtain a first fitting result: Fig.10 The line 102 in FIG. Fig.10 As shown, the starting point of the dividing line and the first fitting result obtained by fitting the dividing line with a second-order Bezier curve are both point s, the end point of the dividing line and the first fitting result obtained by fitting the dividing line with a second-order Bezier curve are both point e, the first fitting result is a single arch curve, and the shape of the first fitting result is controlled by the control point p.

[0160] Step ②, performing third-order Bezier curve fitting on the boundary line to obtain a second fitting result.

[0161] The third-order Bezier curve is a curve defined by a starting point s, an end point e, a first control point p1, and a second control point p2. Therefore, the third-order Bezier curve fitting is performed on the dividing line, that is, the starting point of the dividing line is used as the starting point s of the third-order Bezier curve, and the end point of the dividing line is used as the end point e of the third-order Bezier curve to obtain the first control point p1 and the second control point p2.

[0162] The formula of the third-order Bezier curve is shown in the following formula (14):

[0163] B(t)=(1-t) 3 ·s+3·(1-t) 2 ·t·p1+3·(1-t)·t 2 p2+t 3 ·e (14)

[0164] Among them, t∈[0,1] is the endpoint interpolation variable of the Bezier curve function.

[0165] The boundary line is fitted with a third-order Bezier curve, that is, the coordinates of a set of points on the boundary line (x0, y0),…, (x m ,y m ) is substituted into the following formula (15), and the coordinates of the first control point p1 and the second control point p2 are solved according to the coefficients of the Bezier curve obtained when the value of formula (15) is minimum:

[0166]

[0167] Where, t1 = (1-t) 3 , t2=3(1-t) 2 t, t3=3(1-t)t 2 , t4=t 3 ,s x is the x coordinate of the starting point of the dividing line, P 1x is the x coordinate of the first control point, P2x is the x coordinate of the second control point, e x is the x-coordinate of the end point of the dividing line, s y is the y coordinate of the starting point of the dividing line, P 1y is the y coordinate of the first control point, P 2y is the y coordinate of the second control point, e y is the y coordinate of the end point of the dividing line, X i and Y i are the x-coordinate and y-coordinate of the i-th point on the dividing line, the t corresponding to the i-th point is (i-1) / (N-1), and N is the number of points extracted from the dividing line.

[0168] For P in the above formula (15), 1x , P 1y , P 2x , P 21 Derivatives are taken separately, and the partial derivatives are set to 0. The following matrix can be obtained after sorting:

[0169]

[0170] Solving the above matrix, we can obtain P 1x , P 1y , P 2x , P 2y The value of .

[0171] For example, refer to Fig.11 As shown, Fig.11 The dividing line is Fig.11 As shown in line 111 in FIG. 1 , a third-order Bezier curve is fitted to the boundary line to obtain a second fitting result as shown in FIG. Fig.11 The line 112 in FIG. Fig.11 As shown, the starting points of the dividing line and the second fitting result obtained by fitting the dividing line with a third-order Bezier curve are both point s, the end points of the dividing line and the first fitting result obtained by fitting the dividing line with a second-order Bezier curve are both point e, and the shape of the first fitting result is controlled by the first control point p1 and the second control point p2.

[0172] Step ③, respectively obtain a first fitting accuracy of the first fitting result and a second fitting accuracy of the second fitting result.

[0173] The fitting accuracy is a physical quantity that can characterize the error between the fitting result and the original dividing line. It should be noted that, since the third-order Bezier curve is compatible with the fitting result of the second-order Bezier curve, the second fitting accuracy of the second fitting result obtained by fitting the third-order Bezier curve to the dividing line must be greater than or equal to the first fitting accuracy of the first fitting result obtained by fitting the second-order Bezier curve to the dividing line.

[0174] Step ④: According to the first fitting accuracy and the second fitting accuracy, select the first fitting result or the second fitting result as the segmentation curve of the target image block.

[0175] As described above, the second fitting accuracy is greater than or equal to the first fitting accuracy, and the bitstream overhead of the third-order Bezier curve is greater than the bitstream overhead of the second-order Bezier curve. Therefore, the first fitting result or the second fitting result can be selected as the segmentation curve of the target image block based on the bitstream overhead brought by the third-order Bezier curve and the fitting accuracy improved by the third-order Bezier curve. For example: the difference between the second fitting accuracy and the first fitting accuracy is calculated, and it is determined whether the difference between the second fitting accuracy and the first fitting accuracy is greater than a preset threshold. If the difference between the second fitting accuracy and the first fitting accuracy is greater than the preset threshold, the second fitting result is selected as the segmentation curve of the target image block. If the difference between the second fitting accuracy and the first fitting accuracy is less than the preset threshold, the first fitting result is selected as the segmentation curve of the target image block. Among them, the preset threshold can be set according to the difference between the bitstream overhead brought by the third-order Bezier curve and the bitstream overhead brought by the second-order Bezier curve.

[0176] It should be noted that, in the above-mentioned embodiment, the second-order Bezier curve fitting and the third-order Bezier curve fitting of the dividing line are used as examples for explanation, but the embodiments of the present application are not limited to this. The dividing line can also be fitted by a higher-order Bezier curve to improve the fitting accuracy. However, the amount of calculation brought about by fitting the dividing line by a higher-order Bezier curve is also greater. Therefore, in actual adaptation, the image quality improvement brought about by the amount of calculation and the fitting accuracy can be balanced to select a Bezier curve of a suitable order to fit the dividing line.

[0177] In addition, in the above-mentioned embodiments, the coordinate values ​​of the control points of the Bezier curve are solved by means of a matrix, but in other embodiments, the coordinate values ​​of the control points of the Bezier curve can also be solved by other methods, for example: the coordinate values ​​of the control points of the Bezier curve can also be solved by fitting method, numerical integration method, derivative solution method, etc. The embodiments of the present application do not limit this, and the coordinate values ​​of the control points of the Bezier curve can be solved.

[0178] S510 , sampling the segmentation curve of the target image block with integer pixel accuracy to obtain a sampling result of the segmentation curve.

[0179] Since the Bezier curve (segmentation curve) generated in the above step S509 is a continuous curve and cannot be directly applied to the segmentation of digital images, the sampling result of the discrete segmentation curve is obtained by sampling with integer pixel accuracy, and then the target image block is segmented.

[0180] In some embodiments, sampling the segmentation curve of the target image block with integer pixel accuracy may include: determining a boundary of pixels through which the segmentation curve passes as a sampling result of the segmentation curve.

[0181] For example, refer to Fig.12 As shown, the segmentation curve is Fig.12 The sampling result of the segmentation curve obtained by sampling the segmentation curve of the target image block with integer pixel accuracy can be as follows: Fig.12 As shown by line 122 in FIG. Fig.12 As shown, the sampling results of the segmentation curve are all located on the boundary of the pixel.

[0182] S511 . Divide the target image block into two sub-image blocks according to the sampling result of the segmentation curve.

[0183] For example, Fig.13 As shown, in Fig.12 Based on the sampling result 122 of the segmentation curve, the target image block can be segmented into two sub-image blocks, a sub-image block 131 and a sub-image block 132 .

[0184] S512: Encode the information of the segmentation curve of the target image block to obtain first code stream data.

[0185] In some embodiments, the segmentation curve is a second-order Bezier curve obtained by fitting the dividing line with a second-order Bezier curve, and encoding the information of the segmentation curve of the target image block to obtain the first code stream data includes: encoding the position information of the starting point, the position information of the end point, and the position information of the control point of the second-order Bezier curve to obtain the first code stream data.

[0186] In some embodiments, the segmentation curve is a third-order Bezier curve obtained by fitting the dividing line with a third-order Bezier curve, and encoding the information of the segmentation curve of the target image block to obtain the first code stream data includes: encoding the position information of the starting point, the position information of the end point, the position information of the first control point, and the position information of the second control point of the third-order Bezier curve to obtain the first code stream data.

[0187] S513: Encode the two sub-image blocks respectively to obtain second code stream data and third code stream data.

[0188] In some embodiments, encoding a sub-image block to obtain code stream data of the sub-image block may include: first, performing motion estimation on the sub-image block to obtain a reference block corresponding to the sub-image block, then performing motion compensation on the reference block to obtain a prediction block and motion vector corresponding to each coding block, and calculating the residual between the prediction block and the sub-image block to obtain a prediction residual, and finally transforming and quantizing the prediction residual to obtain a transform coefficient, and entropy encoding the motion vector and transform coefficient to obtain the code stream data of the sub-image block.

[0189] S514: Generate code stream data corresponding to the target image block according to the first code stream data, the second code stream data, and the third code stream data.

[0190] In some embodiments, generating the code stream data corresponding to the target image block according to the first code stream data, the second code stream data and the third code stream data includes: encapsulating the first code stream data, the second code stream data and the third code stream data to obtain the code stream data corresponding to the target image block.

[0191] The present application also provides a video decoding method, referring to Fig.14 As shown, the video decoding method includes the following steps S141 to S145:

[0192] S141. Obtain code stream data corresponding to the target image block.

[0193] The target image block is a rectangular image block obtained by dividing a video frame into blocks; the bitstream data corresponding to the target image block includes: first bitstream data, second bitstream data and third bitstream data; the first bitstream data is bitstream data obtained by encoding information of a segmentation curve used to divide the target image block into two sub-image blocks; the second bitstream data and the third bitstream data are bitstream data obtained by encoding the two sub-image blocks respectively.

[0194] In some embodiments, obtaining code stream data corresponding to the target image block includes: receiving video stream data of a to-be-played video sent by a media resource server, and extracting code stream data corresponding to the target image block from the video stream data.

[0195] S142: Acquire the segmentation curve according to the first bitstream data.

[0196] In some embodiments, the first bitstream data includes: position information of the starting point, the terminal point, and the control point of the second-order Bezier curve, and obtaining the segmentation curve according to the first bitstream data includes: reconstructing the second-order Bezier curve according to the position information of the starting point, the terminal point, and the control point of the second-order Bezier curve to obtain the segmentation curve.

[0197] In some embodiments, the first bitstream data includes: position information of the starting point of the third-order Bezier curve, position information of the terminal point, position information of the first control point, and position information of the second control point. The acquiring the segmentation curve according to the first bitstream data includes: reconstructing the third-order Bezier curve according to the position information of the starting point of the third-order Bezier curve, position information of the terminal point, position information of the first control point, and position information of the second control point to acquire the segmentation curve.

[0198] S143: Segment the target image block based on the segmentation curve to obtain the two sub-image blocks.

[0199] In some embodiments, the above step S143 (segmenting the target image block based on the segmentation curve to obtain the two sub-image blocks) includes the following steps 1431 and 1432:

[0200] Step 1431: Sample the segmentation curve of the target image block with integer pixel accuracy to obtain a sampling result of the segmentation curve.

[0201] Step 1432: divide the target image block into two sub-image blocks according to the sampling result of the segmentation curve.

[0202] The implementation of step 1431 and step 1432 may refer to the implementation of steps S510 and S511 in the above embodiment, and will not be repeated here to avoid redundancy.

[0203] It should be noted that what is obtained in step S143 is only the shape, size and position of the two sub-image blocks, but does not include the image content of the two sub-image blocks. The image content of the two sub-image blocks needs to be reconstructed through subsequent steps.

[0204] S144: Reconstruct the two sub-image blocks according to the second code stream data and the third code stream data.

[0205] In some embodiments, the second code stream data and the third code stream data respectively include prediction residuals and motion vectors of the two sub-image blocks, and reconstructing the two sub-image blocks according to the second code stream data and the third code stream data includes: respectively determining reference blocks corresponding to the two sub-image blocks, respectively performing motion compensation on the reference blocks of the two sub-image blocks according to the motion vectors of the two sub-image blocks, so as to respectively obtain prediction blocks corresponding to the to-be-reconstructed blocks of the two sub-image blocks; and respectively adding and fusing the prediction residuals of the two sub-image blocks with the prediction blocks, so as to obtain the reconstructed two sub-image blocks.

[0206] S145. Obtain the reconstructed target image block according to the two reconstructed sub-image blocks.

[0207] That is, the two reconstructed sub-image blocks are spliced ​​into the reconstructed target image block.

[0208] After obtaining the code stream data corresponding to the target image block, the video decoding method provided in the embodiment of the present application includes the first code stream data obtained by encoding the information of the segmentation curve used to segment the target image block into two sub-image blocks, so the embodiment of the present application can obtain the segmentation curve according to the first code stream data, and segment the target image block based on the segmentation curve to obtain the two sub-image blocks, and because the code stream data corresponding to the target image block also includes the second code stream data and the third code stream data respectively obtained by encoding the two sub-image blocks, so the embodiment of the present application can also reconstruct the two sub-image blocks according to the second code stream data and the third code stream data, and obtain the reconstructed target image block according to the reconstructed two sub-image blocks. Therefore, the embodiment of the present application can realize normal decoding of the video while improving the performance of video encoding.

[0209] In some embodiments, some embodiments of the present application provide a video encoding device, the video encoding device comprising:

[0210] a memory configured to store a computer program;

[0211] The processor is configured to enable the video encoding device to implement the video encoding method described in any of the above embodiments when calling the computer program.

[0212] In some embodiments, some embodiments of the present application provide a video decoding device, the video decoding device comprising:

[0213] a memory configured to store a computer program;

[0214] The processor is configured to enable the video decoding device to implement the video decoding method described in any of the above embodiments when calling the computer program.

[0215] In some embodiments, some embodiments of the present application provide a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a computing device, the computing device implements the video encoding method described in any of the above embodiments or the video decoding method described in any of the above embodiments.

[0216] In some embodiments, some embodiments of the present application provide a computer program product, which, when executed on a computer, enables the computer to implement the video encoding method described in any of the above embodiments or the video decoding method described in any of the above embodiments.

[0217] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0218] For the convenience of explanation, the above description has been made in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or limit the embodiments to the specific forms disclosed above. Based on the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various different variations of the embodiments suitable for specific use considerations.

Claims

1. A video encoding method, characterized in that: include: Obtaining a target image block, wherein the target image block is a rectangular image block obtained by dividing the video frame into blocks; Performing texture edge detection on the target image block to obtain at least one texture edge of the target image block; Dividing the target image block into two image areas based on the at least one texture edge, and acquiring a boundary line between the two image areas; Performing curve fitting on the dividing line to obtain a segmentation curve of the target image block; Splitting the target image block into two sub-image blocks based on a segmentation curve of the target image block; According to the segmentation curve of the target image block and the two sub-image blocks, the code stream data corresponding to the target image block is acquired.

2. The method according to claim 1, characterized in that: The performing texture edge detection on the target image block to obtain at least one texture edge of the target image block includes: Converting the target image block into a grayscale image to obtain a grayscale image block corresponding to the target image block; Performing texture edge detection on the grayscale image block to obtain at least one texture edge of the target image block.

3. The method according to claim 2, characterized in that Before performing texture edge detection on the grayscale image block, the method further includes: The grayscale image block is filtered based on a Gaussian kernel of a preset size.

4. The method according to claim 1, characterized in that The step of dividing the target image block into two image areas based on the at least one texture edge comprises: Performing image region segmentation on the target image block based on the at least one texture edge to obtain an image region set corresponding to the target image block; Obtaining the number of image regions in the image region set; If the number of image regions in the image region set is greater than 2, the image regions in the image region set are merged into two image regions, so as to divide the target image block into two image regions.

5. The method according to claim 4, characterized in that The step of merging the image regions in the image region set into two image regions comprises: Acquire at least two merging schemes of the image region set, wherein the at least two merging schemes include various schemes for merging image regions in the image region set into two image regions; The difference between the at least two merging schemes is obtained respectively, wherein the difference between any merging scheme is the absolute difference between the first similarity and the second similarity of the merging scheme; the first similarity and the second similarity of any merging scheme are respectively the sum of the absolute differences between the grayscale values ​​of each pixel point of the two image regions and the corresponding co-located block under the merging scheme; The image regions in the image region set are merged into two image regions by using a merging scheme with the largest difference among the at least two merging schemes.

6. The method according to claim 1, characterized in that The performing curve fitting on the boundary line to obtain a segmentation curve of the target image block includes: Bezier curve fitting is performed on the boundary line, and a curve obtained by performing Bezier curve fitting on the boundary line is determined as a segmentation curve of the target image block.

7. The method according to claim 6, characterized in that The performing Bezier curve fitting on the boundary line, and determining a curve obtained by performing Bezier curve fitting on the boundary line as a segmentation curve of the target image block, comprises: Performing second-order Bezier curve fitting on the boundary line to obtain a first fitting result; Performing a third-order Bezier curve fitting on the boundary line to obtain a second fitting result; respectively obtaining a first fitting accuracy of the first fitting result and a second fitting accuracy of the second fitting result; According to the first fitting accuracy and the second fitting accuracy, the first fitting result or the second fitting result is selected as the segmentation curve of the target image block.

8. The method according to claim 1, characterized in that The step of dividing the target image block into two sub-image blocks based on the segmentation curve of the target image block comprises: Sampling the segmentation curve of the target image block with integer pixel accuracy to obtain a sampling result of the segmentation curve; The target image block is divided into two sub-image blocks according to the sampling result of the segmentation curve.

9. The method according to claim 1, characterized in that: The step of acquiring the bitstream data corresponding to the target image block according to the segmentation curve of the target image block and the two sub-image blocks includes: Encoding the information of the segmentation curve of the target image block to obtain first code stream data; Encoding the two sub-image blocks respectively to obtain second code stream data and third code stream data; Generate code stream data corresponding to the target image block according to the first code stream data, the second code stream data and the third code stream data.

10. A video decoding method, characterized in that: include: Obtaining bitstream data corresponding to a target image block, wherein the target image block is a rectangular image block obtained by dividing a video frame into blocks; The code stream data corresponding to the target image block includes: first code stream data, second code stream data and third code stream data; the first code stream data is code stream data obtained by encoding information of a segmentation curve used to segment the target image block into two sub-image blocks; the second code stream data and the third code stream data are code stream data obtained by encoding the two sub-image blocks respectively; Acquire the segmentation curve according to the first bitstream data; Segmenting the target image block based on the segmentation curve to obtain the two sub-image blocks; Reconstructing the two sub-image blocks according to the second code stream data and the third code stream data; The reconstructed target image block is obtained according to the two reconstructed sub-image blocks.

11. The method according to claim 10, characterized in that The first bitstream data includes: position information of a starting point, position information of an end point, and position information of a control point of a second-order Bezier curve, and obtaining the segmentation curve according to the first bitstream data includes: The second-order Bezier curve is reconstructed according to the position information of the starting point, the position information of the terminal point, the position information of the control point and a second-order Bezier curve formula to obtain the segmentation curve.

12. The method according to claim 10, characterized in that The first bitstream data includes: position information of a starting point of a third-order Bezier curve, position information of a terminal point, position information of a first control point, and position information of a second control point. The acquiring of the segmentation curve according to the first bitstream data includes: The third-order Bezier curve is reconstructed according to the position information of the starting point, the position information of the terminal point, the position information of the first control point, the position information of the second control point and a third-order Bezier curve formula to obtain the segmentation curve.

13. The method according to claim 10, characterized in that The step of segmenting the target image block based on the segmentation curve to obtain the two sub-image blocks includes: Sampling the segmentation curve of the target image block with integer pixel accuracy to obtain a sampling result of the segmentation curve; The target image block is divided into two sub-image blocks according to the sampling result of the segmentation curve.

14. A video encoding device, characterized in that: include: a memory configured to store a computer program; The processor is configured to, when calling a computer program, enable the video encoding device to implement the video encoding method according to any one of claims 1 to 9.

15. A video decoding device, characterized in that: include: a memory configured to store a computer program; The processor is configured to enable the video decoding device to implement the video decoding method according to any one of claims 10 to 13 when calling the computer program.

Images