Video encoding methods, devices and related equipment

Abstract

This application provides a video encoding method, apparatus, and related devices. The method includes: identifying multiple objects in an image to be encoded to obtain identification information for each object; segmenting the image to be encoded based on the identification information to obtain corresponding sub-images; sorting and grouping the multiple sub-images to obtain a mapping relationship; obtaining matching pixel blocks corresponding to each image sequence in each image sequence and determining a corresponding target sub-image based on the mapping relationship; and encoding each target sub-image sequentially to obtain a video stream. This application improves the video encoding effect by identifying and identifying multiple objects in the image to be encoded, thereby segmenting the image to obtain multiple sub-images, sorting and grouping the multiple sub-images to determine multiple image sequences and mapping relationships, determining multiple matching pixel blocks based on the image sequences, and encoding the sub-images corresponding to the matching pixel blocks.

Description

Technical Field

[0001] This application relates to the field of image coding, and more particularly to a video coding method, apparatus, and related equipment. Background Technology

[0002] In video coding standards, the current image is typically divided into fixed-size 16x16 macroblocks, and each macroblock is encoded from left to right. During encoding, a reference frame is determined for the current macroblock, and motion estimation methods are used to find the best-matching block in the reference frame. This process continues until all macroblocks in the current image are encoded, before proceeding to the next frame. However, this method indiscriminately divides and compresses the image, resulting in artificially created boundaries and a complex search area that fails to identify important macroblocks, leading to poor video coding quality. Summary of the Invention

[0003] This application provides a video encoding method, apparatus, and related equipment to solve the problem of poor video encoding quality in the prior art.

[0004] To solve the above problems, this application is implemented as follows:

[0005] In a first aspect, embodiments of this application provide a video encoding method, the method comprising:

[0006] Multiple objects in the image to be encoded are identified to obtain identification information for each object. Based on the identification information, the image to be encoded is segmented to obtain a sub-image corresponding to each identification information.

[0007] The multiple sub-images are sorted and grouped to obtain multiple image sequences and a mapping relationship between each image sequence and the multiple sub-images, wherein each image sequence includes at least two sub-images;

[0008] In each image sequence, a matching pixel block corresponding to that image sequence is obtained, and a target sub-image corresponding to each matching pixel block is determined based on the mapping relationship; wherein, the target sub-image is any one of the plurality of sub-images, and the matching pixel block is an image region in the image sequence;

[0009] Each of the target sub-images is encoded sequentially to obtain a video stream.

[0010] Optionally, the step of identifying multiple objects in the image to be encoded to obtain identification information corresponding to each object includes:

[0011] The image to be encoded is input into an image recognition model for image recognition to determine the plurality of objects included in the image to be encoded;

[0012] Based on the multiple objects, the image to be encoded is segmented into regions to obtain a target region corresponding to each object;

[0013] Each object within the target area is identified to obtain the identification information corresponding to each object.

[0014] Optionally, the step of performing image segmentation on the image to be encoded based on the identification information to obtain a sub-image corresponding to each identification information includes:

[0015] Traverse the objects corresponding to each identification information and determine the marked coordinates of each object;

[0016] A first maximum coordinate and a first minimum coordinate are determined on a first coordinate axis based on the marked coordinates of each object, and a second maximum coordinate and a second minimum coordinate are determined on a second coordinate axis.

[0017] Based on the first maximum coordinate, the first minimum coordinate, the second maximum coordinate, and the second minimum coordinate, the image pixel region where the object is located is obtained;

[0018] The image to be encoded is segmented based on the image pixel region to determine the sub-image corresponding to each object.

[0019] Optionally, the step of sorting and grouping the multiple sub-images to obtain multiple image sequences and the mapping relationship between each image sequence and the multiple sub-images includes:

[0020] The weight value for each sub-image is obtained by calculating the number of encoded bytes corresponding to each sub-image, the number of times the same number of encoded bytes appears, and the number of times the set of sub-images with the largest number of encoded bytes appears.

[0021] The multiple sub-images are sorted from largest to smallest according to the weight value of each sub-image to obtain the target image sequence;

[0022] The plurality of sub-images are grouped according to the target image sequence to obtain the plurality of image sequences, and the mapping relationship corresponding to each image sequence is determined according to the sub-images included in each image sequence.

[0023] Optionally, the step of obtaining the matching pixel block corresponding to each image sequence in each image sequence, and determining a target sub-image corresponding to each matching pixel block based on the mapping relationship, includes:

[0024] In the plurality of image sequences, for each sub-image, a macroblock with a target area is obtained in each sub-image;

[0025] In each of the sub-images, determine the matching pixel block that best matches the macroblock of the target area of ​​that sub-image;

[0026] Based on the relative displacement between the matching pixel block and the macroblock of the target area, a first motion vector and a second motion vector are determined;

[0027] Based on the mapping relationship, the first motion vector and the second motion vector are mapped to the coordinates of the image sequence corresponding to each sub-image to obtain the third motion vector and the fourth motion vector corresponding to each sub-image;

[0028] A target sub-image corresponding to each matching pixel block is determined based on the third motion vector, the fourth motion vector, and the mapping relationship.

[0029] Optionally, determining the matching pixel block that best matches the macroblock of the target area in each of the sub-images includes:

[0030] In the case that the sub-image is an instantaneous decoded refresh frame (IDR), the matching pixel block that best matches the macroblock of the target area is determined according to the image local matching algorithm;

[0031] If the sub-image is not a refresh frame, the matching pixel block that best matches the macroblock of the target area is determined from the reference sub-image of the previous reference frame of the image to be encoded according to the image local matching algorithm, and the identification information of the reference sub-image is the same as that of the sub-image.

[0032] Secondly, embodiments of this application also provide a video encoding apparatus, the apparatus comprising:

[0033] The identification module is used to identify multiple objects in the image to be encoded, obtain identification information corresponding to each object, and perform image segmentation on the image to be encoded based on the identification information to obtain a sub-image corresponding to each identification information.

[0034] A grouping module is used to sort and group multiple sub-images to obtain multiple image sequences and a mapping relationship between each image sequence and the multiple sub-images, wherein each image sequence includes at least two sub-images;

[0035] The acquisition module is configured to acquire a matching pixel block corresponding to each image sequence in each image sequence, and determine a target sub-image corresponding to each matching pixel block based on the mapping relationship; wherein, the target sub-image is any one of the plurality of sub-images, and the matching pixel block is an image region in the image sequence;

[0036] The encoding module is used to encode each of the target sub-images sequentially to obtain a video stream.

[0037] Thirdly, embodiments of this application also provide an electronic device, including: a transceiver, a memory, a processor, and a program stored in the memory and executable on the processor; the processor is configured to read the program in the memory to implement the steps in the method described in the first aspect above.

[0038] Fourthly, embodiments of this application also provide a readable storage medium for storing a program, which, when executed by a processor, implements the steps of the method described in the first aspect above.

[0039] Fifthly, embodiments of this application also provide a computer program product, which is stored in a storage medium and executed by at least one processor to implement the steps in the method as described in the first aspect.

[0040] This application provides a video encoding method, apparatus, and related devices. The method includes: identifying multiple objects in an image to be encoded to obtain identification information corresponding to each object; segmenting the image to be encoded based on the identification information to obtain sub-images corresponding to each identification information; sorting and grouping the multiple sub-images to obtain multiple image sequences and a mapping relationship between each image sequence and the multiple sub-images, wherein each image sequence includes at least two sub-images; obtaining matching pixel blocks corresponding to each image sequence, and determining a target sub-image corresponding to each matching pixel block based on the mapping relationship; wherein the target sub-image is any one of the multiple sub-images, and the matching pixel block is an image region in the image sequence; and encoding each target sub-image sequentially to obtain a video stream. This application improves the video encoding effect by identifying and representing multiple objects in an image to be encoded, thereby segmenting the image to obtain multiple sub-images, sorting and grouping the multiple sub-images to determine multiple image sequences and mapping relationships, determining multiple matching pixel blocks based on the image sequences, and encoding the sub-images corresponding to the matching pixel blocks. Attached Figure Description

[0041] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a flowchart illustrating the video encoding method provided in an embodiment of this application;

[0043] Figure 2 This is a schematic diagram of the structure of the video encoding device provided in the embodiments of this application;

[0044] Figure 3 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0045] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0046] The terms "first," "second," etc., used in the embodiments of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or devices. Additionally, the use of "and / or" in this application indicates at least one of the connected objects, such as A and / or B and / or C, representing seven possibilities: including A alone, B alone, C alone, and the presence of both A and B, both B and C, both A and C, and the presence of A, B, and C.

[0047] See Figure 1 , Figure 1 This is a flowchart illustrating the video encoding method provided in the embodiments of this application. Figure 1 The video encoding method shown can be performed by an encoder.

[0048] like Figure 1 As shown, a video encoding method may include the following steps:

[0049] Step 101: Identify multiple objects in the image to be encoded to obtain identification information corresponding to each object. Perform image segmentation on the image to be encoded based on the identification information to obtain sub-images corresponding to each identification information.

[0050] In this embodiment, the image to be encoded can be one or more images. This embodiment uses one image as an example; the encoding method for multiple images is the same as for one. Specifically, the image to be encoded includes multiple objects, which can be people, animals, or equipment, etc., and this embodiment does not impose any specific limitations. The identification information can be IDs, for example, assigning numbers 1, 2, and 3 to three different objects, with each identification information corresponding to each object.

[0051] In this embodiment, the image to be encoded is segmented into multiple sub-images based on the determined identification information. Each sub-image includes a numbered object. It should be noted that the segmented images may be of different sizes, so it is necessary to unify the multiple sub-images to obtain multiple sub-images of the same size.

[0052] Step 102: Sort and group the multiple sub-images to obtain multiple image sequences and the mapping relationship between each image sequence and the multiple sub-images, wherein each image sequence includes at least two sub-images.

[0053] In this embodiment, the acquired multiple sub-images are sorted and grouped. The sorting can be performed according to preset weight values, and the groups are formed based on the sorting results, thereby determining multiple image sequences and multiple mapping relationships. Each image sequence includes at least two sub-images. The mapping relationship represents the correspondence between the image sequence and the multiple sub-images.

[0054] Step 103: Obtain the matching pixel block corresponding to each image sequence, and determine a target sub-image corresponding to each matching pixel block based on the mapping relationship; wherein, the target sub-image is any one of the plurality of sub-images, and the matching pixel block is an image region in the image sequence.

[0055] In this embodiment, multiple matching pixel blocks corresponding to each image sequence are determined. A matching pixel block refers to the block that is most matched by calculating the value of the local image matching algorithm (Sum of absolute differences, SAD). Multiple target sub-images corresponding to multiple matching pixel blocks are determined by mapping relationship.

[0056] Step 104: Encode each of the target sub-images sequentially to obtain a video stream.

[0057] In this embodiment, a video stream is obtained by encoding multiple determined target sub-images. Additionally, it is determined whether there are any unprocessed images. These unprocessed images are then encoded after the above steps.

[0058] This application improves video encoding by identifying and representing multiple objects in an image to be encoded, thereby segmenting the image into multiple sub-images, sorting and grouping the sub-images to determine multiple image sequences and mapping relationships, determining multiple matching pixel blocks based on the image sequences, and encoding the sub-images corresponding to the matching pixel blocks.

[0059] In some feasible implementations, optionally, the step of identifying multiple objects in the image to be encoded to obtain identification information corresponding to each object includes:

[0060] The image to be encoded is input into an image recognition model for image recognition to determine the plurality of objects included in the image to be encoded;

[0061] Based on the multiple objects, the image to be encoded is segmented into regions to obtain a target region corresponding to each object;

[0062] Each object within the target area is identified to obtain the identification information corresponding to each object.

[0063] In this embodiment, after acquiring the image to be encoded at the encoding end, a deep learning network is used to perform scene recognition on the input image and segment the objects within it to obtain multiple target regions. By representing the objects included in these multiple target regions, multiple identification information is obtained. The deep learning network can be an image recognition model used to identify objects in the input image; however, this embodiment does not impose specific limitations. For example, object IDs from 0 to n are obtained, denoted as ID0...Idn.

[0064] Optionally, the step of performing image segmentation on the image to be encoded based on the identification information to obtain a sub-image corresponding to each identification information includes:

[0065] Traverse the objects corresponding to each identification information and determine the marked coordinates of each object;

[0066] A first maximum coordinate and a first minimum coordinate are determined on a first coordinate axis based on the marked coordinates of each object, and a second maximum coordinate and a second minimum coordinate are determined on a second coordinate axis.

[0067] Based on the first maximum coordinate, the first minimum coordinate, the second maximum coordinate, and the second minimum coordinate, the image pixel region where the object is located is obtained;

[0068] The image to be encoded is segmented based on the image pixel region to determine the sub-image corresponding to each object.

[0069] In this embodiment, the coordinates of all markers within the current object are obtained, and the maximum and minimum values ​​of the first coordinate axis X direction are calculated as the first maximum coordinate and the first minimum coordinate, i.e., Xmax and Xmin; the maximum and minimum values ​​of the second coordinate axis Y direction are calculated as the second maximum coordinate Ymax and the second minimum coordinate Ymin.

[0070] The four values ​​Xmax / Xmin / Ymax / Ymin are used to obtain the image pixels at the corresponding coordinate positions, resulting in a sub-image Pi. Sub-image Pi is then filled by using the boundary pixels marked by IDi to fill the object, making the object a rectangular area with a minimum size of 16x16, where the length and width are integer multiples of 16. Additionally, it is necessary to determine if there are any remaining unprocessed images, ultimately resulting in multiple sub-images.

[0071] Optionally, the step of sorting and grouping the multiple sub-images to obtain multiple image sequences and the mapping relationship between each image sequence and the multiple sub-images includes:

[0072] The weight value for each sub-image is obtained by calculating the number of encoded bytes corresponding to each sub-image, the number of times the same number of encoded bytes appears, and the number of times the set of sub-images with the largest number of encoded bytes appears.

[0073] The multiple sub-images are sorted from largest to smallest according to the weight value of each sub-image to obtain the target image sequence;

[0074] The plurality of sub-images are grouped according to the target image sequence to obtain the plurality of image sequences, and the mapping relationship corresponding to each image sequence is determined according to the sub-images included in each image sequence.

[0075] In this embodiment, the weight value corresponding to each sub-image is calculated. Specifically, the weight of the sub-image with the current ID can be obtained by counting the number of bytes encoded in the sub-image for each ID, the number of times the same number of bytes appears, and the number of times the sub-image set with the largest number of bytes appears at the top of the pyramid. Based on the weight of the sub-images, the images of the current Pi are sorted to obtain a new sub-image sequence from PP0 to PPn. The coordinate mapping Mapi from the current PP image sequence to the P image sequence is saved, ultimately resulting in multiple image sequences and multiple mapping relationships.

[0076] Optionally, the step of obtaining the matching pixel block corresponding to each image sequence in each image sequence, and determining a target sub-image corresponding to each matching pixel block based on the mapping relationship, includes:

[0077] In the plurality of image sequences, for each sub-image, a macroblock with a target area is obtained in each sub-image;

[0078] In each of the sub-images, determine the matching pixel block that best matches the macroblock of the target area of ​​that sub-image;

[0079] Based on the relative displacement between the matching pixel block and the macroblock of the target area, a first motion vector and a second motion vector are determined;

[0080] Based on the mapping relationship, the first motion vector and the second motion vector are mapped to the coordinates of the image sequence corresponding to each sub-image to obtain the third motion vector and the fourth motion vector corresponding to each sub-image;

[0081] A target sub-image corresponding to each matching pixel block is determined based on the third motion vector, the fourth motion vector, and the mapping relationship.

[0082] In this embodiment, for each sub-image in the plurality of image sequences, the sub-image is encoded from left to right and from top to bottom. Specifically, a 16x16 macroblock is obtained from the PPI sub-image, and the block that best matches the current block is searched from the current sub-image.

[0083] Based on the result of satd, the block with the smallest satd is selected to complete the search result. The first motion vector and the second motion vector, namely MVix and MViy, are calculated based on the relative displacement between the matching pixel block and the macroblock of the target area.

[0084] The motion vectors MVix and MViy of the current block are mapped to the coordinates of the current sub-image PPi in the P image sequence according to the mapping Mapi, to obtain the third motion vector and the fourth motion vector, namely MNixr and MViyr. The third motion vector and the fourth motion vector are used to match in each macroblock of the target area to determine the target sub-image corresponding to the matching pixel block, thereby completing the search of the current block.

[0085] Optionally, determining the matching pixel block that best matches the macroblock of the target area in each of the sub-images includes:

[0086] In the case that the sub-image is an instantaneous decoded refresh frame (IDR), the matching pixel block that best matches the macroblock of the target area is determined according to the image local matching algorithm;

[0087] If the sub-image is not a refresh frame, the matching pixel block that best matches the macroblock of the target area is determined from the reference sub-image of the previous reference frame of the image to be encoded according to the image local matching algorithm, and the identification information of the reference sub-image is the same as that of the sub-image.

[0088] In this embodiment, when the current sub-image is decoded and refreshed in real time (IDR), the matching pixel block that best matches the macroblock of the target area is directly determined according to the local image matching algorithm; when the current sub-image is not a refreshed frame, the block that best matches the current block is searched from the PPi image with the same ID in the previous reference frame, and the encoding process continues according to the best matching block.

[0089] This application improves video encoding by identifying and representing multiple objects in an image to be encoded, thereby segmenting the image into multiple sub-images, sorting and grouping the sub-images to determine multiple image sequences and mapping relationships, determining multiple matching pixel blocks based on the image sequences, and encoding the sub-images corresponding to the matching pixel blocks.

[0090] See Figure 2 , Figure 2 This is a structural diagram of the video encoding apparatus provided in an embodiment of this application. Figure 2 As shown, the video encoding device 200 includes:

[0091] The identification module 210 is used to identify multiple objects in the image to be encoded, obtain identification information corresponding to each object, and perform image segmentation on the image to be encoded based on the identification information to obtain a sub-image corresponding to each identification information.

[0092] Grouping module 220 is used to sort and group multiple sub-images to obtain multiple image sequences and a mapping relationship between each image sequence and the multiple sub-images, wherein each image sequence includes at least two sub-images;

[0093] The acquisition module 230 is used to acquire a matching pixel block corresponding to each image sequence in each image sequence, and determine a target sub-image corresponding to each matching pixel block based on the mapping relationship; wherein, the target sub-image is any one of the plurality of sub-images, and the matching pixel block is an image region in the image sequence;

[0094] The encoding module 240 is used to encode each of the target sub-images sequentially to obtain a video stream.

[0095] Optionally, the identification module 210 includes:

[0096] The recognition submodule is used to input the image to be encoded into the image recognition model for image recognition and to determine the multiple objects included in the image to be encoded.

[0097] The first segmentation submodule is used to perform region segmentation on the image to be encoded based on the multiple objects, so as to obtain a target region corresponding to each object;

[0098] The identification submodule is used to identify objects in each of the target areas to obtain identification information for each object.

[0099] Optionally, the identification module 210 includes:

[0100] The first determining submodule is used to traverse the objects corresponding to each identification information and determine the marked coordinates of each object;

[0101] The second determining submodule is used to determine a first maximum coordinate and a first minimum coordinate on a first coordinate axis based on the marked coordinates of each object, and to determine a second maximum coordinate and a second minimum coordinate on a second coordinate axis.

[0102] The first acquisition submodule is used to acquire the image pixel region where the object is located based on the first maximum coordinate, the first minimum coordinate, the second maximum coordinate, and the second minimum coordinate;

[0103] The second segmentation submodule is used to segment the image to be encoded based on the image pixel region and determine the sub-image corresponding to each object.

[0104] Optionally, the grouping module 220 includes:

[0105] The calculation submodule is used to calculate the number of encoded bytes corresponding to each sub-image, the number of times the same number of encoded bytes appears, and the number of times the set of sub-images with the largest number of encoded bytes appears, so as to obtain the weight value corresponding to each sub-image;

[0106] The sorting submodule is used to sort the multiple sub-images from largest to smallest according to the weight value of each sub-image to obtain the target image sequence;

[0107] The grouping submodule is used to group the plurality of sub-images according to the target image sequence to obtain the plurality of image sequences, and to determine the mapping relationship corresponding to each image sequence according to the sub-images included in each image sequence.

[0108] Optionally, the acquisition module 230 includes:

[0109] The second acquisition submodule is used to acquire a macroblock of a target area in each of the plurality of image sequences for each sub-image;

[0110] The third determining submodule is used to determine, in each of the sub-images, the matching pixel block that best matches the macroblock of the target area of ​​the sub-image;

[0111] The fourth determining submodule is used to determine the first motion vector and the second motion vector based on the relative displacement between the matching pixel block and the macroblock of the target area;

[0112] The conversion submodule is used for

[0113] Based on the mapping relationship, the first motion vector and the second motion vector are mapped to the coordinates of the image sequence corresponding to each sub-image to obtain the third motion vector and the fourth motion vector corresponding to each sub-image;

[0114] The fifth determining submodule is used to determine a target sub-image corresponding to each of the matching pixel blocks based on the third motion vector, the fourth motion vector, and the mapping relationship.

[0115] Optionally, the third determination submodule includes:

[0116] The first determining unit is configured to determine, in the case that the sub-image is an instantaneous decoded refresh frame (IDR), the matching pixel block that best matches the macroblock of the target area according to an image local matching algorithm.

[0117] The second determining unit is configured to, when the sub-image is not a refresh frame, determine, from the reference sub-image of the previous reference frame of the image to be encoded, the matching pixel block that best matches the macroblock of the target area according to the image local matching algorithm, wherein the identification information of the reference sub-image and the sub-image is the same.

[0118] This application improves video encoding by identifying and representing multiple objects in an image to be encoded, thereby segmenting the image into multiple sub-images, sorting and grouping the sub-images to determine multiple image sequences and mapping relationships, determining multiple matching pixel blocks based on the image sequences, and encoding the sub-images corresponding to the matching pixel blocks.

[0119] This application also provides an electronic device. Please refer to [link to relevant documentation]. Figure 3 The electronic device may include a processor 301, a memory 302, and a program 3021 stored in the memory 302 and capable of running on the processor 301.

[0120] When program 3021 is executed by processor 301, it can achieve the following: Figure 1 Any step in the corresponding method embodiment:

[0121] Multiple objects in the image to be encoded are identified to obtain identification information for each object. Based on the identification information, the image to be encoded is segmented to obtain a sub-image corresponding to each identification information.

[0122] The multiple sub-images are sorted and grouped to obtain multiple image sequences and a mapping relationship between each image sequence and the multiple sub-images, wherein each image sequence includes at least two sub-images;

[0123] In each image sequence, a matching pixel block corresponding to that image sequence is obtained, and a target sub-image corresponding to each matching pixel block is determined based on the mapping relationship; wherein, the target sub-image is any one of the plurality of sub-images, and the matching pixel block is an image region in the image sequence;

[0124] Each of the target sub-images is encoded sequentially to obtain a video stream.

[0125] Optionally, the step of identifying multiple objects in the image to be encoded to obtain identification information corresponding to each object includes:

[0126] The image to be encoded is input into an image recognition model for image recognition to determine the plurality of objects included in the image to be encoded;

[0127] Based on the multiple objects, the image to be encoded is segmented into regions to obtain a target region corresponding to each object;

[0128] Each object within the target area is identified to obtain the identification information corresponding to each object.

[0129] Optionally, the step of performing image segmentation on the image to be encoded based on the identification information to obtain a sub-image corresponding to each identification information includes:

[0130] Traverse the objects corresponding to each identification information and determine the marked coordinates of each object;

[0131] A first maximum coordinate and a first minimum coordinate are determined on a first coordinate axis based on the marked coordinates of each object, and a second maximum coordinate and a second minimum coordinate are determined on a second coordinate axis.

[0132] Based on the first maximum coordinate, the first minimum coordinate, the second maximum coordinate, and the second minimum coordinate, the image pixel region where the object is located is obtained;

[0133] The image to be encoded is segmented based on the image pixel region to determine the sub-image corresponding to each object.

[0134] Optionally, the step of sorting and grouping the multiple sub-images to obtain multiple image sequences and the mapping relationship between each image sequence and the multiple sub-images includes:

[0135] The weight value for each sub-image is obtained by calculating the number of encoded bytes corresponding to each sub-image, the number of times the same number of encoded bytes appears, and the number of times the set of sub-images with the largest number of encoded bytes appears.

[0136] The multiple sub-images are sorted from largest to smallest according to the weight value of each sub-image to obtain the target image sequence;

[0137] The plurality of sub-images are grouped according to the target image sequence to obtain the plurality of image sequences, and the mapping relationship corresponding to each image sequence is determined according to the sub-images included in each image sequence.

[0138] Optionally, the step of obtaining the matching pixel block corresponding to each image sequence in each image sequence, and determining a target sub-image corresponding to each matching pixel block based on the mapping relationship, includes:

[0139] In the plurality of image sequences, for each sub-image, a macroblock with a target area is obtained in each sub-image;

[0140] In each of the sub-images, determine the matching pixel block that best matches the macroblock of the target area of ​​that sub-image;

[0141] Based on the relative displacement between the matching pixel block and the macroblock of the target area, a first motion vector and a second motion vector are determined;

[0142] Based on the mapping relationship, the first motion vector and the second motion vector are mapped to the coordinates of the image sequence corresponding to each sub-image to obtain the third motion vector and the fourth motion vector corresponding to each sub-image;

[0143] A target sub-image corresponding to each matching pixel block is determined based on the third motion vector, the fourth motion vector, and the mapping relationship.

[0144] Optionally, determining the matching pixel block that best matches the macroblock of the target area in each of the sub-images includes:

[0145] In the case that the sub-image is an instantaneous decoded refresh frame (IDR), the matching pixel block that best matches the macroblock of the target area is determined according to the image local matching algorithm;

[0146] If the sub-image is not a refresh frame, the matching pixel block that best matches the macroblock of the target area is determined from the reference sub-image of the previous reference frame of the image to be encoded according to the image local matching algorithm, and the identification information of the reference sub-image is the same as that of the sub-image.

[0147] This application improves video encoding by identifying and representing multiple objects in an image to be encoded, thereby segmenting the image into multiple sub-images, sorting and grouping the sub-images to determine multiple image sequences and mapping relationships, determining multiple matching pixel blocks based on the image sequences, and encoding the sub-images corresponding to the matching pixel blocks.

[0148] This application also provides a computer-readable storage medium storing a computer program. When executed by a processor, this computer program implements the various processes of the above-described video encoding method embodiments and achieves the same technical effects. To avoid repetition, it will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0149] This application also provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above-described video encoding method embodiments, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0150] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0151] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0152] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A method of video coding, the method comprising: The method comprises: identifying a plurality of objects in a to-be-encoded image respectively to obtain identification information corresponding to each object, performing image segmentation on the to-be-encoded image based on the identification information to obtain a sub-image corresponding to each identification information, the image segmentation based on the identification information on the to-be-encoded image to obtain a sub-image corresponding to each identification information comprises: traversing each object corresponding to the identification information, determining the marking coordinates of each object; determining the first maximum coordinate and the first minimum coordinate on the first coordinate axis and the second maximum coordinate and the second minimum coordinate on the second coordinate axis according to the marking coordinates of each object; obtaining the image pixel region where the object is located according to the first maximum coordinate, the first minimum coordinate, the second maximum coordinate and the second minimum coordinate; performing image segmentation on the to-be-encoded image according to the image pixel region to determine the sub-image corresponding to each object; sorting and grouping a plurality of the sub-images to obtain a plurality of image sequences and a mapping relationship between each image sequence and the plurality of sub-images, each image sequence comprising at least two sub-images; obtaining a matching pixel block corresponding to each image sequence in each image sequence, and determining a target sub-image corresponding to each matching pixel block based on the mapping relationship; wherein the target sub-image is any one of the plurality of sub-images, and the matching pixel block is an image region in the image sequence; encoding each target sub-image in turn to obtain a video stream.

2. The method of claim 1, wherein, The identification of a plurality of objects in a to-be-encoded image respectively to obtain identification information corresponding to each object comprises: inputting the to-be-encoded image into an image recognition model for image recognition to determine the plurality of objects included in the to-be-encoded image; performing region segmentation on the to-be-encoded image based on the plurality of objects to obtain a target region corresponding to each object respectively; identifying the objects in each target region respectively to obtain identification information corresponding to each object.

3. The method of claim 1, wherein, The sorting and grouping of a plurality of the sub-images to obtain a plurality of image sequences and a mapping relationship between each image sequence and the plurality of sub-images comprises: calculating the number of encoded bytes corresponding to each sub-image, the number of times the same encoded bytes appear, and the number of times the sub-image set with the most encoded bytes appears to obtain a weight value corresponding to each sub-image; sorting the plurality of sub-images from large to small according to the weight value of each sub-image to obtain a target image sequence; grouping the plurality of sub-images according to the target image sequence to obtain the plurality of image sequences, and determining a mapping relationship corresponding to each image sequence according to the sub-images included in each image sequence.

4. The method of claim 1, wherein, The obtaining of a matching pixel block corresponding to each image sequence in each image sequence, and the determination of a target sub-image corresponding to each matching pixel block based on the mapping relationship comprise: in the plurality of image sequences, for each sub-image, obtaining a target area of macroblocks in each sub-image; determining, in each of the sub-images, a matching pixel block that is most matched with the macro block of the target area of the sub-image; determining a first motion vector and a second motion vector based on a relative displacement between the matching pixel block and the macro block of the target area; mapping the first motion vector and the second motion vector into coordinates of an image sequence corresponding to each of the sub-images based on the mapping relationship, to obtain a third motion vector and a fourth motion vector corresponding to each of the sub-images; determining a target sub-image corresponding to each of the matching pixel blocks according to the third motion vector, the fourth motion vector and the mapping relationship.

5. The method of claim 4, wherein, The determining, in each of the sub-images, a matching pixel block that is most matched with the macro block of the target area of the sub-image, comprises: in a case where the sub-image is an instant decoding refresh frame (IDR), determining the matching pixel block that is most matched with the macro block of the target area according to a local image matching algorithm; in a case where the sub-image is not a refresh frame, determining the matching pixel block that is most matched with the macro block of the target area from a reference sub-image of a previous reference frame of the image to be encoded according to a local image matching algorithm, the reference sub-image having the same identification information as the sub-image.

6. A video encoding apparatus, comprising: The apparatus comprises: an identification module configured to identify a plurality of objects in an image to be encoded respectively to obtain identification information corresponding to each of the objects, perform image segmentation on the image to be encoded based on the identification information to obtain a sub-image corresponding to each of the identification information, the identification module comprising: a first determination submodule configured to traverse each of the objects corresponding to the identification information to determine a marker coordinate of each of the objects; a second determination submodule configured to determine a first maximum coordinate and a first minimum coordinate on a first coordinate axis and a second maximum coordinate and a second minimum coordinate on a second coordinate axis according to the marker coordinate of each of the objects; a first acquisition submodule configured to acquire an image pixel region where the object is located according to the first maximum coordinate, the first minimum coordinate, the second maximum coordinate and the second minimum coordinate; and a second segmentation submodule configured to perform image segmentation on the image to be encoded according to the image pixel region to determine a sub-image corresponding to each of the objects; a grouping module configured to sort and group a plurality of the sub-images to obtain a plurality of image sequences and a mapping relationship between each of the image sequences and the plurality of sub-images, each of the image sequences comprising at least two of the sub-images; an acquisition module configured to acquire a matching pixel block corresponding to each of the image sequences and determine a target sub-image corresponding to each of the matching pixel blocks based on the mapping relationship, wherein the target sub-image is any one of the plurality of sub-images and the matching pixel block is an image region in the image sequence; an encoding module configured to encode each of the target sub-images in sequence to obtain a video stream.

7. An electronic device comprising: A memory, a processor, and a program stored in the memory and capable of running on the processor; wherein the processor is configured to read the program in the memory to implement the steps in the video encoding method according to any one of claims 1 to 5.

8. A readable storage medium for storing a program, characterized by The program, when executed by the processor, implements the steps in the video encoding method according to any one of claims 1 to 5.

9. A computer program product, characterised in that, The computer program product is stored in a storage medium, and the computer program product is executed by at least one processor to implement the steps in the video encoding method according to any one of claims 1 to 5.

Images