Video coding method and device and related equipment

Abstract

The invention provides a video coding method and device and related equipment, and the method comprises the steps: carrying out the identification of a plurality of objects in a to-be-coded image, obtaining the identification information corresponding to each object, carrying out the image segmentation of the to-be-coded image based on the identification information, and obtaining corresponding sub-images; sorting and grouping the plurality of sub-images to obtain a mapping relation; obtaining a matched pixel block corresponding to the image sequence in each image sequence, and determining a corresponding target sub-image based on the mapping relation; and coding each target sub-image in sequence to obtain a video stream. According to the method and the device, the plurality of objects in the to-be-coded image are identified and identified, so that image segmentation is performed on the to-be-coded image to obtain the plurality of sub-images, the plurality of sub-images are sorted and grouped to determine the plurality of image sequences and the mapping relationship, and the plurality of matched pixel blocks are determined according to the image sequences; and coding is carried out according to the sub-image corresponding to the matched pixel block, so that the video coding effect is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image coding, and in particular to a video coding method, apparatus and related equipment. Background Art

[0002] In the video coding standard, the current picture is generally divided into 16x16 fixed-size macroblocks, and then each macroblock is encoded from left to right. During the encoding process, a reference frame is determined for the current macroblock, and the motion estimation method is used to find the best matching block in the reference frame until all macroblocks in the current picture are encoded, and then the next frame of the picture is encoded. However, this method will divide and compress the picture indiscriminately, resulting in artificial differentiation boundaries, and the search area is also relatively complex, and it is impossible to determine important macroblocks, resulting in poor video encoding effects. Summary of the invention

[0003] The embodiments of the present application provide a video encoding method, apparatus and related equipment to solve the problem of poor video encoding effect in the prior art.

[0004] To solve the above problems, this application is implemented as follows:

[0005] In a first aspect, an embodiment of the present application provides a video encoding method, the method comprising:

[0006] Identify multiple objects in the image to be encoded respectively to obtain identification information corresponding to each object, and segment the image to be encoded based on the identification information to obtain a sub-image corresponding to each identification information;

[0007] Sorting and grouping the plurality of sub-images to obtain a plurality of image sequences and a mapping relationship between each of the image sequences and the plurality of sub-images, wherein each of the image sequences includes at least two of the sub-images;

[0008] Acquire a matching pixel block corresponding to each of the image sequences, and determine a target sub-image corresponding to each of the matching pixel blocks based on the mapping relationship; wherein the target sub-image is any one of the multiple sub-images, and the matching pixel block is an image region in the image sequence;

[0009] Each of the target sub-images is encoded in turn to obtain a video stream.

[0010] Optionally, the step of respectively identifying multiple objects in the image to be encoded to obtain identification information corresponding to each object includes:

[0011] Inputting the image to be encoded into an image recognition model for image recognition to determine the multiple objects included in the image to be encoded;

[0012] Performing region segmentation on the image to be encoded based on the multiple objects to obtain a target region corresponding to each object;

[0013] Objects in each target area are identified respectively to obtain identification information corresponding to each object.

[0014] Optionally, the performing image segmentation on the image to be encoded based on the identification information to obtain a sub-image corresponding to each piece of identification information includes:

[0015] Traversing the objects corresponding to each identification information, and determining the marking coordinates of each of the objects;

[0016] Determine a first maximum coordinate and a first minimum coordinate on the first coordinate axis according to the marked coordinates of each of the objects, and determine a second maximum coordinate and a second minimum coordinate on the second coordinate axis;

[0017] Acquire an image pixel region where the object is located according to the first maximum coordinate, the first minimum coordinate, the second maximum coordinate, and the second minimum coordinate;

[0018] The image to be encoded is segmented according to the image pixel area to determine a sub-image corresponding to each of the objects.

[0019] Optionally, the sorting and grouping of the plurality of sub-images to obtain a plurality of image sequences and a mapping relationship between each of the image sequences and the plurality of sub-images includes:

[0020] Calculate the number of coded bytes corresponding to each sub-image, the number of occurrences of the same number of coded bytes, and the number of sub-image sets with the largest number of coded bytes, and obtain a weight value corresponding to each sub-image;

[0021] Sort the multiple sub-images from large to small according to the weight value of each sub-image to obtain a target image sequence;

[0022] The multiple sub-images are grouped according to the target image sequence to obtain the multiple image sequences, and a mapping relationship corresponding to each of the image sequences is determined according to the sub-images included in each of the image sequences.

[0023] Optionally, acquiring, in each of the image sequences, a matching pixel block corresponding to the image sequence, and determining, based on the mapping relationship, a target sub-image corresponding to each of the matching pixel blocks, includes:

[0024] In the plurality of image sequences, for each sub-image, obtaining a macroblock of a target area in each sub-image;

[0025] Determine in each of the sub-images a matching pixel block that best matches the macroblock of the target area of ​​the sub-image;

[0026] Determining a first motion vector and a second motion vector based on a relative displacement between the matching pixel block and the macroblock of the target area;

[0027] Mapping the first motion vector and the second motion vector to the coordinates of the image sequence corresponding to each of the sub-images based on the mapping relationship to obtain a third motion vector and a fourth motion vector corresponding to each of the sub-images;

[0028] A target sub-image corresponding to each matching pixel block is determined according to the third motion vector, the fourth motion vector and the mapping relationship.

[0029] Optionally, determining in each of the sub-images a matching pixel block that best matches a macroblock of the target area of ​​the sub-image includes:

[0030] In the case where the sub-image is an instant decoding refresh frame IDR, determining a matching pixel block that best matches the macroblock of the target area according to an image local matching algorithm;

[0031] When the sub-image is not a refresh frame, a matching pixel block that best matches the macroblock of the target area is determined from a reference sub-image of a previous reference frame of the image to be encoded according to an image local matching algorithm, and the reference sub-image has the same identification information as the sub-image.

[0032] In a second aspect, an embodiment of the present application further provides a video encoding device, the device comprising:

[0033] An identification module, used to identify multiple objects in the image to be encoded respectively, obtain identification information corresponding to each object, and perform image segmentation on the image to be encoded based on the identification information to obtain a sub-image corresponding to each identification information;

[0034] a grouping module, used for sorting and grouping the plurality of sub-images to obtain a plurality of image sequences and a mapping relationship between each of the image sequences and the plurality of sub-images, wherein each of the image sequences includes at least two of the sub-images;

[0035] an acquisition module, configured to acquire, in each of the image sequences, a matching pixel block corresponding to the image sequence, and determine, based on the mapping relationship, a target sub-image corresponding to each of the matching pixel blocks; wherein the target sub-image is any one of the multiple sub-images, and the matching pixel block is an image region in the image sequence;

[0036] The encoding module is used to encode each of the target sub-images in turn to obtain a video stream.

[0037] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a transceiver, a memory, a processor, and a program stored in the memory and executable on the processor; the processor is used to read the program in the memory to implement the steps of the method described in the first aspect above.

[0038] In a fourth aspect, an embodiment of the present application further provides a readable storage medium for storing a program, which, when executed by a processor, implements the steps in the method described in the first aspect above.

[0039] In a fifth aspect, an embodiment of the present application further provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the steps in the method described in the first aspect.

[0040] The present application provides a video encoding method, apparatus and related equipment, the method comprising: identifying multiple objects in an image to be encoded respectively, obtaining identification information corresponding to each object, segmenting the image to be encoded based on the identification information, obtaining sub-images corresponding to each identification information; sorting and grouping the multiple sub-images, obtaining multiple image sequences and mapping relationships between each image sequence and the multiple sub-images, each of the image sequences including at least two sub-images; obtaining matching pixel blocks corresponding to the image sequence in each image sequence, and determining a target sub-image corresponding to each matching pixel block based on the mapping relationship; wherein the target sub-image is any one of the multiple sub-images, and the matching pixel block is an image region in the image sequence; encoding each target sub-image in turn to obtain a video stream. The present application identifies multiple objects in an image to be encoded and represents them, thereby segmenting the image to obtain multiple sub-images, sorting and grouping the multiple sub-images to determine multiple image sequences and mapping relationships, and determining multiple matching pixel blocks according to the image sequence, and encoding the sub-images corresponding to the matching pixel blocks, thereby improving the effect of video encoding. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0042] Figure 1 It is a flowchart of a video encoding method provided by an embodiment of the present application;

[0043] Figure 2 is a schematic diagram of the structure of a video encoding device provided in an embodiment of the present application;

[0044] Figure 3 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0046] The terms "first", "second" etc. in the embodiments of the present application are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. In addition, the terms "include" and "have" and any variation thereof are intended to cover non-exclusive inclusions, for example, the process, method, system, product or equipment comprising a series of steps or units need not be limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment. In addition, "and / or" is used in the present application to represent at least one of connected objects, such as A and / or B and / or C, representing the inclusion of separate A, separate B, separate C, and A and B all exist, B and C all exist, A and C all exist, and 7 situations that A, B and C all exist.

[0047] See also Figure 1 , Figure 1 It is a flowchart of the video encoding method provided in an embodiment of the present application. Figure 1 The video encoding method shown may be performed by an encoder.

[0048] like Figure 1 As shown, the video encoding method may include the following steps:

[0049] Step 101: Identify multiple objects in an image to be encoded respectively to obtain identification information corresponding to each object, and segment the image to be encoded based on the identification information to obtain a sub-image corresponding to each identification information.

[0050] In this embodiment, the image to be encoded may be one or more images. In this embodiment, one image is used as an example for description, and the encoding method of the multiple images is the same as that of the single image. Specifically, multiple objects included in the image to be encoded are determined, where the objects may be people, animals, or equipment, etc., which are not specifically limited in this embodiment. The identification information may be an ID, for example, three different objects are assigned numbers 1, 2, and 3, and each identification information corresponds to each object.

[0051] In this embodiment, the image to be encoded is segmented into multiple sub-images based on the determined multiple identification information, and each sub-image includes a numbered object. It should be noted that the sizes of the segmented images can be different, so the multiple sub-images need to be unified to obtain multiple sub-images of the same size.

[0052] Step 102: sort and group the multiple sub-images to obtain multiple image sequences and a mapping relationship between each of the image sequences and the multiple sub-images, each of the image sequences including at least two sub-images.

[0053] In this embodiment, the obtained multiple sub-images are sorted and grouped, wherein the sorting can be performed according to a preset weight value, and the grouping is performed according to the sorting result, so as to determine multiple image sequences and multiple mapping relationships, each of which includes at least two sub-images. The mapping relationship is used to represent the corresponding relationship between the image sequence and the multiple sub-images.

[0054] Step 103, obtaining a matching pixel block corresponding to the image sequence in each of the image sequences, and determining a target sub-image corresponding to each of the matching pixel blocks based on the mapping relationship; wherein the target sub-image is any one of the multiple sub-images, and the matching pixel block is an image area in the image sequence.

[0055] In this embodiment, a plurality of matching pixel blocks corresponding to each image sequence are determined, wherein the matching pixel block refers to a block that is most matched by calculating the value of a local image matching algorithm (Sum of absolute differences, SAD), and a plurality of target sub-images corresponding to the plurality of matching pixel blocks are determined through a mapping relationship.

[0056] Step 104: Encode each of the target sub-images in turn to obtain a video stream.

[0057] In this embodiment, a video stream is obtained by encoding the determined multiple target sub-images. In addition, it is determined whether there are images that have not been processed. After the unprocessed images have gone through the above steps, they are also encoded.

[0058] The present application identifies and represents multiple objects in an image to be encoded, thereby segmenting the image to obtain multiple sub-images, sorting and grouping the multiple sub-images to determine multiple image sequences and mapping relationships, and determining multiple matching pixel blocks based on the image sequence, encoding the sub-images corresponding to the matching pixel blocks, thereby improving the effect of video encoding.

[0059] In some feasible implementation manners, optionally, the step of respectively identifying multiple objects in the image to be encoded to obtain identification information corresponding to each object includes:

[0060] Inputting the image to be encoded into an image recognition model for image recognition to determine the multiple objects included in the image to be encoded;

[0061] Performing region segmentation on the image to be encoded based on the multiple objects to obtain a target region corresponding to each object;

[0062] Objects in each target area are identified respectively to obtain identification information corresponding to each object.

[0064] In this embodiment, after the encoding end obtains the image to be encoded, a deep learning network is used to perform scene recognition on the input image to be encoded, and the objects therein are segmented to obtain multiple target areas, and multiple identification information is obtained by representing the objects included in the multiple target areas. Among them, the deep learning network can be an image recognition model, and the image recognition model is used to identify the objects in the input image, which is not specifically limited in this embodiment. Exemplarily, object IDs from 0 to n are obtained, recorded as ID0...Idn.

[0065] Optionally, the performing image segmentation on the image to be encoded based on the identification information to obtain a sub-image corresponding to each piece of identification information includes:

[0066] Traversing the objects corresponding to each identification information, and determining the marking coordinates of each of the objects;

[0067] Determine a first maximum coordinate and a first minimum coordinate on the first coordinate axis according to the marked coordinates of each of the objects, and determine a second maximum coordinate and a second minimum coordinate on the second coordinate axis;

[0068] Acquire an image pixel region where the object is located according to the first maximum coordinate, the first minimum coordinate, the second maximum coordinate, and the second minimum coordinate;

[0069] The image to be encoded is segmented according to the image pixel area to determine a sub-image corresponding to each of the objects.

[0070] In this embodiment, the coordinates of all markers in the current object are obtained, and the maximum and minimum values ​​of the first coordinate axis in the X direction are obtained as the first maximum coordinate and the first minimum coordinate, i.e., Xmax and Xmin; the maximum and minimum values ​​of the second coordinate axis in the Y direction are obtained as the second maximum coordinate Ymax and the second minimum coordinate Ymin.

[0071] Use the four values ​​of Xmax / Xmin / Ymax / Ymin to obtain the image pixel points at the corresponding coordinate position to obtain the sub-image Pi. Fill the sub-image Pi, that is, use the boundary pixel points at the position marked by IDi to fill the object so that the object is a rectangular area with a minimum size of 16X16, and the length and width of the area are integer multiples of 16. In addition, it is necessary to determine whether there are any remaining images that have not been processed, and finally obtain multiple sub-images.

[0072] Optionally, the sorting and grouping of the plurality of sub-images to obtain a plurality of image sequences and a mapping relationship between each of the image sequences and the plurality of sub-images includes:

[0073] Calculate the number of coded bytes corresponding to each sub-image, the number of occurrences of the same number of coded bytes, and the number of sub-image sets with the largest number of coded bytes, and obtain a weight value corresponding to each sub-image;

[0074] Sort the multiple sub-images from large to small according to the weight value of each sub-image to obtain a target image sequence;

[0075] The multiple sub-images are grouped according to the target image sequence to obtain the multiple image sequences, and a mapping relationship corresponding to each of the image sequences is determined according to the sub-images included in each of the image sequences.

[0076] In this embodiment, the weight value corresponding to each sub-image is calculated. Specifically, the weight of the sub-image of the current id can be obtained by counting the number of bytes encoded by the sub-image of each id, the number of times the same number of encoded bytes appears, and the number of times the sub-image set with the largest number of encoded bytes appears at the top of the pyramid. According to the weight of the sub-image, the current Pi picture is sorted to obtain a new sub-image sequence from PP0 to PPn. And the coordinate mapping Mapi of the current PP image sequence to the P image sequence is saved, and finally multiple image sequences and multiple mapping relationships are obtained.

[0077] Optionally, acquiring, in each of the image sequences, a matching pixel block corresponding to the image sequence, and determining, based on the mapping relationship, a target sub-image corresponding to each of the matching pixel blocks, includes:

[0078] In the plurality of image sequences, for each sub-image, obtaining a macroblock of a target area in each sub-image;

[0079] Determine in each of the sub-images a matching pixel block that best matches the macroblock of the target area of ​​the sub-image;

[0080] Determining a first motion vector and a second motion vector based on a relative displacement between the matching pixel block and the macroblock of the target area;

[0081] Mapping the first motion vector and the second motion vector to the coordinates of the image sequence corresponding to each of the sub-images based on the mapping relationship to obtain a third motion vector and a fourth motion vector corresponding to each of the sub-images;

[0082] A target sub-image corresponding to each matching pixel block is determined according to the third motion vector, the fourth motion vector and the mapping relationship.

[0083] In this embodiment, in the multiple image sequences, for each sub-image, the sub-image is encoded from left to right and from top to bottom. Specifically, a 16x16 macroblock is obtained from the PPI sub-image, and a block that best matches the current block is searched from the current sub-image.

[0084] According to the result of satd, the block with the minimum satd is selected to complete the search result, and the first motion vector and the second motion vector, namely MVix and MViy, are calculated according to the relative displacement between the matching pixel block and the macroblock of the target area.

[0085] The motion vectors MVix and MViy of the current block are mapped to the coordinates of the current sub-picture PPi according to the mapping Mapi to obtain the third motion vector and the fourth motion vector, namely MNixr and MViyr. The third motion vector and the fourth motion vector are matched in each macroblock of the target area to determine the target sub-image corresponding to the matching pixel block, thereby completing the search for the current block.

[0086] Optionally, determining in each of the sub-images a matching pixel block that best matches a macroblock of the target area of ​​the sub-image includes:

[0087] In the case where the sub-image is an instant decoding refresh frame IDR, determining a matching pixel block that best matches the macroblock of the target area according to an image local matching algorithm;

[0088] When the sub-image is not a refresh frame, a matching pixel block that best matches the macroblock of the target area is determined from a reference sub-image of a previous reference frame of the image to be encoded according to an image local matching algorithm, and the reference sub-image has the same identification information as the sub-image.

[0089] In this embodiment, when the current sub-image is instantly decoded as a refresh frame IDR, the matching pixel block that best matches the macroblock of the target area is directly determined based on the image local matching algorithm; when the current sub-image is not a refresh frame, the block that best matches the current block is searched from the PPi picture with the same ID of the previous reference frame, and the encoding process is continued based on the best matching block.

[0090] The present application identifies and represents multiple objects in an image to be encoded, thereby segmenting the image to obtain multiple sub-images, sorting and grouping the multiple sub-images to determine multiple image sequences and mapping relationships, and determining multiple matching pixel blocks based on the image sequence, encoding the sub-images corresponding to the matching pixel blocks, thereby improving the effect of video encoding.

[0091] See also Figure 2 , Figure 2 is a structural diagram of a video encoding device provided in an embodiment of the present application. Figure 2 As shown, the video encoding device 200 includes:

[0092] An identification module 210 is used to identify multiple objects in the image to be encoded respectively, obtain identification information corresponding to each object, and perform image segmentation on the image to be encoded based on the identification information to obtain a sub-image corresponding to each identification information;

[0093] A grouping module 220, configured to sort and group the plurality of sub-images to obtain a plurality of image sequences and a mapping relationship between each of the image sequences and the plurality of sub-images, wherein each of the image sequences includes at least two sub-images;

[0094] An acquisition module 230 is used to acquire, in each of the image sequences, a matching pixel block corresponding to the image sequence, and determine a target sub-image corresponding to each of the matching pixel blocks based on the mapping relationship; wherein the target sub-image is any one of the multiple sub-images, and the matching pixel block is an image region in the image sequence;

[0095] The encoding module 240 is used to encode each of the target sub-images in sequence to obtain a video stream.

[0096] Optionally, the identification module 210 includes:

[0097] A recognition submodule, configured to input the image to be encoded into an image recognition model for image recognition, and determine the multiple objects included in the image to be encoded;

[0098] A first segmentation submodule, configured to perform region segmentation on the image to be encoded based on the multiple objects to obtain a target region corresponding to each object;

[0099] The identification submodule is used to identify each object in the target area respectively to obtain identification information corresponding to each object.

[0100] Optionally, the identification module 210 includes:

[0101] A first determination submodule, configured to traverse the objects corresponding to each identification information and determine the marking coordinates of each object;

[0102] A second determination submodule, configured to determine a first maximum coordinate and a first minimum coordinate on the first coordinate axis according to the marked coordinates of each of the objects, and to determine a second maximum coordinate and a second minimum coordinate on the second coordinate axis;

[0103] A first acquisition submodule, configured to acquire an image pixel region where the object is located according to the first maximum coordinate, the first minimum coordinate, the second maximum coordinate, and the second minimum coordinate;

[0104] The second segmentation submodule is used to segment the image to be encoded according to the image pixel area to determine the sub-image corresponding to each object.

[0105] Optionally, the grouping module 220 includes:

[0106] A calculation submodule, used for calculating the number of coded bytes corresponding to each sub-image, the number of occurrences of the same number of coded bytes and the number of sub-image sets with the largest number of coded bytes, to obtain a weight value corresponding to each sub-image;

[0107] A sorting submodule, used for sorting the plurality of sub-images from large to small according to the weight value of each sub-image, to obtain a target image sequence;

[0108] The grouping submodule is used to group the multiple sub-images according to the target image sequence to obtain the multiple image sequences, and determine the mapping relationship corresponding to each image sequence according to the sub-images included in each image sequence.

[0109] Optionally, the acquisition module 230 includes:

[0110] A second acquisition submodule is used for acquiring a macroblock of a target area in each sub-image in the plurality of image sequences;

[0111] A third determination submodule is used to determine, in each of the sub-images, a matching pixel block that best matches the macroblock of the target area of ​​the sub-image;

[0112] A fourth determination submodule, configured to determine a first motion vector and a second motion vector based on a relative displacement between the matching pixel block and the macroblock of the target area;

[0113] Conversion submodule for

[0114] Mapping the first motion vector and the second motion vector to the coordinates of the image sequence corresponding to each of the sub-images based on the mapping relationship to obtain a third motion vector and a fourth motion vector corresponding to each of the sub-images;

[0116] The fifth determination submodule is used to determine a target sub-image corresponding to each matching pixel block according to the third motion vector, the fourth motion vector and the mapping relationship.

[0117] Optionally, the third determining submodule includes:

[0118] A first determining unit is used to determine, when the sub-image is an instant decoding refresh frame IDR, a matching pixel block that best matches the macroblock of the target area according to an image local matching algorithm;

[0119] The second determination unit is used to determine, when the sub-image is not a refresh frame, a matching pixel block that best matches the macroblock of the target area from a reference sub-image of a previous reference frame of the image to be encoded according to an image local matching algorithm, and the reference sub-image has the same identification information as the sub-image.

[0120] The present application identifies and represents multiple objects in an image to be encoded, thereby segmenting the image to obtain multiple sub-images, sorting and grouping the multiple sub-images to determine multiple image sequences and mapping relationships, and determining multiple matching pixel blocks based on the image sequence, encoding the sub-images corresponding to the matching pixel blocks, thereby improving the effect of video encoding.

[0121] The present application also provides an electronic device. Figure 3 , the electronic device may include a processor 301, a memory 302, and a program 3021 stored in the memory 302 and executable on the processor 301.

[0122] When the program 3021 is executed by the processor 301, it can achieve Figure 1 Any step in the corresponding method embodiment:

[0123] Identify multiple objects in the image to be encoded respectively to obtain identification information corresponding to each object, and segment the image to be encoded based on the identification information to obtain a sub-image corresponding to each identification information;

[0124] Sorting and grouping the plurality of sub-images to obtain a plurality of image sequences and a mapping relationship between each of the image sequences and the plurality of sub-images, wherein each of the image sequences includes at least two of the sub-images;

[0125] Acquire a matching pixel block corresponding to each of the image sequences, and determine a target sub-image corresponding to each of the matching pixel blocks based on the mapping relationship; wherein the target sub-image is any one of the multiple sub-images, and the matching pixel block is an image region in the image sequence;

[0126] Each of the target sub-images is encoded in turn to obtain a video stream.

[0127] Optionally, the step of respectively identifying multiple objects in the image to be encoded to obtain identification information corresponding to each object includes:

[0128] Inputting the image to be encoded into an image recognition model for image recognition to determine the multiple objects included in the image to be encoded;

[0129] Performing region segmentation on the image to be encoded based on the multiple objects to obtain a target region corresponding to each object;

[0130] Objects in each target area are identified respectively to obtain identification information corresponding to each object.

[0131] Optionally, the performing image segmentation on the image to be encoded based on the identification information to obtain a sub-image corresponding to each piece of identification information includes:

[0132] Traversing the objects corresponding to each identification information, and determining the marking coordinates of each of the objects;

[0133] Determine a first maximum coordinate and a first minimum coordinate on the first coordinate axis according to the marked coordinates of each of the objects, and determine a second maximum coordinate and a second minimum coordinate on the second coordinate axis;

[0134] Acquire an image pixel region where the object is located according to the first maximum coordinate, the first minimum coordinate, the second maximum coordinate, and the second minimum coordinate;

[0135] The image to be encoded is segmented according to the image pixel area to determine a sub-image corresponding to each of the objects.

[0136] Optionally, the sorting and grouping of the plurality of sub-images to obtain a plurality of image sequences and a mapping relationship between each of the image sequences and the plurality of sub-images includes:

[0137] Calculate the number of coded bytes corresponding to each sub-image, the number of occurrences of the same number of coded bytes, and the number of sub-image sets with the largest number of coded bytes, and obtain a weight value corresponding to each sub-image;

[0138] Sort the multiple sub-images from large to small according to the weight value of each sub-image to obtain a target image sequence;

[0139] The multiple sub-images are grouped according to the target image sequence to obtain the multiple image sequences, and a mapping relationship corresponding to each of the image sequences is determined according to the sub-images included in each of the image sequences.

[0140] Optionally, acquiring, in each of the image sequences, a matching pixel block corresponding to the image sequence, and determining, based on the mapping relationship, a target sub-image corresponding to each of the matching pixel blocks, includes:

[0141] In the plurality of image sequences, for each sub-image, obtaining a macroblock of a target area in each sub-image;

[0142] Determine in each of the sub-images a matching pixel block that best matches the macroblock of the target area of ​​the sub-image;

[0143] Determining a first motion vector and a second motion vector based on a relative displacement between the matching pixel block and the macroblock of the target area;

[0144] Mapping the first motion vector and the second motion vector to the coordinates of the image sequence corresponding to each of the sub-images based on the mapping relationship to obtain a third motion vector and a fourth motion vector corresponding to each of the sub-images;

[0145] A target sub-image corresponding to each matching pixel block is determined according to the third motion vector, the fourth motion vector and the mapping relationship.

[0146] Optionally, determining in each of the sub-images a matching pixel block that best matches a macroblock of the target area of ​​the sub-image includes:

[0147] In the case where the sub-image is an instant decoding refresh frame IDR, determining a matching pixel block that best matches the macroblock of the target area according to an image local matching algorithm;

[0148] When the sub-image is not a refresh frame, a matching pixel block that best matches the macroblock of the target area is determined from a reference sub-image of a previous reference frame of the image to be encoded according to an image local matching algorithm, and the reference sub-image has the same identification information as the sub-image.

[0149] The present application identifies and represents multiple objects in an image to be encoded, thereby segmenting the image to obtain multiple sub-images, sorting and grouping the multiple sub-images to determine multiple image sequences and mapping relationships, and determining multiple matching pixel blocks based on the image sequence, encoding the sub-images corresponding to the matching pixel blocks, thereby improving the effect of video encoding.

[0150] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, each process of the above-mentioned video encoding method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0151] An embodiment of the present application further provides a computer program product, which is stored in a storage medium. The computer program product is executed by at least one processor to implement the various processes of the above-mentioned video encoding method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0152] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0153] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0154] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A video encoding method, characterized in that: The method comprises: Identify multiple objects in the image to be encoded respectively to obtain identification information corresponding to each object, and segment the image to be encoded based on the identification information to obtain a sub-image corresponding to each identification information; Sorting and grouping the plurality of sub-images to obtain a plurality of image sequences and a mapping relationship between each of the image sequences and the plurality of sub-images, wherein each of the image sequences includes at least two of the sub-images; Acquire a matching pixel block corresponding to each of the image sequences, and determine a target sub-image corresponding to each of the matching pixel blocks based on the mapping relationship; wherein the target sub-image is any one of the multiple sub-images, and the matching pixel block is an image region in the image sequence; Each of the target sub-images is encoded in turn to obtain a video stream.

2. The method according to claim 1, characterized in that The step of respectively identifying multiple objects in the image to be encoded to obtain identification information corresponding to each object includes: Inputting the image to be encoded into an image recognition model for image recognition to determine the multiple objects included in the image to be encoded; Performing region segmentation on the image to be encoded based on the multiple objects to obtain a target region corresponding to each object; Objects in each target area are identified respectively to obtain identification information corresponding to each object.

3. The method according to claim 1, characterized in that The performing image segmentation on the to-be-encoded image based on the identification information to obtain a sub-image corresponding to each piece of identification information includes: Traversing the objects corresponding to each identification information, and determining the marking coordinates of each of the objects; Determine a first maximum coordinate and a first minimum coordinate on the first coordinate axis according to the marked coordinates of each of the objects, and determine a second maximum coordinate and a second minimum coordinate on the second coordinate axis; Acquire an image pixel region where the object is located according to the first maximum coordinate, the first minimum coordinate, the second maximum coordinate, and the second minimum coordinate; The image to be encoded is segmented according to the image pixel area to determine a sub-image corresponding to each of the objects.

4. The method according to claim 1, characterized in that: The step of sorting and grouping the plurality of sub-images to obtain a plurality of image sequences and a mapping relationship between each of the image sequences and the plurality of sub-images includes: Calculate the number of coded bytes corresponding to each sub-image, the number of occurrences of the same number of coded bytes, and the number of sub-image sets with the largest number of coded bytes, and obtain a weight value corresponding to each sub-image; Sort the multiple sub-images from large to small according to the weight value of each sub-image to obtain a target image sequence; The multiple sub-images are grouped according to the target image sequence to obtain the multiple image sequences, and a mapping relationship corresponding to each of the image sequences is determined according to the sub-images included in each of the image sequences.

5. The method according to claim 1, characterized in that The obtaining, in each of the image sequences, a matching pixel block corresponding to the image sequence, and determining, based on the mapping relationship, a target sub-image corresponding to each of the matching pixel blocks, comprises: In the plurality of image sequences, for each sub-image, obtaining a macroblock of a target area in each sub-image; Determine in each of the sub-images a matching pixel block that best matches the macroblock of the target area of ​​the sub-image; Determining a first motion vector and a second motion vector based on a relative displacement between the matching pixel block and the macroblock of the target area; Mapping the first motion vector and the second motion vector to the coordinates of the image sequence corresponding to each of the sub-images based on the mapping relationship to obtain a third motion vector and a fourth motion vector corresponding to each of the sub-images; A target sub-image corresponding to each matching pixel block is determined according to the third motion vector, the fourth motion vector and the mapping relationship.

6. The method according to claim 5, characterized in that The step of determining in each of the sub-images a matching pixel block that best matches a macroblock of the target area of ​​the sub-image comprises: In the case where the sub-image is an instant decoding refresh frame IDR, determining a matching pixel block that best matches the macroblock of the target area according to an image local matching algorithm; When the sub-image is not a refresh frame, a matching pixel block that best matches the macroblock of the target area is determined from a reference sub-image of a previous reference frame of the image to be encoded according to an image local matching algorithm, and the reference sub-image has the same identification information as the sub-image.

7. A video encoding device, characterized in that: The device comprises: An identification module, used to identify multiple objects in the image to be encoded respectively, obtain identification information corresponding to each object, and perform image segmentation on the image to be encoded based on the identification information to obtain a sub-image corresponding to each identification information; a grouping module, used for sorting and grouping the plurality of sub-images to obtain a plurality of image sequences and a mapping relationship between each of the image sequences and the plurality of sub-images, wherein each of the image sequences includes at least two of the sub-images; an acquisition module, configured to acquire, in each of the image sequences, a matching pixel block corresponding to the image sequence, and determine, based on the mapping relationship, a target sub-image corresponding to each of the matching pixel blocks; wherein the target sub-image is any one of the multiple sub-images, and the matching pixel block is an image region in the image sequence; The encoding module is used to encode each of the target sub-images in turn to obtain a video stream.

8. An electronic device, comprising: A memory, a processor, and a program stored in the memory and executable on the processor; wherein the processor is used to read the program in the memory to implement the steps in the video encoding method as described in any one of claims 1 to 6.

9. A readable storage medium for storing a program, characterized in that: When the program is executed by a processor, the steps in the video encoding method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program product is stored in a storage medium, and the computer program product is executed by at least one processor to implement the steps in the video encoding method according to any one of claims 1 to 6.

Images