A block mask based semantic structured image coding method and system

Abstract

The application discloses a semantic structured image coding and decoding method and system based on a block mask, and the image is divided in the mode of the block mask, which is more flexible and changeable, has stronger controllability and expandability, and compared with the original semantic structured coding method, the application can keep coding efficiency and improve flexibility when coding the image of the object overlapping or dense scene by using the semantic structured image coding technology.

Description

Technical Field

[0001] This invention relates to the field of image compression coding technology, and in particular to a semantically structured image encoding and decoding method and system based on block masks. Background Technology

[0002] Existing image compression technologies primarily target human visual image compression. However, with the rapid development of deep learning, machine intelligence analysis is playing an increasingly important role in various fields of human society's production and life. Traditional human-oriented compression methods require compressing and transmitting all information of an image. At the intelligent task analysis end, the entire image needs to be decoded to obtain a complete image before being fed into the subsequent intelligent task analysis model. To more efficiently support human-machine hybrid intelligent applications, existing methods have proposed the concept of image semantic structured bitstreams. For example, Scheme 1: Chinese invention patent CN110225341B, "A Task-Driven Bitstream Structured Image Encoding Method," introduces a region decision network and alignment module for object detection. Based on compressed features, it extracts bounding boxes of the regions where objects exist and performs spatial segmentation of features according to the bounding boxes. The segmented features are then sequentially entropy-encoded to form a structured bitstream.

[0003] However, in many practical applications such as autonomous driving and smart cities, the images to be processed often contain overlapping or even densely packed objects. Solution 1, when processing such images, often uses direct spatial segmentation of the image or compressed features based on detection results. This operation leads to repeated encoding of overlapping regions, which severely impacts encoding efficiency when the overlapping area is too large or the objects are densely packed.

[0004] Solution 2: Chinese invention patent application CN112929662A, entitled "Encoding Method for Solving the Object Overlap Problem in Structured Image Coding Methods," adopts a method of taking the bounding rectangle of overlapping objects, encoding them, and then placing them as a whole segment of the bitstream within the structured bitstream. The problem with this solution is that the bounding rectangles of different objects may also contain a large amount of background information not within the target rectangle, reducing the encoding efficiency for specific intelligent tasks. Summary of the Invention

[0005] The purpose of this invention is to provide a semantically structured image encoding and decoding method and system based on block masks. By using block masks to perform semantically structured encoding on images, the semantically structured image encoding maintains high efficiency and flexibility even when objects overlap or are densely packed in the image.

[0006] The objective of this invention is achieved through the following technical solution:

[0007] A semantically structured image encoding and decoding method based on block masks, comprising:

[0008] The encoding section performs object detection on the input image, obtaining object detection results and instance segmentation results. A block mask is generated by combining predefined block size information, and the block mask is used to distinguish the group to which the target object belongs. Image features of the input image are obtained using the transform operation of a depth image encoder. These image features undergo a priori transformation and quantization to obtain quantized priori features. The quantized priori features undergo an inverse priori transformation to obtain the overall probability distribution. The size information of the input image, the quantized priori features, the object detection results, the block size information, and the block mask are entropy encoded separately and then concatenated to obtain the bitstream header information. The image features are quantized to obtain quantized image features. These quantized image features are then grouped spatially using the block mask, with each group representing a feature of that group, called group features. Entropy encoding is performed on all group features based on the overall probability distribution. All specified groups are selected according to the task settings, and the entropy encoded streams corresponding to all specified groups are combined to form the texture portion of the bitstream. The bitstream header information and the texture portion of the bitstream constitute a semantically structured bitstream.

[0009] Decoding section: The code stream header information in the semantically structured code stream is decoded to obtain the input image size information, quantized prior features, object detection results, block size information, and block mask; the quantized prior features are subjected to inverse prior transform to obtain the overall probability distribution; the code stream corresponding to each specified group is extracted from the texture part, and combined with the overall probability distribution, entropy decoding is performed to obtain the group features corresponding to each specified group; then, combined with the block mask, all the group features corresponding to the specified groups are recombined into reconstructed quantized image features; the reconstructed image is obtained by combining the input image size information and the reconstructed quantized image features through the inverse transform operation of the depth image decoder.

[0010] A semantically structured image encoding and decoding system based on block masks, comprising:

[0011] An encoding unit is used to execute an encoding section, which includes: performing object detection on an input image, obtaining object detection results and instance segmentation results, and generating a block mask by combining predefined block size information, using the block mask to distinguish the group to which the target object belongs; obtaining image features of the input image using the transform operation of a depth image encoder, performing a priori transform and quantization on the image features to obtain quantized prior features, performing an inverse priori transform on the quantized prior features to obtain the overall probability distribution, and combining the size information of the input image, quantized prior features, and object detection... The measurement results, block size information, and block mask are entropy encoded separately and then concatenated to obtain the bitstream header information. Furthermore, the image features are quantized to obtain quantized image features. These quantized image features are then grouped spatially using the block mask, with each group representing a feature category, referred to as a category feature. Entropy encoding is performed on all category features based on the overall probability distribution. All specified categories are selected according to the task settings, and the entropy encoded streams corresponding to all specified categories are combined to form the texture portion of the bitstream. The bitstream header information and the texture portion of the bitstream constitute a semantically structured bitstream.

[0012] A decoding unit is used to execute a decoding section, which includes: decoding the bitstream header information in the semantically structured bitstream to obtain the input image's size information, quantized prior features, target detection results, block size information, and block mask; performing an inverse prior transform on the quantized prior features to obtain the overall probability distribution; extracting the bitstream corresponding to each specified group from the texture portion, combining it with the overall probability distribution, performing entropy decoding to obtain the group features corresponding to each specified group, and then combining it with the block mask to reconstruct all the group features corresponding to the specified groups into reconstructed quantized image features; and combining the input image's size information and the reconstructed quantized image features to obtain the reconstructed image through an inverse transform operation of a depth image decoder.

[0013] A processing device includes: one or more processors; and a memory for storing one or more programs;

[0014] When the one or more programs are executed by the one or more processors, the one or more processors implement the aforementioned method.

[0015] A readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned method.

[0016] As can be seen from the technical solutions provided by the present invention, the method of introducing block masks to divide images is more flexible and versatile, and has stronger controllability and scalability. Compared with the original semantic structured coding method (i.e., the aforementioned Scheme 1 and Scheme 2), the present invention can maintain coding efficiency and improve flexibility when using semantic structured image coding technology to encode images of overlapping or even dense scenes. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A framework diagram of a semantically structured image encoding and decoding method based on block masks provided in an embodiment of the present invention;

[0019] Figure 2 This is a schematic diagram illustrating the intelligent analysis of an input image and the generation of a corresponding block mask, provided by an embodiment of the present invention.

[0020] Figure 3 This is a schematic diagram of the input image and the target detection and instance segmentation results provided in an embodiment of the present invention;

[0021] Figure 4 This is a schematic diagram illustrating how overlapping targets are stitched together as a group based on target detection results, as provided in an embodiment of the present invention.

[0022] Figure 5 This is a schematic diagram illustrating how overlapping targets are grouped together based on instance segmentation results, as provided in an embodiment of the present invention.

[0023] Figure 6 This is a schematic diagram illustrating how overlapping targets are divided into different groups based on instance segmentation results, as provided in an embodiment of the present invention.

[0024] Figure 7 A schematic diagram of a semantically structured image encoding and decoding system based on block masks provided in an embodiment of the present invention;

[0025] Figure 8 This is a schematic diagram of a processing device provided in an embodiment of the present invention. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention.

[0027] First, the following explanations are provided for the terms that may be used in this article:

[0028] The term "and / or" means that either or both can be achieved simultaneously. For example, X and / or Y means that it includes both "X" or "Y" as well as the three cases of "X and Y".

[0029] The terms “including,” “comprising,” “containing,” “having,” or other similar semantic descriptions should be interpreted as non-exclusive inclusion. For example, “including a technical feature element (such as raw material, component, ingredient, carrier, dosage form, material, size, part, component, mechanism, device, step, process, method, reaction conditions, processing conditions, parameter, algorithm, signal, data, product or article of manufacture, etc.)” should be interpreted as including not only the expressly listed technical feature element, but also other technical feature elements that are not expressly listed and are well-known in the art.

[0030] The term "composed of" excludes any technical features not expressly listed. When used in a claim, it closes the claim to exclude all technical features other than those expressly listed, except for associated conventional impurities. If the term appears only in a clause of a claim, it limits the claim to the elements expressly listed in that clause; elements recited in other clauses are not excluded from the overall claim.

[0031] The following is a detailed description of a semantically structured image encoding and decoding method and system based on block masks provided by this invention. Contents not described in detail in the embodiments of this invention are prior art known to those skilled in the art. Where specific conditions are not specified in the embodiments of this invention, they are performed according to conventional conditions in the art or conditions recommended by the manufacturer. Where the manufacturers of reagents or instruments used in the embodiments of this invention are not specified, they are all conventional products that can be purchased commercially.

[0032] Example 1

[0033] This invention provides a semantically structured image encoding and decoding method based on block masks. Figure 1 The overall framework of the method is shown, mainly including the encoding and decoding parts.

[0034] I. Encoding Section.

[0035] 1. Perform target detection on the input image, obtain the detection results, and generate a block mask by combining the predefined block size information. Use the block mask to distinguish the group to which the target object belongs.

[0036] In this embodiment of the invention, the size of the input image is denoted as H×W×C, where H and W represent the height and width of the input image, respectively, and C is the number of channels (all channels are encoded and decoded simultaneously); the predefined block size information is denoted as B, which represents the side length of the image block, and the size of the image block is B×B.

[0037] In this embodiment of the invention, the input image is processed by image intelligent analysis algorithms such as object detection to obtain detection results. The detection results include: object detection results and instance segmentation results; wherein, the object detection results include: the number of object objects, the position of each object object, and its category; the position of each object object includes: the horizontal axis position of the upper left corner coordinate, the vertical axis position of the upper left corner coordinate, the height, and the width; the instance segmentation results are the edge contours of each object object.

[0038] In this embodiment of the invention, a block mask m is generated by combining the detection results (target detection results and instance segmentation results) with predefined block size information. The size of the block mask m is... In a grayscale image, each pixel has a value between 0 and 255. The value of each pixel in the block mask represents the group to which its corresponding image block belongs. Figure 2 The image shown is an example of intelligently analyzing an input image and generating a corresponding block mask. Figure 2 The left side shows the target detection results and instance segmentation results, and the right side shows the generated block mask; where: each rectangle on the left is the target detection result, and the text in the rectangle contains the category and confidence level. This embodiment of the invention mainly considers the category; the edge contours of each target object on the left are the instance segmentation results.

[0039] In this embodiment of the invention, when generating a block mask by combining the detection results (target detection results and instance segmentation results) with predefined block size information: overlapping targets can be grouped together according to the target detection results as the same target object; overlapping targets can also be grouped together according to the instance segmentation results as the same target object; or overlapping targets can be divided into different groups according to their respective instance segmentation results as different target objects. Of course, this invention does not limit the block mask generation method and can be used to select a specified generation method according to the actual situation, and generate a block mask by combining the detection results with predefined block size information.

[0040] In this embodiment of the invention, the instance segmentation result is mainly used when generating the block mask and is not encoded. The target detection result is needed when generating the block mask, and it is also needed by downstream tasks. Therefore, it also needs to be encoded.

[0041] 2. Use the transformation operation of the depth image encoder to obtain the image features of the input image.

[0042] In this embodiment of the invention, the image feature is denoted as y, and the size is... Among them, C y The transformation operations of the depth image encoder involved are implemented with reference to conventional techniques, and will not be elaborated upon in this invention.

[0043] 3. Perform a priori transformation and quantization on the image features to obtain quantized priori features. Perform an inverse priori transformation on the quantized priori features to obtain the overall probability distribution. Perform entropy encoding on the size information of the input image, quantized priori features, target detection results, block size information, and block mask respectively, and then concatenate them to obtain the bitstream header information.

[0044] like Figure 1 As shown on the right and upper left, this part involves two entropy encoding components. The first part entropy encoding is performed on the input image's size information, quantized prior features, and object detection results. Here, entropy encoding of the object detection results separately refers to entropy encoding the number of object objects, as well as the position and category of each object. The second part entropy encoding is performed on the block size information and the block mask separately. Afterwards, the components are concatenated to obtain the bitstream header information. The syntax structure of the bitstream header information is defined in Table 1.

[0045] Table 1: Syntax Structure of Code Stream Header Information

[0046]

[0047] Where: image_height_minus1 represents the image height H; image_width_minus1 represents the image width W; side_information_length represents the quantized super-prior features. The corresponding bitstream length; group_mask_block_size indicates the block size information; group_mask_length_minus1 indicates the bitstream length corresponding to block mask m; bounding_boxes_numbers indicates the number of target objects in the image; bounding_box_x, bounding_box_y, bounding_box_h, bounding_box_w, and bounding_box_category respectively represent the horizontal coordinate, vertical coordinate, height, width, and category information of the upper left corner of the current target object. Only an example of a single target object is provided here. The above 5 information of multiple target objects are grouped together and arranged in order; u indicates an unsigned data type. Taking u(32) as an example, it indicates that the length of the bitstream segment is 32 bits.

[0048] In this embodiment of the invention, the prior transformation can be implemented using conventional techniques, which will not be elaborated here.

[0049] In this embodiment of the invention, the bitstream length is used for subsequent decoding. The principle is as follows: during actual entropy encoding, the length of the data after encoding cannot be known. However, during decoding, the bitstream of the corresponding length needs to be read first before decoding can be performed. Therefore, the corresponding bitstream length needs to be added before storing the bitstream after entropy encoding is completed, so that the decoding end can correctly decode the data.

[0050] In addition, the quantized prior features need to be subjected to an inverse prior transformation to obtain the overall probability distribution. It is then provided for use in group feature entropy encoding.

[0051] 4. Quantize the image features to obtain quantized image features. Then, by combining block masks, the quantized image features are grouped spatially. The features of each group are called group features, which are then combined with the overall probability distribution. Entropy encoding is performed on all group features separately. All specified groups are selected according to the task settings, and the entropy encoded streams corresponding to all specified groups are combined to form the texture part of the bitstream.

[0052] In this embodiment of the invention, the overall probability distribution This refers to the overall probability distribution of quantified image features, which can be obtained through the overall probability distribution. To obtain the probability distribution for each group, specifically: for the k-th group, first, based on the grouping situation and the overall probability distribution... By combining this with an autoregressive model, the corresponding probability distribution is obtained. Then, the entropy model is combined to analyze the corresponding group characteristics. Perform entropy encoding. Repeat the above steps for all groups to obtain all target entropy-encoded bitstreams.

[0053] In this embodiment of the invention, the grouping is based on: the parts with the same value on the block mask m and the quantized image features. The corresponding parts in the spatial dimension are grouped together and called group features. The required groups can be determined according to the needs of downstream tasks. There can be one or more downstream tasks, and the required groups for different downstream tasks can be the same or different. The required number of groups can be less than or equal to the total number of groups. Here, all specified groups are selected according to the settings of the downstream tasks. The entropy encoded streams corresponding to all specified groups are combined to form the texture portion of the bitstream. During combination, the groups are arranged in ascending order according to the value of their corresponding index k, forming the texture portion of the bitstream.

[0054] The semantic structure definition of the texture part is shown in Table 2.

[0055] Table 2: Syntax Structure of Texture Part

[0056]

[0057] Here, `object_texture_length_minus1` represents the bitstream length corresponding to the texture portion of the currently specified group. It should be noted that the above syntax structure only provides examples for a single specified group.

[0058] 5. The bitstream header information and the texture portion of the bitstream constitute a semantically structured bitstream.

[0059] In this embodiment of the invention, the syntactic structure of the semantically structured bitstream includes: the height of the input image, the width of the input image, the bitstream length corresponding to the quantized prior features, block size information, the bitstream length corresponding to the block mask, the number of target objects, the position and category of each target object, and the bitstream length corresponding to the texture portion of all specified groups.

[0060] II. Decoding section.

[0061] 1. Header information decoding. Decode the header information of the semantically structured bitstream to obtain the size information of the input image, quantized prior features, object detection results, block size information, and block mask.

[0062] In this embodiment of the invention, the size information of the input image mainly refers to the height H and width W of the image. The block size information is used to ensure the correct decoding of the block mask. Specifically, the block size information is decoded first, and then the block mask is decoded in combination with the block size information. The target detection results can be used for subsequent downstream tasks.

[0063] 2. Inverse Prior Transform. An inverse prior transform is performed on the quantized prior features to obtain the overall probability distribution. Used for decoding subsequent group features.

[0064] In this embodiment of the invention, the inverse prior transform can be implemented using conventional techniques, which will not be elaborated here.

[0065] 3. Extract the bitstream corresponding to each specified group from the texture portion, and combine it with the overall probability distribution. Entropy decoding is performed to obtain the group features corresponding to each specified group. Specifically, this involves combining the block mask m obtained from decoding with the overall probability distribution. The probability distribution corresponding to each specified group is obtained by using an autoregressive model, and the group features corresponding to each specified group are obtained by entropy decoding.

[0066] Considering the overall probability distribution obtained from the decoding part Similar to the aforementioned encoding section, and furthermore, in both the encoding and decoding sections, the overall probability distribution is utilized. The method for obtaining the probability distribution for each group is also the same, therefore, Figure 1 The relevant process is summarized and represented, omitting the part from quantization → inverse prior transformation → probability estimation. Of course, this is also the common way to represent the relevant process in this field.

[0067] 4. Feature recombination.

[0068] Based on the position provided by the block mask m, the group features corresponding to all specified groups are collectively recombined into recombined quantized image features.

[0069] Considering that a specified group can be a part of all groups, i.e., recombining quantized image features. Number of groups and quantized image features The symbols and names are different, therefore, they are distinguished. Of course, if the specified group includes all groups, then the image features are reconstructed and quantized. Equivalent to quantizing image features

[0070] 5. Combining the size information of the input image with the features of the reconstructed quantized image, the reconstructed image is obtained through the inverse transform operation of the depth image decoder.

[0071] The inverse transformation operations involved in this section can be referred to conventional techniques and will not be elaborated here.

[0072] The above-mentioned solutions provided by the embodiments of the present invention have the following main advantages:

[0073] (1) Based on the semantic structured image coding framework, a method for semantic structured image coding using block masks is proposed, which enables semantic structured image coding to maintain efficiency and flexibility when objects overlap or are dense in the image.

[0074] 2) Compared to the method in Scheme 1, which relies on rectangular boxes to divide the image into regions, this invention uses block masks to divide the image, which is more flexible and versatile, and has stronger controllability and scalability.

[0075] The following section introduces different block mask generation methods to visually demonstrate the advantages of stream structuring flexibility when target objects have overlapping parts.

[0076] like Figure 3 As shown, the object detection results and instance segmentation results are presented, where Figure 3 The left side shows the input image, and the right side shows the target detection results (rectangular boxes) and instance segmentation results (edge ​​expansion).

[0077] Figures 4-6 Three block mask partitioning methods are provided, where each rectangular block represents the B×B image block mentioned above; specifically:

[0078] like Figure 4 As shown, overlapping targets are stitched together into a group based on the target detection results. The dark part is the foreground part containing the person and the umbrella (both are target objects), and the light part is the background part (the use of light and dark colors in the figure is only for easy display. In actual operation, the pixel value of the light part is 0 and the pixel value of the dark part is 1). This method is similar to the idea of ​​Scheme 2, but it does not need to rely on the bounding rectangle containing two targets, thus avoiding the redundancy of compressing part of the background.

[0079] like Figure 5 As shown, overlapping targets are grouped together based on instance segmentation results. This method... Figure 2 Based on this, it can further reduce the redundancy of the background part in the rectangular box of target detection.

[0080] like Figure 6 As shown, overlapping targets are divided into different groups (dark parts and diagonally filled parts) according to the segmentation results of their respective instances and placed in the structured bitstream. This method further ensures the independence of each target in the structured bitstream and can better serve application scenarios where downstream tasks only need a portion of the categories.

[0081] Figures 4-6In the diagram, the left side represents the image where the input image and the block mask overlap, while the right side represents the image of the block mask alone. It should be noted that the block mask generation method is not limited to the three methods mentioned above; users can also optimize the block mask generation process according to their own needs, which demonstrates the flexibility and efficiency of this solution.

[0082] Example 2

[0083] This invention also provides a semantically structured image encoding and decoding system based on block masks, which is mainly implemented based on the methods provided in the foregoing embodiments, such as... Figure 7 As shown, the system mainly includes:

[0084] An encoding unit is used to execute an encoding section, which includes: performing object detection on an input image, obtaining object detection results and instance segmentation results, and generating a block mask by combining predefined block size information, using the block mask to distinguish the group to which the target object belongs; obtaining image features of the input image using the transform operation of a depth image encoder, performing a priori transform and quantization on the image features to obtain quantized prior features, performing an inverse priori transform on the quantized prior features to obtain the overall probability distribution, and combining the size information of the input image, quantized prior features, and object detection... The measurement results, block size information, and block mask are entropy encoded separately and then concatenated to obtain the bitstream header information. Furthermore, the image features are quantized to obtain quantized image features. These quantized image features are then grouped spatially using the block mask, with each group representing a feature category, referred to as a category feature. Entropy encoding is performed on all category features based on the overall probability distribution. All specified categories are selected according to the task settings, and the entropy encoded streams corresponding to all specified categories are combined to form the texture portion of the bitstream. The bitstream header information and the texture portion of the bitstream constitute a semantically structured bitstream.

[0085] A decoding unit is used to execute a decoding section, which includes: decoding the bitstream header information in the semantically structured bitstream to obtain the input image's size information, quantized prior features, target detection results, block size information, and block mask; performing an inverse prior transform on the quantized prior features to obtain the overall probability distribution; extracting the bitstream corresponding to each specified group from the texture portion, combining it with the overall probability distribution, performing entropy decoding to obtain the group features corresponding to each specified group, and then combining it with the block mask to reconstruct all the group features corresponding to the specified groups into reconstructed quantized image features; and combining the input image's size information and the reconstructed quantized image features to obtain the reconstructed image through an inverse transform operation of a depth image decoder.

[0086] Those skilled in the art will understand that, for the sake of convenience and brevity, the above-described division of functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the system can be divided into different functional modules to complete all or part of the functions described above.

[0087] Example 3

[0088] The present invention also provides a processing device, such as Figure 8 As shown, it mainly includes: one or more processors; a memory for storing one or more programs; wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the method provided in the foregoing embodiments.

[0089] Furthermore, the processing device also includes at least one input device and at least one output device; in the processing device, the processor, memory, input device, and output device are connected via a bus.

[0090] In this embodiment of the invention, the specific types of the memory, input device, and output device are not limited; for example:

[0091] Input devices can be touchscreens, image acquisition devices, physical buttons, or mice, etc.

[0092] The output device can be a display terminal;

[0093] The memory can be random access memory (RAM) or non-volatile memory, such as disk storage.

[0094] Example 4

[0095] The present invention also provides a readable storage medium storing a computer program that, when executed by a processor, implements the method provided in the foregoing embodiments.

[0096] In this embodiment of the invention, the readable storage medium is a computer-readable storage medium and can be disposed in the aforementioned processing device, for example, as a memory in the processing device. Furthermore, the readable storage medium can also be any medium capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), magnetic disk, or optical disk.

[0097] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A block mask based semantic structured image coding method, characterized by, The method comprises: The encoding part: target detection is performed on the input image to obtain target detection results and instance segmentation results, and a block mask is generated in combination with predefined block size information, and the block mask is used to distinguish the group to which a target object belongs, wherein when the block mask is generated: overlapping targets are spliced into a group as a same target object according to the target detection results; or the overlapping targets are spliced into a group as a same target object according to the instance segmentation results; or the overlapping targets are divided into different groups as different target objects according to respective instance segmentation results; an image feature of the input image is obtained by using a transformation operation of a depth image encoder, the image feature is subjected to hyper-prior transformation and quantization to obtain quantized hyper-prior features, the quantized hyper-prior features are subjected to hyper-prior inverse transformation to obtain an overall probability distribution, and size information of the input image, the quantized hyper-prior features, the target detection results, the block size information and the block mask are respectively subjected to entropy encoding, and then spliced to obtain bitstream header information; And the image feature is quantized to obtain quantized image features, and the quantized image features are grouped in a spatial dimension in combination with the block mask, each group being a group feature, referred to as a group feature, and all group features are respectively subjected to entropy encoding in combination with the overall probability distribution, all specified groups are selected according to a task setting, and entropy encoding streams corresponding to all specified groups are combined to form a texture part of a bitstream; the bitstream header information and the texture part of the bitstream constitute a semantic structured bitstream; The decoding part: the bitstream header information in the semantic structured bitstream is decoded to obtain size information of the input image, quantized hyper-prior features, target detection results, block size information and a block mask; the quantized hyper-prior features are subjected to hyper-prior inverse transformation to obtain an overall probability distribution; the bitstream corresponding to each specified group is taken out from the texture part, and the group feature corresponding to each specified group is obtained by entropy decoding in combination with the overall probability distribution, and then all group features corresponding to all specified groups are collectively recombined into recombined quantized image features in combination with the block mask; and a reconstructed image is obtained by inverse transformation operation of a depth image decoder in combination with size information of the input image and the recombined quantized image features.

2. The block mask based semantic structured image coding method according to claim 1, characterized in that, The target detection on the input image to obtain the target detection results and the instance segmentation results, and the generation of the block mask in combination with the predefined block size information comprises: The size of the input image is denoted as HxWxC, wherein H and W represent the height and width of the input image respectively, and C is the number of channels; the predefined block size information is denoted as B, which represents the side length of an image block, and the size of the image block is BxB; The block mask m is generated according to the target detection result, the instance segmentation result and the predefined block size information, and the size of the block mask m is a gray image, the value of each pixel is an integer from 0 to 255, and the value of each pixel in the block mask represents the group to which the corresponding image block belongs.

3. The block mask based semantic structured image coding method according to claim 1 or 2, characterized in that, The target detection results comprise: the number of target objects, the position and category of each target object; the position of each target object comprises: the horizontal position of the upper left corner coordinate of the target object, the vertical position of the upper left corner coordinate, the height and the width.

4. The semantic structured image encoding and decoding method based on a block mask according to claim 3, wherein The entropy encoding of the target detection results respectively means that the number of target objects, and the position and category of each target object are respectively subjected to entropy encoding. After entropy encoding each group feature respectively, according to different task settings, all specified groups required are selected, and the texture part of the code stream is formed by arranging the index values corresponding to the specified groups in ascending order from small to large.

5. The block mask based semantic structured image coding method of claim 2, wherein, The syntax structure of the semantic structured code stream includes: the height of the input image, the width of the input image, the code stream length corresponding to the quantized hyper-prior feature, the block size information, the code stream length corresponding to the block mask, the number of target objects, the position and category of each target object, and the code stream length corresponding to the texture part corresponding to all specified groups.

6. A block mask based semantic structured image coding system, characterized by, The system is implemented based on the method of any one of claims 1-5. The encoding unit is configured to perform the encoding part, and the encoding part includes: performing target detection on the input image to obtain a target detection result and an instance segmentation result, and generating a block mask in combination with predefined block size information to distinguish the groups to which the target objects belong through the block mask, wherein, when the block mask is generated: overlapping targets are spliced into a group as a same target object according to the target detection result; or the overlapping targets are spliced into a group as a same target object according to the instance segmentation result; or the overlapping targets are divided into different groups as different target objects according to respective instance segmentation results; an image feature of the input image is obtained by using a transformation operation of a deep image encoder, the image feature is subjected to hyper-prior transformation and quantization to obtain a quantized hyper-prior feature, the quantized hyper-prior feature is subjected to hyper-prior inverse transformation to obtain an overall probability distribution, and the size information of the input image, the quantized hyper-prior feature, the target detection result, the block size information, and the block mask are respectively subjected to entropy encoding to obtain code stream header information; and the image feature is quantized to obtain a quantized image feature, the quantized image feature is grouped in a spatial dimension in combination with the block mask, each group is a group feature, referred to as a group feature, all group features are respectively subjected to entropy encoding in combination with the overall probability distribution, all specified groups are selected according to a task setting, and a texture part of a code stream is formed by combining the entropy encoded streams corresponding to all specified groups in ascending order from small to large according to the numerical values of the indexes corresponding to the specified groups. The decoding unit is configured to perform the decoding part, and the decoding part includes: decoding the code stream header information in the semantic structured code stream to obtain the size information of the input image, the quantized hyper-prior feature, the target detection result, the block size information, and the block mask; performing hyper-prior inverse transformation on the quantized hyper-prior feature to obtain an overall probability distribution; taking out the code stream corresponding to each specified group from the texture part, combining the overall probability distribution, and performing entropy decoding to obtain the group feature corresponding to each specified group, and then recombining the group features corresponding to all specified groups into a recombined quantized image feature in combination with the block mask; and combining the size information of the input image and the recombined quantized image feature to obtain a reconstructed image through an inverse transformation operation of a deep image decoder.

7. A processing device, characterized by The system includes: one or more processors; a memory for storing one or more programs; wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to carry out the method of any one of claims 1-5.

8. A readable storage medium, storing a computer program, characterized in that, The computer program, when executed by the processor, carries out the method of any one of claims 1-5.

Images