A pixel block arrangement method and system for gigapixel

Abstract

This invention discloses a pixel block arrangement method and system for billion-pixel images. The method includes: inputting the billion-pixel image to be labeled into an inference model and constructing a pixel block tree using initial pixel blocks; performing pixel block filtering based on a pixel block filtering model and the pixel block tree to arrange adjacent fine-grained pixel blocks into corresponding coarse-grained pixel blocks; obtaining a preset number of canvases based on the remaining fine-grained pixel blocks, the corresponding coarse-grained pixel blocks, and a pixel block packing model, and generating final layout information of the packed pixel blocks in the preset number of canvases; optimizing the model based on the final layout information of the packed pixel blocks to obtain a trained inference model; and outputting the labeled data of the billion-pixel image to be labeled and the labels of the labeled data based on the trained inference model. The pixel block arrangement framework proposed in this invention significantly improves the inference speed for billion-pixel images while maintaining ideal performance.

Description

Technical Field

[0001] This invention relates to the field of image classification technology, and in particular to a method and system for arranging pixel blocks for billions of pixels. Background Technology

[0002] With the widespread use of high-resolution cameras, the demand for object detection on high-resolution images is increasing. Aerial and remote sensing images typically have relatively high resolutions; for example, images in the VisDrone dataset can reach 2,000 × 1,500 pixels. The recently released PANDA dataset is the first billion-pixel (25,000 × 14,000) human-centered video and image dataset, further expanding the frontiers of high-resolution image analysis while also posing a significant challenge to accelerating object detection on such large images.

[0003] A major direction for accelerating object detection is designing efficient network architectures, such as Faster R-CNN, YOLO, and SSD. However, most frameworks are developed for general images like MSCOCO. Directly applying them to super-resolution images can still incur significant time costs. Furthermore, research on high-speed inference frameworks and specific methods for super-resolution images is limited.

[0004] Therefore, in recent years, some works have been proposed to accelerate object detection in relatively super-resolution images. One team proposed an image-level solution that adaptively selects the resolution for each input image. It is well known that we can only perform necessary computations on local regions, not on the entire super-resolution image. Based on this intuition, most existing works focus on finding local pixel patches where objects may exist. Other teams treat low-resolution detection boxes as pixel patches and perform final detection on these patches at a refined resolution. Furthermore, CRENet clusters coarse detection results into pixel patches, clusterdet follows the idea of ​​RPN and uses neural networks to estimate accurate patches, AutoFocus and GLSAN generate pixel patches based on image features, and DMNe obtains pixel patches guided by density maps. There are also some methods that use reinforcement learning to find valuable pixel patches. However, the above methods usually require a large number of pixel patches to ensure performance. As image resolution increases, when the image resolution reaches the billion-pixel level, the number of pixel patches will be very large (possibly exceeding 12,000). Therefore, their corresponding time costs are still unacceptable in practical applications. Summary of the Invention

[0005] The present invention aims to at least partially solve one of the technical problems in the related art.

[0006] To address the issue that current pixel block arrangement methods in image detection models, which rely on modular design to address the high time cost of independently detecting each local pixel block in billion-pixel images, focus on detecting potential target locations. This invention proposes a novel pixel block arrangement framework for fast target detection in billion-pixel images. Within this framework, a pixel tree is used to accelerate detection and determine which pixel blocks can be grouped together for training. Pixel block filtering and grouping modules generate an ideal pixel block layout, ultimately improving inference speed for billion-pixel images while maintaining good performance.

[0007] Another object of the present invention is to provide a pixel block arrangement system for billion pixels.

[0008] To achieve the above objectives, the present invention provides a pixel block arrangement method for one billion pixels, comprising:

[0009] Obtain the billion-pixel image to be labeled and its corresponding image label, as well as the inference model; wherein, the inference model includes a pixel block tree generation model, a pixel block filtering model, and a pixel block packing model;

[0010] The billion-pixel image to be labeled is input into the inference model to cluster the initial pixel blocks based on the pixel block tree generation model and the image labels, and the initial pixel blocks are used to construct a pixel block tree; wherein, the pixel block tree includes fine-grained pixel blocks and corresponding coarse-grained pixel blocks;

[0011] Pixel block filtering is performed based on the pixel block filtering model and the pixel block tree to arrange adjacent fine-grained pixel blocks into corresponding coarse-grained pixel blocks; and a preset number of canvases are obtained based on the remaining fine-grained pixel blocks, the corresponding coarse-grained pixel blocks, and the pixel block packing model, and the final layout information of the packed pixel blocks is generated in the preset number of canvases.

[0012] The model is optimized based on the final layout information of the packaged pixel blocks to obtain a trained inference model. Based on the trained inference model, the annotation data of the billion-pixel image to be labeled and the labels of the annotation data are output.

[0013] In addition, the pixel block arrangement method for one billion pixels according to the above embodiments of the present invention may also have the following additional technical features:

[0014] Furthermore, in one embodiment of the present invention, before performing pixel block filtering based on the pixel block filtering model and the pixel block tree, the method further includes:

[0015] Based on the pixel block tree and tree structure encoder, the object information of each pixel block node in the pixel block tree and the relationship between pixel block nodes in the tree are obtained.

[0016] Furthermore, in one embodiment of the present invention, the step of constructing a pixel block tree using initial pixel blocks includes:

[0017] Fine-grained pixel blocks are obtained based on the leaf nodes formed in the pixel block tree from the initial pixel blocks;

[0018] The leaf nodes are aggregated to obtain coarse-grained pixel blocks;

[0019] A fine-grained pixel block tree is constructed based on the fine-grained pixel blocks and the coarse-grained pixel blocks, and the root node of the fine-grained pixel block tree is used to represent the complete billion-pixel image to be labeled.

[0020] Furthermore, in one embodiment of the present invention, obtaining a preset number of canvases based on the remaining fine-grained pixel blocks and the corresponding coarse-grained pixel blocks, as well as the pixel block packing model, includes:

[0021] Based on the pixel block packing model, a greedy algorithm is used to fill the first canvas with the remaining fine-grained pixel blocks and the corresponding coarse-grained pixel blocks; and,

[0022] Fill the second canvas with the second pixel block, and stop packing when all pixel blocks have been placed on the canvas.

[0023] Furthermore, in one embodiment of the present invention, the method further includes: using a coarse detector to detect the downscaled billion-pixel image to be labeled, and clustering the obtained coarse detection results.

[0024] To achieve the above objectives, another aspect of the present invention proposes a pixel block arrangement system for billion-pixel pixels, comprising:

[0025] The data acquisition module is used to acquire the billion-pixel image to be labeled and the corresponding image label, as well as the inference model; wherein, the inference model includes a pixel block tree generation model, a pixel block filtering model and a pixel block packing model;

[0026] A pixel block tree construction module is used to input the billion-pixel image to be labeled into the inference model, to cluster the initial pixel blocks based on the pixel block tree generation model and the image labels, and to construct a pixel block tree using the initial pixel blocks; wherein, the pixel block tree includes fine-grained pixel blocks and corresponding coarse-grained pixel blocks;

[0027] The filtering and packing module is used to filter pixels based on the pixel block filtering model and the pixel block tree to arrange adjacent fine-grained pixels into corresponding coarse-grained pixels; and to obtain a preset number of canvases based on the remaining fine-grained pixels, the corresponding coarse-grained pixels, and the pixel block packing model, and to generate the final layout information of the packed pixels in the preset number of canvases.

[0028] The data annotation module is used to optimize the model based on the final layout information of the packaged pixel blocks to obtain a trained inference model, and output the annotation data and labels of the billion-pixel image to be annotated based on the trained inference model.

[0029] The pixel block arrangement method and system for billion-pixel images in this invention improves the efficiency of target detection by learning the pixel block arrangement. A pixel block filtering module is used to select and filter candidate pixel blocks across granularities. A pixel block packing module is used to sequentially pack the remaining pixel blocks into the canvas, improving inference speed. Through modular design, the inference speed for billion-pixel images is improved while maintaining ideal performance.

[0030] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0031] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0032] Figure 1 This is a flowchart of a pixel block arrangement method for one billion pixels according to an embodiment of the present invention;

[0033] Figure 2 This is a pixel block arrangement framework diagram for a billion pixels according to an embodiment of the present invention;

[0034] Figure 3 This is a schematic diagram of a pixel block arrangement system for one billion pixels according to an embodiment of the present invention. Detailed Implementation

[0035] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0036] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0037] The following describes a pixel block arrangement method and system for billion pixels according to embodiments of the present invention, with reference to the accompanying drawings.

[0038] Figure 1 This is a flowchart of a pixel block arrangement method for one billion pixels according to an embodiment of the present invention.

[0039] like Figure 1 As shown, the method includes, but is not limited to, the following steps:

[0040] S1, obtain the billion-pixel image to be labeled and its corresponding image label, as well as the inference model; wherein, the inference model includes a pixel block tree generation model, a pixel block filtering model and a pixel block packing model;

[0041] S2, the billion-pixel image to be labeled is input into the inference model, and initial pixel blocks are obtained by clustering based on the pixel block tree generation model and image labels, and a pixel block tree is constructed using the initial pixel blocks; wherein, the pixel block tree includes fine-grained pixel blocks and corresponding coarse-grained pixel blocks;

[0042] S3, perform pixel block filtering based on pixel block filtering model and pixel block tree to arrange adjacent fine-grained pixel blocks into corresponding coarse-grained pixel blocks; and obtain a preset number of canvases based on the remaining fine-grained pixel blocks, the corresponding coarse-grained pixel blocks and the pixel block packing model, and generate the final layout information of the packed pixel blocks in the preset number of canvases.

[0043] S4. Based on the final layout information of the packaged pixel blocks, the model is optimized to obtain a trained inference model. Based on the trained inference model, the annotation data of the billion-pixel image to be annotated and the labels of the annotation data are output.

[0044] Specifically, such as Figure 2As shown, the pixel block arrangement framework for billion-pixel images proposed in this invention works by constructing an inference model and utilizing a constructed pixel tree, pixel block filtering, and pixel block packing modules to achieve rapid annotation of the data to be labeled. Specifically, the training part is divided into pixel tree generation, pixel block filtering and pixel block packing, and inference model training. First, a multi-granularity pixel tree is constructed for the input billion-pixel image, consisting of fine-grained pixel blocks (leaf layer nodes) and corresponding coarse-grained pixel blocks (intermediate layer nodes). An LSTM-based tree structure encoder is used to embed pixel block information and explore the relationships between pixel blocks. Subsequent learning is based on the multi-granularity pixel block candidate selection process in this tree. Then, the pixel block filtering module learns adaptive pixel block selection between the two granularities, arranging some adjacent fine-grained pixel blocks into corresponding coarse-grained pixel blocks. After that, the pixel block packing module packs all the remaining pixel blocks together to form a smaller number of canvases, generating a compact layout of packed pixel blocks in the canvases. Finally, a reinforcement learning strategy is combined to train the entire framework and complete the annotation and label output of the data to be labeled.

[0045] Based on the above framework, this invention can replace some candidate pixel blocks with fewer coarse pixel blocks, thus accelerating detection. A pixel block packing module is used to arrange adjacent pixel blocks (local view), and pixel block filtering is used to arrange all pixel blocks (global view). Furthermore, experimental results show that the pixel block arrangement framework proposed in this invention significantly improves the inference speed for billion-pixel images while maintaining ideal performance.

[0046] Furthermore, the pixel block arrangement method for one billion pixels proposed in this invention has the following specific implementation steps:

[0047] (1) Pixel tree generation:

[0048] First, a multi-granularity pixel block tree is constructed to explore the intrinsic relationships between each pixel block. Initial pixel blocks are obtained first based on clustering. Specifically, object boxes are clustered according to their positions using average shift, and then these clusters are merged to generate the bounding boxes of the pixel blocks. During training, we directly use the labels for clustering. During inference, a coarse detector is used on the downscaled input image, and the coarse detection results are clustered. Based on the initial pixel blocks, a multi-granularity pixel block tree G = (V, E) is constructed. The initial pixel blocks are first formed into leaf nodes in the pixel block tree; these are considered fine-grained pixel blocks. Then, these leaf nodes are clustered together to obtain the intermediate layer of the new coarse-grained pixel tree. Finally, the root node represents the complete image. The entire structure is represented by V. mid Connect all nodes. In this way, the spatial containment relationships between multi-granularity pixel blocks are reflected through the parent-child relationships between nodes in the tree.

[0049] The set of nodes V is divided into leaf nodes V leaf and intermediate node V mid Each node v i ∈V corresponds to a pixel block, when pixel block v i spatially covering pixel blocks v j At that time, node v i With v j Connected. All pixel block nodes V leaf and V mid Pixel block candidates Where N = |V leaf |+|V mid |

[0050] (2) Tree structure encoding:

[0051] To explore the relationships between pixel block nodes in the tree, a novel tree structure encoder is introduced to embed object information for each pixel block node, as well as the inter- and intra-layer relationships between pixel block nodes in the tree. Specifically, we first use v i =(x i y i w i h i r i y i a i o i n i ) represents each pixel block node v i , where x i and y i The center coordinates of the pixel block, w i h i r i These represent its width, height, and aspect ratio, respectively. i It is the area of ​​this pixel block, o i n i These represent the average area of ​​objects within the pixel block and the number of objects within the pixel block, respectively. We can estimate o using the coarse detection results. i and n i This 8-dimensional feature is then embedded into a high-dimensional representation, and an LSTM-based model consisting of tree-LSTM and chain-LSTM is introduced to explore the relationships between pixel block nodes in the tree. First, the sub-tree and tree-LSTM are used to encode the inter-layer relationships in the tree. Similar to standard LSTM, each Tree-LSTM unit (indexed by j) has its hidden state h. j By analogy, the sum of the hidden states of a Tree-LSTM is set to h. j :

[0052]

[0053] where \(C(j)\) represents the set of child nodes of node \(v\). j In this way, the information of different layers in the tree can be well aggregated in a bottom-up manner. To further understand the intra-layer relationship, first unfold the hidden state of the Tree-LSTM in a chained manner. For flexibility, a node chain is constructed here in a fixed traversal manner, and then the unfolded hidden state is fed into the Chain-LSTM in sequence.

[0054] The overall tree-structured encoder can be expressed as:

[0055] \(z\) i \(=\) LSTM Chain \((\)LSTM Tree \((\)FC\((v\) i \()))\)

[0056] (3) Pixel block filtering:

[0057] In the multi-granularity pixel block tree, a coarse-grained pixel block spatially covers its corresponding fine-grained pixel blocks. Intuitively, only the coarse-grained pixel blocks can be left to significantly reduce the number of candidate pixel blocks to be detected. However, considering the differences in object scales, not all objects can be well detected at a coarser granularity. Therefore, this model introduces a pixel block filtering module that learns to adaptively select candidate pixel blocks between the two granularities. We describe this process as a tree-to-set problem. Since the tree encoding has been written, a filter decoder is directly applied to select pixel blocks from the tree. Specifically, the filter module aims to generate a subset from the pixel block node candidates where \(|S| = M \lt N\). This filtering decoder is constructed by a standard LSTM. At each time step \(j\), the filtering decoder points to a specific pixel block node on the input tree. This pointed node is used as the input for the \((j + 1)\)-th step. Here, the pointing mechanism proposed in PointerNet is adopted, which authorizes the module to point to a specific node in the tree. To avoid repeated selection, the module introduces a mutually exclusive mechanism between the fine-grained pixel blocks and their corresponding coarse-grained pixel blocks. If node 1 is selected, its corresponding fine-grained pixel block nodes 2 and 3 will be masked. Conversely, if node 2 or node 3 is selected, node 1 will be masked.

[0058] (4) Pixel block packing:

[0059] Given the remaining pixel blocks, the pixel block packing module aims to pack specific pixel blocks into the canvas and initialize the layout of pixel blocks within the canvas. Unlike the pixel block filtering module, which focuses on arranging pixel blocks in local and neighboring views, the pixel block packing module arranges pixel blocks in a global view, considering all pixel blocks during packing, regardless of their position within the entire image. Specifically, given the output pixel block node set of the pixel block filtering module... We first reorganize it into a new two-layer pixel tree G. s The root node represents the entire image in the global context, while the pixel block node set consists of nodes from the leaf layer. The earlier tree-structured encoder used in the pixel block wrapping module is then reused to fully utilize the learned tree structure knowledge. The wrapper decoder receives the encoded tree as input and outputs a sequence representing the order of the wrapped pixel blocks. The pixel blocks are then sequentially packed into a canvas. The wrapper decoder's network structure is similar to a filter decoder, also employing a dotted mechanism where each step points to a specific node in the input tree. During the wrapping process, the decoder also initializes the layout of the pixel blocks in the canvas. Specifically, a greedy algorithm is used to ensure that each pixel block initially occupies the bottom space of the canvas. Then, when there is insufficient space to place the next pixel block, a new canvas is created for subsequent pixel blocks. The wrapping process stops when all pixel blocks have been placed into the canvas. The theoretical speedup can be controlled by setting the maximum capacity of each canvas.

[0060] (5) Overall optimization:

[0061] Given an input tree, the filter decoder or packer decoder points to a node at each time step. Taking the pixel block filtering module as an example, the probability of p(S|G) can be decomposed using the chain rule as follows:

[0062]

[0063] Where S(<i) represents the pixel blocks selected before the i-th time step. The goal is to learn a policy.

[0064] The parameters of p(S|G) represent a set of parameters that balance detection speed and accuracy. Inspired by the success of reinforcement learning in solving neural combinatorial problems, policy-based reinforcement learning is employed here to optimize the two main modules of the framework: the pixel block filtering module and the pixel block packing module.

[0065] Two different reward Rs were designed for the pixel block filtering module and the pixel block packing module. filter and R pack To guide their learning process. Specifically, R filter Considering the number of filtered pixel blocks, it is defined as:

[0066]

[0067] Where |·| represents the size of the node set. It instructs the pixel block filtering module to preferentially select coarser pixel blocks to replace the finer-grained pixel blocks it covers. Since the output of the pixel block filtering module is a set, R filter The parameters are designed to be order-independent to adapt to the learning process. For R... pack We make the following settings:

[0068] R pack =AR detector

[0069] Since the pixel block filtering results affect the pixel block packing strategy, we use the cumulative reward after the policy is applied to guide the policy. The training objective of the pixel block filter policy can be updated in the following way:

[0070] R filter =λR filter +R pack

[0071] The optimization process utilizes a weighting factor λ to balance detection speed and performance. Therefore, pixel block filtering...

[0072] The objective function of the module is defined as:

[0073] J(θ|G)=E S～pθ(.| G ) R(S|G)

[0074] Where θ represents the parameters of the pixel block filtering module. Policy gradient and stochastic gradient descent methods are used.

[0075] Optimization. According to the well-known REINFORCE algorithm, the gradient in the above equation can be expressed as:

[0076]

[0077] Where b(S) represents the baseline value independent of S, used to reduce the variance of the gradient. We plot the B samples G1, G2, ..., G... B As a batch size, based on Monte Carlo sampling, the gradient in the above equation can be approximated as:

[0078]

[0079] We use the average of the batch rewards as a baseline value, which is updated incrementally during training. The pixel block packing module will be optimized in a similar manner.

[0080] (6) Repeat step (5):

[0081] The optimization process described above only involves one round of data updates. To achieve better model accuracy, we need to continuously train the model until it reaches a high accuracy level and meets the target. The design of the two permutation modules ensures shorter training time and faster convergence.

[0082] (7) Reasoning about the data to be classified:

[0083] By repeatedly training the model in the previous step, the data to be classified can be preprocessed and then used to obtain the corresponding labels through the inference model.

[0084] In summary, experiments on the PANDA standard dataset (a billion-pixel dataset) demonstrate the superior performance of the pixel block arrangement method proposed in this invention. Specifically, the downsampling factor of the original image to one billion pixels is set to 4 times, the sliding window for decomposition downsampling is 2,048 × 1,024 pixels, a 64-dimensional fully connected layer and a 128-dimensional hidden layer are used, the initial learning rate is 0.001, and the batch size is 64. For the PANDA standard dataset, the proposed method improves the inference speed of one billion-pixel images by 5 times while maintaining ideal performance in the detection of large, medium, and small objects.

[0085] The pixel block arrangement method for billion-pixel images according to embodiments of the present invention improves the efficiency of target detection by learning the pixel block arrangement. A pixel block filtering module is used to select and filter candidate pixel blocks across granularities. A pixel block packing module is used to sequentially pack the remaining pixel blocks into the canvas, improving inference speed. A modular design is employed to improve the inference speed for billion-pixel images while maintaining ideal performance.

[0086] To achieve the above embodiments, such as Figure 3 As shown, this embodiment also provides a pixel block arrangement system 10 for billion pixels. The system 10 includes a data acquisition module 100, a pixel block tree construction module 200, a filtering and packaging module 300, and a data annotation module 400.

[0087] The data acquisition module 100 is used to acquire the billion-pixel image to be labeled and the corresponding image label, as well as the inference model; wherein, the inference model includes a pixel block tree generation model, a pixel block filtering model and a pixel block packing model;

[0088] The pixel block tree construction module 200 is used to input the billion-pixel image to be labeled into the inference model, to cluster based on the pixel block tree generation model and the image label to obtain initial pixel blocks, and to construct a pixel block tree using the initial pixel blocks; wherein, the pixel block tree includes fine-grained pixel blocks and corresponding coarse-grained pixel blocks;

[0089] The filtering and packaging module 300 is used to perform pixel block filtering based on the pixel block filtering model and the pixel block tree, so as to arrange adjacent fine-grained pixel blocks into corresponding coarse-grained pixel blocks; and to obtain a preset number of canvases based on the remaining fine-grained pixel blocks, the corresponding coarse-grained pixel blocks and the pixel block packaging model, and to generate the final layout information of the packaged pixel blocks in the preset number of canvases.

[0090] The data annotation module 400 is used to optimize the model based on the final layout information of the packaged pixel blocks to obtain a trained inference model, and output the annotation data of the billion-pixel image to be annotated and the labels of the annotation data based on the trained inference model.

[0091] Furthermore, 300 steps before the aforementioned filtering and packaging module, it also includes:

[0092] The structure encoding module is used to obtain the object information of each pixel block node in the pixel block tree and the relationship between pixel block nodes in the tree based on the pixel block tree and the tree structure encoder.

[0093] Furthermore, the aforementioned pixel block tree construction module 200 is also used for:

[0094] Fine-grained pixel blocks are obtained based on the leaf nodes formed in the pixel block tree from the initial pixel blocks;

[0095] Aggregate the leaf nodes to obtain coarse-grained pixel blocks;

[0096] A fine-grained pixel block tree is constructed based on fine-grained and coarse-grained pixel blocks, and the root node of the fine-grained pixel block tree is used to represent the complete billion-pixel image to be labeled.

[0097] Furthermore, the aforementioned filter packaging module 300 is also used for:

[0098] Based on the pixel block packing model, a greedy algorithm is used to fill the first canvas with the remaining fine-grained pixel blocks and the corresponding coarse-grained pixel blocks; and,

[0099] Fill the second canvas with the second pixel block, and stop packing when all pixel blocks have been placed on the canvas.

[0100] Furthermore, system 10 also includes: a clustering detection module, used for:

[0101] A coarse detector is used to detect downscaled billion-pixel images to be labeled, and the coarse detection results are clustered.

[0102] The pixel block arrangement system for billion-pixel images according to embodiments of the present invention improves the efficiency of target detection by learning the pixel block arrangement. A pixel block filtering module is used to select and filter candidate pixel blocks across granularities. A pixel block packing module is used to sequentially pack the remaining pixel blocks into the canvas, improving inference speed. A modular design is employed to improve the inference speed for billion-pixel images while maintaining ideal performance.

[0103] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0104] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

Claims

1. A pixel block arrangement method for one billion pixels, characterized in that, Includes the following steps: Obtain the billion-pixel image to be labeled and its corresponding image label, as well as the inference model; wherein, the inference model includes a pixel block tree generation model, a pixel block filtering model, and a pixel block packing model; The billion-pixel image to be labeled is input into the inference model to cluster the initial pixel blocks based on the pixel block tree generation model and the image labels, and the initial pixel blocks are used to construct a pixel block tree; wherein, the pixel block tree includes fine-grained pixel blocks and corresponding coarse-grained pixel blocks; Pixel block filtering is performed based on the pixel block filtering model and the pixel block tree to arrange adjacent fine-grained pixel blocks into corresponding coarse-grained pixel blocks; and a preset number of canvases are obtained based on the remaining fine-grained pixel blocks, the corresponding coarse-grained pixel blocks, and the pixel block packing model, and the final layout information of the packed pixel blocks is generated in the preset number of canvases. The model is optimized based on the final layout information of the packaged pixel blocks to obtain a trained inference model. Based on the trained inference model, the annotation data of the billion-pixel image to be labeled and the labels of the annotation data are output.

2. The method according to claim 1, characterized in that, Before performing pixel block filtering based on the pixel block filtering model and the pixel block tree, the method further includes: Based on the pixel block tree and tree structure encoder, the object information of each pixel block node in the pixel block tree and the relationship between pixel block nodes in the tree are obtained.

3. The method according to claim 1, characterized in that, The process of constructing a pixel block tree using initial pixel blocks includes: Fine-grained pixel blocks are obtained based on the leaf nodes formed in the pixel block tree from the initial pixel blocks; The leaf nodes are aggregated to obtain coarse-grained pixel blocks; A pixel block tree is constructed based on the fine-grained pixel blocks and the coarse-grained pixel blocks, and the root node of the pixel block tree is used to represent the complete billion-pixel image to be labeled.

4. The method according to claim 1, characterized in that, The process of obtaining a preset number of canvases based on the remaining fine-grained pixel blocks, the corresponding coarse-grained pixel blocks, and the pixel block packing model includes: Based on the pixel block packing model, a greedy algorithm is used to fill the first canvas with the remaining fine-grained pixel blocks and the corresponding coarse-grained pixel blocks; and, Fill the second canvas with the second pixel block, and stop packing when all pixel blocks have been placed on the canvas.

5. The method according to claim 1, characterized in that, The method further includes: using a coarse detector to detect the downscaled billion-pixel image to be labeled, and clustering the obtained coarse detection results.

6. A pixel block arrangement system for one billion pixels, characterized in that, include: The data acquisition module is used to acquire the billion-pixel image to be labeled and the corresponding image label, as well as the inference model; wherein, the inference model includes a pixel block tree generation model, a pixel block filtering model and a pixel block packing model; A pixel block tree construction module is used to input the billion-pixel image to be labeled into the inference model, to cluster the initial pixel blocks based on the pixel block tree generation model and the image labels, and to construct a pixel block tree using the initial pixel blocks; wherein, the pixel block tree includes fine-grained pixel blocks and corresponding coarse-grained pixel blocks; The filtering and packing module is used to filter pixels based on the pixel block filtering model and the pixel block tree to arrange adjacent fine-grained pixels into corresponding coarse-grained pixels; and to obtain a preset number of canvases based on the remaining fine-grained pixels, the corresponding coarse-grained pixels, and the pixel block packing model, and to generate the final layout information of the packed pixels in the preset number of canvases. The data annotation module is used to optimize the model based on the final layout information of the packaged pixel blocks to obtain a trained inference model, and output the annotation data and labels of the billion-pixel image to be annotated based on the trained inference model.

7. The system according to claim 6, characterized in that, Prior to the filtering and packaging module, the following is also included: The structure encoding module is used to obtain the object information of each pixel block node in the pixel block tree and the relationship between pixel block nodes in the tree based on the pixel block tree and the tree structure encoder.

8. The system according to claim 6, characterized in that, The pixel block tree construction module is also used for: Fine-grained pixel blocks are obtained based on the leaf nodes formed in the pixel block tree from the initial pixel blocks; The leaf nodes are aggregated to obtain coarse-grained pixel blocks; A pixel block tree is constructed based on the fine-grained pixel blocks and the coarse-grained pixel blocks, and the root node of the pixel block tree is used to represent the complete billion-pixel image to be labeled.

9. The system according to claim 6, characterized in that, The filtering and packaging module is also used for: Based on the pixel block packing model, a greedy algorithm is used to fill the first canvas with the remaining fine-grained pixel blocks and the corresponding coarse-grained pixel blocks. as well as, Fill the second canvas with the second pixel block, and stop packing when all pixel blocks have been placed on the canvas.

10. The system according to claim 6, characterized in that, The system further includes: a clustering detection module, used for: A coarse detector is used to detect downscaled billion-pixel images to be labeled, and the coarse detection results are clustered.

Images