A method for image layer spectrum segmentation based on structural information principle
Through the image layer spectrum segmentation method based on the structural information principle, the graph structure data is constructed using the principles of one-dimensional structure entropy maximization and compressed information minimization, which solves the problem of missing global structure and local detailed information in image segmentation, and realizes automation Segmentation and high interpretability are superior to traditional methods.
Patent Information
- Application Number
- CN202210457184.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-04-27
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2042-04-27
AI Technical Summary
The existing image segmentation algorithm lacks global structural information and local details when processing images, resulting in objects interspersing in the background or belonging to the same object, but cannot be divided into an area due to the gradient of color, and cannot get rid of the influence of artificial selection parameters.
The image layer spectrum segmentation method based on the structural information principle is adopted to construct graph structure data through the principles of one-dimensional structure entropy maximization and compressed information minimization, so as to realize the segmentation of local details and global structure of the image, and use the three-dimensional coding tree to perform image segmentation. , avoid manually selecting parameters.
The precise segmentation of the image is realized, and the structure information of the image can be automatically extracted without manual adjustment of parameters. The segmentation results are highly interpretable, which is better than the performance of traditional methods in various superpixel algorithms.
Smart Images

Figure CN114782379B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of image segmentation, and in particular relates to an image layer spectrum segmentation method based on the principle of structural information. Background Art
[0002] Traditional unsupervised image segmentation algorithms mainly include threshold-based, edge-based, graph-based, and cluster-based algorithms. Threshold-based image segmentation algorithms are suitable for images with large grayscale differences between segmented targets and backgrounds. Edge-based image segmentation algorithms are difficult to form closed and connected areas. Even if closed and connected areas are formed, they usually need to be processed again in combination with other algorithms. Traditional image segmentation algorithms based on graph theory usually compose images through four or eight connections, consider the correlation between local pixels, and segment spatially connected pixels together. This type of method usually has high segmentation accuracy, but the algorithm is relatively time-consuming. Clustering-based image segmentation algorithms need to predetermine the number of clusters and can cluster spatially unconnected pixels together. One of the more popular image segmentation methods today is image segmentation based on deep learning. Image segmentation based on deep learning can achieve good segmentation effects after training, but the objects to be segmented must be pre-trained with data sets, which usually requires high-quality and large amounts of data; image segmentation based on deep learning mostly relies on neural networks to perform semantic segmentation of images based on labels; deep learning models have poor interpretability. At present, there are few composition methods for image segmentation methods based on graph theory.
[0003] Currently, the more popular algorithms among traditional image segmentation algorithms are graph theory-based image segmentation algorithms and clustering-based image segmentation algorithms.
[0004] Unsupervised image segmentation algorithms based on graph theory (such as the EGB algorithm) lack sufficient global structural information when performing image segmentation, resulting in the background being fragmented when objects penetrate the background.
[0005] Clustering-based image segmentation algorithms (such as Kmeans, FCM, and MShift) lack local detail information when performing image segmentation, which may result in the inability to segment the same object into one region due to color gradients.
[0006] The current clustering segmentation algorithm requires manual adjustment of parameters based on the segmentation effect, especially the manual selection of the number of clusters. The current unsupervised image segmentation algorithm cannot get rid of the manual selection of parameters, and the image segmentation results are affected by the parameters. Summary of the invention
[0007] In order to solve the above problems, the present invention proposes an image spectral layer segmentation method based on the principle of structural information to solve the problem of abstract spectral layer segmentation of images. The local detail layer can perform precise segmentation of the image, and the global structure layer can extract the structure of the image. The image can be regionally segmented without selecting parameters.
[0008] To achieve the above object, the technical solution adopted by the present invention is: an image layer spectrum segmentation method based on the principle of structural information, comprising the steps of:
[0009] Preprocess the original image;
[0010] Based on the principle of one-dimensional structural entropy maximization, the graph structure data G of the preprocessed image is constructed. * ;
[0011] In the first segmentation stage, the graph structure data G of the preprocessed image is * Execute the two-dimensional structure entropy minimization algorithm ε on 2 , we get graph G * The optimal partition structure X = {X1, X2, ..., X n}, where n is the graph G * The number of optimal partitioning structures;
[0012] Generate graph structure data G of partition structure X ** ;
[0013] In the second segmentation stage, in Figure G ** Execution algorithm on two-dimensional structure entropy minimization algorithm ε 2 , we get graph G ** The optimal partition structure Y = {Y1, Y2, ..., Y m}, where m is the graph G ** The number of optimal partition structures; for each k = 1, 2, 3 ... m, Y k All nodes in it correspond to the partition structure X;
[0014] In the third segmentation stage, the segmentation structure Y is transformed into the segmentation structure Z = {Z1, Z2, ..., Z m}, construct the structure of a three-dimensional coding tree; for each k = 1, 2, 3 ... m, Z k All nodes in it correspond to pixels in the image.
[0015] Furthermore, the preprocessing of the original image comprises the steps of:
[0016] Get the Gaussian mask;
[0017] Normalize the Gaussian mask;
[0018] Perform two Laplacian product operations on the original image using the Gaussian mask.
[0019] Furthermore, based on the principle of one-dimensional structural entropy maximization, the graph structure data G of the preprocessed image is constructed. * When the image is converted into the range of interaction of each pixel in the graph structure data by using the one-dimensional structure entropy maximization principle generated by the graph, that is, confirming the pixel points that interact with each pixel in the spatial range; including the steps of:
[0020] Given a preprocessed image, for each grid node v, the nearest neighbors are centered on v and located at different levels of square boundaries; connect v with its nearest neighbors; use parameter k to determine which neighbors of v intersect with v, thereby constructing the graph structure data G * =(V * , E * ), where V * is the set of all vertices in the graph, E * is the set of all edges.
[0021] Furthermore, the graph structure data G * The generation of includes the steps:
[0022] (1) Assuming that the neighbor node of node v is node u, the similarity measurement formula between the two is:
[0023]
[0024] In the formula, t is a hyperparameter;
[0025] (2) Assume that parameter k is the number of grid edges or diagonal edges between nodes; by determining k, we can obtain the set of neighbor nodes S that v is expected to interact with. k , Keep S k The nodes that are most similar to v in the graph are retained in a number that is half the number of original nodes. The sorting order in the interaction is: nodes with larger similarity weights are given priority. If the similarities are the same, nodes that have previously interacted with the node are given priority. Thus, the preliminary graph structure data G is constructed. k ,
[0026] satisfy At this time That is the required graph structure data G * .
[0027] Furthermore, the graph structure data G of the generated partition structure X is ** Includes steps:
[0028] For each i=1,2,3...n, set X i Scale to a point independent node xi , for each j = 1, 2, ..., n and i ≠ j, set X j Scale to a point independent node x j ;
[0029] x i and x j The color vector is X i and X j The color vector mean of all independent nodes in the node x j and x j The interaction measurement indicators between independent nodes are obtained to obtain the interaction relationship between independent nodes;
[0030] Then, based on the principle of minimizing compressed information, we construct the graph G. ** .
[0031] Furthermore, the graph G is constructed based on the principle of minimizing compressed information. ** When a special sparse subgraph is found from a weighted complete graph, a graph structure data G is generated. ** =(V ** ,E ** ), where V ** is the set of all vertices in the graph, E ** is the set of all edges.
[0032] Furthermore, the graph G is constructed based on the principle of minimizing compressed information. ** , including the steps of:
[0033] (1) Treat each part of the partition structure X as an independent node, and the similarity measurement formula between nodes is:
[0034]
[0035] In the formula, t is a hyperparameter; similarity is used to obtain interactive nodes and construct a complete graph CG;
[0036] (2) Assume a very small number σ∈(0,1), k2 is the number of edges that each node interacts with in CG; for CG, retain the k2 edges with the largest weight for each node, and obtain a sparse subgraph make The compression rate ρ<σ and k2 is the smallest. At this time, the sparse subgraph That is the required graph structure data G ** .
[0037] The beneficial effects of adopting this technical solution are:
[0038] The present invention realizes the spectral abstract analysis capability of an image based on a coding tree with a spectral abstract structure, including a local detail information layer of the image and a global structure layer of the image. The local detail layer of the image is called the close view of the image, which is used to mine the local detail information of the image. The global structure layer of the image is called the long-distance view of the image, which is used to mine the global structure information of the image. The present invention selects parameters based on the principle of maximizing the one-dimensional structural entropy generated by the graph and the principle of minimizing the compression rate generated by the information system, so as to achieve the purpose of not requiring manual selection of parameters.
[0039] The image segmentation method proposed in the present invention has a layer spectrum abstract structure, including a local detail layer and a global structure layer. Compared with many superpixel algorithms, the local detail layer in the present invention not only shows a high level in indicators that rely on manual annotation (Rec and UE), but also shows a high level in indicators that do not rely on manual annotation (UE), and its value reaches 0.99, which is the best performance among the fifteen superpixel algorithms. The global structure layer in the present invention can perform regional segmentation on the image without artificially specifying the number of clusters; unlike the deep learning algorithm, the algorithm has interpretability of its segmentation results based on the principle of structural information.
[0040] The present invention realizes the layer spectrum abstract analysis of the image based on the coding tree layer spectrum abstract structure, including three image segmentation stages. In the first stage, the one-dimensional structure entropy maximization principle generated by the information system is used to achieve the effect of mining the local detail information of the image. In the second and third stages, the compression rate minimization principle generated by the information system is used to achieve the effect of analyzing the global structure information of the image; and the one-dimensional structure entropy maximization principle, the decoding information maximization principle and the compression information minimization principle are used as the basis for selecting parameters, so as to achieve the purpose of not having to adjust the parameters manually. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] Figure 1 It is a flow chart of an image layer spectrum segmentation method based on the principle of structural information of the present invention;
[0042] Figure 2 The graph structure data G in the embodiment of the present invention * A first structural diagram of the generation process;
[0043] Figure 3 The graph structure data G in the embodiment of the present invention * A second structural diagram of the generation process. DETAILED DESCRIPTION
[0044] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is further described below with reference to the accompanying drawings.
[0045] In this embodiment, see Figure 1As shown, the present invention proposes an image layer spectrum segmentation method based on the principle of structural information, comprising the steps of:
[0046] Preprocess the original image;
[0047] Based on the principle of one-dimensional structural entropy maximization, the graph structure data G of the preprocessed image is constructed. * ;
[0048] In the first segmentation stage, the graph structure data G of the preprocessed image is * Execute the two-dimensional structure entropy minimization algorithm ε on 2 , we get graph G * The optimal partition structure X = {X1, X2, ..., X n}, where n is the graph G * The number of optimal partitioning structures;
[0049] Generate graph structure data G of partition structure X ** ;
[0050] In the second segmentation stage, in Figure G ** Execution algorithm on two-dimensional structure entropy minimization algorithm ε 2 , we get graph G ** The optimal partition structure Y = {Y1, Y2, ..., Y m}, where m is the graph G ** The number of optimal partition structures; for each k = 1, 2, 3 ... m, Y k All nodes in it correspond to the partition structure X;
[0051] In the third segmentation stage, the segmentation structure Y is transformed into the segmentation structure Z = {Z1, Z2, ..., Z m}, construct the structure of a three-dimensional coding tree; for each k = 1, 2, 3 ... m, Z k All nodes in it correspond to pixels in the image.
[0052] As an optimization scheme of the above embodiment, the preprocessing of the original image achieves the purpose of removing noise and enhancing resolution by using a Gaussian mask, making the image smoother and easier to segment; comprising the steps of:
[0053] Get the Gaussian mask; the parameter SIGMA of the Gaussian mask is 0.1, and finally a two-dimensional vector is generated as the mask.
[0054] Normalize the Gaussian mask;
[0055] Perform two Laplacian product operations on the original image using the Gaussian mask.
[0056] The image preprocessing adopted in this article enhances the edge resolution of the image. The operation of this preprocessing actually readjusts the color value of this pixel point according to the surrounding pixel points. Since the SIGMA value adopted here is small, the influence of the color values of the surrounding pixel points on this pixel point will be small. The image after filtering is equivalent to reducing the values of the three RGB channels of this pixel point, that is, the three values in the color vector are reduced to varying degrees. Similarly, this operation avoids the operation of normalizing the Euclidean space distance between two pixel points, and the effect is far better than that of normalization.
[0057] As an optimized solution of the above embodiment, based on the principle of maximizing one-dimensional structural entropy, the graph structure data G of the preprocessed image is constructed * When, the principle of maximizing one-dimensional structural entropy generated by the graph is used to convert the image into the range size of the interaction of each pixel point in the graph structure data, that is, to confirm the pixel points that interact with each pixel point within the spatial range; including the steps:
[0058] Given a preprocessed image, for the nearest neighbors of each grid node v, centered on v and located on the square boundaries at different levels; connect v with its nearest neighbors; use the parameter k to determine which neighbors of v intersect with v, so as to construct the graph structure data G * =(V * , E * ), where V * is the set of all vertices in the graph, and E * is the set of all edges.
[0059] Specifically, the generation of the graph structure data G * includes the steps:
[0060] (1) Assume that the neighbor node of node v is node u, and the similarity measurement formula between the two:
[0061]
[0062] In the formula, t is a hyperparameter, and the hyperparameter t is selected by maximizing the derivative of one-dimensional structural entropy. The finally selected value is 1;
[0063] (2) Assume that the parameter k is the number of grid edges or diagonal edges between nodes; Figure 2 As shown, the neighbors of v present a "rice" shape; by determining k, the set S of neighbor nodes that v is expected to interact with is obtained k , Retain the nodes in S k that are most similar to v, and the number of retained nodes is half of the original number of nodes, as Figure 3 shown; the sorting order in the interaction is: the nodes with larger similarity weights are selected first. If the similarities are the same, the nodes that interacted with this node in advance are selected first; thus, the preliminary graph structure data G is constructedk ,
[0064] satisfy At this time That is the required graph structure data G * .
[0065] Figure 2 and Figure 3 The vertices in satisfy the spatial distribution of the pixels in the image. The black vertices refer to the vertex v, the white vertices refer to the vertices that do not interact with the vertex v, the gray vertices refer to the vertices that interact with the vertex v, the first circle closest to the vertex v is the point that interacts with the vertex v when k = 1, and the second circle point refers to the point that interacts with the vertex v when k = 2. Due to space limitations, this figure only shows the case of k = 2. Figure 2 The part represents all vertices that meet the distance range limit, Figure 3 Part means retaining edges with larger weights, and the number is controlled within Figure 2 Of course, the interacting vertices are not completely similar. Figure 3 In constructing graph G k When the weight of multiple edges is the same, the number of edges with larger weights will be selected. When multiple edges have the same weight, how to select them is also a problem that needs attention. For example, a node can establish connections with eight adjacent nodes. At this time, the node has eight edges when building the graph, but only four edges can be retained during the actual construction process. There may be multiple edges ranked fourth. At this time, they should be sorted strictly according to the order of construction, that is, the edges that interact first are retained.
[0066]
[0067]
[0068] As an optimization solution of the above embodiment, the graph structure data G of the generated partition structure X is ** Includes steps:
[0069] For each i=1, 2, 3...n, X i Scale to a point independent node xi, for each j = 1, 2, ..., n and j ≠ i, X j Scale to a point independent node x j ;
[0070] x i and x j The color vector is X i and X j The color vector mean of all independent nodes in the node x i and x j The interaction measurement indicators between independent nodes are obtained to obtain the interaction relationship between independent nodes;
[0071] Then, based on the principle of minimizing compressed information, we construct the graph G. ** .
[0072] Among them, the graph G is constructed based on the principle of minimizing compressed information. ** When a special sparse subgraph is found from a weighted complete graph, a graph structure data G is generated. ** =(V ** , E ** ), where V ** is the set of all vertices in the graph, E ** is the set of all edges.
[0073] Specifically, the graph G is constructed based on the principle of minimizing compressed information. ** , including the steps of:
[0074] (1) Treat each part of the partition structure X as an independent node, and the similarity measurement formula between nodes is:
[0075]
[0076] In the formula, t is a hyperparameter, which is selected by maximizing the derivative of the one-dimensional structural entropy, and the final selected value is 1. The interaction nodes are obtained by using similarity to construct a complete graph CG.
[0077] (2) Assume a very small number σ∈(0, 1), k2 is the number of edges that each node interacts with in the CG; for the CG, retain the k2 edges with the largest weight for each node, and obtain a sparse subgraph make The compression rate ρ<σ and k2 is the smallest. At this time, the sparse subgraph That is the required graph structure data G ** .
[0078]
[0079]
[0080] The first segmentation stage algorithm of the present invention performs detailed segmentation on the image, and its segmentation results are very fine. According to the boundary quality evaluation experiment, the segmentation results are evaluated, and the Recall of the boundary is approximately equal to 0.99, the error segmentation rate UE is approximately equal to 0.12, and the EV value is equal to 0.99. The algorithm of the present invention performs well in multiple data sets and is better than the other fifteen superpixel segmentation algorithms. The third segmentation stage algorithm of the present invention performs global structure segmentation on the image and performs segmentation based on the results of the second segmentation stage; we used the regional quality evaluation experiment to evaluate its results and compared them with the three unsupervised algorithms. Its PRI value is the highest, reaching 0.7078.
[0081] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.
Claims
1. A method for image layer spectrum segmentation based on the principle of structural information, characterized in that: Includes steps: Preprocess the original image; Based on the principle of one-dimensional structural entropy maximization, the graph structure data G of the preprocessed image is constructed. * ; Use the one-dimensional structure entropy maximization principle generated by the graph to convert the image into the range size of each pixel interaction in the graph structure data, that is, to confirm the pixels that each pixel interacts with within the spatial range; Includes steps: Given a preprocessed image, for each grid node v, the nearest neighbors are centered on v and located at different levels of square boundaries; connect v with its nearest neighbors; use parameter k to determine which neighbors of v intersect with v, thereby constructing the graph structure data G * =(V * ,E * ), where V * is the set of all vertices in the graph, E * is the set of all edges; Graph structure data G * The generation of includes the steps: (1) Assuming that the neighbor node of node v is node u, the similarity measurement formula between the two is: In the formula, t is a hyperparameter; (2) Assume that parameter k is the number of grid edges or diagonal edges between nodes; by determining k, we can obtain the set of neighbor nodes S that v is expected to interact with. k , Keep S k The nodes that are most similar to v in the graph are retained in a number that is half the number of original nodes. The sorting order in the interaction is: nodes with larger similarity weights are given priority. If the similarities are the same, nodes that have previously interacted with the node are given priority. Thus, the preliminary graph structure data G is constructed. k , satisfy At this time That is the required graph structure data G * ; In the first segmentation stage, the graph structure data G of the preprocessed image is * Execute the two-dimensional structure entropy minimization algorithm ε on 2 , we get graph G * The optimal partition structure X={X1,X2,……,X n }, where n is the graph G * The number of optimal partitioning structures; Generate graph structure data G of partition structure X ** , including the steps of: For each i=1,2,3...n, set X i Scale to a point independent node x i , for each j = 1, 2, ..., n and j ≠ i, set X j Scale to a point independent node x j ;x i and x j The color vector is X i and X j The color vector mean of all independent nodes in the node x i and x j The interaction measurement index between them is used to obtain the independent nodes and the interaction relationship between the independent nodes; a weighted complete graph is constructed; Then, based on the principle of minimizing compressed information, we construct the graph G. ** ; In the second segmentation stage, in Figure G ** Execution algorithm on two-dimensional structure entropy minimization algorithm ε 2 , we get graph G ** The optimal partition structure Y={Y1,Y2,……,Y m }, where m is the graph G ** The number of optimal partition structures; for each k = 1, 2, 3 ... m, Y k All nodes in it correspond to the partition structure X; In the third segmentation stage, the segmentation structure Y is transformed into the segmentation structure Z = {Z1, Z2, ..., Z m }, construct the structure of a three-dimensional coding tree; for each k = 1, 2, 3 ... m, Z k All nodes in it correspond to pixels in the image.
2. The image layer spectrum segmentation method based on the structural information principle according to claim 1 is characterized in that: The preprocessing of the original image comprises the steps of: Get the Gaussian mask; Normalize the Gaussian mask; Perform two Laplacian product operations on the original image using the Gaussian mask.
3. The image layer spectrum segmentation method based on the structural information principle according to claim 1 is characterized in that: The graph G is constructed based on the principle of minimizing compressed information. ** When a sparse subgraph is found from a weighted complete graph, a graph structured data G is generated. ** =(V ** ,E ** ), where V ** is the set of all vertices in the graph, E ** is the set of all edges.
4. The image layer spectrum segmentation method based on the structural information principle according to claim 3 is characterized in that: The graph G is constructed based on the principle of minimizing compressed information. ** , including the steps of: (1) Treat each part of the partition structure X as an independent node, and the similarity measurement formula between nodes is: In the formula, t is a hyperparameter; similarity is used to obtain interactive nodes and construct a complete graph CG; (2) Assuming a very small number σ∈(0,1), for CG, retain the k2 edges with the largest weight for each node to obtain a sparse subgraph make The compression rate ρ<σ and k2 is the smallest. At this time, the sparse subgraph That is the required graph structure data G ** .
Citation Information
Patent Citations
Global threshold partitioning method for partitioning target image
CN102819841A
Image region segmentation model training method and device, and image region segmentation method and device
CN111598900A
Entropy based image separation
US20120075440A1
Method for Segmenting Images Using Superpixels and Entropy Rate Clustering
US20120275702A1
