Method, device and equipment for feature element segmentation

Abstract

The present disclosure provides a ground object element segmentation method, device and equipment, which can be applied to the field of computer vision and image processing. The method comprises: clustering pixels in an original image according to a preset feature category to obtain a plurality of superpixel blocks and a mask corresponding to each superpixel block; obtaining a first superpixel feature according to the mask corresponding to each superpixel block and the original image; obtaining a second superpixel feature according to the first superpixel feature, a ground object element category co-occurrence matrix and a ground object element category correlation matrix; obtaining a first original image feature according to the second superpixel feature and the original image; and obtaining a ground object element segmentation result corresponding to the original image according to the first original image feature.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of computer vision, the field of image processing, and more particularly, to a ground feature segmentation method, device and equipment. BACKGROUND

[0002] With the rapid development of remote sensing technology, it is also easier to obtain high-resolution remote sensing images, and the demand for fine interpretation of remote sensing images is increasingly urgent. Ground feature segmentation refers to assigning a ground feature class label to each pixel of a remote sensing image. In recent years, with the development of deep neural networks, methods based on fully convolutional networks have become the mainstream network for semantic segmentation of images in natural scenes.

[0003] In the process of implementing the present disclosure, the inventors have found that the related art has at least the following problem: the existing related art has low boundary accuracy for segmenting ground features in an image. SUMMARY

[0004] Therefore, the present disclosure provides a ground feature segmentation method, device and equipment.

[0005] One aspect of the present disclosure provides a ground feature segmentation method, comprising:

[0006] According to a predetermined feature category, the pixels in the original image are clustered to obtain a plurality of superpixel blocks and a mask corresponding to each superpixel block, wherein the original image includes the ground feature, the size of the mask is the same as the size of the original image, the pixel value of the mask at the pixel position corresponding to the superpixel block is a first value, and the pixel value at other pixel positions is a second value;

[0007] According to the mask corresponding to each superpixel block and the original image, a first superpixel feature is obtained;

[0008] According to the first superpixel feature, a ground feature class co-occurrence matrix and a ground feature class correlation matrix, a second superpixel feature is obtained, wherein the ground feature class co-occurrence matrix represents a matrix recording the probability of multiple ground feature classes appearing in the same image at the same time, and the ground feature class correlation matrix represents a matrix recording the similarity between multiple ground feature classes;

[0009] According to the second superpixel feature and the original image, a first original image feature is obtained;

[0010] According to the first original image feature, a ground feature segmentation result corresponding to the original image is obtained.

[0011] According to an embodiment of the present disclosure, the clustering of the pixels in the original image according to the preset feature category to obtain a plurality of super-pixel blocks and a mask corresponding to each of the super-pixel blocks comprises:

[0012] According to the preset feature category, the original image is processed by using a super-pixel-based neural network to obtain the plurality of super-pixel blocks.

[0013] For each of the super-pixel blocks, the pixel value at the pixel position corresponding to the super-pixel block is set to the first numerical value, and the pixel value at the pixel position other than the pixel position corresponding to the super-pixel block is set to the second numerical value, to obtain the mask.

[0014] According to an embodiment of the present disclosure, the obtaining of the ground feature element segmentation result corresponding to the original image according to the first original image feature comprises:

[0015] The first original image feature is processed by using a semantic segmentation model to obtain the segmentation result corresponding to the original image.

[0016] According to an embodiment of the present disclosure, a plurality of dilated convolution kernels with different dilated coefficients are arranged in the semantic segmentation model, and the processing of the first original image feature by using the semantic segmentation model to obtain the segmentation result corresponding to the original image comprises:

[0017] For each of the plurality of dilated convolution kernels, the original image is convolved by using the dilated convolution kernel to obtain a first dilated convolution feature, wherein the size of the first dilated convolution feature is the same as the size of the first original image feature.

[0018] The first dilated convolution feature and the first original image feature are added to obtain a second dilated convolution feature.

[0019] The second dilated convolution features corresponding to the plurality of dilated convolution kernels are added to obtain a third dilated convolution feature.

[0020] The third dilated convolution feature is processed to obtain the segmentation result corresponding to the original image.

[0021] According to an embodiment of the present disclosure, the obtaining of the second super-pixel feature according to the first super-pixel feature, the ground feature element category co-occurrence matrix, and the ground feature element category correlation matrix comprises:

[0022] The first super-pixel feature, the ground feature element category co-occurrence matrix, and the ground feature element category correlation matrix are processed by using a super-pixel-based graph convolutional neural network to obtain the second super-pixel feature.

[0023] According to an embodiment of the present disclosure, the first original image feature is obtained according to the second superpixel feature and the original image.

[0024] The feature is extracted from the original image to obtain a second original image feature.

[0025] The second superpixel feature and the second original image feature are processed by using a cross-attention mechanism to obtain the first original image feature.

[0026] According to an embodiment of the present disclosure, the first superpixel feature is obtained according to the mask corresponding to each superpixel block and the original image.

[0027] The mask is multiplied by the second original image feature to obtain a third superpixel feature.

[0028] The third superpixel feature is globally averaged and pooled to obtain the first superpixel feature.

[0029] Another aspect of the present disclosure provides a ground object element segmentation device, comprising:

[0030] The superpixel block and the mask obtaining module is used to cluster the pixels in the original image according to a preset feature category to obtain a plurality of superpixel blocks and a mask corresponding to each superpixel block, wherein the original image comprises the ground object element, the size of the mask is the same as the size of the original image, the pixel value of the mask at the pixel position corresponding to the superpixel block is a first value, and the pixel value at other pixel positions is a second value.

[0031] The first superpixel feature obtaining module is used to obtain a first superpixel feature according to the mask corresponding to each superpixel block and the original image.

[0032] The second superpixel feature obtaining module is used to obtain a second superpixel feature according to the first superpixel feature, a ground object element category co-occurrence matrix and a ground object element category correlation matrix, wherein the ground object element category co-occurrence matrix represents a matrix recording the probability of multiple ground object element categories appearing in the same image at the same time, and the ground object element category correlation matrix represents a matrix recording the similarity between multiple ground object element categories.

[0033] The first original image feature obtaining module is used to obtain a first original image feature according to the second superpixel feature and the original image.

[0034] The segmentation result obtaining module is used to obtain a ground object element segmentation result corresponding to the original image according to the first original image feature.

[0035] Another aspect of the present disclosure provides an electronic device, comprising: one or more processors; a memory for storing one or more instructions, wherein when the one or more instructions are executed by the one or more processors, the one or more processors implement the method as described above.

[0036] Another aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, which when executed by a processor, cause the processor to implement the method as described above.

[0037] Another aspect of the present disclosure provides a computer program product, comprising computer-executable instructions for implementing the method as described above when executed.

[0038] According to an embodiment of the present disclosure, by clustering pixels in an original image including ground object elements according to a preset feature category, a plurality of superpixel blocks and a mask plate respectively corresponding to each superpixel block are obtained, an initial ground object segmentation result and a mask plate with a binary label constraint on a ground object boundary are obtained, then according to the mask plate respectively corresponding to each superpixel block and the original image, a first superpixel feature is obtained, an initial superpixel feature respectively corresponding to the plurality of superpixel blocks is obtained, according to the first superpixel feature, a ground object element category co-occurrence matrix and a ground object element category correlation matrix, a second superpixel feature is obtained, the first superpixel feature is corrected by using the ground object element category co-occurrence matrix and the ground object element category correlation matrix, so that the topological relationship between ground object elements is increased in the second superpixel feature, a second superpixel feature respectively corresponding to the plurality of superpixel blocks with more accurate boundary information is obtained, then according to the second superpixel feature with more accurate boundary information and the original image, a first original image feature is obtained, so that the topological relationship between ground object elements is increased in the first original image feature, a first original image feature with more accurate boundary information is obtained, according to the first original image feature, a ground object element segmentation result corresponding to the original image is obtained, and a more accurate segmentation boundary is obtained. BRIEF DESCRIPTION OF DRAWINGS

[0039] The above and other objects, features and advantages of the present disclosure will become more apparent from the following description when taken in conjunction with the accompanying drawings, in which:

[0040] Figure 1 A scenario diagram of an application of a ground object element segmentation method according to an embodiment of the present disclosure is schematically shown;

[0041] Figure 2 A flowchart of a ground object element segmentation method according to an embodiment of the present disclosure is schematically shown;

[0042] Figure 3 A flowchart of another ground object element segmentation method according to an embodiment of the present disclosure is schematically shown;

[0043] Figure 4 a block diagram of a terrain feature segmentation device according to an embodiment of the present disclosure is schematically shown; and

[0044] Figure 5 a block diagram of a computer system suitable for implementing the method described above according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0045] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood, however, that the description which follows is merely illustrative and is not intended to limit the scope of the present disclosure. In the following detailed description of embodiments of the present disclosure, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure. However, it will be apparent to one skilled in the art that one or more embodiments of the present disclosure can be practiced without these specific details. In other instances, well-known structures and functions have not been described in detail in order to avoid obscuring aspects of the present disclosure.

[0046] The terms used herein are merely used to describe specific embodiments and are not intended to limit the present disclosure. The terms "include" and "have" and the like used herein indicate the presence of the described features, steps, operations, and / or components but do not preclude the presence or addition of one or more other features, steps, operations, or components.

[0047] All terms used herein, including technical and scientific terms, have the same meanings as those generally understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having meanings consistent with the context of the present description, and should not be interpreted in an idealized or overly formal way.

[0048] In the case of using expressions similar to "at least one of A, B, and C, etc.", it should generally be interpreted to include at least one of each item enumerated, but not necessarily one of each item enumerated only. In other words, the expression "at least one of A, B, and C, etc." should be interpreted to mean that A alone, B alone, C alone, a combination of A and B, a combination of A and C, a combination of B and C, or a combination of A and B and C are all included. In the case of using expressions similar to "at least one of A, B, or C, etc.", it should generally be interpreted to include at least one of each item enumerated, but not necessarily one of each item enumerated only. In other words, the expression "at least one of A, B, or C, etc." should be interpreted to mean that A alone, B alone, C alone, a combination of A and B, a combination of A and C, a combination of B and C, or a combination of A and B and C are all included.

[0049] In the technical solutions of the present disclosure, the collection, storage, use, processing, transmission, provision, disclosure and application of data (such as, but not limited to, user personal information) comply with relevant laws and regulations, necessary security measures are taken, and the public order and good customs are not violated.

[0050] In view of the low boundary precision of the existing related art for segmenting ground object elements in an image, the present disclosure provides a ground object element segmentation method, device and equipment.

[0051] Embodiments of the present disclosure provide a ground object element segmentation method, which comprises clustering pixels in an original image according to a preset feature category to obtain a plurality of superpixel blocks and a mask corresponding to each superpixel block, wherein the original image comprises ground object elements, the size of the mask is the same as that of the original image, the pixel value of the mask at the pixel position corresponding to the superpixel block is a first numerical value, and the pixel value at other pixel positions is a second numerical value; obtaining first superpixel features according to the mask corresponding to each superpixel block and the original image; obtaining second superpixel features according to the first superpixel features, a ground object element category co-occurrence matrix and a ground object element category correlation matrix, wherein the ground object element category co-occurrence matrix represents a matrix recording the probability of multiple ground object element categories appearing in the same image at the same time, and the ground object element category correlation matrix represents a matrix recording the similarity between multiple ground object element categories; obtaining first original image features according to the second superpixel features and the original image; and obtaining a ground object element segmentation result corresponding to the original image according to the first original image features.

[0052] According to embodiments of the present disclosure, by clustering pixels in an original image comprising ground object elements according to a preset feature category to obtain a plurality of superpixel blocks and a mask corresponding to each superpixel block, an initial ground object segmentation result and a mask plate with a binary label constraint for the ground object boundary are obtained, then first superpixel features are obtained according to the mask plate corresponding to each superpixel block and the original image, initial superpixel features corresponding to the plurality of superpixel blocks are obtained, second superpixel features are obtained according to the first superpixel features, a ground object element category co-occurrence matrix and a ground object element category correlation matrix, the first superpixel features are corrected by using the ground object element category co-occurrence matrix and the ground object element category correlation matrix, the topological relationship between ground object elements is increased in the second superpixel features, second superpixel features corresponding to the plurality of superpixel blocks with more accurate boundary information are obtained, then first original image features are obtained according to the second superpixel features with more accurate boundary information and the original image, the topological relationship between ground object elements is increased in the first original image features, first original image features with more accurate boundary information are obtained, a ground object element segmentation result corresponding to the original image is obtained according to the first original image features, and a more accurate segmentation boundary is obtained.

[0053] Figure 1 An application scenario diagram of the ground feature element segmentation method according to an embodiment of the present disclosure is schematically shown.

[0054] As Figure 1 shown, the application scenario 100 according to this embodiment can include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 can include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0055] A user can use at least one of the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0056] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smartphones, tablet computers, laptop computers, desktop computers, etc.

[0057] The server 105 can be a server providing various services, such as a background management server supporting a website browsed by a user using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (only as an example). The background management server can analyze and process received user requests and other data, and feed back the processing results (such as web pages, information, or data, etc. obtained or generated according to user requests) to the terminal device.

[0058] It should be noted that the ground feature element segmentation method provided by the embodiments of the present disclosure can be generally executed by the server 105. Correspondingly, the ground feature element segmentation apparatus provided by the embodiments of the present disclosure can be generally arranged in the server 105. The ground feature element segmentation method provided by the embodiments of the present disclosure can also be executed by a server or a server cluster different from the server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Correspondingly, the ground feature element segmentation apparatus provided by the embodiments of the present disclosure can also be arranged in a server or a server cluster different from the server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105.

[0059] It should be understood that Figure 1 The number of terminal devices, networks and servers in the above-mentioned system is only illustrative. According to the implementation needs, there can be any number of terminal devices, networks and servers.

[0060] Figure 2 A flowchart of a ground feature element segmentation method according to an embodiment of the present disclosure is schematically shown.

[0061] As shown in FIG. 2, Figure 2 The method includes operations S201-S205.

[0062] In operation S201, pixels in an original image are clustered according to a preset feature category, to obtain a plurality of superpixel blocks and a mask corresponding to each superpixel block, wherein the original image includes a ground feature element, the mask has the same size as the original image, and the pixel value of the mask at a pixel position corresponding to the superpixel block in the original image is a first value, and the pixel value of the mask at other pixel positions is a second value.

[0063] According to an embodiment of the present disclosure, a superpixel block represents a pixel set composed of a plurality of adjacent pixels in an original image.

[0064] According to an embodiment of the present disclosure, there is no overlapping pixel between the plurality of superpixel blocks, and the plurality of superpixel blocks constitute the original image.

[0065] According to an embodiment of the present disclosure, each superpixel block has a mask corresponding thereto, the mask has the same size as the original image, the pixel value of the mask at a pixel position corresponding to the superpixel block in the original image is a first value, and the pixel value of the mask at other pixel positions except the pixel position of the first value is a second value.

[0066] According to an embodiment of the present disclosure, the first numerical value is used to reserve or increase the pixel value at the pixel position in the mask corresponding to the superpixel block, and the second numerical value is used to filter out or reduce the pixel value at the pixel position in the mask other than the pixel position of the first numerical value.

[0067] According to an embodiment of the present disclosure, the first numerical value may, for example, be 1, 2, or 3, and the embodiment of the present disclosure does not limit the specific first numerical value, which can be selected according to actual conditions.

[0068] According to an embodiment of the present disclosure, the second numerical value may, for example, be 0, 0.0001, or 0.00001, and the embodiment of the present disclosure does not limit the specific second numerical value, which can be selected according to actual conditions.

[0069] According to an embodiment of the present disclosure, the preset feature category represents the category of similar features of the pixels in the original image.

[0070] According to an embodiment of the present disclosure, the preset feature category may, for example, be color, texture, and pixel position, and may also be color, texture, pixel position, and a feature element category, and the embodiment of the present disclosure does not limit the specific preset feature category, which can be selected according to actual conditions.

[0071] According to an embodiment of the present disclosure, the pixels in the original image can be clustered according to the preset feature category by using a deep neural network (DNN) to obtain a plurality of superpixel blocks and a mask corresponding to each superpixel block.

[0072] According to an embodiment of the present disclosure, before the pixels in the original image are clustered according to the preset feature category by using the DNN, the DNN can be trained according to the preset feature category, wherein the preset feature category can include one or more of color, texture, pixel position, and a feature element category.

[0073] According to an embodiment of the present disclosure, the size of the original image is selected according to a specific deep neural network.

[0074] According to an embodiment of the present disclosure, by clustering the pixels in the original image according to the preset feature category, a plurality of superpixel blocks and a mask corresponding to each superpixel block are obtained, which realizes the segmentation of the original image into superpixel blocks that are close in the preset feature category (such as color, texture, etc.) and conform to human brain cognition, so that the computer interprets the original image in the manner that the human brain observes images to obtain an initial feature segmentation result and a mask plate with a binary label constraint on the feature boundary.

[0075] In operation S202, a first superpixel feature is obtained according to the mask corresponding to each superpixel block and the original image.

[0076] According to an embodiment of the present disclosure, the original image can be processed by using a full convolutional network to obtain a feature corresponding to the original image, and then the feature corresponding to the original image and a mask corresponding to each superpixel block are multiplied, and then the multiplied result is reduced in dimension to obtain the first superpixel feature.

[0077] In operation S203, the second superpixel feature is obtained according to the first superpixel feature, the ground object element category co-occurrence matrix and the ground object element category correlation matrix, wherein the ground object element category co-occurrence matrix represents a matrix recording the probability of multiple ground object element categories appearing in the same image at the same time, and the ground object element category correlation matrix represents a matrix recording the similarity between multiple ground object element categories.

[0078] According to an embodiment of the present disclosure, the first superpixel feature, the ground object element category co-occurrence matrix and the ground object element category correlation matrix can be processed by using a relevant convolutional neural network to obtain the second superpixel feature.

[0079] According to an embodiment of the present disclosure, the specific ground object element category and the number of ground object element categories can be selected according to actual conditions, and the embodiment of the present disclosure does not limit the specific ground object element category and the number of specific ground object element categories.

[0080] According to an embodiment of the present disclosure, for example, in the case where the ground object element category includes a car, a ground, a house, a tree and a person, the ground object element category co-occurrence matrix can include the probability of the car appearing in the same image at the same time with the ground, the house, the tree and the person respectively, the probability of the ground appearing in the same image at the same time with the house, the tree and the person respectively, the probability of the house appearing in the same image at the same time with the tree and the person respectively, and the probability of the tree and the person appearing in the same image at the same time.

[0081] According to an embodiment of the present disclosure, for example, in the case where the ground object element category includes a car, a ground, a house, a tree and a person, the ground object element category correlation matrix can include the similarity between the car and the ground, the house, the tree and the person respectively, the similarity between the ground and the house, the tree and the person respectively, the similarity between the house and the tree and the person respectively, and the similarity between the tree and the person.

[0082] According to an embodiment of the present disclosure, by obtaining the second superpixel feature according to the first superpixel feature, the ground object element category co-occurrence matrix and the ground object element category correlation matrix, the first superpixel feature is updated, refined and corrected according to the ground object element category co-occurrence matrix and the ground object element category correlation matrix to obtain the second superpixel feature, so that the second superpixel feature includes more topological relationships between ground object element categories, and the second superpixel feature corresponding to multiple superpixel blocks respectively is obtained with more accurate boundary information.

[0083] At operation S204, a first original image feature is obtained according to the second superpixel feature and the original image.

[0084] According to an embodiment of the present disclosure, the original image can be processed by using a full convolution network to obtain a feature corresponding to the original image, and then the feature corresponding to the original image and the second superpixel feature can be processed by using a cross-attention mechanism to obtain the first original image feature, so as to realize the refinement of the feature at the boundary in the original image according to the second superpixel feature.

[0085] According to an embodiment of the present disclosure, by obtaining the first original image feature according to the second superpixel feature and the original image, the feature at the boundary in the original image is refined according to the second superpixel feature to obtain the first original image feature after the boundary feature is refined by the second superpixel feature, so that the first original image feature includes more topological relationships between the feature categories of the ground elements, and the boundary information included in the first original image feature is more accurate.

[0086] At operation S205, a ground element element segmentation result corresponding to the original image is obtained according to the first original image feature.

[0087] According to an embodiment of the present disclosure, the first original image feature can be processed by using a related segmentation model to obtain a ground element element segmentation result corresponding to the original image, and a ground element element segmentation boundary corresponding to the original image is obtained.

[0088] According to an embodiment of the present disclosure, since the first original image feature includes more topological relationships between the feature categories of the ground elements, and the boundary information included in the first original image feature is more accurate, the ground element element segmentation result corresponding to the original image is obtained according to the first original image feature, that is, the ground element element segmentation boundary corresponding to the original image is obtained, so that the ground element element segmentation boundary corresponding to the original image obtained according to the first original image feature is more accurate.

[0089] According to an embodiment of the present disclosure, by clustering pixels in an original image including ground object elements according to preset feature categories, a plurality of superpixel blocks and a mask corresponding to each superpixel block are obtained, an initial ground object segmentation result and a mask with a binary label constraint on a ground object boundary are obtained, then according to the mask corresponding to each superpixel block and the original image, a first superpixel feature is obtained, an initial superpixel feature corresponding to the plurality of superpixel blocks is obtained, according to the first superpixel feature, a ground object element category co-occurrence matrix and a ground object element category correlation matrix, a second superpixel feature is obtained, the first superpixel feature is corrected by using the ground object element category co-occurrence matrix and the ground object element category correlation matrix, so that the topological relationship between ground object elements is increased in the second superpixel feature, a second superpixel feature corresponding to the plurality of superpixel blocks with more accurate boundary information is obtained, then according to the second superpixel feature with more accurate boundary information and the original image, a first original image feature is obtained, so that the first original image feature increases the topological relationship between ground object elements, a first original image feature with more accurate boundary information is obtained, and according to the first original image feature, a ground object element segmentation result corresponding to the original image is obtained, and a more accurate segmentation boundary is obtained.

[0090] According to an embodiment of the present disclosure, according to an embodiment of the present disclosure, for the operation S201 as shown in the figure Figure 2 According to the preset feature categories, the original image is processed by using a superpixel-guided neural network (SGNet) to obtain a plurality of superpixel blocks.

[0091] According to the preset feature categories, the original image is processed by using a superpixel-guided neural network (SGNet) to obtain a plurality of superpixel blocks.

[0092] For each superpixel block, the pixel value at the pixel position corresponding to the superpixel block is set to a first value, and the pixel value at the pixel position other than the pixel position corresponding to the superpixel block is set to a second value, to obtain a mask.

[0093] According to an embodiment of the present disclosure, for example, the SGNet in the deep neural network can be used to cluster the pixels in the original image with a size of 512*512 according to color, texture and pixel position, to obtain 2000 superpixel blocks, and then for each superpixel block in the 2000 superpixel blocks, the pixel value at the pixel position corresponding to the superpixel block is set to 1, and the pixel value at the pixel position other than the pixel position corresponding to the superpixel block is set to 0, to obtain a mask, so as to divide the original image into superpixel blocks that are close in color, texture and the like and conform to human brain cognition, and reduce the amount of calculation while making the computer interpret the original image in the way that the human brain observes images.

[0094] According to embodiments of this disclosure, SGNet can also be used to cluster pixels in an original image of size 512*512 based on color, texture, pixel location, and feature category to obtain 2000 superpixel blocks. Then, for each of the 2000 superpixel blocks, the pixel value at the pixel location corresponding to the superpixel block is set to 1, and the pixel values ​​at other pixel locations are set to 0 to obtain a mask. This achieves the segmentation of the original image into superpixel blocks that are similar in color, texture, etc., in accordance with human brain cognition, reducing the amount of computation while enabling the computer to interpret the original image in the way that the human brain observes images.

[0095] According to embodiments of this disclosure, before using SGNet to cluster pixels in the original image according to preset feature categories, SGNet can be trained according to preset feature categories, wherein the preset feature categories may include one or more of color, texture, pixel location and ground feature categories.

[0096] According to embodiments of this disclosure, for example, before using SGNet to cluster pixels in the original image according to preset feature categories, SGNet can be trained based on color, texture, pixel location, and feature category.

[0097] According to embodiments of this disclosure, the original image is processed using SGNet based on preset feature categories, such as color, texture, pixel location, and land feature category, to obtain multiple superpixel blocks and an initial land feature segmentation result. This achieves the segmentation of the original image into superpixel blocks that are similar in color, texture, etc., in accordance with human brain cognition, reducing the computational load while enabling the computer to interpret the original image in the way the human brain observes images. Then, for each superpixel block, the pixel value at the pixel location corresponding to the superpixel block is set to a first value, and the pixel values ​​at other pixel locations besides the pixel location corresponding to the superpixel block are set to a second value, resulting in a mask corresponding to each superpixel block. This mask has binary label constraints on the land feature boundaries, preparing for subsequent calculation of the superpixel features corresponding to each superpixel block using the mask.

[0098] According to embodiments of this disclosure, such as Figure 2 Operation S202, shown, obtains the first superpixel feature based on the mask and the original image corresponding to each superpixel block, and may include the following operations:

[0099] The mask is multiplied by the second original image feature to obtain the third superpixel feature;

[0100] Global average pooling is performed on the third superpixel feature to obtain the first superpixel feature.

[0101] According to embodiments of this disclosure, the second original image features can be obtained by processing the original image using a fully convolutional network. For example, the second original image features can be obtained by processing the original image using an 18-layer residual network.

[0102] According to embodiments of this disclosure, when the size of the mask is inconsistent with the size of the second original image feature, the larger of the mask and the second original image can be subjected to dimensionality reduction processing so that the size of the mask is consistent with the size of the second original image feature.

[0103] According to embodiments of this disclosure, for example, when the size of the mask is larger than the size of the second original image feature, the mask can be dimensionality reduced so that the size of the mask is consistent with the size of the second original image feature.

[0104] According to an embodiment of this disclosure, when 2000 superpixel blocks are obtained, 2000 masks can be obtained. Then, for each of the 2000 masks, the size of the mask can be reduced to be consistent with the size of the second original image feature. Then, the masks with consistent size are multiplied by the second original image feature to obtain the third superpixel feature, resulting in 2000 third superpixel features.

[0105] According to an embodiment of this disclosure, for each of the 2000 third superpixel features, global average pooling is performed on the third superpixel feature to obtain the first superpixel feature, thereby obtaining 2000 first superpixel features. In the case of an original image of 512*512, 2000 512-dimensional first superpixel features can be obtained.

[0106] According to embodiments of this disclosure, a third superpixel feature is obtained by multiplying the mask with the second original image feature, and then global average pooling is performed on the third superpixel feature to obtain the first superpixel feature, thereby realizing the acquisition of initial superpixel features corresponding to multiple superpixel blocks based on the mask and the original image.

[0107] According to embodiments of this disclosure, such as Figure 2 Operation S203, as shown, obtains the second superpixel feature based on the first superpixel feature, the land feature category co-occurrence matrix, and the land feature category correlation matrix. This operation may include the following steps:

[0108] The first superpixel feature, the feature class co-occurrence matrix and the feature class correlation matrix are processed by using a superpixel-guided graph convolutional network (SGGCN) to obtain a second superpixel feature.

[0109] According to an embodiment of the present disclosure, for example, in the case of obtaining 2000 first superpixel features of 512 dimensions, the 2000 first superpixel features of 512 dimensions can be combined into a feature including the 2000 first superpixel features of 512 dimensions, and then the feature is input into the SGGCN together with the feature class co-occurrence matrix and the feature class correlation matrix to obtain the second superpixel feature.

[0110] According to an embodiment of the present disclosure, by processing the first superpixel feature, the feature class co-occurrence matrix and the feature class correlation matrix by using the SGGCN to obtain the second superpixel feature, the first superpixel feature is corrected and updated by using the prior knowledge, i.e., the feature class co-occurrence matrix and the feature class correlation matrix, so that the topological relationship between features is increased in the second superpixel feature, and the second superpixel feature corresponding to the multiple superpixel blocks respectively and having more accurate boundary information is obtained.

[0111] According to an embodiment of the present disclosure, as shown in operation S204, the first original image feature is obtained according to the second superpixel feature and the original image, which can include the following operations: Figure 2

[0112] The feature of the original image is extracted to obtain a second original image feature;

[0113] The second superpixel feature and the second original image feature are processed by using a cross-attention mechanism to obtain the first original image feature.

[0114] According to an embodiment of the present disclosure, the second original image feature can be obtained by processing the original image by using a full convolutional network, for example, the second original image feature can be obtained by processing the original image by using an 18-layer residual network.

[0115] The feature of the original image can be extracted by using a full convolutional network to obtain the second original image feature, for example, the feature of the original image can be extracted by using an 18-layer residual network to obtain the second original image feature.

[0116] ​According to an embodiment of the present disclosure, by extracting features from the original image, a second original image feature is obtained, and then the second superpixel feature which is more accurate in boundary information and includes the topological relationship between the surface feature elements and the second original image feature are processed by using the cross attention mechanism to obtain the first original image feature, so as to realize the refinement of the boundary features in the second original image feature by using the second superpixel feature, so that the first original image feature increases the topological relationship between the surface feature elements, and the first original image feature which is more accurate in boundary information is obtained.

[0117] According to an embodiment of the present disclosure, for the operation S205 as shown in the figure, Figure 2 According to an embodiment of the present disclosure, the operation S205 as shown in the figure, according to the first original image feature, the surface feature element segmentation result corresponding to the original image can include the following operations:

[0118] The first original image feature is processed by using the semantic segmentation model to obtain the segmentation result corresponding to the original image.

[0119] According to an embodiment of the present disclosure, by processing the first original image feature by using the semantic segmentation model to obtain the segmentation result corresponding to the original image, the segmentation boundary corresponding to the original image is obtained, which realizes the supervision of superpixel segmentation by using the boundary information included in the first original image feature, and further guarantees the accuracy at the surface feature element segmentation boundary.

[0120] According to an embodiment of the present disclosure, a plurality of hollow convolution kernels with different dilation coefficients are set in the semantic segmentation model, and the first original image feature is processed by using the semantic segmentation model to obtain the segmentation result corresponding to the original image, which includes:

[0121] For each hollow convolution kernel in the plurality of hollow convolution kernels, the hollow convolution kernel is convolved with the original image to obtain a first hollow convolution feature, wherein the size of the first hollow convolution feature is the same as the size of the first original image feature;

[0122] The first hollow convolution feature and the first original image feature are added to obtain a second hollow convolution feature;

[0123] The second hollow convolution features corresponding to the plurality of hollow convolution kernels are added to obtain a third hollow convolution feature;

[0124] The third hollow convolution feature is processed to obtain the segmentation result corresponding to the original image.

[0125] According to an embodiment of the present disclosure, the number of the plurality of hollow convolution kernels in the semantic segmentation model can be 3, 4 or 7, etc. The embodiment of the present disclosure does not limit the specific number of hollow convolution kernels, and the number of hollow convolution kernels can be selected according to actual conditions.

[0126] According to an embodiment of the present disclosure, the dilation coefficients of the hole convolution kernels can be selected according to actual conditions, and the present disclosure does not limit the specific dilation coefficients of the hole convolution kernels.

[0127] According to an embodiment of the present disclosure, when the number of hole convolution kernels is 4, the dilation coefficients of the hole convolution kernels can be, for example, 1, 6, 12 and 18.

[0128] According to an embodiment of the present disclosure, for example, the hole convolution kernels with dilation coefficients of 1, 6, 12 and 18 in the semantic segmentation model can be respectively convolved with the original image to obtain first hole convolution features, and four first hole convolution features with the same size as the first original image feature are obtained. Then, the four first hole convolution features are respectively added to the first original image feature to obtain four second hole convolution features with the same size as the first original image feature. Then, the four second hole convolution features are added to obtain third hole convolution features. Then, the third hole convolution features are processed by using the semantic segmentation model to obtain a segmentation result corresponding to the original image and a segmentation boundary corresponding to the original image. Through multiple hole convolutions and pyramid pooling to increase the receptive field, the first hole convolution features, the second hole convolution features and the third hole convolution features obtain more global information, and the boundary segmentation accuracy corresponding to the original image obtained according to the third hole convolution features is further improved.

[0129] According to an embodiment of the present disclosure, for each hole convolution kernel in the plurality of hole convolution kernels, the hole convolution kernel is convolved with the original image to obtain first hole convolution features, and then the first hole convolution features are added to the first original image features to obtain second hole convolution features. The second hole convolution features corresponding to the plurality of hole convolution kernels are added to obtain third hole convolution features, and the third hole convolution features are processed to obtain a segmentation result corresponding to the original image and a segmentation boundary corresponding to the original image. Through multiple hole convolutions and pyramid pooling to increase the receptive field, the first hole convolution features, the second hole convolution features and the third hole convolution features obtain more global information, and the boundary segmentation accuracy corresponding to the original image obtained according to the third hole convolution features is further improved.

[0130] Figure 3 A flowchart of another ground object element segmentation method according to an embodiment of the present disclosure is schematically shown.

[0131] As Figure 3As shown, a remote sensing image is input as an original image in step 301. In step 302, the remote sensing image is superpixel segmented, i.e. clustered, according to color, texture, and ground object element category, etc. by using the DNN which has been trained in step 304, to obtain a plurality of superpixel blocks and a mask corresponding to each of the superpixel blocks, to obtain an initial ground object segmentation result and a mask board with a binary label constraint for the ground object boundary, to realize segmentation of the original image into superpixel blocks which are close in color, texture, etc. in accordance with human brain cognition, and to reduce the amount of calculation while enabling the computer to interpret the original image in the manner of human brain observation of images.

[0132] In step 303, feature extraction is performed on the remote sensing image by using a full convolutional network to obtain second original image features. In step 305, a plurality of masks are multiplied with the second original image features respectively to obtain a plurality of third superpixel features. In step 306, average global pooling is performed on the plurality of third superpixel features respectively, i.e. feature dimension reduction is performed, to obtain a plurality of first superpixel features, to realize obtaining initial superpixel features corresponding to the plurality of superpixel blocks according to the masks and the original image.

[0133] In step 307, the first superpixel features, the ground object element category co-occurrence matrix, and the ground object element category correlation matrix are processed, i.e. graph reasoning is performed, by using a superpixel-based graph convolutional neural network to obtain second superpixel features, to realize correcting the first superpixel features by using the ground object element category co-occurrence matrix and the ground object element category correlation matrix, so that the topological relationship between ground object elements is increased in the second superpixel features, to obtain second superpixel features corresponding to the plurality of superpixel blocks with more accurate boundary information.

[0134] In step 308, the second superpixel features and the second original image features are processed by using a cross-attention mechanism, i.e. feature reconstruction is performed on the second original image features, to obtain first original image features, so that the topological relationship between ground object elements is increased in the first original image features, to obtain first original image features with more accurate boundary information.

[0135] In step 309, semantic segmentation is performed on the first original image features by using a semantic segmentation model to obtain a segmentation result corresponding to the original image, to obtain a more accurate segmentation boundary.

[0136] According to embodiments of the present disclosure, before superpixel segmentation of the remote sensing image according to color, texture, and ground object element category, etc. by using the DNN, the DNN is trained in step 304 by using color, texture, pixel position, and ground object element category, etc. so that the DNN performs boundary division on the remote sensing image according to color, texture, pixel position, and ground object element category, etc. with boundary perception in accordance with preset requirements.

[0137] It should be noted that, unless the execution sequence between different operations is explicitly stated to be different, or the execution sequence between different operations is technically different, the execution sequence between multiple operations can not be distinguished, and multiple operations can be executed simultaneously.

[0138] Figure 4 A block diagram of a ground object element segmentation device according to an embodiment of the present disclosure is schematically shown.

[0139] As shown in Figure 4 The ground object element segmentation device 400 includes a superpixel block and mask obtaining module 410, a first superpixel feature obtaining module 420, a second superpixel feature obtaining module 430, a first original image feature obtaining module 440, and a segmentation result obtaining module 450.

[0140] The superpixel block and mask obtaining module 410 is configured to cluster pixels in an original image according to a preset feature category to obtain a plurality of superpixel blocks and a mask corresponding to each superpixel block, respectively, wherein the original image includes a ground object element, the mask has the same size as the original image, and a pixel value of the mask at a pixel position corresponding to the superpixel block is a first numerical value, and a pixel value of the mask at other pixel positions is a second numerical value.

[0141] The first superpixel feature obtaining module 420 is configured to obtain a first superpixel feature according to the mask corresponding to each superpixel block and the original image.

[0142] The second superpixel feature obtaining module 430 is configured to obtain a second superpixel feature according to the first superpixel feature, a ground object element category co-occurrence matrix, and a ground object element category correlation matrix, wherein the ground object element category co-occurrence matrix represents a matrix recording a probability of multiple ground object element categories appearing in the same image at the same time, and the ground object element category correlation matrix represents a matrix recording a similarity between multiple ground object element categories.

[0143] The first original image feature obtaining module 440 is configured to obtain a first original image feature according to the second superpixel feature and the original image.

[0144] The segmentation result obtaining module 450 is configured to obtain a ground object element segmentation result corresponding to the original image according to the first original image feature.

[0145] According to an embodiment of the present disclosure, the superpixel block and mask obtaining module includes a plurality of superpixel block obtaining sub-modules and a mask obtaining sub-module.

[0146] The plurality of superpixel block obtaining sub-modules are configured to process the original image by using a superpixel-based neural network according to the preset feature category to obtain a plurality of superpixel blocks.

[0147] The mask obtaining submodule is configured to, for each superpixel block, set pixel values at pixel positions corresponding to the superpixel block to a first numerical value, and set pixel values at other pixel positions except the pixel positions corresponding to the superpixel block to a second numerical value, to obtain a mask.

[0148] According to an embodiment of the present disclosure, the segmentation result obtaining module comprises a segmentation result obtaining submodule.

[0149] The segmentation result obtaining submodule is configured to process the first original image feature by using a semantic segmentation model to obtain a segmentation result corresponding to the original image.

[0150] According to an embodiment of the present disclosure, the semantic segmentation model is provided with a plurality of dilated convolution kernels with different dilated coefficients, and the segmentation result obtaining submodule comprises a first dilated convolution feature obtaining unit, a second dilated convolution feature obtaining unit, a third dilated convolution feature obtaining unit, and a segmentation result obtaining unit.

[0151] The first dilated convolution feature obtaining unit is configured to, for each dilated convolution kernel in the plurality of dilated convolution kernels, perform convolution on the original image by using the dilated convolution kernel to obtain a first dilated convolution feature, wherein the size of the first dilated convolution feature is the same as the size of the first original image feature.

[0152] The second dilated convolution feature obtaining unit is configured to add the first dilated convolution feature and the first original image feature to obtain a second dilated convolution feature.

[0153] The third dilated convolution feature obtaining unit is configured to add the second dilated convolution features corresponding to the plurality of dilated convolution kernels respectively to obtain a third dilated convolution feature.

[0154] The segmentation result obtaining unit is configured to process the third dilated convolution feature to obtain a segmentation result corresponding to the original image.

[0155] According to an embodiment of the present disclosure, the second superpixel feature obtaining module comprises a second superpixel feature submodule.

[0156] The second superpixel feature submodule is configured to process the first superpixel feature, the feature class co-occurrence matrix, and the feature class correlation matrix by using a superpixel-based graph convolutional neural network to obtain a second superpixel feature.

[0157] According to an embodiment of the present disclosure, the first original image feature obtaining module comprises a second original image feature obtaining submodule and a first original image feature obtaining submodule.

[0158] The second original image feature obtaining submodule is configured to extract features of the original image to obtain second original image features.

[0159] The first original image feature obtaining submodule is configured to process the second superpixel features and the second original image features by using a cross-attention mechanism to obtain first original image features.

[0160] According to an embodiment of the present disclosure, the first superpixel feature obtaining module includes a third superpixel feature obtaining submodule and a first superpixel feature obtaining submodule.

[0161] The third superpixel feature obtaining submodule is configured to multiply the mask and the second original image features to obtain third superpixel features.

[0162] The first superpixel feature obtaining submodule is configured to perform global average pooling on the third superpixel features to obtain first superpixel features.

[0163] Any one or more of the modules, submodules, units, and subunits according to the embodiments of the present disclosure can be implemented in one module. Any one or more of the modules, submodules, units, and subunits according to the embodiments of the present disclosure can be split into multiple modules. Any one or more of the modules, submodules, units, and subunits according to the embodiments of the present disclosure can be at least partially implemented as a hardware circuit, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on a substrate, a system in a package, an application-specific integrated circuit (ASIC), or any other reasonable manner of hardware or firmware by integrating or packaging a circuit, or in any one of software, hardware, and firmware or in an appropriate combination of any one or more of them. Alternatively, one or more of the modules, submodules, units, and subunits according to the embodiments of the present disclosure can be at least partially implemented as computer program modules that can perform corresponding functions when the computer program modules are run.

[0164] For example, any of the superpixel block and mask obtaining module 410, the first superpixel feature obtaining module 420, the second superpixel feature obtaining module 430, the first original image feature obtaining module 440, and the segmentation result obtaining module 450 can be combined in one module / unit / sub-unit, or any of the modules / units / sub-units can be split into multiple modules / units / sub-units. Alternatively, at least part of the function of one or more of the modules / units / sub-units can be combined with at least part of the function of other modules / units / sub-units, and implemented in one module / unit / sub-unit. According to an embodiment of the present disclosure, at least one of the superpixel block and mask obtaining module 410, the first superpixel feature obtaining module 420, the second superpixel feature obtaining module 430, the first original image feature obtaining module 440, and the segmentation result obtaining module 450 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on board, a system on package, an application specific integrated circuit (ASIC), or any other reasonable manner of integrating or packaging a circuit, etc. hardware or firmware, or in any one of software, hardware, and firmware implementation or in a proper combination of any of them. Alternatively, at least one of the superpixel block and mask obtaining module 410, the first superpixel feature obtaining module 420, the second superpixel feature obtaining module 430, the first original image feature obtaining module 440, and the segmentation result obtaining module 450 can be at least partially implemented as a computer program module that can perform corresponding functions when the computer program module is run.

[0165] It should be noted that the feature element segmentation device part in the embodiments of the present disclosure corresponds to the feature element segmentation method part in the embodiments of the present disclosure, and the description of the feature element segmentation device part is specifically referred to the feature element segmentation method part, which will not be repeated here.

[0166] Figure 5 A block diagram of a computer system suitable for implementing the above-described methods according to an embodiment of the present disclosure is schematically shown. Figure 5 The computer system shown is only an example, and should not bring any limitation to the function and use range of the embodiments of the present disclosure.

[0167] As Figure 5As shown, the computer system 500 according to embodiments of the present disclosure includes a processor 501 that can perform various appropriate actions and processes according to programs stored in a read only memory (ROM) 502 or loaded into a random access memory (RAM) 503 from a storage section 508. The processor 501 can include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor, and / or a related chip set, and / or a dedicated microprocessor (e.g., an application specific integrated circuit (ASIC)), and so on. The processor 501 can also include an on-board memory for cache use. The processor 501 can include a single processing unit or multiple processing units for executing different actions of the method processes according to embodiments of the present disclosure.

[0168] In the RAM 503, various programs and data required for the operation of the system 500 are stored. The processor 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. The processor 501 performs various operations of the method processes according to embodiments of the present disclosure by executing the programs in the ROM 502 and / or the RAM 503. Note that the programs can also be stored in one or more memories other than the ROM 502 and the RAM 503. The processor 501 can also perform various operations of the method processes according to embodiments of the present disclosure by executing the programs stored in the one or more memories.

[0169] According to embodiments of the present disclosure, the system 500 can also include an input / output (I / O) interface 505 that is also connected to the bus 504. The system 500 can further include one or more of the following components connected to the I / O interface 505: an input section 506 including a keyboard, a mouse, etc.; an output section 507 including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, a modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as necessary. A removable recording medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is attached to the drive 510 as necessary, so that a computer program read therefrom is installed into the storage section 508 as necessary.

[0170] According to an embodiment of the present disclosure, the method flow according to the embodiments of the present disclosure can be implemented as a computer software program. For example, the embodiments of the present disclosure include a computer program product comprising a computer program carrying computer program code for executing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network by the communication section 509, and / or installed from the detachable medium 511. When the computer program is executed by the processor 501, the above-described functions defined in the system, device, apparatus, module, unit, etc. of the embodiments of the present disclosure are performed. According to an embodiment of the present disclosure, the system, device, apparatus, module, unit, etc. described above can be implemented by computer program modules.

[0171] The present disclosure also provides a computer-readable storage medium, which can be included in the device / apparatus / system described in the above embodiments, or can exist separately without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, which when executed, implement the methods according to the embodiments of the present disclosure.

[0172] According to an embodiment of the present disclosure, the computer-readable storage medium can be a non-volatile computer-readable storage medium. For example, it can include but is not limited to a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any appropriate combination thereof. In the present disclosure, the computer-readable storage medium can be any tangible medium that contains, or stores a program for use by or in connection with an instruction execution system, apparatus, or device.

[0173] For example, according to an embodiment of the present disclosure, the computer-readable storage medium can include one or more memories of the ROM 502 and / or the RAM 503 described above and / or one or more memories other than the ROM 502 and the RAM 503.

[0174] The embodiments of the present disclosure also include a computer program product comprising a computer program containing program codes for executing the method provided by the embodiments of the present disclosure, which program codes are used to make the electronic device implement the ground feature element segmentation method provided by the embodiments of the present disclosure when the computer program product is running on the electronic device.

[0175] When the computer program is executed by the processor 501, the above-described functions defined in the system / apparatus of the embodiments of the present disclosure are performed. According to an embodiment of the present disclosure, the system, device, apparatus, module, unit, etc. described above can be implemented by computer program modules.

[0176] In one embodiment, the computer program can be embodied on a tangible memory device, such as a magnetic storage device, an optical storage device, etc. In another embodiment, the computer program can be transmitted in a signal over a network, distributed across networks, downloaded and installed, and / or installed from a removable memory media 511. The computer program comprising the program code can be transmitted using any suitable network medium, including, but not limited to wireless, wired, etc., or any suitable combination of the foregoing.

[0177] According to an embodiment of the disclosure, program code for execution by a computer program can be written in any combination of one or more programming languages, and can specifically be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages include, but are not limited to, Java, C++, python, "C", or similar programming languages. Program code can execute entirely on a user's computing device, partly on a user's device, partly on a remote computing device, or entirely on a remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, such as through the Internet using an Internet Service Provider.

[0178] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions. Those skilled in the art will understand that the features recited in the various embodiments and / or claims of this disclosure can be combined and / or combined in various ways, even if such combinations or combinations are not expressly described in this disclosure. In particular, the features described in the various embodiments and / or claims of this disclosure may be combined and / or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0179] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. The scope of this disclosure is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. A method for feature segmentation, comprising: clustering pixels in an original image according to a preset feature category to obtain a plurality of superpixel blocks and a mask corresponding to each of the superpixel blocks, wherein the original image comprises a feature, the mask has the same size as the original image, and a pixel value of the mask at a pixel position corresponding to the superpixel block is a first value and a pixel value of the mask at other pixel positions is a second value; obtaining a first superpixel feature according to the mask corresponding to each of the superpixel blocks and the original image; obtaining a second superpixel feature according to the first superpixel feature, a feature category co-occurrence matrix and a feature category correlation matrix, wherein the feature category co-occurrence matrix represents a matrix recording a probability of a plurality of feature categories appearing in the same image at the same time, and the feature category correlation matrix represents a matrix recording a similarity between the plurality of feature categories; obtaining a first original image feature according to the second superpixel feature and the original image; obtaining a feature segmentation result corresponding to the original image according to the first original image feature, comprising: processing the first original image feature by using a semantic segmentation model to obtain a segmentation result corresponding to the original image; the semantic segmentation model is provided with a plurality of dilated convolution kernels with different dilated coefficients, and the processing of the first original image feature by using the semantic segmentation model to obtain a segmentation result corresponding to the original image comprises: for each dilated convolution kernel of the plurality of dilated convolution kernels, performing convolution on the original image by using the dilated convolution kernel to obtain a first dilated convolution feature, wherein the first dilated convolution feature has the same size as the first original image feature; adding the first dilated convolution feature and the first original image feature to obtain a second dilated convolution feature; adding the second dilated convolution features corresponding to the plurality of dilated convolution kernels to obtain a third dilated convolution feature; and processing the third dilated convolution feature to obtain a segmentation result corresponding to the original image.

2. The method of claim 1, wherein, the clustering of the pixels in the original image according to the preset feature category to obtain the plurality of superpixel blocks and the mask corresponding to each of the superpixel blocks comprises: processing the original image by using a superpixel-based neural network according to the preset feature category to obtain the plurality of superpixel blocks; for each superpixel block, setting a pixel value at a pixel position corresponding to the superpixel block to the first value and setting a pixel value at other pixel positions except the pixel position corresponding to the superpixel block to the second value to obtain the mask.

3. The method of claim 1, wherein, the obtaining of the second superpixel feature according to the first superpixel feature, the feature category co-occurrence matrix and the feature category correlation matrix comprises: processing the first superpixel feature, the feature category co-occurrence matrix and the feature category correlation matrix by using a superpixel-based graph convolutional neural network to obtain the second superpixel feature.

4. The method of claim 1, wherein, The first original image feature is obtained according to the second superpixel feature and the original image. Features are extracted from the original image to obtain second original image features. The second superpixel feature and the second original image feature are processed by using a cross-attention mechanism to obtain the first original image feature.

5. The method of claim 4, wherein, The first superpixel feature is obtained according to the mask corresponding to each superpixel block and the original image. The mask is multiplied by the second original image feature to obtain third superpixel features. The third superpixel features are globally averaged and pooled to obtain the first superpixel feature.

6. A ground object element segmentation device, comprising: A superpixel block and mask obtaining module is configured to cluster pixels in an original image according to a preset feature category to obtain a plurality of superpixel blocks and a mask corresponding to each superpixel block, wherein the original image includes the ground object element, the size of the mask is the same as the size of the original image, and the pixel value of the mask at the pixel position corresponding to the superpixel block is a first value, and the pixel value at other pixel positions is a second value. A first superpixel feature obtaining module is configured to obtain a first superpixel feature according to the mask corresponding to each superpixel block and the original image. A second superpixel feature obtaining module is configured to obtain a second superpixel feature according to the first superpixel feature, a ground object element category co-occurrence matrix, and a ground object element category correlation matrix, wherein the ground object element category co-occurrence matrix represents a matrix recording the probability of multiple ground object element categories appearing in the same image at the same time, and the ground object element category correlation matrix represents a matrix recording the similarity between multiple ground object element categories. A first original image feature obtaining module is configured to obtain a first original image feature according to the second superpixel feature and the original image. A segmentation result obtaining module is configured to obtain a ground object element segmentation result corresponding to the original image according to the first original image feature. The segmentation result obtaining module includes a segmentation result obtaining submodule. The segmentation result obtaining submodule is configured to process the first original image feature by using a semantic segmentation model to obtain a segmentation result corresponding to the original image. The semantic segmentation model is provided with a plurality of dilated convolution kernels with different dilation coefficients, and the segmentation result obtaining submodule includes a first dilated convolution feature obtaining unit, a second dilated convolution feature obtaining unit, a third dilated convolution feature obtaining unit, and a segmentation result obtaining unit. The first dilated convolution feature obtaining unit is configured to, for each dilated convolution kernel in the plurality of dilated convolution kernels, perform convolution on the dilated convolution kernel and the original image to obtain a first dilated convolution feature, wherein the size of the first dilated convolution feature is the same as the size of the first original image feature. The second dilated convolution feature obtaining unit is configured to add the first dilated convolution feature and the first original image feature to obtain a second dilated convolution feature. The third dilated convolution feature obtaining unit is configured to add the second dilated convolution feature and the first original image feature to obtain a third dilated convolution feature. The segmentation result obtaining unit is configured to obtain a segmentation result corresponding to the original image according to the third dilated convolution feature. The third hollow convolution feature obtaining unit is configured to add the second hollow convolution features corresponding to the plurality of hollow convolution kernels respectively to obtain third hollow convolution features; The segmentation result obtaining unit is configured to process the third hollow convolution features to obtain a segmentation result corresponding to the original image. 7.An electronic device, comprising: one or more processors; a memory storing one or more instructions, wherein the one or more instructions, when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1 to 5. 8.A computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to implement the method of any one of claims 1 to 5.

Images