Remote sensing image segmentation method and device, terminal equipment and storage medium
By acquiring remote sensing images and descriptive text to generate reference images, multi-scale division and feature similarity calculation methods are used to solve the problem of low accuracy of remote sensing images, and high accuracy segmentation of large-scale remote sensing images is achieved.
Patent Information
- Application Number
- CN202411299555.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-14
- Publication Date
- 2025-01-14
AI Technical Summary
When existing machine learning models deal with large-scale remote sensing images, there is a problem of low accuracy in remote sensing image segmentation.
By acquiring the remote sensing image and the description text related to the target object, a reference image is generated, and the characteristics of the target object are segmented from the remote sensing image based on the reference image, multi-scale division and feature similarity calculation methods are used to improve segmentation accuracy.
The accuracy of remote sensing image segmentation is improved, and the problem of low segmentation accuracy in existing models when processing large-scale remote sensing images is solved, so that target objects in remote sensing images can be more accurately identified and segmented.
Smart Images
Figure CN119313683A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of image processing technology, and in particular, relates to a remote sensing image segmentation method, apparatus, terminal equipment and storage medium. Background Art
[0002] Remote sensing images are images obtained using remote sensing technology and are widely used. Specifically, they provide important data support for environmental monitoring, land use, resource management, urban planning, disaster warning, and other aspects by segmenting and extracting the content of remote sensing images.
[0003] Currently, machine learning models can be used to segment and extract the content of remote sensing images. However, since remote sensing images usually have high resolution and large data volumes, existing models are difficult to process large-scale remote sensing images. Even if they can be processed, there will be a problem of low accuracy in remote sensing image segmentation. Summary of the invention
[0004] The embodiments of the present application provide a remote sensing image segmentation method, apparatus, terminal device and storage medium, aiming to solve the problem of low accuracy of remote sensing image segmentation when existing models process large-scale remote sensing images.
[0005] In a first aspect, an embodiment of the present application provides a remote sensing image segmentation method, the method comprising:
[0006] Acquisition of remote sensing images;
[0007] Acquire description text related to the target object, and generate a reference image corresponding to the description text;
[0008] A target segmentation image including the target object is segmented from the remote sensing image according to the reference image.
[0009] In a possible implementation manner of the first aspect, before segmenting a target segmented image including the target object from the remote sensing image according to the reference image, the method further includes:
[0010] Determine first remote sensing images of the remote sensing image at different scales, and divide each of the first remote sensing images into at least one second remote sensing image;
[0011] The step of segmenting a target segmented image including the target object from the remote sensing image according to the reference image comprises:
[0012] A target segmented image including the target object is segmented from the second remote sensing image according to the reference image.
[0013] In a possible implementation manner of the first aspect, segmenting a target segmented image including the target object from the second remote sensing image according to the reference image includes:
[0014] determining objects in the second remote sensing image and corresponding image features thereof;
[0015] Calculating feature similarity between each of the image features and the reference image;
[0016] According to the feature similarity, a target segmented image including the target object is segmented from the second remote sensing image.
[0017] In a possible implementation manner of the first aspect, before calculating the feature similarity between each of the image features and the reference image, the method further includes:
[0018] Determine a target image feature corresponding to the description text from all the image features;
[0019] The calculating the feature similarity between each of the image features and the reference image comprises:
[0020] Calculate the feature similarity between the target image feature and the reference image feature.
[0021] In a possible implementation manner of the first aspect, segmenting a target segmented image including the target object from the second remote sensing image according to the feature similarity includes:
[0022] Determining a target remote sensing image; wherein the target remote sensing image is a second remote sensing image at a target scale;
[0023] Performing feature segmentation on the second remote sensing images according to the feature similarity to obtain a segmentation result corresponding to each of the second remote sensing images; wherein the segmentation result includes a first segmentation result of the target remote sensing image and a second segmentation result of other remote sensing images, and the other remote sensing images are second remote sensing images other than the target remote sensing image;
[0024] A target segmentation image including the target object at the target scale is generated according to the first segmentation result and the second segmentation result.
[0025] In a possible implementation manner of the first aspect, generating a target segmentation image including the target object at the target scale according to the first segmentation result and the second segmentation result includes:
[0026] Determining scale ratio information between the target remote sensing image and the other remote sensing images;
[0027] Determine, according to the scale ratio information and the position information of the second segmentation result, a third segmentation result in the target remote sensing image corresponding to the second segmentation result;
[0028] A target segmentation image including the target object at the target scale is generated according to the first segmentation result and the third segmentation result.
[0029] In a possible implementation manner of the first aspect, dividing each of the first remote sensing images into at least one second remote sensing image includes:
[0030] When the image resolution of the first remote sensing image meets a preset segmentation condition, the first remote sensing image is divided into at least one second remote sensing image according to a preset segmentation rule.
[0031] In a possible implementation manner of the first aspect above, the feature similarity includes one or more of color similarity, shape similarity, and spectral similarity.
[0032] In a second aspect, an embodiment of the present application provides a remote sensing image segmentation device, the device comprising:
[0033] An acquisition module, used for acquiring remote sensing images;
[0034] A generation module, used to obtain description text related to the target object and generate a reference image corresponding to the description text;
[0035] The segmentation module is used to segment a target segmentation image including the target object from the remote sensing image according to the reference image.
[0036] In a third aspect, an embodiment of the present application provides a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the remote sensing image segmentation method provided in the first aspect above is implemented.
[0037] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the remote sensing image segmentation method provided in the first aspect above is implemented.
[0038] It can be understood that the beneficial effects of the second to fourth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here.
[0039] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:
[0040] In an embodiment of the present application, by acquiring a remote sensing image, acquiring a descriptive text related to the target object, and generating a reference image corresponding to the descriptive text, that is, obtaining a reference image related to the target object, it is possible to segment features similar to the reference image from the remote sensing image based on the reference image, that is, it is possible to segment features related to the target object from the remote sensing image, and obtain a target segmented image including the target object, thereby improving the accuracy of segmenting the remote sensing image. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] Figure 1 is a flowchart of a remote sensing image segmentation method provided by an embodiment of the present application;
[0042] Figure 2 is a schematic diagram of dividing a first remote sensing image provided by an embodiment of the present application;
[0043] Figure 3 is a schematic diagram of a multi-scale first remote sensing image provided by an embodiment of the present application;
[0044] Figure 4 is a flowchart of another remote sensing image segmentation method provided by an embodiment of the present application;
[0045] Figure 5 It is a flowchart of a remote sensing image segmentation method provided in one embodiment of the present application;
[0046] Figure 6 It is a structural schematic diagram of a remote sensing image segmentation device provided in one embodiment of the present application;
[0047] Figure 7 It is a structural block diagram of a terminal device provided in one embodiment of the present application. DETAILED DESCRIPTION
[0048] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0049] Remote sensing images are images obtained using remote sensing technology. With the development of remote sensing technology, their applications are becoming more and more extensive. Specifically, they can be segmented and extracted to provide important data support for environmental monitoring, land use, resource management, urban planning, disaster warning and other aspects based on the extracted content.
[0050] At present, machine learning models can be used to segment and extract the content of remote sensing images, such as using the U-net (U-shaped network, U-shaped convolutional neural network) model as the backbone network for remote sensing image content segmentation. However, since remote sensing images usually have high resolution and large data volumes, existing models are difficult to process large-scale remote sensing images. Even if they can be processed, there will be a problem of low accuracy in remote sensing image segmentation.
[0051] Based on this, an embodiment of the present application provides a remote sensing image segmentation method, which obtains a remote sensing image, obtains a descriptive text related to the target object, and generates a reference image corresponding to the descriptive text to obtain a reference image related to the target object. According to the reference image, features related to the target object can be segmented from the remote sensing image to obtain a target segmented image including the target object, thereby improving the accuracy of remote sensing image segmentation, thereby solving the current problem of low accuracy in remote sensing image segmentation.
[0052] The following is a detailed description of the remote sensing image segmentation method provided in the embodiment of the present application in conjunction with the accompanying drawings:
[0053] See also Figure 1 , Figure 1 A flowchart of a remote sensing image segmentation method provided by an embodiment of the present application is shown. Figure 1 As shown, the above remote sensing image segmentation method may specifically include the following steps:
[0054] Step 101, acquiring a remote sensing image.
[0055] The remote sensing image may be an image obtained by using remote sensing technology, such as an image of surface information obtained by remote sensing platforms such as airplanes and satellites. The remote sensing image may be any one of aerial photographs and satellite photographs.
[0056] In practical applications, remote sensing technology can be used to obtain remote sensing images in real time through a remote sensing platform, or remote sensing technology can be used in advance to obtain remote sensing images of different regions and store them on a cloud platform or in a local database. Then, when it is necessary to segment the content of remote sensing images of a certain area, the remote sensing images stored in the cloud platform or database can be directly obtained.
[0057] Step 102: Acquire description text related to the target object and generate a reference image corresponding to the description text.
[0058] The target object refers to the object that needs to be segmented from the remote sensing image. The target object can be determined by the user based on the needs or based on the business needs. For example, it can be a road object determined based on the user needs or a river object determined based on the business of environmental monitoring. The description text can be a text used to describe the target object, specifically a text describing the shape, color, size, material and other features of the target object. For example, for a road object, the description text can be: a two-way four-lane asphalt road. For a building object, the description text can be: a red circular building. The reference image can be an image generated based on the description text to represent the target object.
[0059] When the remote sensing image needs to be segmented based on user needs or business needs, the object that needs to be segmented from the remote sensing image, that is, the target object, can be determined based on the user needs or the business needs.
[0060] For example, a user may propose a demand to segment an object from a remote sensing image, and the system can respond to the user's demand and determine the target object based on the user's demand; since remote sensing images can provide important data support for environmental monitoring, land use, resource management, urban planning, disaster warning and other aspects, the system can provide services in environmental monitoring, land use, resource management, urban planning, disaster warning and other aspects, and different businesses will propose different business requirements, that is, different businesses will require different content to be segmented from remote sensing images, and the target object of each business can be determined based on the needs of different businesses.
[0061] For example, based on the needs of environmental monitoring business, the target objects of environmental monitoring business can be determined as various water objects such as rivers and lakes and various vegetation objects such as forests and crops. Based on the needs of urban planning business, the target objects of urban planning business can be determined as various artificial construction objects such as buildings and roads.
[0062] After determining the target object, the description text entered by the user can be obtained, and the description text related to the target object can be determined from the description text entered by the user. The description text fed back by each business can also be obtained, and the description text related to the target object can be determined from the description text fed back by each business.
[0063] In one embodiment of the present application, a remote sensing image segmentation method provided in this embodiment can be applied to a terminal device including a memory, a processor, and a computer program stored in the memory and executable on the processor. The terminal device can provide an interface for interacting with a user, and when the user needs to segment an object from a remote sensing image, the user can enter a description text through the interface provided by the terminal device to describe the need to segment an object from the remote sensing image; the terminal device can also be connected to each service communication, and can obtain the description text fed back by each service.
[0064] In a specific implementation, the terminal device can be connected to the cloud platform for communication, and one or more services can be configured in the cloud platform, and then the terminal device can obtain the description text fed back by each service in the cloud platform.
[0065] After obtaining the description text related to the target object, a reference image may be generated according to the description text related to the target object.
[0066] In practical applications, various objects that will exist in remote sensing images can be determined in advance, and the shape of each object can be defined. An image of each object can be generated based on the shape of each object, and the generated image can be stored in a cloud platform or a local database. After obtaining the descriptive text related to the target object, the image corresponding to the target object in the descriptive text can be retrieved, and the color, size and other features of the image can be adjusted according to the descriptive text to obtain a reference image corresponding to the target object.
[0067] In a specific implementation, a pre-trained machine learning model may be used to generate a reference image corresponding to the description text, such as a distributed autoencoder for language and image (DALL-E), a stable diffusion model, or other machine learning models.
[0068] Step 103: segmenting a target segmentation image including the target object from the remote sensing image according to the reference image.
[0069] The target segmented image may be an image segmented from a remote sensing image and including the target object.
[0070] After obtaining the reference image, an image matching the shape of the reference image can be segmented from the remote sensing image, that is, a target segmented image including the target object.
[0071] In practical applications, the reference image and the remote sensing image can be input into a pre-trained neural network model based on the self-attention mechanism, such as a Transformer model, and then the segmentation result output by the neural network model can be obtained, which is the target segmentation image.
[0072] In an embodiment of the present application, before step 103, the following steps may also be included:
[0073] First remote sensing images of remote sensing images at different scales are determined, and each first remote sensing image is divided into at least one second remote sensing image.
[0074] The scale may be the remote sensing accuracy of the remote sensing image, and the scale of the remote sensing image may include meter-level accuracy, decimeter-level accuracy, centimeter-level accuracy, etc. The first remote sensing image may be an image of the remote sensing image at each scale, and the second remote sensing image may be a block image obtained by dividing the first remote sensing image into blocks.
[0075] After acquiring the remote sensing image, the remote sensing image can be scaled to obtain remote sensing images at multiple scales, that is, multiple first remote sensing images are obtained, and each first remote sensing image is divided into at least one second remote sensing image to obtain multiple second remote sensing images at different scales.
[0076] In an embodiment of the present application, each first remote sensing image may be divided into at least one second remote sensing image in the following manner:
[0077] When the image resolution of the first remote sensing image meets a preset segmentation condition, the first remote sensing image is divided into at least one second remote sensing image according to a preset segmentation rule.
[0078] Among them, the blocking condition can be used to determine whether the first remote sensing image needs to be divided, the image resolution can be the resolution of the first remote sensing image, the division rule can be used to divide the first remote sensing image, the division rule can be a rule for dividing the first remote sensing image according to a preset order and a preset size, the fixed size can be the size of the second remote sensing image, and the division rule can be user-defined or generated based on the image resolution.
[0079] After the first remote sensing image is obtained, the image resolution of the first remote sensing image may be determined, and it may be judged whether the image resolution of the first remote sensing image satisfies a blocking condition.
[0080] In practical applications, it is possible to detect whether the image resolution of the first remote sensing image is less than an image resolution threshold value corresponding to the segmentation condition. When it is detected that the image resolution of the first remote sensing image is less than or equal to the image resolution threshold value, it can be determined that the resolution of the first remote sensing image is relatively small, and there is no need to divide the first remote sensing image, that is, it is determined that the image resolution of the first remote sensing image does not meet the segmentation condition. When it is detected that the image resolution of the first remote sensing image is greater than the image resolution threshold value, it can be determined that the resolution of the first remote sensing image is relatively large, and the first remote sensing image needs to be divided, that is, it is determined that the image resolution of the first remote sensing image meets the segmentation condition.
[0081] Specifically, the image resolution threshold may be the maximum resolution defined for content segmentation of the remote sensing image. The image resolution threshold may be used to determine whether the resolution of the remote sensing image is too large to result in the inability to perform content segmentation on the remote sensing image or to result in reduced efficiency in performing content segmentation on the remote sensing image. The image resolution threshold may be determined by the user based on experience, or may be the maximum resolution allowed for content segmentation of the remote sensing image.
[0082] In a specific implementation, by dividing the first remote sensing image when the image resolution of the first remote sensing image is relatively large, a second remote sensing image with a smaller image resolution is obtained, that is, an image with an image resolution less than or equal to an image resolution threshold is obtained, thereby reducing the situation where the remote sensing image cannot be segmented for content or the efficiency of content segmentation of the remote sensing image is reduced due to the image resolution of the remote sensing image being too large, so that content segmentation can be performed on remote sensing images with too large an image resolution.
[0083] When the image resolution of the first remote sensing image meets the preset segmentation condition, the first remote sensing image can be divided into at least one image of a preset size according to the order in the preset segmentation rule, that is, at least one second remote sensing image is obtained.
[0084] Exemplarily, the division rule may be a rule of division from left to right and according to a 3*3px window. Specifically, a 3*3 window means that the width and height of the window are both 3 pixels. Based on the division rule, the first remote sensing image can be divided into multiple images of size 3*3px in order from left to right, thus obtaining a second remote sensing image with a width and height of 3 pixels.
[0085] When the image resolution of the first remote sensing image does not meet the preset blocking conditions, there is no need to divide the first remote sensing image, and the first remote sensing image is used to perform the above-mentioned step of segmenting a target segmented image including the target object from the remote sensing image according to the reference image, that is, segmenting a target segmented image including the target object from the first remote sensing image according to the reference image.
[0086] In an embodiment of the present application, step 103 may include the following steps:
[0087] According to the reference image, a target segmented image including the target object is segmented from the second remote sensing image.
[0088] After the second remote sensing image is obtained, a target segmented image matching the shape of the reference image may be segmented from the second remote sensing image.
[0089] In practical applications, by dividing the first remote sensing image to obtain at least one second remote sensing image, and segmenting a target segmented image from each second remote sensing image, it is possible to process large-scale remote sensing images, that is, when processing remote sensing images with excessively large image resolution, by dividing the remote sensing image and segmenting the target segmented image from the divided remote sensing image, thereby solving the problem that the existing model cannot process large-scale remote sensing images.
[0090] In addition, in the process of dividing the first remote sensing image, an object in the first remote sensing image may be divided into multiple second remote sensing images, that is, an object is distributed in one or more second remote sensing images, and then in the process of segmenting the second remote sensing image, the object cannot be accurately identified or segmented, which reduces the accuracy of remote sensing image segmentation.
[0091] See also Figure 2 , Figure 2 FIG. 4 shows a schematic diagram of dividing a first remote sensing image provided by an embodiment of the present application, such as Figure 2 As shown, the first remote sensing image A may be composed of 9 rows and 9 columns of pixels, and the image resolution of the first remote sensing image A may be 9*9px, and then the first remote sensing image A may be divided in order from left to right and according to a 3*3px window to obtain 9 second remote sensing images, namely, second remote sensing images a1 to a9. The first remote sensing image A may include object B, and the shape of object B may be a rectangle with a center coordinate of (3, 1), a length of 2, and a width of 1, and the center coordinate (3, 1) may indicate that the center of object B is located at the position of the pixel of the 3rd row and the 1st column in the first remote sensing image A, and then in the process of dividing the first remote sensing image A, object B will be divided into second remote sensing image a7 and second remote sensing image a8, and second remote sensing image a7 and second remote sensing image a8 only include partial features of object B, so object B cannot be accurately identified from second remote sensing image a7 and second remote sensing image a8, and object B cannot be segmented.
[0092] In one embodiment of the present application, since the first remote sensing image can be a remote sensing image at different scales, and the second remote sensing image can be a remote sensing image divided from the first remote sensing image, the second remote sensing image can be represented as a remote sensing image at different scales. It should be understood that a second remote sensing image at a certain scale may not be able to accurately identify an object, and for an object that cannot be accurately identified, the remote sensing image can be scaled so that the object that cannot be accurately identified is also scaled, so that the object can be accurately identified in the scaled remote sensing image, that is, an object that cannot be identified by a second remote sensing image of another scale can be identified in a second remote sensing image of another scale.
[0093] See also Figure 3 , Figure 3 A schematic diagram of a multi-scale first remote sensing image provided by an embodiment of the present application is shown. Figure 3 As shown, the first remote sensing image B may be an image of the remote sensing image b at the first scale, and the first remote sensing image B may be composed of 12 rows and 12 columns of pixels at the first scale, that is, the image resolution of the first remote sensing image B may be 12*12px, and then the coordinate system corresponding to the first remote sensing image B may be established based on the 12 rows and 12 columns of pixels, and the second remote sensing image C may be an image of the remote sensing image b at the second scale, and the first remote sensing image C may be composed of 4 rows and 4 columns of pixels at the second scale, that is, the image resolution of the first remote sensing image C may be 4*4px, and then the coordinate system corresponding to the first remote sensing image C may be established based on the 4 rows and 4 columns of pixels. Among them, the first remote sensing image B may be divided in order from left to right and in a 3*3px window to obtain second remote sensing images b1 to second remote sensing images b16, and the first remote sensing image C may be divided in order from left to right and in a 2*2px window to obtain second remote sensing images c1 to second remote sensing images c4. The remote sensing image b may include object D, and the position of object D in the first remote sensing image B may be (3, 2), that is, it is at the position of the pixel in the 3rd row and the 2nd column, and the shape of object D may be a circle with the center coordinates (3, 2) and the diameter 1, that is, object D is divided into the second remote sensing image b13 and the second remote sensing image b14, and the position of object D in the first remote sensing image C may be (1, 2 / 3), and the shape of object D may be a circle with the center coordinates (1, 2 / 3) and the diameter 1 / 3, that is, object D is in the second remote sensing image c3.
[0094] In the first remote sensing image B, object D can be divided into the second remote sensing image b9 and the second remote sensing image b10, so object D cannot be accurately identified at the first scale. In the first remote sensing image B, object D is only divided into the second remote sensing image c3, and is not divided into multiple second remote sensing images, so object D can be accurately identified at the second scale.
[0095] In a specific implementation, by segmenting a target segmented image from remote sensing images at different scales, objects in the remote sensing image can be identified, and the accuracy of identifying and segmenting objects in the remote sensing image is improved, thereby solving the problem of low accuracy of remote sensing image segmentation in existing models when processing large-scale remote sensing images.
[0096] In an embodiment of the present application, by acquiring a remote sensing image, acquiring a descriptive text related to the target object, and generating a reference image corresponding to the descriptive text, a reference image related to the target object is obtained, and based on the reference image, features related to the target object can be segmented from the remote sensing image to obtain a target segmented image including the target object, thereby improving the accuracy of segmentation of the remote sensing image, thereby solving the current problem of low accuracy in remote sensing image segmentation.
[0097] See also Figure 4 , Figure 4 A flowchart of another remote sensing image segmentation method provided in an embodiment of the present application is shown, which may specifically include the following steps:
[0098] Step 401, acquiring a remote sensing image.
[0099] Step 402: Acquire description text related to the target object and generate a reference image corresponding to the description text.
[0100] Step 403: determine first remote sensing images of remote sensing images at different scales, and divide each first remote sensing image into at least one second remote sensing image.
[0101] For the related descriptions of step 401 to step 403, please refer to step 101 to step 103, which will not be repeated here.
[0102] Step 404: Determine objects in the second remote sensing image and their corresponding image features.
[0103] Among them, image features may include color features, texture features, shape features, spectral features and other features.
[0104] After the second remote sensing image is obtained, feature recognition may be performed on the second remote sensing image, thereby being able to identify objects included in the second remote sensing image and determine image features of each object.
[0105] In practical applications, since the remote sensing image can be an image of the ground surface obtained by remote sensing technology, the remote sensing image can include images of objects existing on the ground surface, that is, the remote sensing image can include objects existing on the ground surface, such as various rivers, lakes and other water objects, various forests, crops and other vegetation objects, buildings, roads and other artificial construction objects, etc., and then feature recognition can be used to determine the objects in the second remote sensing image and determine the image features of each object.
[0106] Step 405, calculating the feature similarity between each image feature and the reference image.
[0107] The feature similarity may include one or more of color similarity, shape similarity, and spectral similarity. The color similarity may be the similarity between color features, the shape similarity may be the similarity between shape features, and the spectral similarity may be the similarity between spectral features.
[0108] After determining the objects and image features of the second remote sensing image, the image features of the reference image can be determined, and then the image features of the reference image can be compared with the image features of each object in the second remote sensing image to obtain the feature similarity between the image features of each object and the image features of the reference image.
[0109] In practical applications, the color features of the reference image can be compared with the color features of each object in the second remote sensing image to obtain the color similarity between the color features of the object and the reference image. Similarly, the shape similarity between the shape features of the object and the reference image can be obtained, as well as the spectral similarity between the spectral features of the object and the reference image. This can then lead to the feature similarity between the image features of each object in the second remote sensing image and the reference image.
[0110] It should be understood that the reference image can be an image related to the target object. For example, the target object can be a red playground. The user can enter a description text of the red playground and generate a reference image representing the red playground through the Stable Diffusion model. It can be said that the object described by the reference image is a playground, and the image features included in the reference image include color features and shape features, wherein the color feature is red, and the shape feature is the shape of the playground. In the process of calculating feature similarity, the image features in the reference image can be directly obtained, and the feature similarity can be calculated based on the image features of the reference image.
[0111] In an embodiment of the present application, before step 405, the following steps may also be included:
[0112] Determine the target image feature corresponding to the description text from all image features.
[0113] The target image feature may be an image feature of an object described in the description text.
[0114] After determining the object of the second remote sensing image and its image features, the object described by the descriptive text can also be determined, and then based on the object described by the descriptive text, the object in the second remote sensing image that matches the descriptive text can be determined, and the image features of the matching object in the second remote sensing image can be determined as the target image features corresponding to the descriptive text.
[0115] Exemplarily, the description text input by the user may be a red playground, and the object described by the description text can be determined as the playground. By performing feature recognition on the second remote sensing image, the objects in the second remote sensing image can be determined, and the objects represented as the playground can be determined from all the objects in the second remote sensing image, that is, the objects that match the description text, and then the image features of all objects represented as the playground in the second remote sensing image can be determined as the target image features.
[0116] In an embodiment of the present application, step 405 may include the following steps:
[0117] Calculate the feature similarity between the target image features and the reference image.
[0118] After the target image features are determined, the image features of the reference image may be compared with the target image features to obtain feature similarity between the target image features and the image features of the reference image.
[0119] In a specific implementation, by determining the target image features corresponding to the description text from the second remote sensing image, it is only necessary to compare the target image features with the image features of the reference image, without comparing the image features of all objects in the second remote sensing image with the image features of the reference image. This can reduce the steps of comparing image features, thereby improving the efficiency of segmenting the remote sensing image. In addition, by determining the target image features corresponding to the description text from the second remote sensing image, it can be ensured that the image features related to the target object in the remote sensing image are compared with the reference image, thereby reducing the situation where an object in the remote sensing image that is similar in shape to the reference image is segmented but the object is not the target object. That is, remote sensing image segmentation can also be performed based on the description text, thereby improving the accuracy of remote sensing image segmentation.
[0120] Step 406 : Segment a target segmentation image including the target object from the second remote sensing image according to the feature similarity.
[0121] After calculating the feature similarity, the feature similarity of each image feature in the second remote sensing image can be compared with a preset similarity threshold, and then the image features in the second remote sensing image whose feature similarity is greater than or equal to the similarity threshold can be determined, and the image features whose feature similarity is greater than or equal to the similarity threshold can be segmented out, that is, the target segmented image including the target object.
[0122] Specifically, the similarity threshold may be used to determine whether the image features of the remote sensing image are similar to the image features of the reference image. The similarity threshold may be user-defined or may be obtained based on experience.
[0123] In practical applications, when the feature similarity between a certain image feature in a remote sensing image and a reference image is greater than a similarity threshold, the image feature can be considered similar to the reference image, that is, the image feature can be considered as the image feature of the target object, and then the image feature can be segmented to obtain a target segmented image including the target object.
[0124] In an embodiment of the present application, step 406 may include steps 4061 to 4063:
[0125] Step 4061, determine the target remote sensing image.
[0126] The target remote sensing image may be a second remote sensing image at a target scale, and the target scale may be a scale set by a user, or may be a scale with the lowest accuracy or a scale with the largest size and the most objects among all scales.
[0127] After the feature similarity is calculated, the target scale can be determined, and the second remote sensing image at the target scale can be determined as the target remote sensing image.
[0128] Exemplarily, when a remote sensing image is scaled, first remote sensing images at three scales can be obtained, including a first remote sensing image a at a first scale, a first remote sensing image b at a second scale, and a first remote sensing image c at a third scale. Each first remote sensing image is divided to obtain second remote sensing images at the first scale to the third scale. The second scale can be defined as the target scale, and the second remote sensing image at the second scale can be the target remote sensing image.
[0129] Step 4062: perform feature segmentation on the second remote sensing images according to feature similarity to obtain a segmentation result corresponding to each second remote sensing image.
[0130] Among them, the segmentation result can be an image obtained by segmenting the image features of the remote sensing image, and the segmentation result can include a first segmentation result of the target remote sensing image and a second segmentation result of other remote sensing images. The other remote sensing image can be a second remote sensing image other than the target remote sensing image.
[0131] After determining the target remote sensing image, remote sensing images other than the target remote sensing image can be determined as other remote sensing images from all second remote sensing images, and feature segmentation can be performed on all second remote sensing images according to feature similarity to obtain a segmentation result corresponding to each second remote sensing image, that is, feature segmentation can be performed on the target remote sensing image and other remote sensing images respectively according to feature similarity to obtain a first segmentation result of the target remote sensing image, and a second segmentation result of the other remote sensing images can be obtained.
[0132] Step 4063: Generate a target segmentation image including the target object at the target scale according to the first segmentation result and the second segmentation result.
[0133] After obtaining the first segmentation result and the second segmentation result, the second segmentation result may be merged with the first segmentation result, that is, the second segmentation result may be correspondingly merged into the first segmentation result to obtain a target segmented image including the target object at the target scale.
[0134] In practical applications, all first segmentation results can be merged in advance in the same coordinate system. For example, the coordinate system of the first remote sensing image at the target scale can be used as the coordinate system for merging all the first segmentation results to obtain a segmented image including all the first segmentation results at the target scale, and all the second segmentation results can be correspondingly merged in the segmented image including all the first segmentation results to obtain a target segmented image including the target object at the target scale.
[0135] In an embodiment of the present application, step 4063 may be performed in the following manner:
[0136] Determine the scale ratio information between the target remote sensing image and other remote sensing images, determine the third segmentation result corresponding to the second segmentation result in the target remote sensing image according to the scale ratio information and the position information of the second segmentation result, and generate a target segmentation image including the target object at the target scale according to the first segmentation result and the third segmentation result.
[0137] Among them, the scale ratio information can be the ratio between the scale of other remote sensing images and the target scale, the position information can be the position of the second segmentation result in the corresponding first remote sensing image, that is, the position in the first remote sensing image with the same scale as the second segmentation result, and the third segmentation result can be the image feature corresponding to the position information in the target remote sensing image.
[0138] After obtaining the first segmentation result and the second segmentation result, for each second segmentation result, a first remote sensing image with the same scale as the second segmentation result can be determined, and the position of the second segmentation result in the first remote sensing image with the same scale can be determined, which is the position information corresponding to the second segmentation result, and then the position information corresponding to each second segmentation result can be obtained.
[0139] After obtaining the first segmentation result and the second segmentation result, the scale ratio information between the scales of other remote sensing images and the target scale can also be determined, and then the scale ratio information between each second segmentation result and the target scale can be obtained.
[0140] Exemplarily, when the scales of the remote sensing image include a first scale, a second scale, and a third scale, and the target scale is the second scale, the scale ratio information between the other remote sensing images and the target scale can be expressed as the scale ratio information between the second scale and the first scale, and the scale ratio information between the second scale and the third scale. For the first scale, the position of each second segmentation result at the scale in the first remote sensing image at the scale can be determined, that is, the position information corresponding to the second segmentation result at the first scale, and similarly, the position information corresponding to the second segmentation result at each scale can be obtained.
[0141] After obtaining the position information corresponding to each second segmentation result and the scale ratio information between each second segmentation result and the target scale, the position of each second segmentation result in the first remote sensing image at the target scale can be determined according to the position information and the scale ratio information, and the image features of the first remote sensing image at the target scale at the position can be segmented, which is the third segmentation result. Furthermore, a target segmented image including the target object at the target scale can be generated according to the first segmentation result and the third segmentation result.
[0142] Exemplarily, when the scale of the remote sensing image includes a first scale and a second scale, and the first scale is the target scale, the first remote sensing image at each scale can be divided to obtain multiple second remote sensing images. Based on the position information corresponding to each second segmentation result at the second scale and the scale ratio information between the second scale and the target scale, the position of each second segmentation result in the first remote sensing image at the target scale can be determined, and the image features of the first remote sensing image at the target scale at the position can be segmented out, which is the third segmentation result.
[0143] In practical applications, by performing feature segmentation on remote sensing images at each scale, the segmentation results at each scale can be obtained, and then the segmentation results at each scale can be merged to obtain the segmentation result at the target scale. The segmentation result at the target scale is used as the output for segmenting the remote sensing image, which is the target segmented image.
[0144] In an embodiment of the present application, by acquiring a remote sensing image and determining a first remote sensing image of the remote sensing image at different scales, each first remote sensing image is divided into at least one second remote sensing image, and then the object in the second remote sensing image and its corresponding image features are determined, so that the object in the second remote sensing image and its image features can be identified and segmented, which can solve the problem that the existing model cannot process remote sensing images with too large image resolution, and by acquiring a descriptive text related to the target object, a reference image corresponding to the descriptive text is generated, and the feature similarity between each image feature and the reference image is calculated. According to the feature similarity, the features related to the target object are segmented from the second remote sensing image to obtain a target segmented image including the target object, thereby improving the accuracy of segmenting the remote sensing image.
[0145] See also Figure 5 , Figure 5 A schematic flow chart of a remote sensing image segmentation method provided in an embodiment of the present application is shown, which may specifically include the following steps:
[0146] Step 501, acquiring a remote sensing image.
[0147] Step 502: Obtain description text related to the target object.
[0148] Step 503, performing scale scaling and image division operations on the remote sensing image in sequence to obtain a second remote sensing image at multiple scales.
[0149] The scaling operation may be an operation of determining a first remote sensing image of a remote sensing image at multiple scales, and the image division operation may be an operation of dividing each first remote sensing image into at least one second remote sensing image;
[0150] Step 504: input the description text into the Stable Diffusion model to generate a reference image related to the target object.
[0151] Step 505: input the second remote sensing image, description text and reference image at each scale into the Transformer model.
[0152] In step 506, the Transformer model determines the target image features corresponding to the description text from the second remote sensing image, and calculates the feature similarity between the target image features and the reference image, so as to perform feature segmentation on the second remote sensing image based on the feature similarity to obtain a target segmented image.
[0153] In an embodiment of the present application, a remote sensing image is acquired and scaled and divided in sequence to obtain a second remote sensing image at multiple scales, thereby reducing the image resolution of the second remote sensing image, so that the second remote sensing image after scaled and divided can be processed, thereby reducing the situation where remote sensing images with too large an image resolution cannot be processed. By acquiring a descriptive text related to the target object and inputting the descriptive text into a StableDiffusion model, a reference image related to the target object can be generated. The second remote sensing image, the descriptive text and the reference image at each scale are input into a Transformer model, and the Transformer model can determine the target image features corresponding to the descriptive text from the second remote sensing image, and calculate the feature similarity between the target image features and the reference image, so as to perform feature segmentation on the second remote sensing image based on the feature similarity, thereby segmenting the features related to the target object from the second remote sensing image to obtain a target segmented image, thereby improving the accuracy of segmenting the remote sensing image.
[0154] See also Figure 6 , Figure 6 The following is a schematic diagram showing the structure of a remote sensing image segmentation device provided in an embodiment of the present application. The device may specifically include the following modules:
[0155] An acquisition module 601 is used to acquire remote sensing images;
[0156] A generating module 602 is used to obtain description text related to the target object and generate a reference image corresponding to the description text;
[0157] The segmentation module 603 is used to segment a target segmentation image including a target object from the remote sensing image according to the reference image.
[0158] In one implementation, the device may further include the following modules:
[0159] A segmentation module, used for determining first remote sensing images of the remote sensing image at different scales and dividing each first remote sensing image into at least one second remote sensing image before segmenting a target segmented image including a target object from the remote sensing image according to a reference image;
[0160] In one implementation, the segmentation module 603 may also be used to:
[0161] According to the reference image, a target segmented image including the target object is segmented from the second remote sensing image.
[0162] In one implementation, the segmentation module 603 may also be used to:
[0163] determining objects in the second remote sensing image and their corresponding image features;
[0164] Calculate the feature similarity between each image feature and the reference image;
[0165] According to the feature similarity, a target segmentation image including the target object is segmented from the second remote sensing image.
[0166] In one implementation, the device may further include the following modules:
[0167] A determination module, used for determining a target image feature corresponding to the description text from all image features before calculating the feature similarity between each image feature and the reference image;
[0168] In one implementation, the segmentation module 603 may also be used to:
[0169] Calculate the feature similarity between the target image features and the reference image.
[0170] In one implementation, the segmentation module 603 may also be used to:
[0171] Determine a target remote sensing image; wherein the target remote sensing image is a second remote sensing image at a target scale;
[0172] Performing feature segmentation on the second remote sensing images according to feature similarity to obtain a segmentation result corresponding to each second remote sensing image; wherein the segmentation result includes a first segmentation result of the target remote sensing image and a second segmentation result of other remote sensing images, and the other remote sensing images are second remote sensing images other than the target remote sensing image;
[0173] A target segmentation image including the target object at the target scale is generated according to the first segmentation result and the second segmentation result.
[0174] In one implementation, the segmentation module 603 may also be used to:
[0175] Determine the scale ratio information between the target remote sensing image and other remote sensing images;
[0176] Determine a third segmentation result corresponding to the second segmentation result in the target remote sensing image according to the scale ratio information and the position information of the second segmentation result;
[0177] A target segmentation image including the target object at the target scale is generated according to the first segmentation result and the third segmentation result.
[0178] In one implementation, the above-mentioned division module may also be used for:
[0179] When the image resolution of the first remote sensing image meets a preset segmentation condition, the first remote sensing image is divided into at least one second remote sensing image according to a preset segmentation rule.
[0180] In one implementation, the feature similarity includes one or more of color similarity, shape similarity, and spectral similarity.
[0181] In an embodiment of the present application, by acquiring a remote sensing image, acquiring descriptive text related to the target object, and generating a reference image corresponding to the descriptive text, a reference image related to the target object is obtained. Based on the reference image, features related to the target object can be segmented from the remote sensing image to obtain a target segmented image including the target object, thereby improving the accuracy of segmenting the remote sensing image.
[0182] It should be noted that the information interaction, execution process and other contents between the above-mentioned devices are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.
[0183] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.
[0184] See also Figure 7 , Figure 5 A structural block diagram of a terminal device provided by an embodiment of the present application is shown as follows: Figure 7 As shown, this embodiment provides a terminal device 71, which includes: at least one processor 711, a memory 712, and a computer program 7121 stored in the memory 712 and executable on at least one processor 711, and when the processor 711 executes the computer program 7121, the steps in any of the above-mentioned method embodiments are implemented.
[0185] The embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in any one of the above method embodiments can be implemented.
[0186] An embodiment of the present application provides a computer program product. When the computer program product runs on a mobile terminal, the mobile terminal can implement the steps in the above-mentioned method embodiments when executing the computer program product.
[0187] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device capable of carrying the computer program code to the camera device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium.
[0188] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.
Claims
1. A remote sensing image segmentation method, characterized in that: The method comprises: Acquisition of remote sensing images; Acquire description text related to the target object, and generate a reference image corresponding to the description text; A target segmentation image including the target object is segmented from the remote sensing image according to the reference image.
2. The remote sensing image segmentation method according to claim 1, characterized in that: Before segmenting a target segmented image including the target object from the remote sensing image according to the reference image, the method further includes: Determine first remote sensing images of the remote sensing image at different scales, and divide each of the first remote sensing images into at least one second remote sensing image; The step of segmenting a target segmented image including the target object from the remote sensing image according to the reference image comprises: A target segmented image including the target object is segmented from the second remote sensing image according to the reference image.
3. The remote sensing image segmentation method according to claim 2, characterized in that: The step of segmenting a target segmented image including the target object from the second remote sensing image according to the reference image comprises: determining objects in the second remote sensing image and corresponding image features thereof; Calculating feature similarity between each of the image features and the reference image; According to the feature similarity, a target segmented image including the target object is segmented from the second remote sensing image.
4. The remote sensing image segmentation method according to claim 3, characterized in that: Before calculating the feature similarity between each of the image features and the reference image, the method further includes: Determine a target image feature corresponding to the description text from all the image features; The calculating the feature similarity between each of the image features and the reference image comprises: Calculate the feature similarity between the target image feature and the reference image feature.
5. The remote sensing image segmentation method according to claim 3, characterized in that: The step of segmenting a target segmented image including the target object from the second remote sensing image according to the feature similarity comprises: Determining a target remote sensing image; wherein the target remote sensing image is a second remote sensing image at a target scale; Performing feature segmentation on the second remote sensing images according to the feature similarity to obtain a segmentation result corresponding to each of the second remote sensing images; wherein the segmentation result includes a first segmentation result of the target remote sensing image and a second segmentation result of other remote sensing images, and the other remote sensing images are second remote sensing images other than the target remote sensing image; A target segmentation image including the target object at the target scale is generated according to the first segmentation result and the second segmentation result.
6. The remote sensing image segmentation method according to claim 5, characterized in that: The step of generating a target segmentation image including the target object at the target scale according to the first segmentation result and the second segmentation result includes: Determining scale ratio information between the target remote sensing image and the other remote sensing images; Determine, according to the scale ratio information and the position information of the second segmentation result, a third segmentation result in the target remote sensing image corresponding to the second segmentation result; A target segmentation image including the target object at the target scale is generated according to the first segmentation result and the third segmentation result.
7. The remote sensing image segmentation method according to claim 2, characterized in that: The step of dividing each of the first remote sensing images into at least one second remote sensing image comprises: When the image resolution of the first remote sensing image meets a preset segmentation condition, the first remote sensing image is divided into at least one second remote sensing image according to a preset segmentation rule.
8. The remote sensing image segmentation method according to claim 3, characterized in that: The feature similarity includes one or more of color similarity, shape similarity, and spectrum similarity.
9. A remote sensing image segmentation device, characterized in that: The device comprises: An acquisition module, used for acquiring remote sensing images; A generation module, used to obtain description text related to the target object and generate a reference image corresponding to the description text; The segmentation module is used to segment a target segmentation image including the target object from the remote sensing image according to the reference image.
10. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 8 is implemented.