A remote sensing image interpretation method and device based on image segmentation model

Through the method based on the image segmentation model, the image segmentation model is trained using the conversion and recognition technology of historical remote sensing images, and the problems of inaccurate interpretation results and low efficiency in the prior art are solved, and efficient remote sensing image interpretation is achieved.

CN118736328BActive Publication Date: 2024-11-22ZHEJIANG GUOYAO GEOGRAPHIC INFORMATION TECH CO LTD +2
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411187379.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-28
Publication Date
2024-11-22
Estimated Expiration
2044-08-28

AI Technical Summary

Technical Problem

The existing remote sensing image interpretation technology relies on grayscale values, colors and texture features. Combined with traditional machine learning models, it requires manual intervention to draw the contours of the land, and the interpretation results are low, which affects the interpretation efficiency.

Method used

Using the method based on the image segmentation model, by acquiring multiple historical remote sensing images for conversion processing, dividing the pattern outline and identifying mask information, forming a training and verification image set, training the image segmentation model, reducing manual intervention and improving interpretation accuracy .

Benefits of technology

It improves the accuracy and efficiency of land object interpretation, reduces manual operations, and improves the automation level of remote sensing image interpretation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118736328B_ABST
    Figure CN118736328B_ABST
Patent Text Reader

Abstract

The present application belongs to the technical field of remote sensing image processing, and discloses a remote sensing image interpretation method and device based on an image segmentation model. By converting and processing a plurality of historical remote sensing images, corresponding mask images are obtained, and the contour of each patch divided by each mask image is identified and processed to obtain corresponding mask information; then, a training image set and a verification image set can be divided according to all historical remote sensing images and all corresponding mask information to train a first image segmentation model, and an image interpretation result can be obtained according to a target remote sensing image, a first click operation input by a user, and the trained first image segmentation model. This not only avoids excessive manual intervention operations, but also utilizes the first image segmentation model trained by historical remote sensing images and corresponding mask information to improve the accuracy of ground object interpretation results, thereby effectively ensuring the interpretation efficiency of remote sensing images.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application belongs to the technical field of remote sensing image processing, and in particular, relates to a remote sensing image interpretation method and device based on an image segmentation model. Background Art

[0002] Land object interpretation refers to the process of identifying and analyzing surface objects from acquired images (or images). The goal is to extract information about surface cover type, land use status, vegetation distribution, or water body location from images. It is widely used in natural resource management, environmental monitoring, urban planning, agricultural applications, and disaster assessment.

[0003] With the rapid development of remote sensing technology, researchers are paying more and more attention to the use of high-resolution images. It has become a common method to use high-resolution remote sensing images combined with remote sensing extraction technology to identify information about surface objects. Existing extraction technologies mainly use features such as grayscale value, color, and texture of remote sensing images, combined with traditional machine learning models for image interpretation. Not only does it require manual intervention to draw the contours of objects, but the accuracy of the object interpretation results is also low, which in turn affects the efficiency of remote sensing image interpretation. Summary of the invention

[0004] In order to solve the above-mentioned technical problems that the existing extraction technology mainly uses the gray value, color and texture features of remote sensing images, combined with traditional machine learning models to interpret images, not only requires manual intervention to draw the contours of objects, but also the accuracy of the object interpretation results is low, which further affects the interpretation efficiency of remote sensing images, this application proposes a remote sensing image interpretation method and device based on an image segmentation model, and its technical solution is as follows:

[0005] In a first aspect, an embodiment of the present application provides a remote sensing image interpretation method based on an image segmentation model, comprising:

[0006] According to the model task requirements, at least two historical remote sensing images collected by the UAV in at least one region are obtained, and each historical remote sensing image is converted to obtain a corresponding mask image; wherein each region corresponds to at least one historical remote sensing image;

[0007] Dividing a spot contour corresponding to each grayscale value from each mask image, and performing recognition processing on each spot contour to obtain corresponding mask information; wherein each mask image includes at least one grayscale value and a spot contour corresponding to each grayscale value;

[0008] All historical remote sensing images are divided into a first image set and a second image set, each historical remote sensing image in the first image set is preprocessed, and a training image set is obtained according to each preprocessed historical remote sensing image and all corresponding mask information;

[0009] According to each historical remote sensing image in the second image set and all corresponding mask information, a verification image set is obtained, and the first image segmentation model is trained based on the training image set and the verification image set;

[0010] The target remote sensing image collected by the UAV is obtained, and the image interpretation result is obtained according to the target remote sensing image, the first click operation input by the user and the trained first image segmentation model.

[0011] In an optional solution of the first aspect, each historical remote sensing image is converted to obtain a corresponding mask image, including:

[0012] According to the nth ground feature attribute value input by the user, the ground feature category corresponding to the nth ground feature attribute value is queried in the preset ground feature database, and the ground feature area corresponding to the ground feature category is framed in each historical remote sensing image; wherein n is a positive integer greater than or equal to 1, and the preset ground feature database includes at least two ground feature attribute values ​​and the ground feature category corresponding to each ground feature attribute value;

[0013] When a second click operation input by the user in the feature area is detected, a mark point of the second click operation is generated on the corresponding historical remote sensing image;

[0014] Inputting the historical remote sensing image containing the marked points into the second image segmentation model to obtain the corresponding spot contour; wherein the second image segmentation model is trained by at least two sample remote sensing images containing sample marked points, the sample spot contour corresponding to each sample marked point, and the marking frame corresponding to each sample spot contour;

[0015] According to the preset correspondence between ground object category and grayscale value, the contour of the spots contained in each ground object area in each historical remote sensing image is converted to obtain a mask image of the corresponding historical remote sensing image; wherein the preset correspondence between ground object category and grayscale value includes at least two ground object categories and the grayscale value corresponding to each ground object category.

[0016] In another optional solution of the first aspect, before converting the image spot contour contained in each ground object area in each historical remote sensing image according to the preset ground object category-grayscale value correspondence relationship to obtain the mask image of the corresponding historical remote sensing image, the method further includes:

[0017] When a reduction operation corresponding to the spot contour input by the user is detected, the spot contour is segmented to obtain a first segmented contour and a second segmented contour;

[0018] When the click position of the narrowing operation is within the first segmentation contour, updating the image spot contour to the first segmentation contour; or

[0019] When the click position of the narrowing-down operation is within the second segmentation contour, the spot contour is updated to the second segmentation contour.

[0020] In another optional solution of the first aspect, before converting the image spot contour contained in each ground object area in each historical remote sensing image according to the preset ground object category-grayscale value correspondence relationship to obtain the mask image of the corresponding historical remote sensing image, the method further includes:

[0021] When a first range-increasing operation corresponding to the spot contour input by the user is detected, at least one first newly added contour adjacent to the spot contour is determined based on a click position of the first range-increasing operation;

[0022] Select the first newly added contour with the largest area from all the first newly added contours, merge the spot contour and the first newly added contour with the largest area, and update the spot contour to the merged spot contour;

[0023] When a second range increasing operation corresponding to the spot contour input by the user is detected, the second newly added contour corresponding to the second range increasing operation and the merged spot contour are merged, and the spot contour is updated to the merged spot contour.

[0024] In another optional solution of the first aspect, each spot contour is identified and processed to obtain corresponding mask information, including:

[0025] Determine two first contour points on each pattern spot contour along a preset first axis direction, and determine two second contour points on the corresponding pattern spot contour along a preset second axis direction;

[0026] Generate a circumscribed rectangular frame according to two first contour points and two second contour points on each pattern spot contour, and identify vertex coordinates and size parameters of the circumscribed rectangular frame;

[0027] Generate a random point in each circumscribed rectangular frame, and identify the coordinates of the random point when the random point is detected to be within the corresponding pattern outline;

[0028] The vertex coordinates, size parameters and coordinates of random points corresponding to each spot outline are format converted, and the vertex coordinates, size parameters and coordinates of random points after the format conversion are used as corresponding mask information.

[0029] In yet another optional solution of the first aspect, preprocessing each historical remote sensing image in the first image set includes:

[0030] Performing flip processing on at least one historical remote sensing image in the first image set according to a preset flip ratio;

[0031] Scaling all the historical remote sensing images in the first image set according to a preset scaling ratio;

[0032] All historical remote sensing images after scaling are normalized.

[0033] In another optional solution of the first aspect, obtaining an image interpretation result according to the target remote sensing image, the first click operation input by the user, and the trained first image segmentation model includes:

[0034] When the first click operation input by the user is a point operation, a mark point of the point operation is generated on the target remote sensing image;

[0035] Inputting the target remote sensing image containing the marked points into the trained first image segmentation model to obtain an image interpretation result; or

[0036] When the first click operation input by the user is a frame operation, a marking frame corresponding to the frame operation is generated on the target remote sensing image; wherein the marking frame includes at least one ground feature area on the target remote sensing image;

[0037] The target remote sensing image containing the marked frame is input into the trained first image segmentation model to obtain the image interpretation result.

[0038] In a second aspect, an embodiment of the present application provides a remote sensing image interpretation device based on an image segmentation model, comprising:

[0039] An image processing module is used to obtain at least two historical remote sensing images collected by the drone in at least one region according to the model task requirements, and convert each historical remote sensing image to obtain a corresponding mask image; wherein each region corresponds to at least one historical remote sensing image;

[0040] An information recognition module is used to divide the spot contour corresponding to each grayscale value from each mask image, and perform recognition processing on each spot contour to obtain corresponding mask information; wherein each mask image includes at least one grayscale value and a spot contour corresponding to each grayscale value;

[0041] A set generation module is used to divide all historical remote sensing images into a first image set and a second image set, preprocess each historical remote sensing image in the first image set, and obtain a training image set based on each preprocessed historical remote sensing image and all corresponding mask information;

[0042] A model training module, used to obtain a verification image set according to each historical remote sensing image in the second image set and all corresponding mask information, and train the first image segmentation model based on the training image set and the verification image set;

[0043] The image interpretation module is used to obtain the target remote sensing image collected by the UAV, and obtain the image interpretation result according to the target remote sensing image, the first click operation input by the user and the trained first image segmentation model.

[0044] In a third aspect, an embodiment of the present application further provides a remote sensing image interpretation device based on an image segmentation model, comprising a processor and a memory;

[0045] The processor is connected to the memory;

[0046] A memory for storing executable program codes;

[0047] The processor runs the program corresponding to the executable program code by reading the executable program code stored in the memory, so as to implement the remote sensing image interpretation method based on the image segmentation model provided by the first aspect of the embodiment of the present application or any one of the implementation methods of the first aspect.

[0048] In a fourth aspect, an embodiment of the present application provides a computer storage medium, which stores a computer program. The computer program includes program instructions. When the program instructions are executed by a processor, the remote sensing image interpretation method based on the image segmentation model provided by the first aspect of the embodiment of the present application or any one of the implementation methods of the first aspect can be implemented.

[0049] In an embodiment of the present application, when interpreting the ground objects in a remote sensing image, a plurality of historical remote sensing images can be converted to obtain corresponding mask images, and each pattern spot outline divided by each mask image can be identified to obtain corresponding mask information; then, a training image set and a verification image set can be divided according to all historical remote sensing images and all corresponding mask information to train the first image segmentation model, and an image interpretation result can be obtained according to the target remote sensing image, a first click operation input by the user, and the trained first image segmentation model. This not only avoids excessive manual intervention operations, but also utilizes the first image segmentation model trained by the historical remote sensing images and the corresponding mask information to improve the accuracy of the ground object interpretation results, thereby effectively ensuring the interpretation efficiency of the remote sensing images. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0051] Figure 1 An overall flow chart of a remote sensing image interpretation method based on an image segmentation model provided in an embodiment of the present application;

[0052] Figure 2 A schematic diagram of the effect of a mask image provided in an embodiment of the present application;

[0053] Figure 3 A schematic diagram of the effect of mask information provided by an embodiment of the present application;

[0054] Figure 4 A schematic diagram of an image interpretation result provided in an embodiment of the present application;

[0055] Figure 5 A schematic diagram of the structure of a remote sensing image interpretation device based on an image segmentation model provided in an embodiment of the present application;

[0056] Figure 6 A schematic structural diagram of another remote sensing image interpretation device based on an image segmentation model provided in an embodiment of the present application. DETAILED DESCRIPTION

[0057] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0058] In the following introduction, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance. The following introduction provides multiple embodiments of the present application, and different embodiments can be replaced or combined, so the present application can also be considered to include all possible combinations of the same and / or different embodiments recorded. Therefore, if one embodiment includes features A, B, and C, and another embodiment includes features B and D, then the present application should also be considered to include embodiments containing one or more of all other possible combinations of A, B, C, and D, although the embodiment may not be clearly recorded in the following text.

[0059] The following description provides examples and does not limit the scope, applicability or examples set forth in the claims. Changes may be made to the functions and arrangements of the elements described without departing from the scope of the present application. Various processes or components may be appropriately omitted, substituted or added to each example. For example, the described method may be performed in an order different from the order described, and various steps may be added, omitted or combined. In addition, features described in some examples may be combined in other examples.

[0060] See also Figure 1 , Figure 1 The overall flow chart of a remote sensing image interpretation method based on an image segmentation model provided in an embodiment of the present application is shown.

[0061] like Figure 1 As shown, the remote sensing image interpretation method based on the image segmentation model may at least include the following steps:

[0062] Step 102: acquiring at least two historical remote sensing images collected by the UAV in at least one region according to the model mission requirements, and converting each historical remote sensing image to obtain a corresponding mask image.

[0063] In an embodiment of the present application, the remote sensing image interpretation method based on the image segmentation model can be but is not limited to being applied to a platform terminal. The platform terminal can be provided with a user interaction interface to receive different types of operations input by the user on the user interaction interface, and the platform terminal can also establish a wireless communication connection with a drone to obtain multiple historical remote sensing images collected by the drone in different regions. The preset first image segmentation model can be trained in combination with the different types of operations input by the user and the historical remote sensing images, and the trained first image segmentation model can be used to efficiently and accurately interpret the target remote sensing images collected by the drone.

[0064] Among them, the platform terminal can display each historical remote sensing image or target remote sensing image collected by the drone on the user interaction interface, so that the user can click or select each historical remote sensing image or target remote sensing image. The click operation can be but not limited to a click operation on a certain position in the remote sensing image or a frame operation on a certain area in the remote sensing image (that is, selecting a certain area through a rectangular frame); the selection operation can be but not limited to a reduction range operation or a magnification operation on a certain area. For example, when the user performs a left-click operation on a certain position outside a certain area in a certain historical remote sensing image, it indicates that the user wants to perform a magnification operation on the area, that is, to increase the area; when the user performs a right-click operation on a certain position within a certain area in a certain historical remote sensing image, it indicates that the user wants to perform a reduction range operation on the area, that is, to reduce the area.

[0065] Of course, the user can also input conventional operations such as confirmation or cancellation in the user interaction interface, which will not be elaborated here.

[0066] In order to reduce the impact of sample imbalance in different regions, the multiple historical remote sensing images collected by the drone in different regions may include, but are not limited to, one or more historical remote sensing images collected in hilly and mountainous areas, one or more historical remote sensing images collected in plain areas, one or more historical remote sensing images collected in urban areas, and one or more historical remote sensing images collected in rural areas. Here, each historical remote sensing image may include at least one or more types of land objects, which may be, but are not limited to, buildings, grasslands, trees, water bodies, or tea trees, etc., and the platform terminal may also assign land object attribute values ​​to the buildings, grasslands, trees, water bodies, and tea trees, respectively, so that when the user enters any land object attribute value on a remote sensing image, the land object area corresponding to the land object attribute value can be selected on the remote sensing image.

[0067] It can be understood that the preset first image segmentation model structure built into the platform terminal can be but is not limited to referring to the Segment Anything Model (SAM) structure well known in the art, for example, it may include an efficientvit-sam picture encoder, a prompt encoder and a mask decoder. Here, the efficientvit-sam picture encoder is taken as an example. The efficientvit-sam picture encoder can be specifically composed of an input layer, a convolutional layer, a residual module, 3 F-MBConv convolutional layers, two efficient VIT modules (each efficient VIT module can be composed of F-MBConv and Efficientvit core modules), an F-MBConv convolutional layer and an output layer in sequence, and the prompt encoder and the mask decoder can both include a composition structure similar to that of the efficientvit-sam picture encoder, which will not be described in detail here.

[0068] In addition, the preset first image segmentation model may also be set with various operating parameters by the platform terminal during the training process, such as but not limited to setting the basic learning rate of the preset first image segmentation model to 0.0000004 (a lower learning rate is used to ensure that the preset first image segmentation model can stably adapt while learning new tasks, and retain most of the valuable pre-trained features, reducing the risk of overfitting), setting the learning rate adjustment strategy of the preset first image segmentation model to cosine (that is, gradually reducing the learning rate according to the cosine function), setting the optimizer of the weight minimization loss function of the preset first image segmentation model to AdamW (to process weight decay and gradient update separately, with better generalization ability and more effective learning rate scheduling), setting the weight decay of the preset first image segmentation model to 0.1 (to prevent overfitting), setting the gradient clipping threshold of the preset first image segmentation model to 2.0, and setting the training rounds of the preset first image segmentation model to 200, and not limited to this.

[0069] It can also be understood that the platform terminal obtains corresponding mask images by converting multiple historical remote sensing images, and identifies and processes the contours of each patch divided by each mask image to obtain corresponding mask information; then, a training image set and a verification image set can be divided according to all historical remote sensing images and all corresponding mask information to train the first image segmentation model, and an image interpretation result can be obtained according to the target remote sensing image, the first click operation input by the user, and the trained first image segmentation model. This not only avoids excessive manual intervention, but also utilizes the first image segmentation model trained by historical remote sensing images and corresponding mask information to improve the accuracy of the ground object interpretation results, thereby effectively ensuring the interpretation efficiency of remote sensing images.

[0070] Specifically, when interpreting the objects in the remote sensing images, the platform terminal can, but is not limited to, screen out multiple historical remote sensing images collected from at least one region within a specified period of time from all the historical remote sensing images collected by the drone according to the model task requirements. The specified period of time can be, but is not limited to, the xth quarter of xx year. The at least one region can be, but is not limited to, at least one of the hilly and mountainous areas, plain areas, urban areas, and rural areas mentioned according to the model task requirements, and each region corresponds to at least one historical remote sensing image. Here, each of the screened historical remote sensing images can include at least one of the above-mentioned buildings, grasslands, trees, water bodies, or tea trees, so as to ensure the subsequent user's needs for object interpretation of the target remote sensing image.

[0071] Furthermore, after determining a plurality of historical remote sensing images, the platform terminal may also sequentially display each historical remote sensing image to the user through a user interaction interface, so as to determine one or more spot contours in the corresponding historical remote sensing image according to different types of operations input by the user, and obtain a corresponding mask image by converting the one or more spot contours. Here, the size of the mask image may be consistent with the size of the historical remote sensing image (for example, both may be 512*512, and the image format may be set to PNG), and the mask image may be composed of a spot contour represented by at least one grayscale value and a white background. The grayscale values ​​corresponding to the spot contours of different land object categories are different, and the same land object category may correspond to one or more spot contours of the same grayscale value, and the position of each spot contour in the mask image is consistent with the position of the spot contour in the corresponding historical remote sensing image.

[0072] It is understandable that the method of converting one or more spot contours in the historical remote sensing image may be a conventional technical means in the art for creating a mask image based on the spot contours, which will not be elaborated herein.

[0073] As an option of the embodiment of the present application, each historical remote sensing image is converted to obtain a corresponding mask image, including:

[0074] According to the nth ground feature attribute value input by the user, the ground feature category corresponding to the nth ground feature attribute value is queried in the preset ground feature database, and the ground feature area corresponding to the ground feature category is framed in each historical remote sensing image; wherein n is a positive integer greater than or equal to 1, and the preset ground feature database includes at least two ground feature attribute values ​​and the ground feature category corresponding to each ground feature attribute value;

[0075] When a second click operation input by the user in the feature area is detected, a mark point of the second click operation is generated on the corresponding historical remote sensing image;

[0076] Inputting the historical remote sensing image containing the marked points into the second image segmentation model to obtain the corresponding spot contour; wherein the second image segmentation model is trained by at least two sample remote sensing images containing sample marked points, the sample spot contour corresponding to each sample marked point, and the marking frame corresponding to each sample spot contour;

[0077] According to the preset correspondence between ground object category and grayscale value, the contour of the spots contained in each ground object area in each historical remote sensing image is converted to obtain a mask image of the corresponding historical remote sensing image; wherein the preset correspondence between ground object category and grayscale value includes at least two ground object categories and the grayscale value corresponding to each ground object category.

[0078] Specifically, when converting each historical remote sensing image to obtain a corresponding mask image, the platform terminal can obtain the nth ground feature attribute value input by the user on the historical remote sensing image after displaying the historical remote sensing image to the user through the user interaction interface, where n can be a positive integer greater than or equal to 1 (maximum not more than 5), that is, the user can input at least one ground feature attribute value on the historical remote sensing image, and each ground feature attribute value can correspond to any ground feature category of the above-mentioned buildings, grasslands, trees, water bodies or tea trees, and after the user inputs a ground feature attribute value each time, the platform terminal can also query the ground feature category corresponding to the currently input ground feature attribute value in the preset ground feature database, so as to select the ground feature area corresponding to the ground feature category in the historical remote sensing image by image recognition and other methods. It can be understood that when the user inputs multiple ground feature attribute values ​​on the historical remote sensing image, the platform terminal can determine the ground feature area corresponding to each ground feature category in the historical remote sensing image in turn according to the ground feature category corresponding to each ground feature attribute value.

[0079] Here, the preset feature database may include at least two feature attribute values ​​and a feature category corresponding to each feature attribute value. For example, it may include five feature attribute values ​​represented as 1, 2, 3, 4 and 5, wherein the feature category corresponding to the feature attribute value of 1 may be a building, the feature category corresponding to the feature attribute value of 2 may be a grassland, the feature category corresponding to the feature attribute value of 3 may be a tree, the feature category corresponding to the feature attribute value of 4 may be a water body, and the feature category corresponding to the feature attribute value of 5 may be a tea tree.

[0080] It can also be understood that the above-mentioned processing method of selecting the object area corresponding to the object category in the historical remote sensing image can be, but is not limited to, inputting the historical remote sensing image and the object category into a preset image processing model, so as to mark the object area corresponding to the object category on the historical remote sensing image by means of model prediction. Of course, in the embodiments of the present application, image processing technical means well known in the art can also be used, without limitation thereto.

[0081] Next, after selecting the feature area corresponding to the feature category in the historical remote sensing image, the platform terminal can receive a second click operation input by the user in the feature area, which can be understood as a click operation on a certain position in the feature area, for example, a click operation on a certain building in the building area, and a mark point corresponding to the certain position can be generated on the historical remote sensing image. Here, the coordinates of the mark point on the historical remote sensing image can be consistent with the coordinates of the click operation on the above-mentioned certain position on the historical remote sensing image.

[0082] Next, the platform terminal may input the historical remote sensing image containing the marking point into the second image segmentation model, so as to mark the spot contour corresponding to the marking point on the historical remote sensing image by means of model prediction. For example, but not limited to, when the position of the marking point is consistent with the click operation position of a building in the building area, the spot contour of the corresponding building may be marked on the historical remote sensing image by the second image segmentation model. Here, the second image segmentation model may be, but is not limited to, a large image segmentation model well known in the art, which is trained by at least two sample remote sensing images containing sample marking points, a sample spot contour corresponding to each sample marking point, and a marking frame corresponding to each sample spot contour. The sample spot contour corresponding to each sample marking point and the marking frame corresponding to each sample spot contour may be obtained by manual annotation or machine calculation, and are not limited thereto.

[0083] It can be understood that each feature area in each historical remote sensing image may correspond to one or more map contours, that is, the user can input one or more location click operations in a certain feature area, and each time the user inputs a location click operation, the platform terminal can generate a corresponding mark point on the historical remote sensing image, and use the second image segmentation model to mark the corresponding map contour on the historical remote sensing image; or after the user inputs the click operation of all locations, the platform terminal can generate all corresponding mark points on the historical remote sensing image, and use the second image segmentation model to mark all corresponding map contours on the historical remote sensing image, but is not limited to this.

[0084] Then, after marking all the spot contours on the historical remote sensing image, the platform terminal can also determine the grayscale value of the ground object category corresponding to each spot contour according to the preset ground object category-grayscale value correspondence relationship, and then convert all the spot contours contained in each ground object area in the historical remote sensing image according to the corresponding grayscale value, and then obtain the corresponding mask image. Here, the preset ground object category-grayscale value correspondence relationship may include at least two ground object categories and the grayscale value corresponding to each ground object category. For example, when the ground object category is a building, the corresponding grayscale value can be set to 15 (that is, the RGB channel corresponds to 0, 0, 128, and the display color can be blue); when the ground object category is a water body, the corresponding grayscale value can be set to 173 (that is, the RGB channel corresponds to 255, 165, 0, and the display color can be orange).

[0085] Also see here Figure 2 The schematic diagram of the effect of a mask image provided by an embodiment of the present application is shown as follows: Figure 2 As shown, 2A may be a historical remote sensing image of a certain area containing water bodies and trees, 2a may correspond to the mask image of 2A, it can be seen that the number of spot contours corresponding to water bodies is greater than one, the number of spot contours corresponding to trees is equal to one, and the white background area in 2a is the image area in 2A excluding water bodies and trees; 2B may be a historical remote sensing image of a certain area containing buildings, grasslands, water bodies and trees, 2b may correspond to the mask image of 2B, it can be seen that the number of spot contours corresponding to buildings is greater than one, the number of spot contours corresponding to grasslands is equal to one, and the number of spot contours corresponding to water bodies is greater than one. The number of spot contours corresponding to buildings is greater than one, the number of spot contours corresponding to trees is greater than one, and the white background area in 2b is the image area excluding buildings, grasslands, water bodies and trees in 2B; 2C may be a historical remote sensing image of a certain area containing buildings, grasslands and trees, and 2c may correspond to the mask image of 2C. It can be seen that the number of spot contours corresponding to buildings is greater than one, the number of spot contours corresponding to grasslands is greater than one, the number of spot contours corresponding to trees is greater than one, and the white background area in 2c is the image area excluding buildings, grasslands and trees in 2C.

[0086] As another option of the embodiment of the present application, before converting the image spot contour contained in each ground object area in each historical remote sensing image according to the preset ground object category-grayscale value correspondence relationship to obtain the mask image of the corresponding historical remote sensing image, it also includes:

[0087] When a reduction operation corresponding to the spot contour input by the user is detected, the spot contour is segmented to obtain a first segmented contour and a second segmented contour;

[0088] When the click position of the narrowing operation is within the first segmentation contour, updating the image spot contour to the first segmentation contour; or

[0089] When the click position of the narrowing-down operation is within the second segmentation contour, the spot contour is updated to the second segmentation contour.

[0090] Specifically, in order to ensure the range accuracy of each spot outline, after obtaining the historical remote sensing image marked with the spot outline through the second image segmentation model, when the platform terminal detects that the user inputs a reduction operation corresponding to the spot outline on the historical remote sensing image marked with the spot outline, it indicates that the range of the spot outline is large. The reduction operation here can be but is not limited to the right-click operation input by the user within the spot outline, and then the spot outline can be segmented through the image segmentation technology well known in the art to obtain the first segmented outline and the second segmented outline. It can be understood that when the embodiment of the present application performs segmentation processing on the spot outline, it can be but is not limited to first identifying the characteristic contour line within the spot outline to segment the spot outline according to the characteristic contour line, and is not limited to this.

[0091] Next, when it is detected that the click position of the narrowing operation is within the first segmentation contour, it indicates that the spot contour that meets the user's needs is the first segmentation contour, and the spot contour in the historical remote sensing image can be updated to the first segmentation contour, that is, the range of the spot contour is narrowed. It can be understood that if the narrowing operation input by the user continues to be detected within the first segmentation contour, it indicates that the spot contour that meets the user's needs is within the first segmentation contour, and the first segmentation contour needs to be segmented until the spot contour that meets the user's needs is obtained.

[0092] When it is detected that the click position of the narrowing operation is within the second segmentation contour, it indicates that the spot contour that meets the user's needs is the second segmentation contour, and the spot contour in the historical remote sensing image can be updated to the second segmentation contour, that is, the range of the spot contour is narrowed. Of course, if the narrowing operation input by the user continues to be detected within the second segmentation contour, it indicates that the spot contour that meets the user's needs is within the second segmentation contour, and the second segmentation contour needs to be segmented until the spot contour that meets the user's needs is obtained.

[0093] As another option of the embodiment of the present application, before converting the image spot contour contained in each ground object area in each historical remote sensing image according to the preset ground object category-grayscale value correspondence relationship to obtain the mask image of the corresponding historical remote sensing image, it also includes:

[0094] When a first range-increasing operation corresponding to the spot contour input by the user is detected, at least one first newly added contour adjacent to the spot contour is determined based on a click position of the first range-increasing operation;

[0095] Filter out the first newly added contour with the largest area from all newly added contours, merge the spot contour and the first newly added contour with the largest area, and update the spot contour to the merged spot contour;

[0096] When a second range increasing operation corresponding to the spot contour input by the user is detected, the second newly added contour corresponding to the second range increasing operation and the merged spot contour are merged, and the spot contour is updated to the merged spot contour.

[0097] Specifically, in order to ensure the range accuracy of each spot outline, after obtaining the historical remote sensing image marked with the spot outline through the second image segmentation model, when the platform terminal detects that the user inputs the first range increase operation corresponding to the spot outline on the historical remote sensing image marked with the spot outline, it indicates that the range of the spot outline is small. The first range increase operation here can be but is not limited to the left-click operation input by the user in the area close to the spot outline, and then the contour addition direction of the spot outline can be determined according to the click position of the first range increase operation. For example, when the click position of the first range increase operation is on the left side of the spot outline, it can be determined that the contour addition direction of the spot outline is on the left side, and at least one first new contour adjacent to the spot outline can be identified along the contour addition direction of the spot outline on the historical remote sensing image marked with the spot outline through image recognition technology well known in the art.

[0098] It can be understood that when the embodiment of the present application identifies at least one first newly added contour adjacent to the spot contour, it can, but is not limited to, first identifying the characteristic contour line of the spot contour on the historical remote sensing image, and at least one newly added characteristic contour line identified along the newly added contour direction of the spot contour, and then identifying and processing the characteristic contour line of the spot contour and the area enclosed by all the newly added characteristic contour lines to obtain at least one first newly added contour adjacent to the spot contour, and is not limited to this.

[0099] Then, after determining at least one first newly added contour adjacent to the spot contour, the platform terminal may also select the first newly added contour with the largest area from all the first newly added contours, merge the first newly added contour with the spot contour (or perform image stitching processing, etc.), and when the second increase range operation input by the user in the area close to the merged spot contour is not detected within a preset time interval, it indicates that the merged spot contour meets the user's needs, and then the spot contour in the historical remote sensing image can be updated to the merged spot contour, that is, the range of the spot contour is increased. Here, the second increase range operation can be, but is not limited to, a left-click operation input by the user in the area close to the merged spot contour.

[0100] If a second range-increasing operation input by the user in an area close to the merged spot contour is detected within a preset time interval, it indicates that the merged spot contour has not yet met the user's needs, that is, the range of the merged spot contour is small, and then the second newly added contour corresponding to the second range-increasing operation (that is, the newly added contour with the largest area among at least one newly added contour adjacent to the merged spot contour) can be determined by referring to the above method, and the currently merged spot contour is merged again until a spot contour that meets the user's needs is obtained. Here, the second range-increasing operation can be, but is not limited to, a left-click operation input by the user in an area close to the spot contour, and is not limited to this.

[0101] Step 104: Divide the spot contour corresponding to each gray value from each mask image, and perform recognition processing on each spot contour to obtain corresponding mask information.

[0102] Specifically, when the platform terminal obtains the mask image corresponding to each historical remote sensing image, it can also, but is not limited to, save and process each mask image according to a preset storage path to facilitate subsequent data tracing by users; and after saving the mask images of all historical remote sensing images, the platform terminal can read each mask image in sequence according to the chronological order of the saving time, and when reading the mask image, it can specifically divide one or more spot contours corresponding to each grayscale value according to the grayscale value mode, for example, but not limited to, each grayscale value can be used as a mask object, and the find Countours function of the opencv library can be used to divide one or more spot contours corresponding to each grayscale value in the mask image.

[0103] Furthermore, after dividing all the pattern spot contours in each mask image, the platform terminal can, but is not limited to, identify two first contour points on each pattern spot contour along a preset first axis direction, and the preset first axis direction can be a horizontal axis direction, that is, the two first contour points can be respectively the leftmost contour point and the rightmost contour point of each pattern spot contour in the horizontal direction; in addition, the platform terminal can also identify two second contour points on each pattern spot contour along a preset second axis direction, and the preset second axis direction can be a vertical axis direction, that is, the two second contour points can be respectively the highest contour point and the lowest contour point of each pattern spot contour in the vertical direction.

[0104] Further, after obtaining the two first contour points and the two second contour points of each spot contour, the platform terminal can generate a bounding rectangle of the corresponding spot contour (i.e., the minimum bounding rectangle containing the corresponding spot contour) according to the two first contour points and the two second contour points, and the two first contour points and the two second contour points are all located on the bounding rectangle, and the vertex coordinates and size parameters of the bounding rectangle can be identified by image processing techniques well known in the art. Here, the vertex coordinates of the bounding rectangle can be, but are not limited to, the coordinates of any vertex in the plane rectangular coordinate system corresponding to the historical remote sensing image, the size parameters of the bounding rectangle can be, but are not limited to, the length and width, the origin of the plane rectangular coordinate system corresponding to the historical remote sensing image can be, but are not limited to, the lower left vertex of the historical remote sensing image, and the X-coordinate axis and the Y-coordinate axis of the plane rectangular coordinate system corresponding to the historical remote sensing image can be, respectively, the long side and the wide side of the lower left vertex, and are not limited thereto.

[0105] Furthermore, after determining the circumscribed rectangular box, vertex coordinates and size parameters corresponding to each spot outline, the platform terminal can also randomly generate a point within the circumscribed rectangular box, so that when it is detected that the randomly generated point is within the corresponding spot outline, the coordinates of the randomly generated point can be determined in combination with the above-mentioned plane rectangular coordinate system, and the vertex coordinates, size parameters and coordinates of the randomly generated point corresponding to each spot outline can be format converted, for example but not limited to conversion into a json file format, and the vertex coordinates, size parameters and coordinates of the randomly generated point after the format conversion can be used as the mask information corresponding to the corresponding spot outline on the historical remote sensing image.

[0106] It is understandable that, in addition to using the vertex coordinates, size parameters, and coordinates of randomly generated points as mask information corresponding to the spot contour on the historical remote sensing image, the platform terminal can also, but is not limited to, identify the spot mask corresponding to the spot contour on the mask image corresponding to the historical remote sensing image, so as to convert the spot mask into an RLE code, and similarly, by performing format conversion processing on the RLE code, use the vertex coordinates, size parameters, coordinates of randomly generated points, and RLE code after format conversion as mask information corresponding to the corresponding spot contour on the historical remote sensing image. Of course, in the embodiment of the present application, the number information, area, size parameters, and other information of the spot contour can also be format converted to serve as the mask information corresponding to the corresponding spot contour on the historical remote sensing image, but it is not limited to this.

[0107] Also see here Figure 3 The schematic diagram of the effect of a mask information provided by an embodiment of the present application is shown as follows: Figure 3 As shown, 3A may correspond to a historical remote sensing image containing two land feature areas, namely buildings and trees; 3B may correspond to the mask image of 3A; 3C may correspond to the circumscribed rectangular box generated for each pattern spot outline in the mask image of 3B and the randomly generated points within the circumscribed rectangular box; the mask information of each pattern spot outline may be the vertex coordinates, size parameters and coordinates of the randomly generated points of the corresponding circumscribed rectangular box after format conversion.

[0108] Step 106: divide all historical remote sensing images into a first image set and a second image set, preprocess each historical remote sensing image in the first image set, and obtain a training image set based on each preprocessed historical remote sensing image and all corresponding mask information.

[0109] Specifically, after determining the mask information corresponding to each patch outline in each historical remote sensing image, the platform terminal can, but is not limited to, divide all historical remote sensing images into a first image set representing a training set and a second image set representing a verification set according to a preset ratio, and then preprocess all historical remote sensing images in the first image set to ensure the training effect of the first image segmentation model, and determine the training image set based on each preprocessed historical remote sensing image and the mask information corresponding to each historical remote sensing image.

[0110] It can be understood that when determining a training image set based on each preprocessed historical remote sensing image and the mask information corresponding to each historical remote sensing image, it can be but is not limited to the user inputting one or more click operations at positions on each preprocessed historical remote sensing image through the user interaction interface, so that the platform terminal generates marking points corresponding to all click operations on the historical remote sensing image; then, the platform terminal can also combine the mask image corresponding to the historical remote sensing image to generate a patch outline corresponding to each marking point on the historical remote sensing image containing the marking point, so as to use the historical remote sensing image containing the marking point and the patch outline corresponding to the marking point as the real image of the corresponding historical remote sensing image containing the marking point; in addition, the platform terminal can also filter out the mask information corresponding to the marking point in the corresponding mask information based on the historical remote sensing image containing the marking point, and can use each preprocessed historical remote sensing image containing the marking point, the real image corresponding to each historical remote sensing image and the filtered mask information as the training image set.

[0111] As another option of the embodiment of the present application, preprocessing each historical remote sensing image in the first image set includes:

[0112] Performing flip processing on at least one historical remote sensing image in the first image set according to a preset flip ratio;

[0113] Scaling all the historical remote sensing images in the first image set according to a preset scaling ratio;

[0114] All historical remote sensing images after scaling are normalized.

[0115] Specifically, when preprocessing each historical remote sensing image in the first image set, the platform terminal may flip at least one historical remote sensing image in the first image set according to a preset flipping ratio. For example, but not limited to, when the preset flipping ratio is 30%, 30% of the historical remote sensing images in the first image set may be horizontally flipped to enhance the robustness of the first image segmentation model to directional changes of historical remote sensing images during training.

[0116] Then, after flipping at least one historical remote sensing image in the first image set, the platform terminal can also scale each historical remote sensing image in the first image set according to a preset scaling ratio to ensure the consistency and accuracy of all historical remote sensing images while maintaining the aspect ratio of each historical remote sensing image unchanged.

[0117] Next, after scaling all the historical remote sensing images in the first image set, the platform terminal can also perform image normalization processing on each historical remote sensing image in the first image set, and can, but is not limited to, set the average value list of the normalization processing to [0.485, 0.456, 0.406] and the standard deviation list to [0.229, 0.224, 0.225], which not only helps the first image segmentation model to converge quickly during training, but also improves the generalization ability of the first image segmentation model.

[0118] Of course, after performing image normalization processing on each historical remote sensing image in the first image set, the embodiment of the present application can also, but is not limited to, perform image filling processing on each historical remote sensing image to further ensure the consistency and accuracy of all historical remote sensing images.

[0119] Step 108: Obtain a verification image set according to each historical remote sensing image in the second image set and all corresponding mask information, and train the first image segmentation model based on the training image set and the verification image set.

[0120] Specifically, after obtaining the training image set, the platform terminal may, but is not limited to, preprocess each historical remote sensing image in the second image set, and the preprocessing method may be any one of scaling processing, image normalization processing, or image filling processing, and the verification image set may be determined based on each preprocessed historical remote sensing image and the mask information corresponding to each historical remote sensing image. Here, the method for determining the verification image set can refer to the above description, and will not be repeated here.

[0121] Of course, the platform terminal in the embodiment of the present application can also be but is not limited to being provided with a data provider, by inputting all historical remote sensing images and the mask information corresponding to each historical remote sensing image into the data provider, so that the data provider generates a training image set and a verification image set according to a preset automatic control program, but this is not limited to this here.

[0122] Furthermore, after obtaining the verification image set, the platform terminal can, but is not limited to, set operating parameters such as basic learning rate, learning rate adjustment strategy, optimizer of weight minimization loss function, weight decay, gradient clipping threshold and training rounds for the built-in first image segmentation model, and substitute the data in the training image set and the data in the verification image set into the first image segmentation model for training.

[0123] Here, taking the training of the first image segmentation model in combination with the training image set as an example, the platform terminal can input the historical remote sensing image containing the marked points in the training image set into the efficientvit-sam picture encoder, so that the efficientvit-sam picture encoder downsamples the historical remote sensing image and obtains the corresponding embedding vector; then, the platform terminal can identify the spot contour corresponding to the marked point in the corresponding mask image, and perform convolution calculation on the mask image marked with the spot contour and the embedding vector to obtain the convolution image; then, the platform terminal can use the prompt encoder to convolve the historical remote sensing image with the historical remote sensing image. The mask information corresponding to the historical remote sensing image is encoded, for example but not limited to encoding vertex coordinates and size parameters, or encoding coordinates of randomly generated points, and the encoding result and the convolution image mentioned above are input into the mask decoder to obtain a predicted image; then, the platform terminal can also calculate the total loss function (for example, including mask loss and Dice loss) and evaluation index (for example, average intersection-over-union ratio) according to the predicted image and the real image corresponding to the historical remote sensing image, and optimize the operating parameters of the first image segmentation model in combination with the total loss function and the evaluation index until the training round reaches the set threshold.

[0124] In order to better prove that the first image segmentation model trained in the present application is superior to the conventional large image segmentation model (ie, the SAM large model) in the art, the following table can also be referred to:

[0125]

[0126] Among them, the point prompt can be understood as the coordinates of the randomly generated points mentioned above are used as the prompt encoder, and are input into the SAM large model and the first image segmentation model for prediction. The box prompt can be understood as the vertex coordinates and size parameters mentioned above are used as the prompt encoder, and are input into the SAM large model and the first image segmentation model for prediction. The data in the table can be understood as the evaluation index between the predicted image and the real image obtained by the corresponding model. It can be seen that whether it is a verification image set or a training image set, the evaluation index of the first image segmentation model under the point prompt is better than the evaluation index of the SAM large model, and the evaluation index of the first image segmentation model under the box prompt is also better than the evaluation index of the SAM large model, and the performance improvement of the first image segmentation model under the point prompt is significantly better than the performance of the first image segmentation model predicted under the box prompt.

[0127] Step 110: Acquire the target remote sensing image collected by the drone, and obtain an image interpretation result according to the target remote sensing image, the first click operation input by the user, and the trained first image segmentation model.

[0128] Specifically, after the training of the first image segmentation model is completed, the platform terminal can, but is not limited to, receive the target remote sensing image collected by the drone in the designated area, and display it to the user through the user interaction interface, so that the user can input a first click operation on the target remote sensing image. The first click operation can, but is not limited to, a point operation on a certain position on the target remote sensing image or a box operation on a certain area, and the platform terminal generates a marking point or a marking box corresponding to the first click operation on the target remote sensing image.

[0129] Furthermore, the platform terminal can input the target remote sensing image containing the marking point or the marking box into the trained first image segmentation model to predict the image interpretation result through the trained first image segmentation model, that is, to mark the spot contour area corresponding to the marking point or the marking box in the target remote sensing image.

[0130] Also see here Figure 4 FIG. 1 is a schematic diagram showing an effect of an image interpretation result provided by an embodiment of the present application, such as Figure 4 As shown, 4A corresponds to a target remote sensing image containing a marking point, and 4B corresponds to a target remote sensing image containing a spot contour area corresponding to the marking point predicted by the first image segmentation model.

[0131] As another option of the embodiment of the present application, an image interpretation result is obtained according to the target remote sensing image, the first click operation input by the user, and the trained first image segmentation model, including:

[0132] When the first click operation input by the user is a point operation, a mark point of the point operation is generated on the target remote sensing image;

[0133] Inputting the target remote sensing image containing the marked points into the trained first image segmentation model to obtain an image interpretation result; or

[0134] When the first click operation input by the user is a frame operation, a marking frame corresponding to the frame operation is generated on the target remote sensing image; wherein the marking frame includes at least one ground feature area on the target remote sensing image;

[0135] The target remote sensing image containing the marked frame is input into the trained first image segmentation model to obtain the image interpretation result.

[0136] It can be understood that when the first click operation input by the user is a point operation, the first image segmentation model can be trained by the coordinates of the randomly generated points mentioned above; when the first click operation input by the user is a box operation, the first image segmentation model can be trained by the vertex coordinates and size parameters mentioned above to ensure the prediction accuracy of the first image segmentation model, but is not limited to this.

[0137] See also Figure 5 , Figure 5 A schematic structural diagram of a remote sensing image interpretation device based on an image segmentation model provided in an embodiment of the present application is shown.

[0138] like Figure 5 As shown, the remote sensing image interpretation device based on the image segmentation model may include at least an image processing module 501, an information recognition module 502, a set generation module 503, a model training module 504 and an image interpretation module 505, wherein:

[0139] The image processing module 501 is used to obtain at least two historical remote sensing images collected by the drone in at least one region according to the model task requirements, and convert each historical remote sensing image to obtain a corresponding mask image; wherein each region corresponds to at least one historical remote sensing image;

[0140] The information recognition module 502 is used to divide the spot contour corresponding to each gray value from each mask image, and perform recognition processing on each spot contour to obtain corresponding mask information; wherein each mask image includes at least one gray value and the spot contour corresponding to each gray value;

[0141] A set generation module 503 is used to divide all historical remote sensing images into a first image set and a second image set, preprocess each historical remote sensing image in the first image set, and obtain a training image set based on each preprocessed historical remote sensing image and all corresponding mask information;

[0142] A model training module 504 is used to obtain a verification image set according to each historical remote sensing image in the second image set and all corresponding mask information, and train the first image segmentation model based on the training image set and the verification image set;

[0143] The image interpretation module 505 is used to obtain the target remote sensing image collected by the drone, and obtain the image interpretation result according to the target remote sensing image, the first click operation input by the user and the trained first image segmentation model.

[0144] In some possible embodiments, each historical remote sensing image is converted to obtain a corresponding mask image, including:

[0145] According to the nth ground feature attribute value input by the user, the ground feature category corresponding to the nth ground feature attribute value is queried in the preset ground feature database, and the ground feature area corresponding to the ground feature category is framed in each historical remote sensing image; wherein n is a positive integer greater than or equal to 1, and the preset ground feature database includes at least two ground feature attribute values ​​and the ground feature category corresponding to each ground feature attribute value;

[0146] When a second click operation input by the user in the feature area is detected, a mark point of the second click operation is generated on the corresponding historical remote sensing image;

[0147] Inputting the historical remote sensing image containing the marked points into the second image segmentation model to obtain the corresponding spot contour; wherein the second image segmentation model is trained by at least two sample remote sensing images containing sample marked points, the sample spot contour corresponding to each sample marked point, and the marking frame corresponding to each sample spot contour;

[0148] According to the preset correspondence between ground object category and grayscale value, the contour of the spots contained in each ground object area in each historical remote sensing image is converted to obtain a mask image of the corresponding historical remote sensing image; wherein the preset correspondence between ground object category and grayscale value includes at least two ground object categories and the grayscale value corresponding to each ground object category.

[0149] In some possible embodiments, before converting the image spot contour contained in each ground object area in each historical remote sensing image according to the preset ground object category-grayscale value correspondence relationship to obtain the mask image of the corresponding historical remote sensing image, the method further includes:

[0150] When a reduction operation corresponding to the spot contour input by the user is detected, the spot contour is segmented to obtain a first segmented contour and a second segmented contour;

[0151] When the click position of the narrowing operation is within the first segmentation contour, updating the image spot contour to the first segmentation contour; or

[0152] When the click position of the narrowing-down operation is within the second segmentation contour, the spot contour is updated to the second segmentation contour.

[0153] In some possible embodiments, before converting the image spot contour contained in each ground object area in each historical remote sensing image according to the preset ground object category-grayscale value correspondence relationship to obtain the mask image of the corresponding historical remote sensing image, the method further includes:

[0154] When a first range-increasing operation corresponding to the spot contour input by the user is detected, at least one first newly added contour adjacent to the spot contour is determined based on a click position of the first range-increasing operation;

[0155] Select the first newly added contour with the largest area from all the first newly added contours, merge the spot contour and the first newly added contour with the largest area, and update the spot contour to the merged spot contour;

[0156] When a second range increasing operation corresponding to the spot contour input by the user is detected, the second newly added contour corresponding to the second range increasing operation and the merged spot contour are merged, and the spot contour is updated to the merged spot contour.

[0157] In some possible embodiments, each spot contour is identified and processed to obtain corresponding mask information, including:

[0158] Determine two first contour points on each pattern spot contour along a preset first axis direction, and determine two second contour points on the corresponding pattern spot contour along a preset second axis direction;

[0159] Generate a circumscribed rectangular frame according to two first contour points and two second contour points on each pattern spot contour, and identify vertex coordinates and size parameters of the circumscribed rectangular frame;

[0160] Generate a random point in each circumscribed rectangular frame, and identify the coordinates of the random point when the random point is detected to be within the corresponding pattern outline;

[0161] The vertex coordinates, size parameters and coordinates of random points corresponding to each spot outline are format converted, and the vertex coordinates, size parameters and coordinates of random points after the format conversion are used as corresponding mask information.

[0162] In some possible embodiments, preprocessing each historical remote sensing image in the first image set includes:

[0163] Performing flip processing on at least one historical remote sensing image in the first image set according to a preset flip ratio;

[0164] Scaling all the historical remote sensing images in the first image set according to a preset scaling ratio;

[0165] All historical remote sensing images after scaling are normalized.

[0166] In some possible embodiments, obtaining an image interpretation result according to the target remote sensing image, the first click operation input by the user, and the trained first image segmentation model includes:

[0167] When the first click operation input by the user is a point operation, a mark point of the point operation is generated on the target remote sensing image;

[0168] Inputting the target remote sensing image containing the marked points into the trained first image segmentation model to obtain an image interpretation result; or

[0169] When the first click operation input by the user is a frame operation, a marking frame corresponding to the frame operation is generated on the target remote sensing image; wherein the marking frame includes at least one ground feature area on the target remote sensing image;

[0170] The target remote sensing image containing the marked frame is input into the trained first image segmentation model to obtain the image interpretation result.

[0171] See also Figure 6 , Figure 6 A structural schematic diagram of another remote sensing image interpretation device based on an image segmentation model provided in an embodiment of the present application is shown.

[0172] like Figure 6 As shown, the remote sensing image interpretation device 600 based on the image segmentation model may include at least one processor 601 , at least one network interface 604 , a user interface 603 , a memory 605 and at least one communication bus 602 .

[0173] The communication bus 602 may be used to realize the connection and communication among the above-mentioned components.

[0174] The user interface 603 may include buttons, and the optional user interface may also include a standard wired interface or a wireless interface.

[0175] The network interface 604 may include, but is not limited to, a Bluetooth module, an NFC module, a Wi-Fi module, etc.

[0176] Among them, the processor 601 may include one or more processing cores. The processor 601 uses various interfaces and lines to connect various parts in the remote sensing image interpretation device 600 based on the image segmentation model, and executes various functions and processes data of the remote sensing image interpretation device 600 based on the image segmentation model by running or executing instructions, programs, code sets or instruction sets stored in the memory 605, and calling data stored in the memory 605. Optionally, the processor 601 can be implemented in at least one hardware form of DSP, FPGA, and PLA. The processor 601 can integrate one or a combination of CPU, GPU, modem, etc. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 601, and it can be implemented separately through a chip.

[0177] The memory 605 may include a RAM or a ROM. Optionally, the memory 605 includes a non-transitory computer-readable medium. The memory 605 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 605 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 605 may optionally be at least one storage device located away from the aforementioned processor 601. As Figure 6 As shown, the memory 605 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a remote sensing image interpretation application based on an image segmentation model.

[0178] Specifically, the processor 601 may be used to call the remote sensing image interpretation application based on the image segmentation model stored in the memory 605, and specifically perform the following operations:

[0179] According to the model task requirements, at least two historical remote sensing images collected by the UAV in at least one region are obtained, and each historical remote sensing image is converted to obtain a corresponding mask image; wherein each region corresponds to at least one historical remote sensing image;

[0180] Dividing a spot contour corresponding to each grayscale value from each mask image, and performing recognition processing on each spot contour to obtain corresponding mask information; wherein each mask image includes at least one grayscale value and a spot contour corresponding to each grayscale value;

[0181] All historical remote sensing images are divided into a first image set and a second image set, each historical remote sensing image in the first image set is preprocessed, and a training image set is obtained according to each preprocessed historical remote sensing image and all corresponding mask information;

[0182] According to each historical remote sensing image in the second image set and all corresponding mask information, a verification image set is obtained, and the first image segmentation model is trained based on the training image set and the verification image set;

[0183] The target remote sensing image collected by the UAV is obtained, and the image interpretation result is obtained according to the target remote sensing image, the first click operation input by the user and the trained first image segmentation model.

[0184] The present application also provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the steps of the above method are implemented. The computer-readable storage medium may include, but is not limited to, any type of disk, including a floppy disk, an optical disk, a DVD, a CD-ROM, a micro drive, and a magneto-optical disk, a ROM, a RAM, an EPROM, an EEPROM, a DRAM, a VRAM, a flash memory device, a magnetic card or an optical card, a nanosystem (including a molecular memory IC), or any type of medium or device suitable for storing instructions and / or data.

[0185] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

Claims

1. A remote sensing image interpretation method based on an image segmentation model, characterized in that: include: According to the model task requirements, at least two historical remote sensing images collected by the drone in at least one region are obtained, and each of the historical remote sensing images is converted to obtain a corresponding mask image; wherein each of the regions corresponds to at least one of the historical remote sensing images; Dividing a spot contour corresponding to each grayscale value from each mask image, and performing recognition processing on each spot contour to obtain corresponding mask information; wherein each mask image includes at least one grayscale value and the spot contour corresponding to each grayscale value; Dividing all the historical remote sensing images into a first image set and a second image set, preprocessing each of the historical remote sensing images in the first image set, and obtaining a training image set based on each of the preprocessed historical remote sensing images and all the corresponding mask information; According to each of the historical remote sensing images in the second image set and all the corresponding mask information, a verification image set is obtained, and a first image segmentation model is trained based on the training image set and the verification image set; a target remote sensing image collected by the drone is obtained, and an image interpretation result is obtained according to the target remote sensing image, a first click operation input by a user, and the trained first image segmentation model; The converting process of each of the historical remote sensing images to obtain a corresponding mask image includes: According to the nth ground feature attribute value input by the user, the ground feature category corresponding to the nth ground feature attribute value is queried in a preset ground feature database, and the ground feature area corresponding to the ground feature category is framed in each of the historical remote sensing images; wherein n is a positive integer greater than or equal to 1, and the preset ground feature database includes at least two of the ground feature attribute values ​​and the ground feature category corresponding to each of the ground feature attribute values; When a second click operation input by the user in the feature area is detected, a mark point of the second click operation is generated on the corresponding historical remote sensing image; Inputting the historical remote sensing image containing the marking point into a second image segmentation model to obtain a corresponding spot contour; wherein the second image segmentation model is trained by at least two sample remote sensing images containing sample marking points, a sample spot contour corresponding to each sample marking point, and a marking frame corresponding to each sample spot contour; According to a preset corresponding relationship between ground object category and grayscale value, the image spot outline contained in each ground object area in each of the historical remote sensing images is converted to obtain a mask image of the corresponding historical remote sensing image; wherein the preset corresponding relationship between ground object category and grayscale value includes at least two ground object categories and grayscale values ​​corresponding to each ground object category; Wherein, before converting the image spot contour contained in each of the ground object areas in each of the historical remote sensing images according to the preset ground object category-grayscale value correspondence relationship to obtain the mask image corresponding to the historical remote sensing image, it also includes: When a reduction operation corresponding to the image spot outline input by the user is detected, segmenting the image spot outline to obtain a first segmented outline and a second segmented outline; When the click position of the narrowing operation is within the first segmentation outline, updating the image spot outline to the first segmentation outline; or When the click position of the narrowing-down operation is within the second segmentation outline, the image spot outline is updated to the second segmentation outline.

2. The method according to claim 1, characterized in that Before converting the image spot contour contained in each of the ground object regions in each of the historical remote sensing images according to the preset ground object category-grayscale value correspondence to obtain the mask image corresponding to the historical remote sensing image, the method further includes: When a first range-increasing operation corresponding to the image spot contour input by a user is detected, at least one first newly added contour adjacent to the image spot contour is determined based on a click position of the first range-increasing operation; Selecting the first newly added contour with the largest area from all the first newly added contours, merging the spot contour and the first newly added contour with the largest area, and updating the spot contour to the merged spot contour; When a second range increasing operation corresponding to the spot contour input by the user is detected, the second newly added contour corresponding to the second range increasing operation and the spot contour after the merge processing are merged, and the spot contour is updated to the spot contour after the merge processing.

3. The method according to claim 1, characterized in that The identifying process of each of the pattern spots to obtain corresponding mask information includes: Determine two first contour points on each of the pattern spot contours along a preset first axis direction, and determine two second contour points on the corresponding pattern spot contours along a preset second axis direction; Generate a circumscribed rectangular frame according to two of the first contour points and two of the second contour points on each of the pattern spots, and identify vertex coordinates and size parameters of the circumscribed rectangular frame; Generate a random point in each of the circumscribed rectangular frames, and when it is detected that the random point is within the corresponding pattern spot outline, identify the coordinates of the random point; The vertex coordinates, the size parameters and the coordinates of the random points corresponding to each of the pattern spot contours are format converted, and the vertex coordinates, the size parameters and the coordinates of the random points after the format conversion are used as corresponding mask information.

4. The method according to claim 1, characterized in that: The preprocessing of each of the historical remote sensing images in the first image set comprises: Performing flip processing on at least one of the historical remote sensing images in the first image set according to a preset flip ratio; Scaling all the historical remote sensing images in the first image set according to a preset scaling ratio; All the historical remote sensing images after scaling are normalized.

5. The method according to claim 1, characterized in that The step of obtaining an image interpretation result according to the target remote sensing image, the first click operation input by the user, and the trained first image segmentation model includes: When the first click operation input by the user is a point operation, generating a mark point of the point operation on the target remote sensing image; inputting the target remote sensing image containing the mark point into the trained first image segmentation model to obtain an image interpretation result; or When the first click operation input by the user is a frame operation, a marking frame corresponding to the frame operation is generated on the target remote sensing image; wherein the marking frame includes at least one of the ground feature areas on the target remote sensing image; The target remote sensing image including the marking frame is input into the trained first image segmentation model to obtain an image interpretation result.

6. A remote sensing image interpretation device based on an image segmentation model, characterized in that: include: An image processing module is used to obtain at least two historical remote sensing images collected by the drone in at least one region according to the model task requirements, and convert each of the historical remote sensing images to obtain a corresponding mask image; wherein each of the regions corresponds to at least one of the historical remote sensing images; An information recognition module, used to divide a spot contour corresponding to each grayscale value from each mask image, and perform recognition processing on each spot contour to obtain corresponding mask information; wherein each mask image includes at least one grayscale value and the spot contour corresponding to each grayscale value; A set generation module, used for dividing all the historical remote sensing images into a first image set and a second image set, preprocessing each of the historical remote sensing images in the first image set, and obtaining a training image set according to each of the preprocessed historical remote sensing images and all the corresponding mask information; A model training module, configured to obtain a verification image set according to each of the historical remote sensing images in the second image set and all the corresponding mask information, and to train a first image segmentation model based on the training image set and the verification image set; An image interpretation module, used to obtain the target remote sensing image collected by the drone, and obtain an image interpretation result according to the target remote sensing image, the first click operation input by the user, and the trained first image segmentation model; The converting process of each of the historical remote sensing images to obtain a corresponding mask image includes: According to the nth ground feature attribute value input by the user, the ground feature category corresponding to the nth ground feature attribute value is queried in a preset ground feature database, and the ground feature area corresponding to the ground feature category is framed in each of the historical remote sensing images; wherein n is a positive integer greater than or equal to 1, and the preset ground feature database includes at least two of the ground feature attribute values ​​and the ground feature category corresponding to each of the ground feature attribute values; When a second click operation input by the user in the feature area is detected, a mark point of the second click operation is generated on the corresponding historical remote sensing image; Inputting the historical remote sensing image containing the marking point into a second image segmentation model to obtain a corresponding spot contour; wherein the second image segmentation model is trained by at least two sample remote sensing images containing sample marking points, a sample spot contour corresponding to each sample marking point, and a marking frame corresponding to each sample spot contour; According to a preset corresponding relationship between ground object category and grayscale value, the image spot outline contained in each ground object area in each of the historical remote sensing images is converted to obtain a mask image of the corresponding historical remote sensing image; wherein the preset corresponding relationship between ground object category and grayscale value includes at least two ground object categories and grayscale values ​​corresponding to each ground object category; Wherein, before converting the image spot contour contained in each of the ground object areas in each of the historical remote sensing images according to the preset ground object category-grayscale value correspondence relationship to obtain the mask image corresponding to the historical remote sensing image, it also includes: When a reduction operation corresponding to the image spot outline input by the user is detected, segmenting the image spot outline to obtain a first segmented outline and a second segmented outline; When the click position of the narrowing operation is within the first segmentation outline, updating the image spot outline to the first segmentation outline; or When the click position of the narrowing-down operation is within the second segmentation outline, the image spot outline is updated to the second segmentation outline.

7. A remote sensing image interpretation device based on an image segmentation model, characterized in that: including a processor and a memory; The processor is connected to the memory; The memory is used to store executable program code; The processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to execute the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed on a computer or a processor, the computer or the processor executes the steps of the method according to any one of claims 1 to 5.