Image segmentation method, model training method, related device and electronic equipment

Abstract

The present invention provides an image segmentation method, a model training method, related devices, and electronic equipment. The method includes: acquiring L first remote sensing images; performing image segmentation on the L first remote sensing images based on a target model to obtain target land segmentation results; the target model is trained based on N second target training sample sets and a training sample set corresponding to the N first target training sample sets, the training sample set including M remote sensing image samples; a second target training sample set is obtained by foreground labeling of remote sensing image samples in the first target training sample set based on a target category corresponding to the first target training sample set; the N first target training sample sets are obtained by dividing the M remote sensing image samples based on N target categories corresponding to the training sample sets, wherein the proportion of image content of the target category in the M remote sensing image samples is less than a preset threshold. The present invention can improve the accuracy of image segmentation of remote sensing images using a neural network model.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of image processing technology, and in particular to an image segmentation method, a model training method, related devices and electronic equipment. Background Art

[0002] With the rapid development of image processing technology, image segmentation technology has been widely used. Image segmentation technology refers to the process of dividing an image into several regions with similar properties. In the field of remote sensing image processing, remote sensing image segmentation is a very important step in the process of map data generation.

[0003] Currently, neural network models are commonly used to segment remote sensing images. However, due to the uneven distribution of content in remote sensing images, some categories of image content, such as roads and lakes, account for a relatively small proportion of the image. Consequently, these categories contribute little to the iterative update of the neural network parameters during training. Consequently, the accuracy of remote sensing image segmentation using neural network models is relatively low. Summary of the Invention

[0004] The embodiments of the present invention provide an image segmentation method, a model training method, related devices and electronic equipment to solve the problem of low accuracy when using a neural network model to segment remote sensing images in the prior art.

[0005] In a first aspect, an embodiment of the present invention provides an image segmentation method, the method comprising:

[0006] Acquire L first remote sensing images, where L is a positive integer;

[0007] Perform image segmentation on the L first remote sensing images based on the target model to obtain a target land segmentation result;

[0008] In which, the target model is trained based on N second target training sample sets and training sample sets corresponding to N first target training sample sets, the training sample sets include M remote sensing image samples, a second target training sample set is based on a target category corresponding to a first target training sample set, and the remote sensing image samples in the first target training sample set are marked with foreground annotations, the N first target training sample sets are divided into the M remote sensing image samples based on the N target categories corresponding to the training sample sets, the proportion of image content of the target category in the M remote sensing image samples is less than a preset threshold, M is an integer greater than 1, and N is a positive integer.

[0009] In a second aspect, an embodiment of the present invention provides a model training method, the method comprising:

[0010] Obtain N target categories corresponding to a training sample set, the training sample set including M remote sensing image samples, the proportion of image content of the target category in the M remote sensing image samples being less than a preset threshold, where M is an integer greater than 1 and N is a positive integer;

[0011] Dividing the M remote sensing image samples based on the N target categories to obtain N first target training sample sets, where one first target training sample set corresponds to one target category;

[0012] For each first target training sample set, based on the target category corresponding to the first target training sample set, performing foreground labeling on the remote sensing image samples in the first target training sample set to generate a second target training sample set corresponding to the first target training sample set;

[0013] The target model is trained based on the N second target training sample sets corresponding to the N first target training sample sets and the training sample set.

[0014] In a third aspect, an embodiment of the present invention provides an image segmentation device, the device comprising:

[0015] A first acquisition module is used to acquire L first remote sensing images, where L is a positive integer;

[0016] An image segmentation module is used to perform image segmentation on the L first remote sensing images based on the target model to obtain a target land segmentation result;

[0017] In which, the target model is trained based on N second target training sample sets and training sample sets corresponding to N first target training sample sets, the training sample sets include M remote sensing image samples, a second target training sample set is based on a target category corresponding to a first target training sample set, and the remote sensing image samples in the first target training sample set are marked with foreground annotations, the N first target training sample sets are divided into the M remote sensing image samples based on the N target categories corresponding to the training sample sets, the proportion of image content of the target category in the M remote sensing image samples is less than a preset threshold, M is an integer greater than 1, and N is a positive integer.

[0018] In a fourth aspect, an embodiment of the present invention provides a model training device, comprising:

[0019] A second acquisition module is configured to acquire N target categories corresponding to a training sample set, wherein the training sample set includes M remote sensing image samples, and the proportion of image content of the target category in the M remote sensing image samples is less than a preset threshold, where M is an integer greater than 1 and N is a positive integer;

[0020] a division module, configured to divide the M remote sensing image samples based on the N target categories to obtain N first target training sample sets, where one first target training sample set corresponds to one target category;

[0021] a labeling module, configured to, for each first target training sample set, perform foreground labeling on remote sensing image samples in the first target training sample set based on the target category corresponding to the first target training sample set, so as to generate a second target training sample set corresponding to the first target training sample set;

[0022] A training module is used to train a target model based on the N second target training sample sets corresponding to the N first target training sample sets and the training sample set.

[0023] In a fifth aspect, an embodiment of the present invention provides an electronic device comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the steps of the above-mentioned image segmentation method or the steps of the above-mentioned model training method are implemented.

[0024] In a sixth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned image segmentation method or the steps of the above-mentioned model training method are implemented.

[0025] In an embodiment of the present invention, L first remote sensing images are acquired; image segmentation is performed on the L first remote sensing images based on a target model to obtain a target plot segmentation result; the target model is trained based on N second target training sample sets and a training sample set corresponding to the N first target training sample sets, the training sample sets including M remote sensing image samples, a second target training sample set obtained by foreground labeling of remote sensing image samples in the first target training sample set based on a target category corresponding to a first target training sample set, the N first target training sample sets obtained by dividing the M remote sensing image samples based on N target categories corresponding to the training sample sets, the proportion of image content of the target category in the M remote sensing image samples being less than a preset threshold. In this way, when training the target model, by dividing the training data based on target categories with a relatively small proportion of image content in the training data and relabeling based on the target category, the problem of uneven distribution of the training data can be fundamentally resolved, the segmentation accuracy of the target model for target categories with a relatively small proportion can be improved, and the accuracy of image segmentation of remote sensing images based on the target model can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in describing the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0027] Figure 1 1 is a flow chart of an image segmentation method provided by an embodiment of the present invention;

[0028] Figure 2 Schematic diagram of the model training method provided by the embodiment of the present invention;

[0029] Figure 3 Schematic diagram of the structure of an image segmentation device provided by an embodiment of the present invention;

[0030] Figure 4 Schematic diagram of the structure of the model training device provided by an embodiment of the present invention;

[0031] Figure 5 It is a structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments derived by ordinary technicians in this field based on the embodiments of the present invention without creative work are within the scope of protection of the present invention.

[0033] The following first describes the image segmentation method provided by the embodiment of the present invention.

[0034] It should be noted that the image segmentation method provided in the embodiments of the present invention relates to artificial intelligence technologies such as computer vision and deep learning. It can be widely applied in the field of geographic information, for example, by segmenting remote sensing images to generate map data. This method can be performed by the image segmentation device of the embodiments of the present invention. The image segmentation device can be configured in any electronic device to perform the image segmentation method, and the electronic device can be a server or a terminal, without specific limitation.

[0035] See also Figure 1 , the figure shows a flow chart of the image segmentation method provided by an embodiment of the present invention.

[0036] like Figure 1 As shown, the method may include the following steps:

[0037] Step 101: Acquire L first remote sensing images.

[0038] Wherein, L is a positive integer.

[0039] Here, remote sensing images generally refer to remote sensing images, which are films or photographs that record the electromagnetic wave size of various ground objects. They are mainly divided into aerial photographs and satellite photographs. In other words, remote sensing images generally include image content of ground objects on the Earth's surface, such as rivers, forests, roads, buildings, mountains, etc. The purpose of the embodiments of this application is to detect the image content representing the ground objects in remote sensing images, segment the image content in the remote sensing images, and obtain the land segmentation results of each feature.

[0040] L can be 1 or greater than 1, and is not specifically limited here.

[0041] When L is equal to 1, the first remote sensing image can be obtained in a variety of ways. For example, a remote sensing image can be acquired by satellite as the first remote sensing image, a pre-stored remote sensing image can be acquired as the first remote sensing image, a remote sensing image sent by another electronic device can be acquired as the first remote sensing image, a remote sensing image can be downloaded from the Internet as the first remote sensing image, or a second remote sensing image can be acquired and image enhancement processing performed on the second remote sensing image to obtain the first remote sensing image. The method for acquiring the second remote sensing image is similar to that for acquiring the first remote sensing image and will not be described in detail here.

[0042] When L is greater than 1, the L first remote sensing images are usually remote sensing images of the same scene, that is, the L first remote sensing images usually include the same image content, so as to improve the accuracy of ground object segmentation of the remote sensing image in the scene.

[0043] There are also multiple ways to acquire the L first remote sensing images. For example, multiple remote sensing images of the same scene can be collected by satellite as the L first remote sensing images, or a second remote sensing image can be acquired (the second remote sensing image is a remote sensing image that requires image segmentation), and the second remote sensing image can be subjected to image enhancement processing such as proportional scaling, random flipping, random selection, random color jittering, random exposure, and random cropping to generate L first remote sensing images.

[0044] Step 102: perform image segmentation on the L first remote sensing images based on the target model to obtain a target plot segmentation result; the target model is trained based on N second target training sample sets and a training sample set corresponding to the N first target training sample sets, the training sample set including M remote sensing image samples, a second target training sample set is obtained by performing foreground annotation on the remote sensing image samples in the first target training sample set based on a target category corresponding to a first target training sample set, the N first target training sample sets are obtained by dividing the M remote sensing image samples based on N target categories corresponding to the training sample sets, and the proportion of image content of the target category in the M remote sensing image samples is less than a preset threshold.

[0045] Wherein, M is an integer greater than 1, and N is a positive integer.

[0046] In this step, the target model can be a neural network model, which can be a single model, such as the semantic segmentation model OCRNet whose backbone network is HRNetW48. This model can be a multi-classification model to segment the land parcels of various categories of objects in the remote sensing image.

[0047] The target model may also include multiple sub-models, for example, including N first sub-models and second sub-models, and the number of second sub-models is one or more, which is not specifically limited here. Among them, the first sub-model can be a binary classification model, such as the semantic segmentation model of the network such as UNet, FCN, etc. can be used as the first sub-model to segment the land parcel segmentation results of the target category of the land objects in the remote sensing image, and the second sub-model can be a multi-classification model, such as the semantic segmentation model OCRNet with the backbone network of HRNetW48 can be used as the second sub-model to segment the land parcel segmentation results of each category of land objects in the remote sensing image.

[0048] The target model can be obtained by training based on N second target training sample sets and the training sample sets corresponding to the N first target training sample sets. The following will introduce in detail the process of obtaining the N second target training sample sets.

[0049] Specifically, a training sample set can be obtained. The training sample set can include M remote sensing image samples. The acquisition method of each remote sensing image sample is similar to the acquisition method of the first remote sensing image, and is not further described here. The training sample set can also include land parcel labels for each feature in each remote sensing image sample. The training sample set can include only land parcel category labels or both land parcel category labels and land parcel edge contour labels.

[0050] The M remote sensing image samples can include multiple feature categories. For example, they can include 10 categories, including rivers, forests, roads, buildings, mountains, etc. Ideally, each category should account for 10%. However, due to the different proportions of features on the Earth's surface, the data share of each category can vary greatly. For example, the data share of forests and roads can differ by more than 100 times. Therefore, it is necessary to obtain data from categories with a smaller share. This category is called the target category, and there can be one or more target categories.

[0051] In a specific implementation, each pixel in the M remote sensing image samples can be traversed to calculate the proportion of each category of ground feature data in the training data. For example, ocean data accounts for 10% of the training data, forest data accounts for 11%, road data accounts for 1%, and building data accounts for 2%. Classes whose proportion in the training data is less than a preset threshold can be used as target categories. The preset threshold can be set according to actual conditions, such as 5%. In this case, the target categories include roads and buildings.

[0052] Afterwards, the land parcel category labels of the M remote sensing image samples may be traversed. If the land parcel category label of the remote sensing image sample includes a target category, the remote sensing image may be classified into the first target training sample set corresponding to the target category.

[0053] For example, the training sample set includes remote sensing image sample A, remote sensing image sample B and remote sensing image sample C. Remote sensing image sample A includes road data and mountain data, remote sensing image sample B includes road data, building data and ocean data, remote sensing image sample C includes ocean data and forest data, and the target categories include roads and buildings.

[0054] At this point, two new training sample sets can be divided, namely two first target training sample sets, one of which corresponds to roads and the other corresponds to buildings. The land category labels of the remote sensing image samples in the original training sample set can be traversed. Since remote sensing image sample A and remote sensing image sample B both include road data, remote sensing image sample A and remote sensing image sample B can be divided into the first target training sample set corresponding to roads, which can be called the first target training sample set D. Since remote sensing image sample B includes building data, remote sensing image sample B can be divided into the first target training sample set corresponding to buildings, which can be called the first target training sample set E.

[0055] Based on the target categories corresponding to the first target training sample set, the remote sensing image samples in the first target training sample set are respectively labeled with foreground labels to obtain a second target training sample set. Specifically, the image content of the target category in the remote sensing image samples of the first target training sample set can be labeled with a value of 1, which represents the foreground, and other image contents can be labeled with a value of 0, which represents the background.

[0056] For example, for the first target training sample set D, the road data in remote sensing image sample A and remote sensing image sample B are labeled as 1, and the data other than road data (including mountain data, building data, and ocean data) are labeled as 0. For the first target training sample set E, the building data in remote sensing image sample B is labeled as 1, and the data other than building data (including road data and ocean data) are labeled as 0.

[0057] In this way, new labeled data can be regenerated for target categories with a relatively small proportion, so that the remote sensing image samples can be divided into a task with only two categories, increasing the proportion of labeled data of the target category in the entire training sample set and solving the problem of uneven data distribution in the training sample set.

[0058] In addition, based on the target category corresponding to the first target training sample set, the remote sensing image samples in the first target training sample set are annotated with foreground annotation, and the second target training sample set can be directly obtained. Alternatively, after foreground annotation, sliding windows of various sizes can be used to traverse the image content of the foreground-annotated remote sensing image samples, and the window containing the image content of the target category can be cropped out. Then, image enhancement processing such as scaling and flipping can be performed to generate new remote sensing image samples. At the same time, the label data of the foreground-annotated remote sensing image samples can be processed in the same way, and finally N second target training sample sets corresponding to N target categories are generated, with one second target training sample set corresponding to one target category. In this way, the proportion of the image content of the target category in the remote sensing image samples can be further increased, the training effect of the target model can be improved, and the image segmentation accuracy of the target model for remote sensing images can be further improved.

[0059] These N second target training samples and the training sample set ultimately constitute the entire training sample set for the target model. Based on this entire training sample set, a target model can be trained. Because the new training sample set increases the proportion of target category training data in the overall training data, this improves the target model's ability to extract features from target category data, thereby improving the target model's training effectiveness. Therefore, while maintaining the accuracy of other ground feature recognition categories, the recognition accuracy of the target category with a smaller proportion can be improved, thereby improving the recognition accuracy of the target model across the entire training data.

[0060] In this embodiment, L first remote sensing images are acquired; image segmentation is performed on the L first remote sensing images based on a target model to obtain a target plot segmentation result; the target model is trained based on N second target training sample sets and a training sample set corresponding to the N first target training sample sets, the training sample set including M remote sensing image samples, a second target training sample set is obtained by performing foreground labeling on remote sensing image samples in the first target training sample set based on a target category corresponding to a first target training sample set, the N first target training sample sets are obtained by dividing the M remote sensing image samples based on N target categories corresponding to the training sample sets, and the proportion of image content of the target category in the M remote sensing image samples is less than a preset threshold. In this way, when training the target model, by dividing the training data based on target categories with a relatively small proportion of image content in the training data and relabeling based on the target category, the problem of uneven distribution of the training data can be fundamentally solved, the segmentation accuracy of the target model on target categories with a relatively small proportion can be improved, and the accuracy of image segmentation of remote sensing images based on the target model can be improved.

[0061] Optionally, the target model includes N first sub-models and second sub-models, the N first sub-models are trained based on the N second target training sample sets, one first sub-model is trained based on one second target training sample set, and the second sub-model is trained based on the training sample set, and the target land segmentation result is obtained by performing image segmentation on the L first remote sensing images based on the target model, including:

[0062] Based on each first sub-model, the L first remote sensing images are segmented to obtain N first land parcel segmentation results;

[0063] Performing image segmentation on the L first remote sensing images based on the second sub-model to obtain a second land parcel segmentation result;

[0064] The N first land parcel segmentation results and the second land parcel segmentation result are fused to obtain the target land parcel segmentation result.

[0065] In this embodiment, a model fusion technology may be used to fuse the land parcel segmentation results of multiple sub-models.

[0066] Specifically, the target model may include multiple sub-models, including N first sub-models and second sub-models, that is, at least N+1 sub-models. The following description will be made by taking the example that the target model includes N+1 sub-models.

[0067] The first sub-model is a binary classification model, whose training data is a second target training sample set. After training, the first sub-model can identify the ground object data of the target category corresponding to the second target training sample set in the remote sensing image. The second sub-model is a multi-classification model, whose training data is the original training sample set. After training, the second sub-model can identify the ground object data of each category corresponding to the original training sample set in the remote sensing image.

[0068] Image segmentation can be performed on the L first remote sensing images using an existing or new image segmentation method based on each first sub-model to obtain N first land parcel segmentation results. For example, the target model includes a first sub-model F and a first sub-model G. Based on the first sub-model F, image segmentation is performed on the L first remote sensing images to obtain a first land parcel segmentation result P. Based on the first sub-model G, image segmentation is performed on the L first remote sensing images to obtain a first land parcel segmentation result Q.

[0069] When L is greater than 1, multiple land parcel segmentation results can be obtained based on a first sub-model. At this time, these multiple land parcel segmentation results can be fused to finally obtain a first land parcel segmentation result. The specific fusion method will be described in detail in the following implementation method.

[0070] Based on the second sub-model, an existing or new image segmentation method can be used to perform image segmentation on the L first remote sensing images to obtain a second land parcel segmentation result.

[0071] The N+1 sub-models can finally obtain N+1 land segmentation results, and the land segmentation result is a matrix with a length and width consistent with the first remote sensing image. At the same time, the third channel of the land segmentation result corresponds to the number of categories. The land segmentation result specifically includes the recognition probability of each pixel point in the first remote sensing image belonging to each category (including land object category and background).

[0072] Since the second target training sample set is binary data, the length of the third channel of the land parcel segmentation results output by the N first sub-models is 2, while the original training sample set is multi-class data, and the length of the third channel of the land parcel segmentation results output by the second sub-model is greater than 2, for example, 10. Therefore, it is necessary to expand the first land parcel segmentation results output by the N models into a land parcel segmentation result that matches the size of the second land parcel segmentation result output by the second sub-model.

[0073] Specifically, based on the position of the target category in the second plot segmentation result, the data corresponding to the target category in the first plot segmentation result can be placed in the corresponding position. For example, the target category corresponding to the first plot segmentation result is road, and its foreground in the second target training sample set is marked as 1. The data of its recognition probability belonging to this target category is located in the first column of the first plot segmentation result, while the foreground of road in the original training sample set is marked as 6, and its recognition probability data belonging to this target category is located in the sixth column of the second plot segmentation result. At this time, the first plot segmentation result can be expanded to make its size consistent with the second plot segmentation result, and the data in the first column can be placed in the sixth column, and the other expanded new data can be filled with 0.

[0074] Afterwards, the land segmentation results output by the N+1 sub-models are fused by voting. Specifically, based on the land segmentation results output by the N+1 sub-models, for each pixel point in the first remote sensing image, the average value of the recognition probability of the pixel point belonging to each category is calculated (which can be the arithmetic mean or geometric mean), and the maximum value voting is adopted to select the category with the largest average value of the recognition probability as the output result of the pixel point. The model fusion formula is shown in the following formula (1).

[0075] pre=max(out1,out2...out n ...out N+1 ) (1)

[0076] Among them, pre is the fusion result of the land segmentation results output by multiple sub-models, that is, the target land segmentation result, and out n The land parcel segmentation result output by the nth sub-model.

[0077] For example, for a pixel point in the first remote sensing image, in the first plot segmentation result P, the recognition probability of the pixel point belonging to the road is 0.9, and the recognition probability of belonging to the background is 0.1. In the first plot segmentation result Q, the recognition probability of the pixel point belonging to the building is 0.2, and the recognition probability of belonging to the background is 0.8. In the second plot segmentation result, the recognition probability of the pixel point belonging to the road is 0.6, the recognition probability of belonging to the building is 0.1, the recognition probability of belonging to the forest is 0.05, the recognition probability of belonging to the ocean is 0.05, and the recognition probability of belonging to the background is 0.05. At this time, after integrating the three plot segmentation results, the average value of the recognition probability of belonging to the road is 0.5, and the calculation formula can be (0.9+0.6) / 3. The average value of the recognition probability of belonging to the building is 0.1, and the average value of the recognition probability of belonging to the background is about 0.3. Therefore, the prediction result of the pixel point is that it belongs to the road.

[0078] In this implementation, sub-models with different structures are trained on different training data. These sub-models are then used to segment the first remote sensing image, yielding multiple land parcel segmentation results. During the prediction phase, multi-model fusion techniques are used to fuse the predictions from these sub-models using a voting method to produce the final prediction. This allows for the extraction of more features from a remote sensing image of the same scene using multiple sub-models for image segmentation. By fusing the land parcel segmentation results through voting, the segmentation accuracy of the remote sensing image can be further improved.

[0079] In addition, the network structures and algorithms used by these multiple sub-models can be quite different in order to extract different features of remote sensing images, extract as many features as possible, and improve the performance of tasks such as ground object recognition, target detection, and change detection in remote sensing images.

[0080] Optionally, when L is greater than 1, the L first remote sensing images are obtained by performing image enhancement processing based on the second remote sensing image, and the image segmentation is performed on the L first remote sensing images based on each first sub-model to obtain N first land parcel segmentation results, including:

[0081] For each first sub-model, performing image segmentation on the L first remote sensing images based on the first sub-model to obtain L third land parcel segmentation results corresponding to the first sub-model;

[0082] The L third land parcel segmentation results corresponding to the first sub-model are fused to obtain a first land parcel segmentation result corresponding to the first sub-model.

[0083] In this embodiment, a second remote sensing image can be obtained (the second remote sensing image is a remote sensing image that needs to be segmented), and the second remote sensing image can be subjected to image enhancement processing such as proportional scaling, random flipping, random selection, random color jittering, random exposure, and random cropping to generate L first remote sensing images.

[0084] For each first sub-model, an existing or new image segmentation method is used to perform image segmentation on the L first remote sensing images respectively to obtain L third land parcel segmentation results corresponding to the first sub-model.

[0085] The L third plot segmentation results can be fused in a manner similar to the fusion of the N+1 plot segmentation results output by the N+1 sub-models, and finally the prediction result of the single first sub-model, i.e., the first plot segmentation result, is obtained.

[0086] In this embodiment, by performing image enhancement processing on the input data of each first sub-model, multiple first remote sensing images are obtained, and image segmentation is performed on these multiple first remote sensing images based on the first sub-model, so that a single first sub-model can extract more features of the remote sensing image when segmenting the remote sensing image, and finally multiple prediction results of the single first sub-model are obtained. The multiple prediction results of the single first sub-model are first fused and voted to obtain the prediction result of the single first sub-model. After that, the prediction results of multiple sub-models (including N first sub-models) can be fused for a second time, and the prediction results output by these multiple sub-models can be fused by voting as the final segmentation result, which can greatly improve the segmentation accuracy of the remote sensing image.

[0087] The following describes the model training method provided by an embodiment of the present invention.

[0088] See also Figure 2 , the figure shows a flow chart of the model training method provided by an embodiment of the present invention.

[0089] like Figure 2 As shown, the method may include the following steps:

[0090] Step 201: Obtain N target categories corresponding to a training sample set, wherein the training sample set includes M remote sensing image samples, and the proportion of image content of the target category in the M remote sensing image samples is less than a preset threshold, where M is an integer greater than 1 and N is a positive integer;

[0091] Step 202: dividing the M remote sensing image samples based on the N target categories to obtain N first target training sample sets, where one first target training sample set corresponds to one target category.

[0092] Step 203: for each first target training sample set, based on the target category corresponding to the first target training sample set, perform foreground labeling on the remote sensing image samples in the first target training sample set to generate a second target training sample set corresponding to the first target training sample set;

[0093] Step 204 : Training the target model based on the N second target training sample sets corresponding to the N first target training sample sets and the training sample set.

[0094] This embodiment describes the training process of the target model, wherein the quality of the training data of the target model can directly affect the training effect of the target model, thereby indirectly affecting the segmentation accuracy of the target model for remote sensing images.

[0095] In reality, the distribution of objects on the earth's surface is uneven, which will lead to an uneven distribution of data in the collected original training sample set. In this case, this embodiment can optimize the training data of the target model to increase the proportion of object data that accounts for a relatively small proportion in the training data, so as to achieve the purpose of balancing the data of each category in the training data.

[0096] Specifically, a training sample set can be obtained. The training sample set can include M remote sensing image samples. The acquisition method of each remote sensing image sample is similar to the acquisition method of the first remote sensing image, and is not further described here. The training sample set can also include land parcel labels for each feature in each remote sensing image sample. The training sample set can include only land parcel category labels or both land parcel category labels and land parcel edge contour labels.

[0097] The M remote sensing image samples can include multiple feature categories. For example, they can include 10 categories, including rivers, forests, roads, buildings, mountains, etc. Ideally, each category should account for 10%. However, due to the different proportions of features on the Earth's surface, the data share of each category can vary greatly. For example, the data share of forests and roads can differ by more than 100 times. Therefore, it is necessary to obtain data from categories with a smaller share. This category is called the target category, and there can be one or more target categories.

[0098] In a specific implementation, each pixel in the M remote sensing image samples can be traversed to calculate the proportion of each category of ground feature data in the training data. For example, ocean data accounts for 10% of the training data, forest data accounts for 11%, road data accounts for 1%, and building data accounts for 2%. Classes whose proportion in the training data is less than a preset threshold can be used as target categories. The preset threshold can be set according to actual conditions, such as 5%. In this case, the target categories include roads and buildings.

[0099] Afterwards, the land parcel category labels of the M remote sensing image samples may be traversed. If the land parcel category label of the remote sensing image sample includes a target category, the remote sensing image may be classified into the first target training sample set corresponding to the target category.

[0100] For example, the training sample set includes remote sensing image sample A, remote sensing image sample B and remote sensing image sample C. Remote sensing image sample A includes road data and mountain data, remote sensing image sample B includes road data, building data and ocean data, remote sensing image sample C includes ocean data and forest data, and the target categories include roads and buildings.

[0101] At this point, two new training sample sets can be divided, namely two first target training sample sets, one of which corresponds to roads and the other corresponds to buildings. The land category labels of the remote sensing image samples in the original training sample set can be traversed. Since remote sensing image sample A and remote sensing image sample B both include road data, remote sensing image sample A and remote sensing image sample B can be divided into the first target training sample set corresponding to roads, which can be called the first target training sample set D. Since remote sensing image sample B includes building data, remote sensing image sample B can be divided into the first target training sample set corresponding to buildings, which can be called the first target training sample set E.

[0102] Based on the target categories corresponding to the first target training sample set, the remote sensing image samples in the first target training sample set are respectively labeled with foreground labels to obtain a second target training sample set. Specifically, the image content of the target category in the remote sensing image samples of the first target training sample set can be labeled with a value of 1, which represents the foreground, and other image contents can be labeled with a value of 0, which represents the background.

[0103] For example, for the first target training sample set D, the road data in remote sensing image sample A and remote sensing image sample B are labeled as 1, and the data other than road data (including mountain data, building data, and ocean data) are labeled as 0. For the first target training sample set E, the building data in remote sensing image sample B is labeled as 1, and the data other than building data (including road data and ocean data) are labeled as 0.

[0104] In this way, new labeled data can be regenerated for target categories with a relatively small proportion, so that the remote sensing image samples can be divided into a task with only two categories, increasing the proportion of labeled data of the target category in the entire training sample set and solving the problem of uneven data distribution in the training sample set.

[0105] In addition, based on the target category corresponding to the first target training sample set, the remote sensing image samples in the first target training sample set are annotated with foreground annotation, and the second target training sample set can be directly obtained. Alternatively, after foreground annotation, sliding windows of various sizes can be used to traverse the image content of the foreground-annotated remote sensing image samples, and the window containing the image content of the target category can be cropped out. Then, image enhancement processing such as scaling and flipping can be performed to generate new remote sensing image samples. At the same time, the label data of the foreground-annotated remote sensing image samples can be processed in the same way, and finally N second target training sample sets corresponding to N target categories are generated, with one second target training sample set corresponding to one target category. In this way, the proportion of the image content of the target category in the remote sensing image samples can be further increased, the training effect of the target model can be improved, and the image segmentation accuracy of the target model for remote sensing images can be further improved.

[0106] These N second target training samples and the training sample set ultimately constitute the entire training sample set for the target model. Based on this entire training sample set, a target model can be trained. Because the new training sample set increases the proportion of target category training data in the overall training data, this improves the target model's ability to extract features from target category data, thereby improving the target model's training effectiveness. Therefore, while maintaining the accuracy of other ground feature recognition categories, the recognition accuracy of the target category with a smaller proportion can be improved, thereby improving the recognition accuracy of the target model across the entire training data.

[0107] Optionally, the target model includes N first sub-models and a second sub-model, and the training of the target model based on the N second target training sample sets and the training sample set includes:

[0108] Training the N first sub-models based on the N second target training sample sets, where one first sub-model is trained based on one second target training sample set;

[0109] The second sub-model is trained based on the training sample set.

[0110] In this embodiment, the target model may include multiple sub-models, including N first sub-models and second sub-models, that is, at least N+1 sub-models. The following description takes the target model including N+1 sub-models as an example.

[0111] The first sub-model is a binary classification model, whose training data is a second target training sample set. The trained first sub-model can identify the ground object data of the target category corresponding to the second target training sample set in the remote sensing image. The first sub-model can use a semantic segmentation model such as a UNet or FCN network.

[0112] The second sub-model is a multi-classification model, trained on the original training sample set. The trained second sub-model can identify the various categories of ground features in the remote sensing image corresponding to the original training sample set. The second sub-model can use the semantic segmentation model OCRNet with the HRNetW48 backbone network.

[0113] During the specific training process, the training sample set of each sub-model can be divided into a training set, a validation set, and a test set, so that the sub-model can be trained based on the training set, the sub-model can be validated based on the validation set, and the sub-model can be tested based on the test set.

[0114] In this embodiment, the target model is divided into multiple sub-models, and the N first sub-models are trained based on the N second target training sample sets. Each first sub-model is trained based on a second target training sample set; and each second sub-model is trained based on the training sample set. In this way, when performing image segmentation based on multiple sub-models for the same scene remote sensing image, more features of the remote sensing image can be extracted. Fusion of the land parcel segmentation results from these multiple sub-models can improve the segmentation accuracy of the remote sensing image.

[0115] Optionally, the training sample set includes a land parcel category label and a land parcel edge contour label corresponding to a first remote sensing image sample, the first remote sensing image sample is any remote sensing image sample among the M remote sensing image samples, and the training of the second sub-model based on the training sample set includes:

[0116] Inputting the first remote sensing image sample into the second sub-model for image segmentation to obtain land parcel category information and land parcel edge contour information corresponding to the first remote sensing image sample;

[0117] Determining first difference information between the land parcel category information and the land parcel category label, and determining second difference information between the land parcel edge contour information and the land parcel edge contour label;

[0118] Based on the first difference information and the second difference information, network parameters of the second sub-model are updated.

[0119] In this embodiment, the second sub-model can select the semantic segmentation model OCRNet with the backbone network HRNetW48, and the first remote sensing image sample can be input into the second sub-model for image segmentation. The top layer of the second sub-model can include two network branches, one network branch is output as the result of image segmentation to obtain the plot category information, and the other is output as the result of the plot edge to obtain the plot edge contour information.

[0120] An existing or new comparison algorithm can be used to determine the first difference information between the parcel category information and the parcel category label to obtain the loss between the predicted parcel category information and the actual parcel category label. For example, the parcel segmentation loss function shown in the following formula (2) can be used to determine the first difference information.

[0121]

[0122] Among them, J CE is the first difference information, c i is the land parcel category of the i-th pixel, For c i The predicted probability under the plot category, For c i The parcel category label under the parcel category, V1 is the number of pixels in the remote sensing image sample in the original training sample set.

[0123] An existing or new comparison algorithm can be used to determine second difference information between the land parcel edge contour information and the land parcel edge contour label to obtain a loss between the predicted land parcel edge contour information and the actual land parcel edge contour label. This loss can be used as an auxiliary loss to add constraints to the land parcel edge.

[0124] Since the difference between the amount of edge information and background information is relatively large, the second difference information can be determined by using the land edge loss function shown in the following formula (3).

[0125] FL(pt)=-(1-pt) γ log(pt) (3)

[0126] Wherein, FL(pt) is the second difference information, pt can be the probability of the edge contour of the land parcel, and γ is a preset parameter, which can be a constant.

[0127] Afterwards, the first difference information and the second difference information can be superimposed to obtain the network loss of the second sub-model. Based on the network loss, the network parameters of the second sub-model are updated using methods such as gradient descent until the network loss of the second sub-model converges and is less than the set threshold. At this time, the training of the second sub-model is completed.

[0128] In addition, the training process of the first sub-model can be similar to that of the second sub-model, except that the first sub-model is a binary classification model, and networks such as UNet and FCN can be selected as semantic segmentation models. The land parcel segmentation loss function of the binary classification model can be shown as follows (4).

[0129]

[0130] Among them, J CM is the loss of the first sub-model for the land parcel category, y i is the land parcel category label of the i-th pixel, p i is the predicted probability, and V2 is the number of pixels in the remote sensing image sample in the second target training sample set.

[0131] In this embodiment, when training the sub-model, while determining the loss between the predicted plot category information and the actual plot category label, the loss between the predicted plot edge contour information and the actual plot edge contour label can be determined as an auxiliary loss to add constraints on the plot edge. This can improve the training effect of the target model, thereby reducing the misrecognition rate of the target model, and further improving the recognition accuracy of the target model and improving the performance of the target model.

[0132] Optionally, the performing foreground labeling on remote sensing image samples in the first target training sample set based on the target category corresponding to the first target training sample set to generate a second target training sample set corresponding to the first target training sample set includes:

[0133] Based on the target category corresponding to the first target training sample set, performing foreground labeling on the remote sensing image samples in the first target training sample set to obtain a third target training sample set;

[0134] The remote sensing image samples in the third target training sample set are cropped to obtain a second target training sample set corresponding to the first target training sample set, wherein the remote sensing image samples in the second target training sample set include image content of a target area, and the target area is an area in the remote sensing image samples of the third target training sample set that includes image content of the target category.

[0135] In this embodiment, for the remote sensing image samples after foreground annotation, sliding windows of various sizes can be used to traverse the image content, and the window containing the image content of the target category can be cropped out. Then, image enhancement processing such as scaling and flipping can be performed to generate new remote sensing image samples. At the same time, the label data of the remote sensing image samples after foreground annotation can be processed in the same way, and finally N second target training sample sets corresponding to N target categories are generated, with one second target training sample set corresponding to one target category. In this way, the proportion of the image content of the target category in the remote sensing image samples can be further increased, the training effect of the target model can be improved, and the image segmentation accuracy of the target model for remote sensing images can be further improved.

[0136] The image segmentation device provided by the embodiment of the present invention is described below.

[0137] See also Figure 3 , the figure shows a structural schematic diagram of the image segmentation device provided by an embodiment of the present invention.

[0138] like Figure 3 As shown, the image segmentation device 300 includes:

[0139] A first acquisition module 301 is configured to acquire L first remote sensing images, where L is a positive integer;

[0140] An image segmentation module 302 is configured to perform image segmentation on the L first remote sensing images based on the target model to obtain a target land segmentation result;

[0141] In which, the target model is trained based on N second target training sample sets and training sample sets corresponding to N first target training sample sets, the training sample sets include M remote sensing image samples, a second target training sample set is based on a target category corresponding to a first target training sample set, and the remote sensing image samples in the first target training sample set are marked with foreground annotations, the N first target training sample sets are divided into the M remote sensing image samples based on the N target categories corresponding to the training sample sets, the proportion of image content of the target category in the M remote sensing image samples is less than a preset threshold, M is an integer greater than 1, and N is a positive integer.

[0142] Optionally, the target model includes N first sub-models and second sub-models, the N first sub-models are trained based on the N second target training sample sets, and one first sub-model is trained based on one second target training sample set, and the image segmentation module 302 includes:

[0143] A first image segmentation unit is configured to perform image segmentation on the L first remote sensing images based on each first sub-model to obtain N first land parcel segmentation results;

[0144] a second image segmentation unit, configured to perform image segmentation on the L first remote sensing images based on the second sub-model to obtain a second land parcel segmentation result;

[0145] A fusion unit is used to fuse the N first land parcel segmentation results and the second land parcel segmentation result to obtain the target land parcel segmentation result.

[0146] Optionally, when L is greater than 1, the L first remote sensing images are obtained by performing image enhancement processing based on the second remote sensing image, and the first image segmentation unit is specifically configured to:

[0147] For each first sub-model, performing image segmentation on the L first remote sensing images based on the first sub-model to obtain L third land parcel segmentation results corresponding to the first sub-model;

[0148] The L third land parcel segmentation results corresponding to the first sub-model are fused to obtain a first land parcel segmentation result corresponding to the first sub-model.

[0149] The image segmentation device 300 can implement each process implemented in the above-mentioned image segmentation method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described here.

[0150] The model training device provided by an embodiment of the present invention is described below.

[0151] See also Figure 4 , the figure shows a schematic structural diagram of the model training device provided by an embodiment of the present invention.

[0152] like Figure 4 As shown, the model training device 400 includes:

[0153] A second acquisition module 401 is configured to acquire N target categories corresponding to a training sample set, wherein the training sample set includes M remote sensing image samples, and the proportion of image content of the target category in the M remote sensing image samples is less than a preset threshold, where M is an integer greater than 1 and N is a positive integer;

[0154] A division module 402 is configured to divide the M remote sensing image samples based on the N target categories to obtain N first target training sample sets, where one first target training sample set corresponds to one target category;

[0155] a labeling module 402 configured to, for each first target training sample set, perform foreground labeling on remote sensing image samples in the first target training sample set based on the target category corresponding to the first target training sample set, so as to generate a second target training sample set corresponding to the first target training sample set;

[0156] The training module 404 is configured to train a target model based on the N second target training sample sets corresponding to the N first target training sample sets and the training sample set.

[0157] Optionally, the target model includes N first sub-models and N second sub-models, and the training module 404 includes:

[0158] A first training unit is configured to train the N first sub-models based on the N second target training sample sets, wherein one first sub-model is trained based on one second target training sample set;

[0159] The second training unit is used to train the second sub-model based on the training sample set.

[0160] Optionally, the training sample set includes a land parcel category label and a land parcel edge contour label corresponding to a first remote sensing image sample, the first remote sensing image sample is any remote sensing image sample among the M remote sensing image samples, and the second training unit is specifically configured to:

[0161] Inputting the first remote sensing image sample into the second sub-model for image segmentation to obtain land parcel category information and land parcel edge contour information corresponding to the first remote sensing image sample;

[0162] Determining first difference information between the land parcel category information and the land parcel category label, and determining second difference information between the land parcel edge contour information and the land parcel edge contour label;

[0163] Based on the first difference information and the second difference information, network parameters of the second sub-model are updated.

[0164] Optionally, the marking module 402 is specifically configured to:

[0165] Based on the target category corresponding to the first target training sample set, performing foreground labeling on the remote sensing image samples in the first target training sample set to obtain a third target training sample set;

[0166] The remote sensing image samples in the third target training sample set are cropped to obtain a second target training sample set corresponding to the first target training sample set, wherein the remote sensing image samples in the second target training sample set include image content of a target area, and the target area is an area in the remote sensing image samples of the third target training sample set that includes image content of the target category.

[0167] The model training device 400 can implement each process implemented in the above-mentioned model training method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described here.

[0168] The electronic device provided by the embodiment of the present invention is described below.

[0169] See also Figure 5 , the figure shows a schematic diagram of the structure of the electronic device provided by the embodiment of the present invention. Figure 5 As shown, the electronic device 500 includes a processor 501 , a memory 502 , a user interface 503 and a bus interface 504 .

[0170] The processor 501 is configured to read the program in the memory 502 and execute the following process:

[0171] Acquire L first remote sensing images, where L is a positive integer;

[0172] Perform image segmentation on the L first remote sensing images based on the target model to obtain a target land segmentation result;

[0173] In which, the target model is trained based on N second target training sample sets and training sample sets corresponding to N first target training sample sets, the training sample sets include M remote sensing image samples, a second target training sample set is based on a target category corresponding to a first target training sample set, and the remote sensing image samples in the first target training sample set are marked with foreground annotations, the N first target training sample sets are divided into the M remote sensing image samples based on the N target categories corresponding to the training sample sets, the proportion of image content of the target category in the M remote sensing image samples is less than a preset threshold, M is an integer greater than 1, and N is a positive integer.

[0174] exist Figure 5 In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically linking together various circuits of one or more processors represented by processor 501 and memory represented by memory 502. The bus architecture may also link together various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are all well known in the art and therefore will not be further described herein. The bus interface 504 provides an interface. For different user devices, the user interface 503 may also be an interface capable of connecting external or internal devices as required, including but not limited to a keypad, display, speaker, microphone, joystick, etc.

[0175] The processor 501 is responsible for managing the bus architecture and general processing, and the memory 502 can store data used by the processor 501 when performing operations.

[0176] Optionally, the target model includes N first sub-models and second sub-models, the N first sub-models are trained based on the N second target training sample sets, one first sub-model is trained based on one second target training sample set, and the second sub-model is trained based on the training sample set. The processor 501 is further configured to:

[0177] Based on each first sub-model, the L first remote sensing images are segmented to obtain N first land parcel segmentation results;

[0178] Performing image segmentation on the L first remote sensing images based on the second sub-model to obtain a second land parcel segmentation result;

[0179] The N first land parcel segmentation results and the second land parcel segmentation result are fused to obtain the target land parcel segmentation result.

[0180] Optionally, when L is greater than 1, the L first remote sensing images are obtained by performing image enhancement processing based on the second remote sensing image, and the processor 501 is further configured to:

[0181] For each first sub-model, performing image segmentation on the L first remote sensing images based on the first sub-model to obtain L third land parcel segmentation results corresponding to the first sub-model;

[0182] The L third land parcel segmentation results corresponding to the first sub-model are fused to obtain a first land parcel segmentation result corresponding to the first sub-model.

[0183] In addition, the processor 501 is further configured to read the program in the memory 502 and execute the following process:

[0184] Obtain N target categories corresponding to a training sample set, the training sample set including M remote sensing image samples, the proportion of image content of the target category in the M remote sensing image samples being less than a preset threshold, where M is an integer greater than 1 and N is a positive integer;

[0185] Dividing the M remote sensing image samples based on the N target categories to obtain N first target training sample sets, where one first target training sample set corresponds to one target category;

[0186] For each first target training sample set, based on the target category corresponding to the first target training sample set, performing foreground labeling on the remote sensing image samples in the first target training sample set to generate a second target training sample set corresponding to the first target training sample set;

[0187] The target model is trained based on the N second target training sample sets corresponding to the N first target training sample sets and the training sample set.

[0188] Optionally, the target model includes N first sub-models and N second sub-models, and the processor 501 is further configured to:

[0189] Training the N first sub-models based on the N second target training sample sets, where one first sub-model is trained based on one second target training sample set;

[0190] The second sub-model is trained based on the training sample set.

[0191] Optionally, the training sample set includes a land parcel category label and a land parcel edge contour label corresponding to a first remote sensing image sample, where the first remote sensing image sample is any remote sensing image sample among the M remote sensing image samples. The processor 501 is further configured to:

[0192] Inputting the first remote sensing image sample into the second sub-model for image segmentation to obtain land parcel category information and land parcel edge contour information corresponding to the first remote sensing image sample;

[0193] Determining first difference information between the land parcel category information and the land parcel category label, and determining second difference information between the land parcel edge contour information and the land parcel edge contour label;

[0194] Based on the first difference information and the second difference information, network parameters of the second sub-model are updated.

[0195] Optionally, the processor 501 is further configured to:

[0196] Based on the target category corresponding to the first target training sample set, performing foreground labeling on the remote sensing image samples in the first target training sample set to obtain a third target training sample set;

[0197] The remote sensing image samples in the third target training sample set are cropped to obtain a second target training sample set corresponding to the first target training sample set, wherein the remote sensing image samples in the second target training sample set include image content of a target area, and the target area is an area in the remote sensing image samples of the third target training sample set that includes image content of the target category.

[0198] Preferably, an embodiment of the present invention further provides an electronic device, comprising a processor 501, a memory 502, and a computer program stored in the memory 502 and executable on the processor 501. When the computer program is executed by the processor 501, the computer program implements the various processes of the above-mentioned image segmentation method embodiment, or implements the various processes of the above-mentioned model training method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be described here.

[0199] The embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the computer program implements the various processes of the above-mentioned image segmentation method embodiment, or the various processes of the above-mentioned model training method embodiment, and can achieve the same technical effect. To avoid repetition, the details are not described here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0200] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0201] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0202] In the embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0203] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the objectives of the embodiments of the present invention.

[0204] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0205] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard drive, ROM, RAM, a magnetic disk, or an optical disk.

[0206] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. An image segmentation method, characterized in that: The method comprises: Acquire L first remote sensing images, where L is a positive integer; Perform image segmentation on the L first remote sensing images based on the target model to obtain a target land segmentation result; The target model is obtained by training based on N second target training sample sets and a training sample set corresponding to N first target training sample sets, the training sample set includes M remote sensing image samples, a second target training sample set is obtained by performing foreground annotation on the remote sensing image samples in the first target training sample set based on the target category corresponding to the first target training sample set, the N first target training sample sets are obtained by dividing the M remote sensing image samples based on the N target categories corresponding to the training sample sets, the proportion of image content of the target category in the M remote sensing image samples is less than a preset threshold, M is an integer greater than 1, and N is a positive integer; The target model includes N first sub-models and second sub-models, the N first sub-models are trained based on the N second target training sample sets, one first sub-model is trained based on one second target training sample set, and the second sub-model is trained based on the training sample set, and the target land segmentation result is obtained by performing image segmentation on the L first remote sensing images based on the target model, including: Based on each first sub-model, the L first remote sensing images are segmented to obtain N first land parcel segmentation results; Performing image segmentation on the L first remote sensing images based on the second sub-model to obtain a second land parcel segmentation result; The N first land parcel segmentation results and the second land parcel segmentation result are fused to obtain the target land parcel segmentation result.

2. The method according to claim 1, characterized in that When L is greater than 1, the L first remote sensing images are obtained by performing image enhancement processing based on the second remote sensing image, and the image segmentation is performed on the L first remote sensing images based on each first sub-model to obtain N first land parcel segmentation results, including: For each first sub-model, performing image segmentation on the L first remote sensing images based on the first sub-model to obtain L third land parcel segmentation results corresponding to the first sub-model; The L third land parcel segmentation results corresponding to the first sub-model are fused to obtain a first land parcel segmentation result corresponding to the first sub-model.

3. A model training method, characterized in that: The method comprises: Obtain N target categories corresponding to a training sample set, the training sample set including M remote sensing image samples, the proportion of image content of the target category in the M remote sensing image samples being less than a preset threshold, where M is an integer greater than 1 and N is a positive integer; Dividing the M remote sensing image samples based on the N target categories to obtain N first target training sample sets, where one first target training sample set corresponds to one target category; For each first target training sample set, based on the target category corresponding to the first target training sample set, performing foreground labeling on the remote sensing image samples in the first target training sample set to generate a second target training sample set corresponding to the first target training sample set; Training a target model based on the N second target training sample sets corresponding to the N first target training sample sets and the training sample set; The target model includes N first sub-models and a second sub-model, and the training of the target model based on the N second target training sample sets and the training sample set includes: Training the N first sub-models based on the N second target training sample sets, where one first sub-model is trained based on one second target training sample set; The second sub-model is trained based on the training sample set.

4. The method according to claim 3, characterized in that The training sample set includes a land parcel category label and a land parcel edge contour label corresponding to a first remote sensing image sample, the first remote sensing image sample being any remote sensing image sample among the M remote sensing image samples, and training the second sub-model based on the training sample set includes: Inputting the first remote sensing image sample into the second sub-model for image segmentation to obtain land parcel category information and land parcel edge contour information corresponding to the first remote sensing image sample; Determining first difference information between the land parcel category information and the land parcel category label, and determining second difference information between the land parcel edge contour information and the land parcel edge contour label; Based on the first difference information and the second difference information, network parameters of the second sub-model are updated.

5. The method according to claim 3, characterized in that The step of performing foreground labeling on remote sensing image samples in the first target training sample set based on the target category corresponding to the first target training sample set to generate a second target training sample set corresponding to the first target training sample set includes: Based on the target category corresponding to the first target training sample set, performing foreground labeling on the remote sensing image samples in the first target training sample set to obtain a third target training sample set; The remote sensing image samples in the third target training sample set are cropped to obtain a second target training sample set corresponding to the first target training sample set, wherein the remote sensing image samples in the second target training sample set include image content of a target area, and the target area is an area in the remote sensing image samples of the third target training sample set that includes image content of the target category.

6. An image segmentation device, characterized in that: The device comprises: A first acquisition module is used to acquire L first remote sensing images, where L is a positive integer; An image segmentation module is used to perform image segmentation on the L first remote sensing images based on the target model to obtain a target land segmentation result; The target model is obtained by training based on N second target training sample sets and a training sample set corresponding to N first target training sample sets, the training sample set includes M remote sensing image samples, a second target training sample set is obtained by performing foreground annotation on the remote sensing image samples in the first target training sample set based on the target category corresponding to the first target training sample set, the N first target training sample sets are obtained by dividing the M remote sensing image samples based on the N target categories corresponding to the training sample sets, the proportion of image content of the target category in the M remote sensing image samples is less than a preset threshold, M is an integer greater than 1, and N is a positive integer; The target model includes N first sub-models and second sub-models, the N first sub-models are trained based on the N second target training sample sets, and one first sub-model is trained based on one second target training sample set. The image segmentation module includes: A first image segmentation unit is configured to perform image segmentation on the L first remote sensing images based on each first sub-model to obtain N first land parcel segmentation results; a second image segmentation unit, configured to perform image segmentation on the L first remote sensing images based on the second sub-model to obtain a second land parcel segmentation result; A fusion unit is used to fuse the N first land parcel segmentation results and the second land parcel segmentation result to obtain the target land parcel segmentation result.

7. A model training device, characterized in that: The device comprises: A second acquisition module is configured to acquire N target categories corresponding to a training sample set, wherein the training sample set includes M remote sensing image samples, and the proportion of image content of the target category in the M remote sensing image samples is less than a preset threshold, where M is an integer greater than 1 and N is a positive integer; a division module, configured to divide the M remote sensing image samples based on the N target categories to obtain N first target training sample sets, where one first target training sample set corresponds to one target category; a labeling module, configured to, for each first target training sample set, perform foreground labeling on remote sensing image samples in the first target training sample set based on the target category corresponding to the first target training sample set, so as to generate a second target training sample set corresponding to the first target training sample set; A training module, configured to train a target model based on the N second target training sample sets corresponding to the N first target training sample sets and the training sample set; The target model includes N first sub-models and second sub-models, and the training module includes: A first training unit is configured to train the N first sub-models based on the N second target training sample sets, wherein one first sub-model is trained based on one second target training sample set; The second training unit is used to train the second sub-model based on the training sample set.

8. An electronic device, characterized in that: The electronic device includes: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the steps of the image segmentation method described in any one of claims 1 to 2 are implemented, or the steps of the model training method described in any one of claims 3 to 5 are implemented.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the image segmentation method according to any one of claims 1 to 2, or implements the steps of the model training method according to any one of claims 3 to 5.

Images