A method, device and medium for quality assessment of remote sensing image segmentation data

Abstract

The present application discloses a method, device and medium for evaluating the quality of remote sensing image segmentation data. The method includes: obtaining remote sensing image samples and their automatic segmentation data sets; segmenting the remote sensing image samples to obtain image slices; respectively determining a first parameter for characterizing the degree of overlap between the image slices and the target image blocks, and a second parameter for characterizing the difference in regional categories; the target image blocks are image blocks corresponding to the image slices in the automatic segmentation data sets; according to the first parameter and the second parameter, determining the integrity score of the regional segmentation and the accuracy score of the regional categories in the automatic segmentation data sets to determine the automatic segmentation quality of the automatic segmentation data sets. Thus, the integrity of the regional segmentation and the classification accuracy of the regional categories are determined by the area overlap between the image slices and the target image blocks and the difference in regional categories, and then a fast, efficient and accurate remote sensing image segmentation data quality evaluation can be achieved based on the integrity and accuracy.

Description

Technical Field

[0001] The present application relates to the field of remote sensing image technology, and in particular to a method, device and medium for quality assessment of remote sensing image segmentation data. Background Art

[0002] The purpose of intelligent interpretation of remote sensing images is to automatically and efficiently extract valuable information from remote sensing images through deep learning models. Among them, automatic segmentation and annotation of remote sensing images to obtain pixel-level remote sensing image segmentation data is crucial for extremely complex application scenarios.

[0003] In the pixel-level remote sensing image segmentation task, the core of segmentation lies in the accurate depiction of the contours and boundaries between objects, and the integrity of the internal areas of objects of the same category. High-quality remote sensing image segmentation data can be provided to the deep learning model as training data, and the model can be continuously iterated to improve the accuracy of the model's automatic segmentation. In addition, obtaining high-quality remote sensing image segmentation data is crucial for complex application scenarios such as land resource management, environmental monitoring, urban planning, and earthquake prevention and disaster reduction.

[0004] At present, the quality of automatically segmented remote sensing image segmentation data is mainly evaluated by manual inspection of pixel categories and positions, so as to verify the accuracy and integrity of remote sensing image segmentation data, correct low-quality segmentation data, and improve the overall quality of remote sensing image segmentation data. However, this quality assessment method is prone to errors, inefficient, and requires a lot of labor costs.

[0005] Therefore, how to quickly, efficiently and accurately evaluate the quality of remote sensing image segmentation data and reduce labor costs is an urgent problem to be solved by technical personnel in this field. Summary of the invention

[0006] In view of this, one aspect of the present application provides a method for evaluating the quality of remote sensing image segmentation data, the method comprising:

[0007] Acquire remote sensing image samples and automatic segmentation datasets of the remote sensing image samples;

[0008] Slicing the remote sensing image sample to obtain image slices;

[0009] Determining respectively a first parameter for characterizing the degree of overlap of the area of ​​the region and a second parameter for characterizing the difference in the category of the region between the image slice and the target image block; the target image block is an image block in the automatic segmentation data set corresponding to the image slice;

[0010] According to the first parameter and the second parameter, a completeness score of the region segmentation and an accuracy score of the region category in the automatically segmented data set are determined to determine the automatic segmentation quality of the automatically segmented data set.

[0011] Optionally, segmenting the remote sensing image sample to obtain image slices includes:

[0012] Using a corner point detection algorithm, calibrating the corner point coordinates of the remote sensing image sample;

[0013] Determining whether the number of corner point coordinates exceeds a threshold;

[0014] If it does not exceed, taking the corner point coordinates as designated points in the image slices, performing segmentation of a designated size to obtain the image slices; wherein the number of the image slices is the same as the number of the corner point coordinates;

[0015] If it exceeds, the coordinates of the corner points with a preset proportion are randomly selected as the designated points, and the image is divided into the designated size to obtain the image slice.

[0016] Optionally, determining a first parameter for characterizing a degree of overlap between the image slice and the target image block includes:

[0017] Evenly setting a preset number of seed points on the image slice;

[0018] Based on the seed point, extract the foreground object target in the image slice to obtain a first segmentation result; and convert the first segmentation result into a first binary segmentation result;

[0019] Converting the second segmentation result of the target image block into a second binary segmentation result;

[0020] The first parameter is determined according to the first binary segmentation result and the second binary segmentation result.

[0021] Optionally, determining the first parameter according to the first binary segmentation result and the second binary segmentation result includes:

[0022] Obtaining a preset weighting coefficient; wherein the preset weighting coefficient is set according to the area of ​​the foreground object target and the total area of ​​the image slice;

[0023] Based on the preset weighting coefficient, a weighted intersection-and-union ratio of the first binary segmentation result and the second binary segmentation result is determined; and the weighted intersection-and-union ratio is used as the first parameter.

[0024] Optionally, determining a second parameter for characterizing a difference in region category between the image slice and the target image block includes:

[0025] Assign category values ​​to different land feature area categories;

[0026] Performing region category inference on the image slice to obtain a first region category; and obtaining a second region category corresponding to a pre-set position in a result of sorting the target image block from large to small areas;

[0027] The second parameter is determined according to the category value, the first area category and the second area category.

[0028] Optionally, determining the second parameter according to the category value, the first area category, and the second area category includes:

[0029] Determine, according to the category value, a first value corresponding to the first area category and a second value corresponding to the second area category;

[0030] According to the first value and the second value, the multi-label hinge loss values ​​of the first area category and the second area category are determined; and the multi-label hinge loss values ​​are used as the second parameter.

[0031] Optionally, determining the completeness score of the region segmentation and the accuracy score of the region category in the automatically segmented data set according to the first parameter and the second parameter to determine the automatic segmentation quality of the automatically segmented data set includes:

[0032] The mean value of the first parameter corresponding to each of the image slices is used as the completeness score; and the mean value of the second parameter corresponding to each of the image slices is used as the accuracy score;

[0033] assigning corresponding weight coefficients to the completeness score and the accuracy score;

[0034] Determining a quality score of the automatically segmented data set based on the weight coefficient, the completeness score, and the accuracy score;

[0035] When the quality score is greater than the score threshold, it is determined that the automatic segmentation quality meets the standard.

[0036] Another aspect of the present application provides a remote sensing image segmentation data quality assessment device, the device comprising:

[0037] An acquisition module, used for acquiring remote sensing image samples and automatic segmentation data sets of the remote sensing image samples;

[0038] A segmentation module, used for segmenting the remote sensing image sample to obtain image slices;

[0039] a parameter determination module, used to respectively determine a first parameter for characterizing the degree of overlap of the area between the image slice and a target image block, and a second parameter for characterizing the difference in the area categories; the target image block is an image block in the automatic segmentation data set corresponding to the image slice;

[0040] The segmentation quality determination module is used to determine the completeness score of the region segmentation and the accuracy score of the region category in the automatic segmentation data set according to the first parameter and the second parameter, so as to determine the automatic segmentation quality of the automatic segmentation data set.

[0041] Another aspect of the present application provides a remote sensing image segmentation data quality assessment device, including a memory and a processor, wherein the memory stores a computer program that can be executed on the processor, and when the processor executes the program, the steps of the remote sensing image segmentation data quality assessment method are implemented.

[0042] Another aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the program is executed by a processor, the steps of the remote sensing image segmentation data quality assessment method are implemented.

[0043] The present application provides a method, device and medium for quality assessment of remote sensing image segmentation data, which have the following beneficial effects: by determining the area overlap between the image slice and the target image block, the integrity of the regional segmentation in the automatic segmentation data set is determined, that is, the accuracy of the contour and boundary segmentation between objects is determined. In addition, by determining the regional category difference between the image slice and the target image block, the classification accuracy of the regional category of the automatic segmentation data set can be determined, that is, whether the internal area of ​​the same category of objects is complete. Furthermore, the quality of the automatic segmentation data set is assessed based on the completeness and accuracy. In this way, not only the high cost caused by manual quality assessment can be avoided, but also fast, efficient and accurate quality assessment of remote sensing image segmentation data can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 A flowchart of a method for quality assessment of remote sensing image segmentation data provided in an embodiment of the present application;

[0045] Figure 2 A schematic diagram of the structure of a remote sensing image segmentation quality assessment system provided in an embodiment of the present application;

[0046] Figure 3 A schematic diagram of the principle of a quality assessment method for remote sensing image segmentation data provided in an embodiment of the present application;

[0047] Figure 4 A schematic diagram of a slice of remote sensing image segmentation provided in an embodiment of the present application;

[0048] Figure 5 A flowchart of a method for quality assessment of remote sensing image segmentation data provided by another embodiment of the present application;

[0049] Figure 6 A schematic diagram of the principle of a quality assessment method for remote sensing image segmentation data provided by another embodiment of the present application;

[0050] Figure 7 A flowchart of a method for quality assessment of remote sensing image segmentation data provided by another embodiment of the present application;

[0051] Figure 8 A schematic diagram of the principle of a method for quality assessment of remote sensing image segmentation data provided by another embodiment of the present application;

[0052] Fig. 9 A schematic diagram of the structure of a remote sensing image segmentation data quality assessment device provided in an embodiment of the present application;

[0053] Fig.10 A schematic structural diagram of a remote sensing image segmentation data quality assessment device provided in another embodiment of the present application.

[0054] The accompanying drawings are marked as follows: 1 is a quality assessment device, 2 is an inspection platform, 3 is a database, 20 is a memory, 21 is a processor, 22 is a display screen, 23 is an input and output interface, 24 is a communication interface, 25 is a power supply, 26 is a communication bus, 201 is a computer program, 202 is an operating system, and 203 is data. DETAILED DESCRIPTION

[0055] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0056] It should be understood that although the terms first, second, third, etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0057] Figure 1 A flowchart of a method for evaluating the quality of remote sensing image segmentation data provided by an embodiment of the present application is shown in FIG. Figure 1 As shown, the method includes:

[0058] S10: Obtain remote sensing image samples and automatic segmentation datasets of remote sensing image samples;

[0059] Figure 2 This is a schematic diagram of a remote sensing image segmentation quality assessment system provided in an embodiment of the present application. First, it should be noted that the remote sensing image segmentation data quality assessment method provided in the embodiment of the present application is applied to Figure 2 The quality assessment device 1 in the remote sensing image segmentation quality assessment system shown in FIG. 1 , that is, in an optional embodiment, the execution subject may be as follows Figure 2 The quality assessment device 1 is shown.

[0060] In an optional embodiment, if Figure 2 As shown, the quality assessment device 1 may include multiple, that is, n, and this application does not limit the specific number, which can be set according to actual business needs. The quality assessment device 1 specifies steps S10 to S13 to implement the remote sensing image segmentation data quality assessment method of this application.

[0061] Specifically, a remote sensing image sample is first obtained, and an automatically segmented data set is obtained after the remote sensing image sample is automatically segmented and annotated using deep learning technology. It should be noted that the present application does not limit the method for automatically segmenting the remote sensing image sample to obtain the automatically segmented data set.

[0062] In addition, it should be noted that the automatic segmentation data set includes regional segmentation of remote sensing image samples and regional annotation data, where regional segmentation refers to the delineation and segmentation of the contours and boundaries between objects, and regional annotation refers to the annotation of objects of the same category within the object in order to distinguish the categories of different object areas. Object area categories can include but are not limited to forests, farmlands, rivers, houses, and roads.

[0063] In an optional embodiment, the remote sensing image samples are segmented at the pixel level, and the ground object region categories in the obtained automatic segmentation data set can be distinguished based on pixels, and the same pixel value represents the same ground object region category.

[0064] It is worth noting that the method provided in the embodiment of the present application can perform quality assessment on multiple remote sensing image samples at the same time, that is, the present application does not limit the number of remote sensing image samples obtained in step S10.

[0065] S11: Slice the remote sensing image sample to obtain image slices;

[0066] Furthermore, in an optional embodiment, after obtaining the remote sensing image sample, the remote sensing image sample is segmented to obtain image slices. It should be noted that the size and segmentation method of the image slices are not limited in this application.

[0067] However, it is worth noting that the automatic segmentation dataset includes multiple image blocks, and the number of image blocks is the same as the number of image slices after segmentation. It is understandable that in order to evaluate the quality of the automatic segmentation dataset, it is necessary to compare the current segmented image slices with the image blocks in the automatic segmentation dataset to determine whether the integrity of the regional segmentation in the image block and the accuracy of the regional classification meet expectations. Therefore, in addition to the number of image slices after segmentation being the same as the number of image blocks, the image slices and image blocks are in a one-to-one correspondence, and the regional segmentation and regional classification between the one-to-one corresponding image slices and image blocks are similar.

[0068] S12: respectively determining a first parameter for characterizing the degree of overlap of the area between the image slice and the target image block, and a second parameter for characterizing the difference in the area categories; the target image block is an image block in the automatic segmentation data set corresponding to the image slice;

[0069] In an optional embodiment, after the remote sensing image sample is segmented, the first parameter and the second parameter between the image slice and the target image block are determined. It can be understood that the image slice and the image block in the automatic segmentation data set are in a one-to-one correspondence. When judging the regional segmentation completeness and regional category accuracy of the image slice, it is necessary to retrieve the image block corresponding to the current image slice from the automatic segmentation data set, that is, to obtain the target image block corresponding to the image slice.

[0070] Furthermore, a first parameter for characterizing the area overlap between the image slice and the target image block, and a second parameter for characterizing the area category difference are calculated. It should be noted that the present application does not limit the method for determining the area overlap between the two images, and the method for determining the area category difference between the two images, that is, the method for calculating the first parameter and the second parameter is not limited.

[0071] It should be noted that, in a specific embodiment, the first parameters and the second parameters of all image slices may be determined simultaneously, or the image slices may be traversed sequentially to calculate the first parameters and the second parameters corresponding to different image slices one by one.

[0072] S13: Determine, according to the first parameter and the second parameter, a completeness score of the region segmentation and an accuracy score of the region category in the automatic segmentation dataset to determine the automatic segmentation quality of the automatic segmentation dataset.

[0073] In fact, after step S12, the integrity of the regional segmentation in the automatic segmentation data set can be determined based on the first parameter corresponding to all image slices in the remote sensing image sample. The integrity of the regional segmentation refers to whether the outlines and boundaries between objects are accurately depicted, that is, whether the internal areas of the same object category are complete. In other words, the first parameter is mainly used to determine whether the polygons used to segment different object categories in the automatic segmentation data set are depicted correctly.

[0074] In addition, according to the second parameters corresponding to all image slices, the accuracy score of the region category in the automatic segmentation dataset can be determined. The accuracy of the region category refers to whether the recognition of the region category of the feature is correct, for example, whether the forest is correctly identified and labeled as a forest.

[0075] It should be noted that in order to intuitively evaluate the completeness of region segmentation and the accuracy of region classification, a score that can intuitively determine the completeness and accuracy, that is, a completeness score and an accuracy score, can be calculated based on the first parameter and the second parameter.

[0076] In an optional embodiment, further, the automatic segmentation quality of the automatically segmented data set may be directly determined according to the completeness score and the accuracy score, that is, the segmentation quality of the automatically segmented data set may be determined.

[0077] Furthermore, in an optional embodiment, after determining the segmentation quality of the automatic segmentation data set, if it is determined that the segmentation quality is poor, a prompt signal is sent so that the user can adjust the automatic segmentation data set with poor quality in time, thereby improving the overall automatic segmentation quality of the remote sensing image samples.

[0078] In an optional embodiment, in order to further improve the quality assessment accuracy of the automatically segmented data set, such as Figure 2 As shown, the remote sensing image segmentation quality assessment system provided by the present application also includes a review platform 2, which is used to review the assessment results of the quality assessment device 1. In an optional embodiment, in order to ensure the assessment quality while ensuring the assessment efficiency, the review platform 2 can conduct random checks on the assessment results of the quality assessment device 1.

[0079] In addition, if Figure 2 As shown, the remote sensing image segmentation quality assessment system also includes a database 3, which is used to store sample data that meets quality standards.

[0080] The spot check of the review platform 2 is implemented manually. In an optional embodiment, in order to ensure the efficiency of the spot check, the review platform 2 may include multiple review platforms. That is, different quality assessment devices 1 can be connected to multiple review platforms 2 in communication. The number of review platforms 2 connected to different quality assessment devices 1 is not limited in this application. Figure 2As shown, the present application does not limit n and m, and n and m can be the same or different.

[0081] In a specific embodiment, different review platforms 2 perform different random inspections, that is, the random inspections of remote sensing image samples are different. In an optional embodiment, different reviewers conduct quality random inspections on different review platforms 2. Specifically, the reviewers can use a browser to log in to their user accounts, and the quality assessment device 1 will automatically distribute the data corresponding to the quality assessment results to different reviewers. After the reviewers conduct random inspections, if the number of qualified samples exceeds 80%, the quality assessment device 1 will submit all the corresponding sample data to the database 3 for storage, so that high-quality remote sensing image sample segmentation data sets can be directly retrieved in actual application scenarios later.

[0082] Thus, the quality assessment method of remote sensing image segmentation data provided by the embodiment of the present application determines the completeness of the regional segmentation in the automatic segmentation data set by determining the area overlap between the image slice and the target image block, that is, determines the accuracy of the contour and boundary segmentation between the objects. In addition, by determining the regional category difference between the image slice and the target image block, the classification accuracy of the regional category of the automatic segmentation data set can be determined, that is, whether the internal area of ​​the same category of objects is complete. Further, the quality of the automatic segmentation data set is assessed based on the completeness and accuracy. Thus, not only can the high cost caused by manual quality assessment be avoided, but also fast, efficient and accurate quality assessment of remote sensing image segmentation data can be achieved.

[0083] As an optional embodiment, the remote sensing image sample is segmented to obtain image slices, including:

[0084] The corner point coordinates of remote sensing image samples are calibrated through the corner point detection algorithm;

[0085] Determine whether the number of corner point coordinates exceeds a threshold;

[0086] If it does not exceed, the corner point coordinates are used as designated points in the image slices, and the image slices are divided into designated sizes to obtain image slices; wherein the number of image slices is the same as the number of corner point coordinates;

[0087] If it exceeds, the coordinates of the corner points with a preset proportion are randomly selected as the specified points, and the image is divided into specified sizes to obtain image slices.

[0088] Figure 3 A schematic diagram of the principle of a remote sensing image segmentation data quality assessment method provided in an embodiment of the present application. In a specific embodiment, Figure 3 As shown, when the remote sensing image samples are segmented, the corner point coordinates of the remote sensing image samples are first calibrated through the corner point detection algorithm, which can be understood as selecting the corner point coordinates in the remote sensing image samples through the corner point detection algorithm.

[0089] Furthermore, it is determined whether the selected corner point coordinates are too many. Specifically, it is determined whether the number of corner point coordinates exceeds a threshold. If not, the corner point coordinates are used as designated points in the image slice, and then the remote sensing image sample is segmented into a specified size based on the designated points to obtain image slices.

[0090] It is worth noting that one corner point coordinate can be used to divide an image slice. Therefore, the number of image slices is the same as the number of corner point coordinates, and the image slices and corner point coordinates correspond one to one, that is, there is one corner point coordinate in an image slice, and the corner point coordinate is located at a specified point of the image slice.

[0091] For example, in an optional embodiment, the designated point may be the midpoint of the image slice, the designated size may be a 64*64 rectangle, and then the 64*64 rectangle is segmented using the corner point coordinates as the midpoint of the image slice.

[0092] It should be noted that the designated point can be the midpoint of the image slice or any vertex of the image slice rectangle, which is not limited in this application. In addition, it should be noted that when selecting the size of the segmentation, it can be set according to actual business needs. However, under normal circumstances, in order to ensure that the image slices after segmentation include as few regional categories as possible, thereby improving the subsequent calculation of regional category accuracy, therefore, in an optional embodiment, it can be set according to the area of ​​each regional category in the remote sensing image sample to ensure that there are no more than two regional categories in the image slices after segmentation.

[0093] Figure 4 A schematic diagram of a slice of remote sensing image segmentation provided in an embodiment of the present application, in Figure 4 In an optional embodiment described in the left figure, after the remote sensing image sample is segmented, an image slice set is obtained, including image slice A1, image slice A2, image slice A3 and image slice A4, and each image slice includes a corner point coordinate.

[0094] Correspondingly, the automatic segmentation dataset of the remote sensing image sample includes image blocks corresponding to the image slices, such as Figure 4 As shown in the right figure, it includes image block B1, image block B2, image block B3 and image block B4. In fact, it should be noted that image slices and image blocks are sub-images after the remote sensing image samples are segmented. In order to distinguish the current segmented image from the automatically segmented image, different naming methods are used.

[0095] In another optional embodiment, if there are too many current corner point coordinates, that is, the number of corner point coordinates exceeds a threshold, image segmentation based on all corner point coordinates will increase the overall computational complexity and affect the efficiency of quality assessment.

[0096] In addition, if there are too many corner point coordinates, the texture of multiple image slices will be repeated in some scenes. For example, farmland usually covers a large area. When the farmland area is segmented, multiple image slices may be farmland. Repeating the same image slice will lead to a waste of resources and affect the overall quality assessment efficiency.

[0097] Therefore, in this embodiment, when the number of corner point coordinates exceeds the threshold, the corner point coordinates of a preset proportion are randomly selected as designated points in the image slice, for example, 70% of the corner point coordinates are randomly selected as designated points for segmentation. The preset proportion is negatively correlated with the difference between the number of corner point coordinates and the threshold. That is, when the difference is larger, the preset proportion is smaller, and when the difference is smaller, the preset proportion is larger.

[0098] When segmenting, the coordinates of each corner point may be traversed in sequence to perform segmentation of a specified size, or segmentation may be performed simultaneously, which is not limited in this application.

[0099] Figure 5 A flowchart of a method for evaluating the quality of remote sensing image segmentation data provided by another embodiment of the present application is provided as an optional embodiment. Figure 5 As shown, determining a first parameter for characterizing the degree of overlap between the image slice and the target image block includes:

[0100] S50: evenly setting a preset number of seed points on the image slice;

[0101] S51: extracting foreground objects in the image slice based on the seed point to obtain a first segmentation result; and converting the first segmentation result into a first binary segmentation result;

[0102] Figure 6 A schematic diagram of the principle of a remote sensing image segmentation data quality assessment method provided by another embodiment of the present application. In a specific embodiment, Figure 6 As shown, a preset number of seed points are evenly set in the image slice, for example, 9 seed points are set, and based on the seed points, the foreground object target in the image slice is extracted to obtain the first segmentation result.

[0103] Specifically, the segmentation may be performed by specifying an image segmentation algorithm. For example, in an optional embodiment, the image segmentation algorithm may be a SNIC segmentation algorithm. Further, the first segmentation result obtained by the segmentation is converted into a first binary segmentation result.

[0104] It is worth noting that when extracting foreground objects from image slices, an initial value needs to be given, and the seed point is this initial value. Therefore, it is necessary to ensure that the seed points are evenly set on the image slices, and the number of seed points must ensure that the front part of the foreground object can be extracted.

[0105] It is understandable that if too many seed points are set, the image segmentation algorithm will have too many iterations, affecting efficiency. However, if too few seed points are set, the accuracy is insufficient and the accuracy of quality assessment cannot be guaranteed. Therefore, in an optional embodiment, the number of seed points can be set according to the specified size of the image slices, and the larger the specified size, the more seed points there are. Of course, in another optional embodiment, it can also be set according to actual business needs, which is not limited in this application.

[0106] S52: converting the second segmentation result of the target image block into a second binary segmentation result;

[0107] S53: Determine a first parameter according to the first binary segmentation result and the second binary segmentation result.

[0108] Furthermore, in order to facilitate determination of the regional area overlap between the image slice and the target image block, in a specific embodiment, it is necessary to convert the first segmentation result into a first binary segmentation result, and convert the second segmentation result of the target image block into a second binary segmentation result.

[0109] In multi-category image segmentation tasks, each pixel may be labeled as a different category. For example, different types of farmland and forests are represented by different values. In order to facilitate the determination of regional area overlap, all pixels with original label values ​​greater than or equal to 1 are considered "foreground", and pixels with label values ​​of 0 are considered "background". In other words, all pixels are classified into foreground and background.

[0110] like Figure 6 As shown in FIG. 1 , after the image slice is subjected to the setting of seed points and the image segmentation algorithm is used to extract the foreground object target, the image slice including the first foreground can be obtained, and the target image block corresponding to the image slice exists in the automatic segmentation data set. Figure 6 As shown, the target image block includes a second foreground, and the first foreground corresponds to the second foreground. Based on the first foreground and the second foreground, the area overlap between the image slice and the target image block can be determined, that is, the first parameter can be determined according to the first binary segmentation result and the second binary segmentation result.

[0111] Specifically, in an optional embodiment, as Figure 5 As shown, step S53 determines the first parameter according to the first binary segmentation result and the second binary segmentation result, including:

[0112] S530: Obtaining a preset weighting coefficient; wherein the preset weighting coefficient is set according to the area of ​​the foreground object target and the total area of ​​the image slice;

[0113] S531: Based on a preset weighting coefficient, determine a weighted intersection-and-union ratio of the first binary segmentation result and the second binary segmentation result; and use the weighted intersection-and-union ratio as a first parameter.

[0114] In an optional embodiment, if Figure 6 As shown, the area overlap between the image slice and the target image block is determined by weighted intersection-and-union calculation, and the calculated weighted intersection-and-union ratio is used as the first parameter. Specifically, the calculation formula is:

[0115]

[0116] in, is the weighted intersection-and-union ratio of the Nth image slice in the remote sensing image sample, is the total number of pixels in the image slice, The target image block is The value of the pixel, For image slices in The value of the pixel, is the preset weighting factor.

[0117] It can be understood that after converting both the image slice and the target image block into binary segmentation results, when 1 represents the foreground and 0 represents the background, the product , which can be used to characterize the current target image block in The pixel and image slice are in Specifically, if the result is 0, they are not the same, and if the result is 1, they are the same.

[0118] Among them, the weighting coefficient Used to control the weighted intersection-and-union ratio of the current image slice, that is, the weight of the quality score of the automatic segmentation data set, the weight coefficient It can be set according to the area of ​​the foreground object and the total area of ​​the image slice. The specific calculation formula is:

[0119]

[0120] in, is the area of ​​the foreground object, that is, the area of ​​the first foreground, is the total area of ​​the image tiles.

[0121] In a specific embodiment, it can be understood that the quality of the annotation of a small area of ​​fragmented land has little impact on the overall annotation sample, so a smaller impact factor can be given, that is, the weighting coefficient It can be set to a smaller value to avoid the impact of small mislabeled fragments on quality assessment.

[0122] like Figure 6 As shown, after calculating the weighted intersection-and-union ratio of the first binary segmentation result and the second binary segmentation result, the area overlap region between the image slice and the target image block can be determined. The larger the overlap region, the better the regional segmentation integrity.

[0123] Figure 7 A flowchart of a method for evaluating the quality of remote sensing image segmentation data provided by another embodiment of the present application is provided as an optional embodiment. Figure 7 As shown, determining a second parameter for characterizing the difference in regional categories between the image slice and the target image block includes:

[0124] S70: assigning category values ​​to different land feature area categories;

[0125] In a specific embodiment, for the convenience of calculation, different category values ​​are assigned to different feature area categories in step S70. The feature area categories may include but are not limited to forests, residential areas, farmlands, cities, and roads. Correspondingly, different category values ​​are pre-assigned to different feature area categories.

[0126] S71: performing region category inference on the image slice to obtain a first region category; and obtaining a second region category corresponding to a pre-set position in the result of sorting the area from large to small from the target image block;

[0127] S72: Determine a second parameter according to the category value, the first area category, and the second area category.

[0128] Figure 8 This is a schematic diagram of the principle of a remote sensing image segmentation data quality assessment method provided by another embodiment of the present application. For ease of understanding, Figure 8 The example shown is used for illustration.

[0129] In an optional embodiment, if Figure 8 As shown, the image slice is inferred to obtain the first regional category. Figure 8 In the example shown, the first region category includes rivers and roads. In addition, based on the size of the feature region, the feature regions in the target image block are sorted from large to small, and the second region category corresponding to the first preset position in the sorting result is obtained, where the preset position can be the first two positions, or the second region category corresponding to the largest area, which is not limited in this application.

[0130] It should be noted that when inferring the regional categories of image slices, in an optional embodiment, multiple public remote sensing image scene classification sample sets can be selected to perform migration training on the ResNet50 neural network model, and the trained ResNet50 neural network model can be used for category reasoning of image slices.

[0131] exist Figure 8 In the example shown, the second area category corresponding to the first two digits is obtained, and the second area category includes roads and cultivated land. That is, for the image slice, the first two categories with the largest pixel proportion in the target image block, that is, the largest area, are obtained as the second area category. It can be understood that different category values ​​are assigned to different land feature areas, and the first area category in the image slice and the second area category in the target image block are determined, and then the second parameter can be further determined.

[0132] Specifically, in an optional embodiment, as Figure 7 As shown, determining the second parameter according to the category value, the first area category and the second area category includes:

[0133] S720: Determine, according to the category value, a first value corresponding to the first area category and a second value corresponding to the second area category;

[0134] S721: Determine the multi-label hinge loss value of the first area category and the second area category according to the first value and the second value; and use the multi-label hinge loss value as the second parameter.

[0135] In a specific embodiment, according to the category value, a first value corresponding to the first area category and a second value corresponding to the second area category can be determined. Further, the multi-label hinge loss value is calculated by the first value and the second value. The specific calculation formula is:

[0136]

[0137] in, is the multi-label hinge loss value of the Nth image slice in the remote sensing image sample, is the total number of regional categories, is the number of slices in the Nth image. the first value of the first region category, is the first image block in the Nth image block a second value of a second area category, For all region categories in N image slices.

[0138] Among them, it should be noted that The value of is related to the specified size of the segmented image slice. In the specific embodiment, in order to ensure the accuracy of the calculation, The value of is not greater than 2.

[0139] like Figure 8 As shown, after the multi-label hinge loss value is calculated according to the first value and the second value, the multi-label hinge loss value is used as the second parameter to provide data support for the subsequent evaluation of the quality of the entire automatic segmentation data set.

[0140] Based on the above embodiment, as an optional embodiment, determining the completeness score of the region segmentation and the accuracy score of the region category in the automatically segmented data set according to the first parameter and the second parameter to determine the automatic segmentation quality of the automatically segmented data set includes:

[0141] The mean value of the first parameter corresponding to each image slice is used as the completeness score; and the mean value of the second parameter corresponding to each image slice is used as the accuracy score;

[0142] Assign corresponding weight coefficients to the completeness score and the accuracy score;

[0143] Determine the quality score of the automatically segmented data set based on the weight coefficient, the completeness score, and the accuracy score;

[0144] When the quality score is greater than the score threshold, it is determined that the automatic segmentation quality meets the standard.

[0145] In a specific embodiment, the mean value of the first parameter corresponding to each image slice is used as the integrity score, and the specific calculation formula is as follows:

[0146]

[0147] in, is the completeness score, is the weighted intersection-and-union ratio of the Nth image slice in the remote sensing image sample (i.e., the first parameter), The number of image slices cut for remote sensing image samples.

[0148] In addition, the mean value of the second parameter corresponding to each image slice is used as the accuracy score. The specific calculation formula is as follows:

[0149]

[0150] in, is the accuracy score, is the multi-label hinge loss value of the Nth image slice in the remote sensing image sample (i.e., the second parameter).

[0151] It is understandable that in a specific embodiment, some application scenarios pay more attention to the completeness of region segmentation, while some application scenarios pay more attention to the accuracy of region classification. Therefore, in order to meet the needs of different application scenarios, corresponding weight coefficients are assigned to the completeness score and the accuracy score, and the quality score of the automatically segmented data set is determined based on the weight coefficient, the completeness score and the accuracy score.

[0152] Of course, in an optional embodiment, when completeness and accuracy are equally important, the sum of the completeness score and the accuracy score is used as the quality score of the automatically segmented data set. Specifically, the calculation formula is:

[0153]

[0154] in, is the quality score of the automatically segmented dataset.

[0155] In an optional embodiment, when the quality score is greater than the score threshold, it is determined that the automatic segmentation quality meets the standard. Therefore, through the quality assessment method of remote sensing image segmentation data provided by this application, after determining the automatic segmentation quality, the corresponding quality standard mark can be performed so that the user can intuitively view the quality of the remote sensing image segmentation.

[0156] In order to make it easier for those skilled in the art to understand the technical solution provided by this application, Figure 3 For explanation. Figure 3 In the example shown, the algorithm for extracting foreground objects in image slices, that is, segmenting image slices, takes the SNIC segmentation algorithm as an example. The model for reasoning about the regional categories of image slices takes the ResNet50 neural network model as an example. In addition, integrity and accuracy are equally important, and the sum of the integrity score and the accuracy score is used as an example to illustrate the quality score of the automatically segmented data set.

[0157] like Figure 3 As shown, a remote sensing image sample and an automatic segmentation dataset of the remote sensing image sample are taken as input. First, the remote sensing image sample is segmented by a corner detection algorithm to obtain multiple image slices, and the target image blocks corresponding to the image slices are obtained from the automatic segmentation dataset. Among them, one image slice corresponds to one target image block.

[0158] Furthermore, the processing is divided into two branches, specifically, including the SNIC segmentation algorithm and the ResNet50 model. That is, it includes two branches: the completeness score calculation of the region segmentation and the accuracy score calculation of the region category.

[0159] In the branch of completeness score calculation, Figure 6The completeness score is determined by the weighted intersection-over-union calculation shown in Figure 2. At the same time, in the branch of accuracy score calculation, the accuracy score is determined by multi-label hinge loss calculation. Finally, the completeness score and the accuracy score are summed to obtain the quality score of the automatically segmented dataset.

[0160] In the above-mentioned embodiments, the quality assessment method for remote sensing image segmentation data is described in detail. The present application also provides an embodiment corresponding to a quality assessment device for remote sensing image segmentation data.

[0161] Fig. 9 A schematic diagram of a remote sensing image segmentation data quality assessment device provided in an embodiment of the present application is shown in FIG. Fig. 9 As shown, the device comprises:

[0162] An acquisition module 90 is used to acquire remote sensing image samples and automatic segmentation data sets of remote sensing image samples;

[0163] A segmentation module 91 is used to segment the remote sensing image sample to obtain image slices;

[0164] A parameter determination module 92 is used to respectively determine a first parameter for characterizing the degree of overlap of the area between the image slice and the target image block, and a second parameter for characterizing the difference in the area category; the target image block is an image block in the automatic segmentation data set corresponding to the image slice;

[0165] The segmentation quality determination module 93 is used to determine the completeness score of the region segmentation and the accuracy score of the region category in the automatic segmentation data set according to the first parameter and the second parameter, so as to determine the automatic segmentation quality of the automatic segmentation data set.

[0166] In addition, the remote sensing image segmentation data quality assessment device provided in the embodiment of the present application also includes:

[0167] Corner point coordinate calibration module, used to calibrate the corner point coordinates of remote sensing image samples through corner point detection algorithm;

[0168] The first processing module is used to determine whether the number of corner point coordinates exceeds a threshold; if not, the corner point coordinates are used as designated points in the image slices, and the image slices are divided into designated sizes to obtain image slices; wherein the number of image slices is the same as the number of corner point coordinates; if exceeded, the corner point coordinates with a preset proportion are randomly selected as designated points, and the image slices are divided into designated sizes to obtain image slices.

[0169] A seed point setting module is used to evenly set a preset number of seed points on the image slice;

[0170] An extraction module is used to extract the foreground object target in the image slice based on the seed point to obtain a first segmentation result; and convert the first segmentation result into a first binary segmentation result;

[0171] A conversion module, used for converting the second segmentation result of the target image block into a second binary segmentation result;

[0172] A first parameter determination module is used to determine a first parameter according to the first binary segmentation result and the second binary segmentation result.

[0173] A preset weighting coefficient acquisition module is used to acquire a preset weighting coefficient; wherein the preset weighting coefficient is set according to the area of ​​the foreground object target and the total area of ​​the image slice;

[0174] The weighted intersection-and-union ratio determination module is used to determine the weighted intersection-and-union ratio of the first binary segmentation result and the second binary segmentation result based on a preset weighting coefficient; and use the weighted intersection-and-union ratio as the first parameter.

[0175] Category value assignment module, used to assign category values ​​to different area categories of land features;

[0176] The second processing module is used to infer the region category of the image slice to obtain the first region category; and obtain the second region category corresponding to the pre-set position in the result of sorting the area from large to small from the target image block;

[0177] The second parameter determination module is used to determine the second parameter according to the category value, the first area category and the second area category.

[0178] A third processing module, used to determine a first value corresponding to the first area category and a second value corresponding to the second area category according to the category value;

[0179] The multi-label hinge loss value determination module is used to determine the multi-label hinge loss value of the first area category and the second area category according to the first value and the second value; and use the multi-label hinge loss value as the second parameter.

[0180] The fourth processing module is used to use the mean value of the first parameter corresponding to each image slice as the completeness score; and use the mean value of the second parameter corresponding to each image slice as the accuracy score;

[0181] A weight coefficient allocation module, used to allocate corresponding weight coefficients to the completeness score and the accuracy score;

[0182] A quality score determination module, used for determining the quality score of the automatically segmented data set according to the weight coefficient, the completeness score and the accuracy score;

[0183] The quality standard determination module is used to determine whether the automatic segmentation quality meets the standard when the quality score is greater than the score threshold.

[0184] Fig.10 A schematic diagram of a remote sensing image segmentation data quality assessment device provided in another embodiment of the present application is shown in FIG. Fig.10 As shown, the remote sensing image segmentation data quality assessment device includes: a memory 20 for storing a computer program;

[0185] The processor 21 is used to implement the steps of the remote sensing image segmentation data quality assessment method mentioned in the above embodiment when executing the computer program.

[0186] The remote sensing image segmentation data quality assessment device provided in this embodiment may include but is not limited to a smart phone, a tablet computer, a laptop computer or a desktop computer.

[0187] Among them, the processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 can be implemented in at least one hardware form of a digital signal processor (Digital Signal Processor, referred to as DSP), a field programmable gate array (Field-Programmable Gate Array, referred to as FPGA), and a programmable logic array (Programmable Logic Array, referred to as PLA). The processor 21 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a central processing unit (Central Processing Unit, referred to as CPU); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 21 may be integrated with a graphics processing unit (Graphics Processing Unit, referred to as GPU), and the GPU is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 21 may also include an artificial intelligence (Artificial Intelligence, referred to as AI) processor, which is used to process computing operations related to machine learning.

[0188] The memory 20 may include one or more computer-readable storage media, which may be non-transitory. The memory 20 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 20 is at least used to store the following computer program 201, wherein, after the computer program is loaded and executed by the processor 21, it can implement the relevant steps of the quality assessment method of remote sensing image segmentation data disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory 20 may also include an operating system 202 and data 203, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 202 may include Windows, Unix, Linux, etc. The data 203 may include, but is not limited to, relevant data involved in the quality assessment method of remote sensing image segmentation data, etc.

[0189] In some embodiments, the remote sensing image segmentation data quality assessment device may further include a display screen 22 , an input and output interface 23 , a communication interface 24 , a power supply 25 , and a communication bus 26 .

[0190] Those skilled in the art will understand that Fig.10 The structure shown in the figure does not constitute a limitation on the quality assessment device for remote sensing image segmentation data, and may include more or fewer components than those shown in the figure.

[0191] The remote sensing image segmentation data quality assessment device provided in the embodiment of the present application includes a memory and a processor. When the processor executes the program stored in the memory, it can implement the remote sensing image segmentation data quality assessment method in the above embodiment.

[0192] It should be noted that, although the operations are depicted in a specific order in the accompanying drawings, this should not be understood as requiring these operations to be performed in the specific order shown or to be performed sequentially, or requiring all illustrated operations to be performed to achieve the desired results. In some cases, multitasking and parallel processing may be advantageous. In addition, the separation of various system modules and components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product, or packaged into multiple software products.

Claims

1. A method for quality assessment of remote sensing image segmentation data, characterized in that: The method comprises: Acquire remote sensing image samples and automatic segmentation datasets of the remote sensing image samples; Slicing the remote sensing image sample to obtain image slices; Determining respectively a first parameter for characterizing the degree of overlap of the area of ​​the region and a second parameter for characterizing the difference in the category of the region between the image slice and the target image block; the target image block is an image block in the automatic segmentation data set corresponding to the image slice; Determine, according to the first parameter and the second parameter, a completeness score of the region segmentation and an accuracy score of the region category in the automatically segmented data set to determine the automatic segmentation quality of the automatically segmented data set; Determining a first parameter for characterizing a degree of overlap between the image slice and the target image block comprises: Evenly setting a preset number of seed points on the image slice; Based on the seed point, extract the foreground object target in the image slice to obtain a first segmentation result; and convert the first segmentation result into a first binary segmentation result; Converting the second segmentation result of the target image block into a second binary segmentation result; Determining the first parameter according to the first binary segmentation result and the second binary segmentation result; The determining the first parameter according to the first binary segmentation result and the second binary segmentation result includes: Obtaining a preset weighting coefficient; wherein the preset weighting coefficient is set according to the area of ​​the foreground object target and the total area of ​​the image slice; Based on the preset weighting coefficient, a weighted intersection-and-union ratio of the first binary segmentation result and the second binary segmentation result is determined; and the weighted intersection-and-union ratio is used as the first parameter.

2. The remote sensing image segmentation data quality assessment method according to claim 1, characterized in that: Slicing the remote sensing image sample to obtain image slices includes: Using a corner point detection algorithm, calibrating the corner point coordinates of the remote sensing image sample; Determining whether the number of corner point coordinates exceeds a threshold; If it does not exceed, taking the corner point coordinates as designated points in the image slices, performing segmentation of a designated size to obtain the image slices; wherein the number of the image slices is the same as the number of the corner point coordinates; If it exceeds, the coordinates of the corner points with a preset proportion are randomly selected as the designated points, and the image is divided into the designated size to obtain the image slice.

3. The remote sensing image segmentation data quality assessment method according to claim 1, characterized in that: Determining a second parameter for characterizing a difference in region category between the image slice and the target image block comprises: Assign category values ​​to different land feature area categories; Performing region category inference on the image slice to obtain a first region category; and obtaining a second region category corresponding to a pre-set position in a result of sorting the target image block from large to small areas; The second parameter is determined according to the category value, the first area category and the second area category.

4. The remote sensing image segmentation data quality assessment method according to claim 3, characterized in that: Determining the second parameter according to the category value, the first area category, and the second area category includes: Determine, according to the category value, a first value corresponding to the first area category and a second value corresponding to the second area category; According to the first value and the second value, the multi-label hinge loss values ​​of the first area category and the second area category are determined; and the multi-label hinge loss values ​​are used as the second parameter.

5. The remote sensing image segmentation data quality assessment method according to claim 1, characterized in that: Determining the completeness score of the region segmentation and the accuracy score of the region category in the automatically segmented data set according to the first parameter and the second parameter to determine the automatic segmentation quality of the automatically segmented data set includes: The mean value of the first parameter corresponding to each of the image slices is used as the completeness score; and the mean value of the second parameter corresponding to each of the image slices is used as the accuracy score; assigning corresponding weight coefficients to the completeness score and the accuracy score; Determining a quality score of the automatically segmented data set based on the weight coefficient, the completeness score, and the accuracy score; When the quality score is greater than the score threshold, it is determined that the automatic segmentation quality meets the standard.

6. A remote sensing image segmentation data quality assessment device, characterized in that: The device comprises: An acquisition module, used for acquiring remote sensing image samples and automatic segmentation data sets of the remote sensing image samples; A segmentation module, used for segmenting the remote sensing image sample to obtain image slices; a parameter determination module, used to respectively determine a first parameter for characterizing the degree of overlap of the area between the image slice and a target image block, and a second parameter for characterizing the difference in the area categories; the target image block is an image block in the automatic segmentation data set corresponding to the image slice; a segmentation quality determination module, configured to determine a completeness score of region segmentation and an accuracy score of region category in the automatically segmented data set according to the first parameter and the second parameter, so as to determine an automatic segmentation quality of the automatically segmented data set; A seed point setting module, used for evenly setting a preset number of seed points on the image slice; An extraction module, used for extracting the foreground ground object target in the image slice based on the seed point to obtain a first segmentation result; and converting the first segmentation result into a first binary segmentation result; A conversion module, used for converting the second segmentation result of the target image block into a second binary segmentation result; A first parameter determination module, configured to determine the first parameter according to the first binary segmentation result and the second binary segmentation result; A preset weighting coefficient acquisition module, used to acquire a preset weighting coefficient; wherein the preset weighting coefficient is set according to the area of ​​the foreground object target and the total area of ​​the image slice; A weighted intersection-and-union ratio determination module is used to determine a weighted intersection-and-union ratio of the first binary segmentation result and the second binary segmentation result based on the preset weighting coefficient; and use the weighted intersection-and-union ratio as the first parameter.

7. A remote sensing image segmentation data quality assessment device, comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor executes the program, the steps of the remote sensing image segmentation data quality assessment method described in any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the remote sensing image segmentation data quality assessment method according to any one of claims 1 to 5 are implemented.

Images