Sample labeling method and system for remote sensing image, computer device and storage medium
By generating and modifying pseudo-labels, the problem of low efficiency and accuracy in remote sensing image annotation was solved, and semi-automatic annotation of remote sensing image samples was achieved, improving the annotation accuracy and efficiency of natural features.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HAINAN UNIV
- Filing Date
- 2023-05-08
- Publication Date
- 2026-05-15
AI Technical Summary
Traditional remote sensing image annotation methods are inefficient and have low accuracy, making it difficult to accurately annotate natural features without fixed shapes, such as water bodies and landslides.
By acquiring remote sensing image samples, pseudo-labels are generated using a label generation algorithm, and noise reduction and feature contour extraction are performed. Combined with pseudo-label modification instructions and tools such as flood filling and live-line lasso, semi-automatic labeling is achieved, improving labeling accuracy and efficiency.
It reduces the time cost of deep learning sample annotation for remote sensing images, improves the annotation accuracy and precision of irregular features, and supports direct annotation of multispectral remote sensing images.
Smart Images
Figure CN116645545B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image annotation technology, and in particular to a method, system, computer equipment, and storage medium for annotating remote sensing images. Background Technology
[0002] Currently, pixel-level semantic segmentation datasets in the remote sensing field are all generated manually, which requires a significant amount of time. Therefore, annotation software has emerged. Traditional annotation software mostly does not support direct annotation of multispectral remote sensing images; it typically annotates on blank labels that cannot be modified, requiring manual point selection to synthesize polygons to fit the target contour. The current annotation process for deep learning samples in the remote sensing field is generally as follows: First, use specialized software to convert the remote sensing image into RGB image format; then, use annotation tools, such as Labelme, to manually select points on blank labels for the target features to fit polygons to the target contour; finally, export the labels.
[0003] However, traditional sample annotation methods are difficult to use for direct annotation of multispectral remote sensing images. For natural features without fixed shapes, such as water bodies and landslides, the method of manually selecting points and performing polygon fitting is difficult to achieve high-precision annotation of such natural features. Therefore, traditional sample annotation methods suffer from low efficiency and low annotation accuracy. Summary of the Invention
[0004] Therefore, in order to solve the above-mentioned technical problems, a method, system, computer equipment and storage medium for sample annotation of remote sensing images are provided, which can improve the efficiency and accuracy of sample annotation.
[0005] A method for labeling samples of remote sensing images, the method comprising:
[0006] Acquire remote sensing images and segment the remote sensing images to obtain remote sensing image samples;
[0007] Extract the sample features of the remote sensing image sample, obtain the label generation algorithm corresponding to the sample features, and use the label generation algorithm to generate pseudo-labels corresponding to the remote sensing image sample;
[0008] The pseudo-labels are denoised, and the contours of ground features are extracted from the remote sensing image samples to obtain the extracted samples.
[0009] Obtain a pseudo-label modification instruction, modify the pseudo-label according to the pseudo-label modification instruction and the extracted sample, and obtain the target label;
[0010] The target tags are reviewed, and once approved, they are stored in the dataset.
[0011] In one embodiment, noise reduction processing is performed on the pseudo-tags, including:
[0012] The pseudo-label is filtered using a connectivity filtering algorithm to identify and filter regions in the pseudo-label whose four-connected regions are less than a threshold, thus obtaining the filtered pseudo-label.
[0013] The jagged edges of the filtered pseudo-labels are smoothed using a median filtering algorithm to obtain the processed pseudo-labels.
[0014] In one embodiment, the remote sensing image sample is subjected to ground feature contour extraction to obtain an extracted sample, including:
[0015] Tool for finding and extracting ground features;
[0016] The contours of ground features are extracted from the remote sensing image samples using the ground feature extraction tool to obtain the extracted samples.
[0017] In one embodiment, modifying the pseudo-label according to the pseudo-label modification instruction and the extracted sample to obtain the target label includes:
[0018] The operation interface is displayed according to the pseudo-label modification instructions, and the seed point selected through the operation interface is obtained;
[0019] Centered on the seed point, calculate the difference between the pixel value of the pixel point in the four connected regions and the pixel value of the seed point, and add the pixels with a difference less than the threshold into the target region;
[0020] The newly added pixel is used as a new seed point, and new pixels are added to the target area with the new seed point as the center until no new pixels are added.
[0021] If the selected new seed point is a normal seed point, the new pixel point is weighted and fused with the extracted sample to obtain the target label;
[0022] If the selected new seed point is an anti-seed point, then the pixels surrounded by the extracted sample will be used as new labels and weighted image fusion will be performed with the extracted sample to obtain the target label.
[0023] In one embodiment, modifying the pseudo-label according to the pseudo-label modification instruction and the extracted sample to obtain the target label includes:
[0024] Calculate the cost of each side of the extracted sample, and determine the target contour based on the cost of each side;
[0025] Based on the target contour, a starting point is selected on the extracted sample as a seed point, and the target point is obtained;
[0026] Calculate the minimum average cost between the seed point and the target point, determine the segmentation contour based on the minimum average cost, and determine a new label based on the segmentation contour;
[0027] The new label is then fused with the extracted sample using a weighted image fusion process to obtain the target label.
[0028] In one embodiment, modifying the pseudo-label according to the pseudo-label modification instruction and the extracted sample to obtain the target label includes:
[0029] The cost of the extracted sample is calculated as a cost connectivity graph. The target region image is determined through the cost connectivity graph, and each corner point of the target region image is calculated as a pseudo seed point.
[0030] Each of the pseudo-seed points is used as a vertex in the improved livewire algorithm to generate a segmentation plane, and the pixels contained in the segmentation plane are labeled with new tags;
[0031] The new label is then fused with the extracted sample using a weighted image fusion process to obtain the target label.
[0032] In one embodiment, modifying the pseudo-label according to the pseudo-label modification instruction and the extracted sample to obtain the target label includes:
[0033] According to the pseudo-label modification instruction, only the cost in the region of interest (ROI) that needs to be modified in the pseudo-label is calculated, and the cost of the non-ROI region is filled to form a cost connectivity graph with low computational complexity.
[0034] Calculate the pseudo-seed points of the regions that need to be modified in the pseudo-labels, and generate a lasso that can be adjusted or have additional pseudo-seed points added to perform the adjustment, wherein the pseudo-seed points are connected by segmenting the contours with minimal cost.
[0035] The adjusted pseudo-seed points are used as vertices to generate a plane, and the pixels contained in the plane are labeled with new tags;
[0036] The new label is then fused with the extracted sample using a weighted image fusion process to obtain the target label.
[0037] A sample annotation system for remote sensing images, the system comprising:
[0038] The image segmentation processing module is used to acquire remote sensing images and segment the remote sensing images to obtain remote sensing image samples.
[0039] The pseudo-label generation module is used to extract the sample features of the remote sensing image sample, obtain the label generation algorithm corresponding to the sample features, and use the label generation algorithm to generate pseudo-labels corresponding to the remote sensing image sample.
[0040] The contour extraction module is used to perform noise reduction processing on the pseudo-labels and extract the contours of ground features from the remote sensing image samples to obtain the extracted samples.
[0041] The label modification module is used to obtain pseudo-label modification instructions, modify the pseudo-labels according to the pseudo-label modification instructions and the extracted samples, and obtain the target label.
[0042] The tag review module is used to review the target tags and store them in the dataset after the review is passed.
[0043] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program performing the following steps:
[0044] Acquire remote sensing images and segment the remote sensing images to obtain remote sensing image samples;
[0045] Extract the sample features of the remote sensing image sample, obtain the label generation algorithm corresponding to the sample features, and use the label generation algorithm to generate pseudo-labels corresponding to the remote sensing image sample;
[0046] The pseudo-labels are denoised, and the contours of ground features are extracted from the remote sensing image samples to obtain the extracted samples.
[0047] Obtain a pseudo-label modification instruction, modify the pseudo-label according to the pseudo-label modification instruction and the extracted sample, and obtain the target label;
[0048] The target tags are reviewed, and once approved, they are stored in the dataset.
[0049] A computer-readable storage medium having a computer program stored thereon, the computer program performing the following steps when executed by a processor:
[0050] Acquire remote sensing images and segment the remote sensing images to obtain remote sensing image samples;
[0051] Extract the sample features of the remote sensing image sample, obtain the label generation algorithm corresponding to the sample features, and use the label generation algorithm to generate pseudo-labels corresponding to the remote sensing image sample;
[0052] The pseudo-labels are denoised, and the contours of ground features are extracted from the remote sensing image samples to obtain the extracted samples.
[0053] Obtain a pseudo-label modification instruction, modify the pseudo-label according to the pseudo-label modification instruction and the extracted sample, and obtain the target label;
[0054] The target tags are reviewed, and once approved, they are stored in the dataset.
[0055] The aforementioned remote sensing image sample annotation method, system, computer equipment, and storage medium involve acquiring remote sensing images, segmenting the images to obtain remote sensing image samples, extracting sample features from the remote sensing image samples, obtaining a label generation algorithm corresponding to the sample features, and using the label generation algorithm to generate pseudo-labels corresponding to the remote sensing image samples. Noise reduction processing is applied to the pseudo-labels, and ground feature contour extraction is performed on the remote sensing image samples to obtain extracted samples. A pseudo-label modification instruction is obtained, and the pseudo-labels are modified according to the pseudo-label modification instruction and the extracted samples to obtain target labels. The target labels are then reviewed, and after approval, the target labels are stored in the dataset. The pseudo-labels generated by the label generation algorithm and the ground feature contour extraction enable semi-automatic label annotation, reducing the time cost of deep learning sample annotation for remote sensing images and improving the annotation accuracy of irregular ground features in remote sensing images. The modification instructions allow for modification of the pseudo-labels, further improving the accuracy of sample annotation. Attached Figure Description
[0056] Figure 1 This is an application environment diagram of a sample annotation method for remote sensing images in one embodiment;
[0057] Figure 2 This is a flowchart illustrating a sample annotation method for remote sensing images in one embodiment;
[0058] Figure 3 This is a schematic diagram illustrating the use of a flood filling tool to assist an operator in modifying pseudo-labels in one embodiment.
[0059] Figure 4 This is a schematic diagram illustrating the process of using a live-line lasso tool to assist an operator in correcting errors in a pseudo-label in one embodiment.
[0060] Figure 5 This is a schematic diagram illustrating the process of using the Shi-Tomasi corner detection tool to assist an operator in modifying pseudo-labels in one embodiment.
[0061] Figure 6 This is a schematic diagram illustrating the process of using the ROI live lasso modification tool to correct errors in pseudo-labels in one embodiment.
[0062] Figure 7 This is a schematic diagram illustrating the label review process in one embodiment;
[0063] Figure 8 This is a schematic diagram of the label modification process in one embodiment;
[0064] Figure 9 This is a structural block diagram of a sample annotation system for remotely sensed images in one embodiment;
[0065] Figure 10 This is a structural diagram of a sample annotation system for remote sensing images in another embodiment;
[0066] Figure 11 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation
[0067] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0068] The remote sensing image sample annotation method provided in this application embodiment can be applied to, for example... Figure 1 The application environment shown. For example... Figure 1 As shown, the application environment includes computer device 110. Computer device 110 can acquire remote sensing images and segment them to obtain remote sensing image samples; computer device 110 can extract sample features from the remote sensing image samples, obtain a label generation algorithm corresponding to the sample features, and use the label generation algorithm to generate pseudo-labels corresponding to the remote sensing image samples; computer device 110 can perform noise reduction processing on the pseudo-labels and extract ground feature contours from the remote sensing image samples to obtain extracted samples; computer device 110 can obtain pseudo-label modification instructions, modify the pseudo-labels according to the pseudo-label modification instructions and the extracted samples, and obtain target labels; computer device 110 can perform label review on the target labels, and after approval, store the target labels in the dataset. The computer device 110 can be, but is not limited to, various personal computers, laptops, smartphones, robots, unmanned aerial vehicles, tablets, and other devices.
[0069] In one embodiment, such as Figure 2 As shown, a sample annotation method for remote sensing images is provided, including the following steps:
[0070] Step 202: Acquire remote sensing images and segment them to obtain remote sensing image samples.
[0071] Computer equipment can acquire multispectral remote sensing imagery, and it can be equipped with a built-in multispectral remote sensing image cropping tool. After acquiring the remote sensing imagery, the computer equipment can use the built-in multispectral remote sensing image cropping tool to divide the imagery into multiple parts, thus obtaining multiple remote sensing image samples. These remote sensing image samples can be placed in a folder to be labeled for subsequent labeling work.
[0072] Step 204: Extract the sample features of the remote sensing image samples, obtain the label generation algorithm corresponding to the sample features, and use the label generation algorithm to generate pseudo-labels corresponding to the remote sensing image samples.
[0073] Computer equipment can generate pseudo-labels based on remote sensing image samples using appropriate algorithms. Specifically, computer equipment can extract sample features from remote sensing image samples to find corresponding label generation algorithms, such as the spectral features of the target or deep learning methods. The label generation algorithm selected based on the sample features can improve the accuracy of the generated pseudo-labels.
[0074] Step 206: Denoise the pseudo-labels and extract the contours of ground features from the remote sensing image samples to obtain the extracted samples.
[0075] The pseudo-labels automatically acquired by the algorithm for remote sensing image samples have problems such as classification errors, noise, and classification failures, so it is necessary to correct the pseudo-labels.
[0076] Computer equipment can perform noise reduction on fake labels. At the same time, the computer equipment can also have built-in high-precision tools for extracting ground feature contours, which can extract ground feature contours from remote sensing image samples to obtain extracted samples.
[0077] Step 208: Obtain the pseudo-label modification instruction, modify the pseudo-label according to the pseudo-label modification instruction and the extracted sample, and obtain the target label.
[0078] The computer device can obtain pseudo-tag modification instructions. In this embodiment, the computer device may have a built-in pseudo-tag modification tool. When modifying a tag, the computer device's display interface can show a comparison image of the modified tag. The operator can quickly identify the parts of the pseudo-tag that need modification and then click the modification control to trigger the pseudo-tag modification instruction. The computer device can use the pseudo-tag modification tool and a high-precision feature contour extraction tool to further refine the tag, thereby obtaining the target tag.
[0079] Step 210: Review the target labels and store them in the dataset after approval.
[0080] After obtaining the target labels, the computer device can review the labels, expose labels that do not meet the requirements and send them back for modification, and store the approved target labels into the dataset.
[0081] In this embodiment, the computer device acquires remote sensing images and segments them to obtain remote sensing image samples. It then extracts sample features from these samples, obtains a label generation algorithm corresponding to these features, and uses this algorithm to generate pseudo-labels corresponding to the remote sensing image samples. The pseudo-labels are then denoised, and ground feature contours are extracted from the remote sensing image samples to obtain extracted samples. A pseudo-label modification instruction is obtained, and based on the instruction and the extracted samples, the pseudo-labels are modified to obtain target labels. The target labels are then reviewed, and once approved, they are stored in the dataset. The pseudo-labels generated by the label generation algorithm and the extracted ground feature contours enable semi-automatic labeling, reducing the time cost of deep learning sample labeling for remote sensing images and improving the labeling accuracy of irregular ground features in remote sensing images. The modification instructions allow for the modification of pseudo-labels, further improving the accuracy of sample labeling.
[0082] In one embodiment, a sample annotation method for remote sensing images may further include a process of denoising pseudo-labels. The specific process includes: using a connected component filtering algorithm to filter the pseudo-labels, identifying and filtering regions in the pseudo-labels whose four-connected regions are less than a threshold, and obtaining filtered pseudo-labels; using a median filtering algorithm to smooth the jagged edges in the filtered pseudo-labels, and obtaining processed pseudo-labels.
[0083] Since the generated pseudo-labels often contain a large amount of fine noise, this embodiment uses a connected component filtering algorithm to filter the pseudo-labels. The connected component filtering algorithm can identify regions in the label whose four-connected regions are less than a threshold and filter them, thereby achieving noise reduction. In addition, this embodiment also uses a median filtering algorithm to smooth the jagged edges of the pseudo-labels, thus optimizing the pseudo-label effect.
[0084] In one embodiment, a sample annotation method for remote sensing images may further include a process of extracting ground feature contours, specifically including: finding a ground feature contour extraction tool; and extracting ground feature contours from the remote sensing image sample using the ground feature contour extraction tool to obtain the extracted sample.
[0085] In one embodiment, a sample annotation method for remote sensing images may further include using an improved flooding tool to assist the operator in identifying errors in pseudo-labels and thus modifying them. The specific process includes: displaying an operation interface based on pseudo-label modification instructions and obtaining seed points selected through the interface; calculating the difference between the pixel value of a pixel in a four-connected region and the pixel value of the seed point, centered on the seed point, and adding pixels with differences less than a threshold to the target region; using the newly added pixels as new seed points, and adding new pixels to the target region centered on the new seed points until no new pixels are added; if the selected new seed point is a normal seed point, then the new pixels are fused with the extracted samples using a weighted image fusion to obtain the target label; if the selected new seed point is an inverse seed point, then the pixels surrounded by the extracted samples are used as new labels and fused with the extracted samples using a weighted image fusion to obtain the target label.
[0086] Among them, the flood fill tool can automatically extract pixels that are related to the color of the selected pixel, so as to quickly identify features with the same color characteristics. The improved flood fill tool can not only select seed points, but also select inverse seed points to quickly extract small features.
[0087] Specifically, the process of using the flood fill tool to help operators find errors in pseudo-tags and thus correct them is as follows: Figure 3 As shown, users can select the label color to correspond to the background value in the label color table on the operation interface, select a seed point (x, y) on the image in the new window, and then, using the seed point as the center, determine the difference between the pixel value of the 4-neighborhood and the pixel value of the seed point. Pixels with a difference less than a threshold are added to the region. Then, the computer device can use the newly added pixels as new seed points, repeatedly using the new seed point as the center, determining the difference between the pixel value of the 4-neighborhood and the pixel value of the seed point, and adding pixels with a difference less than a threshold to the region until no new pixels are added to the region. If the seed point is a normal seed point, the new label is fused with the original image using weighted image fusion. If the seed point is an inverse seed point, the pixels surrounded by the extracted samples are used as new labels and fused with the original image using weighted image fusion. The fused image can be judged visually for its quality. Qualified new labels are merged with old labels.
[0088] In one embodiment, a sample labeling method for remote sensing images may further include using a live lasso tool to assist the operator in identifying errors in false labels and thus modifying the false labels. The specific process includes: calculating the cost of each edge of the extracted sample and determining the target contour based on the cost of each edge; selecting a starting point as a seed point on the extracted sample based on the target contour and obtaining the target point; calculating the minimum average cost between the seed point and the target point, determining the segmentation contour based on the minimum average cost, and determining a new label based on the segmentation contour; and performing weighted image fusion of the new label and the extracted sample to obtain the target label.
[0089] In this embodiment, the live wire lasso tool can be based on an improved livewire algorithm, which can automatically search for dividing lines that fit the edge of the target between the seed point and the target point. When a series of dividing lines are connected end to end to form a plane, the plane is identified as the target label. The live wire lasso tool can quickly identify features with relatively clear edges.
[0090] Specifically, the process of using the live lasso tool to assist the operator in discovering errors in the pseudo-labels and thus correcting them is as follows: Figure 4 As shown, a computer device can calculate the cost of the entire graph and treat it as a cost-connected graph, where the local cost l(p,q) between two points is determined by the following formula: l(p,q)=ω Z *f Z (q)+ω D *f D (p,q)+ω G *f G (q). In the improved livewire algorithm, f... Z The Laplace operator function of (q) is replaced with the improved Canny operator function, which can more clearly identify edges. The characteristic equation of the improved Canny operator is: The introduction of corner points can further improve the accuracy of the livewire algorithm; f G (q) is the cost function related to the gradient magnitude. For digital images, this is equivalent to finding the gradient of a two-dimensional discrete function, and the partial derivative I can be approximated using the difference function. x and I y The gradient magnitude G can be equivalent to: To assign low cost to strong edges and high cost to weak edges, the gradient magnitude is multiplied by an inverse linear function, i.e.: f D (p,q) is the gradient direction-dependent cost function, which reflects the gradient direction changes between nodes. Its expression is: D(p)=[I x (p),I y (p)],D'(p)=[I y(p),-I x (p)]), Where: V p (p,q)=D'(p)·L(p,q), V q (p,q)=L(p,q)·D'(q), ω Z ω D ω G Let ω be the weight of each factor, and the following relationship exists: Z +ω D +ω G =1.
[0091] Next, the computer device can select a starting point on the image based on the target contour, designated as seed point p; then, it takes the pixel currently pointed to by the mouse and designates it as target point p. It then calculates the minimum average cost along the path between the two points, using Dijkstra's algorithm to find the minimum cost path from seed point p to target point q, and displays the segmented contour on the interface. The improved Livewire replaces the cumulative cost in the traditional algorithm with the minimum average cost, effectively reducing computational load and path error rate.
[0092] The computer device can repeatedly select a starting point on the image as a seed point based on the target contour, then obtain the pixel currently pointed to by the mouse and designate it as the target point. It then calculates the minimum average cost on the path between the two points, finds the minimum cost path from the seed point to the target point, and displays the segmentation contour on the interface until an end signal is received. The last target point and the starting point are treated as the seed point and target point for calculation, generating a plane with n points as vertices. The pixels contained in the plane are labeled as required. The new labels are then weighted and fused with the original image. The fused image can be visually judged to be qualified. Qualified new labels are merged with old labels. Compared with the polygon lasso used by other annotation software, the live lasso tool can obtain accurate edge segmentation with fewer seed points.
[0093] In one embodiment, a sample labeling method for remote sensing images may further include the process of using the Shi-Tomasi live-wire lasso tool to help the operator detect errors in pseudo-labels and thus modify the pseudo-labels. The specific process includes: calculating the overall image cost of the extracted samples and using it as a cost connectivity graph; determining the target region image through the cost connectivity graph and calculating each corner point of the target region image as a pseudo-seed point; using each pseudo-seed point as a vertex of the improved livewire algorithm to generate a segmentation plane, and labeling the pixels contained in the plane as new labels; and performing weighted image fusion of the new labels and the extracted samples to obtain the target label.
[0094] In this embodiment, since the Livewire algorithm can achieve better edge segmentation by selecting the corner points of the target as seed points, this embodiment proposes an improved Shi-Tomasi corner detection tool that can automatically identify the corner points of the target in the remote sensing image as seed points, so that the seed points can be referenced without relying on visual judgment.
[0095] Specifically, the Shi-Tomasi corner detection tool helps operators better identify the edge corners of missing targets in labels, thereby enabling them to better use the live lasso tool to modify pseudo-labels. The process is as follows: Figure 5 As shown, by using a computer device to select the area of the target in the image, the corner points of the target area will be calculated. The principle of the improved Shi-Tomasi corner detection algorithm is as follows: The Shi-Tomasi algorithm detects corner points by calculating the gray-scale changes of points within the window. As the window moves in multiple directions, if the gray-scale changes within the window are not significant, it is considered a flat area. If the gray-scale changes in some directions but not in others, it is considered a straight line. If the gray-scale changes in all directions, it is considered a corner point.
[0096] Using the Shi-Tomasi algorithm, the magnitude of the grayscale change after a local window W(x,y) moves (u,v) can be detected as: F(u,v)=∑ (x,y) ω(x,y)[I(x,y)-I(x+u,y+v)] 2 In the formula, ω(x,y) is a Gaussian weighted function, which can be obtained from the Taylor expansion. in The eigenvalue response function of the Shi-Tomasi corner is: R = min(λ1,λ2), where λ1 and λ2 are two eigenvalues of M(x,y).
[0097] Traditional Shi-Tomasi algorithms are prone to corner clusters and false corners. This embodiment proposes a live-line lasso tool based on an improved Shi-Tomasi corner detection algorithm. It detects corners by utilizing grayscale changes within a window and performs maximum suppression on the corners, thus resolving corner clustering and false corner phenomena. Furthermore, by selecting cluster centers and performing K-means clustering on all corners based on pixel similarity, corners of the same category can be more evenly distributed near the target contour. This corner detection-based method provides effective seed point references for the live-line lasso, helping annotators complete target area annotation tasks more quickly and accurately.
[0098] The computer device can treat the corner points generated by the improved Shi-Tomasi algorithm as pseudo-seed points and select pseudo-seed points sequentially based on visual observation. Based on the selected n pseudo-seed points, the live-line lasso algorithm is executed group by group to generate a plane with n points as vertices. The pixels contained therein are labeled as the required labels. The new labels are then fused with the original image using weighted image fusion. The fused image can be judged as qualified by visual observation. Qualified new labels are then merged with old labels.
[0099] In one embodiment, such as Figure 6 As shown, the provided remote sensing image sample labeling method can also include the process of correcting errors in pseudo-labels using a ROI live-line lasso tool. The specific process includes: calculating the cost of only the ROI in the pseudo-label that needs modification according to the pseudo-label modification instruction; filling the cost of non-ROI regions to form a cost connectivity graph with low computational complexity; calculating pseudo-seed points in the pseudo-label of the region that needs modification; generating a lasso that can be adjusted or have additional pseudo-seed points added for adjustment, wherein the pseudo-seed points are connected by contour segmentation with minimum cost; generating a plane using the adjusted pseudo-seed points as vertices; and labeling the pixels contained within the plane as new labels; and performing weighted image fusion of the new labels and extracted samples to obtain the target label.
[0100] Specifically, the ROI live wire lasso modification tool based on the livewire algorithm of ROI local cost obtains the pseudo-label to be modified, treats the area to be modified in the pseudo-label as the ROI region, calculates the cost in the ROI region of interest, and fills in the cost of non-ROI regions, thereby forming a cost connectivity graph with low computational complexity.
[0101] By calculating the pseudo-seed points that need to be modified for the labels, a lasso is generated that can be adjusted or added to implement the adjustment, where pseudo-seed points are connected by segmenting the contour at the minimum cost; annotators can further correct pseudo-labels by adjusting the position of pseudo-seed points or increasing the number of pseudo-seed points.
[0102] The adjusted pseudo-seed points are used as vertices to generate a plane, and the pixels contained in the plane are labeled as new labels. The new labels are then fused with the extracted samples using a weighted image fusion process. The fused image can be visually judged to determine whether it is qualified. Qualified new labels are then merged with the old labels.
[0103] In one embodiment, a sample labeling method for remote sensing images can also help labelers identify and correct errors in pseudo-labels through a label color reference table and label modification tools.
[0104] The process involves extracting the original label colors from the pseudo-labels and writing them into a label color lookup table. Additionally, labelers can select the corresponding color for the label within the 256-level 8-bit full value range RGB. For example, a vegetation cover area can be labeled as #00FF00 (green) and a body of water as #0000FF (blue). The selected label colors will be displayed in the label color lookup table in the set order and with an encoding.
[0105] Labelers can select label colors from the label color correspondence table or color palette on the operation interface, and use the slider to adjust the brush size or eraser size to supplement label error areas, erase noise, and correct edges.
[0106] In one embodiment, a sample annotation method for remote sensing images may further include a label review process, which includes: simultaneously displaying target labels and extracted samples on a display interface, obtaining review marks for the target labels through the display interface; determining whether the target labels have passed review based on the review marks; if the target labels have passed review, storing the target labels in the dataset; if the target labels have failed review, storing the target labels in the label modification database.
[0107] Even after the labelers have corrected the labels, errors may still exist, requiring further review by the auditors. Figure 7 As shown, the review software in the computer equipment can automatically generate a composite image of multispectral remote sensing images and labels displayed in RGB color. The left side of the composite image displays the fused image of the semi-transparent label and the original image, while the right side displays the remote sensing image of the non-label area. The obvious contrast allows reviewers to quickly identify problems in the labels, making it convenient to circle and process them. Unqualified labels and images, as well as composite images, are placed in a specific folder for labelers to modify.
[0108] In the sample annotation method for remote sensing images proposed in this application, the label modification system is as follows: Figure 8As shown, pseudo-labels, label modification tools, and high-precision feature contour extraction tools are used to ensure the accuracy and efficiency of sample labeling. Meanwhile, the flood fill tool, live-line lasso tool, Shi-Tomasi corner detection tool, and ROI live-line lasso modification tool can more accurately and efficiently segment irregularly shaped natural features. Furthermore, the labeling software can not only label blank labels but also directly modify pseudo-labels, which greatly improves the labeling efficiency of remote sensing images. In addition, the flood fill tool, live-line lasso tool, Shi-Tomasi corner detection tool, and ROI live-line lasso modification tool effectively solve the problem that existing labeling methods cannot perform high-precision labeling of natural features without fixed shapes. Moreover, a multispectral remote sensing image compatibility module is adopted, which can directly read multispectral remote sensing images as RGB images, eliminating the need for conversion to RGB images for labeling, and allowing for free combination of bands for more refined modifications.
[0109] In one embodiment, the sample annotation method for remote sensing images of this application is used to annotate a semantic segmentation dataset of water bodies. The specific steps include:
[0110] Acquire remote sensing images and then divide them into multiple parts after segmentation.
[0111] In this embodiment of the invention, for the creation of a water body semantic segmentation dataset, the images must contain water bodies. After segmentation and division using the built-in multispectral remote sensing image cropping tool, the images are placed in a folder to be labeled for subsequent labeling work.
[0112] Use appropriate algorithms to generate pseudo-labels for multispectral remote sensing images.
[0113] In this embodiment of the invention, the Normalized Difference Water Index (NDWI) from a built-in traditional remote sensing algorithm is used to create pseudo-labels for the water body semantic segmentation dataset. The principle behind this is based on the spectral characteristics of ground features within the water body. The pseudo-labels can be generated not only using traditional remote sensing algorithms, but also any deep learning or machine learning algorithm capable of semantic segmentation. Furthermore, this invention can also create blank labels, which can be used as a type of pseudo-label.
[0114] Noise reduction processing is performed on the generated pseudo-labels.
[0115] Since the generated pseudo-labels often contain a large amount of fine noise, this embodiment of the invention employs a connected component filtering algorithm to filter the pseudo-labels. This algorithm can identify regions in the label whose 4-connected regions are less than a threshold and filter them, thereby achieving a noise reduction effect. Furthermore, this embodiment of the invention also uses a median filtering algorithm to smooth the jagged edges in the pseudo-labels, thus optimizing the pseudo-label effect.
[0116] Use tag editing software to modify pseudo-tags.
[0117] This invention provides a label modification software. Users can select the path of multispectral remote sensing imagery and labels on the user interface, choosing R, G, and NIR bands for input to highlight water bodies. The editing window provides a comparison function; users can enter edit mode by right-clicking, displaying an image composed of a semi-transparent image and a semi-transparent label, clearly showing the error areas of false labels. Furthermore, users can use label modification tools, flood fill tools, live-line lasso tools, Shi-Tomasi corner detection tools, and ROI live-line lasso modification tools to further process the labels, reducing label precision to the pixel level.
[0118] Use label verification software to verify new labels.
[0119] After selecting the paths for the multispectral remote sensing image and its corresponding label, the label review software automatically synthesizes a composite image of the multispectral remote sensing image and the label, displayed in RGB color. The left side of this composite image shows the label-covered area, which is covered by a semi-transparent mask of the corresponding label color, while the right side shows the non-label area. This allows reviewers to quickly identify problems in the labels, especially classification errors, noise, and classification failures. Once a problem is found, reviewers can circle and mark the incorrect parts of the composite image and place the rejected original image and label into the unqualified folder. The information is also automatically written to the log for labelers to make further modifications.
[0120] In one embodiment, the sample annotation method of the remote sensing imagery of this application is used to annotate a remote sensing ground feature semantic segmentation dataset. The specific steps include:
[0121] Acquire multispectral remote sensing images and then divide them into multiple parts after segmentation.
[0122] In this embodiment of the invention, in order to meet the need for creating a remote sensing land cover semantic segmentation dataset, the remote sensing image must contain one or more of the land covers to be classified. The remote sensing image can be segmented and divided using the built-in multispectral remote sensing image cropping tool, and the results are placed in the folder to be labeled.
[0123] Use appropriate algorithms to generate pseudo-labels for multispectral remote sensing images.
[0124] In this embodiment of the invention, pseudo-labels for the multispectral remote sensing ground feature semantic segmentation dataset can be generated using a semi-supervised deep learning algorithm with the aid of a large-scale pre-trained model, while the labeled data used for training can be generated by labeling blank labels using a label modifier. Furthermore, this invention provides a label color reference card, allowing users to customize and select the RGB colors of the corresponding ground feature labels from the 256-level 8-bit full-value RGB range and store them in the label color card. These selected colors can then be used for label annotation.
[0125] Noise reduction processing is performed on the generated pseudo-labels.
[0126] Since the generated pseudo-labels generally contain a large amount of fine noise, this embodiment of the invention employs a connected component filtering algorithm to filter the pseudo-labels. This algorithm can identify regions in the label whose 4-connected regions are less than a threshold and filter them, thereby achieving a noise reduction effect. Furthermore, this embodiment of the invention also uses a median filtering algorithm to smooth the jagged edges in the pseudo-labels, thus optimizing the pseudo-label effect.
[0127] Use tag editing software to modify pseudo-tags.
[0128] The label modification software proposed in this invention can improve label accuracy to the pixel level by selecting the path of multispectral remote sensing imagery and corresponding labels, using the R, G, and B bands of the multispectral remote sensing imagery and the labels as input, and then further processing them using label modification tools, flood fill tools, live-line lasso tools, Shi-Tomasi corner detection tools, and ROI live-line lasso modification tools. Furthermore, to improve the accuracy of pseudo-labels generated by deep learning algorithms, newly modified high-precision datasets can be added to the algorithm's dataset to expand the data volume before training and prediction, further improving the accuracy of the generated pseudo-labels.
[0129] Use label verification software to verify new labels.
[0130] After selecting the paths for the multispectral remote sensing image and its corresponding label, the label review software automatically synthesizes a composite image of the multispectral remote sensing image and the label, displayed in RGB color. The left side of the composite image shows the label-covered area, which is covered by a semi-transparent mask of the corresponding label color, while the right side shows the non-labeled area. This allows reviewers to quickly identify problems in the labels, especially classification errors, noise, and classification failures. After identifying problems, reviewers circle and mark the incorrect parts of the composite image and place the original image and label that failed the review into a "failed review" folder. The information is automatically written to the log so that the labelers can make further modifications.
[0131] It should be understood that although the steps in the above flowcharts are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the above flowcharts may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.
[0132] In one embodiment, such as Figure 9 As shown, a sample annotation system for remote sensing images is provided, including: an image segmentation processing module 910, a pseudo-label generation module 920, a contour extraction module 930, a label modification module 940, and a label verification module 950, wherein:
[0133] The image segmentation processing module 910 is used to acquire remote sensing images and segment the remote sensing images to obtain remote sensing image samples.
[0134] The pseudo-label generation module 920 is used to extract sample features of remote sensing image samples, obtain the label generation algorithm corresponding to the sample features, and use the label generation algorithm to generate pseudo-labels corresponding to the remote sensing image samples.
[0135] The contour extraction module 930 is used to denoise the pseudo-labels and extract the contours of the ground features from the remote sensing image samples to obtain the extracted samples.
[0136] The label modification module 940 is used to obtain pseudo-label modification instructions, modify the pseudo-labels according to the pseudo-label modification instructions and the extracted samples, and obtain the target label.
[0137] The tag review module 950 is used to review target tags and store them in the dataset after the review is passed.
[0138] In one embodiment, a sample annotation system for remote sensing images is provided, such as Figure 10 As shown, it includes a multispectral remote sensing image compatibility module, a pseudo-label modification module, a high-precision tool for extracting ground feature outlines, and label review software.
[0139] In one embodiment, the contour extraction module 930 is further configured to use a connected filtering algorithm to filter the pseudo-label, identify the regions in the pseudo-label whose four-connected regions are less than a threshold and filter them to obtain the filtered pseudo-label; and use a median filtering algorithm to smooth the jagged edges in the filtered pseudo-label to obtain the processed pseudo-label.
[0140] In one embodiment, the contour extraction module 930 is also used to locate a ground feature contour extraction tool; and to extract ground feature contours from remote sensing image samples using the ground feature contour extraction tool to obtain extracted samples.
[0141] In one embodiment, the label modification module 940 is further configured to display an operation interface according to the pseudo-label modification instruction and obtain the seed point selected through the operation interface; calculate the difference between the pixel value corresponding to the pixel point in the four-connected region and the pixel value of the seed point, with the seed point as the center, and add the pixel points with a difference less than a threshold into the target region; take the newly added pixel point as the new seed point, and add new pixel points into the target region with the new seed point as the center, until no new pixel points are added; if the selected new seed point is a normal seed point, then the new pixel point and the extracted sample are weighted and fused to obtain the target label; if the selected new seed point is an anti-seed point, then the pixel points surrounded by the extracted sample are taken as the new label and weighted and fused with the extracted sample to obtain the target label.
[0142] In one embodiment, the label modification module 940 is further configured to calculate the cost of each side of the extracted sample and determine the target contour based on the cost of each side; select a starting point as a seed point on the extracted sample based on the target contour and obtain the target point; calculate the minimum average cost between the seed point and the target point, determine the segmentation contour based on the minimum average cost, and determine a new label based on the segmentation contour; and perform weighted image fusion of the new label and the extracted sample to obtain the target label.
[0143] In one embodiment, the label modification module 940 is further configured to calculate the whole image cost of the extracted samples and use it as a cost connectivity graph, determine the target region image through the cost connectivity graph, and calculate each corner point of the target region image as each pseudo seed point; use each pseudo seed point as the vertex of the improved livewire algorithm to generate a segmentation plane, and label the pixels contained in the plane as new labels; and perform weighted image fusion with the extracted samples to obtain the target label.
[0144] In one embodiment, the label review module 950 is further configured to calculate only the cost of the region of interest (ROI) that needs to be modified in the pseudo-label according to the pseudo-label modification instruction, fill the cost of the non-ROI region to form a cost connectivity graph with low computational complexity; calculate the pseudo-seed points of the region that needs to be modified in the pseudo-label, and generate a lasso that can be adjusted or added to implement the adjustment, wherein the pseudo-seed points are connected by segmenting the contour with minimum cost; generate a plane as vertices using the adjusted pseudo-seed points, and label the pixels contained in the plane as new labels; and perform weighted image fusion of the new labels and the extracted samples to obtain the target label.
[0145] In one embodiment, the label review module 950 is also used to simultaneously display the target label and the extracted sample in the display interface, obtain the review mark of the target label through the display interface; determine whether the target label has passed the review based on the review mark; if the target label has passed the review, store the target label in the dataset; if the target label has failed the review, store the target label in the label modification database.
[0146] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 11 As shown, the computer device includes a processor, memory, network interface, display screen, and input system connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a sample annotation method for remote sensing images. The display screen can be a liquid crystal display (LCD) or an e-ink display. The input system can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.
[0147] Those skilled in the art will understand that Figure 11 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0148] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:
[0149] Acquire remote sensing images and segment them to obtain remote sensing image samples;
[0150] Extract the sample features of remote sensing image samples, obtain the label generation algorithm corresponding to the sample features, and use the label generation algorithm to generate pseudo-labels corresponding to the remote sensing image samples.
[0151] The pseudo-labels are denoised, and the contours of ground features are extracted from the remote sensing image samples to obtain the extracted samples.
[0152] Obtain the pseudo-label modification instruction, modify the pseudo-label according to the pseudo-label modification instruction and the extracted sample, and obtain the target label;
[0153] The target labels are reviewed and approved before being stored in the dataset.
[0154] In one embodiment, when the processor executes the computer program, it further performs the following steps: using a connected component filtering algorithm to filter the pseudo-labels, identifying and filtering regions in the pseudo-labels whose four-connected regions are less than a threshold, to obtain filtered pseudo-labels; using a median filtering algorithm to smooth the jagged edges in the filtered pseudo-labels, to obtain processed pseudo-labels.
[0155] In one embodiment, when the processor executes the computer program, it also performs the following steps: finding a feature contour extraction tool; extracting feature contours from remote sensing image samples using the feature contour extraction tool to obtain extracted samples.
[0156] In one embodiment, when the processor executes the computer program, it further performs the following steps: displaying an operation interface according to the pseudo-label modification instruction and obtaining the seed point selected through the operation interface; calculating the difference between the pixel value corresponding to the pixel point in the four-connected region and the pixel value of the seed point, with the seed point as the center, and adding the pixel points with a difference less than a threshold into the target region; using the newly added pixel point as the new seed point, and adding new pixels into the target region with the new seed point as the center, until no new pixels are added; if the selected new seed point is a normal seed point, then the new pixel point and the extracted sample are weighted and fused to obtain the target label; if the selected new seed point is an inverse seed point, then the pixels surrounded by the extracted sample are used as the new label and weighted and fused with the extracted sample to obtain the target label.
[0157] In one embodiment, when the processor executes the computer program, it further performs the following steps: calculating the cost of each side of the extracted sample and determining the target contour based on the cost of each side; selecting a starting point as a seed point on the extracted sample based on the target contour and obtaining the target point; calculating the minimum average cost between the seed point and the target point, determining the segmentation contour based on the minimum average cost, and determining a new label based on the segmentation contour; and performing weighted image fusion of the new label and the extracted sample to obtain the target label.
[0158] In one embodiment, when the processor executes the computer program, it further performs the following steps: calculating the whole image cost of the extracted samples and using it as a cost connected graph; determining the target region image through the cost connected graph and calculating each corner point of the target region image as each pseudo-seed point; using each pseudo-seed point as a vertex of the improved livewire algorithm to generate a segmentation plane, and labeling the pixels contained in the plane as new labels; and performing weighted image fusion with the extracted samples using the new labels to obtain the target label.
[0159] In one embodiment, when the processor executes the computer program, it further performs the following steps: calculating only the cost in the Region of Interest (ROI) that needs to be modified in the pseudo-label according to the pseudo-label modification instruction, filling the cost of non-ROI regions to form a cost connectivity graph with low computational complexity; calculating the pseudo-seed points of the regions that need to be modified in the pseudo-label, and generating a lasso that can be adjusted or have additional pseudo-seed points added to perform the adjustment, wherein the pseudo-seed points are connected by segmenting the contour with minimum cost; generating a plane as vertices using the adjusted pseudo-seed points, and labeling the pixels contained in the plane as new labels; and performing weighted image fusion of the new labels and extracted samples to obtain the target label.
[0160] In one embodiment, when the processor executes the computer program, it further performs the following steps: simultaneously displaying the target label and the extracted sample in the display interface, and obtaining the review mark of the target label through the display interface; determining whether the target label has passed the review based on the review mark; if the target label has passed the review, storing the target label in the dataset; if the target label has failed the review, storing the target label in the label modification database.
[0161] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:
[0162] Acquire remote sensing images and segment them to obtain remote sensing image samples;
[0163] Extract the sample features of remote sensing image samples, obtain the label generation algorithm corresponding to the sample features, and use the label generation algorithm to generate pseudo-labels corresponding to the remote sensing image samples.
[0164] The pseudo-labels are denoised, and the contours of ground features are extracted from the remote sensing image samples to obtain the extracted samples.
[0165] Obtain the pseudo-label modification instruction, modify the pseudo-label according to the pseudo-label modification instruction and the extracted sample, and obtain the target label;
[0166] The target labels are reviewed and approved before being stored in the dataset.
[0167] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: using a connected component filtering algorithm to filter the pseudo-labels, identifying and filtering regions in the pseudo-labels whose four-connected regions are less than a threshold, to obtain filtered pseudo-labels; using a median filtering algorithm to smooth the jagged edges in the filtered pseudo-labels, to obtain processed pseudo-labels.
[0168] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: finding a feature contour extraction tool; extracting feature contours from remote sensing image samples using the feature contour extraction tool to obtain extracted samples.
[0169] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: displaying an operation interface according to the pseudo-label modification instruction and obtaining the seed point selected through the operation interface; calculating the difference between the pixel value corresponding to the pixel point in the four-connected region and the pixel value of the seed point, with the seed point as the center, and adding the pixel points with a difference less than a threshold into the target region; taking the newly added pixel point as the new seed point, and adding new pixels into the target region with the new seed point as the center, until no new pixels are added; if the selected new seed point is a normal seed point, then performing weighted image fusion of the new pixel point and the extracted sample to obtain the target label; if the selected new seed point is an inverse seed point, then performing weighted image fusion of the pixels surrounded by the extracted sample as the new label and the extracted sample to obtain the target label.
[0170] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: calculating the cost of each side of the extracted sample and determining the target contour based on the cost of each side; selecting a starting point as a seed point on the extracted sample based on the target contour and obtaining the target point; calculating the minimum average cost between the seed point and the target point, determining the segmentation contour based on the minimum average cost, and determining a new label based on the segmentation contour; and performing weighted image fusion of the new label and the extracted sample to obtain the target label.
[0171] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: calculating the overall image cost of the extracted samples and using it as a cost connected graph; determining the target region image through the cost connected graph and calculating each corner point of the target region image as each pseudo-seed point; using each pseudo-seed point as a vertex of the improved livewire algorithm to generate a segmentation plane, and labeling the pixels contained in the plane as new labels; and performing weighted image fusion with the extracted samples using the new labels to obtain the target label.
[0172] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: calculating the cost of the Region of Interest (ROI) that needs to be modified in the pseudo-label only according to the pseudo-label modification instruction, filling the cost of non-ROI regions to form a cost connectivity graph with low computational complexity; calculating the pseudo-seed points of the regions that need to be modified in the pseudo-label, and generating a lasso that can be adjusted or have additional pseudo-seed points added to perform the adjustment, wherein the pseudo-seed points are connected by segmenting the contour with minimum cost; generating a plane as vertices using the adjusted pseudo-seed points, and labeling the pixels contained in the plane as new labels; and performing weighted image fusion of the new labels and extracted samples to obtain the target label.
[0173] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: simultaneously displaying the target label and the extracted sample in the display interface, and obtaining the review mark of the target label through the display interface; determining whether the target label has passed the review based on the review mark; if the target label has passed the review, storing the target label in the dataset; if the target label has failed the review, storing the target label in the label modification database.
[0174] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.
[0175] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0176] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.
Claims
1. A sample annotation method for remote sensing images, characterized in that, The method includes: Acquire remote sensing images and segment the remote sensing images to obtain remote sensing image samples; Extract the sample features of the remote sensing image sample, obtain the label generation algorithm corresponding to the sample features, and use the label generation algorithm to generate pseudo-labels corresponding to the remote sensing image sample; The noise reduction process for the pseudo-labels includes: applying a connected component filtering algorithm to filter the pseudo-labels, identifying and filtering regions in the pseudo-labels whose four-connected regions are less than a threshold, to obtain filtered pseudo-labels; applying a median filtering algorithm to smooth the jagged edges in the filtered pseudo-labels, to obtain processed pseudo-labels; and extracting ground feature contours from the remote sensing image samples to obtain extracted samples, including: finding a ground feature contour extraction tool; and using the ground feature contour extraction tool to extract ground feature contours from the remote sensing image samples to obtain extracted samples. Obtaining a pseudo-label modification instruction, modifying the pseudo-label according to the pseudo-label modification instruction and the extracted sample to obtain a target label, includes: calculating the overall image cost of the extracted sample and using it as a cost connectivity graph; determining the target region image through the cost connectivity graph and calculating each corner point of the target region image as a pseudo-seed point; using each pseudo-seed point as a vertex of the improved livewire algorithm to generate a segmentation plane, and labeling the pixels contained in the segmentation plane as new labels; performing weighted image fusion of the new labels and the extracted sample to obtain the target label; wherein, the local cost between points p and q in the cost connectivity graph... Determined by the following formula: In the improved livewire algorithm, The Laplace operator function is replaced with the improved Canny operator function, which can more clearly identify edges. The characteristic equation of the improved Canny operator is: ; The cost function is related to the gradient magnitude. For digital images, this is equivalent to finding the gradient of a two-dimensional discrete function, using differences to approximate the partial derivatives. and The gradient magnitude G is equivalent to: To assign low cost to strong edges and high cost to weak edges, the gradient magnitude is multiplied by an inverse linear function, i.e.: The gradient-direction-dependent cost function reflects the changes in gradient direction between nodes, and its expression is: in: , , Let the weights of each factor be denoted as , and the following relationship exists: ; The target tags are reviewed, and once approved, they are stored in the dataset.
2. The sample annotation method for remote sensing images according to claim 1, characterized in that, The step of modifying the pseudo-label according to the pseudo-label modification instruction and the extracted sample to obtain the target label includes: The operation interface is displayed according to the pseudo-label modification instructions, and the seed point selected through the operation interface is obtained; Centered on the seed point, calculate the difference between the pixel value of the pixel point in the four connected regions and the pixel value of the seed point, and add the pixels with a difference less than the threshold into the target region; The newly added pixel is used as a new seed point, and new pixels are added to the target area with the new seed point as the center until no new pixels are added. If the selected new seed point is a normal seed point, the new pixel point is weighted and fused with the extracted sample to obtain the target label; If the selected new seed point is an anti-seed point, then the pixels surrounded by the extracted sample will be used as new labels and weighted image fusion will be performed with the extracted sample to obtain the target label.
3. The sample annotation method for remote sensing images according to claim 1, characterized in that, The step of modifying the pseudo-label according to the pseudo-label modification instruction and the extracted sample to obtain the target label includes: Calculate the cost of each side of the extracted sample, and determine the target contour based on the cost of each side; Based on the target contour, a starting point is selected on the extracted sample as a seed point, and the target point is obtained; Calculate the minimum average cost between the seed point and the target point, determine the segmentation contour based on the minimum average cost, and determine a new label based on the segmentation contour; The new label is then fused with the extracted sample using a weighted image fusion process to obtain the target label.
4. The sample annotation method for remote sensing images according to claim 1, characterized in that, The step of modifying the pseudo-label according to the pseudo-label modification instruction and the extracted sample to obtain the target label includes: According to the pseudo-label modification instruction, only the cost in the region of interest (ROI) that needs to be modified in the pseudo-label is calculated, and the cost of the non-ROI region is filled to form a cost connectivity graph with low computational complexity. Calculate the pseudo-seed points of the regions that need to be modified in the pseudo-labels, and generate a lasso that can be adjusted or have additional pseudo-seed points added to perform the adjustment, wherein the pseudo-seed points are connected by segmenting the contours with minimal cost. The adjusted pseudo-seed points are used as vertices to generate a plane, and the pixels contained in the plane are labeled with new tags; The new label is then fused with the extracted sample using a weighted image fusion process to obtain the target label.
5. A sample annotation system for remote sensing images, characterized in that, The system includes: The image segmentation processing module is used to acquire remote sensing images and segment the remote sensing images to obtain remote sensing image samples. The pseudo-label generation module is used to extract the sample features of the remote sensing image sample, obtain the label generation algorithm corresponding to the sample features, and use the label generation algorithm to generate pseudo-labels corresponding to the remote sensing image sample. The contour extraction module is used to perform noise reduction processing on the pseudo-labels, including: filtering the pseudo-labels using a connected component filtering algorithm, identifying and filtering regions in the pseudo-labels whose four-connected regions are less than a threshold, to obtain filtered pseudo-labels; smoothing the jagged edges in the filtered pseudo-labels using a median filtering algorithm, to obtain processed pseudo-labels; and extracting ground feature contours from the remote sensing image samples to obtain extracted samples, including: finding a ground feature contour extraction tool; and using the ground feature contour extraction tool to extract ground feature contours from the remote sensing image samples to obtain extracted samples. A label modification module is used to obtain pseudo-label modification instructions, modify the pseudo-labels according to the pseudo-label modification instructions and the extracted samples, and obtain target labels. This includes: calculating the overall image cost of the extracted samples and using it as a cost connectivity graph; determining the target region image through the cost connectivity graph and calculating each corner point of the target region image as a pseudo-seed point; using each pseudo-seed point as a vertex of the improved livewire algorithm to generate a segmentation plane, and labeling the pixels contained within the segmentation plane as new labels; and performing weighted image fusion of the new labels and the extracted samples to obtain the target label; wherein, the local cost between points p and q in the cost connectivity graph... Determined by the following formula: In the improved livewire algorithm, The Laplace operator function is replaced with the improved Canny operator function, which can more clearly identify edges. The characteristic equation of the improved Canny operator is: ; The cost function is related to the gradient magnitude. For digital images, this is equivalent to finding the gradient of a two-dimensional discrete function, using differences to approximate the partial derivatives. and The gradient magnitude G is equivalent to: To assign low cost to strong edges and high cost to weak edges, the gradient magnitude is multiplied by an inverse linear function, i.e.: The gradient-direction-dependent cost function reflects the changes in gradient direction between nodes, and its expression is: in: , , Let the weights of each factor be denoted as , and the following relationship exists: ; The tag review module is used to review the target tags and store them in the dataset after the review is passed.
6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 4.
7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.