A method for producing road sample data based on vector graphics

Abstract

The invention discloses a method for producing road sample data based on vector map spots, which belongs to the field of remote sensing technology and information science and technology. The method comprises: performing super-resolution processing on a remote sensing image to obtain a remote sensing image with enhanced road elements; cutting the remote sensing image with enhanced road elements based on prior information of the vector map spots to obtain sample data to produce a background image; producing the background image based on the prior information of the vector map spots and the sample data to obtain a first road element mask; performing semantic segmentation on the background image produced by the sample data using SAM segmentation to obtain a semantic segmentation result; and performing similarity judgment on the semantic segmentation result and the first road element mask to obtain road element sample data.

Description

Technical Field

[0001] The invention belongs to the field of remote sensing technology and information science and technology, and in particular relates to a method for producing road sample data based on vector graphics spots. Background Art

[0002] Deep learning semantic segmentation technology has become one of the supporting technologies for intelligent interpretation of remote sensing images of road elements. However, the primary condition for the success of deep learning semantic segmentation tasks is to produce high-quality road sample data with sufficient quantity, comprehensive types and balanced proportions based on remote sensing images, which usually accounts for more than 60% of the entire intelligent interpretation workload.

[0003] There are two conventional methods for producing road feature sample data: (1) Using geographic information system software (such as ArcGIS, QGis, etc.), through visual interpretation and sketching, manually superimposing vector patches on remote sensing images, and then producing sample data through image cropping and vector rasterization. (2) Using interactive deep learning technology software (such as RITM, EISeg, Labelme, etc.), through interactive methods such as clicking, graffiti, rectangular boxes, extreme points, etc., semi-automatically marking and generating sample labels on remote sensing images, and then producing sample data through remote sensing image and sample label cropping.

[0004] Whether it is a fully manual method or a semi-automatic method, it is inseparable from a large number of human-computer interactions to obtain masks. In the intelligent interpretation tasks of monthly and quarterly remote sensing images, there are often a large number of road elements that change with resolution, season, phase, climate, weather, and dynamics. In view of these situations, it is difficult for human-computer interaction to obtain high-quality road masks with sufficient quantity, comprehensive types, and uniform distribution at low cost and high efficiency.

[0005] The above reasons make it extremely difficult to form intelligent interpretation capabilities of remote sensing images of road elements in a short time and at low cost, which seriously affects the large-scale and engineering application of semantic segmentation deep learning technology in the field of surface cover survey and monitoring. Therefore, the present invention proposes a method for producing road sample data based on vector map spots. Summary of the invention

[0006] In order to solve the above technical problems, the present invention proposes a method for producing road sample data based on vector graphics spots to solve the problems existing in the above prior art.

[0007] To achieve the above object, the present invention provides a method for producing road sample data based on vector map spots, comprising:

[0008] Super-resolution processing is performed on remote sensing images to obtain remote sensing images with enhanced road features;

[0009] Based on the prior information of the vector map, the remote sensing image enhanced by the road element is cut to obtain sample data to produce a background image;

[0010] A background image is produced based on the prior information of the vector map patch and the sample data to obtain a first road element mask;

[0011] Using SAM segmentation to perform semantic segmentation on the background image produced by the sample data to obtain a semantic segmentation result;

[0012] A similarity judgment is performed on the semantic segmentation result and the first road element mask to obtain road element sample data.

[0013] Optionally, the process of obtaining the remote sensing image with enhanced road features includes:

[0014] Performing super-resolution enhancement on the remote sensing image based on VQGAN to obtain a first super-resolution image;

[0015] Performing super-resolution enhancement on the remote sensing image based on VDSR to obtain a second super-resolution image;

[0016] The first super-resolution image and the second super-resolution image are weightedly fused to obtain a remote sensing image with enhanced road elements.

[0017] Optionally, the calculation formula of the road element enhanced remote sensing image is:

[0018] SRIMG=a*VQGAN_IMG+(1-a)*VDSR_IMG

[0019] Where SRIMG is the remote sensing image with enhanced road features, VQGAN_IMG is the 4x super-resolution image based on VQGAN technology, VDSR_IMG is the super-resolution image based on VDSR technology, and a is the image weighting coefficient.

[0020] Optionally, the prior information of the vector patch includes the patch road element type and the polygonal vector boundary; and the process of cutting the road element enhanced remote sensing image based on the prior information of the vector patch to obtain sample data and produce the background image includes:

[0021] Select road-related land-type patches based on the prior attribute data of the vector patches;

[0022] The maximum outer intercepted rectangle of the land type patch is divided into blocks and screened to obtain the final patch;

[0023] After the remote sensing image enhanced with the road elements is cropped based on the polygonal vector boundary of the final image patch, sample data is obtained by screening based on the road element type of the image patch to produce a background image.

[0024] Optionally, the calculation formula for dividing the maximum outer intercepted rectangle of the land type patch into blocks is:

[0025] M=floor(H p / R / w)*round(W p / R / w)

[0026] Where M is the number of sub-block sets, floor is the rounding function, and H p and W p are the height and width of the maximum circumscribed rectangle, R is the spatial resolution of the remote sensing image enhanced with road features, and w is the number of pixels in the sample image length.

[0027] Optionally, the process of obtaining the first road feature mask includes:

[0028] Based on the sample data, a background image is produced to obtain a corresponding image spot outer intercept rectangle;

[0029] Based on the image spot outer interception rectangle, the SAM model is used to segment the background image produced by the sample data to obtain a background image semantic segmentation result;

[0030] Detecting the sample data based on the GroundingDINO model to produce a background image to obtain a road sample target object;

[0031] The road sample target object and the background image semantic segmentation result are superimposed to obtain a first road element mask.

[0032] Optionally, the segmentation granularity index of the SAM model is PSN, and the calculation formula of PSN is:

[0033] PSN = int((L min +L max )*1.414) / R

[0034] Where, L min is the minimum road width, L max is the maximum road width, int is the rounding function, and 1.414 means the pixel width considering the oblique road.

[0035] Optionally, the process of obtaining road feature sample data includes:

[0036] Selecting the mask of the semantic segmentation result to obtain a second road element mask;

[0037] Calculating a difference in spectral characteristics between the first road feature mask and the second road feature mask;

[0038] Road element sample data is obtained based on the spectral feature differences.

[0039] Compared with the prior art, the present invention has the following advantages and technical effects:

[0040] The present invention discloses a method for producing road sample data based on vector map spots, which includes super-resolution processing of remote sensing images to obtain remote sensing images with enhanced road elements; cutting the remote sensing images with enhanced road elements based on prior information of vector map spots to obtain sample data to produce background images; producing background images based on prior information of vector map spots and sample data to obtain a first road element mask; performing semantic segmentation on the background images produced by sample data using SAM segmentation to obtain a semantic segmentation result; and performing similarity judgment on the semantic segmentation result and the first road element mask to obtain road element sample data. Based on this, the present invention overcomes the limitations of current road element sample data production.

[0041] The technical effects of the present invention also include: (1) The present invention fully utilizes the prior category attributes of the existing vector image patches to improve the category accuracy of the road element sample data; (2) The present invention fully utilizes the prior boundaries of the existing vector image patches to reduce the complexity of the SAM segmentation model for processing images, thereby improving the area accuracy of the road element sample data; (3) The present invention performs mask screening within the same background image, thereby greatly limiting the sample complexity of the same spectrum heterogeneous objects. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0043] Figure 1 It is a flow chart of a method for producing road sample data based on vector graphics spots according to an embodiment of the present invention. DETAILED DESCRIPTION

[0044] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0045] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0046] Embodiment 1

[0047] The purpose of the present invention is to provide a road element sample data production based on vector spots, so as to achieve low-cost, high-efficiency, and high-quality construction of large-scale road element sample data production capabilities. In this embodiment, a road sample data production method based on vector spots is provided, including: based on image super-resolution and feature fusion technology, a remote sensing image with enhanced road elements is obtained; based on the existing prior attributes and range data of the vector spots, the remote sensing image is screened and cut using the road element type and the spot boundary to obtain sample data to produce a background image; using the existing prior spot range of the vector spots, the spot outer rectangle is used as the interactive input of the SAM segmentation model to obtain the background image semantic segmentation result 1; using the road element type name as text input, based on the Grounding DINO unsupervised target detection model, the road element target is detected, and it is superimposed with the background image semantic segmentation result 1 and the repeated area is selected as the first road element mask; according to the first road element mask, using the spot feature similarity, similar spots are selected on the result of the global segmentation of the SAM segmentation model to obtain road elements outside the polygon.

[0048] like Figure 1 As shown, as a specific implementation scheme of this scheme, the present invention provides a method for producing road element sample data based on vector map spots, including:

[0049] Super-resolution processing is performed on the remote sensing image to obtain a remote sensing image with enhanced road features; the specific implementation process is as follows:

[0050] (1) Obtaining remote sensing images with enhanced road features based on image super-resolution and feature fusion technology;

[0051] The remote sensing images are processed by super-resolution based on VQGAN (Vector Quantized Generative Adversaral Network) and VDSR (Very Deep Super Resolution) technologies, and weighted feature fusion is performed on the two super-resolution images to obtain a remote sensing image with enhanced road features. This image can achieve a balance between improving clarity and suppressing false information, and improve the information representation of road features in the image.

[0052] VQGAN is a generative model based on GAN. Although it can convert images into high-quality images, it is essentially learning the mapping relationship between the low-resolution and high-resolution context information of the original images and their high-quality images obtained from other channels, and then migrating the mapping relationship to the input low-resolution images, and generating higher-resolution high-quality images. The super-resolution images generated by this method have good clarity, but when the original images and their high-quality images obtained from other channels have incomplete coverage and unbalanced types for the representation of road element images, the generated high resolution will contain more false information, which will greatly affect the production accuracy of road element samples.

[0053] VDSR is a mean square error control model for infinite image magnification. It learns the mean square error loss of low-resolution-high-resolution context information of original images and their high-quality images obtained from other channels, and then migrates the best super-resolution model to the input low-resolution image, and generates a higher-resolution high-quality image. The overall goal of this method is to increase the details and control the signal-to-noise ratio after image magnification. The generated super-resolution image retains the original features of the object well, but the clarity is generally not high.

[0054] SRIMG=a*VQGAN_IMG+(1-a)*VDSR_IMG

[0055] Among them, SRIMG is the remote sensing image with enhanced road features, VQGAN_IMG is the 4x super-resolution image based on VQGAN technology, VDSR_IMG is the super-resolution image based on VDSR technology, and a is the image weighting coefficient.

[0056] Based on the prior information of the vector map, the remote sensing image enhanced with road features is cropped to obtain sample data to produce the background image; the specific implementation process is as follows:

[0057] (2) Based on the prior information of the vector image patch, the feature type and patch boundary of the patch are used to screen and crop the remote sensing image SRIMG enhanced with road features to obtain sample data to produce the background image;

[0058] 1) Select the patches of road-related land types in the existing vector data.

[0059] P1={P j ,j∈field in(DLBM,DLDM,LXBH,......road-related land class code,or DLMC,LBMC......road-related land class, etc.)}.

[0060] Among them, P1 is the vector data of road-related land class maps, P is the existing vector data, field is the attribute domain, (DLBM, DLDM, LXBH,...) is the category code attribute column, (DLMC, LBMC,...) is the category name attribute column, and or is or.

[0061] 2) Screen the spots according to the spatial distribution of P1 spots.

[0062] The maximum circumscribed rectangle of P1 is divided into blocks, and the number of blocks is:

[0063] M=floor(H p / R / w)*round(W p / R / w)

[0064] Where M is the number of sub-block sets, floor is the rounding function, and H p and W p are the height and width of the maximum circumscribed rectangle of P1, R is the spatial resolution of SRIMG, and w is the number of pixels in the sample image length.

[0065] According to the number of patches existing in all blocks of P1 and the number of required samples, the minimum value of the two is taken to determine the number of screening prior patches.

[0066] N=min(N P1 ,N Wanted )

[0067] Among them, N is the number of samples to be screened, min is the minimum function, and N P1 is the number of spots in all blocks of P1, N Wanted is the number of samples required.

[0068] From the M sub-blocks, N spots are selected on average, and there is at most one spot in each sub-block. Considering that each sub-block often contains a different number of spots, in order to make N points have an equal chance of being selected, [0,1] is divided into M parts, each part of length l i / ∑il i , where l i is the number of spots in the ith sub-block. In the process of selecting spots, a number between [0,1] is randomly generated, and the sequence number of the sub-block selected this time is determined according to the position of the random number in the l parts. Then a spot is randomly selected from the sub-block and set as a spot that does not meet the conditions, so that the same spot will not be selected in the subsequent spot selection process, and then repeated N times until all spots are selected.

[0069] 3) According to the polygonal vector boundary of each selected patch, the SRIMG is cropped with the center point of the polygon as the center and w pixels as the window size to obtain sample data to produce the background image.

[0070] Based on the prior information of the vector map and the sample data, a background image is produced to obtain the first road element mask; the specific implementation process is as follows:

[0071] (3) Using the existing polygons of the vector map, obtain the road elements within the polygons;

[0072] A background image is created for the sample data, and its maximum circumscribed rectangle in the corresponding polygon is obtained. The rectangle is used as the input of the rectangular box for interactive deep learning. The SAM segmentation model is adopted to segment the sample objects in the rectangular box from the background image created by the sample data, and the result is counted as the semantic segmentation result. PSN (Point Side Number) is the segmentation granularity indicator of the SAM segmentation model, which directly determines the level of detail of the segmented object. In order to ensure the SAM segmentation effect, the PSN value is set using the maximum and minimum road widths.

[0073] PSN = int((L min + Lmax )*1.414) / R

[0074] Among them, L min is the minimum road width, L max is the maximum road width, int is the rounding function, and 1.414 is the width of the oblique road pixel.

[0075] Taking the road feature type name 'road' as the text input for interactive deep learning, the GroundingDINO model is used to detect road sample objects in the background image made from sample data, superimpose the semantic segmentation results, and select the repeated area as the first road feature mask.

[0076] SAM segmentation is used to produce a background image for the sample data and perform semantic segmentation to obtain a semantic segmentation result; similarity judgment is performed on the semantic segmentation result and the first road element mask to obtain road element sample data. The specific implementation process is as shown in step (4):

[0077] (4) Based on the first road element mask and utilizing the similarity of the image patches, similar image patches are selected from the results of the global segmentation of all large SAM models to obtain the road elements outside the polygon.

[0078] The SAM segmentation model is adopted to semantically segment the background image and obtain the semantic segmentation result of the background image. The mask in the semantic segmentation result of the background image outside the polygon is selected to obtain the second road element mask. The difference in spectral mean variance characteristics between the characteristics of the second road element mask and the road element mask one is calculated to obtain the road element sample data.

[0079] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for producing road sample data based on vector map spots, characterized in that: include: Super-resolution processing is performed on remote sensing images to obtain remote sensing images with enhanced road features; Based on the prior information of the vector map, the remote sensing image enhanced by the road element is cut to obtain sample data to produce a background image; A background image is produced based on the prior information of the vector map patch and the sample data to obtain a first road element mask; Using SAM segmentation to perform semantic segmentation on the background image produced by the sample data to obtain a semantic segmentation result; Performing similarity judgment on the semantic segmentation result and the first road element mask to obtain road element sample data; The process of obtaining the first road feature mask includes: Based on the sample data, a background image is produced to obtain a corresponding image spot outer intercept rectangle; Based on the image spot outer interception rectangle, the SAM model is used to segment the background image produced by the sample data to obtain a background image semantic segmentation result; Detecting the sample data based on the GroundingDINO model to produce a background image to obtain a road sample target object; Superimposing the road sample target object and the background image semantic segmentation result to obtain a first road element mask; The segmentation granularity index of the SAM model is PSN, and the calculation formula of PSN is: PSN=int((L min +L max )*1.414) / R Where, L min is the minimum road width, L max is the maximum road width, int is the rounding function, and 1.414 means the pixel width considering the oblique road.

2. The method for producing road sample data based on vector map spots according to claim 1 is characterized in that: The process of obtaining remote sensing images with enhanced road features includes: Performing super-resolution enhancement on the remote sensing image based on VQGAN to obtain a first super-resolution image; Performing super-resolution enhancement on the remote sensing image based on VDSR to obtain a second super-resolution image; The first super-resolution image and the second super-resolution image are weightedly fused to obtain a remote sensing image with enhanced road elements.

3. The method for producing road sample data based on vector map spots according to claim 2 is characterized in that: The calculation formula of the remote sensing image enhanced by the road element is: SRIMG=a*VQGAN_IMG+(1-a)*VDSR_IMG Where SRIMG is the remote sensing image with enhanced road features, VQGAN_IMG is the 4x super-resolution image based on VQGAN technology, VDSR_IMG is the super-resolution image based on VDSR technology, and a is the image weighting coefficient.

4. The method for producing road sample data based on vector map spots according to claim 2 is characterized in that: The prior information of the vector patch includes the patch road element type and the polygon vector boundary; the process of cutting the road element enhanced remote sensing image based on the prior information of the vector patch to obtain sample data and prepare the background image includes: Select road-related land-type patches based on the prior attribute data of the vector patches; The maximum outer intercepted rectangle of the land type patch is divided into blocks and screened to obtain the final patch; After the remote sensing image enhanced with the road elements is cropped based on the polygonal vector boundary of the final image patch, sample data is obtained by screening based on the road element type of the image patch to produce a background image.

5. The method for producing road sample data based on vector map spots according to claim 4 is characterized in that: The calculation formula for dividing the maximum outer intercepted rectangle of the land type map into blocks is: M=floor(H p / R / w)*round(W p / R / w) In the formula, M is the number of sub-block sets, floor is the rounding function, and H p and W p are the height and width of the maximum circumscribed rectangle, R is the spatial resolution of the remote sensing image enhanced with road features, and w is the number of pixels in the sample image length.

6. The method for producing road sample data based on vector map spots according to claim 1, characterized in that: The process of obtaining road feature sample data includes: Selecting the mask of the semantic segmentation result to obtain a second road element mask; Calculate the spectral feature difference between the first road feature mask and the second road feature mask; and obtain road feature sample data based on the spectral feature difference.

Images