Method for determining seam line of orthoimages, method for generating mosaic image of orthoimages, and device and medium

Abstract

The present application relates to the field of remote sensing image data processing, and in particular to a method for determining a seam line of orthoimages, a method for generating a mosaic image of orthoimages, and a device and a medium. The method for determining a seam line of orthoimages comprises: using a crawling search method to respectively extract valid area contours of a master image and a slave image; calculating difference values of valid pixels within an overlapping area between the master image and the slave image by means of a difference function, so as to obtain a difference map of a valid area, and if the number of intersection points is greater than 2, calculating the position of a centroid of the overlapping area on the basis of the coordinates of each contour point of an intersection contour, then determining an internal local window centered on the centroid in the difference map, searching the internal local window, and using a position with the minimum difference value as an optimization end point and the intersection points as optimization start points; and using an A star algorithm to determine a seam line on the basis of the optimization start points and the optimization end point. The present application can more accurately determine a seam line of a plurality of orthorectified remote sensing images with irregular contours.

Description

Orthophoto mosaic line determination method, mosaic image generation method, device and medium TECHNICAL FIELD

[0001] The present application relates to the field of remote sensing image data processing, and in particular to an orthophoto mosaic line determination method, a mosaic image generation method, a device and a medium. BACKGROUND

[0002] With the rapid development of aerospace remote sensing technology and the wide application of geographic information systems, ground observation by optical or synthetic aperture radar (SAR) remote sensing satellites has become the main way to obtain geographic information. Among them, the orthophoto image has the advantages of high precision, simple recognition, rich readable geographic information, etc., and plays a very important role in map making, environment detection, land use planning and disaster monitoring. The basic steps of generating an orthophoto image include image preprocessing, image matching, image positioning, image stitching and image output. Among them, image stitching is one of the key steps of orthophoto production, and its role is to stitch multiple remote sensing images with overlapping areas into an orthophoto image after processing. At the current development stage, seamless stitching of automatic production of orthophoto images is a difficult problem in photogrammetry and remote sensing development. Due to the differences in imaging parameters of different sensors, the complexity of the relationship between ground objects, and the changes in ground objects caused by different shooting times, different remote sensing images may have some inconsistencies in the overlapping area. Therefore, mosaic lines are needed to avoid these areas with significant differences, so as to avoid obvious geometric misalignment and color difference on the final mosaic image. However, how to automatically generate mosaic lines is a key problem in the automatic production of orthophoto images.

[0003] At present, in the research of remote sensing image mosaic, the common research idea is: firstly, a global initial seam line grid is generated, and then local optimization of the mosaic line is carried out according to the grid nodes. The initial seam line grid is the basis and premise of the local optimization of the seam line, and the complexity of the grid has a direct impact on the local optimization of the seam line. The generation method of the initial seam line grid can be roughly divided into two categories: geometric method and pixel method. At present, the mainstream is the geometric method, and there is a basic premise condition for using the geometric method, that is: the effective area of the orthographic image and the overlapping area between the images can be represented by a convex quadrilateral or an approximate convex quadrilateral. For the complex overlapping condition of any convex polygon or concave polygon, the geometric method cannot guarantee that the generated mosaic line is in the overlapping area of the adjacent orthographic images. This increases the difficulty of the subsequent local optimization algorithm of the mosaic line, and even leads to the failure of the optimization algorithm. On the other hand, the geometric method uses a limited number of nodes to represent the contour of the effective area of the remote sensing image. When the contour is a relatively regular quadrilateral, only four corner points of the effective area can be used to describe the contour of the area. However, when processing the orthographic images generated by SAR, the contour of the remote sensing image is irregular in most cases. At this time, using a small number of nodes to approximate the contour can easily lead to the misjudgment of part of the invalid pixels or the valid pixels into the effective area or the invalid area, resulting in the existence of holes in the mosaic result or the loss of part of the information of the image.

[0004] Therefore, how to more accurately determine the mosaic line in a plurality of orthographic remote sensing images with irregular contours is a problem to be solved.

[0005] SUMMARY

[0006] The application provides a method for determining an orthographic image mosaic line, which can more accurately determine the mosaic line in a plurality of orthographic remote sensing images with irregular contours.

[0007] In order to achieve the above purpose, the application adopts the following technical scheme:

[0008] In a first aspect, the application provides a method for determining an orthographic image mosaic line, which comprises:

[0009] Pretreating the orthographic images to obtain a master image and a slave image, wherein the slave image has an overlapping area with the master image;

[0010] Extracting the effective area contour of the master image and the slave image by using a crawling search method;

[0011] Determining the intersection of the effective area contours of the master image and the slave image;

[0012] Obtaining the intersection contour of the master image and the slave image;

[0013] The difference values ​​of effective pixels within the overlapping area of ​​the main image and the secondary image are calculated using a difference function to obtain a difference map of the effective area. If the number of intersection points is greater than 2, the position of the centroid of the overlapping area is calculated based on the coordinates of each contour point of the intersection contour. Then, an inner local window is determined in the difference map with the centroid as the center. The inner local window is searched, and the position with the smallest difference value is taken as the optimization endpoint, and the intersection point is taken as the optimization starting point. If the number of intersection points is equal to 2, they are taken as the optimization starting point and optimization endpoint, respectively.

[0014] Using the A* algorithm, the mosaic lines of the main image and the secondary image are determined based on the optimization start point and optimization end point.

[0015] In a preferred embodiment of this application, the preprocessing of the orthophoto image may further include:

[0016] Obtain the latitude and longitude coordinates corresponding to the index values ​​of each pixel in the orthophoto image to obtain the reference latitude and longitude of this orthophoto image;

[0017] The orthophotos are sorted based on the reference latitude and longitude.

[0018] In a preferred embodiment of this application, the preprocessing of the orthophoto image may further include:

[0019] Through interpolation, pixels from different orthophotos are represented in the same index coordinate system;

[0020] Set the latitude and longitude resolution of different orthophotos to be the same.

[0021] In a preferred embodiment of this application, determining the intersection point of the effective region contours of the main image and the secondary image may further include:

[0022] Obtain the contour points with the same index coordinates that exist in the effective region contours of the main image and the secondary image, and obtain the intersection points.

[0023] In a preferred example of this application, it can be further configured to include:

[0024] A contour point quick lookup table is constructed based on a hash table and a list, and the contour points of the effective region contour are stored through the contour point quick lookup table.

[0025] During the crawling search process, the searched contour points are determined by the contour point quick lookup table to see if they are on an existing contour line. If they are, the contour line is clipped.

[0026] In a preferred example of this application, it can be further configured to include:

[0027] When the search direction of the crawling search method when detecting the valid pixel is not parallel to one of the pixel grid lines of the orthographic image, the valid pixel is determined to be a diagonal virtual contour point, and the valid pixel is taken as a contour point of the effective area contour;

[0028] An intersection point is obtained by comparing the contour points of the main image and the slave image in the contour point lookup table.

[0029] In a preferred example of the present application, the intersection contour of the main image and the slave image can be further obtained by:

[0030] The position relationship between the contour points of the effective area contour and the grid points is determined.

[0031] A starting point is determined, and the starting point is located in the overlapping area of the main image and the slave image and belongs to only one of the contour lines.

[0032] Based on the position relationship, the intersection contour is obtained by traversing the contour points of the main image and the slave image from the starting point.

[0033] In a preferred example of the present application, the intersection contour of the main image and the slave image can be further obtained by:

[0034] When there are multiple overlapping areas of the main image and the slave image, the starting point of each overlapping area is determined by a contour starting point determination algorithm.

[0035] In a preferred example of the present application, the intersection contour of the main image and the slave image can be further obtained by:

[0036] When the effective area contour of the main image and the slave image has a large number of redundant intersection points due to coincidence or proximity in some areas, the redundant intersection points are merged by using a contour intersection point merging algorithm.

[0037] In a preferred example of the present application, the intersection contour of the main image and the slave image can be further obtained by:

[0038] The orthographic image is preprocessed to obtain a main image and a slave image, and the slave image has an overlapping area with the main image.

[0039] The effective area contour of the main image and the slave image is extracted by using a crawling search method.

[0040] The intersection points of the effective area contour of the main image and the slave image are determined.

[0041] The intersection contour of the main image and the slave image is obtained.

[0042] The difference values of the effective pixels in the overlapping area of the main image and the slave image are calculated by a difference function to obtain a difference map of the effective area; if the number of the intersection points is greater than 2, the position of the centroid of the overlapping area is calculated according to the coordinates of the profile points of the intersection profile, then an inner local window is determined in the difference map with the centroid as the center, the inner local window is searched, the position with the minimum difference value is taken as the optimization end point, and the intersection points are taken as the optimization start points; if the number of the intersection points is equal to 2, the intersection points are respectively taken as the optimization start point and the optimization end point;

[0043] The inlay line of the main image and the slave image is determined based on the optimization start point and the optimization end point by using an A-star algorithm; the pixels near the inlay line are feathered, and then the weight values of the main image and the slave image in the overlapping area are calculated;

[0044] The main image and the slave image are weightedly fused based on the weight values to obtain an inlaid image.

[0045] In a preferred example of the present application, before the pixels near the inlay line are feathered, the following steps can be further included:

[0046] The attribution information of the intersection points in the intersection profile is obtained;

[0047] After the inlay line is generated, the attribution of the area in the overlapping area of the main image and the slave image which is divided according to the inlay line is judged based on the attribution information.

[0048] In a third aspect, the present application provides a computer device, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the orthographic image inlay line determination method according to any one of the above aspects when executing the computer program.

[0049] In a fourth aspect, the present application provides a computer device, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the orthographic image inlaid image generation method according to any one of the above aspects when executing the computer program.

[0050] In a fifth aspect, the present application provides a computer readable storage medium, which stores a program, and the program is executable on a processor to implement the orthographic image inlay line determination method according to any one of the above aspects.

[0051] In a sixth aspect, the present application provides a computer readable storage medium, which stores a program, and the program is executable on a processor to implement the orthographic image inlaid image generation method according to any one of the above aspects.

[0052] To sum up, compared with the prior art, the technical scheme provided by the embodiment of the application has at least the following beneficial effects:

[0053] The method for determining the orthographic image mosaic line provided by the application uses a pixel-based contour generation method for the orthographic image, ensures that the generated mosaic line is in the overlapping area of adjacent orthographic images, is conducive to the execution of the optimization algorithm, can accurately preserve the geometric information of the image and its overlapping area, is suitable for the determination of the mosaic line of remote sensing images with various complex overlapping conditions and arbitrary concave-convex polygon contours, and has stronger universality than mainstream methods. By using the crawling search method to search for the effective contour, the method is more suitable for the extraction of irregular concave-convex polygon contours, avoids the situation that the mosaic result has a hole or part of the information of the image is lost due to the fact that part of the invalid pixels are misjudged into the effective area or the effective pixels are misjudged into the invalid area, makes the determination result of the mosaic line more accurate, and improves the determination efficiency. Considering that the overlapping area can be an arbitrary concave-convex polygon, the application adopts the "centroid" and "local search" methods to determine the position of the optimization end point, so that the generation end point of the mosaic line is more accurate. The difference of the image overlapping area is quantitatively represented as a "difference map" through the "difference function", the process of determining the optimal mosaic line is converted into the process of finding the optimal path from the optimization start point to the optimization end point by means of the A-star algorithm, the complex overlapping area is divided by the mosaic line, and the accuracy of the determination of the mosaic line is improved. BRIEF DESCRIPTION OF DRAWINGS

[0054] Fig. 1 is a flowchart of a method for determining an orthographic image mosaic line provided by an embodiment of the application.

[0055] Fig. 2 is a schematic diagram of the azimuth angle distribution of adjacent grids of a crawling search method of a method for determining an orthographic image mosaic line provided by an embodiment of the application.

[0056] Fig. 3 is a process schematic diagram of a crawling search method of a method for determining an orthographic image mosaic line provided by an embodiment of the application.

[0057] Fig. 4 is a schematic diagram of a mosaic line provided by a method for determining an orthographic image mosaic line provided by an embodiment of the application.

[0058] Fig. 5 is a schematic diagram of a contour quick search provided by a method for determining an orthographic image mosaic line provided by an embodiment of the application.

[0059] Fig. 6 is a contour line clipping situation diagram of a method for determining an orthographic image mosaic line provided by an embodiment of the application.

[0060] Fig. 7 is a schematic diagram of the intersection type of a method for determining an orthographic image mosaic line provided by an embodiment of the application.

[0061] FIG. 8 is a schematic diagram of the position relationship between the contour point and the grid point in a method for determining a mosaic line of orthographic images according to an embodiment of the present application.

[0062] FIG. 9 is a schematic diagram of the relationship between the next contour point and the current contour point in a method for determining a mosaic line of orthographic images according to an embodiment of the present application.

[0063] FIG. 10 is a schematic diagram of the main image and the slave image in a method for determining a mosaic line of orthographic images according to an embodiment of the present application.

[0064] FIG. 11 is a schematic diagram of the relationship between the number of times of leaving the overlapping area and the number of overlapping areas of the main image and the slave image in a method for determining a mosaic line of orthographic images according to an embodiment of the present application.

[0065] FIG. 12 is a schematic diagram of the merging of the intersection points of the dense contours in a method for determining a mosaic line of orthographic images according to an embodiment of the present application.

[0066] FIG. 13 is a main image weight map in a method for generating a mosaic image of orthographic images according to an embodiment of the present application.

[0067] FIG. 14 is a mosaic image of the main image and the slave image in a method for generating a mosaic image of orthographic images according to an embodiment of the present application.

[0068] FIG. 15 is a schematic diagram of the attribution judgment of the overlapping area in a method for generating a mosaic image of orthographic images according to an embodiment of the present application. DETAILED DESCRIPTION

[0069] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all the other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0070] In an embodiment of the present application, a method for determining a mosaic line of orthographic images is provided. Please refer to FIG. 1, the method is executed sequentially from S100 to S600, comprising:

[0071] S100: pre-processing the orthographic images to obtain a main image and a slave image, the slave image and the main image have an overlapping area;

[0072] Specifically, the orthographic image is one of geographic information data obtained by earth observation of an optical or synthetic aperture radar (SAR) remote sensing satellite. The master image is an image that has been processed and can come from the result of the last time splicing. The slave image is an image used for splicing on the master image and comes from an unprocessed image in the orthographic image. After the preprocessing, the two have overlapping parts.

[0073] S200: Extracting the effective area contour of the master image and the slave image respectively using a crawling search method;

[0074] Specifically, the crawling search method includes the following steps: first, for one of the master image or the slave image, find the coordinates of the first effective pixel by searching row by row and column by column, and take the pixel as the starting point of the "crawling search". This pixel is also the starting point of the contour line of the effective area contour. The effective pixel is a pixel that can accurately reflect the shape, size and position of the ground object. The determination standard of the effective pixel can be preset according to the specific application requirement and the data quality.

[0075] As shown in FIG. 2, the position relationship between the outer 8 grids of the 3x3 sub-block and the search starting point is described using the azimuth angle, and the azimuth angle distribution is 0° azimuth along the positive and negative directions of the x-axis, and the remaining 7 azimuths are sequentially increased by 45° along the counterclockwise direction.

[0076] FIG. 3 demonstrates the process of "crawling search", wherein (a) is a local search schematic diagram, and the effective area, contour point and invalid area in the grid or pixel are shown as follows: the effective area is an area composed of effective pixels of the orthographic image, and the contour point divides the effective area and the invalid area. Let the search azimuth of the last iteration be θ pre , then the initial search azimuth of the current iteration is θ cur = mod(θ pre + 45°, 360°), that is, counterclockwise rotation by 45°; in particular, let θ pre = 45° at the first iteration. Starting from the initial search azimuth, the adjacent grids of the 8 azimuths with the search starting point are judged in sequence along the clockwise direction, and after each judgment, the search azimuth needs to be rotated by 45° along the clockwise direction, that is, θ cur = mod(θ cur- 45°, 360°); when the valid pixel value is detected for the first time, the current search ends, and the grid coordinates and search orientation of the valid pixel detected at this time are saved, which prepares for the starting point of the search and the initialization of the search orientation of the next iteration. When the detected contour point coincides with the contour starting point, the iteration terminates, and then the contour line composed of the set of contour points is obtained, at which time the contour extraction of the valid region of the image is completed. The contour line is taken as the valid region contour. As shown in (b) of FIG. 3, the overall search is schematically shown, as shown in the figure, the search starting point is determined row by row and column by column, and the iteration terminates when the contour starting point is detected.

[0077] S300: determining the intersection point of the valid region contours of the main image and the slave image;

[0078] Specifically, after the valid region contours of the main image and the slave image are extracted respectively in step S200, the contour points of the intersection position of the two valid region contours in the same coordinate system are determined, and the intersection point is obtained.

[0079] S400: obtaining the intersection contour of the main image and the slave image;

[0080] Specifically, the intersection contour is a contour point that is located inside or on the contour line of both the main image and the slave image.

[0081] S500: calculating the difference value of the valid pixels in the overlapping region of the main image and the slave image by a difference function to obtain a difference map of the valid region; if the number of intersection points is greater than 2, the position of the centroid of the overlapping region is calculated according to the coordinates of each contour point of the intersection contour, and then a local window is determined in the difference map with the centroid as the center, the inner local window is searched, the position with the minimum difference value is taken as the optimization end point, and the intersection point is taken as the optimization starting point; if the number of intersection points is equal to 2, they are respectively taken as the optimization starting point and the optimization end point;

[0082] Specifically, first, the difference value of the valid pixels in the overlapping region of the two images is calculated by a difference function to obtain a difference map of the valid region. The overlapping region is the overlapping part of the valid regions of the main image and the slave image. The difference function is a function that quantitatively describes the difference between the two images. The specific form of the function can be replaced according to the requirements, as long as it satisfies "the difference function value is positively correlated with the image difference", that is, the greater the image difference, the greater the difference function value. For example, the difference function is defined as:

[0083] wherein I1 and I2 are respectively the pixel values corresponding to the main image and the slave image, represents the gradient in the x direction of the pixel value, represents the gradient of the y direction of the pixel value.

[0084] In order to avoid the generated mosaic line coinciding with the intersection contour, a larger value is assigned to the pixel points located on the intersection contour, which needs to be greater than the maximum value of the difference map.

[0085] Then, the optimization end point is calculated in different ways according to the number of intersection points. If the intersection points of the intersection contour are not more than 2, one of the intersection points is directly selected as the optimization start point and the other is the optimization end point. If the intersection points of the intersection contour are more than 2, the "centroid" and "local search" methods are used to determine the position of the optimization end point, and the steps include: first, the centroid position of the overlapping area is calculated according to the coordinates of each contour point of the intersection contour; then, the centroid is taken as the center to search in the local window within a certain search width, and the position with the minimum difference value is found as the final "optimization end point", and the width is adjusted according to the image size. When there are multiple intersection points of the intersection contour, the optimization end point obtained by this calculation method successfully divides the complex overlapping area into several regions, and at this time there are multiple optimization start points and one optimization end point.

[0086] S600: using A-star algorithm to determine the mosaic line of the master image and the slave image based on the optimization start point and the optimization end point.

[0087] Specifically, the intersection point of each intersection contour is taken as the optimization start point, and the A-star algorithm is used to find the optimal path from the optimization start point to the optimization end point on the difference map, which is the mosaic line. The A-star algorithm is a heuristic search algorithm. This algorithm combines the completeness of Dijkstra algorithm and the high efficiency of heuristic search, and can find the optimal path in directed graph or weighted graph, and is widely used in graph search and artificial intelligence field. As shown in FIG. 4, (a) and (b) are the generated mosaic lines of the master image and the slave image in the case of having two intersection points and having multiple intersection points, respectively.

[0088] In this embodiment, the pixel-based contour generation method is used for orthographic image, which ensures that the generated tessellation line is in the overlapping area of adjacent orthographic images, is conducive to the execution of the optimization algorithm, can accurately retain the geometric information of the image and its overlapping area, and is suitable for tessellation line determination of remote sensing images with various complex overlapping conditions and arbitrary concave-convex polygon contours. Compared with mainstream methods, it has stronger universality. By using the crawling search method to search for effective contours, it is more suitable for irregular concave-convex polygon contour extraction, avoids the situation that part of the invalid pixels are misjudged into the effective area or the effective pixels are misjudged into the invalid area, resulting in the existence of holes in the tessellation result or the loss of part of the image information, makes the tessellation line determination result more accurate, and improves the determination efficiency; considering that the overlapping area can be an arbitrary concave-convex polygon, the application adopts the "centroid" and "local search" methods to determine the position of the optimization end point, so that the generation end point of the tessellation line is more accurate. The difference between the image overlapping area is quantitatively represented as a difference map by using the "difference function", and the process of determining the optimal tessellation line is converted into the process of finding the optimal path from the optimization starting point to the optimization end point by means of A-star algorithm, which realizes the division of the tessellation line for the complex overlapping area and improves the accuracy of the tessellation line determination.

[0089] In some embodiments, the preprocessing of the orthographic image comprises:

[0090] Obtaining the longitude and latitude coordinates corresponding to the index value of each pixel of the orthographic image to obtain the reference longitude and latitude of the orthographic image.

[0091] Sorting the orthographic image based on the reference longitude and latitude.

[0092] In specific implementation, since each grid, i.e. pixel, of the orthographic image is essentially the response characteristic of a certain geographic area in the imaging system, each grid contains the geographic information of the position, i.e. the longitude and latitude represented by the pixel. For a remote sensing image with a size of N x ×N y , the longitude and latitude calculation formula corresponding to the index value (x, y) is:

[0093] Where (lon0, lat0) is the longitude and latitude coordinates corresponding to the index value (0, 0), which is the reference longitude and latitude; Δx and Δy are the longitude resolution and latitude resolution of the image, respectively. These parameters can be obtained from the metadata of the image file of the orthographic image.

[0094] The multiple orthographic images are spliced in a "two-by-two processing" manner, that is, each time of splicing contains two input images: a master image and a slave image, wherein the master image is from the result of the last splicing, and the slave image is from an unprocessed orthographic image. Specifically, the reference longitude and latitude of the input image is sorted according to the strategy of "longitude priority comparison, longitude ascending order, and latitude descending order", and the sorted image order is the execution order of the "two-by-two processing" algorithm.

[0095] In this embodiment, it is ensured that there is an overlapping area between the master image and the slave image in each splicing.

[0096] In some embodiments, the pre-processing of the orthographic image comprises:

[0097] The pixels of different orthographic images are represented in the same index coordinate system through interpolation processing.

[0098] The longitude and latitude resolutions of different orthographic images are set to be the same.

[0099] In specific implementation, the reference longitude and latitude and the longitude and latitude resolution of each orthographic image to be spliced are made the same through interpolation processing of the master image and the slave image. In the implementation process, the minimum longitude and maximum latitude pixel points in the input image list are usually selected as the reference longitude and latitude of the final spliced image, and the longitude and latitude resolution can be customized according to user demand or automatically set to the maximum resolution in the input image list.

[0100] In this embodiment, because the reference longitude and latitude of each orthographic image to be spliced are different in actual image pre-processing, and it is difficult to ensure that the input images have the same longitude and latitude resolution, the input image grid, that is, the pixel, needs to be unified to the same index coordinate system for subsequent pixel-based contour extraction. Through interpolation processing, the pixels of different orthographic images are represented in the same index coordinate system, and then the longitude and latitude resolutions of different orthographic images are set to be the same, avoiding frequent coordinate conversion and interpolation calculation in subsequent steps, and improving the execution efficiency of the method.

[0101] In some embodiments, the intersection of the effective area contours of the master image and the slave image is determined by:

[0102] The contour points of the same index coordinates existing in the effective area contours of the master image and the slave image are obtained to obtain the intersection.

[0103] In the embodiment, the intersection is obtained by acquiring the profile points of the same index coordinates existing in the effective area profiles of the main image and the slave image, and the execution efficiency of the method is improved by taking advantage of the convenience that the pixels of different orthographic images are represented in the same index coordinate system.

[0104] In some embodiments, comprising:

[0105] A profile point quick lookup table is constructed based on a hash table and a list, and the profile points of the effective area profiles are stored in the profile point quick lookup table.

[0106] In the execution process of the crawling search method, whether the searched profile point is on the existing profile line to which it belongs is determined by the profile point quick lookup table, and if so, the profile line is trimmed.

[0107] In the specific implementation, the positions of any grid of different orthographic images can be represented by the index coordinates in the same coordinate system after the orthographic images are preprocessed. When there are profile points of the same index coordinates on two profile lines, it indicates that the two profiles intersect at the point. Therefore, the intersection of two profiles or whether a profile point is on another profile can be calculated by comparing the index coordinates. In order to improve the judgment efficiency, the profile point information is stored by combining the two data structures of "hash table + list" in the embodiment, which is referred to as "profile point quick lookup table". The "list" data structure can retain the relative position relationship of each profile point and can quickly acquire the index coordinates of adjacent profile points. The "hash table" data structure can realize fast data retrieval by mapping the key to a specific index position through a hash function, and can realize fast data searching, inserting and deleting operations, and the time complexity of each operation is usually O(1). As shown in FIG. 5, the index coordinates of the profile points are taken as the keys of the hash table, and the serial numbers of the profile points in the list are taken as the corresponding key values of the hash table. By the profile point quick lookup table, the position of a profile point in the current profile point and the positional relationship between a profile point and the current profile point can be quickly queried.

[0108] After the profile point lookup table is obtained, in the process of the crawling search method, it is judged by the profile point lookup table whether the searched profile point is on the existing profile line to which it belongs, and if so, the profile line is pruned. Specifically, in the process of "crawling search", when a new profile point B is detected, it is judged by the "profile point lookup table" whether the profile point B is already on the profile line: if not, the index coordinates of the profile point B are inserted into the "profile point lookup table"; if so, the profile line needs to be "pruned": let the starting point of the current profile line be O and the ending point be C, there is a profile point A on the profile line whose coordinates are the same as those of the newly detected profile point B, let the number of profile points from the profile point O to the profile point A be length(OA), and the number of profile points from the profile point A to the profile point C be length(AC), if length(OA) is greater than or equal to length(AC), all the profile points between the profile point A and the profile point C are deleted from the profile point lookup table, if length(OA) is less than length(AC), all the profile points between the profile point O and the profile point A are deleted. After the pruning is completed, the profile point B is inserted into the profile lookup table. Figure 6 shows some cases where profile pruning is needed, where (a) is the case where the starting point has "fluff", (b) is the case where the search process has "small rings", and (c) is the case where the search process has "fluff".

[0109] In this embodiment, the efficiency of judging the intersection of two effective area profiles or judging whether a profile point of an effective area is on another profile is improved. Due to the uniqueness of the hash table key value, the profile point lookup table cannot have profile points with the same position but different list numbers, i.e. the profile line cannot have "sub-rings", so the "crawling search method" needs to be appropriately improved so that the profile points of the effective area profiles obtained can be stored in the profile point lookup table.

[0110] In some embodiments, comprising:

[0111] When the search direction of the crawling search method when detecting an effective pixel is not parallel to one of the pixel grid lines of the orthographic image, the effective pixel is determined to be a diagonal virtual profile point, and it is taken as a profile point of the effective area profile;

[0112] By comparing the profile points of the main image and the slave image in the profile point lookup table, the intersection point is obtained.

[0113] In implementation, as shown in FIG. 7, there are two cases for the intersection of the pixel-based contour lines: "cross" intersection and "X" intersection. In FIG. 7, (a) and (b) are schematic diagrams of the "cross" intersection and "X" intersection, respectively. When the "X" intersection occurs, the actual intersection coordinates have decimals. To avoid missing the contour points when calculating the intersection, the intersection of the "X" intersection is referred to as a "diagonal virtual contour point". The contour point does not correspond to the actual grid data, but represents the geometric position relationship of the two contours at this point. When performing the crawling search method, if the search direction when detecting the valid pixel is 45°, 135°, 225°, or 315°, a "diagonal virtual contour point" needs to be inserted first, and then the valid pixel is inserted into the "contour point lookup table". Regardless of the general grid (pixel) intersection or the diagonal virtual contour point, the contour line clipping operation needs to be determined when inserting the contour point lookup table.

[0114] In this embodiment, the accuracy of the intersection point generation is improved, which is beneficial to improve the accuracy of the inlay line determination.

[0115] In some embodiments, the intersection contour of the main image and the slave image is obtained, including:

[0116] The position relationship between the contour point of the effective area contour and the grid point is determined.

[0117] A starting point is determined, which is located in the overlapping area of the main image and the slave image and only belongs to one of the contour lines.

[0118] Based on the position relationship, the contour points of the main image and the slave image are traversed and compared one by one from the starting point to obtain the intersection contour.

[0119] In implementation, the position relationship between the contour point of the effective area contour and the grid point is:

[0120] As shown in FIG. 8, the purple circle center represents a certain contour point; the blue circle center is the last contour point of the current contour point, referred to as the start of the current contour point; and the red circle center is the next contour point of the current contour point, referred to as the end of the current contour point. In a 3x3 sub-block centered on the current contour point, the position relationship between the grid points and the contour points can be divided into the following five types: (1) start-to-end: refers to the grid between the start and the end in the clockwise direction, such as the light green grid in FIG. 8, and this part of the grid is outside the image; (2) end-to-start: refers to the grid between the end and the start in the clockwise direction, such as the light brown grid in FIG. 8, and this part of the grid is inside the image; (3) on-start: refers to the grid located at the start position; (4) on-end: refers to the grid located at the end position; and (5) origin: refers to the grid located at the current contour point position.

[0121] After the position relationship is determined, an intersection contour obtaining step is performed. First, a starting point is found on the main contour by means of a "contour point lookup table", which satisfies the condition of being located in the overlapping area and belonging to only one contour line. If no contour line satisfying the condition is found, the main contour can not intersect with the slave contour, and the algorithm is terminated. The main contour is the effective contour of the main image. Then, the main contour is set as a "reference contour", the slave image is set as a "comparison contour", and the starting point is taken as a current contour point to perform the following steps: if the current contour point is not an intersection point of the two contours, the next contour point of the reference contour is taken as the current contour point; if the current contour point is an intersection point of the two contours, the index position and attribution of the current intersection point are saved, the attribution indicating whether the contour point is from the main contour or the slave contour, and then the next contour point of the comparison contour is compared with the current contour point according to the position relationship. FIG. 9 is a schematic diagram of the relationship between the next contour point of the comparison contour and the current contour point in the intersection contour obtaining process, in which the blue color represents the reference contour line, the green color represents the comparison contour line, the red color represents the intersection point of the two contour lines, O represents the current contour point, A represents the starting point of O, B represents the ending point of O, C represents the next contour point of the comparison contour, the "+" area represents the contour outside, and the "-" area represents the contour inside. As shown in FIG. 9, if the current contour point is an intersection point of the two contours, based on the position relationship, there are four cases, which correspond to four different execution modes respectively, and the four different execution modes are as follows: "on-end" as shown in FIG. 9 (c), the next contour point of the reference contour is taken as the current contour; "start-to-end" as shown in FIG. 9 (a), the next contour point of the reference contour is taken as the current contour; "end-to-start" as shown in FIG. 9 (b), the next contour point of the comparison contour is taken as the current contour point, and then the reference contour and the comparison contour are interchanged; and "on-start", which is usually not present, and if present, it means that the starting point of the algorithm does not satisfy the condition, and the algorithm is terminated. In addition, as shown in FIG. 9 (d), there is a case that the reference contour turns at point P to point C, and Q point is not accessed. In this case, the main image and the slave image have multiple overlapping areas.

[0122] In the embodiment, the intersection contour can be determined more accurately, and the inlay line can be determined more accurately.

[0123] In some embodiments, the method comprises:

[0124] When the main image and the slave image have multiple overlapping areas, the starting point of each overlapping area is determined by the contour starting point determination algorithm.

[0125] In implementation, as shown in FIG. 10, (a) is a schematic diagram when the main image and the slave image have only one overlapping area, and (b) is a schematic diagram when the main image and the slave image have multiple overlapping areas. When the main image and the slave image have multiple overlapping areas, the contour starting point determination algorithm is used to determine the starting point of each overlapping area. The execution process of the contour starting point determination algorithm includes: first, as in the intersection contour acquisition algorithm, a starting point is found on the main contour, which meets the following conditions: located in the overlapping area and only belongs to a contour line; then, starting from this starting point, the main contour line is circled, and the number of times of leaving the overlapping area during the circling is counted, which is the maximum number of overlapping areas of the two images; then, each time the overlapping area is left, a contour point in the overlapping area is reserved as the execution starting point of the overlapping area; the obtained execution starting points are traversed, and before being input into the intersection contour acquisition algorithm, it is determined whether the current execution starting point is on the existing intersection contour. If yes, the starting point is discarded, otherwise the intersection contour acquisition algorithm is continued to execute; finally, the algorithm is executed, and the intersection contour of each overlapping area and each corresponding intersection point are output. FIG. 11 is a schematic diagram of the relationship between the number of times of leaving the overlapping area and the number of overlapping areas, wherein (a) is a schematic diagram of the position relationship between the main image and the slave image when the main image contour leaves the overlapping area twice, the two overlapping areas are not connected, and there are two overlapping areas, and (b) is a schematic diagram of the position relationship between the main image and the slave image when the main image contour leaves the overlapping area twice, there is only one overlapping area, and the number of times of leaving is less than the number of overlapping areas.

[0126] In the embodiment, as can be seen from the fourth case in FIG. 9, the main image and the slave image can have multiple overlapping areas, so there are multiple intersection contours and corresponding intersection points, and a suitable starting point needs to be found for each overlapping area. The contour starting point determination algorithm is used to determine the starting point of each overlapping area, which prepares for the subsequent steps and is conducive to more accurately determining the inlay line.

[0127] In some embodiments, before the intersection contour of the main image and the slave image is acquired, the following steps are included:

[0128] When the effective area contour of the main image and the slave image has a large number of redundant intersection points due to partial area coincidence or proximity, the contour intersection point merging algorithm is used to merge the redundant intersection points.

[0129] In implementation, first, the attribution information in the intersection contour is acquired, which determines whether the intersection point belongs to the main contour or the slave contour. Then, the contour intersection point merging algorithm is used to merge the redundant intersection points, which includes the following steps: before merging, the contour intersection point list is sorted clockwise as [a1, a2, …, an], and the first element a1 is the starting point of the first overlapping area; then, the following steps are performed: if a1 is on the main contour, a1 is the starting point of the first overlapping area, and the next element a2 is the starting point of the second overlapping area; if a1 is on the slave contour, a2 is the starting point of the first overlapping area, and the next element a3 is the starting point of the second overlapping area; the above steps are repeated until the last element an is obtained; finally, the obtained starting points of the overlapping areas are output. n-1 ,a n], traversing the intersection list, adjacent and belonging to the same intersection is divided into a group, to get the grouping [g1, g2, …, g k-1 k ] ; each group contains at least one profile intersection, for each group, only the last profile intersection in the group is retained as the "merged intersection point" of the current group, to get a merged intersection list [b1, b2, …, b k-1 k ] ; then check the attribution of the first intersection and the last intersection of the merged intersection list, if the same, delete the last merged intersection; finally, the merged intersection list is substituted into the above-mentioned tessellation line determination and overlapping area attribution judgment algorithm. Specifically, before merging, the attribution of the intersection list is [main, main, …, main, slave, slave, …, slave]; it can be divided into two groups [[main, main, …, main], [slave, slave, …, slave]]; the last intersection of each group is retained to get the merged intersection list [main, slave]; check the attribution of the first and last intersection, if different, the final merged intersection list is [main, slave]. Figure 12 shows a case of dense contour intersection, where (a) is a schematic diagram before intersection merging, and (b) is a schematic diagram after intersection merging.

[0130] In this embodiment, it is considered that the main and slave contours may overlap or be close in some areas, resulting in a large number of redundant intersections in the area. If all intersections are determined for tessellation lines and overlapping area attribution division, it will cause unnecessary running time growth of the algorithm. Through the contour intersection merging algorithm of this embodiment, the execution efficiency of the method is improved.

[0131] The application also provides a method for generating an orthophoto mosaic image, comprising:

[0132] Pretreating the orthophoto to obtain a main image and a slave image, the slave image having an overlapping area with the main image;

[0133] Extracting the effective area contours of the main image and the slave image respectively using a crawling search method;

[0134] Determining the intersection of the effective area contours of the main image and the slave image;

[0135] Obtaining the intersection contour of the main image and the slave image;

[0136] ​​calculate a difference value of the effective pixels in the overlapping area of the main image and the slave image by a difference function to obtain a difference map of the effective area; if the number of intersection points is greater than 2, a position of a centroid of the overlapping area is calculated according to coordinates of each profile point of the intersection profile, then a local inner window is determined in the difference map with the centroid as a center, the intersection points are searched in the local inner window, a position with a minimum difference value is taken as an optimization end point, and the intersection points are taken as optimization start points; if the number of intersection points is equal to 2, the intersection points are respectively taken as the optimization start points and the optimization end points;

[0137] an inlaid line of the main image and the slave image is determined based on the optimization start points and the optimization end points by using an A-star algorithm; pixels near the inlaid line are feathered, and then a weight value of the main image and the slave image in the overlapping area is calculated;

[0138] the main image and the slave image are weightedly fused based on the weight value to obtain an inlaid image.

[0139] In a specific implementation, first, the inlaid line of the main image and the slave image is generated based on the inlaid line generation method of the orthographic image. Then, the attribution of the main image and the slave image in the overlapping area is divided based on the inlaid line. After the attribution division result is obtained, the weight value of the main image and the slave image in the overlapping area can be calculated. The weight value of the main image in the overlapping area attributed to the main profile is 1, and the weight value of the main image in the overlapping area attributed to the slave profile is 0. The pixels near the inlaid line are appropriately feathered to avoid a large change distance of the weight value on the inlaid line, thereby achieving a better visual effect. Assuming that the weight value of the main profile in the overlapping area of a certain pixel (i, j) is W(i, j), the pixel value of the inlaid image is the weighted fusion of the main image and the slave image, that is: I(i, j) = I master ·W(i, j) + I slave (i, j)·(1-W(i, j)).

[0140] After the main image and the slave image are weightedly fused, the inlaid image shown in FIG. 14 is obtained.

[0141] In the embodiment, the orthographic image is spliced according to the inlaid line, and the accuracy of splicing the orthographic image according to the inlaid line is improved.

[0142] In some embodiments, before the pixels near the inlaid line are feathered, the following operations are included:

[0143] Attribution information of the intersection points in the intersection profile is obtained.

[0144] After the inlaid line is generated, the attribution of the region in the overlapping area of the main image and the slave image divided according to the inlaid line is judged based on the attribution information.

[0145] In the implementation, firstly, the attribution information of the intersection points in the intersection contour is acquired. Then, according to the attribution information, the attribution of the region divided by the inlaid line can be determined, and the attribution determination step includes: if the intersection point belongs to the main contour (as shown in A1 in FIG. 15), the divided region between the intersection point and the last intersection point in the counterclockwise direction (the clockwise direction is shown by the arrow F in the figure) (as shown in B1 in FIG. 15) and the optimization end point (as shown in E in FIG. 15) belongs to the slave image (the region filled with blue in FIG. 15); if the intersection point belongs to the slave contour (as shown in B2 in FIG. 15), the divided region between the intersection point and the last intersection point in the counterclockwise direction (A1 in FIG. 15) and the optimization end point (E in FIG. 15) belongs to the main image (the region filled with red in FIG. 15). Finally, the attribution determination result of the overlapping region shown in FIG. 15 is obtained, in which A i the contour belonging to the main image, B i the contour belonging to the slave image, and E is the optimization end point.

[0146] In the embodiment, the accuracy in splicing the main image and the slave image is improved.

[0147] The application further provides a computer device, which comprises a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and the processor implements the steps of the orthographic image inlaid line determination method according to any one of the above embodiments when executing the computer program.

[0148] The application further provides a computer device, which comprises a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and the processor implements the steps of the orthographic image inlaid image generation method according to any one of the above embodiments when executing the computer program.

[0149] The application further provides a computer readable storage medium, which stores a program, wherein the computer readable storage medium refers to a carrier for storing data, which can be but is not limited to a floppy disk, an optical disk, a hard disk, a flash memory, a USB flash disk, a Memory Stick, and the like, and the computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The working process, working details, and technical effects of the computer readable storage medium provided in the embodiment can be referred to the above embodiments of the orthographic image inlaid line determination method, and will not be described herein.

[0150] The application further provides a computer readable storage medium, which stores a program, wherein the computer readable storage medium refers to a carrier storing data, which can include, but is not limited to, a floppy disk, an optical disk, a hard disk, a flash memory, a USB flash disk and / or a Memory Stick, etc., and the computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable devices. The working process, working details and technical effects of the computer readable storage medium provided in the embodiment can be referred to the above embodiments of the orthographic image mosaic image generation method, which will not be described here.

[0151] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware, and the computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, the processes of the above-mentioned embodiments can be included. Any reference to memory, storage, database or other medium used in the embodiments provided in the application can include non-volatile and / or volatile memory. The non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. The volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM) and memory bus dynamic RAM (RDRAM).

[0152] The technical features of the above embodiments can be combined in any manner. To make the description concise, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combinations of the technical features do not exist contradictory, they should be considered as the scope of the present application. The above embodiments only express several implementation manners of the application, and the description is more specific and detailed, but it should not be understood as a limitation on the patent scope of the application. It should be noted that for those skilled in the art, without departing from the concept of the application, some modifications and improvements can be made, which are within the scope of the application. Therefore, the patent protection scope of the application should be subject to the appended claims.

Claims

1. A method for determining orthographic image seam lines, characterized in that The method comprises the following steps: preprocessing orthographic images to obtain a main image and a slave image, the slave image having an overlapping area with the main image; extracting effective area contours of the main image and the slave image respectively by using a crawling search method; determining intersection points of the effective area contours of the main image and the slave image; obtaining intersection contours of the main image and the slave image; calculating difference values of effective pixels in the overlapping area of the main image and the slave image by using a difference function to obtain a difference map of the effective area; if the number of intersection points is greater than 2, the position of the centroid of the overlapping area is calculated according to the coordinates of each contour point of the intersection contours, then a local window is determined in the difference map with the centroid as the center, the local window is searched, the position with the minimum difference value is taken as the optimization end point, and the intersection points are taken as the optimization start points; if the number of intersection points is equal to 2, the intersection points are respectively taken as the optimization start point and the optimization end point; determining the inlay line of the main image and the slave image based on the optimization start point and the optimization end point by using an A-star algorithm.

2. The orthographic image tiling line determination method of claim 1, wherein, The preprocessing of the orthographic images comprises the following steps: obtaining the latitude and longitude coordinates corresponding to the index values of each pixel of the orthographic images to obtain reference latitude and longitude of the orthographic images; sorting the orthographic images based on the reference latitude and longitude.

3. The orthographic image tiling line determination method of claim 2, wherein, The preprocessing of the orthographic images comprises the following steps: representing the pixels of different orthographic images in the same index coordinate system by interpolation processing; setting the latitude and longitude resolutions of different orthographic images to be the same.

4. The orthographic image tiling line determination method of claim 3, wherein, The determination of the intersection points of the effective area contours of the main image and the slave image comprises the following steps: obtaining contour points of the same index coordinates existing in the effective area contours of the main image and the slave image to obtain intersection points.

5. The orthographic image tiling line determination method of claim 1, wherein, The method comprises the following steps: constructing a contour point quick lookup table based on a hash table and a list, and storing the contour points of the effective area contours in the contour point quick lookup table; in the process of executing the crawling search method, it is judged by the contour point quick lookup table whether the searched contour point is on an existing contour line to which the contour point belongs, and if so, the contour line is trimmed.

6. The orthographic image tiling line determination method of claim 5, wherein, The method comprises the following steps: when the search direction of the effective pixel detected by the crawling search method is not parallel to one of the pixel grid lines of the orthographic image, the effective pixel is determined to be a diagonal virtual contour point, and the effective pixel is taken as a contour point of the effective area contour; the intersection points are obtained by comparing the contour points of the main image and the slave image in the contour point quick lookup table.

7. The orthographic image tiling line determination method of claim 6, wherein, The method for obtaining the intersection contours of the main image and the slave image comprises the following steps: determining the positional relationship between the contour points and the grid points of the effective area contours; determining a start point, the start point being located in the overlapping area of the main image and the slave image and belonging to only one contour line; based on the positional relationship, the intersection contours are obtained by traversing and comparing the contour points of the main image and the slave image from the start point.

8. The orthographic image tiling line determination method of claim 7, wherein, The method comprises the following steps: when there are multiple overlapping areas of the main image and the slave image, the start points of each overlapping area are determined by a contour start point determination algorithm.

9. The orthographic image tiling line determination method of claim 1, wherein, Before the intersection contours of the main image and the slave image are obtained, the method comprises the following steps: When the intersection points of the effective area profiles of the main image and the slave image are largely redundant due to the partial area overlap or proximity, a profile intersection merging algorithm is used to merge the redundant intersection points.

10. A method for generating orthophoto mosaic images, characterized in that, The method comprises the following steps: preprocessing the orthographic image to obtain a main image and a slave image, the slave image having an overlapping area with the main image; extracting the effective area profiles of the main image and the slave image respectively using a crawling search method; determining the intersection points of the effective area profiles of the main image and the slave image; obtaining the intersection profile of the main image and the slave image; calculating the difference values of the effective pixels in the overlapping area of the main image and the slave image by a difference function to obtain a difference map of the effective area; if the number of the intersection points is greater than 2, the position of the centroid of the overlapping area is calculated according to the coordinates of the profile points of the intersection profile, then a local inner window is determined in the difference map with the centroid as the center, the inner local window is searched, the position with the minimum difference value is taken as the optimization end point, and the intersection points are taken as the optimization start points; if the number of the intersection points is equal to 2, the intersection points are respectively taken as the optimization start point and the optimization end point; determining the inlay line of the main image and the slave image based on the optimization start point and the optimization end point using an A-star algorithm; feathering the pixels near the inlay line, and then calculating the weight values of the main image and the slave image in the overlapping area; based on the weight values, performing weighted fusion on the main image and the slave image to obtain an inlaid image.

11. The orthographic image mosaic generation method of claim 10, wherein, Before the pixels near the inlay line are feathered, the method comprises the following steps: obtaining the attribution information of the intersection points in the intersection profile; after the inlay line is generated, based on the attribution information, performing attribution judgment on the area in the overlapping area of the main image and the slave image which is divided according to the inlay line.

12. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the computer program comprises instructions for causing the processor to perform the method of any one of claims 1-11. The processor executes the computer program to implement the steps of the orthographic image inlay line determination method according to any one of claims 1 to 9.

13. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the steps of the orthographic image inlaid image generation method according to any one of claims 10 to 11.

14. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a program, wherein the program is executed by the processor to implement the orthographic image inlay line determination method according to any one of claims 1 to 9.

15. A computer readable storage medium characterized by: The computer readable storage medium stores a program, wherein the program is executed by the processor to implement the orthographic image inlaid image generation method according to any one of claims 10 to 11.

Images