A method for automatically determining image control points based on aerial images

Abstract

This invention discloses an automated method for determining ground control points (GCPs) based on aerial imagery. The method includes: registering numerous color aerial images corresponding to the measured terrain in a geographic coordinate system, where the total number of images is M; using N pre-measured geographic coordinates marked on each of the N color aerial images as recommended GCPs, where N is a positive integer between 2 and half of M; defining a neighborhood centered on each recommended GCP in each color aerial image with a pre-defined size as a reference pixel block; and scaling up auxiliary GCPs based on the N reference pixel blocks within the spatial area shared by the M color aerial images in the pre-defined coordinate system. This method can assist in non-full field deployment schemes by referencing a small number of pixel blocks within the large-scale imagery itself, eliminating the need for manual mapping of other data types different from the imagery, thus balancing simplicity, accuracy, and computational efficiency.

Description

Technical Field

[0001] This invention belongs to the field of aerial image recognition technology, and in particular relates to an automated method for determining control points based on aerial images. Background Technology

[0002] Image control points (ADCs) are fundamental to aerial photogrammetry operations, controlling aspects such as aerial triangulation and mapping. Their deployment has always been challenging, generally falling into two categories: full field deployment and partial field deployment. Full field deployment involves surveying all ADCs required for orientation and correction in the office before aerial photography. While offering high aerial accuracy, this method suffers from significant disadvantages, including a large workload and high costs. Partial field deployment, on the other hand, uses aerial triangulation to obtain orientation and correction results, allowing for a smaller number of field measurements. This method is more efficient than full field deployment. However, partial field deployment requires strict adherence to aerial photogrammetry office specifications, which can deviate from actual production needs. For example, deployment standards may be relaxed for special terrains like virgin forests and deserts, and deployment is unsuitable for bodies of water such as rivers and lakes.

[0003] To address the challenges of non-full field deployment schemes, some methods, such as control point deployment, employ computer algorithms like graph matching or machine learning models, but still require manual mapping assistance, and the algorithms themselves are extremely complex. Summary of the Invention

[0004] The purpose of this invention is to provide an automated method for determining control points based on aerial imagery, in order to solve the problems existing in the prior art.

[0005] To achieve the above objectives, the present invention provides a method for automated determination of control points based on aerial imagery, comprising:

[0006] Acquire several color aerial images of the terrain to be measured, and register the color aerial images in a geographic coordinate system;

[0007] Several geographic coordinates measured in the field are marked in the registered color aerial image, and the marked points are used as recommended ground control points;

[0008] Based on a preset size, a reference pixel block is obtained centered on the recommended control point;

[0009] Based on the reference pixel block, auxiliary image control points are added to the spatial region of the coordinate system where the color aerial image is located, thereby realizing the construction of the image control library.

[0010] Optionally, the geographic coordinates and the color aerial image have a quantitative relationship: N∈[2, M / 2], where N is a positive integer;

[0011] Where N is the number of geographic coordinates measured in the field, and M is the number of color aerial images.

[0012] Optionally, the process of adding camera control points includes:

[0013] Reference points are set in each color aerial image according to a preset point distance range; a field control area centered on the reference points is defined in each color aerial image according to a preset rule template; target pixel blocks in each field control area that match the reference pixel blocks are detected; and pixels located in the middle of each target pixel block are recorded as backup field control points.

[0014] Optionally, the process of setting a reference point includes:

[0015] A point learning model is constructed and trained. The trained point learning model identifies several points of interest in each of the color aerial images. For two adjacent color aerial images, if a pair of points of interest matches a preset point distance range, they are both set as reference points.

[0016] Optionally, the process of defining the image control region includes:

[0017] Align the center of the preset rule template with the reference point;

[0018] The preset rule template is adjusted to be parallel to the color aerial image where the reference point being aligned is located;

[0019] In the color aerial image where the reference point is aligned, locate the image region to be determined that is projected by the preset rule template;

[0020] If the area of ​​the image region to be determined is equal to that of the preset rule template, then the entire image region to be determined is set as the image control area. If not, then within the image region to be determined, the largest sub-region with the same shape as the preset rule template, centered on the reference point being aligned, is set as the image control area.

[0021] Optionally, the process of detecting target pixel blocks includes:

[0022] Based on preset values, each of the image control regions is defined as several undetermined pixel blocks. The average similarity between each undetermined pixel block and each of the reference pixel blocks is calculated, and the largest average similarity among all the pixels belonging to the same image control region is selected as the target similarity.

[0023] The undetermined pixel blocks that correspond one-to-one with the target similarity are determined as the target pixel blocks.

[0024] Optionally, the formula for calculating the average similarity is:

[0025] α1×C(x, y)+α2×T(x, y)+α3×SSIM(x, y)

[0026] In the formula, α1, α2 and α3 represent three different weighting coefficients, C(x, y) represents color similarity, T(x, y) represents texture similarity, SSIM(x, y) represents structural similarity index, x represents any of the proposed pixel blocks, and y represents any of the proposed reference pixel blocks.

[0027] Optionally, the preset rule template is set with a preset magnification factor for any color aerial image, and the preset magnification factor is set to [2, 10].

[0028] The technical effects of this invention are as follows:

[0029] In this invention, numerous color aerial images are compared together with the actual terrain in a geographic coordinate system. Based on a suitable amount of pre-measured geographic coordinates, recommended ground control points are marked on no more than half of the color aerial images, corresponding to the actual points. This allows for the subsequent positioning of a suitable number of reference pixel blocks, eliminating the need for extensive field measurements and reducing fieldwork. These reference pixel blocks serve as reference data for the large-scale addition of points to numerous color aerial images. Referencing a small number of pixel blocks within the large-scale images themselves can assist in non-full field point deployment schemes. There is no need to manually delineate other types of data different from the images, such as flight strip network parameters and 3D point clouds, which helps to balance simplicity, accuracy, and computational efficiency. Attached Figure Description

[0030] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0031] Figure 1 This is a schematic flowchart of the aerial image control point determination method in an embodiment of the present invention;

[0032] Figure 2 This is a flowchart illustrating the process of adding auxiliary image control points in an embodiment of the present invention;

[0033] Figure 3 This is a schematic diagram of the architecture of a control point determination device based on aerial imagery in an embodiment of the present invention;

[0034] Figure 4 This is a circuit diagram of a computing device according to an embodiment of the present invention. Detailed Implementation

[0035] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0036] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0037] Example 1

[0038] like Figure 1-4 As shown, this embodiment provides a method for automated determination of control points based on aerial imagery, including:

[0039] S1, registers numerous color aerial images corresponding to the measured terrain in the geographic coordinate system, with a total number of M images.

[0040] S2, based on the N geographic coordinates measured in the field beforehand, mark them one by one in the N color aerial images as recommended ground control points. There are no ground control points in the remaining MN color aerial images. N is a positive integer in [2, M / 2].

[0041] For example, the terrain being measured can be an urban area or a mountainous area. In the field, multiple physical points evenly distributed in the terrain can be selected for on-site measurement. These physical points can include, but are not limited to, the corners of rooftops, the edges of zebra crossings, and the corners of roads. This helps to ensure the success rate of subsequent pixel block matching. Each geographic coordinate is automatically converted to the geographic coordinate system, and the points are automatically marked in the color aerial image according to the converted coordinates.

[0042] For example, N and M can be in an increasing relationship, M can be 500 and N can be 10, or M can be 1000 and N can be 20.

[0043] S3. In each color aerial image marked with recommended control points, a neighborhood centered on the recommended control points is defined as a reference pixel block according to a preset size. For example, the preset size can be a square size such as 3*3 or 5*5, so that the reference pixel block is square.

[0044] S4, in the spatial region occupied by M color aerial images in the preset coordinate system, add auxiliary image control points in a scaled manner according to N reference pixel blocks.

[0045] Using the above-described method for determining ground control points based on aerial imagery, numerous color aerial images are compared with the actual terrain in a geographic coordinate system. Based on a suitable amount of pre-measured geographic coordinates, recommended ground control points are marked on less than half of the color aerial images, corresponding to the actual points. This provides a suitable number of reference pixel blocks for subsequent positioning. This eliminates the need for extensive field measurements, reducing the workload. These reference pixel blocks serve as reference data for large-scale addition of points to numerous color aerial images. Referencing a small number of pixel blocks within the large-scale imagery itself can assist in non-full field point deployment schemes. There is no need to manually delineate other types of data different from the imagery, such as flight strip network parameters and 3D point clouds. This helps to balance simplicity, accuracy, and computational efficiency.

[0046] Optionally, see Figure 2 S4 includes S41 to S43.

[0047] S41, set reference points in each color aerial image according to a preset point distance range.

[0048] For example, aerial images are generally rectangular, and the preset point spacing range is appropriately set between the image width and twice the image width. For example, if the aerial image is a 20cm*30cm rectangle, the preset point spacing range can be set to [32cm, 38cm]. Any value can be selected from the preset point spacing range, and points are marked at equal intervals in the spatial area mentioned above according to the selected point spacing.

[0049] Optionally, S41 includes: identifying several points of interest in each color aerial image using a pre-trained point learning model; and setting each pair of points of interest that matches a preset point distance range as reference points for two adjacent color aerial images.

[0050] After quickly and accurately identifying a large number of points of interest in numerous color aerial images using a machine learning model, the large number of points of interest are filtered by using a preset point spacing range. This ensures that each image contains at least one reference point, thereby ensuring that at least M reference points have a certain degree of uniformity and preventing the reduction of subsequent calculation efficiency due to too many reference points.

[0051] S42 defines a field control area centered on a reference point in each color aerial image according to a preset rule template.

[0052] Optionally, S42 includes: aligning the center of the preset rule template with each reference point; adjusting the preset rule template to be parallel to the color aerial image where the aligned reference point is located; locating the image region to be determined projected by the preset rule template in the color aerial image where the aligned reference point is located; detecting whether the image region to be determined has the same area as the preset rule template; if so, setting the entire image region to be determined as a field control area; if not, setting the largest sub-region with the same shape as the preset rule template within the image region to be determined, centered on the aligned reference point.

[0053] For example, the preset rule template can be either a rectangle or a square. The distance from the reference point along the length of the image to the long side of the projection area is measured, and the distance from the reference point along the width of the image to the short side of the projection area is measured, which are used as the size of the largest sub-region.

[0054] After registering the preset rule template to the color aerial image, if the area of ​​the preset rule template and its projection on the color aerial image are equal, it can be determined that the preset rule template does not exceed the color aerial image; otherwise, it can be determined that the preset rule template exceeds the color aerial image. In the latter case, the preset rule template and its projection area on the color aerial image can have the same shape, but the center of the projection area may deviate from the reference point. Therefore, the largest sub-region centered on the reference point within the projection area should be determined as the image control area.

[0055] Optionally, the preset rule template is set with a preset magnification factor for any color aerial image, with the preset magnification factor set to [2, 10].

[0056] For example, the preset rule template can be a black or semi-transparent graphic, which can be square, and the size of the color aerial image can be 8 or 10 times larger.

[0057] By using preset rule templates, we can prevent the success rate of subsequent pixel matching from being reduced due to the control area being too small compared to the aerial image, and we can also prevent the efficiency of subsequent pixel matching from being reduced due to the control area being too large compared to the aerial image.

[0058] S43, detect the target pixel block in each image control region that is most similar to the N reference pixel blocks.

[0059] Optionally, S43 includes: defining each of the image control regions into a plurality of undetermined pixel blocks according to a preset size; calculating the average similarity between each undetermined pixel block and each reference pixel block; selecting the largest average similarity among all the pixels belonging to the same image control region as the target similarity; and determining the undetermined pixel blocks that correspond one-to-one with the target similarity as the target pixel blocks.

[0060] By using a pixel block similarity matching method, the accuracy and success rate of finding pixel blocks suitable for setting up image control points are ensured.

[0061] Alternatively, any average similarity can be expressed as:

[0062] α1×C(x, y)+α2×T(x, y)+α3×SSIM(x, y)

[0063] Where α1, α2 and α3 represent three different weighting coefficients, C(x, y) represents color similarity, T(x, y) represents texture similarity, SSIM(x, y) represents structural similarity index, x represents any undetermined pixel block, and y represents any reference pixel block.

[0064] By using a weighted average method to comprehensively measure the similarity of multi-dimensional image features, the reliability and accuracy of the average similarity are ensured.

[0065] S44, the pixel located in the middle of each target pixel block is recorded as a spare control point.

[0066] See Figure 3 Another embodiment of the present invention provides a control point determination device based on aerial imagery, comprising: an image registration module, a first point placement module, a neighborhood delineation module, and a second point placement module.

[0067] The image registration module is used to register numerous color aerial images corresponding to the measured terrain in the geographic coordinate system. The total number of images is M.

[0068] The first point placement module is used to mark N geographic coordinates in N color aerial images as recommended ground control points, where N is a positive integer in [2, M / 2].

[0069] The neighborhood delineation module is used to define a neighborhood centered on the recommended control point as a reference pixel block in each color aerial image marked with the recommended control point, according to a preset size.

[0070] The second point-deployment module is used to add auxiliary image control points in a spatial area shared by M color aerial images in a preset coordinate system, based on N reference pixel blocks.

[0071] See Figure 4 Another embodiment of the present invention provides a computing device including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the aforementioned method for determining control points based on aerial imagery. The processor can be connected to the memory via a universal serial bus. It is understood that the aforementioned computing device can be a server or a terminal device.

[0072] Another embodiment of the present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method for determining control points based on aerial images.

[0073] Generally, the computer instructions for implementing the method of the present invention can be carried on any combination of one or more computer-readable storage media. Non-transitory computer-readable storage media can include any computer-readable medium except for the signal itself, which is temporarily propagating.

[0074] Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device.

[0075] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, and C++—as well as conventional procedural programming languages—such as the "C" language or similar programming languages. In particular, Python, suitable for neural network computation, and platform frameworks based on TensorFlow, PyTorch, etc., can be used. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or connected to an external computer (e.g., via the Internet using an Internet service provider).

[0076] The above description is merely a preferred embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for automatically determining control points based on aerial imagery, characterized in that, Includes the following steps: Acquire several color aerial images of the terrain to be measured, and register the color aerial images in a geographic coordinate system; Several geographic coordinates measured in the field are marked in the registered color aerial image, and the marked points are used as recommended ground control points; Based on a preset size, a reference pixel block is obtained centered on the recommended control point; Based on the reference pixel block, auxiliary image control points are added to the spatial region of the coordinate system where the color aerial image is located to realize the automatic determination of image control points; The process of adding external image control points includes: Reference points are set in each color aerial image according to a preset point distance range; a field control area centered on the reference points is defined in each color aerial image according to a preset rule template; target pixel blocks in each field control area that match the reference pixel blocks are detected; and pixels located in the middle of each target pixel block are recorded as backup field control points. The process of setting a reference point includes: A point learning model is constructed and trained. The trained point learning model identifies several points of interest in each of the color aerial images. For two adjacent color aerial images, if a pair of points of interest matches a preset point distance range, they are both set as reference points. The process of defining the image control area includes: Align the center of the preset rule template with the reference point; The preset rule template is adjusted to be parallel to the color aerial image where the reference point being aligned is located; In the color aerial image where the reference point is aligned, locate the image region to be determined that is projected by the preset rule template; If the area of ​​the image region to be determined is equal to that of the preset rule template, then the entire image region to be determined is set as the image control area. If not, then within the image region to be determined, with the reference point being aligned as the center, the largest sub-region with the same shape as the preset rule template is set as the image control area. The process of detecting target pixel blocks includes: Based on preset values, each of the image control regions is defined as several undetermined pixel blocks. The average similarity between each undetermined pixel block and each of the reference pixel blocks is calculated, and the largest average similarity among all the pixels belonging to the same image control region is selected as the target similarity. The undetermined pixel blocks that correspond one-to-one with the target similarity are determined as the target pixel blocks.

2. The method for automated determination of control points based on aerial imagery according to claim 1, characterized in that, The geographic coordinates and the color aerial image have a quantitative relationship: N∈[2, M / 2], where N is a positive integer; Wherein, N is the number of geographic coordinates measured in the field, M is the number of color aerial images, and N geographic coordinates are marked in the registered N color aerial images.

3. The method for automated determination of control points based on aerial imagery according to claim 1, characterized in that, The formula for calculating the average similarity is: α1×C(x, y)+α2×T(x, y)+α3×SSIM(x, y) In the formula, α1, α2 and α3 represent three different weighting coefficients, C(x, y) represents color similarity, T(x, y) represents texture similarity, SSIM(x, y) represents structural similarity index, x represents any of the proposed pixel blocks, and y represents any of the proposed reference pixel blocks.

4. The method for automated determination of control points based on aerial imagery according to claim 1, characterized in that, The preset rule template is set with a preset magnification factor for any color aerial image, and the preset magnification factor is set to [2, 10].

Images