Image parameter determination method and device, storage medium and program product

Abstract

The invention discloses an image parameter determination method and device, a storage medium and a program product. The method comprises the following steps: acquiring a first orthographic image and a second orthographic image, wherein the first orthographic image and the second orthographic image have an image overlapping region; performing image matching on the first orthographic image and the second orthographic image, and determining matched feature point pairs between the first orthographic image and the second orthographic image; constructing a geographic coordinate function of a mapping geographic point of the feature point pair based on the pixel coordinates of the feature point pair and to-be-determined image parameters of the first orthographic image and the second orthographic image; constructing a loss function based on a geographic coordinate function of the mapping geographic points of the feature point pairs; the to-be-determined image parameters of the first orthographic image under the condition that the loss function meets the preset loss condition are solved, and the image parameters of the first orthographic image can be rapidly and accurately determined.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and in particular, relates to a method, device, storage medium and program product for determining image parameters. Background Art

[0002] Currently, daily management and maintenance of photovoltaic power stations are usually carried out based on images of photovoltaic power stations taken by drones while moving.

[0003] However, the images of photovoltaic power stations taken by drones while moving have inaccurate image parameters, which causes certain difficulties in the daily management and maintenance of photovoltaic power stations. Summary of the invention

[0004] The embodiments of the present application provide a method, device, storage medium and program product for determining image parameters, which can solve the problem of being unable to quickly and accurately determine the image parameters corresponding to the orthographic image.

[0005] In a first aspect, an embodiment of the present application provides a method for determining image parameters, including: acquiring a first orthographic image and a second orthographic image, wherein the first orthographic image and the second orthographic image have an image overlapping area; performing image matching on the first orthographic image and the second orthographic image to determine a pair of feature points that match between the first orthographic image and the second orthographic image; constructing a geographic coordinate function of the mapped geographic points of the feature point pair based on the pixel coordinates of the feature point pair and the image parameters to be determined of the first orthographic image and the second orthographic image; constructing a loss function based on the geographic coordinate function of the mapped geographic points of the feature point pair; and solving the image parameters to be determined of the first orthographic image when the loss function satisfies a preset loss condition.

[0006] In an optional implementation of the first aspect, the loss function is related to the distance between the mapped geographic points of the feature point pair, and the loss function is constructed based on the geographic coordinate function of the mapped geographic points of the feature point pair, including: determining the geographic coordinate deviation between the first orthographic image and the second orthographic image based on the first geographic coordinates of the first orthographic image and the second geographic coordinates of the second orthographic image; and constructing the loss function based on the geographic coordinate deviation and the geographic coordinate function of the mapped geographic points of the feature point pair.

[0007] In an optional implementation of the first aspect, based on the first geographic coordinates of the first orthographic image and the second geographic coordinates of the second orthographic image, determining the geographic coordinate deviation between the first orthographic image and the second orthographic image includes: determining the first geographic coordinates based on the latitude and longitude parameters of the first orthographic image; determining the second geographic coordinates based on the latitude and longitude parameters of the second orthographic image; determining the geographic coordinate deviation between the first orthographic image and the second orthographic image based on the first geographic coordinates and the second geographic coordinates.

[0008] In an optional implementation of the first aspect, the image parameter to be determined includes at least one of a yaw angle, a pitch angle, a roll angle, and an elevation.

[0009] In an optional implementation of the first aspect, based on the pixel coordinates of the feature point pair and the image parameters to be determined of the first orthographic image and the second orthographic image, a geographic coordinate function of the mapping geographic point of the feature point pair is constructed, including: constructing a first three-dimensional rotation matrix based on the yaw angle, pitch angle and roll angle of the first orthographic image; constructing a first scaling matrix based on the elevation of the first orthographic image, wherein at least one of the yaw angle, pitch angle, roll angle and elevation is the image parameter to be determined; obtaining an intrinsic parameter matrix of an image acquisition device, the image acquisition device being an image acquisition device that takes the first orthographic image and the second orthographic image; constructing a geographic coordinate function of the mapping geographic point of the first feature point based on the pixel coordinates of the first feature point in the feature point pair, the intrinsic parameter matrix, the first three-dimensional rotation matrix and the first scaling matrix.

[0010] In an optional implementation of the first aspect, based on the pixel coordinates of the feature point pair and the image parameters to be determined of the first orthographic image and the second orthographic image, a geographic coordinate function of the mapping geographic point of the feature point pair is constructed, including: constructing a second three-dimensional rotation matrix based on the yaw angle, pitch angle and roll angle of the second orthographic image; constructing a second scaling matrix based on the elevation of the second orthographic image, wherein at least one of the yaw angle, pitch angle, roll angle and elevation is the image parameter to be determined; obtaining an intrinsic parameter matrix of an image acquisition device, the image acquisition device being an image acquisition device that takes the first orthographic image and the second orthographic image; and constructing a geographic coordinate function of the mapping geographic point of the second feature point based on the intrinsic parameter matrix, the pixel coordinates of the second feature point in the feature point pair, the second three-dimensional rotation matrix and the second scaling matrix.

[0011] In an optional implementation of the first aspect, a loss function is constructed based on a geographic coordinate function of a mapped geographic point of a feature point pair, including: constructing a sub-loss function corresponding to the feature point pair based on the geographic coordinate function of the mapped geographic point of the feature point pair between a first orthographic image and any second orthographic image, the sub-loss function being related to the distance between the mapped geographic points of the feature point pair; and constructing a loss function based on the sub-loss function of the feature point pair between the first orthographic image and multiple second orthographic images.

[0012] In an optional implementation of the first aspect, a loss function is constructed based on a sub-loss function of feature point pairs between a first orthographic image and multiple second orthographic images, including: determining the total number of feature point pairs between the first orthographic image and the multiple second orthographic images; and constructing a loss function based on the sub-loss function of the feature point pairs between the first orthographic image and the multiple second orthographic images and the total number of feature point pairs.

[0013] In an optional implementation of the first aspect, acquiring the first orthographic image and the second orthographic image includes: determining, from a plurality of orthographic images, a second orthographic image having an image overlapping area with the first orthographic image.

[0014] In an optional implementation of the first aspect, determining a second orthographic image having an image overlapping area with a first orthographic image from a plurality of orthographic images comprises: determining a geographical distance between the first orthographic image and each orthographic image based on geographical coordinates of the first orthographic image and each orthographic image; determining whether the geographical distance satisfies a preset condition; and determining the orthographic image as the second orthographic image if the geographical distance satisfies the preset condition.

[0015] In an optional implementation of the first aspect, before performing image matching on the first orthographic image and determining the feature point pairs that match the first orthographic image and the second orthographic image, the method for determining image parameters also includes: performing distortion correction on the first orthographic image and the second orthographic image to obtain the first orthographic image and the second orthographic image after distortion correction, and the first orthographic image and the second orthographic image after distortion correction are used to extract feature points.

[0016] In the second aspect, an embodiment of the present application provides a device for determining image parameters, including: an acquisition unit, used to acquire a first orthographic image and a second orthographic image, the first orthographic image and the second orthographic image having an image overlapping area; an image matching unit, used to perform image matching on the first orthographic image and the second orthographic image, and determine the feature point pairs that match between the first orthographic image and the second orthographic image; a first construction unit, used to construct a geographic coordinate function of the mapped geographic points of the feature point pair based on the pixel coordinates of the feature point pair and the image parameters to be determined of the first orthographic image and the second orthographic image; a second construction unit, used to construct a loss function based on the geographic coordinate function of the mapped geographic points of the feature point pair; and a calculation unit, used to solve the image parameters to be determined of the first orthographic image when the loss function satisfies a preset loss condition.

[0017] In a third aspect, an embodiment of the present application provides a computer-readable storage medium having computer program instructions stored thereon, and when the computer program instructions are executed by a processor, the method for determining image parameters of the first aspect is implemented.

[0018] In a fourth aspect, an embodiment of the present application provides a computer program product. When instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the method for determining image parameters of the first aspect.

[0019] In an embodiment of the present application, since the first orthographic image and the second orthographic image have an image overlapping area, image matching is performed on the first orthographic image and the second orthographic image, and the feature point pairs that match between the first orthographic image and the second orthographic image can be quickly determined. Using the pixel coordinates of the feature point pairs that match between the first orthographic image and the second orthographic image, and the image parameters to be determined of the first orthographic image and the second orthographic image, a geographic coordinate function of the mapped geographic points corresponding to the feature point pairs in the geographic coordinate system can be cleverly constructed, so that the image parameters to be determined are included in the geographic coordinate function. Since the feature point pairs between the two orthographic images are determined by image matching, the geographic coordinates of the mapped geographic points corresponding to the feature point pairs are also matched. Based on this, a loss function can be constructed based on the geographic coordinate function of the mapped geographic points corresponding to the feature point pairs. By solving the loss function, the image parameters to be determined that meet the preset loss conditions are obtained. It can be seen that in the method for determining image parameters of the present application, the loss function is constructed using the geographic coordinate function of the mapped geographic point of the feature point pair, so that the loss function corresponds to the geographic coordinate function of the mapped geographic point, which contains the image parameters to be determined, thereby solving the loss function to obtain the image parameters to be determined of the first orthographic image that meets the preset loss conditions, and the image parameters can be determined quickly and accurately. On this basis, when the image acquisition device configured by the drone takes multiple images during movement, it keeps the lens vertically downward toward the ground to take multiple orthographic images. According to this method, the image parameters to be determined of these orthographic images can be quickly and accurately determined, and the determined image parameters are used to splice and process the multiple images of the photovoltaic power station, thereby analyzing the overall area of ​​the photovoltaic power station and locating problems, effectively improving the efficiency of daily management and maintenance of the photovoltaic power station. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solution of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0021] Figure 1 It is a schematic diagram of the architecture of a system for determining image parameters provided by an embodiment of the present application; Figure 2 is a flow chart of a method for determining image parameters provided by an embodiment of the present application; Figure 3 is a flow chart of a geographic coordinate function for constructing a mapping geographic point of a first feature point provided by an embodiment of the present application; Figure 4 is a flow chart of a geographic coordinate function for constructing a mapping geographic point of a second feature point provided by an embodiment of the present application; Figure 5 is a flow chart of constructing a loss function based on a geographic coordinate function of a mapped geographic point based on a feature point pair provided by an embodiment of the present application; Figure 6 This is a flow chart of a method for determining image parameters in a scenario where a drone inspects a photovoltaic power station, provided by an embodiment of the present application; Figure 7 is a structural schematic diagram of an apparatus for determining image parameters provided by yet another embodiment of the present application; Figure 8 It is a structural schematic diagram of an electronic device provided in yet another embodiment of the present application.

[0022] The above drawings include the following reference numerals: 100. electronic device; 200. drone; 700. device for determining image parameters; 710. acquisition unit; 720. image matching unit; 730. first construction unit; 740. second construction unit; 750. computing unit; 801. processor; 802. memory; 803. communication interface; 810. bus. DETAILED DESCRIPTION

[0023] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.

[0024] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0025] At present, the daily management and maintenance of photovoltaic power stations are usually carried out based on the images of photovoltaic power stations taken. However, there are some inaccurate image parameters in the relevant image parameters of the images taken of photovoltaic power stations, which makes it impossible to further process the images, such as stitching multiple images of photovoltaic power stations taken into a continuous image, so that the overall area of ​​photovoltaic power stations cannot be analyzed and the problem cannot be located, which leads to low efficiency in the daily management and maintenance of photovoltaic power stations. For example, in the scenario of using drones to inspect photovoltaic power stations, the image acquisition equipment configured by the drones can take images of photovoltaic power stations on the inspection route. However, due to the inaccurate image parameters of the images of photovoltaic power stations taken by drones, it is impossible to further process the images, such as stitching multiple images of photovoltaic power stations taken into a continuous image, so that the overall area of ​​photovoltaic power stations cannot be accurately analyzed and the problem cannot be located.

[0026] The present application provides a method, device, storage medium and program product for determining image parameters. Since the first orthographic image and the second orthographic image have an image overlapping area, the first orthographic image and the second orthographic image are image matched, and the feature point pairs that match between the first orthographic image and the second orthographic image can be quickly determined. Using the pixel coordinates of the feature point pairs that match between the first orthographic image and the second orthographic image, and the image parameters to be determined of the first orthographic image and the second orthographic image, a geographic coordinate function of the mapped geographic points corresponding to the feature point pairs in the geographic coordinate system can be cleverly constructed, so that the geographic coordinate function contains the image parameters to be determined. Since the feature point pairs between the two orthographic images are determined by image matching, the geographic coordinates of the mapped geographic points corresponding to the feature point pairs are also matched. Based on this, a loss function can be constructed based on the geographic coordinate function of the mapped geographic points corresponding to the feature point pairs. By solving the loss function, the image parameters to be determined of the first orthographic image that meet the preset loss conditions are obtained. It can be seen that in the method for determining image parameters of the present application, the loss function is constructed using the geographic coordinate function of the mapped geographic point of the feature point pair, so that the loss function corresponds to the geographic coordinate function of the mapped geographic point, which contains the image parameters to be determined, thereby solving the loss function to obtain the image parameters to be determined of the first orthographic image that meets the preset loss conditions, and the image parameters can be determined quickly and accurately. On this basis, the determined image parameters can be used to further process the image, such as stitching multiple images of the photovoltaic power station, so as to analyze the overall area of ​​the photovoltaic power station and locate problems, effectively improving the daily management and maintenance efficiency of the photovoltaic power station.

[0027] For ease of understanding, here we take the example of drone inspection of photovoltaic power stations to briefly introduce the application architecture of the image parameter determination method, device, storage medium and program product provided in this application. Figure 1 A schematic diagram of the application architecture of a system for determining image parameters provided in an embodiment of the present application. The system for determining image parameters includes a drone 200 and an electronic device 100. The drone 200 inspects a photovoltaic power station based on an inspection route, and uses an image acquisition device configured on the drone to capture multiple images. It should be noted that when capturing multiple images, the image acquisition device configured on the drone keeps the lens pointed vertically downward at the ground to ensure that the multiple images captured are orthographic images. After receiving the multiple orthographic images captured by the drone 200, the electronic device 100 determines the image parameters of the multiple orthographic images according to the image parameter determination method provided in the present application, and obtains the determined image parameters corresponding to each orthographic image.

[0028] In practical applications, in the embodiments of the present application, the method for determining image parameters provided in the embodiments of the present application can be implemented when the processor of the electronic device executes a program or instruction. However, in some embodiments, the image acquisition device of the drone or other devices may also have similar functions. For example, the method for determining image parameters provided in the embodiments of the present application is implemented when the image acquisition device of the drone executes a program or instruction. The embodiments of the present application do not limit this.

[0029] It should be noted that the application scenarios described in the above embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. A person of ordinary skill in the art will know that with the emergence of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems. The method for determining image parameters provided by the embodiments of the present application can be applied to various application scenarios where image parameters need to be determined.

[0030] The following describes the method, device, storage medium and program product for determining image parameters provided by the present application.

[0031] Figure 2 FIG. 1 is a flow chart of a method for determining image parameters provided by an embodiment of the present application. Figure 2 As shown, the method for determining image parameters provided in the embodiment of the present application may include steps S201 to S205.

[0032] In step S201 , a first orthographic image and a second orthographic image are acquired, and the first orthographic image and the second orthographic image have an image overlap area.

[0033] The first orthographic image and the second orthographic image are taken when the lens of the image acquisition device of the drone is pointed vertically downward to the ground.

[0034] The image overlap region refers to a region having the same or similar content between the first orthographic image and the second orthographic image. Optionally, the image overlap between the first orthographic image and the second orthographic image may be a complete overlap or a partial overlap.

[0035] In some embodiments, any one of the plurality of orthographic images may be determined as a first orthographic image, and then a second orthographic image having an image overlap area with the first orthographic image may be determined from orthographic images other than the first orthographic image.

[0036] Take the inspection of photovoltaic power stations by drones as an example. In actual applications, when drones perform inspection tasks, the image acquisition equipment configured by the drones takes pictures of the photovoltaic power station and obtains an orthographic image set containing n orthographic images. For example, in determining When the image parameters are As the first orthographic image, and can be obtained from { } There is at least one second orthographic image in the image overlap region. When the image parameters are As the first orthographic image, it can be obtained from Determine and There is at least one second orthographic image of the image overlap region.

[0037] Since there is an image overlap area between the first orthographic image and the second orthographic image, it is ensured that the first orthographic image and the second orthographic image have more common features, so feature point pairs can be quickly and accurately determined between the first orthographic image and the second orthographic image later.

[0038] There are many ways to implement determining at least one second orthographic image having an image overlap area with the first orthographic image from the plurality of orthographic images.

[0039] In one embodiment, a neural network model may be used to perform target detection on a first orthographic image and multiple orthographic images, and then determine an intersection over union (IOU) value between a bounding box obtained by target detection in the first orthographic image and a bounding box obtained by target detection in each orthographic image. Finally, based on the IOU values ​​of the first orthographic image and each orthographic image, the orthographic image corresponding to an IOU value greater than a preset threshold is determined as a second orthographic image having an image overlapping area with the first orthographic image.

[0040] In another embodiment, a second orthographic image having an image overlapping area with the first orthographic image can be determined based on continuous shooting time screening, for example, an orthographic image whose shooting time interval with the first orthographic image is less than a preset threshold is determined as the second orthographic image.

[0041] In another embodiment, a second orthographic image having an image overlap area with the first orthographic image may be determined based on the geographic distance between the first orthographic image and each orthographic image. Specifically, based on the geographic coordinates of the first orthographic image and each orthographic image, the geographic distance between the first orthographic image and each orthographic image is determined; whether the geographic distance meets a preset condition is determined; and if the geographic distance meets the preset condition, the orthographic image is determined as the second orthographic image.

[0042] Since the geographic coordinates of the first orthographic image and each orthographic image are two-dimensional plane coordinates, we can use , determine the geographical distance between the first orthographic image and each orthographic image, where ( , ) is used to represent the geographic coordinates of the first orthographic image, It is used to represent the horizontal coordinate of the first orthographic image in the geographic coordinate system. Used to represent the vertical coordinate of the first orthographic image in the geographic coordinate system, ( , ) is used to represent the geographic coordinates of the orthographic image, Used to represent the horizontal coordinate of the orthographic image in the geographic coordinate system. Used to represent the vertical coordinate of the orthographic image in the geographic coordinate system.

[0043] The preset condition may be that the geographical distance between the first orthographic image and each orthographic image is less than a distance threshold, and the distance threshold may be set based on the shooting height, shooting angle, and focal length of the image acquisition device.

[0044] Based on the geographical distance between the first orthographic image and each orthographic image, each orthographic image is screened, so that the second orthographic image having an overlapping area with the first orthographic image can be quickly determined.

[0045] In step S202, image matching is performed on the first orthographic image and the second orthographic image to determine matching feature point pairs between the first orthographic image and the second orthographic image.

[0046] Image matching is to determine the feature points of similar parts between different images and establish the correspondence between the feature points of similar parts between different images. Usually, image matching between two images can be performed based on feature point matching, region matching or global matching.

[0047] In one embodiment, the image matching is further explained by taking the inspection of a photovoltaic power station by a drone as an example. First, taking a first orthographic image and a second orthographic image as an example, feature points in the first orthographic image and feature points in the second orthographic image are extracted. Afterwards, based on the feature vector distance between the feature points in the first orthographic image and the feature points in the second orthographic image, it is determined whether the feature points in the first orthographic image and the feature points in the second orthographic image correspond to the same feature. If the feature vector distance between the feature points in the first orthographic image and the feature points in the second orthographic image meets the requirements, that is, the two feature points correspond to the same feature, then there is a corresponding relationship between the feature points in the first orthographic image and the feature points in the second orthographic image, so that the feature points in the first orthographic image and the feature points in the second orthographic image are determined as a feature point pair.

[0048] A feature point pair is two feature points with corresponding relationship in two images.

[0049] In actual application, any feasible feature extraction method in the prior art can be used to extract feature points in the first orthographic image or the second orthographic image. For example, algorithms such as Scale Invariant Feature Transform (SIFT), Oriented FAST and Rotated BRIEF (ORB), or Binary Robust Invariant Scalable Keypoints (BRISK) can be used to extract feature points in the first orthographic image or the second orthographic image.

[0050] In an optional embodiment, when there are multiple second orthographic images, taking the SIFT algorithm as an example, the process of performing image matching on the first orthographic image and each second orthographic image to determine the feature point pairs between the first orthographic image and each second orthographic image is further described. The process includes a feature extraction step, a feature matching step, a feature matching result screening step, and a false matching removal step.

[0051] Feature extraction step: Extract SIFT feature points of the first orthographic image and each second orthographic image to obtain a feature point set SIFT={ , , , …, }, each element in the feature point set SIFT is used to represent a SIFT feature point in the first orthographic image or the second orthographic image, which is in the form of, for example but not limited to, a feature vector.

[0052] Feature matching step: using the nearest neighbor matching method, two feature points whose feature vectors are closest to the feature points in the first orthographic image are determined from each second orthographic image.

[0053] Feature matching result screening steps: For ease of understanding, a feature point in the first orthographic image and two feature points with the closest feature vector distance in the corresponding second orthographic image are used for illustration. , the feature vector distance between the feature point in the first orthophoto and the two feature points in the corresponding second orthophoto is screened, and the feature vector distance greater than the feature threshold is screened out At the same time, determine the ratio of the feature vector distance between the feature point in the first orthographic image and the closest feature point in the second orthographic image to the feature vector distance between the feature point in the first orthographic image and the second closest feature point in the second orthographic image. In the case of, the feature point in the first orthographic image and the closest feature point in the second orthographic image are determined as a feature point pair. For example, the ratio threshold Can be set to 0.75.

[0054] Steps to remove mismatches: Use the Random Sample Consensus (RANSAC) algorithm to remove mismatched feature point pairs. RANSAC estimates a geometric transformation model (such as a homography matrix) by iteratively selecting feature point pairs, and then verifies other feature point pairs based on the geometric transformation model to exclude feature point pairs that do not conform to the geometric transformation model.

[0055] Based on the above process, the feature point pairs between the first orthographic image and each second orthographic image can be obtained. The first orthographic image and the second orthographic image The feature point pairs that match each other are The first orthographic image The pixel coordinates of the feature points in can be expressed as: With the first orthographic image The second orthographic image with one-to-one correspondence between the feature points in The pixel coordinates of the feature points in can be expressed as: in, Used to represent the first orthographic image middle The pixel coordinates of the feature points, ( , ) is used to represent the first orthographic image Middle The pixel coordinates of the feature points, Used to indicate the Feature points in the first orthographic image The horizontal coordinate in the image coordinate system is Used to indicate the Feature points in the first orthographic image The ordinate in the image coordinate system is Used to represent the second orthographic image middle The pixel coordinates of the feature points, ( , ) is used to represent the second orthographic image Middle The pixel coordinates of the feature points, Used to indicate the feature points in the second orthographic image The horizontal coordinate in the image coordinates is Used to indicate the feature points in the second orthographic image The vertical coordinate in image coordinates.

[0056] In step S203, a geographic coordinate function of the mapped geographic points of the feature point pair is constructed based on the pixel coordinates of the feature point pair and the image parameters to be determined of the first orthographic image and the second orthographic image.

[0057] The mapped geographic point of the feature point pair is to map the feature point in the feature point pair from the image coordinate system to the geographic coordinate system to obtain the mapped geographic point corresponding to the feature point in the feature point pair.

[0058] Based on the pixel coordinates of the feature point pair and the image parameters to be determined of the first orthographic image and the second orthographic image, a geographic coordinate function of the mapping geographic point of the feature point pair is constructed, that is, the image parameters to be determined of the first orthographic image and the second orthographic image are regarded as unknown parameters, and the feature points in the feature point pair are mapped from the image coordinate system to the geographic coordinate system using the camera imaging model, so that the geographic coordinate function of the mapping geographic point of the feature point pair can be constructed based on the pixel coordinates of the feature point pair, so that the geographic coordinate function contains the image parameters to be determined. Therefore, the geographic coordinate function of the mapping geographic point of the feature point pair is used to characterize the geographic coordinates of the mapping geographic point corresponding to the feature point in the feature point pair in the geographic coordinate system.

[0059] For orthographic images, the image parameters that need to be determined may include yaw angle and / or elevation.

[0060] In one embodiment, a geographic coordinate function of the mapped geographic point of the feature point pair is constructed based on the pixel coordinates of the feature point pair and the yaw angles of the first orthographic image and the second orthographic image. Optionally, in this embodiment, the elevation may be a known accurate quantity.

[0061] In another embodiment, a geographic coordinate function of the mapped geographic point of the feature point pair is constructed based on the pixel coordinates of the feature point pair, the elevations of the first orthographic image and the second orthographic image. Optionally, in this embodiment, the yaw angle may be a known accurate quantity.

[0062] In yet another embodiment, a geographic coordinate function of the mapped geographic point of the feature point pair is constructed based on the pixel coordinates of the feature point pair, the yaw angle and the elevation of the first orthographic image and the second orthographic image.

[0063] For an orthographic image, the angle values ​​of the pitch angle and the roll angle are theoretically zero, but considering various errors, the angles of the pitch angle and the roll angle may not be zero in practice. Therefore, in addition to the yaw angle and the elevation, the pitch angle and / or the roll angle may also be determined.

[0064] In summary, the embodiments of the present application do not specifically limit the image parameters that need to be determined, which can be at least one of the yaw angle, elevation, pitch angle, and roll angle, or any other suitable parameters that can be used to construct the geographic coordinates corresponding to the image coordinates, and can be set according to actual needs.

[0065] In step S204, a loss function is constructed based on the geographic coordinate function of the mapped geographic points of the feature point pairs.

[0066] Since the feature point pairs between the first orthographic image and the second orthographic image are determined by image matching, the geographic coordinates of the mapped geographic points corresponding to the feature point pairs are also matched. Based on this, the loss function is constructed using the geographic coordinate function of the mapped geographic points of the feature point pairs, so that the loss function corresponds to the geographic coordinate function of the mapped geographic points. The loss function can be used to reflect the matching loss between the mapped geographic points of the feature point pairs. The matching loss can be characterized, for example, by the geographic distance between the mapped geographic points of the feature point pairs, that is, the loss function is related to the distance between the mapped geographic points of the feature point pairs. The image parameters of the first orthographic image can be solved later when the loss function meets the preset loss condition, so that the image parameters of the first orthographic image obtained are more accurate.

[0067] For example, a feature point pair is used to represent a common feature between the first orthographic image and the second orthographic image. In the real world, the feature should correspond to a geographic coordinate. Based on this, the feature points in the feature point pair are mapped from the image coordinate system to the geographic coordinate system, and the geographic coordinates of the feature points in the feature point pair in the geographic coordinate system can be obtained, and the geographic coordinates of the two feature points in the feature point pair are theoretically the same or similar, that is, the geographic coordinates of the mapped geographic points of the feature point pair are theoretically the same or similar. Therefore, a loss function can be constructed based on the geographic coordinate function of the mapped geographic points of the feature point pair.

[0068] A loss function is constructed based on a geographic coordinate function of the mapped geographic points of the feature point pair. The specific implementation method may be: based on the first geographic coordinates of the first orthographic image and the second geographic coordinates of the second orthographic image, determine the geographic coordinate deviation between the first orthographic image and the second orthographic image; based on the geographic coordinate deviation and the geographic coordinate function of the mapped geographic points of the feature point pair, construct a loss function.

[0069] The geographic coordinate deviation is the deviation between the first geographic coordinate of the first orthographic image and the second geographic coordinate of the second orthographic image, for example, it can be the deviation between the coordinate components corresponding to the first geographic coordinate and the second geographic coordinate.

[0070] There are multiple implementations for obtaining the first geographic coordinates of the first orthographic image and the second geographic coordinates of the second orthographic image. For example, the first geographic coordinates of the first orthographic image and the second geographic coordinates of the second orthographic image can be directly obtained based on an image acquisition device, or the first geographic coordinates of the first orthographic image and the second orthographic image can be obtained based on the latitude and longitude parameters of the first orthographic image and the second orthographic image.

[0071] In some embodiments, the geographic coordinates obtained using pixel coordinates, image parameters, and camera imaging models are geographic coordinates in the coordinate system of the image acquisition device itself, rather than in the geodetic coordinate system. Since the first orthographic image and the second orthographic image are shot at different locations, this results in a deviation between the geographic coordinates of the first orthographic image and the second orthographic image in the coordinate system of the image acquisition device itself. Correspondingly, there is also a deviation in the geographic coordinates of the mapped geographic points of the feature point pair. Therefore, in the process of constructing the loss function, it is necessary to consider the geographic coordinate deviation between the first orthographic image and the second orthographic image, so that the constructed loss function can accurately reflect the matching relationship between the geographic coordinates of the mapped geographic points of the feature point pair that match the first orthographic image and the second orthographic image.

[0072] In some embodiments, if the geographic coordinates obtained using pixel coordinates, image parameters and camera imaging models are geographic coordinates in a geodetic coordinate system, then in the process of constructing the loss function, there is no need to consider the geographic coordinate deviation between the first orthographic image and the second orthographic image.

[0073] There are multiple implementations of determining the geographic coordinate deviation between the first orthographic image and the second orthographic image based on the first geographic coordinate of the first orthographic image and the second geographic coordinate of the second orthographic image.

[0074] In one embodiment, the geographic coordinate deviation between the first orthographic image and the second orthographic image is determined based on the difference between the first geographic coordinate of the first orthographic image and the second geographic coordinate of the second orthographic image. As an example, if the first geographic coordinate of the first orthographic image and the second geographic coordinate of the second orthographic image are geographic coordinates in the coordinate system of the image acquisition device itself, then the deviation between the first geographic coordinate and the second geographic coordinate can be directly determined; if the first geographic coordinate of the first orthographic image and the second geographic coordinate of the second orthographic image are geographic coordinates in the geodetic coordinate system, then the deviation between the first geographic coordinate and the second geographic coordinate in the geodetic coordinate system can be used as the deviation between the first geographic coordinate and the second geographic coordinate in the coordinate system of the image acquisition device itself.

[0075] In another embodiment, based on the latitude and longitude parameters of the first orthographic image, a first geographic coordinate is determined, and the geographic coordinate is a geographic coordinate in a geodetic coordinate system; based on the latitude and longitude parameters of the second orthographic image, a second geographic coordinate is determined, and the geographic coordinate is a geographic coordinate in a geodetic coordinate system; based on the first geographic coordinate and the second geographic coordinate, a geographic coordinate deviation between the first orthographic image and the second orthographic image is determined, that is, the deviation between the first geographic coordinate and the second geographic coordinate in the geodetic coordinate system is used as the deviation between the first geographic coordinate and the second geographic coordinate in the coordinate system of the image acquisition device itself.

[0076] There are multiple ways to obtain the longitude and latitude parameters of the first orthographic image and the second orthographic image.

[0077] In one embodiment, the longitude and latitude parameters of the first orthographic image and the second orthographic image may be obtained by manual annotation.

[0078] In another embodiment, it can be directly obtained from the Exchangeable Image File Format (EXIF) of the first orthographic image and the second orthographic image. The EXIF ​​file refers to metadata embedded in the image. The EXIF ​​file records detailed information about the shooting conditions of the image, the relevant settings of the image acquisition device, and the editing software. And the EXIF ​​file can be applied to various types of image formats, for example, JPEG format, PNG format, TIFF format.

[0079] In a scenario where a drone inspects a photovoltaic power station, if the orthographic image captured by the image acquisition device configured for the drone is in JPEG format, the longitude and latitude parameters corresponding to the first orthographic image and the second orthographic image can be directly obtained from the EXIF ​​files corresponding to the first orthographic image and the second orthographic image.

[0080] There are many implementations for determining the first geographic coordinates of the first orthographic image and the second geographic coordinates of the second orthographic image based on the latitude and longitude parameters of the first orthographic image and the second orthographic image.

[0081] In one embodiment, the longitude and latitude parameters of the first orthographic image and the second orthographic image are respectively converted to the National Geodetic Coordinate System 2000 (CGCS2000) coordinate system to obtain the first geographic coordinates of the first orthographic image and the second geographic coordinates of the second orthographic image.

[0082] For example, the latitude and longitude parameters of the orthographic image are determined from the EXIF ​​file of the orthographic image. The latitude and longitude parameters of the orthographic image can be expressed as LON={ } LAT={ } The LON set is a longitude parameter set, and the LON set element It is used to represent the longitude parameter of the nth orthographic image. The LAT set is a latitude parameter set, and the LAT set contains Used to represent the latitude parameter of the nth orthographic image.

[0083] Based on the conversion of the latitude and longitude parameters of the orthographic image to the CGCS2000 coordinate system, the geographic coordinates of the orthographic image are obtained. The geographic coordinates of the orthographic image can be expressed as: LX= LY={ } Among them, the LX set is the horizontal axis set, and the LX set It is used to represent the horizontal coordinate of the nth orthographic image in the geographic coordinate system. The LY set is the vertical coordinate set. Used to represent the vertical coordinate of the nth orthographic image in the geographic coordinate system.

[0084] In another embodiment, the longitude and latitude parameters of the first orthographic image and the second orthographic image are converted into the World Geodetic System (WGS84) coordinate system to obtain the first geographic coordinates of the first orthographic image and the second geographic coordinates of the second orthographic image.

[0085] In yet another embodiment, the longitude and latitude parameters of the first orthographic image and the second orthographic image are converted into the Beidou coordinate system to obtain the first geographic coordinates of the first orthographic image and the second geographic coordinates of the second orthographic image.

[0086] The feature point pair includes a first feature point and a second feature point. Based on the geographic coordinate deviation and the geographic coordinate function of the mapped geographic points of the feature point pair, a loss function is constructed. There are multiple implementation methods.

[0087] In one embodiment, based on the geographic coordinate deviation, a geographic coordinate function of the mapped geographic point of the first feature point is constructed to obtain the geographic coordinate function of the mapped geographic point of the first feature point, and based on the geographic coordinate function of the mapped geographic point of the first feature point and the geographic coordinate function of the mapped geographic point of the second feature point, a loss function is constructed.

[0088] In another embodiment, based on the geographic coordinate deviation, a geographic coordinate function of the mapped geographic point of the second feature point is constructed to obtain the geographic coordinate function of the mapped geographic point of the second feature point, and a loss function is constructed based on the geographic coordinate function of the mapped geographic point of the second feature point and the geographic coordinate function of the mapped geographic point of the first feature point.

[0089] In another embodiment, a loss function is constructed based on a geographic coordinate function of a mapped geographic point of a feature point pair, and the loss function is constructed based on a geographic coordinate deviation to obtain a loss function constructed based on the geographic coordinate deviation.

[0090] The first geographic coordinates are determined based on the latitude and longitude parameters of the first orthographic image, and the second geographic coordinates are determined based on the latitude and longitude parameters of the second orthographic image, so that the first geographic coordinates and the second geographic coordinates can be obtained more accurately. Based on the first geographic coordinates and the second geographic coordinates, the geographic coordinate deviation between the first orthographic image and the second orthographic image is determined, so that the determined geographic coordinate deviation is accurate, and subsequently based on the geographic coordinate deviation and the geographic coordinate function of the mapped geographic point of the feature point pair, the loss function can be constructed more accurately.

[0091] In step S205, the image parameters to be determined of the first orthographic image are solved so that the loss function satisfies the preset loss condition.

[0092] There are many ways to set the preset loss condition. For example, the preset loss condition can be that the loss function reaches a convergence state, the preset loss condition can also be that the function value of the loss function has reached a minimum, of course, the preset loss condition can also be that the number of iterations reaches a preset number of iterations, and the preset loss condition can also be that the function value of the loss function reaches a preset function value.

[0093] In one embodiment, if a geographic coordinate function of the mapped geographic points of the feature point pair is constructed based on the pixel coordinates of the feature point pair and the yaw angles of the first orthographic image and the second orthographic image, the yaw angle of the first orthographic image can be solved when the loss function meets the preset loss condition.

[0094] In another embodiment, if a geographic coordinate function of the mapped geographic points of the feature point pair is constructed based on the pixel coordinates of the feature point pair and the elevations of the first orthographic image and the second orthographic image, the elevation of the first orthographic image can be solved when the loss function meets the preset loss condition.

[0095] In another embodiment, if a geographic coordinate function of the mapping geographic point of the feature point pair is constructed based on the pixel coordinates of the feature point pair, the yaw angle and elevation of the first orthographic image, and the yaw angle and elevation of the second orthographic image, the yaw angle and elevation of the first orthographic image can be solved when the loss function meets the preset loss condition. As described above, the pitch angle, roll angle or other image parameters of the first orthographic image can also be used when the loss function meets the preset loss condition.

[0096] There are multiple ways to solve the image parameters of the first orthographic image when the loss function satisfies the preset loss condition.

[0097] In one embodiment, a nonlinear optimization method is used to solve the loss function until the function value of the loss function satisfies a preset loss condition, thereby obtaining image parameters of the first orthographic image.

[0098] When the image parameters of the first orthographic image are determined by using a nonlinear optimization method, specifically, a Powell optimization method (Powell) may be used to solve the loss function until the function value of the loss function satisfies a preset loss condition, thereby obtaining the image parameters of the first orthographic image.

[0099] In the process of solving the loss function using Powell, the search range of image parameters such as the yaw angle, elevation, pitch angle, and roll angle of the first orthographic image can be specifically configured according to the operating conditions and parameter accuracy of the corresponding equipment to narrow the search range as much as possible.

[0100] For example, in a scenario where a drone is inspecting a photovoltaic power station, the search range of image parameters such as the yaw angle, elevation, pitch angle, and roll angle of the first orthographic image can be specifically configured according to the operating conditions and parameter accuracy of the drone to narrow the search range as much as possible and to more quickly obtain image parameters such as the yaw angle, elevation, pitch angle, and roll angle of the first orthographic image that meet the preset loss conditions.

[0101] In another embodiment, a genetic algorithm is used to solve the loss function until the function value of the loss function satisfies a preset loss condition, thereby obtaining the image parameters of the first orthographic image.

[0102] In yet another embodiment, a particle swarm algorithm is used to solve the loss function until a function value of the loss function satisfies a preset loss condition, thereby obtaining image parameters of the first orthographic image.

[0103] After obtaining image parameters such as the yaw angle, elevation, pitch angle, and roll angle of the first orthographic image, the obtained image parameters such as the yaw angle, elevation, pitch angle, and roll angle of the first orthographic image may be written into the EXIF ​​file of the first orthographic image.

[0104] In an embodiment of the present application, since the first orthographic image and the second orthographic image have an image overlapping area, image matching is performed on the first orthographic image and the second orthographic image, and the feature point pairs that match between the first orthographic image and the second orthographic image can be quickly determined. Using the pixel coordinates of the feature point pairs that match between the first orthographic image and the second orthographic image, and the image parameters to be determined of the first orthographic image and the second orthographic image, a geographic coordinate function of the mapped geographic points corresponding to the feature point pairs in the geographic coordinate system can be cleverly constructed, so that the image parameters to be determined are included in the geographic coordinate function. Since the feature point pairs between the two orthographic images are determined by image matching, the geographic coordinates of the mapped geographic points corresponding to the feature point pairs are also matched. Based on this, a loss function can be constructed based on the geographic coordinate function of the mapped geographic points corresponding to the feature point pairs. By solving the loss function, the image parameters to be determined that meet the preset loss conditions are obtained. It can be seen that in the method for determining image parameters of the present application, the loss function is constructed using the geographic coordinate function of the mapped geographic point of the feature point pair, so that the loss function corresponds to the geographic coordinate function of the mapped geographic point, which contains the image parameters to be determined, thereby solving the loss function to obtain the image parameters to be determined of the first orthographic image that meets the preset loss conditions, and the image parameters can be determined quickly and accurately. On this basis, the determined image parameters can be used to stitch multiple images of the photovoltaic power station, so as to analyze the overall area of ​​the photovoltaic power station and locate problems, effectively improving the daily management and maintenance efficiency of the photovoltaic power station.

[0105] like Figure 3 As shown, another embodiment of the present application provides a flowchart of constructing a geographic coordinate function of a mapping geographic point of a first feature point. Figure 3 The difference from other embodiments is that step S203 in other embodiments can be specifically refined as follows: Figure 3 Steps S2031 to S2034 in .

[0106] In step S2031, a first three-dimensional rotation matrix is ​​constructed based on the yaw angle, pitch angle and roll angle of the first orthographic image. It should be noted that in this step, the yaw angle, pitch angle and roll angle may all be unknown parameters, or one or more of them may be known parameters.

[0107] The yaw angle, pitch angle, and roll angle are three Euler angles that describe the rotation state of an object. The yaw angle is used to indicate the angle of rotation of an object around the vertical Z axis, the roll angle is used to indicate the angle of rotation of an object around the X axis, and the pitch angle is used to indicate the angle of rotation of an object around the Y axis. In the scenario where a drone inspects a photovoltaic power station, the yaw angle of the drone can be understood as the angle between the actual flight direction of the drone and the planned flight direction, the roll angle can be understood as the left and right tilt angle of the drone, and the pitch angle can be understood as the angle between the front and back direction of the drone and the horizontal plane.

[0108] The three-dimensional rotation matrix can be used to describe the posture of the image acquisition device relative to the geographic coordinate system. Therefore, based on the three-dimensional rotation matrix, the pixel points in the image acquisition device can be accurately mapped to the geographic coordinate system. Based on this, the first three-dimensional rotation matrix can be constructed based on the yaw angle, pitch angle, and roll angle of the first orthographic image.

[0109] In step S2032, a first scaling matrix is ​​constructed based on the elevation of the first orthographic image, wherein at least one of the yaw angle, the pitch angle, the roll angle and the elevation is an image parameter to be determined. In this step, the parameter value of the elevation may be known or to be determined.

[0110] Elevation is the distance between the image acquisition device and the ground. Due to the different heights between the image acquisition device and the ground, the physical sizes of the pixels in the captured orthographic images are different.

[0111] Since there may be size differences between different coordinate systems, the size differences between different coordinate systems can be adjusted based on the scaling matrix. Based on this, the first scaling matrix can be constructed based on the elevation of the first orthographic image.

[0112] In step S2033, an internal parameter matrix of an image acquisition device is obtained, where the image acquisition device is an image acquisition device that captures the first orthographic image and the second orthographic image.

[0113] In a scenario where a drone inspects a photovoltaic power station, the image acquisition device may be an image acquisition device configured on the drone.

[0114] The intrinsic parameter matrix is ​​used to describe the geometric characteristics and optical properties inside the image acquisition device. Therefore, based on the intrinsic parameter matrix, the first feature point can be mapped from the pixel coordinate system to the geographic coordinate system more accurately. In one embodiment, the intrinsic parameter matrix can be expressed as in, Used to represent the effective focal length of the orthographic image on the x-axis, Used to represent the effective focal length of the orthographic image on the y-axis, and is the principal point coordinate, that is Used to represent the horizontal coordinate of the center point of the image in the pixel coordinate system. Used to represent the vertical coordinate of the center point of the image in the pixel coordinate system.

[0115] In step S2034, a geographic coordinate function of mapping the first feature point to a geographic point is constructed based on the pixel coordinates of the first feature point in the feature point pair, the intrinsic parameter matrix, the first three-dimensional rotation matrix and the first scaling matrix.

[0116] In one embodiment, a geographic coordinate function of a mapping geographic point of the first feature point may be constructed based on the pixel coordinates of the first feature point in the feature point pair, the inverse matrix of the intrinsic parameter matrix, the first three-dimensional rotation matrix and the first scaling matrix.

[0117] For example, if the image parameters of the first orthographic image are to be determined, and the Represents the first orthographic image, using represents the second orthographic image, the first orthographic image The first feature point in And the second orthographic image The second feature point in is a feature point pair. The process of constructing the geographic coordinate function of the first feature point mapping geographic point with the pixel coordinate of the first feature point, the inverse matrix of the intrinsic parameter matrix, the first three-dimensional rotation matrix and the first scaling matrix is ​​as follows.

[0118] Based on the first orthographic image The yaw angle, pitch angle and roll angle are used to construct the first three-dimensional rotation matrix. The first three-dimensional rotation matrix can be expressed as = in, Used to represent the first three-dimensional rotation matrix, Used to represent the first orthographic image The yaw angle, Used to represent the first orthographic image The roll angle, Used to represent the first orthographic image The pitch angle. It should be noted that the image parameters with known parameter values ​​can be substituted into specific numerical values. For example, if only the parameter value of the yaw angle needs to be determined, the angle values ​​of the pitch angle and the roll angle (under the premise of the orthographic image, the ideal angle value is zero) are substituted into the first three-dimensional rotation matrix; if only the parameter value of the pitch angle needs to be determined, the angle values ​​of the yaw angle and the roll angle are substituted into the first three-dimensional rotation matrix; if only the parameter value of the roll angle needs to be determined, the angle values ​​of the yaw angle and the pitch angle are substituted into the first three-dimensional rotation matrix; if the parameter values ​​of two angles need to be determined, the remaining angle values ​​are substituted into the first three-dimensional rotation matrix; if the parameter values ​​of three angles need to be determined, the above-mentioned representation of the first three-dimensional rotation matrix is ​​used.

[0119] Based on the first orthographic image The elevation of constructs the first scaling matrix, which can be expressed as = in, Used to represent the first scaling matrix, Used to represent the first orthographic image If the parameter value of the elevation is known, the specific parameter value can be substituted into the first scaling matrix.

[0120] Based on the first orthographic image The first characteristic point The pixel coordinates and the inverse matrix of the intrinsic matrix , the first three-dimensional rotation matrix and the first scaling matrix , construct the first feature point The geographic coordinate function of the mapped geographic point, then the first feature point The geographic coordinate function of the mapped geographic point can be expressed as = in, Used to represent the inverse matrix of the internal parameter matrix, Used to represent the first feature point Function to map geographic points to geographic coordinates.

[0121] Based on the first feature point The pixel coordinates and the inverse matrix of the intrinsic matrix , the first three-dimensional rotation matrix and the first scaling matrix , construct the first feature point The geographic coordinate function of the mapped geographic point In the process, due to the inverse matrix of the internal parameter matrix , the first three-dimensional rotation matrix and the first scaling matrix They are all three-dimensional matrices, so the first feature point obtained is The geographic coordinate function of the mapped geographic point The geographic coordinates represented are also three-dimensional. The two-dimensional pixel coordinates are converted to the three-dimensional geographic coordinates of the mapped geographic point. During the conversion process, the elevation axis of the three-dimensional geographic coordinates of the mapped geographic point is inaccurate. Therefore, in order to reduce the impact on the accuracy of the image parameters, in one embodiment, the first feature point The geographic coordinate function of the mapped geographic point The elevation axis is normalized to make the first feature point The three-dimensional geographic coordinates of the mapped geographic point are converted into two-dimensional geographic coordinates, that is, = in, Used to represent the first feature point The two-dimensional geographic coordinates of the mapped geographic points, Used to represent the first feature point The horizontal coordinate of the three-dimensional geographic coordinates of the mapped geographic point, Used to represent the first feature point The vertical coordinate of the three-dimensional geographic coordinates of the mapped geographic point, Used to represent the first feature point The elevation coordinates of the mapped geographic point in three-dimensional geographic coordinates.

[0122] By utilizing the intrinsic parameter matrix of the image acquisition device, the first three-dimensional rotation matrix and the first scaling matrix, the first feature point is mapped from the pixel coordinate system to the geographic coordinate system, so that the mapped geographic point corresponding to the first feature point can be accurately obtained, and the geographic coordinate function of the mapped geographic point of the first feature point can be more accurately constructed.

[0123] like Figure 4 As shown, another embodiment of the present application provides a flowchart of constructing a geographic coordinate function of a mapping geographic point of a second feature point. Figure 4 The difference from other embodiments is that step S203 in other embodiments can be specifically refined as follows: Figure 4 Steps S2035 to S2038 in .

[0124] In step S2035, a second three-dimensional rotation matrix is ​​constructed based on the yaw angle, pitch angle and roll angle of the second orthographic image. It should be noted that in this step, the yaw angle, pitch angle and roll angle may all be unknown parameters, or one or more of them may be known parameters.

[0125] The three-dimensional rotation matrix can be used to describe the posture of the image acquisition device relative to the geographic coordinate system. Therefore, based on the three-dimensional rotation matrix, the pixel points in the image acquisition device can be accurately mapped to the geographic coordinate system. Based on this, the second three-dimensional rotation matrix can be constructed based on the yaw angle, pitch angle and roll angle of the second orthographic image.

[0126] In step S2036, a second scaling matrix is ​​constructed based on the elevation of the second orthographic image, wherein at least one of the yaw angle, the pitch angle, the roll angle and the elevation is an image parameter to be determined. In this step, the parameter value of the elevation may be known or to be determined.

[0127] Since there may be size differences between different coordinate systems, the size differences between different coordinate systems can be adjusted based on the scaling matrix. Based on this, a second scaling matrix can be constructed based on the elevation of the second orthographic image.

[0128] In step S2037, an internal parameter matrix of an image acquisition device is obtained, where the image acquisition device is an image acquisition device that captures the first orthographic image and the second orthographic image.

[0129] In a scenario where a drone inspects a photovoltaic power station, the image acquisition device may be an image acquisition device configured on the drone.

[0130] In step S2038, a geographic coordinate function of mapping the second feature point to the geographic point is constructed based on the intrinsic parameter matrix, the pixel coordinates of the second feature point in the feature point pair, the second three-dimensional rotation matrix and the second scaling matrix.

[0131] The translation matrix can represent the position offset of the image acquisition device relative to the geographic coordinate system. In the embodiment where the geographic coordinates of the first orthographic image and the second orthographic image are the geographic coordinates in the coordinate system of the image acquisition device itself rather than the geodetic coordinate system, due to the different shooting positions of the first orthographic image and the second orthographic image, there is a deviation in the geographic coordinates of the common features between the first orthographic image and the second orthographic image. Therefore, in order to more accurately determine the geographic coordinate function of the mapping geographic point of the second feature point, in one embodiment, a translation matrix can be constructed based on the geographic coordinate deviation between the first orthographic image and the second orthographic image, and the second feature point is mapped from the image coordinate system to the geographic coordinate system using the translation matrix, the inverse matrix of the intrinsic parameter matrix, the second three-dimensional rotation matrix, and the second scaling matrix, so as to construct the geographic coordinate function of the mapping geographic point of the second feature point.

[0132] For example, if the image parameters of the first orthographic image are to be determined, and Represents the first orthographic image, using represents the second orthographic image, the first orthographic image The first feature point in And the second orthographic image The second feature point in is a feature point pair.

[0133] Determine the first orthographic image With the second orthographic image The geographic coordinate deviation between , and the geographic coordinate deviation can be expressed as in, Used to represent the first orthographic image The horizontal coordinate in the geographic coordinate system, Used to represent the first orthographic image The vertical coordinate in the geographic coordinate system, Used to represent the second orthographic image The horizontal coordinate in the geographic coordinate system, Used to represent the second orthographic image The vertical coordinate in the geographic coordinate system, Used to represent the first orthographic image With the second orthographic image The deviation of the horizontal axis between Used to represent the first orthographic image With the second orthographic image The deviation between the vertical axes.

[0134] Based on the first orthographic image With the second orthographic image The geographic coordinate deviation between them is used to construct the translation matrix = in, Used to represent the translation matrix.

[0135] Based on the second orthographic image The yaw angle, pitch angle and roll angle are used to construct the second three-dimensional rotation matrix. The second three-dimensional rotation matrix can be expressed as in, Used to represent the second three-dimensional rotation matrix, Used to represent the second orthographic image The yaw angle, Used to represent the second orthographic image The roll angle, Used to represent the second orthographic image The pitch angle. It should be noted that the image parameters with known parameter values ​​can be substituted into specific values. For example, if only the parameter value of the yaw angle needs to be determined, the angle values ​​of the pitch angle and the roll angle (under the premise of the orthographic image, the ideal angle value is zero) are substituted into the second three-dimensional rotation matrix; if only the parameter value of the pitch angle needs to be determined, the angle values ​​of the yaw angle and the roll angle are substituted into the second three-dimensional rotation matrix; if only the parameter value of the roll angle needs to be determined, the angle values ​​of the yaw angle and the pitch angle are substituted into the second three-dimensional rotation matrix; if the parameter values ​​of two angles need to be determined, the remaining angle values ​​are substituted into the second three-dimensional rotation matrix; if the parameter values ​​of three angles need to be determined, the above-mentioned representation of the second three-dimensional rotation matrix is ​​used.

[0136] Based on the second orthographic image The second scaling matrix can be expressed as = in, Used to represent the second scaling matrix, Used to represent the second orthographic image If the parameter value of the elevation is known, the specific parameter value can be substituted into the first scaling matrix.

[0137] Based on the translation matrix , the inverse matrix of the internal parameter matrix , the second three-dimensional rotation matrix and the second scaling matrix , the second feature point Map to geographic coordinates and construct the second feature point The geographic coordinate function of the mapped geographic point , the second feature point The geographic coordinate function of the mapped geographic point can be expressed as = in, Used to represent the inverse matrix of the intrinsic parameter matrix.

[0138] Based on the translation matrix , the inverse matrix of the internal parameter matrix , the second three-dimensional rotation matrix and the second scaling matrix , the second feature point Map to geographic coordinates and construct the second feature point The geographic coordinate function of the mapped geographic point In the process, due to the inverse matrix of the internal parameter matrix , the second three-dimensional rotation matrix , the second scaling matrix And the translation matrix are all three-dimensional matrices, so the second characteristic point obtained is The geographic coordinate function of the mapped geographic point The geographic coordinates represented are also three-dimensional. The two-dimensional pixel coordinates are converted to the three-dimensional geographic coordinates of the mapped geographic point. During the conversion process, the elevation axis of the three-dimensional geographic coordinates of the mapped geographic point is inaccurate. Therefore, in order to reduce the impact on the accuracy of the image parameters, in one embodiment, the second feature point The geographic coordinate function of the mapped geographic point The elevation axis is normalized to make the second feature point The three-dimensional geographic coordinates of the mapped geographic point are converted into two-dimensional geographic coordinates, that is, = in, Used to represent the second feature point The two-dimensional geographic coordinates of the mapped geographic points, Used to represent the second feature point The horizontal coordinate of the three-dimensional geographic coordinates of the mapped geographic point, Used to represent the second feature point The vertical coordinate of the three-dimensional geographic coordinates of the mapped geographic point, Used to represent the second feature point The elevation coordinates of the mapped geographic point in three-dimensional geographic coordinates.

[0139] By using the intrinsic parameter matrix of the image acquisition device, the second three-dimensional rotation matrix and the second scaling matrix, the second feature point is mapped to the geographic coordinates, so as to accurately obtain the mapped geographic point of the second feature point and more cleverly construct the geographic coordinate function of the mapped geographic point of the second feature point.

[0140] It should be noted that the translation matrix can also be used to construct the geographic coordinate function of the mapping geographic point of the first feature point. The specific construction method can refer to the geographic coordinate function of the mapping geographic point of the second feature point. In some embodiments, the translation matrix can be used to construct the geographic coordinate function of the mapping geographic point of the first feature point and the second feature point without considering the translation matrix. Instead, when calculating the loss function, that is, the geographic distance between the mapping geographic points, the geographic coordinates of one of the feature points are added or subtracted from the geographic coordinate deviation, and then the geographic distance is calculated with the geographic coordinates of the other feature point.

[0141] It can be understood that the embodiments of constructing the geographic coordinate function in the present application are not limited to the form of the above three-dimensional rotation matrix and scaling matrix. If other types of image parameters need to be determined, a transformation matrix for transforming pixel coordinates into geographic coordinates can be constructed based on the mapping relationship between the image parameters and the geographic coordinates.

[0142] like Figure 5 As shown, another embodiment of the present application provides a flowchart for constructing a loss function based on a geographic coordinate function of a mapped geographic point of a feature point pair. Figure 5 The difference from other embodiments is that step S204 in other embodiments can be specifically refined as follows: Figure 5 Step S2041 and step S2042 in .

[0143] In step S2041, based on the geographic coordinate function of the mapped geographic points of the feature point pairs between the first orthographic image and any second orthographic image, a sub-loss function corresponding to the feature point pairs is constructed, and the sub-loss function is related to the distance between the mapped geographic points of the feature point pairs.

[0144] The sub-loss function can be used to represent the distance between the mapped geographic points of a feature point pair. The sub-loss function can be expressed as = in, Used to represent the sub-loss function, Used to represent the transpose operator.

[0145] In step S2042, a loss function is constructed based on the sub-loss functions of the feature point pairs between the first orthographic image and the plurality of second orthographic images.

[0146] When there are multiple second orthophoto images, a loss function is constructed based on the sub-loss function of the feature point pairs between the first orthophoto image and the multiple second orthophoto images. In this way, the constructed loss function can be used to reflect the geographical distance between the feature points in the first orthophoto image and the corresponding feature points in each second orthophoto image, so that the constructed loss function is more accurate, and further makes the image parameters of the subsequently obtained first orthophoto image more accurate.

[0147] In some examples, a loss function is constructed based on a sub-loss function of feature point pairs between a first orthographic image and multiple second orthographic images, including: determining the total number of feature point pairs between the first orthographic image and the multiple second orthographic images; and constructing a loss function based on the sub-loss function of feature point pairs between the first orthographic image and the multiple second orthographic images and the total number of feature point pairs.

[0148] Based on the sub-loss functions of the feature point pairs between the first orthographic image and multiple second orthographic images and the total number of feature point pairs, a loss function in an average state can be constructed, so that in the subsequent process of solving the loss function, the loss function can reach a convergence state relatively quickly, thereby quickly solving the image parameters in the optimal state corresponding to the first orthographic image.

[0149] In one embodiment, based on the sub-loss functions of all feature point pairs between the first orthographic image and a second orthographic image, a loss function between the first orthographic image and the second orthographic image is constructed, then = in, Used to represent the loss function between the first orthographic image and the second orthographic image.

[0150] Based on the total number of feature point pairs between the first orthographic image and the second orthographic image, an average loss function between the first orthographic image and the second orthographic image is constructed, and then: = in, Used to represent the average loss function between the first orthographic image and the second orthographic image.

[0151] A loss function is constructed based on an average loss function between the first orthographic image and each of the second orthographic images.

[0152] For example, when there is one first orthographic image and three second orthographic images, the first orthographic image is represented as , the three second orthographic images are represented as , as well as , and There are feature point pairs, and There are feature point pairs, and There are feature point pairs.

[0153] based on and between The sub-loss function of feature point pairs is constructed and The loss function between ,based on and between The sub-loss function of feature point pairs is constructed and The loss function between ,based on and between The sub-loss function of feature point pairs is constructed and The loss function between .

[0154] based on and Loss function between And the number of feature point pairs ,get and The average loss function between ,based on and Loss function between And the number of feature point pairs ,get and The average loss function between ,based on and Loss function between And the number of feature point pairs ,get and The average loss function between .

[0155] based on and The average loss function between , and The average loss function between as well as and The average loss function between , we can get the loss function E, which can be expressed as + + In another embodiment, a loss function between the first orthographic image and a second orthographic image is constructed based on a sub-loss function of all feature point pairs between the first orthographic image and the second orthographic image; a loss function between the first orthographic image and multiple second orthographic images is constructed based on the loss function between the first orthographic image and each of the second orthographic images; and a loss function is constructed by averaging the loss functions between the first orthographic image and multiple second orthographic images based on the total number of feature point pairs between the first orthographic image and the multiple second orthographic images.

[0156] For example, when there is one first orthographic image and three second orthographic images, the first orthographic image is represented as , the three second orthographic images are represented as , as well as , and There are feature point pairs, and There are feature point pairs, and There are feature point pairs.

[0157] based on and between The sub-loss function of feature point pairs is constructed and The loss function between ,based on and between The sub-loss function of feature point pairs is constructed and The loss function between ,based on and between The sub-loss function of feature point pairs is constructed and The loss function between .

[0158] based on and The loss function between , and The loss function between as well as and The loss function between ,get and , as well as The loss function between + + .

[0159] based on and Between feature point pairs, and Between feature point pairs, and Between feature point pairs, we get and , as well as The total number of feature point pairs between = + + .

[0160] and , as well as The loss function between as well as , we get the loss function E, which can be expressed as E= In some examples, the method for determining image parameters of the present application also includes: before performing image matching on the first orthographic image and the second orthographic image to determine the matching feature point pairs between the first orthographic image and the second orthographic image, performing distortion correction on the first orthographic image and the second orthographic image to obtain the distortion-corrected first orthographic image and the second orthographic image, and the distortion-corrected first orthographic image and the second orthographic image are used to extract feature points.

[0161] By performing distortion correction on the first orthographic image and the second orthographic image, geometric distortion introduced in the imaging process can be eliminated, further ensuring that feature points of the first orthographic image and the plurality of second orthographic images after distortion correction can be extracted more accurately subsequently.

[0162] In one embodiment, the first orthographic image and the second orthographic image may be subjected to distortion correction by the following steps, wherein the specific process includes a first determination step, a distortion correction step and a distortion correction processing step, wherein: The first determination step: determine the internal parameter matrix K, which can be expressed as in, Used to indicate the effective focal length of the orthographic image on the x-axis. Used to represent the effective focal length of the orthographic image on the y-axis, and is the principal point coordinate, that is Used to represent the horizontal coordinate of the center point of the image in the pixel coordinate system. Used to represent the vertical coordinate of the center point of the image in the pixel coordinate system.

[0163] Distortion correction steps: Image distortion correction coefficients usually include radial distortion coefficients and tangential distortion coefficients. The radial distortion coefficients include , as well as , the tangential distortion coefficients include and Based on this, the distortion model of the orthographic image can be expressed as (1+ + )+2 xy+ ) (1+ + )+ xy+ ) in,( ) is used to represent the pixel coordinates of the pixel points in the orthographic image after distortion correction in the pixel coordinate system. It is used to represent the horizontal coordinate of the pixel point in the orthographic image after distortion correction in the pixel coordinate system. It is used to represent the ordinate of the pixel point in the orthographic image after distortion correction in the pixel coordinate system. (x, y) is the pixel coordinate of the pixel point in the orthographic image before distortion correction in the pixel coordinate system. x is used to represent the abscissa of the pixel point in the orthographic image before distortion correction in the pixel coordinate system. y is used to represent the ordinate of the pixel point in the orthographic image before distortion correction in the pixel coordinate system. = .

[0164] Distortion correction processing step: construct a mapping matrix, and iteratively interpolate the pixel coordinates of the pixel points in the orthographic image before distortion correction in the distortion correction step and the corresponding pixel coordinates of the pixel points after distortion correction, so as to obtain the mapping matrix of the orthographic image before and after distortion correction, that is, obtain a mapping table of the distorted coordinates and the undistorted coordinates of the orthographic image. Then, based on the mapping table, the pixel points in the orthographic image are mapped one by one to obtain the corresponding image after distortion correction.

[0165] For ease of understanding, the following uses an example of a drone inspecting a photovoltaic power station to illustrate the processing of the method for determining image parameters of the present application. Figure 6 The flowchart of the method for determining image parameters in the scenario of the drone inspecting the photovoltaic power station provided in the embodiment of the present application. The method for determining image parameters in the scenario of the drone inspecting the photovoltaic power station includes steps S601 to S610.

[0166] In step S601, the drone inspects the photovoltaic power station and takes n orthographic images, that is, the orthographic image set .

[0167] In step S602, a first orthographic image and a plurality of second orthographic images are determined from the plurality of orthographic images.

[0168] In determining When the image parameters are As the first orthographic image, and can be obtained from { } There is a second orthographic image of the image overlap area. For ease of understanding, assume and If the geographical distance of each orthographic image in satisfies the preset conditions, The orthophoto images in are all determined as the second orthophoto images, that is, is the first orthographic image, Each of the orthographic images in is a second orthographic image.

[0169] In step S603, distortion correction is performed on the first orthographic image and the plurality of second orthographic images.

[0170] In step S604, image matching is performed on the first orthographic image and each of the second orthographic images to determine feature point pairs between the first orthographic image and each of the second orthographic images.

[0171] Step S605 and step S606 take a feature point pair as an example to illustrate the construction of a geographic coordinate solution function for mapping the first feature point in the feature point pair and a geographic coordinate solution function for mapping the second feature point in the feature point pair.

[0172] In step S605, a geographic coordinate function of a mapping geographic point of the first feature point in the feature point pair is constructed based on the pixel coordinates of the first feature point in the feature point pair and the image parameters to be determined of the first orthographic image.

[0173] A first three-dimensional rotation matrix is ​​constructed based on the yaw angle, the pitch angle, and the roll angle of the first orthographic image.

[0174] A first scaling matrix is ​​constructed based on the elevation of the first orthographic image, wherein at least one of the yaw angle, the pitch angle, the roll angle and the elevation is an image parameter to be determined.

[0175] Based on the pixel coordinates of the first feature point of the first orthographic image, the inverse matrix of the intrinsic parameter matrix, the first three-dimensional rotation matrix and the first scaling matrix, a geographic coordinate function of the mapping geographic point of the first feature point is constructed, and by normalizing the elevation axis in the geographic coordinate function of the mapping geographic point of the first feature point, the three-dimensional geographic coordinates represented by the geographic coordinate function of the mapping geographic point of the first feature point are converted into two-dimensional geographic coordinates.

[0176] In step S606, a geographic coordinate function of a mapping geographic point of the second feature point in the feature point pair is constructed based on the pixel coordinates of the second feature point in the feature point pair, the image parameters to be determined of the second orthographic image, and the geographic coordinate deviation.

[0177] Based on the first geographic coordinates of the first orthoimage and the second geographic coordinates of the second orthoimage, a geographic coordinate deviation is determined.

[0178] A translation matrix is ​​constructed based on a geographic coordinate deviation between the first orthographic image and the second orthographic image.

[0179] A second three-dimensional rotation matrix is ​​constructed based on the yaw angle, the pitch angle, and the roll angle of the second orthographic image.

[0180] Based on the elevation of the second orthographic image, a second scaling matrix is ​​constructed, wherein at least one of the yaw angle, the pitch angle, the roll angle and the elevation is an image parameter to be determined.

[0181] Based on the translation matrix, the inverse matrix of the intrinsic parameter matrix, the second three-dimensional rotation matrix and the second scaling matrix, a geographic coordinate function of the mapping geographic point of the second feature point is constructed, and the elevation axis in the geographic coordinate function of the mapping geographic point of the second feature point is normalized so that the three-dimensional geographic coordinates represented by the geographic coordinate function of the mapping geographic point of the second feature point is converted into two-dimensional geographic coordinates.

[0182] In step S607, a sub-loss function is constructed based on the geographic coordinate function of the mapped geographic points of the feature point pairs.

[0183] In step S608 , a loss function is constructed based on the sub-loss functions of the feature point pairs between the first orthographic image and the plurality of second orthographic images and the total number of feature point pairs.

[0184] In step S609, a nonlinear programming algorithm is used to solve the image parameters to be determined of the first orthographic image when the loss function satisfies the preset loss condition.

[0185] In the solution process, the initial values ​​of the image parameters of each orthographic image are first obtained, and the optimal solution is searched from the initial values ​​until the loss function reaches a convergence state, thereby obtaining the image parameters of the first orthographic image. As an example, the initial values ​​of the image parameters can be specified or obtained from an image acquisition device, such as from an EXIF ​​file of the image.

[0186] In step S610 , the determined image parameters of the first orthographic image are written into the EXIF ​​file of the first orthographic image.

[0187] In the orthographic image set I, other orthographic images, namely { }, steps S602 to S610 may be repeated.

[0188] It should be noted that the device for determining image parameters is a device corresponding to the above-mentioned method for determining image parameters. All implementation methods in the above-mentioned method embodiment are applicable to the embodiment of the device and can achieve the same technical effect, which will not be repeated here.

[0189] Based on the same inventive concept, the embodiment of the present application also provides a device for determining image parameters. Figure 7 The device for determining image parameters provided in the embodiment of the present application is described in detail.

[0190] Figure 7 It is a structural schematic diagram of a device for determining image parameters provided in an embodiment of the present application.

[0191] like Figure 7 As shown, the image parameter determination device 700 may include an acquisition unit 710 , an image matching unit 720 , a first construction unit 730 , a second construction unit 740 , and a calculation unit 750 .

[0192] The acquisition unit 710 may be used to acquire a first orthographic image and a second orthographic image, wherein the first orthographic image and the second orthographic image have an image overlap area.

[0193] The image matching unit 720 may be used to perform image matching on the first orthographic image and the second orthographic image to determine matching feature point pairs between the first orthographic image and the second orthographic image.

[0194] The first construction unit 730 may be configured to construct a geographic coordinate function of a mapping geographic point of the feature point pair based on the pixel coordinates of the feature point pair and the image parameters to be determined of the first orthographic image and the second orthographic image.

[0195] The second construction unit 740 may be used to construct a loss function based on a geographic coordinate function of a mapped geographic point of a feature point pair.

[0196] The calculation unit 750 may be used to solve the image parameters to be determined of the first orthographic image when the loss function satisfies a preset loss condition.

[0197] In one embodiment, the second construction unit can also be used to correlate the loss function with the distance between the mapped geographical points of the feature point pair, determine the geographical coordinate deviation between the first orthographic image and the second orthographic image based on the first geographical coordinates of the first orthographic image and the second geographical coordinates of the second orthographic image; and construct the loss function based on the geographical coordinate deviation and the geographical coordinate function of the mapped geographical points of the feature point pair.

[0198] In one embodiment, the second construction unit can also be used to determine the first geographic coordinates based on the latitude and longitude parameters of the first orthographic image; determine the second geographic coordinates based on the latitude and longitude parameters of the second orthographic image; and determine the geographic coordinate deviation between the first orthographic image and the second orthographic image based on the first geographic coordinates and the second geographic coordinates.

[0199] In one embodiment, the image parameter to be determined includes at least one of a yaw angle, a pitch angle, a roll angle, and an elevation.

[0200] In one embodiment, the first construction unit can also be used to construct a first three-dimensional rotation matrix based on the yaw angle, pitch angle and roll angle of the first orthographic image; construct a first scaling matrix based on the elevation of the first orthographic image, wherein at least one of the yaw angle, pitch angle, roll angle and elevation is an image parameter to be determined; obtain an intrinsic parameter matrix of an image acquisition device, wherein the image acquisition device is an image acquisition device that takes the first orthographic image and the second orthographic image; and construct a geographic coordinate function of a mapping geographic point of the first feature point based on the pixel coordinates of the first feature point in the feature point pair, the intrinsic parameter matrix, the first three-dimensional rotation matrix and the first scaling matrix.

[0201] In one embodiment, the first construction unit can also be used to construct a second three-dimensional rotation matrix based on the yaw angle, pitch angle and roll angle of the second orthographic image; construct a second scaling matrix based on the elevation of the second orthographic image, wherein at least one of the yaw angle, pitch angle, roll angle and elevation is an image parameter to be determined; obtain an intrinsic parameter matrix of an image acquisition device, wherein the image acquisition device is an image acquisition device that takes the first orthographic image and the second orthographic image; and construct a geographic coordinate function of a mapping geographic point of the second feature point based on the intrinsic parameter matrix, the pixel coordinates of the second feature point in the feature point pair, the second three-dimensional rotation matrix and the second scaling matrix.

[0202] In one embodiment, the second construction unit can also be used to construct a sub-loss function corresponding to the feature point pair based on the geographic coordinate function of the mapped geographic points of the feature point pair between the first orthographic image and any second orthographic image, and the sub-loss function is related to the distance between the mapped geographic points of the feature point pair; and construct a loss function based on the sub-loss function of the feature point pair between the first orthographic image and multiple second orthographic images.

[0203] In one embodiment, the second construction unit can also be used to determine the total number of feature point pairs between the first orthographic image and the multiple second orthographic images; and construct a loss function based on the sub-loss function of the feature point pairs between the first orthographic image and the multiple second orthographic images and the total number of feature point pairs.

[0204] In one embodiment, the acquisition unit may be further configured to determine, from the plurality of orthographic images, a second orthographic image having an image overlap area with the first orthographic image.

[0205] In one embodiment, the acquisition unit can also be used to determine the geographical distance between the first orthographic image and each orthographic image based on the geographical coordinates of the first orthographic image and each orthographic image; determine whether the geographical distance meets a preset condition; and when the geographical distance meets the preset condition, determine the orthographic image as the second orthographic image.

[0206] In one embodiment, the apparatus for determining image parameters of the present application further includes a distortion correction unit. The distortion correction unit may be used to perform distortion correction on the first orthographic image and the second orthographic image before performing image matching on the first orthographic image and determining the matching feature point pairs between the first orthographic image and the second orthographic image, so as to obtain the first orthographic image and the second orthographic image after the distortion correction, and the first orthographic image and the second orthographic image after the distortion correction are used to extract feature points.

[0207] Figure 8 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application is shown.

[0208] The electronic device may include a processor 801 and a memory 802 storing computer program instructions.

[0209] Specifically, the processor 801 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0210] The memory 802 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 802 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 802 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 802 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 802 is a non-volatile solid-state memory.

[0211] The memory may include read-only memory (ROM), random access memory (RAM), magnetic disk storage media devices, optical storage media devices, flash memory devices, electrical, optical or other physical / tangible memory storage devices. Thus, typically, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to one aspect of the present application.

[0212] The processor 801 implements any one of the methods for determining image parameters in the above embodiments by reading and executing computer program instructions stored in the memory 802 .

[0213] In one example, the electronic device may further include a communication interface 803 and a bus 810. Figure 8 As shown, the processor 801, the memory 802, and the communication interface 803 are connected via a bus 810 and communicate with each other.

[0214] The communication interface 803 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0215] Bus 810 includes hardware, software or both, and couples the components of online data traffic billing equipment to each other. For example, but not limitation, the bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front-side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnect (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. Where appropriate, bus 810 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the present application considers any suitable bus or interconnection.

[0216] In addition, in combination with the method for determining image parameters in the above embodiments, the present application embodiment may provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any one of the methods for determining image parameters in the above embodiments is implemented.

[0217] An embodiment of the present application also provides a computer program product. When instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes a method for determining image parameters as provided in an embodiment of the present application.

[0218] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0219] The functional blocks shown in the structural block diagram described above can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application-specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link through a data signal carried in a carrier. "Machine-readable medium" may include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROMs, flash memories, erasable ROMs (EROMs), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, and the like. The code segments can be downloaded via computer networks such as the Internet, intranets, and the like. It should also be noted that the exemplary embodiments mentioned in the present application describe some methods or systems based on a series of steps or devices. However, the present application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or they can be different from the order in the embodiment, or several steps can be performed simultaneously.

[0220] The above describes various aspects of the present application with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the functions / actions specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0221] The above is only a specific implementation of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.

Claims

1. A method for determining image parameters, characterized in that: include: Acquire a first orthographic image and a second orthographic image, wherein the first orthographic image and the second orthographic image have an image overlap area; Performing image matching on the first orthographic image and the second orthographic image to determine matching feature point pairs between the first orthographic image and the second orthographic image; constructing a geographic coordinate function of a mapped geographic point of the feature point pair based on the pixel coordinates of the feature point pair and the image parameters to be determined of the first orthographic image and the second orthographic image; constructing a loss function based on the geographic coordinate function of the mapped geographic points of the feature point pairs; Solve the image parameters to be determined of the first orthographic image when the loss function satisfies a preset loss condition.

2. The method according to claim 1, characterized in that The loss function is related to the distance between the mapped geographical points of the feature point pair, and the loss function is constructed based on the geographical coordinate function of the mapped geographical points of the feature point pair, including: determining a geographic coordinate deviation between the first orthographic image and the second orthographic image based on a first geographic coordinate of the first orthographic image and a second geographic coordinate of the second orthographic image; The loss function is constructed based on the geographic coordinate deviation and the geographic coordinate function of the mapped geographic point of the feature point pair.

3. The method according to claim 2, characterized in that Determining a geographic coordinate deviation between the first orthographic image and the second orthographic image based on a first geographic coordinate of the first orthographic image and a second geographic coordinate of the second orthographic image includes: Determining the first geographic coordinates based on the latitude and longitude parameters of the first orthographic image; Determining the second geographic coordinates based on the latitude and longitude parameters of the second orthographic image; Based on the first geographic coordinates and the second geographic coordinates, a geographic coordinate deviation between the first orthographic image and the second orthographic image is determined.

4. The method according to claim 1, characterized in that: The image parameter to be determined includes at least one of a yaw angle, a pitch angle, a roll angle, and an elevation.

5. The method according to any one of claims 1 to 4, characterized in that: Constructing a geographic coordinate function of a mapped geographic point of the feature point pair based on the pixel coordinates of the feature point pair and the image parameters to be determined of the first orthographic image and the second orthographic image, comprising: constructing a first three-dimensional rotation matrix based on the yaw angle, the pitch angle, and the roll angle of the first orthographic image; constructing a first scaling matrix based on the elevation of the first orthographic image, wherein at least one of the yaw angle, the pitch angle, the roll angle and the elevation is the image parameter to be determined; Acquire an intrinsic parameter matrix of an image acquisition device, where the image acquisition device is an image acquisition device that captures the first orthographic image and the second orthographic image; A geographic coordinate function of a mapping geographic point of the first feature point is constructed based on the pixel coordinates of the first feature point in the feature point pair, the intrinsic parameter matrix, the first three-dimensional rotation matrix, and the first scaling matrix.

6. The method according to any one of claims 1 to 4, characterized in that Constructing a geographic coordinate function of a mapped geographic point of the feature point pair based on the pixel coordinates of the feature point pair and the image parameters to be determined of the first orthographic image and the second orthographic image, comprising: constructing a second three-dimensional rotation matrix based on the yaw angle, the pitch angle, and the roll angle of the second orthographic image; constructing a second scaling matrix based on the elevation of the second orthographic image, wherein at least one of the yaw angle, the pitch angle, the roll angle and the elevation is the image parameter to be determined; Acquire an intrinsic parameter matrix of an image acquisition device, where the image acquisition device is an image acquisition device that captures the first orthographic image and the second orthographic image; A geographic coordinate function of mapping the second feature point to a geographic point is constructed based on the intrinsic parameter matrix, the pixel coordinates of the second feature point in the feature point pair, the second three-dimensional rotation matrix, and the second scaling matrix.

7. The method according to any one of claims 1 to 4, characterized in that Constructing a loss function based on the geographic coordinate function of the mapped geographic point of the feature point pair includes: constructing a sub-loss function corresponding to the feature point pair based on a geographic coordinate function of the mapped geographic points of the feature point pair between the first orthoimage and any of the second orthoimages, wherein the sub-loss function is related to the distance between the mapped geographic points of the feature point pair; The loss function is constructed based on the sub-loss functions of the feature point pairs between the first orthographic image and a plurality of the second orthographic images.

8. The method according to claim 7, characterized in that Constructing the loss function based on the sub-loss function of the feature point pairs between the first orthographic image and the plurality of second orthographic images comprises: determining a total number of the feature point pairs between the first orthographic image and a plurality of the second orthographic images; The loss function is constructed based on the sub-loss functions of the feature point pairs between the first orthoimage and a plurality of the second orthoimages and a total number of the feature point pairs.

9. The method according to any one of claims 1 to 4, characterized in that: The acquiring of the first orthographic image and the second orthographic image comprises: The second orthographic image having an image overlapping area with the first orthographic image is determined from the plurality of orthographic images.

10. The method according to claim 9, characterized in that The step of determining the second orthographic image having an image overlap area with the first orthographic image from the plurality of orthographic images comprises: determining a geographical distance between the first orthographic image and each of the orthographic images based on the geographical coordinates of the first orthographic image and each of the orthographic images; Determining whether the geographical distance meets a preset condition; When the geographical distance satisfies the preset condition, the orthographic image is determined as the second orthographic image.

11. The method according to any one of claims 1 to 4, characterized in that: Before performing image matching on the first orthographic image and the second orthographic image to determine the feature point pairs that match between the first orthographic image and the second orthographic image, the method further includes: Distortion correction is performed on the first orthographic image and the second orthographic image to obtain the first orthographic image and the second orthographic image after distortion correction, and the first orthographic image and the second orthographic image after distortion correction are used to extract the feature points.

12. A device for determining image parameters, characterized in that: include: An acquisition unit, configured to acquire a first orthographic image and a second orthographic image, wherein the first orthographic image and the second orthographic image have an image overlap area; an image matching unit, configured to perform image matching on the first orthographic image and the second orthographic image, and determine a pair of feature points that match between the first orthographic image and the second orthographic image; a first construction unit, configured to construct a geographic coordinate function of a mapping geographic point of the feature point pair based on pixel coordinates of the feature point pair and image parameters to be determined of the first orthographic image and the second orthographic image; A second construction unit, configured to construct a loss function based on the geographic coordinate function of the mapped geographic point of the feature point pair; A calculation unit is used to solve the image parameters to be determined of the first orthographic image when the loss function satisfies a preset loss condition.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by a processor, the method for determining image parameters according to any one of claims 1 to 11 is implemented.

14. A computer program product, characterized in that When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the method for determining image parameters as described in any one of claims 1 to 11.

Images