Spatial coordinate determination method, apparatus, and computer-readable storage medium

Abstract

The application provides a space coordinate determination method, device and computer readable storage medium, and belongs to the technical field of positioning. The method comprises the following steps: acquiring a first image and a second image of a target object; wherein the first image and the second image are images of the target object at different angles, and the first image comprises a target point corresponding to the target object; calculating an epipolar equation of the target point in the second image according to the target point in the first image; determining a pixel coordinate of the target point in the second image according to the epipolar equation of the target point in the second image; and determining a space coordinate of the target point according to the pixel coordinate of the target point in the first image and the pixel coordinate of the target point in the second image. The space coordinate positioning of the target object can be realized by shooting the target object at different angles. In the case that the GNSS signal is weak, the space positioning of the target object is relatively accurate, and the positioning accuracy in the case that the GNSS signal is weak is improved.

Description

Technical Field

[0001] The present application relates to the field of positioning technology, and in particular to a method and device for determining spatial coordinates and a computer-readable storage medium. Background Art

[0002] Currently, the Global Navigation Satellite System (GNSS) is mostly used to locate targets. Specifically, a GNSS receiver uses satellite pseudoranges, ephemeris, satellite launch time and other observation quantities to locate the target.

[0003] However, in certain specific scenarios (such as obstructions by trees, houses, tunnels, severe weather conditions, etc.), GNSS signals are susceptible to interference, resulting in low positioning accuracy of the GNSS for targets. Summary of the Invention

[0004] The purpose of the embodiments of the present application is to provide a method, device and computer-readable storage medium for determining spatial coordinates, so as to solve the problem that the positioning accuracy of the target object is low when the GNSS positioning method is subjected to signal interference.

[0005] In a first aspect, the present invention provides a method for determining spatial coordinates, the method comprising: acquiring a first image and a second image of a target object; wherein the first image and the second image are images of the target object at different angles, and the first image includes a target point corresponding to the target object; based on the target point in the first image, calculating the kernel line equation of the target point in the second image; based on the kernel line equation of the target point in the second image, determining the pixel coordinates of the target point in the second image; and determining the spatial coordinates of the target point based on the pixel coordinates of the target point in the first image and the pixel coordinates of the target point in the second image.

[0006] The spatial coordinate determination method designed above first obtains the first image and the second image of the target object at different angles, and then calculates the kernel equation of the target point in the second image based on the target point in the first image, and determines the pixel coordinates of the target point in the second image based on the kernel equation of the target point in the second image, and finally determines the spatial coordinates of the target point based on the pixel coordinates of the target point in the first image and the pixel coordinates of the target point in the second image, so that the spatial coordinate positioning of the target object can be achieved by photographing the target object at different angles, solving the problem of low positioning accuracy of the target object caused by signal interference in the GNSS positioning method, obtaining more accurate spatial positioning of the target object when the GNSS signal is weak, and improving the positioning accuracy when the GNSS signal is weak.

[0007] In an optional embodiment of the first aspect, determining the pixel coordinates of the target point in the second image based on the kernel equation of the target point in the second image includes: generating multiple to-be-matched pixel grids based on the kernel equation of the target point in the second image; generating a target pixel grid based on the first image and the target point in the first image; determining an optimal matching pixel grid based on the target pixel grid and the multiple to-be-matched pixel grids; determining whether the optimal matching pixel grid meets preset matching requirements; and if it is determined that the optimal matching pixel grid meets the preset matching requirements, determining the pixel coordinates of the target point in the second image based on the optimal matching pixel grid. This embodiment finds the area in the second image that is most similar to the target point by matching the multiple to-be-matched pixel grids of the second image with the target pixel grid of the first image, thereby determining the pixel coordinates of the target point in the second image, thereby achieving accurate determination of the pixel coordinates of the target point in the second image.

[0008] In an optional embodiment of the first aspect, generating multiple pixel grids to be matched based on the epipolar equation of the target point in the second image includes performing a multiple translation of the epipolar equation of the target point in the second image at a preset translation interval of pixels to obtain multiple pixel grids to be matched of a preset size. This embodiment simplifies and rapidly generates multiple pixel grids to be matched by performing multiple translations of the epipolar equation.

[0009] In an optional embodiment of the first aspect, determining an optimal matching pixel grid based on a target pixel grid and a plurality of to-be-matched pixel grids includes: calculating the disparity and similarity between the target pixel grid and each to-be-matched pixel grid; and determining the optimal matching pixel grid among the plurality of to-be-matched pixel grids based on the disparity and similarity between the target pixel grid and each to-be-matched pixel grid. This embodiment determines the optimal matching pixel grid based on the disparity and similarity between the target pixel grid and the to-be-matched pixel grid, thereby improving the accuracy of the optimal matching pixel grid obtained by matching.

[0010] In an optional implementation of the first aspect, after determining whether the best matching pixel grid meets the preset matching requirements, the method further includes: if it is determined that the best matching pixel grid does not meet the preset matching requirements, increasing the pixel sizes of the to-be-matched pixel grid and the target pixel grid, and determining a second best matching pixel grid based on the multiple to-be-matched pixel grids and the target pixel grid after the sizes are increased, until the second best matching pixel grid meets the preset matching requirements.

[0011] In an optional embodiment of the first aspect, determining whether the optimal matching pixel grid meets the preset matching requirements includes: obtaining the pixel coordinates of the center point of the optimal matching pixel grid; determining whether the pixel coordinates of the center point of the optimal matching pixel grid are within a preset range of the epipolar equation; if the pixel coordinates of the center point of the optimal matching pixel grid are determined to be within the preset range of the epipolar equation, determining that the optimal matching pixel grid meets the preset matching requirements; if the pixel coordinates of the center point of the optimal matching pixel grid are determined to be not within the preset range of the epipolar equation, determining that the optimal matching pixel grid does not meet the preset matching requirements. Because the correct area of ​​the target point in the second image must be near the epipolar line, this embodiment determines whether the optimal matching pixel grid meets the preset matching requirements by determining whether the pixel coordinates of the center point of the optimal matching pixel grid are within the preset range of the epipolar equation, thereby improving the accuracy of the determined optimal matching pixel grid.

[0012] In an optional implementation of the first aspect, determining the pixel coordinates of the target point in the second image based on the optimal matching pixel grid includes: obtaining the pixel coordinates of the center point of the optimal matching pixel grid, and using the pixel coordinates of the center point of the optimal matching pixel grid as the pixel coordinates of the target point in the second image.

[0013] In an optional implementation of the first aspect, calculating the kernel line equation of the target point in the second image based on the target point in the first image includes: obtaining a pre-calibrated camera pose, the camera pose including the pose from the camera coordinate system to the UTM coordinate system; calculating the kernel line equation of the target point in the second image based on the pre-calibrated camera pose, the pixel coordinates of the target point in the first image, and the second image.

[0014] In a second aspect, the present invention provides a spatial coordinate determination device, which includes: an acquisition module, a calculation module and a determination module; the acquisition module is used to acquire a first image and a second image of a target object; wherein the first image and the second image are images of the target object at different angles, and the first image includes a target point corresponding to the target object; the calculation module is used to calculate the kernel line equation of the target point in the second image based on the target point in the first image; the determination module is used to determine the pixel coordinates of the target point in the second image based on the kernel line equation of the target point in the second image; and, determine the spatial coordinates of the target point based on the pixel coordinates of the target point in the first image and the pixel coordinates of the target point in the second image.

[0015] The spatial coordinate determination device designed above, this scheme first obtains the first image and the second image of the target object at different angles, and then calculates the kernel line equation of the target point in the second image based on the target point in the first image, and determines the pixel coordinates of the target point in the second image based on the kernel line equation of the target point in the second image, and finally determines the spatial coordinates of the target point based on the pixel coordinates of the target point in the first image and the pixel coordinates of the target point in the second image, so that the spatial coordinate positioning of the target object can be achieved by shooting the target object at different angles, solving the problem of low positioning accuracy of the target object caused by signal interference in the GNSS positioning method, obtaining more accurate spatial positioning of the target object when the GNSS signal is weak, and improving the positioning accuracy when the GNSS signal is weak.

[0016] In an optional implementation of the second aspect, the determination module is specifically configured to generate a plurality of pixel squares to be matched based on an epipolar line equation of the target point in the second image; generate a target pixel square based on the first image and the target point of the first image; determine an optimal matching pixel square based on the target pixel square and the plurality of pixel squares to be matched; determine whether the optimal matching pixel square meets preset matching requirements; and if it is determined that the optimal matching pixel square meets the preset matching requirements, determine the pixel coordinates of the target point in the second image based on the optimal matching pixel square.

[0017] In an optional implementation of the second aspect, the determination module is further specifically configured to translate the epipolar equation of the target point in the second image multiple times according to a preset translation spacing of pixels to obtain multiple pixel grids of a preset size to be matched.

[0018] In an optional implementation of the second aspect, the determination module is further specifically configured to calculate the disparity and similarity between the target pixel grid and each pixel grid to be matched; and determine the optimal matching pixel grid among multiple pixel grids to be matched based on the disparity and similarity between the target pixel grid and each pixel grid to be matched.

[0019] In an optional implementation manner of the second aspect, the determination module is further specifically configured to, if it is determined that the optimal matching pixel grid does not meet the preset matching requirements, increase the pixel sizes of the to-be-matched pixel grid and the target pixel grid, and determine a second optimal matching pixel grid based on the multiple to-be-matched pixel grids and the target pixel grid after the sizes are increased, until the second optimal matching pixel grid meets the preset matching requirements.

[0020] In an optional implementation of the second aspect, the determination module is further specifically used to obtain the pixel coordinates of the center point of the optimal matching pixel grid; determine whether the pixel coordinates of the center point of the optimal matching pixel grid are within a preset range of the epipolar line equation; if it is determined that the pixel coordinates of the center point of the optimal matching pixel grid are within the preset range of the epipolar line equation, then determine that the optimal matching pixel grid meets the preset matching requirements; if it is determined that the pixel coordinates of the center point of the optimal matching pixel grid are not within the preset range of the epipolar line equation, then determine that the optimal matching pixel grid does not meet the preset matching requirements.

[0021] In an optional implementation of the second aspect, the determination module is further specifically configured to obtain the pixel coordinates of the center point of the optimal matching pixel grid, and use the pixel coordinates of the center point of the optimal matching pixel grid as the pixel coordinates of the target point in the second image.

[0022] In an optional implementation of the second aspect, the calculation module is specifically used to obtain a pre-calibrated camera pose, which includes the pose from the camera coordinate system to the UTM coordinate system; based on the pre-calibrated camera pose, the pixel coordinates of the target point in the first image and the second image, calculate the kernel line equation of the target point in the second image.

[0023] In a third aspect, the present application provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it executes the method in the first aspect or any optional implementation of the first aspect.

[0024] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method in the first aspect or any optional implementation of the first aspect is executed.

[0025] In a fifth aspect, the present application provides a computer program product, which, when running on a computer, enables the computer to execute the method in the first aspect or any optional implementation of the first aspect.

[0026] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0028] Figure 1 A schematic diagram of the flow of the spatial coordinate positioning method provided in an embodiment of the present application;

[0029] Figure 2 A schematic structural diagram of a spatial coordinate positioning device provided in an embodiment of the present application;

[0030] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0031] Icons: 200 - acquisition module; 210 - calculation module; 220 - determination module; 3 - electronic device; 301 - processor; 302 - memory; 303 - communication bus. DETAILED DESCRIPTION

[0032] The following embodiments of the technical solution of the present application will be described in detail with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present application and are therefore only examples and are not intended to limit the scope of protection of the present application.

[0033] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application belongs; the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit this application; the terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned figure descriptions are intended to cover non-exclusive inclusions.

[0034] In the description of the embodiments of this application, the technical terms "first" and "second" are used only to distinguish different objects and should not be understood to indicate or imply relative importance or implicitly specify the quantity, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, the meaning of "plurality" is more than two, unless otherwise clearly and specifically defined.

[0035] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0036] In the description of the embodiments of this application, the term "and / or" is simply a description of the association relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent the following three situations: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this document generally indicates that the associated objects are in an "or" relationship.

[0037] In the description of the embodiments of the present application, the term "multiple" refers to more than two (including two). Similarly, "multiple groups" refers to more than two groups (including two groups), and "multiple pieces" refers to more than two pieces (including two pieces).

[0038] At present, the positioning of target objects is mostly achieved by the Global Navigation Satellite System (GNSS). For example, the spatial position of vehicles and people is mostly achieved by the Global Navigation Satellite System (GNSS). When GNSS positioning is achieved, the GNSS receiver usually locates the target object using observation quantities such as satellite pseudorange, ephemeris, and satellite launch time.

[0039] The inventors of this application have discovered that in certain specific scenarios, such as when obstructed by trees, houses, and tunnels, and in severe weather conditions (such as foggy days), the global navigation satellite system signal is easily interfered with, making the global navigation satellite system signal weak, which in turn leads to low positioning accuracy of the global navigation satellite system for the target object.

[0040] In response to the above problems, the inventors of this application have designed a method, device and computer-readable storage medium for determining spatial coordinates. The method uses a monocular camera, an inertial measurement unit (IMU) and a GNSS to fuse information, and calculates the spatial coordinates of the target point by searching for points of the same name in multiple images of the target object. The spatial coordinates of the target object can be determined by photographing the target object from multiple angles, effectively solving the problem of poor positioning accuracy in environments with weak GNSS signals, and improving the positioning accuracy and convenience of the target object.

[0041] Based on the above ideas, the present application provides a method for determining spatial coordinates, which can be applied to computing devices, including but not limited to computers, servers, controllers, chips, and GNSS receivers, etc. Figure 1 As shown, the spatial coordinate determination method can be implemented by the following methods, including:

[0042] Step S100: Acquire a first image and a second image of a target object.

[0043] Step S110: Calculate the epipolar equation of the target point in the second image according to the target point in the first image.

[0044] Step S120: determining the pixel coordinates of the target point in the second image according to the epipolar equation of the target point in the second image.

[0045] Step S130: determining the spatial coordinates of the target point according to the pixel coordinates of the target point in the first image and the pixel coordinates of the target point in the second image.

[0046] In the above embodiment, the first image and the second image of the target object are images of the target object taken from different angles. Specifically, this solution can capture the target object from multiple angles using a camera, thereby obtaining multiple images of the target object. Based on this, the user can select a target point corresponding to the target object in any of the multiple images of the target object. The image with the selected target point becomes the first image of the target object, while the image without the selected point becomes the second image of the target object. The target point corresponding to the target object selected by the user can be any point on the target object in the first image, or any point around the target object in the first image, and this solution does not impose any specific limitations on this.

[0047] As a possible implementation, the second image may contain only one image. For example, a camera is used to shoot the target object at two different angles to obtain two images A1 and A2 of the target object. The user selects the target point P on image A1. Then, according to the description of this solution, image A1 is the first image and image A2 is the second image.

[0048] As another possible implementation, the second image may include multiple images. For example, a camera is used to capture the target object from five different angles to obtain five images A1, A2, A3, A4, and A5 of the target object. The user selects the target point P on image A3. Then, according to the description of this solution, image A3 is the first image; images A1, A2, A4, and A5 are all second images.

[0049] In addition, it should be noted here that multiple images of the target object from multiple angles can be collected and stored in advance. After the user selects the target point P, the above process can be executed to determine the spatial coordinates of the target point P; of course, in addition to the method of collecting and storing in advance, multiple images of the target object from multiple angles can be captured and obtained in real time.

[0050] After acquiring the first image and the second image of the target object in the above manner, this solution can calculate the epipolar equation of the target point in the second image based on the target point in the first image.

[0051] It should be noted here that this solution can calibrate the camera and other sensors in advance. Specifically, the camera of this solution can be embedded in the GNSS receiver. In terms of space, this solution can use the existing open source calibration method to calibrate the camera's intrinsic parameters, the extrinsic parameters between the camera and the IMU in advance, and the attitude of the camera coordinate system and the Universal Transverse Mercator Grid System (UTM) coordinate system corresponding to the GNSS; this solution can calculate the translation parameters of the satellite navigation positioning antenna phase center relative to the IMU center based on the structural design obtained by the GNSS receiver. In terms of time, this solution can use time synchronization and interpolation processing to adjust the sampling frequency of the IMU and the camera, so that the IMU and the camera collect data at the same time, thereby obtaining sensor and camera information in the same time and space.

[0052] Based on the above, this solution can calculate the epipolar equation of the target point in the second image in the following way. Specifically, through the above calibration, the posture of the camera coordinate system to the Universal Transverse Mercator Grid System (UTM) coordinate system, that is, the camera pose, can be obtained, that is, T = (R1|t1), where R1 represents the transformation matrix between the camera and UTM, and t1 represents the translation vector between the camera and UTM.

[0053] Then the posture of the first image and the corresponding second image can be obtained through posture transformation, that is,

[0054] Then through the essential matrix and the basic matrix we can get:

[0055] E=t Λ R;

[0056] F=K-TRK -1 ;

[0057] Where t represents the translation matrix of the first image and the second image, R represents the rotation matrix of the first image and the second image, and K represents the camera intrinsic parameter. Based on the above conditions, the coefficients of the epipolar equation of the target point P in the second image can be obtained as follows:

[0058] L = FP;

[0059] Wherein, P is the pixel coordinate of the target point P in the first image, and L is the coefficients a, b, and c of the epipolar equation ax+by-c=0, thereby obtaining the epipolar equation of the target point in the second image.

[0060] It should be noted here that, when the second image consists of multiple images, the parameters of different second images can be adjusted to the parameters of the corresponding image.

[0061] After obtaining the epipolar equation of the target point in the second image in the above manner, the pixel coordinates of the target point in the second image, i.e., the pixel coordinates of the same name point of the target point in the second image, can be determined based on the epipolar equation of the target point in the second image.

[0062] As a possible implementation, this solution may first generate a plurality of pixel grids to be matched according to the epipolar line equation of the target point in the second image.

[0063] Specifically, the epipolar equation of the target point in the second image is translated multiple times at a preset pixel translation interval to obtain multiple pixel grids of a preset size to be matched. For example, this solution can translate the epipolar equation of the target point in the second image multiple times at a translation interval of 1 pixel, with the epipolar equation as the center and N / 2 above and below, to obtain multiple N*N size grids to be matched.

[0064] In addition, this solution can also generate a target pixel grid based on the first image and the target point in the first image. Specifically, a matching grid of a preset size is generated with the target point in the first image as the center. For example, this solution generates an N*N target grid centered on the target point P in the first image, where N is the number of pixels. For example, when N is 3, the size of the matching grid and the target grid are both 3*3 pixels.

[0065] Based on the above, this solution determines the optimal matching square among the multiple matching squares based on the target pixel square and the multiple matching squares. Because this solution captures images of the target object from multiple angles, the target point will be present in both the first and second images. The target point's location in the first image and the target point's location in the second image will undoubtedly be similar. Therefore, as a possible implementation, this solution can calculate the disparity and similarity between the target pixel square and each matching pixel square, and determine the optimal matching pixel square based on the disparity and similarity between the target pixel square and each matching pixel square.

[0066] Specifically, this solution can assign corresponding weights to disparity and similarity. For example, disparity corresponds to the first weight and similarity corresponds to the second weight. On this basis, this solution can calculate disparity * first weight + similarity * second weight to obtain the consideration value for each pixel grid to be matched, thereby determining the pixel grid to be matched with the largest consideration value as the optimal matching pixel grid.

[0067] In addition to the above methods, this solution can use disparity alone or similarity alone to determine the optimal matching pixel grid. For example, this solution can determine the matching pixel grid with the smallest disparity with the target pixel grid among multiple matching pixel grids as the optimal matching pixel grid; or this solution can determine the matching pixel grid with the greatest similarity with the target pixel grid among multiple matching pixel grids as the optimal matching pixel grid.

[0068] After determining the optimal matching pixel grid in the above manner, this solution can determine whether the optimal matching pixel grid meets the preset matching requirements. Specifically, the epipolar line means the correct area of ​​the target point P in the second image, so the position of the same-name point of the target point P in the second image should be within a certain range of the epipolar line of the target point in the second image. Therefore, this solution can specifically determine whether the pixel coordinates of the center point of the optimal matching pixel grid are within the preset range of the epipolar line equation. If the pixel coordinates of the center point of the optimal matching pixel grid are determined to be within the preset range of the epipolar line equation, then the optimal matching pixel grid is determined to meet the preset matching requirements; if the pixel coordinates of the center point of the optimal matching pixel grid are determined not to be within the preset range of the epipolar line equation, then the optimal matching pixel grid is determined to not meet the preset matching requirements.

[0069] For example, this solution can determine whether the pixel coordinates of the center point of the optimal matching pixel square are within the range of 3 pixels above and below the epipolar line. If the pixel coordinates of the center point of the optimal matching pixel square are within the range of 3 pixels above and below the epipolar line, then the optimal matching pixel square is determined to meet the preset matching requirements; if the pixel coordinates of the center point of the optimal matching pixel square are not within the range of 3 pixels above and below the epipolar line, then the optimal matching pixel square is determined to not meet the preset matching requirements.

[0070] As a possible implementation, if the best-matching pixel grid is determined to meet preset matching requirements, the present solution determines the pixel coordinates of the target point in the second image based on the best-matching pixel grid. Specifically, the present solution may obtain the pixel coordinates of the center point of the best-matching pixel grid and use the pixel coordinates of the center point of the best-matching pixel grid as the pixel coordinates of the target point in the second image.

[0071] As another possible implementation, if the present solution determines that the optimal matching pixel grid does not meet the preset matching requirements, the present solution may increase the pixel sizes of the to-be-matched pixel grid and the target pixel grid, and determine a second optimal matching pixel grid based on the increased pixel sizes of the to-be-matched pixel grid and the target pixel grid, until the second optimal matching pixel grid meets the preset matching requirements. For example, if the size of the to-be-matched pixel grid and the target pixel grid described above is 3*3, and if the optimal matching pixel grid is determined not to meet the preset matching requirements, the present solution may increase N to 4 or 5, thereby repeating the previously described steps of determining the pixel coordinates of the target point in the second image.

[0072] In addition, it should be noted that before dividing the grid into multiple to-be-matched squares, this solution can also determine whether there is an offset in the x and y directions of the epipolar line based on the slope of the epipolar line equation. If there is an offset, epipolar line correction can be performed to simplify the calculation, thereby changing the first image and the second image from a two-dimensional matching search to a one-dimensional matching search.

[0073] After obtaining the pixel coordinates of the target point in the second image in the above manner, this solution can determine the spatial coordinates of the target point based on the pixel coordinates of the target point in the first image and the pixel coordinates of the target point in the second image.

[0074] As a possible implementation, this solution can use feature point triangulation to determine the spatial coordinates of the target point using the pixel coordinates of the target point in the first image and the pixel coordinates of the target point in the second image. Of course, in addition to using feature point triangulation, this solution can also use any other method that converts image coordinate points into three-dimensional spatial coordinates.

[0075] In addition, since the second image of this scheme described above may include multiple images, on this basis, this scheme can use the pixel coordinates of the target point in multiple second images to respectively determine the multiple spatial coordinates of the target point, and then analyze the multiple spatial coordinates (such as weighted averaging, etc.) to obtain the final spatial coordinates of the target point, thereby improving the accuracy of the spatial coordinates of the target point.

[0076] The spatial coordinate determination method designed above first obtains the first image and the second image of the target object at different angles, and then calculates the kernel equation of the target point in the second image based on the target point in the first image, and determines the pixel coordinates of the target point in the second image based on the kernel equation of the target point in the second image, and finally determines the spatial coordinates of the target point based on the pixel coordinates of the target point in the first image and the pixel coordinates of the target point in the second image, so that the spatial coordinate positioning of the target object can be achieved by photographing the target object at different angles, solving the problem of low positioning accuracy of the target object caused by signal interference in the GNSS positioning method, obtaining more accurate spatial positioning of the target object when the GNSS signal is weak, and improving the positioning accuracy when the GNSS signal is weak.

[0077] Figure 2 The present application provides a schematic structural block diagram of a spatial coordinate determination device. It should be understood that the device is Figure 1 The method embodiment executed in corresponds to the embodiment of the method, and can execute the steps involved in the aforementioned method. The specific functions of the device can be found in the description above. To avoid repetition, detailed description is appropriately omitted here. The device includes at least one software function module that can be stored in a memory in the form of software or firmware or solidified in the operating system (OS) of the device. Specifically, the device includes: an acquisition module 200, a calculation module 210 and a determination module 220; the acquisition module 200 is used to acquire a first image and a second image of the target object; wherein the first image and the second image are images of the target object at different angles, and the first image includes a target point corresponding to the target object; the calculation module 210 is used to calculate the epipolar equation of the target point in the second image based on the target point in the first image; the determination module 220 is used to determine the pixel coordinates of the target point in the second image based on the epipolar equation of the target point in the second image; and, based on the pixel coordinates of the target point in the first image and the pixel coordinates of the target point in the second image, the spatial coordinates of the target point are determined.

[0078] The spatial coordinate determination device designed above, this scheme first obtains the first image and the second image of the target object at different angles, and then calculates the kernel line equation of the target point in the second image based on the target point in the first image, and determines the pixel coordinates of the target point in the second image based on the kernel line equation of the target point in the second image, and finally determines the spatial coordinates of the target point based on the pixel coordinates of the target point in the first image and the pixel coordinates of the target point in the second image, so that the spatial coordinate positioning of the target object can be achieved by shooting the target object at different angles, solving the problem of low positioning accuracy of the target object caused by signal interference in the GNSS positioning method, obtaining more accurate spatial positioning of the target object when the GNSS signal is weak, and improving the positioning accuracy when the GNSS signal is weak.

[0079] In an optional implementation manner of this embodiment, the determination module 220 is specifically configured to generate a plurality of pixel squares to be matched based on the epipolar line equation of the target point in the second image; generate a target pixel square based on the first image and the target point in the first image; determine an optimal matching pixel square based on the target pixel square and the plurality of pixel squares to be matched; determine whether the optimal matching pixel square meets a preset matching requirement; and if it is determined that the optimal matching pixel square meets the preset matching requirement, determine the pixel coordinates of the target point in the second image based on the optimal matching pixel square.

[0080] In an optional implementation of this embodiment, the determination module 220 is further specifically configured to translate the epipolar equation of the target point in the second image multiple times according to a preset translation interval of pixels to obtain multiple pixel grids to be matched of a preset size.

[0081] In an optional implementation manner of this embodiment, the determination module 220 is further specifically configured to calculate the disparity and similarity between the target pixel grid and each pixel grid to be matched; and determine the best matching pixel grid among the multiple pixel grids to be matched based on the disparity and similarity between the target pixel grid and each pixel grid to be matched.

[0082] In an optional implementation manner of this embodiment, the determination module 220 is further specifically configured to increase the pixel sizes of the to-be-matched pixel square and the target pixel square if it is determined that the best matching pixel square does not meet the preset matching requirement, and determine a second best matching pixel square based on the multiple to-be-matched pixel squares and the target pixel square after the sizes are increased, until the second best matching pixel square meets the preset matching requirement.

[0083] In an optional implementation manner of this embodiment, the determination module 220 is further specifically used to obtain the pixel coordinates of the center point of the optimal matching pixel grid; determine whether the pixel coordinates of the center point of the optimal matching pixel grid are within a preset range of the epipolar line equation; if it is determined that the pixel coordinates of the center point of the optimal matching pixel grid are within the preset range of the epipolar line equation, then the optimal matching pixel grid is determined to meet the preset matching requirements; if it is determined that the pixel coordinates of the center point of the optimal matching pixel grid are not within the preset range of the epipolar line equation, then the optimal matching pixel grid is determined to not meet the preset matching requirements.

[0084] In an optional implementation of this embodiment, the determination module 220 is further specifically configured to obtain the pixel coordinates of the center point of the optimal matching pixel grid, and use the pixel coordinates of the center point of the optimal matching pixel grid as the pixel coordinates of the target point in the second image.

[0085] In an optional implementation of this embodiment, the calculation module 210 is specifically used to obtain a pre-calibrated camera pose, which includes the pose from the camera coordinate system to the UTM coordinate system; based on the pre-calibrated camera pose, the pixel coordinates of the target point in the first image and the second image, calculate the kernel line equation of the target point in the second image.

[0086] According to some embodiments of the present application, Figure 3 As shown, the present application provides an electronic device 3, including: a processor 301 and a memory 302, the processor 301 and the memory 302 are interconnected and communicate with each other through a communication bus 303 and / or other forms of connection mechanisms (not shown), and the memory 302 stores a computer program executable by the processor 301. When the computing device is running, the processor 301 executes the computer program to execute the method executed in the aforementioned implementation, such as steps S100 to S130: acquiring a first image and a second image of the target object; calculating the kernel line equation of the target point in the second image based on the target point in the first image; determining the pixel coordinates of the target point in the second image based on the kernel line equation of the target point in the second image; determining the spatial coordinates of the target point based on the pixel coordinates of the target point in the first image and the pixel coordinates of the target point in the second image.

[0087] The present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the aforementioned execution method is executed.

[0088] Among them, the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0089] The present application provides a computer program product, which enables the computer to execute the aforementioned method when running on the computer.

[0090] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application, and they should all be included in the scope of the claims and specification of the present application. In particular, as long as there is no structural conflict, the various technical features mentioned in the various embodiments can be combined in any way. The present application is not limited to the specific embodiments disclosed herein, but includes all technical solutions that fall within the scope of the claims.

Claims

1. A method for determining spatial coordinates, characterized in that: The method comprises: Acquire a first image and a second image of a target object; wherein the first image and the second image are images of the target object at different angles, and the first image includes a target point corresponding to the target object; Calculating, based on the target point in the first image, an epipolar equation of the target point in the second image; determining the pixel coordinates of the target point in the second image according to the epipolar equation of the target point in the second image; determining the spatial coordinates of the target point according to the pixel coordinates of the target point in the first image and the pixel coordinates of the target point in the second image; Determining the pixel coordinates of the target point in the second image according to the epipolar equation of the target point in the second image includes: generating a plurality of pixel grids to be matched according to an epipolar line equation of the target point in the second image; generating a target pixel grid according to the first image and the target point of the first image; Determining an optimal matching pixel grid according to the target pixel grid and a plurality of pixel grids to be matched; Determining whether the optimal matching pixel grid meets preset matching requirements; If it is determined that the optimal matching pixel grid meets the preset matching requirement, the pixel coordinates of the target point in the second image are determined according to the optimal matching pixel grid.

2. The method according to claim 1, characterized in that Generating a plurality of pixel grids to be matched according to the epipolar equation of the target point in the second image includes: The epipolar line equation of the target point in the second image is translated multiple times according to a preset translation interval of pixels to obtain multiple pixel grids to be matched with a preset size.

3. The method according to claim 1, characterized in that Determining the optimal matching pixel grid according to the target pixel grid and the plurality of pixel grids to be matched includes: Calculating the disparity and similarity between the target pixel grid and each pixel grid to be matched; According to the disparity and similarity between the target pixel grid and each pixel grid to be matched, the best matching pixel grid is determined among the multiple pixel grids to be matched.

4. The method according to claim 1, wherein After determining whether the optimal matching pixel grid meets a preset matching requirement, the method further includes: If it is determined that the best matching pixel grid does not meet the preset matching requirements, the pixel sizes of the to-be-matched pixel grid and the target pixel grid are increased, and a second best matching pixel grid is determined based on the multiple to-be-matched pixel grids and the target pixel grid after the increased sizes, until the second best matching pixel grid meets the preset matching requirements.

5. The method according to claim 1, wherein Determining whether the optimal matching pixel grid meets a preset matching requirement includes: Obtaining the pixel coordinates of the center point of the optimal matching pixel grid; Determining whether the pixel coordinates of the center point of the optimal matching pixel grid are within a preset range of the epipolar equation; If it is determined that the pixel coordinates of the center point of the optimal matching pixel grid are within the preset range of the epipolar equation, then it is determined that the optimal matching pixel grid meets the preset matching requirements; If it is determined that the pixel coordinates of the center point of the optimal matching pixel grid are not within the preset range of the epipolar equation, it is determined that the optimal matching pixel grid does not meet the preset matching requirements.

6. The method according to claim 1, characterized in that Determining the pixel coordinates of the target point in the second image according to the optimal matching pixel grid includes: The pixel coordinates of the center point of the optimal matching pixel grid are obtained, and the pixel coordinates of the center point of the optimal matching pixel grid are used as the pixel coordinates of the target point in the second image.

7. The method according to claim 1, characterized in that The step of calculating the epipolar equation of the target point in the second image according to the target point in the first image includes: Obtain a pre-calibrated camera pose, where the camera pose includes the pose from the camera coordinate system to the UTM coordinate system; The epipolar line equation of the target point in the second image is calculated according to the pre-calibrated camera pose, the pixel coordinates of the target point in the first image, and the second image.

8. A spatial coordinate determination device, characterized in that: The device includes: an acquisition module, a calculation module and a determination module; The acquisition module is configured to acquire a first image and a second image of the target object; wherein the first image and the second image are images of the target object at different angles, and the first image includes a target point corresponding to the target object; The calculation module is used to calculate the epipolar equation of the target point in the second image according to the target point in the first image; The determining module is configured to determine the pixel coordinates of the target point in the second image based on the epipolar equation of the target point in the second image; and determine the spatial coordinates of the target point based on the pixel coordinates of the target point in the first image and the pixel coordinates of the target point in the second image; The determination module is specifically configured to generate a plurality of to-be-matched pixel grids based on an epipolar line equation of the target point in the second image; generate a target pixel grid based on the first image and the target point in the first image; determine an optimal matching pixel grid based on the target pixel grid and the plurality of to-be-matched pixel grids; determine whether the optimal matching pixel grid meets preset matching requirements; and if it is determined that the optimal matching pixel grid meets the preset matching requirements, determine the pixel coordinates of the target point in the second image based on the optimal matching pixel grid.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images