Method for assigning geographical coordinates

Abstract

The invention relates to a method for assigning geographical coordinates to image coordinates of a camera image, wherein optically coded markers (4) having reference points (7) are arranged in the camera image region (3). The detected image coordinates of the reference points (7) are assigned, on the basis of the marker IDs, to the geographical coordinates, which were determined by means of a receiver of a satellite navigation system integrated into a portable device (8), wherein the process is repeated with a plurality of differently coded markers (4) in a plane (1).

Description

[0001] Methods for assigning geographic coordinates

[0002] The invention relates to a method for assigning geographical coordinates to image coordinates of a camera image.

[0003] The invention further relates to a system for carrying out such a method.

[0004] The assignment of geographic coordinates to image coordinates of depicted objects in the camera image of a camera whose position is fixed to a plane is necessary in order to be able to assign objects across the viewing areas of several cameras, to be able to determine the speeds of the objects, or to enable cameras attached to mobile robots to determine the distance to the target or obstacle for target or obstacle detection.

[0005] To determine the geographic coordinates of objects depicted in the camera image of a camera whose position is fixed to a plane, transfer functions are known that assign geographic coordinates of the depicted plane to each pixel of the camera. This involves a mathematical transformation based on the camera's image coordinates, which converts them into corresponding geographic coordinates of the depicted plane. Calculating the parameters of the transfer functions requires a dataset of at least four corresponding image coordinates and geographic coordinates. Typically, to create such a dataset, characteristic objects are identified in the image, their position in the plane is determined geodetically, for example, from georeferenced aerial photographs, and their image coordinates are manually marked in the image. The disadvantages of this method are its time and complexity.Further disadvantages include inaccuracies in the dataset, stemming from the determination of geographic coordinates and the manual marking of characteristic objects. Particularly for camera systems used for object detection, which are regularly set up at different locations, an automated, user-friendly, and precise method is desirable.

[0006] The invention therefore aims to determine several corresponding image coordinates and geographical coordinates in a camera image of a plane whose position is fixed to the camera.

[0007] Furthermore, the invention aims to at least partially fulfill the following requirements:

[0008] • The process should be largely automated and easy to use.

[0009] • The geographical image coordinates should be determined precisely.

[0010] • The device should be inexpensive to manufacture.

[0011] • The image coordinates should be determined precisely.

[0012] • The procedure should also be feasible in the event of temporary partial obscuration of the plane, for example by traffic.

[0013] • The success and progress of the data set collection should be immediately assessable by the user.

[0014] To solve this problem, the invention, according to a first aspect, provides a method comprising the steps: a) arranging a marker having an optical coding in the image area of ​​a camera, wherein the optical coding comprises an identifier of the marker and wherein the marker has a reference point identifiable by an image evaluation,

[0015] b) Evaluating the camera image to capture the image coordinates of the reference point and the identifier, c) Storing the image coordinates together with the marker identifier,

[0016] d) Positioning a receiver of a satellite navigation system integrated into a portable device at the marker's reference point and recording the geographic coordinates and identifier,

[0017] e) Storing the geographic coordinates together with the marker identifier,

[0018] f) Matching the stored geographic coordinates to the stored image coordinates using the matching identifier,

[0019] g) Repeat steps a) to f) with a plurality of markers having different identifiers, which are arranged in a plane distributed in the image area of ​​the camera.

[0020] The use of optically coded markers with unique identifiers and characteristic reference points enables precise and automated acquisition of image coordinates. Manual positioning of a portable device with an integrated satellite navigation receiver at the markers' reference points ensures accurate determination of the geographic coordinates. By matching the stored image and geographic coordinates based on the corresponding marker identifiers, a precise data set for the transfer functions is generated. The method according to the invention is characterized by its simplicity and user-friendliness, as the markers can be removed from the image area after the geographic coordinates have been acquired and do not require permanent installation.The optical coding of the markers enables reliable detection of the characteristic reference points using known algorithms, even with partial occlusion of the plane. By capturing multiple markers step by step, the user receives immediate feedback on the progress and success of the data set creation.

[0021] Overall, the invention offers a cost-effective, precise and efficient solution for the assignment of geographic coordinates to image coordinates, which is particularly advantageous for camera systems regularly set up at different locations for object acquisition.

[0022] The accuracy of the geographic coordinate acquisition plays a crucial role in the precision of the entire process. Preferably, a satellite navigation receiver is used in the portable device, utilizing RTK (Real-Time Kinematic) corrected GNSS (Global Navigation Satellite System) signals from multiple navigation satellite constellations, such as GPS and Galileo. This achieves highly accurate determination of geographic coordinates. RTK technology enables real-time correction of the satellite signals by receiving and processing correction data from a reference station. By combining signals from different satellite systems and applying advanced error correction algorithms, centimeter-level accuracy is achieved.

[0023] In a preferred embodiment, positioning the satellite navigation system receiver at the marker reference point comprises positioning an antenna of the receiver at the reference point, preferably at a distance of less than 10 cm, preferably less than 5 cm, and in particular less than 2 cm from the reference point.

[0024] Preferably, as part of step d), it may be provided that an indicator light on the portable device signals to the user stable reception and sufficient accuracy of the data from the satellite navigation receiver and thus operational readiness.

[0025] In a preferred embodiment of the invention, the marker's reference point is part of the optical coding. This means that the marker's optical coding not only contains the unique identifier but also integrates the characteristic reference point, at which the geographic coordinates are recorded, directly into the code pattern.

[0026] In one exemplary embodiment of the invention, the optical coding of the markers is implemented as an AprilTag or QR code. These established coding methods are characterized by their robust detectability in images, even under varying lighting conditions or partial obscuration. The error correction mechanisms incorporated in AprilTags or QR codes ensure reliable and unambiguous identification of the marker. Furthermore, these coding methods enable precise localization of the marker's reference point with sub-pixel accuracy. By applying edge detection and interpolation algorithms, the image coordinates of the reference point can be determined with a resolution finer than that of individual pixels in the camera image.Preferably, the marker is positioned at a distance from the camera such that its size in the camera image is at least 40 pixels. This ensures robust and precise marker detection.

[0027] The image analysis performed in step b) of the method according to the invention enables the acquisition of the optical coding and the reference point, as well as the decoding of the marker's identifier. The image processing methods required for this can include pattern recognition techniques to locate the marker's optical coding within the image. Techniques such as edge detection, Hough transform, or template matching can be used for this purpose. Once the coding is located, the marker's reference point is determined. Depending on the type of coding, this can be done using methods such as centroid calculation, vertex detection, or the identification of specific landmarks within the code. Furthermore, the optical coding is decoded to extract the marker's unique identifier.For this purpose, the procedures corresponding to the respective coding scheme are applied, such as the decoding of QR codes or AprilTags. The result of the decoding is the marker's identifier, which is used for the subsequent steps of the process.

[0028] In a preferred embodiment of the invention, the markers are implemented as printed panels. Using printed panels allows for cost-effective production of the markers, as established printing processes can be utilized. The panels can be made from various materials, such as paper, cardboard, or plastic, depending on the requirements for durability and weather resistance. A further advantage of printed panels lies in their easy adaptability to different camera distances. By selecting appropriate panel sizes, it can be ensured that the markers are always clearly visible in the camera image, regardless of the distance between the camera and the surface.

[0029] The described method does not require all markers to be positioned simultaneously within the camera's image area.

[0030] Rather, it is possible to arrange and capture the markers sequentially, so that the procedure can be carried out even if the plane is partially obscured, for example by traffic.

[0031] The identifier of each marker is crucial for assigning geographic coordinates to image coordinates. This identifier is determined both during the acquisition of the image coordinates (step b) and during the acquisition of the geographic coordinates (step d). In step b), the identifier is acquired by analyzing the image to decode the identifier encoded in the optical code. Several methods are conceivable for acquiring the identifier in step d).

[0032] In a preferred embodiment of the invention, the identifier is acquired in step d) using input devices on the portable device, preferably via keys or a user interface. One possibility is to label the four markers with the numbers one to four and to provide corresponding number keys on the portable device. The user positions the portable device at the characteristic point of each marker and confirms this by pressing the associated number key. Once the portable device has been positioned at all markers, the process is complete for the user, and the subsequent steps are automated. This embodiment is characterized by its ease of use, as the user only needs to perform a few simple steps.

[0033] An alternative implementation involves equipping the portable device with a display that shows the user the identification of the next marker. The user positions the device at the displayed marker and sends the position by pressing a button on the device. In this version, no separate number keys are required, as the markers are identified via the display.

[0034] Both implementation methods simplify operation for the user and reduce potential sources of error when assigning geographic coordinates to the markers. The input devices on the portable device ensure that the recorded coordinates are correctly linked to the corresponding markers. The automated processing after the user input is complete contributes to the efficiency and user-friendliness of the method according to the invention.

[0035] In a preferred embodiment of the invention, the identifier is acquired in step d) by wirelessly reading an electronic memory of the marker. For this purpose, the markers are equipped, for example, with RFID (Radio-Frequency Identification) tags near the respective reference point, while the portable device is equipped with an RFID reader. The RFID tags store unique identifiers that are assigned to the respective markers. When the user positions the portable device at the reference point of a marker, the RFID reader automatically reads the unique identifier of the tag and assigns it to the acquired geographic coordinates. This embodiment replaces the manual entry of the marker identification by the user using the numeric keypad of the portable device. Instead, only a single button is required, which triggers the transmission of the coordinates.The use of RFID technology further automates and simplifies the data collection process.

[0036] Preferably, the method is implemented server-based, with the server running separately, as a cloud solution, or integrated into the camera. The camera and the portable device are equipped with mobile data modems or other wireless telecommunications interfaces, and the progress of the data acquisition is immediately reported back to the user on a webpage, which the user can access, for example, on their mobile phone. This allows the user to immediately assess the success and progress of the data acquisition.

[0037] In another embodiment, the function of the portable device is fulfilled by a software application running on a smartphone, preferably equipped with an RTK-GNSS receiver.

[0038] In general, it should be noted that steps a) to g) of the invention need not necessarily be carried out in the specified order, unless a particular step requires the prior execution of another step. For example, step b) requires the prior execution of step a), and step c) presupposes the execution of step b). Likewise, step e) requires the prior execution of step d). With respect to a particular marker, step f) requires that, for that marker, steps a) to c) and steps d) to e) have been carried out beforehand.

[0039] Repeating steps a) to f) as described in step g) does not require that all steps a) to f) have already been performed for a marker. If all markers remain in the image area, steps a) to c) can first be performed for all markers (e.g., first step a) for all markers, then steps b) and c) for all markers), followed by steps d) to e) for all markers. The mapping according to step f) can then be performed for all markers collectively.

[0040] The repetition of steps a) to f) is preferably performed for at least four different markers.

[0041] The camera used in step b) of the method according to the invention can be implemented in various forms. In a preferred embodiment, a digital camera integrated into a handheld device, such as a smartphone or tablet, is used. This embodiment offers the advantage that many users already own a suitable device, thus avoiding additional acquisition costs. Furthermore, the cameras built into modern handheld devices are generally of high quality and provide images with sufficient resolution and sharpness for carrying out the method. An alternative embodiment involves the use of a standalone digital camera specifically optimized for the method. This camera can be equipped with a higher-quality lens and a larger image sensor to achieve even better image quality.According to a second aspect, the invention relates to a system for carrying out a method according to the first aspect of the invention, comprising a plurality of markers having an optical coding, a data processing device, a digital camera and a portable device with an integrated receiver of a satellite navigation system, wherein the optical coding comprises a unique identifier for each marker and wherein the marker has a reference point identifiable by an image evaluation, wherein the portable device has input or reading means for capturing the identifier, and wherein the portable device, the camera and the data processing device each have communication interfaces to allow data transmission from the portable device to the data processing device and from the camera to the data processing device, wherein the system is configured to carry out the following steps:

[0042] Evaluating the camera image in the camera or data processing unit to capture the image coordinates of the reference point and the identifier, and, if necessary, transmitting them to the data processing unit.

[0043] Storing the image coordinates together with the marker identifier in the data processing device, capturing the geographic coordinates of the marker's reference point and the identifier by the portable device and transmitting them to the data processing device,

[0044] Store, in the data processing facility, the geographic coordinates together with the marker identifier,

[0045] Assigning the stored geographic coordinates to the stored image coordinates in the data processing unit using the matching identifier,

[0046] Repeat the preceding steps with a plurality of markers having different identifiers.

[0047] Further features and advantages of the invention will become apparent from the following description of a preferred embodiment of the invention which does not limit the scope of protection, with reference to the attached drawings.

[0048] Figure 1 shows a perspective view of the camera and the plane, Figure 2 a perspective view of the camera and the plane with the markers arranged therein, Figure 3 the camera image of the plane with the markers arranged therein, Figure 4 a view of a marker and Figure 5 the portable device.

[0049] Figure 1 shows that camera 1 is directed towards a plane 2. Camera 1 is fixed relative to this plane 2. According to Figures 2 and 3, four different markers 4, each with a unique identifier and an optical code 6, are temporarily arranged in the image area 3 of camera 1. The markers 4 are labeled with a digit 5 ​​and are uniquely identified by this digit. The optical code 6 includes an identifier corresponding to the digit and has a reference point 7, which is marked on the marker 4 and can be located in the camera image by known recognition algorithms.

[0050] As shown in Fig. 5, an indicator light 9 on the portable device 8 signals stable reception and sufficient accuracy of the data from the satellite navigation receiver integrated into the portable device 8, and thus operational readiness. Number keys 1 to 4 11 are arranged on the portable device. The user confirms the positioning of the portable device 8 with the cylindrical target marker 12 at the reference point 7 of the respective marker 4 using the number keys 11. The portable device 8 then transmits its geographic coordinates, along with the number of the pressed number key, to a server via a cellular modem integrated into the portable device 8, which stores this data. The geographic coordinates are determined from the GNSS signals of the GPS and Galileo satellite navigation constellations received by the antenna integrated into the target marker 12, as well as from correction data (RTK) received by the integrated cellular modem.

[0051] For the user, the process is now complete; the markers can be removed from the camera's field of view. During this process, the images from digital camera 1 are transmitted to the server and stored via a cellular modem integrated into the camera. The geographic coordinates transmitted when the number keys are pressed, together with the coordinates of the characteristic points of the respective markers determined from the camera images stored at the time of their transmission, form a dataset of corresponding image coordinates and geographic coordinates. This dataset is sufficient to assign the parameters for the transfer functions to all image coordinates of the camera with geographic coordinates.

Claims

Patent claims:

1. Method for assigning geographic coordinates to image coordinates of a camera image, comprising: a) Arranging a marker (4) having an optical coding ( 6) in the image area (3) of a camera ( 1 ), wherein the optical coding ( 6) comprises an identifier of the marker (4 ) and wherein the marker (4 ) has a reference point (7) identifiable by an image evaluation, b) Evaluating the camera image to record the image coordinates of the reference point (7) and the identifier, c) Storing the image coordinates together with the identifier of the marker (4), d) Positioning a receiver of a satellite navigation system integrated into a portable device ( 8 ) at the reference point (7 ) of the marker (4 ) and recording the geographic coordinates and the identifier, e) Storing the geographic coordinates together with the marker identifier (4 ) , f) Matching the stored geographic coordinates to the stored image coordinates using the matching identifier, g) Repeating steps a) to f) with a plurality of markers (4) having different identifiers, which are arranged distributed in a plane (2) in the image area (3) of the camera ( 1 ).

2. Method according to claim 1, characterized in that the detection of the identifier in step d) is carried out using input means, preferably keys ( 11 ) or a user interface, of the portable device ( 8 ).

3. Method according to claim 1, characterized in that the detection of the identifier in step d) is carried out by wirelessly reading an electronic memory of the marker (4 ).

4. Method according to claim 1, 2 or 3, characterized in that the optical coding ( 6 ) of the marker (4 ) has the reference point (7 ).

5. Method according to one of claims 1 to 4, characterized in that the positioning of the satellite navigation system receiver at the reference point (7 ) of the marker (4 ) comprises the positioning of an antenna of the receiver at the reference point (7 ) , preferably at a distance of less than 5 cm, in particular less than 2 cm from the reference point (7 ).

6. System for carrying out a method according to any one of claims 1 to 5, comprising a plurality of markers (4) having an optical coding (6), a data processing device, a digital camera (1), and a portable device (8) with an integrated receiver of a satellite navigation system, wherein the optical coding (6) comprises a unique identifier for each marker (4), and wherein the markers (4) have a reference point (7) identifiable by an image evaluation, wherein the portable device (8) has input or read means for capturing the identifier, and wherein the portable device (8), the camera (1), and the data processing device each have a communication interface to allow data transmission from the portable device (8) to the data processing device and from the camera (1) to the data processing device, wherein the system is configured to perform the following steps: Evaluating, in the camera or the data processing device, the camera image of the camera ( 1 ) to record the image coordinates of the reference point (7 ) and the identifier and, if necessary, transmitting them to the data processing device, Storing, in the data processing device, the image coordinates together with the marker identifier, capturing the geographic coordinates of the reference point (7) of the marker (4) and the identifier by the portable device (8) and transmitting them to the data processing device, Store, in the data processing facility, the geographic coordinates together with the identifier of the marker ( 4 ) , Assigning the stored geographic coordinates to the stored image coordinates in the data processing unit using the matching identifier, Repeat the preceding steps with a plurality of markers having different identifiers ( 4 ).

Images