A method enabling mutual guidance between aircraft and ground forces using mobile devices

Abstract

The invention relates to a method enabling ground forces to mark a target point on a photo taken with a mobile device, calculate the coordinates and distance from the current location, and instantly share this data with aircraft to guide them to the designated point.

Description

[0001] DESCRIPTION

[0002] A METHOD ENABLING MUTUAL GUIDANCE BETWEEN AIRCRAFT

[0003] AND GROUND FORCES USING MOBILE DEVICES

[0004] Technical Field

[0005] The invention relates to a method enabling ground forces to mark a target point on a photo taken with a mobile device camera, calculate the coordinate information of the selected point and the distance from the current location, and instantly share this information with aircraft so they can be directed to the designated point.

[0006] Prior Art

[0007] In military operations, it is crucial to quickly and effectively direct both ground and air units to the correct location for mission success. The ability of ground forces to accurately and rapidly indicate perceived threats to aircraft, and vice versa, is of great importance.

[0008] In existing techniques, ground forces use devices such as laser pointers to direct aircraft. These devices are not cost-effective. Alternatively, target locations are described verbally via radio communication. This method leads to challenges in describing the location, potential confusion with similar locations, and thus misdirection.

[0009] Upon examining the current state of the art, a need has arisen for the development of a method allowing ground forces to mark a target on a photo taken with a mobile device, calculate the coordinates and distance, and instantly share this information with aircraft for precise navigation.

[0010] Objectives of the Invention

[0011] The objective of the invention is to develop a method allowing ground forces to mark a target point on a photo taken by a mobile device, calculate the coordinates and distance from the current location, and share this data in real time with aircraft to direct them accurately. Another objective is to develop a method to eliminate the need for hardware such as laser designators and radios, enabling more effective coordination between ground forces and aircraft.

[0012] Detailed Description of the Invention

[0013] The invention relates to a method that enables the sharing of information in real time with aerial vehicles and the guidance of aerial vehicles to a specified point, after calculating the coordinate information of the selected point and the distance from the current location by marking the target point identified on a photograph taken with a mobile device camera by ground forces, it comprises;

[0014] Taking a photo using a mobile device camera,

[0015] Storing the photo along with location data (latitude, longitude, altitude) and orientation (yaw, pitch, roll) at the time of capture in the device memory, Correcting the photo using camera calibration techniques,1

[0016] Determining the coordinates of the selected point using the Moller- Trumbore ray-triangle intersection algorithm2and marking a target point on the calibrated photo by user,

[0017] Calculating the distance, direction, and angle information of the target point relative to the current location using a ray tracing algorithm2based on the determined coordinates,

[0018] Transmitting the calculated data to ground and air units via wireless network, SMS, or email,

[0019] Classifying the location in the captured image as a safe, risky, or dangerous area using convolutional neural networks (CNN) and Long Short-Term Memory (LSTM) methods,

[0020] Sending the classification result to ground or air units.

[0021] In the method of the invention, the user opens the camera of a mobile device (such as a smartphone or tablet) and takes a photo by pointing the camera toward an area that is considered potentially threatening. At the moment the photo is taken, the device’s location information (latitude, longitude, altitude) and orientation angles (yaw, pitch, roll) are recorded along with the photo.

[0022] The camera sensor model is calculated using parameters such as camera resolution and pixel size. The captured photo is then corrected using camera calibration techniques.1

[0023] Then, a target point is marked on the calibrated photo, and using the location and orientation data, the coordinates of the selected point are determined using the Moller-Trumbore ray-triangle intersection algorithm.2

[0024] Based on the determined coordinates, the distance, direction, and angle of the target point relative to the current location are calculated using a ray tracing algorithm.2

[0025] Subsequently, the user’s current location, the target point specified by other units, elevation information, and identifying features are displayed on the photo used for the calculation. The distance and direction of the units to the target point can also be calculated. The calculated identifying feature information is transmitted via message tones or email.

[0026] The unit transmitting the coordinates will activate a message alert system indicating the security status of the corresponding coordinate point (safe area / risky area / dangerous area) and notify the other units accordingly.

[0027] Initially, many photos taken with mobile phones were labeled into three different classes (safe area, risky area, and dangerous area) according to the level of threat. A deep learning model was trained on these labeled data. Now, when we input a new photo into the model, it can automatically classify it. This means that instead of the user having to make a judgment, the Al model can make its own decisions, enabling it to be used quickly and effectively in the field.

[0028] Artificial intelligence (Al) processes the image to assess area safety using CNN and LSTM-based models, either individually or combined. This Al evaluation provides automatic safety assessments, including potential terror threats, terrain steepness, and geographic intelligence.

[0029] The classified safety information is sent to air units via message or email, categorized into three levels (safe, risky, dangerous). Distinct alert tones for each level enable operators to quickly assess whether the target location is suitable for engagement. Thus, both geographical and intelligence data of the target point are transferred to the air unit.

[0030] The captured image will be processed using a pre-trained artificial intelligence model to automatically assess the security risk status of the area. This Al model can be developed by employing convolutional neural networks (CNN) and long shortterm memory (LSTM) based networks either individually or in combination. By processing the captured image, information regarding the security status of the relevant region will be obtained automatically. Geographical and intelligence data such as the presence of potential terrorist threats, the terrain conditions, and sharp elevation transitions within the marked area will be extracted using the Al model. The threat status information will be transmitted to the air unit via message or email. The threat level will be classified into three categories: safe area, risky area, and dangerous area. For each threat level, different alert tones will be played through the application to enable the operator to quickly recognize the situation. In other words, the message tone will inform the operator at the air unit whether it is appropriate to proceed to that point. Thus, both the coordinates and elevation data of the targeted location, as well as related geographical and intelligence information, will be communicated to the air unit.

[0031] References:

[0032] [1] Barbara Frank, Cyrill Stachniss, Giorgio Grisetti, Kai Arras, Wolfram Burgard Robotics 2: Camera Calibration (http: / / ais.informatik.uni- freiburg.de / teaching / wsl0 / robotics2 / pdfs / rob2-10-camera-calibration.pdf) [2] Tomas Moller & Ben Trumbore (1997) Fast, Minimum Storage Ray-Triangle Intersection, Journal of Graphics Tools, 2: 1, 21-28, DOI:

[0033] 10.1080 / 10867651.1997.10487468.

Claims

CLAIMS1. A method that enables the sharing of information in real time with aerial vehicles and the guidance of aerial vehicles to a specified point, after calculating the coordinate information of the selected point and the distance from the current location by marking the target point identified on a photograph taken with a mobile device camera by ground forces, characterized in that, it comprises;Taking a photo using a mobile device camera,Storing the photo along with location data (latitude, longitude, altitude) and orientation (yaw, pitch, roll) at the time of capture in the device memory,Correcting the photo using camera calibration techniques,1Determining the coordinates of the selected point using the Moller- Trumbore ray-triangle intersection algorithm2and marking a target point on the calibrated photo by user,Calculating the distance, direction, and angle information of the target point relative to the current location using a ray tracing algorithm2based on the determined coordinates,Transmitting the calculated data to ground and air units via wireless network, SMS, or email,Classifying the location in the captured image as a safe, risky, or dangerous area using convolutional neural networks (CNN) and Long Short-Term Memory (LSTM) methods,Sending the classification result to ground or air units.

2. A method accoring to claim 1, characterized in that the mobile device is a smartphone, tablet, or similar.