Self-Learning Command and Control Module for Navigation (GENISYS), and System Thereof

Abstract

Title: Self-Learning Commands and Control Module (GENISYS) for Navigation and Its System. A navigation system (300) for land, air, sea, or underwater transportation means (302) includes a remote control workstation (301) having a manual control mode (310), a mission planning mode (330), and a tactical control mode (360) that initiate command and control options; a navigation module (100) disposed retrofittably on the transportation means (302); and a plurality of perception sensors (318) disposed on the transportation means (302). The system (300) receives manual, electrical, wireless, and voice commands from a human operator (305) in the manual control (310) and mission planning modes (330) and converts those commands into a data set for training a navigation model having a navigation algorithm. The perception sensors (318) generate a data set for self-learning in real time in the manual control mode (310), the mission control mode (330), and the tactical control mode (360). The navigation system (300) navigates spontaneously whether or not a communication network (390) exists.

Description

Technical Field

[0001] Claim of Priority The present invention claims priority from the Indian patent application No. 202221024830, filed on July 27, 2022, with the title of the invention "Self-Learning Command and Control Module (GENISYS)".

[0002] The present invention relates to the control and management of the operation of means of transportation. Specifically, the present invention relates to the operation control and management of land, sea, or air transportation means. More specifically, the present invention relates to the operation control and management in a situation where communication from the base station of the means of transportation is lost.

Background Art

[0003] As land, sea, and air transportation means, those that are not only manned but also unmanned are known. The unknown routes and journeys of such means of transportation are not only predictable but also unpredictable.

[0004] Unmanned means of transportation are controlled by sensors and communication. The Chinese Patent Application No. 3067575 discloses a self-learning autonomous navigation for unmanned underwater vehicles, particularly based on an improved recurrent neural network. The Chinese Patent Application No. 109521454 discloses a navigation method based on a volumetric Kalman filter. The Chinese Patent Application No. 102778237 provides a map self-learning system in vehicle navigation. In this system, the self-learning function of the electronic navigation map in the vehicle navigation system is achieved by curve fitting and prediction algorithms, as a result, the positioning accuracy of the vehicle is improved and the electronic navigation map is completed.

[0005] In some cases, means of transportation that are manned but unmanned in different situations have problems. Such problems become more complex when communication is interrupted.

[0006] The present invention attempts to overcome this important industrial requirement. Object of the invention

[0007] One object of the present invention is to invent a device and method that capture individual instructions during operation by an on-board operator and incorporate such instructions as self-learning.

[0008] Another object of the present invention is to invent a device and method that capture instructions during operation by a remote operator and incorporate such instructions as self-learning.

[0009] Yet another object of the present invention is to invent a device and method that capture macro and micro instructions during operation by an on-board and remote operator and incorporate such instructions as self-learning.

[0010] Yet another object of the present invention is to invent a device that can be retrofitted to any means of conveyance to be suitable for autonomous operation, particularly when there is no communication network.

Summary of the Invention

[0011] The present invention is a navigation system having a navigation module named GENISYS that can be retrofitted to any manned or unmanned land / air / sea / underwater means of conveyance. GENISYS has the ability to capture all operator instructions and commands, including voice, manual, electrical, and wireless commands, and convert them into executable data / algorithms having time and position coordinates. GENISYS is connected, either wired or wirelessly, to a plurality of sensors and the intelligence of the means of conveyance. GENISYS incorporates the capabilities specific to the means of conveyance with executable data and algorithms constructed from the operator's instructions and commands.

[0012] A drone, hereinafter referred to as a platform or a means of transportation, equipped with a navigation module, can self-navigate to an obscured unknown destination or an obscured unknown target even when there is no communication network and while obstacles are hiding the direct line of sight to the target. In the following description, the hardware and software of the navigation module and the navigation system of the present invention that provide the above capabilities will be described in sequence.

[0013] The navigation system around the means of transportation includes · A remote control workstation or RCW, · A navigation module or GENISYS including a command and control unit, and · A plurality of perception sensors.

[0014] The three types of control deployed for the navigation module are 1. Manual control, 2. Mission planning, 3. Tactical control.

[0015] Manual control means direct human control, including throttle and handle control, either physically present on the tactical platform of the means of transportation or remotely, especially when the means of transportation is a land - suitable means of transportation, or an aircraft, or a drone or ship carrying a human, or a sub - sea or underwater means of transportation. In mission - type control, a complete plan is implemented before the start of the operation. This plan includes how the means of transportation should be executed and how it should behave in various scenarios. All commands and control factors for cases where it does not apply (if - else) or where it applies (if) are pre - planned and supplied to the RCW system. Data from multiple sensors is acquired, graphed, illustrated, processed, or used to generate a user interface and to perform multiple jobs such as control, monitoring, withdrawal, interference, blasting, etc. with the help of the system / means of transportation / platform. Tactical control includes a wide - ranging guidance and control system that includes a handshake with the mission plan for decision - making in important scenarios such as obstacle avoidance and response guidance scenarios, and multiple inputs and outputs. Tactical control maps the actual trajectory of the system to the desired trajectory. The difference is calculated and the offset is supplied to the system for learning to help the system gradually better match the actual trajectory with the desired trajectory.

[0016] The command control unit (CCU) comprises a self - learning command control unit, unmanned control, and an evaluation and correction platform. Manual control directly controls the engine rudder and throttle without autonomous operation. The control of the means of transportation sends a signal to the VDC (vehicle direct control) that manages all controls of the means of transportation. Here, the hybrid - signal - type control operates where the command control unit processes data and is passed on to the vehicle direct control. The self - learning command control unit consists of subsystems including vehicle guidance and course control.

[0017] Unmanned control mainly relies on multiple perception sensors. The data of the perception sensors are processed, evaluated for precision, and response guidance is provided to the mission plan.

[0018] For an autonomous maritime platform, a tactical platform can execute a set of various missions such as sea denial, escort, surveillance, logistics, and counter-target functions. Therefore, such a platform conducts asset planning by mission objective, data collection and analysis, asset availability and capabilities, optimization algorithms, resource allocation, dynamic adaptation, as well as communication and coordination. By combining data analysis, optimization algorithms, and compliance planning strategies, an autonomous maritime tactical platform can efficiently conduct asset planning for autonomous navigation vessels. This platform optimizes the utilization of assets, enhances mission fulfillment, and enables effective decision-making in a dynamic maritime environment. The evaluation and correction platform implements the actions and corrective measures to be taken to ensure that the tactical platform achieves its objectives.

[0019] The multiple perception sensors connected to the response guidance system include accelerometers, gyroscopes, compasses, magnetic heading sensors, barometric pressure sensors, multiple GNSS, visual sensors including cameras, stereo cameras, omnidirectional cameras, IR cameras, ultrasonic sensors, laser rangefinders, Li-Dar, sonars, radars, optical sensors, and depth sensors. Such types of sensor data using sensor fusion algorithms provide appropriate data. The sensor fusion algorithm is a computational method aimed at combining measurements from multiple sensors that jointly provide more precise information regarding the system being measured than any single sensor alone.

[0020] Data from such sensors, along with data from a human operator or third-party user or device, is useful for a human operator within the loop to perform important tasks at a remote control workstation (RCW). This data is used to perform course control of the platform or vehicle. Course control includes multiple movements of the platform or vehicle. Movements such as roll, pitch, yaw, right, left, port, starboard, altitude increase, altitude decrease, thrust, throttle, gear shift, etc. are thereby performed.

[0021] Therefore, those data from the sensors are used to obtain feedback from the system in a closed control loop, instead of monitoring the control and accepting data from the system in an open loop. Also, the current system is platform-agnostic, and thus can function in both a closed-loop and an open-loop system. In an open-loop system, the data is managed by cognitive / fusion sensor data management, and the decision / control / monitoring / management is completed at a remote control workstation (RCW).

[0022] The vehicle or tactical platform has vehicle direct control including vehicle actuators that are operable under manual control. Vehicle dynamics, including speed and inclination, etc., are affected by the type, weight, acceleration ability, night vision, turning radius, etc. of the vehicle, as well as by the navigation payload of such, and at the same time are affected by the environmental scenario including terrain parameters, and are applied to the output of the sensors to provide feedback to the mission plan.

[0023] In the manual control mode, commands from the human operator that cause changes detected by the throttle or changes to the steering wheel are saved in the self-learning module for subsequent autonomous operations. Importantly, the perception sensors do not actively participate in supporting navigation in manual control but continue to generate a real-time dataset for self-learning. Even when the vehicle is manually controlled, the perception sensors incompletely collect information on the surrounding environment, terrain, drift, atmospheric pressure, including the route, buildings, and distances between buildings, and construct a training dataset for use in mission planning and tactical control situations.

[0024] In the mission planning mode, the vehicle guidance module generates a route for the vehicle to reach a given destination relative to the home position by obtaining feedback from the navigation payload and the perception sensors to complete the assigned task. The course control module generates commands to the vehicle actuators regarding the assigned task. The perception sensors continue to generate a real-time dataset for self-learning regarding the assigned task while the vehicle guidance model updates the dataset in real time.

[0025] In the mission planning mode using automatic piloting in tactical control with situation awareness and interactive collision avoidance, the vehicle guidance module generates a path for the vehicle towards completing the assigned task of reaching a predetermined destination relative to the home position by obtaining feedback from the navigation payload and the perception sensors. The course control module generates commands to the vehicle actuators regarding the assigned task. The navigation system introduces an interactive payload such as AIS, an environmental scenario, and an evaluation and correction module for accurate navigation. The navigation system obtains collision-related data from prior information such as map data including trees, mountains, building information, and terrain information. The perception sensors continue to generate a dataset for self-learning in real time regarding the assigned task while the vehicle guidance model updates the dataset in real time.

[0026] In the mission planning mode using automatic piloting in tactical control with situation awareness and reactive collision avoidance, the navigation system uses data from cameras, LIDAR, sonar, ADSB in addition to the AIS that provides real-time data, and based on that, response guidance performs course control for collision avoidance (through the evaluation and correction module). The perception sensors continue to generate a dataset for self-learning in real time regarding the assigned task while the vehicle guidance model updates the dataset in real time.

[0027] A vehicle assigned a known destination relative to the home position calculates more than one possible path. The path prediction has predetermined basic settings and limitations based on a cognitive algorithm for a pre-trained model from past real-time datasets. The flight may use reactive collision avoidance technology or interactive collision avoidance technology.

[0028] Importantly, in all of the above embodiments, a communication network including Internet communication or satellite communication is available. The present invention is equally possible even when no communication network of any or all types exists. The transport means is assigned the task of reaching an unknown destination, which is a drowning person in need of help. The unknown destination is hidden behind a large obstacle, which is a ship. In this scenario, navigation is performed based on a local reference frame in the form of a virtual cube with situation-dependent X, Y, and Z dimensions generated by a deep learning grid algorithm. The X, Y, and Z dimensions are determined according to the situation based on the home position, the unknown destination, and the determination that the obstacle is fully within the cube. The transport means moves within the cube and recognizes its position relative to the home position with the help of a compass, gyroscope, accelerometer, acoustic sensor, camera, and other non-network devices and perception sensors. The transport means, here a drone, predicts a safe altitude based on the type of task input, weather conditions, and position information.

Brief Description of the Drawings

[0029]

Figure 1

Figure 2A

Figure 2B

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11A

Figure 11B

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Mode for Carrying Out the Invention

[0030] Hereinafter, the present invention will be described with the aid of the drawings. The detailed description is a preferred embodiment that can develop several variations, and therefore, the present invention should not be construed as limiting in any way.

[0031] The present invention is a navigation module named GENISYS that can be retrofitted to any manned or unmanned land / air / sea / subsea transportation means and its system. Preferred embodiments will be described in conjunction with the drawings of unmanned aerial transportation means generally known as drones.

[0032] GENISYS and its surrounding systems have the ability to capture all operator instructions and commands, including voice, manual, electrical, and wireless commands, and convert them into executable data / algorithms with time and position coordinates.

[0033] GENISYS is connected, either wired or wirelessly, to a plurality of sensors and the intelligence of the transportation means. GENISYS incorporates the capabilities specific to the transportation means with executable data and algorithms constructed from operator instructions and commands.

[0034] In FIG. 1, a drone, hereinafter referred to as a platform or a means of transportation (302) and arranged together with a navigation module (100), can self-navigate to an obstructed unknown destination or an obstructed unknown target (82), here a drowning person, even in the absence of a communication network. An obstacle (85), here a ship, is obstructing the direct line of sight to the target (82). In the following description, the hardware and software of the navigation module (100) of the present invention that provide the above capabilities will be described in sequence.

[0035] In FIGS. 2A-2B, a navigation system (300) around the means of transportation (302) includes · a remote control workstation or RCW (301), · a navigation module or GENISYS (100) including a command and control unit (311) and a means of transportation platform (302) including a tactical platform (302A), · a perception sensor (318).

[0036] The means of transportation (302) refers to a platform or a transportation system or a means of transportation that provides specific capabilities and affects the operating field. The means of transportation is essentially a platform or a means for delivering various assets, weapons, or equipment to the action space. Such a means of transportation / tactical platform (302 / 302A) varies in size, mobility, and function according to the operating requirements.

[0037] Remote control workstation or RCW (301) - RCW (301) or ground control station (GCS) is the section where all command and control options are initiated, which includes initial command and control by a human operator (305) responsible for all operations planned to be executed on a manual or semi-autonomous or autonomous system platform. Command and control includes complete control and monitoring of system data or system video, or payload data or payload control, or both (FIG. 3).

[0038] A human operator (305) is a person who controls, monitors, or both controls and monitors system data or system video, or payload data or payload control, or both. The human operator (305) performs a plurality of tasks such as manual operation (310), mission planning (330), and tactical control (360) by a remote control workstation (RCW) or a ground control station (GCS) (301).

[0039] Three types of control deployed in the navigation module (100) are 1. Manual control (310), 2. Mission planning (330), 3. Tactical control (360).

[0040] Manual control (310) means direct human control including throttle and steering control, either physically present on the tactical platform of the vehicle (302) or remotely, especially when the vehicle is a land - suitable vehicle, or an aircraft, or a drone carrying a human or a ship, or a marine or underwater vehicle. Here, the human operator (305) takes full responsibility for the operation. The human operator (305) needs to undertake all individual parameters or settings or states or decisions or tactical decisions of the system.

[0041] In mission plan (330) type of control, a complete plan is implemented before operation starts. This plan includes how the means of transportation should be executed and how the means of transportation should behave in which scenarios, and all commands and control factors for cases where they do not apply (if - else) or apply (if) are pre - planned and supplied to the RCW system. Once supplied, after start, the means of transportation / command control will execute the functions as set, and the RCW will not avoid the planned operations. Mission plan (330) is the formal definition of any type of unmanned platform or system or means of transportation where a complete mission or plan or route plan and the preparation and execution of the complete mission are carried out. The monitoring / control / managing of mission plan (330) and real - time data are fully incorporated into the system or platform. This helps to manage the complete route / plan / mission of the system / platform / means of transportation. Data from multiple sensors are acquired, graphed, or illustrated, or processed, or used to generate a user interface and execute multiple jobs such as control / monitoring / exit / obstruction / explosion with the help of the system / means of transportation / platform.

[0042] Tactical control (360) - Tactical control includes a wide - ranging guidance navigation and control system that includes a handshake with mission plan (330) for decision - making in important scenarios such as obstacle avoidance and response guidance, and includes multiple inputs and outputs. The wide - ranging guidance navigation includes a hardware - software correlation system that determines a desired movement path called a trajectory from the current position of the platform or means of transportation to the specified target position. Tactical control maps the actual trajectory of the system to the desired trajectory. The difference is calculated and the offset is supplied to the system for learning to help the system gradually match the actual trajectory and the desired trajectory better.

[0043] Generally, two horizontal dimensions: · Pitch or pitch angle (88), · Yaw or yaw angle (87), and · Roll or roll angle (86), are divided.

[0044] In FIG. 21, the movement of the auxiliary wing (86a) of the air transportation means controls roll (86), the movement of the rudder (87a) controls yaw (87), and the movement of the elevator (88a) controls pitch (88). These three route control outputs are mixed to generate fin commands from angular velocity to acceleration feedback. The control strategy is either a proportional-integral (PI) controller or a proportional-integral-derivative (PID) controller.

[0045] The command control unit or CCU (311) - manual control (310) functions together with the command control unit (311). The command control unit (CCU) includes a self-learning command control unit (314), unmanned control (320), and an evaluation and correction platform (320). Manual control directly controls the rudder and throttle of the engine without autonomous operation. Vehicle control sends a signal to the VDC (Vehicle Direct Control) that manages all controls of the vehicle. Here, the hybrid signal type of control operates where the command control unit processes data and is passed on to vehicle direct control (FIG. 4).

[0046] The self-learning command control unit (314) includes the following subsystems: · Vehicle guidance (316), · Course control (317).

[0047] The transportation means guidance system (316) is a part of the control structure of the transportation means and consists of a route generation means, a flight plan algorithm, and a sensor fusion module. The stereoscopic sensor and the GFPS sensor are used as position sensors. The trajectory for the operation of the transportation means is generated, as a first step, by using only the information from the digital map. Object detection sensors such as the stereoscopic sensor, three laser scanners, and a radar sensor observe the transportation means environment and report the detected objects to the sensor fusion module. This information is used to dynamically update the planned transportation means trajectory for the final operation of the transportation means. This helps the transportation means guidance system (316) to track corrections and complete navigation guidance.

[0048] The course control system (317) derives a model for transportation means route tracking and course control through the management of the throttle and the nose. The pitch angle and the yaw angle can be treated as control inputs together with the rapid climb rate.

[0049] Unmanned control (320) mainly relies on a plurality of perception sensors (318), and the data of the perception sensors are processed, evaluated for precision, and response guidance (319) is provided to the mission plan (330).

[0050] Asset plan (321): For an autonomous maritime platform, the tactical platform can execute various mission sets such as sea denial, escort, surveillance, logistics, and counter-target functions.

[0051] Therefore, such a platform will implement the asset plan as follows.

[0052] · Mission objectives: The tactical platform starts by defining the mission objectives based on the requirements or tasks to be achieved. These objectives can include patrolling a specific area, conducting search and rescue operations, monitoring maritime traffic, or any other mission-specific objectives.

[0053] · Data correction and analysis: The platform collects relevant data from various information sources to recognize the situation. This data can include real-time information on weather conditions, ship traffic, sensor readings, geographical features, and mission constraints. The platform analyzes this data to understand the current operating environment and identify potential risks or opportunities.

[0054] · Asset availability and capabilities: The platform freely evaluates the availability and capabilities of assets. This includes considering the type of autonomous navigation vessels, sensor suites, communication systems, durability, speed, and any other relevant characteristics. The platform also takes into account the operational constraints and limitations of each asset.

[0055] · Optimization algorithms: The tactical platform utilizes optimization algorithms to determine the most effective deployment and utilization of assets. These algorithms take into account factors such as asset capabilities, mission objectives, constraints, and operating parameters to generate an optimized plan. The plan can include routes, schedules, task assignments, and adjustment strategies.

[0056] · Resource allocation: Based on the optimized plan, the tactical platform assigns tasks and allocates resources to autonomous navigation vessels. The tactical platform ensures that assets are efficiently distributed to maximize coverage, minimize response time, and optimize overall mission fulfillment. This may include considering factors such as the proximity of assets to target areas, current status, and suitability for specific tasks.

[0057] ·Dynamic Adaptation: The tactical platform continuously monitors the operating environment and adapts the asset plan as needed. The tactical platform can dynamically adjust the plan in response to changes in the situation, the emergence of threats, or new mission requirements. This adaptability enables the platform to optimize asset allocation in real time and make decisions based on up-to-date information.

[0058] ·Communication and Coordination: The tactical platform facilitates communication and coordination among autonomous navigation vessels and other entities involved in the mission. The tactical platform establishes communication links, provides real-time updates, and enables collaboration among assets. This ensures that assets are synchronized, share relevant information, and jointly achieve mission objectives efficiently.

[0059] By combining data analysis, optimization algorithms, and adaptive planning strategies, the autonomous maritime tactical platform can efficiently perform asset planning for autonomous navigation vessels. This platform optimizes the utilization of assets, enhances mission fulfillment, and enables effective decision-making in a dynamic maritime environment.

[0060] Evaluation and Correction Platform (320): Based on environmental perception and recognition, a given task / objective, and the payload on board, the platform continuously evaluates the objective, the effects to be achieved, and whether corrections are necessary for execution. Subsequently, corrective measures are implemented and actions are taken to ensure that the tactical platform achieves the objective.

[0061] A plurality of perception sensors (318) connected to the response guidance system (319) include ·An accelerometer, ·A gyroscope, ·A compass, ·A magnetic heading sensor, ·An air pressure sensor, ·A plurality of GNSSs, · Visual sensors such as cameras, stereo cameras, omnidirectional cameras, IR cameras, · Ultrasonic sensors, · Laser rangefinders, · Li-Dar, sonar, radar · Optical sensors, · GNSS sensors, · Depth sensors, are included.

[0062] Sensor data of such types using sensor fusion algorithms provides appropriate data. Sensor fusion algorithms are said to be computational methods aimed at combining measurements from multiple sensors that jointly provide more precise information than any single sensor alone with respect to the system being measured.

[0063] Data from such sensors and data from a human operator or third-party user or device are useful for the human operator within the loop to perform important tasks at a remote control workstation (RCW). This data is used for course control of the platform or means of conveyance (302). Course control includes multiple movements of the platform or means of conveyance (302). Thereby, movements such as roll, pitch, yaw, right, left, port, starboard, altitude increase, altitude decrease, thrust, throttle, gear shift, etc. are performed.

[0064] Therefore, those data from the sensors are used to obtain feedback from the system in a closed control loop instead of monitoring the control and accepting data from the system in an open loop. Also, the current system is platform-agnostic and thus can function in both closed-loop and open-loop systems. In an open-loop system, the data is managed by cognitive / fusion sensor data management, and decision / control / monitoring / management is completed at a remote control workstation (RCW).

[0065] The self-learning command control unit (314) includes a plurality of FRAM and NVRAM memory storage devices that store main control codes in the directory of the boot controller.

[0066] In FIG. 5, the transport means (302) or the tactical platform (302A) has a transport means direct control (315) that includes a transport means actuator (323) operable under manual control (310). The transport means dynamics (324), including speed, inclination, etc., is affected by the type, weight, acceleration performance, night vision, turning radius, etc. of the transport means, and at the same time is affected by the environmental scenario / constraint (325) including terrain parameters, and is applied to the output of the sensor (318) to provide feedback to the mission plan (330).

[0067] In FIGS. 6 and 7, in the manual control mode (310), commands by a human operator that cause a change (326) detected at the throttle or a change (327) at the handle are stored in the self-learning module for subsequent autonomous operation (328). Importantly, the perception sensor does not actively participate in supporting navigation in manual control (310), but continues to generate a data set for self-learning in real time (329). Exemplarily, in FIG. 8, even if the transport means (302) is manually controlled, the perception sensor incompletely collects information on the surrounding environment, terrain, drift, air pressure, including the route, buildings (151) and the distance between buildings (152), and constructs a training data set for use in the mission plan (330) and tactical control (360) situations.

[0068] In FIGS. 9 and 10, in the mission planning mode (330) which is the semi-autonomous mode, the vehicle guidance module (316) generates a path for the vehicle (302) to reach a predetermined destination with respect to the home position by obtaining feedback from the navigation payload (326) and the perception sensor (318). The course control module (317) generates commands for the vehicle actuator (323) regarding the assigned task. While the vehicle guidance model updates the data set in real time (332), the perception sensor (318) continues to generate a data set for self-learning in real time regarding the assigned task (329).

[0069] In FIGS. 9, 11A to 11B, in the mission planning mode (330) using automatic piloting in tactical control having situation recognition and interactive collision avoidance, the tactical platform (302A) is equipped with a payload / smart alarm (361). The vehicle guidance module (316) generates a path for the vehicle (302) to reach a predetermined destination with respect to the home position by obtaining feedback from the navigation payload (326) and the perception sensor (318). The course control module (317) generates commands for the vehicle actuator (323) regarding the assigned task. The navigation system (100) introduces an interactive payload such as an aeronautical information service or AIS (362), an environmental scenario / constraint (325), and an evaluation and correction module (320) for accurate navigation. The navigation system (100) obtains collision-related data from prior information such as map data including tree, mountain, building information, and terrain information. While the vehicle guidance model updates the data set in real time (332), the perception sensor (318) continues to generate a data set for self-learning in real time regarding the assigned task (329).

[0070] In FIGS. 9 and 12, in a mission planning mode (330) with automatic piloting in tactical control with situation awareness and reactive collision avoidance, a vehicle guidance module (316) obtains feedback from a navigation payload (326) and a perception sensor (318) to generate a path for a vehicle (302) towards completion of an assigned task of reaching a predetermined destination relative to a home position. A course control module (317) generates commands to a vehicle actuator (323) regarding the assigned task. A navigation system (300) incorporates an interactive payload (362) such as an AIS, environmental scenarios / constraints (325), and an evaluation and correction module (320) for accurate navigation. The navigation system (300) uses data from a camera, LIDAR, sonar, ADSB in addition to an aviation information service (AIS) providing real-time data, and based on this, response guidance (319) performs course control for collision avoidance (through the evaluation and correction module (320)). The perception sensor (318) continues to generate a data set for real-time self-learning regarding the assigned task while the vehicle guidance model updates the data set in real time (332).

[0071] In FIGS. 9 and 13, vehicles (302) assigned a known destination (80) relative to a home position (81) calculate one or more possible paths. Safety altitude prediction (370) and path prediction (371) have predetermined basic settings and limitations based on a cognitive algorithm for a pre-trained model from past real-time data sets. If any object is detected en route, the vehicle (302) changes its path in a predetermined order (379). The flight may use reactive collision avoidance technology or interactive collision avoidance technology.

[0072] Importantly, in all of the above embodiments, a communication network (390) including Internet communication or satellite communication is available. The present invention is equally possible even when there is no arbitrary or any kind of communication network (390). In FIG. 1, the carrier means (302) is assigned the task of reaching an unknown destination (82), which is a drowning person (82a) in need of help. The unknown destination (82) is hidden behind a large obstacle (85) that is a ship. In this scenario, navigation is performed based on a local reference frame in the form of a virtual cube (90) with situation-dependent X, Y, and Z dimensions generated by a deep learning grid algorithm (FIG. 14). The cube (90) comprises a precise three-dimensional grid that generates a plurality of nodes (91) by the intersection of X, Y, and / or Z coordinates. While moving within the cube (90), the carrier means (302) recognizes its precise position by the corresponding node(s) (91). The X, Y, and Z dimensions are determined according to the situation based on the judgment that the home position (81), the unknown destination (82), and the obstacle (85) are sufficiently within the cube (90). The carrier means moves within the cube (90) and, with the help of a compass, gyroscope, accelerometer, acoustic sensor, camera, and other non-network devices and perception sensors (318), knows its relative position with respect to the home position (81). In FIGS. 1, 15, and 16, the carrier means (302), which is a drone here, predicts a safety altitude (373) based on the type of task manually input, weather conditions, and position information (372). Such information is relayed to the self-learning module (383) of the navigation module (100). Based on the safety altitude / cruise speed learned from past operations and the sensor gain adjusted by the drone itself, the drone spontaneously completes the task (384).

[0073] The grid is based on a deep learning algorithm and constitutes a grid with a coarser or finer pitch based on a predetermined task. As an example, for identifying the position of a person, the grid has a foot pitch, and when the destination is a ship or a building, the grid has a much longer pitch.

[0074] Search direction (380) - When the search direction cannot be confirmed, the home position (81) is at the center (375) of the cube (90), or when the search direction is confirmed, the home position is at the corner (374) of the cube (90) (Fig. 17). When the transport means (302) is a marine transport means, the home position (81) is at the upper edge (376) of the cube (90) (Fig. 18).

[0075] As a variant, the virtual cube (90) is an electromagnetic field or any other energy field locally generated by the transport means (302) or the tactical platform (302A) around it. Thus, the virtual cube (90) effectively navigates the transport means (302) or the tactical platform (302A), particularly with the help of a self-learning module (314) based on a sensor fusion algorithm, carefully ensuring precision and redundancy. As described above, the sensor fusion algorithm is a computational method introduced to analyze and compare sensor data related to mismatches and use "fused" data for training the AI model (377) instead of simply using captured data.

[0076] The related data includes, by way of example, directly or indirectly comparing measurement units from two or more sensors. Thus, geographical rules are applied to multiple orthogonal measurement units to calculate non-orthogonal measurement units.

[0077] Standalone electromechanical non-network-based sensors are generally known to be less accurate, and the transport means (302) obtains regular feedback from an AI model (377) based on a pre-trained cognitive navigation algorithm. In Fig. 19, the navigation switches between network-based navigation (391) and grid-based navigation (392) with intermittent communication network availability to continuously obtain an accurate trajectory until the assigned task is completed.

[0078] The algorithms and computer programs present in multiple layers are for the self-learning command control unit (314), in particular, · User interface, · In-communication layer, · Multiple vehicle libraries, · Here, these libraries determine the behavior of the control system according to vehicles that can be, for example, ground vehicles, UAVs, USVs, AUVs, etc. · Vehicle unique code, · Control and thread management layer, · Hardware, · Perception system or payload, · Sensors, · Payload control, · Exist during the operation of the sandbox layer for control.

[0079] User interface: The user interface is the medium for interaction between the human operator and the system. All control and monitoring parameters and control analogs are available on the user interface of the system.

[0080] In-communication layer: The in-communication layer is the layer where communication between the computer system and multiple payloads such as the perception system, input channels, output channels, etc. is managed. The most important part of this layer is to manage and initiate communication, including switching navigation from one communication network (390) to another communication network (390) using another communication channel if one channel of the communication medium is being affected. Therefore, all monitoring and control are carried out through this layer.

[0081] Multiple vehicle libraries: The operation control of multiple transport means related to all threads is managed by these libraries. With these libraries, the hardware can understand the type of transport means that a specific hardware and the corresponding software are scheduled to control. For example, when an automobile is about to be controlled from a specific hardware, the control and thread management layer uses the transport means library of the automobile type from the dataset, so that the software and the hardware can understand the type of commands required for that specific transport means.

[0082] Control and thread management layer: This layer enables the processor to control the clock and timing of the entire system so that multiple threads / multiple processes can operate simultaneously.

[0083] Hardware: In this layer, hardware is allocated. For example, compatibility issues can be resolved with the initialization of software drivers and DLLs required for specific hardware peripherals.

[0084] Perception system and payload: The payload and navigation sensors such as accelerometers, vision systems, compasses / heading sensors, gyroscopes, camera systems, etc. are allocated to this system.

[0085] Sandbox layer for control and testing: This layer is an isolated test environment that enables the user to execute and determine programs that can perform control without affecting the applications and systems or platforms being operated.

[0086] In FIGS. 23 and 24, the navigation module (100) has the following core hardware components: · Control unit (110), · Feedback unit (120), · Control processor (130), · Self-learning processor (140) - The self-learning processor (140) receives a training dataset and comprises a model having a cognitive navigation algorithm including subroutines according to an embodiment. · Sensor system (150) - The sensor system (150) receives inputs from the perception sensors (318) and converts all inputs suitable for PID control of the vehicle actuator (323). · Power distribution unit (160) including filters and isolation, · Power supply management (170), · Communication module (180).

[0087] The navigation system (300) equipped with the navigation module (100) controls the tactical platform or vehicle (302) in micro management with the help of a plurality of perception sensors (318), and all operator instructions and commands including voice, manual, electrical, and wireless commands are captured by the navigation algorithm using a deep learning-based command and control unit.

[0088] All pre-recorded position and relative time information is retrieved and compared to the position and relative time information that occurs during autonomous operation. The difference between the pre-recorded information and the actual information is used to generate an error signal, which is converted into a command signal and learned / self-learned. In FIG. 22, the core of the invention is the approximation of a predicted trajectory (381) and an actual trajectory (382) as a result of continuous real-time learning and transmission of a training data set to a model-executing navigation algorithm. The self-learning command and control system according to the invention functions closely with a human operator by utilizing a level of "managed autonomy". Thereby, the operator can concentrate on nearby operations rather than directly navigating the vehicle. Technical advantages include hybrid technology with smart algorithm hardware for both manned and unmanned operation. This can be configured to suit a particular vehicle and is not limited to the type of vehicle. This technology is equally applicable to the control of torpedo-type vehicles as well as high-speed vehicles. It uses a self-tuning algorithm that learns from the vehicle's response and adapts itself. It can also be used as a simple autopilot for high-end stand-alone unmanned systems. This technology is utilized in survey, security, and military applications. The self-learning command and control system is modular and application-adapted, and thus, a suitably named navigation module (100) can be retrofitted to any vehicle regardless of type, size, shape, application, and means of propulsion. FIG. 24 is an image of a navigation module (100) during deployment. A rugged, vibration-resistant, water- and intrusion-resistant housing (101) houses rugged electronics.

Claims

**Claim 1** A navigation system (300) for a means of transportation (302) on land, in the air, on the sea, or underwater, wherein the means of transportation (302) comprises manual control and a plurality of means-of-transportation actuators, - A remote control workstation (RCW) (301) having a manual control mode (310) that initiates command and control options including initial and intervention commands and control by a human operator (305), a mission planning mode (330), and a tactical control mode (360), - A navigation module (100) comprising a command control unit (311) having a self-learning command control module (314), - A plurality of perception sensors (318) disposed on the means of transportation (302), characterized in that the navigation system (300) receives manual, electrical, wireless, and voice commands of the human operator (305) in the manual control mode (310) and the mission planning mode (330), converts the commands into a data set for training a navigation model having a navigation algorithm resident in the self-learning command control module (314), and the plurality of perception sensors (318) generate a data set for self-learning in real time in the manual control mode (310), the mission control mode (330), and the tactical control mode (360), the navigation system (300) that can navigate spontaneously not only when a communication network (390) exists but also when no communication network (390) exists. **Claim 2** The navigation module (100) further comprises - A control unit (110), - A feedback unit (120), - A control processor (130), - A self-learning processor (140), - A sensor system (150), - A power distribution unit (160) including filters and isolation, - A power supply management (170), and - A communication module (180). The plurality of perception sensors (318) are connected to the sensor system (150) either wired or wirelessly through the communication module (180), the sensor system (150) receives inputs from the perception sensors (318) and converts all the inputs into something suitable for a proportional-integral-derivative (PID) controller (378) for the vehicle actuator (323), and the self-learning processor (140) receives a training data set for a cognitive navigation algorithm model. The navigation system (300) according to claim 1.

3. The mission planning mode (330) is executed before the start of operation, and the data set from the plurality of perception sensors (318) is acquired and graphed, or illustrated, or processed, or used to generate a user interface and execute a plurality of jobs such as control / monitoring / exit / obstruction / explosion with the help of the system / vehicle / platform. The navigation system (300) according to claim 1.

4. The tactical control (360) includes a handshake with the mission planning (330) for decision-making in critical scenarios including obstacle avoidance and response guidance, determines a desired movement path or trajectory from the current position of the platform or the vehicle to a specified target position, maps the actual trajectory and the desired trajectory in the system, calculates the difference therebetween, and an offset is supplied to the system for learning to gradually better match the actual trajectory (382) and the desired trajectory (381). The navigation system (300) according to claim 1.

5. The mission planning mode (330) performs automatic piloting in tactical control with situation awareness and interactive collision avoidance, the vehicle guidance module (316) generates a path for the vehicle (302) towards completion of the assigned task of reaching a predetermined destination relative to the home position, the navigation system (300) obtains collision-related data from a pre-set of prior information of map data including tree, mountain, building information, and terrain information, and the perception sensor (318) continuously generates a data set for self-learning in real time regarding the assigned task while the vehicle guidance model is updating the data set in real time (332). The navigation system (300) according to claim 1.

6. The mission planning mode (330) performs automatic piloting in tactical control with situation awareness and reactive collision avoidance, the vehicle guidance module (316) generates a path for the vehicle (302) towards completion of the assigned task of reaching a predetermined destination relative to the home position, the course control module (317) generates commands for the vehicle actuators (323) regarding the assigned task, the navigation system (300) uses data from cameras, LiDAR, sonar, ADS-B in addition to an aeronautical information service providing real-time data, and based on this, response guidance (319) performs course control through an evaluation and correction module (320) for collision avoidance, and the perception sensor (318) continuously generates a data set for self-learning in real time regarding the assigned task while the vehicle guidance model is updating the data set in real time (332). The navigation system (300) according to claim 1.

7. The command control unit or CCU (311) comprises a self-learning command control unit (314), unmanned control (320), and an evaluation and correction platform (320), and the self-learning command control unit (314) Vehicle guidance (316), wherein the vehicle guidance system (316) has a route generation means, a travel plan algorithm, and a sensor fusion module, a stereoscopic sensor and a GPS sensor are used as position sensors, and the track for the operation of the vehicle is, as a first step, generated by using only information from a digital map, while object detection sensors such as the stereoscopic sensor, three laser scanners, and a radar sensor observe the environment of the vehicle, report the detected objects to the sensor fusion module, and dynamically update the planned vehicle track for the final vehicle operation to track corrections and complete navigation guidance, vehicle guidance (316), and Course control (317), wherein the course control system (317) derives a model for vehicle path tracking and course control by operating the throttle and the nose, and the pitch angle (88), roll angle (86), yaw angle (87), and rapid ascent speed are treated as inputs, course control (317), comprising the navigation system (300) according to claim 1.

8. The tactical control (360) that handshakes with the mission plan (330) performs, in real time, an asset plan (321) for the mission objective by means of an optimization algorithm for resource allocation and dynamic adaptation in response to fluctuations in the situation, the emergence of threats, or new mission requirements, and promotes communication and coordination among other entities involved in the mission to ensure synchronization, the navigation system (300) according to claim 1.

9. The evaluation and correction platform (320) continuously evaluates the objectives, the effects to be achieved, and the corrections required for execution, and implements correction measures and actions taken, the navigation system (300) according to claim 8.

10. The plurality of perception sensors (318) connected to the response guidance system (319) include an accelerometer, a gyroscope, a compass, a magnetic heading sensor, a barometric pressure sensor, a plurality of GNSSs, a vision sensor, an acoustic sensor, a laser rangefinder, Li-Dar, a sonar, a radar, an optical sensor, and a depth sensor, and the sensor data is fused using a plurality of sensor fusion algorithms, the navigation system (300) according to claim 1.

11. The navigation system (300) according to claim 1, wherein the conveying means (302) to which a known destination (80) with respect to the home position (81) is assigned generates a plurality of route predictions (371) having predetermined basic settings and limitations based on a cognitive navigation algorithm for a navigation model pre-trained from past real-time data sets.

12. When there is no communication network (390), the navigation module (100) executes navigation through a local reference frame in the form of a virtual cube (90) with situation-dependent X, Y, and Z dimensions generated by a deep learning grid algorithm, wherein the X, Y, and Z dimensions are determined according to the situation based on the determination that the home position (81), the unknown destination (82), and the obstacles (85) are sufficient within the cube (90), and the conveying means moves within the cube (90) and knows its relative position with respect to the home position (81) with the help of a plurality of electromechanical non-network-based sensors including a compass, a gyroscope, an accelerometer, an acoustic sensor, a camera, and other non-network devices and perception sensors (318). The navigation system (300) according to claim 1.

13. The navigation system (300) according to claim 1, wherein the navigation module (100) of the conveying means (302), which is a drone here, predicts a safety altitude (373) based on the type of task manually input, the weather situation, and the position information (372).

14. The navigation system (300) according to claim 12, wherein the virtual cube (90) is an electromagnetic field locally generated by the conveying means (302) or its surrounding tactical platform (302A).

15. The navigation system (300) according to claim 12, wherein the virtual cube (90) is an energy field locally generated by the conveying means (302) or its surrounding tactical platform (302A).

16. The navigation system (300) according to claim 12, wherein the virtual cube (90) comprises a precision three-dimensional grid that generates a plurality of nodes (91) by the intersection of X, Y, and / or Z coordinates.

17. The navigation system (300) according to claim 12, wherein when the search direction cannot be confirmed, the home position (81) is the center (375) of the cube (90), or when the search direction is confirmed, the home position (81) is a corner (374) of the cube (90).

18. The navigation system (300) according to claim 12, wherein when the conveying means (302) is a marine conveying means, the home position (81) is the upper edge portion (376) of the cube (90).

19. The navigation system (300) according to claim 12, wherein the conveyance means guidance system (316) of the conveyance means (302) obtains periodic feedback from an AI navigation model (377) based on a pre-trained cognitive navigation algorithm.

20. The navigation system (300) according to claim 12, wherein the navigation switches between network-based navigation (391) and grid-based navigation (392) with intermittent network availability in order to continuously acquire an accurate trajectory.

21. The navigation system (300) according to claim 1, comprising a communication inner layer computer program for switching navigation from one communication network (390) to another communication network (390).

22. The navigation system (300) according to claim 1, comprising a plurality of conveyance means libraries for selecting appropriate conveyance means control and conveyance means guidance corresponding to the navigation of conveyance means on land, in the air, at sea, or underwater.

23. The navigation system (300) according to claim 1, comprising a control and thread management layer computer program for synchronizing and controlling the clocks and timings of a plurality of threads / plural processes.

24. The navigation system (300) according to claim 1, comprising a sandbox layer for control and testing in an isolated test environment.

25. The navigation system (300) according to claim 1, wherein the navigation module (100) is disposed so as to be retrofittable to the transport means (302).

26. The navigation system (300) according to claim 1, wherein the data set is a "fusion" data set based on a sensor fusion algorithm.

Images