Short-distance air support system and method based on man-machine hybrid intelligent decision

Abstract

The invention relates to a short-distance air support system and method based on man-machine hybrid intelligent decision making. The system comprises a first man-machine interaction module used for extracting and fusing acquired sea-air multimode environment situation data to obtain environment situation information, feeding back the environment situation information to a command terminal, performing task planning according to the environment situation information and task requirements, and optimizing alternative schemes according to adjustment instructions to generate recommendation schemes; the command terminal is used for generating a decision scheme according to the recommendation scheme and expert experience; the second man-machine interaction module is used for adjusting a flight path and a mounting resource allocation scheme in real time according to the acquired environment data and the recommended scheme, and adjusting a mounting resource flight path in real time; and the operation terminal is used for completing short-distance air support of sea rescue according to expert experience, the flight path, the mounting resource distribution scheme and the mounting resource flight path. By adopting the system, the safety and success rate of rescue actions can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a close air support system and method based on human-machine hybrid intelligent decision-making. Background Art

[0002] In a maritime rescue scenario, when a maritime rescue unit (such as a rescue vessel or offshore platform) approaches a target but is unable to independently complete the rescue due to environmental risks, equipment limitations, or geographical obstacles, aviation rescue forces, at the request of the maritime unit, conduct an emergency air intervention operation to provide close air support to the trapped target. The maritime rescue unit and the target are already in close proximity, requiring the use of air power to overcome the rescue bottleneck.

[0003] In this context, when maritime rescue units approach a target but are unable to independently complete the rescue due to factors such as severe storms, complex sea conditions, equipment limitations, or geographical obstacles, traditional methods that rely on manual experience and simple communication equipment face significant challenges in information processing, rescue command and dispatch, and decision-making security. Manual processing of multi-source rescue data is inefficient, making it difficult to quickly integrate information to form effective decisions. Poor coordination between air and maritime rescue units can lead to wasted resources and delays in timely arrival at the scene. Human decision-making alone is prone to misjudgment in complex and changing scenarios, making it difficult to ensure safety and failing to meet the high demands of modern maritime rescue for timeliness, accuracy, and security. Summary of the Invention

[0004] Based on this, it is necessary to provide a close air support system and method based on human-machine hybrid intelligent decision-making to address the above technical problems.

[0005] A close air support system based on human-machine hybrid intelligent decision-making, the system comprising: Command terminal, first human-computer interaction module, operation terminal and second human-computer interaction module; The first human-computer interaction module is used to extract and fuse the acquired sea and air multi-modal environmental situation data to obtain environmental situation information, feed the environmental situation information back to the command terminal, receive the task requirements fed back by the command terminal, perform task planning based on the environmental situation information and the task requirements, obtain alternative plans, feed the alternative plans back to the command terminal, receive the plan adjustment instructions fed back by the command terminal, optimize the alternative plans based on the adjustment instructions, and generate a recommended plan; the recommended plan includes a rescue path planning plan and a task resource allocation plan; The command terminal is used to generate a decision plan based on the recommended plan and expert experience and send it to the operation terminal and the second human-computer interaction module, and send the decision plan to the first human-computer interaction module to optimize the task planning capability of the first human-computer interaction module; The second human-computer interaction module is used to adjust the flight path in real time according to the acquired environmental data and the rescue path planning plan, adjust the mount resource allocation plan according to the mission resource allocation plan, monitor the dynamic position of the rescue target and the flight trajectory of the mount resource to adjust the flight trajectory of the mount resource in real time; The operation terminal is used to complete close air support for maritime rescue based on expert experience, the flight path fed back by the second human-computer interaction module, the mount resource allocation plan and the mount resource flight trajectory.

[0006] A close air support method based on human-machine hybrid intelligent decision-making, the method comprising: The first human-computer interaction module extracts and fuses the acquired sea and air multi-modal environmental situation data to obtain environmental situation information, feeds the environmental situation information back to the command terminal, receives the task requirements fed back by the command terminal, performs task planning based on the environmental situation information and the task requirements, obtains alternative plans, feeds the alternative plans back to the command terminal, receives plan adjustment instructions fed back by the command terminal, optimizes the alternative plans based on the adjustment instructions, and generates a recommended plan; the recommended plan includes a rescue path planning plan and a task resource allocation plan; The command terminal generates a decision plan based on the recommended plan and expert experience and sends it to the operation terminal and the second human-computer interaction module, and sends the decision plan to the first human-computer interaction module to optimize the task planning capability of the first human-computer interaction module; The second human-computer interaction module adjusts the flight path in real time based on the acquired environmental data and the rescue path planning plan, adjusts the mount resource allocation plan based on the mission resource allocation plan, monitors the dynamic position of the rescue target and the flight trajectory of the mount resources, and adjusts the flight trajectory of the mount resources in real time; The operation terminal completes close air support for maritime rescue based on expert experience, the flight path fed back by the second human-computer interaction module, the mount resource allocation plan, and the mount resource flight trajectory.

[0007] The aforementioned close air support system and method based on human-machine hybrid intelligent decision-making, through the extraction and fusion of multimodal sea and air environmental situation data by the first human-machine interaction module, can rapidly integrate massive amounts of data such as the location of the distressed vessel, sea conditions, and weather conditions into environmental situation information. Alternative plans, based on mission requirements, are generated and dynamically optimized, addressing the inefficiency of manual data processing and delayed decision-making, making rescue decisions more timely and accurate. The command terminal combines recommended plans with expert experience to generate decision plans, which feed back into the task planning capabilities of the first human-machine interaction module. This enables continuous evolution of human-machine collaboration, improves the adaptability of rescue plans in multiple scenarios, and avoids the limitations of purely manual decision-making. The second human-machine interaction module adjusts flight paths and resource allocation in real time based on environmental data and rescue routes, monitors target dynamics and resource trajectories, and precisely controls them. This enhances the collaborative efficiency of air and sea rescue units, reduces resource waste, and ensures the rapid arrival of air rescue forces. The operation terminal combines expert experience with real-time feedback to complete rescue missions, leveraging the advantages of human decision-making in complex sea conditions while combining the precise control capabilities of machines to improve the safety and success rate of rescue operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 1 is a schematic structural diagram of a close air support system based on human-machine hybrid intelligent decision-making in one embodiment; Figure 2 A block diagram of a human-machine hybrid decision-making module in a human-in-the-loop embodiment; Figure 3 2 is a structural block diagram of a human-machine hybrid control module in an embodiment. DETAILED DESCRIPTION

[0009] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0010] In one embodiment, Figure 1 As shown, a close air support system based on human-machine hybrid intelligent decision-making is provided, including: Command terminal, first human-computer interaction module, operation terminal and second human-computer interaction module; The first human-computer interaction module is used to extract and fuse the acquired sea and air multi-modal environmental situation data to obtain environmental situation information, feed the environmental situation information back to the command terminal, receive the task requirements fed back by the command terminal, perform task planning based on the environmental situation information and task requirements, obtain alternative plans, feed the alternative plans back to the command terminal, receive the plan adjustment instructions fed back by the command terminal, optimize the alternative plans based on the adjustment instructions, and generate a recommended plan; the recommended plan includes a rescue path planning plan and a task resource allocation plan; The command terminal is used to generate a decision plan based on the recommended plan and expert experience and send it to the operation terminal and the second human-computer interaction module, and send the decision plan to the first human-computer interaction module to optimize the task planning capability of the first human-computer interaction module; The second human-computer interaction module is used to adjust the flight path in real time according to the acquired environmental data and the rescue path planning plan, adjust the mount resource allocation plan according to the mission resource allocation plan, monitor the dynamic position of the rescue target and the flight trajectory of the mount resources to adjust the flight trajectory of the mount resources in real time; The operation terminal is used to complete close air support for maritime rescue based on expert experience, the flight path fed back by the second human-computer interaction module, the mount resource allocation plan and the mount resource flight trajectory.

[0011] In this close air support system based on hybrid human-machine intelligent decision-making, the first human-machine interaction module extracts and integrates multimodal sea and air situational data, rapidly integrating massive amounts of data such as the location of distressed vessels, sea conditions, and weather conditions into environmental situation information. Alternative plans are generated and dynamically optimized based on mission requirements, addressing the inefficiency of manual data processing and delayed decision-making, making rescue decisions more timely and accurate. The command terminal combines recommended solutions with expert experience to generate decision plans, which in turn feeds back into the task planning capabilities of the first human-machine interaction module. This enables continuous evolution of human-machine collaboration, improves the adaptability of rescue plans across multiple scenarios, and avoids the limitations of purely manual decision-making. The second human-machine interaction module adjusts flight paths and resource allocation in real time based on environmental data and rescue routes, monitors target dynamics and resource trajectories, and precisely controls them. This enhances the collaborative efficiency of air and sea rescue units, reduces resource waste, and ensures the rapid arrival of air rescue forces. By integrating expert experience with real-time feedback, the operation terminal completes rescue missions, leveraging the advantages of human decision-making in complex sea conditions while combining them with the precise control capabilities of machines to improve the safety and success rate of rescue operations.

[0012] In one embodiment, the acquired environmental situation data is extracted and integrated to obtain environmental situation information, including: receiving sea and air multimodal environmental situation data, continuously processing the sea and air multimodal environmental situation data to obtain continuous sea and air multimodal environmental situation data; identifying the relationship between entities in the continuous sea and air multimodal environmental situation data, and using a graph neural network to construct an emergency rescue knowledge graph based on the relationship between entities; using a pre-trained first language model to extract key information from the continuous sea and air multimodal environmental situation data; the key information includes natural environment blind spots, human interference blind spots and rescue target priority rankings; and obtaining environmental situation information based on the emergency rescue knowledge graph and the key information.

[0013] In this embodiment, the primary task is to comprehensively model and integrate multi-source data about the search and rescue environment. By combining an emergency rescue knowledge graph constructed using a graph neural network (GNN) with the semantic understanding capabilities of a multimodal LLM, data from various sensors (such as radar signals, terrain imagery, and textual intelligence) can be extracted and integrated to comprehensively represent the situational context of both parties. Specifically, a Kalman filter is employed for time series prediction of dynamically changing target trajectories, ensuring the continuity and accuracy of situational data. Furthermore, the LLM's multimodal capabilities (such as support for integrated analysis of text, images, and video) enable the extraction of key information from complex data, such as the identification of interference blind spots (such as natural blind spots (such as reefs, vortex areas, and severe weather) and man-made interference blind spots (such as communication jamming blind spots)) and the prioritization of rescue targets. This information is converted into concise and intuitive situation reports using natural language generation (NLG) technology and graphically presented using visualization tools, helping commanders quickly understand the search and rescue environment.

[0014] To illustrate, consider a specific scenario: a sudden onslaught of severe convective weather in a certain sea area causes a small fishing boat to run aground and list, trapping five people on board and imminently sinking. A nearby rescue vessel is unable to approach due to strong winds and waves, necessitating air support. After the fishing boat sends a distress signal, the system collects multi-source data through radar, satellite communications, drone cameras, and other equipment. This data includes the boat's real-time location, list angle, and vital signs of the crew (transmitted via wearable devices). Environmental data includes wind speed and direction, cloud height, and wave patterns. Rescue resource data includes the fuel level, maximum payload, and takeoff and landing conditions of the standby helicopter. The system continuously processes this data, removes outliers, and then uses a graph neural network to analyze relationships between the data. For example, it associates "strong winds" with rules such as "helicopter launch altitude must be increased" and "rescue vessel must maintain a safe distance" to construct an emergency rescue knowledge graph. At the same time, the pre-trained large language model extracts key information from the data, such as "natural environment blind spots" (cloud cover causes limited drone vision) and "rescue target priority" (prioritize the rescue of seriously injured people), and ultimately generates complete environmental situation information.

[0015] In one embodiment, continuous processing of sea and air multimodal environmental situation data to obtain continuous sea and air multimodal environmental situation data includes: receiving sea and air multimodal environmental situation data; the sea and air multimodal environmental situation data includes rescue target location and dynamic information, rescuer resource status information, terrain information and meteorological condition information; using a Kalman filter to combine the terrain information and meteorological condition information to perform time series prediction on the rescue target trajectory to obtain a continuous trajectory, and obtaining continuous sea and air multimodal environmental situation data based on the continuous trajectory, the rescuer resource status information, terrain information and meteorological condition information that have been time-aligned.

[0016] In this embodiment, a Kalman filter is used to combine terrain and meteorological data to predict the trajectory of the rescue target, achieving data time alignment. This can accurately predict the drift path of the rescue target under the influence of ocean currents and guide the early deployment of helicopters.

[0017] In one embodiment, task planning is performed based on environmental situation information and task requirements to obtain alternative solutions, including: parsing the task requirements into structured constraints through a pre-trained second language model; generating an initial solution based on the structured constraints and sea and air multimodal environmental situation data using an optimization algorithm; the initial solution includes a rescue path planning solution and a task resource allocation solution; gamification modeling is performed on the rescuer and the environment to obtain a game theory model, the initial solution is used as the rescuer strategy, the environmental situation is predicted using a pre-trained spatiotemporal convolutional neural network, the environment strategy is simulated based on the predicted environmental situation using a pre-trained third language model, game deduction is performed based on the rescuer strategy and the environment strategy using the game theory model, Nash equilibrium analysis is performed on the deduction results, and the rescuer strategy with the highest benefit and lowest risk in the equilibrium state is selected; the structured constraints are updated based on the inference results of the third language model, the optimization algorithm is used to recalculate based on the updated structured constraints to generate a better solution, the above process is iterated until the iteration stop condition is met, and the current solution is output as the alternative solution.

[0018] In this embodiment, during the mission planning and decision-making recommendation phase, the LLM's mission generation capabilities automatically generate multiple alternative scenarios. These scenarios rationally allocate mission resources based on optimization algorithms (such as integer programming and dynamic programming). LLM's logical reasoning and causal analysis capabilities are then combined to recommend optimal rescue paths and resource allocation strategies to commanders. Furthermore, a game theory model is introduced, leveraging the LLM's reasoning capabilities to simulate the game dynamics between rescuers and the environment / risks. Nash equilibrium points are used to analyze vulnerabilities in environmental risk strategies and optimize rescue plans. Furthermore, deep learning techniques (such as spatiotemporal convolutional neural networks (ST-CNN)) are incorporated to dynamically model the search and rescue environment in real time, generating more accurate search and rescue and risk avoidance strategies.

[0019] Taking a fishing boat grounded scenario as an example, the system interprets mission requirements (e.g., "rescue all five people") into structured constraints. For example, the helicopter must arrive before the boat sinks; lifesaving equipment must be deployed before personnel are hoisted; and flight conflicts with other air rescue units must be avoided. After generating an initial plan using an optimization algorithm, the system uses a spatiotemporal convolutional neural network to predict future weather changes (e.g., increasing wind speed, decreasing visibility) and simulates environmental strategies (e.g., the impact of wind and waves on flight safety). Simultaneously, the initial plan serves as the rescuer's strategy, and a game theory model is used to perform deductions. For example, selecting a "low-altitude, fast release" strategy could result in a delivery error due to wind and waves; selecting a "high-altitude, slow release" strategy could result in exceeding fuel limits. The system uses Nash equilibrium analysis to select the plan with the optimal risk-reward ratio and iterates the optimization until the constraints are met.

[0020] In one embodiment, an optimization algorithm is used to generate an initial plan based on structured constraints and sea-air multimodal environmental situation data, including: when performing a rescue path planning task, a dynamic programming algorithm is used to generate a rescue path planning plan based on structured constraints and sea-air multimodal environmental situation data; when performing task resource allocation planning, an integer programming algorithm is used to generate a task resource allocation plan based on structured constraints and sea-air multimodal environmental situation data; and an initial plan is obtained based on the rescue path planning plan and the task resource allocation plan.

[0021] In this embodiment, a dynamic programming algorithm automatically plans the shortest and safest flight path based on structured constraints and environmental data, avoiding meteorological exclusion zones (such as cumulonimbus clouds) and geographical obstacles (reefs). For example, when a thunderstorm is predicted ahead, the system automatically calculates a detour to ensure the helicopter remains at a safe altitude. An integer programming algorithm considers the helicopter's payload capacity, the amount of life-saving equipment, and personnel requirements to develop a precise plan. For example, it allocates a corresponding number of life jackets and life rafts based on the number of trapped personnel, ensuring that medical equipment is prioritized for the seriously injured. Ultimately, the path planning and resource allocation plan are integrated into a complete initial plan.

[0022] In one embodiment, the system also includes: the first human-computer interaction module is also used to present the continuous sea and air multi-mode environmental situation data in a visual manner as a fused situation map, and generate a natural language briefing based on the recommended plan, key information and hierarchical analysis of local situation data, and send the fused situation map and the natural language briefing to the command terminal; the fused situation map includes the position distribution, resource distribution and terrain of both parties.

[0023] In this embodiment, at the data input and output level, the focus is on the structured processing and efficient transmission of multi-source data in the search and rescue environment. Input data includes sensor-collected information on the location and dynamics of search and rescue targets, the status of rescuer resources, terrain and weather conditions, etc.; output data includes fused situation maps, recommended mission planning solutions (such as rescue paths and resource allocation strategies), and natural language briefings for commanders' reference. In specific scenarios, the module's efficiency also benefits from LLM's ability to process long input contexts. It can support large-scale data input and automatically extract key information in multi-level command systems. For example, in multi-level command nodes, LLM generates semantically simplified text reports through hierarchical analysis of local situation data, reducing communication pressure and improving transmission efficiency.

[0024] In maritime rescue scenarios, the Large Language Model (LLM) constructs a dynamic threat assessment framework by integrating multi-source data, including radar tracks, optical recognition, and electromagnetic signals, in real time. First, it transforms raw sensor input into a unified semantic space, identifying key entities (such as unregistered speedboats and groups of people drowning) and their behavioral patterns (such as sudden speed changes and AIS shutdown). It then combines rescue rules with the maritime environment (such as wind and wave levels and oil spill spread) to quantify the target threat index and predict resource conflicts (such as airspace overlap and spectrum interference). Finally, it generates a hierarchical semantic report, providing immediate action recommendations (such as "launch warning flares") to rescue units, prioritizing response options (such as removing interfering vessels and using non-lethal deterrence) for command, and automatically optimizing rescue resource scheduling (such as helicopter routing and drone tasking). This process transforms traditional, decentralized manual analysis into a machine-enhanced cognitive closed loop, reducing communication overhead while improving decision-making timeliness by an order of magnitude, ensuring rapid deployment of protective intervention capabilities in complex maritime conditions.

[0025] Taking a fishing boat running aground as an example, the first human-computer interaction module transforms continuous environmental situation data into a 3D fused situation map, visually displaying information such as the fishing boat's location, helicopter route, and meteorological risk areas. Simultaneously, it generates natural language briefings, such as: "Current wind speeds are high; helicopters are advised to maintain high altitude; the fishing boat continues to drift southeast and is expected to approach a reef in 15 minutes; deploying lifesaving equipment is a priority." Commanders can quickly grasp the overall situation and make decisions by viewing the situation map and briefings.

[0026] like Figure 2The figure shows a block diagram of the human-in-the-loop hybrid decision-making module. The human-machine interaction fully utilizes the natural language processing capabilities of the LLM to achieve an efficient and user-friendly collaborative model. First, the LLM further optimizes recommended solutions by semantically parsing the commander's input mission requirements (such as objective priorities and resource constraints). Second, the module allows the commander to directly adjust the machine-generated solutions through natural language. This dynamic feedback mechanism helps the LLM learn the commander's logical preferences and optimize the quality of subsequent decision-making. The LLM's multimodal generation capabilities also enable the presentation of mission planning results in various formats (such as text and visual charts), significantly improving the interpretability and communication efficiency of mission results. The "human-in-the-loop hybrid decision-making module" deeply embeds the LLM's multimodal understanding and natural language generation capabilities into the decision-making process, achieving an organic integration of information processing, mission planning, and human-machine interaction. The module not only significantly improves decision-making efficiency in close air support missions but also effectively alleviates the cognitive stress on commanders in high-intensity search and rescue environments. In the future, as LLM's capabilities in complex problem reasoning, feature extraction, and task generation continue to improve, the module will demonstrate its potential in a wider range of emergency rescue application scenarios.

[0027] In one embodiment, real-time adjustment of the flight path based on the acquired environmental data and the rescue path planning scheme includes: acquiring a recommended scheme; the recommended scheme includes a rescue path planning scheme and a task resource allocation scheme; the rescue path planning scheme includes an initial rescue route; acquiring environmental data collected by onboard sensors, using a pre-set flight path planning algorithm, and generating a flight path based on the initial rescue route and environmental data; the environmental data includes rescue target situation data, environmental characteristic data, and aircraft status data.

[0028] like Figure 3The figure shows a block diagram of a human-in-the-loop (HIL) hybrid control module. During maritime emergency rescue missions, due to the limited range of rescue helicopters and the need for efficient rescue, rescue vehicles often need to operate deep into complex sea conditions. This choice of action inevitably exposes our rescue forces to challenging environmental challenges, requiring them to continuously contend with harsh sea conditions and complex hydrological conditions. In this high-risk, highly dynamic rescue environment, mission success depends not only on the rescuers' skilled operational skills and emergency preparedness, but also on rapid and accurate decision-making to address complex emergencies. To this end, this paper proposes a "HIL" HIL hybrid control module. By deeply integrating human control with machine-assisted decision-making, this module provides efficient coordinated control of support aircraft. The core of this "HIL" framework is that humans are the primary operators and decision-makers, while machine intelligence provides auxiliary functions, primarily in the command and guidance module and the weapon control module. This framework not only emphasizes real-time human participation during mission execution but also fully leverages the high-speed computing power of machines to ensure the achievement of mission objectives in a dynamic search and rescue environment. During the human-machine hybrid control process, the command and guidance module provides guidance for navigation, maneuvering and rescue operations to support aircraft. At the same time, real-time data feedback helps commanders adjust rescue strategies, and the rescue resource deployment control module allocates, deploys and provides subsequent guidance for rescue resources.

[0029] The command and guidance module uses onboard sensors to collect search and rescue environment data, such as the dynamic location of the search and rescue target, the coverage of severe sea conditions, and the characteristics of the sea terrain. It then uses an agent-based algorithm to generate an optimal flight path and maneuver strategy. This data input includes the aircraft's real-time position, target situation, and environmental characteristics. After algorithmic processing, the module generates dynamically updated path recommendations and risk avoidance plans, ultimately outputting the optimal flight route and rescue instructions.

[0030] However, when an aircraft enters an area with complex sea conditions or encounters an emergency, the complexity and randomness of the search and rescue environment may exceed the rapid processing capabilities of the intelligent agent algorithm. In such cases, the module plays an auxiliary role, providing necessary support to rescuers through the visualization of sensor data and warning information generated by the intelligent algorithm. For example, when radar detects an approaching severe storm, the module will display a safe area on the pilot's operating interface and recommend an evacuation direction. The final flight and rescue decision is made by the pilot based on real-time conditions and his own judgment, ensuring flexible response to complex emergencies. This collaborative approach effectively combines the computing power of machine intelligence with the decision-making advantages of humans, enabling pilots to make precise adjustments in a short period of time.

[0031] The rescue resource deployment control module is a critical component of mission execution. Based on command, the module allocates and prioritizes onboard rescue resources. For example, it allocates the type and quantity of life-saving equipment based on the urgency and priority of the target. Data inputs include the target's real-time location, the optimal rescue time window, and rescue resource performance parameters. The module's output is a precise rescue resource allocation plan and deployment recommendations. This process emphasizes the human role, with the pilot always holding the final decision on deployment, ensuring mission safety and controllability.

[0032] Once rescue resources are deployed, the resource guidance module takes over, dynamically adjusting the deployment trajectory to accommodate changes in the rescue target. For example, if the target moves or environmental conditions change, the module uses real-time data to adjust the deployment path to ensure accurate resource delivery. However, all pre-deployment steps (such as deployment height and equipment selection) must be confirmed and implemented by rescue personnel. This design not only meets the ethical requirements of maritime rescue but also reduces the risk of algorithmic misjudgment.

[0033] In one embodiment, a pre-set flight path planning algorithm is used to generate a flight path based on the initial rescue route and environmental data, including: using the A* algorithm to optimize the initial rescue path according to environmental feature data, rescue target situation data and static constraints to obtain a baseline route; the static constraints include the boundary of the no-fly zone and the long-term weather warning area; using the dynamic window method, according to the baseline route, real-time collected environmental obstacle data and aircraft status data, a set of obstacle avoidance candidate paths is generated within a local window; using the trained TinyLSTM neural network to predict dynamic threats that may appear in the short term and generate a risk heat map; inputting the obstacle avoidance candidate path set and the risk heat map into the fuzzy logic controller, using multi-objective weights to make weighted decisions, and outputting the optimal path as the flight path; the multi-objective weights include rescue timeliness weights, fuel efficiency weights and safety margin weights.

[0034] In this embodiment, after the helicopter takes off, the system uses the A* algorithm to globally optimize the initial rescue route, avoiding no-fly zones (such as military exercise areas) and long-term weather warning zones (typhoon paths). During flight, a dynamic window method monitors obstacles (such as floating ship debris) in real time and generates multiple candidate obstacle avoidance paths within a local window. Simultaneously, a TinyLSTM neural network predicts dynamic threats (such as sudden wind speed increases) in the short term and generates a risk heat map. A fuzzy logic controller selects the optimal path from the candidate paths, taking into account rescue timeliness, fuel efficiency, and safety margins. For example, if floating objects are detected ahead and fuel is sufficient, a safe detour is prioritized to ensure flight safety.

[0035] In one embodiment, the operation terminal is deployed on a control unit of an air rescue platform.

[0036] In addition to the core mission components, evaluation and feedback are also essential components of mission completion. After deploying rescue equipment, environmental obstruction may prevent visual evaluation for several minutes. This requires real-time monitoring by the equipment control system and environmental sensors, which must assess the effectiveness based on incomplete information. If additional rescue resources are needed, the sequence must be reorganized and a new round of rescue operations planned. The rescue operation concludes, and the rescue vehicle returns, marking the end of the close air support mission. Finally, an overall mission evaluation is conducted, documenting any unexpected situations encountered during actual mission execution and comparing them to the Markov decision process used in the simulation. The loss function is then calculated, backpropagated, and the agent parameters are optimized. Lessons learned are also shared with human commanders to build experience and enhance their resilience.

[0037] Aiming at the classic complex decision-making tasks in the field of close air support, the present invention proposes a human-machine hybrid intelligent decision-making model to solve the challenges faced by traditional command and control models in modern collaborative operations. Traditional command and control models lack the ability to quickly integrate information and coordinate efficient collaborative operations of multiple units. Simply relying on commanders to make decisions is difficult to adapt to the needs of modern collaborative operations. Although machine intelligence has advantages in information processing efficiency and accuracy, its comprehensive cognitive ability is relatively weak, and fully granting machine decision-making power may bring about ethical and uncertainty issues. The human-machine hybrid intelligent decision-making model has become the best feasible solution to the problem of close air support. It proposes a universal method for dividing the collaborative scope of "human" and "machine", studies the human-machine hybrid intelligent decision-making models at different stages, and elaborates on the human-machine collaboration mode, realizing the effective integration of human intelligence and machine intelligence.

[0038] In one embodiment, a close air support method based on human-machine hybrid intelligent decision-making is provided, comprising: The first human-computer interaction module extracts and integrates the acquired sea and air multi-modal environmental situation data to obtain environmental situation information, feeds the environmental situation information back to the command terminal, receives the mission requirements fed back by the command terminal, performs mission planning based on the environmental situation information and mission requirements, obtains alternative plans, feeds the alternative plans back to the command terminal, receives the plan adjustment instructions fed back by the command terminal, optimizes the alternative plans based on the adjustment instructions, and generates a recommended plan; the recommended plan includes a rescue path planning plan and a mission resource allocation plan; The command terminal generates a decision plan based on the recommended plan and expert experience and sends it to the operation terminal and the second human-computer interaction module, and sends the decision plan to the first human-computer interaction module to optimize the task planning capability of the first human-computer interaction module; The second human-computer interaction module adjusts the flight path in real time based on the acquired environmental data and the rescue path planning plan, adjusts the mount resource allocation plan based on the mission resource allocation plan, monitors the dynamic position of the rescue target and the flight trajectory of the mount resources, and adjusts the flight trajectory of the mount resources in real time; The operating terminal completes close air support for maritime rescue based on expert experience, the flight path fed back by the second human-computer interaction module, the mount resource allocation plan and the mount resource flight trajectory.

[0039] Each module in the aforementioned close air support system based on human-machine hybrid intelligent decision-making can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in the form of software in a memory in the computer device, so that the processor can call and execute the corresponding operations of each module.

[0040] For the specific limitations of the close air support method based on human-machine hybrid intelligent decision-making, please refer to the limitations of the close air support system based on human-machine hybrid intelligent decision-making above, which will not be repeated here.

[0041] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0042] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and such modifications and improvements are intended to fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A close air support system based on human-machine hybrid intelligent decision-making, characterized by: The system includes a command terminal, a first human-computer interaction module, an operation terminal and a second human-computer interaction module; The first human-computer interaction module is used to extract and fuse the acquired sea and air multi-modal environmental situation data to obtain environmental situation information, feed the environmental situation information back to the command terminal, receive the task requirements fed back by the command terminal, perform task planning based on the environmental situation information and the task requirements, obtain alternative plans, feed the alternative plans back to the command terminal, receive the plan adjustment instructions fed back by the command terminal, optimize the alternative plans according to the adjustment instructions, and generate a recommended plan; The recommended plan includes a rescue path planning plan and a task resource allocation plan; The command terminal is used to generate a decision plan based on the recommended plan and expert experience and send it to the operation terminal and the second human-computer interaction module, and send the decision plan to the first human-computer interaction module to optimize the task planning capability of the first human-computer interaction module; The second human-computer interaction module is used to adjust the flight path in real time according to the acquired environmental data and the rescue path planning plan, adjust the mount resource allocation plan according to the mission resource allocation plan, monitor the dynamic position of the rescue target and the flight trajectory of the mount resource to adjust the flight trajectory of the mount resource in real time; The operation terminal is used to complete close air support for maritime rescue based on expert experience, the flight path fed back by the second human-computer interaction module, the mount resource allocation plan and the mount resource flight trajectory.

2. The system according to claim 1, wherein: The extraction and fusion of the acquired environmental situation data to obtain environmental situation information includes: receiving sea and air multi-mode environmental situation data, and continuously processing the sea and air multi-mode environmental situation data to obtain continuous sea and air multi-mode environmental situation data; Identify the relationships between entities in continuous sea and air multimodal environmental situation data, and use graph neural networks to construct an emergency rescue knowledge graph based on the entities and the relationships between them; Extracting key information from the continuous sea-air multimodal environmental situation data using a pre-trained first language model; the key information includes natural environment blind spots, human interference blind spots, and rescue target priority ranking; Environmental situation information is obtained based on the emergency rescue knowledge graph and the key information.

3. The system according to claim 2, characterized in that The continuous processing of the sea and air multi-mode environmental situation data to obtain the continuous sea and air multi-mode environmental situation data includes: Receive sea and air multi-mode environmental situation data; the sea and air multi-mode environmental situation data includes rescue target location and dynamic information, rescuer resource status information, terrain information and weather condition information; A Kalman filter is used to perform time series prediction of the rescue target trajectory in combination with terrain information and meteorological conditions information to obtain a continuous trajectory. Based on the continuous trajectory, the rescuer's resource status information, terrain information and meteorological conditions information that have been time-aligned, continuous sea and air multi-mode environmental situation data are obtained.

4. The system according to claim 1, wherein: Mission planning is performed based on environmental situation information and mission requirements, and the alternative plans include: Parsing task requirements into structured constraints through a pre-trained second-largest language model; Using an optimization algorithm, an initial plan is generated based on the structured constraints and the sea-air multi-mode environmental situation data; the initial plan includes a rescue path planning plan and a task resource allocation plan; Gaming modeling is performed on the rescuer and the environment to obtain a game theory model. The initial plan is used as the rescuer strategy. The environmental situation is predicted using a pre-trained spatiotemporal convolutional neural network. The pre-trained third language model is used to simulate the environmental strategy based on the predicted environmental situation. The game theory model is used to perform game deduction based on the rescuer strategy and the environmental strategy. Nash equilibrium analysis is performed on the deduction results, and the rescuer strategy with the highest benefit and lowest risk in the equilibrium state is selected; The structural constraints are updated based on the inference results of the third language model. An optimization algorithm is used to recalculate the updated structural constraints to generate a better solution. The above process is iterated until the iteration stop condition is met, and the current solution is output as an alternative solution.

5. The system according to claim 4, characterized in that Using an optimization algorithm, generating an initial solution based on the structural constraints and the sea and air multi-mode environmental situation data includes: When performing the rescue path planning task, a dynamic programming algorithm is used to generate a rescue path planning scheme based on the structured constraints and the sea and air multi-mode environmental situation data; When performing task resource allocation planning, an integer programming algorithm is used to generate a task resource allocation plan based on the structured constraints and the sea and air multi-mode environmental situation data; The initial plan is obtained based on the rescue path planning plan and task resource allocation plan.

6. The system according to claim 1, wherein: The flight path control module includes: Obtaining a recommended solution; the recommended solution includes a rescue path planning solution and a task resource allocation solution; the rescue path planning solution includes an initial rescue route; Acquire environmental data collected by onboard sensors, and use a pre-set flight path planning algorithm to generate a flight path based on the initial rescue route and the environmental data; the environmental data includes rescue target situation data, environmental characteristic data, and aircraft status data.

7. The system according to claim 6, characterized in that Using a preset flight path planning algorithm to generate a flight path according to the initial rescue route and the environmental data includes: Using an A* algorithm, the initial rescue path is optimized based on the environmental characteristic data, the rescue target situation data, and static constraints to obtain a reference route; the static constraints include the no-fly zone boundary and the long-term weather warning area; A dynamic window method is used to generate a set of obstacle avoidance candidate paths within a local window based on the reference route, real-time collected environmental obstacle data, and aircraft status data; Use the trained TinyLSTM neural network to predict dynamic threats that may occur in the short term and generate a risk heat map; The obstacle avoidance candidate path set and the risk heat map are input into a fuzzy logic controller, and a weighted decision is made using multi-objective weights, and the optimal path is output as the flight path; the multi-objective weights include a rescue timeliness weight, a fuel efficiency weight, and a safety margin weight.

8. The system according to claim 1, wherein: The system further comprises: The first human-computer interaction module is also used to present the continuous sea and air multi-modal environmental situation data in a visual manner as a fused situation map, and generate a natural language briefing based on the recommended plan, key information and hierarchical analysis of local situation data, and send the fused situation map and natural language briefing to the command terminal; the fused situation map includes the position distribution, resource distribution and terrain of both parties.

9. The system according to claim 1, wherein: The operation terminal is deployed on the control unit of the air rescue platform.

10. A close air support method based on human-machine hybrid intelligent decision-making implemented in the system according to any one of claims 1 to 9, characterized in that: The method comprises: The first human-computer interaction module extracts and fuses the acquired sea and air multi-modal environmental situation data to obtain environmental situation information, feeds the environmental situation information back to the command terminal, receives the task requirements fed back by the command terminal, performs task planning based on the environmental situation information and the task requirements, obtains alternative plans, feeds the alternative plans back to the command terminal, receives plan adjustment instructions fed back by the command terminal, optimizes the alternative plans based on the adjustment instructions, and generates a recommended plan; the recommended plan includes a rescue path planning plan and a task resource allocation plan; The command terminal generates a decision plan based on the recommended plan and expert experience and sends it to the operation terminal and the second human-computer interaction module, and sends the decision plan to the first human-computer interaction module to optimize the task planning capability of the first human-computer interaction module; The second human-computer interaction module adjusts the flight path in real time based on the acquired environmental data and the rescue path planning plan, adjusts the mount resource allocation plan based on the mission resource allocation plan, monitors the dynamic position of the rescue target and the flight trajectory of the mount resources, and adjusts the flight trajectory of the mount resources in real time; The operation terminal completes close air support for maritime rescue based on expert experience, the flight path fed back by the second human-computer interaction module, the mount resource allocation plan, and the mount resource flight trajectory.

Images