Unmanned system intelligent combat platform based on large model and digital twinning

Abstract

The invention relates to an unmanned system intelligent combat platform based on a large model and digital twinning, belongs to the technical field of strike evaluation, and solves the problems of low intelligent level, poor combat effectiveness evaluation and low visualization degree in the unmanned system technology in the prior art. The large combat model is used for generating combat area environment information and target information and further generating each combat scheme so as to obtain an optimal combat scheme; the digital twin virtual battlefield is used for reconstructing a virtual combat environment; the controller is further used for reconstructing a virtual target in the virtual combat environment and performing fine adjustment on the local combat environment, then striking the virtual target in the virtual combat environment according to the received optimal striking scheme, and collecting an image set after striking of the virtual target; the large combat model is also used for obtaining a simulated damage evaluation result according to the received image set after the strike; and the command center is also used for performing actual strike based on the simulated damage evaluation result. And intelligent combat of the unmanned system is realized.

Description

Technical Field

[0001] The present invention relates to the field of strike assessment technology, and in particular to an unmanned system intelligent combat platform based on a large model and digital twins. Background Art

[0002] With the rapid advancement of global science and technology, the application scope and strategic value of unmanned systems have significantly increased. Unmanned equipment has become a key means of enhancing comprehensive combat effectiveness and significantly reducing casualties. Specifically, cutting-edge equipment such as drones and unmanned combat vehicles have demonstrated their indispensable role in modern combat operations. They can efficiently and accurately perform multiple tasks such as reconnaissance and surveillance, precision strikes, and material transportation, providing extremely reliable and comprehensive information support for command and decision-making systems. This development trend not only reflects the latest achievements in military technology but also foreshadows profound changes in the future form and style of warfare.

[0003] However, while existing intelligent unmanned equipment and technologies have made some progress, overall they still suffer from low intelligence levels, shallow operational applications, and limited training formats. On the one hand, existing unmanned equipment relies significantly on human intervention in actual combat applications and lacks the ability to autonomously complete the full chain of enemy target reconnaissance and identification, automatic generation of strike strategies, pre-strike effect estimation, and post-strike assessment. On the other hand, existing drone combat simulation formats, visualization platform functions, and presentations are limited, with limited support for functions such as spatiotemporal data processing and calculation. This hinders comprehensive and in-depth perception of both enemy and friendly data and the overall combat situation, and fails to meet the demands of modern combat strategies. Summary of the Invention

[0004] In view of the above analysis, the embodiments of the present invention aim to provide an unmanned system intelligent combat platform based on large models and digital twins to solve the problems of low intelligence level, poor combat effectiveness evaluation, and low visualization in existing unmanned system technologies.

[0005] The embodiment of the present invention provides an unmanned system intelligent combat platform based on a large model and digital twins, including a large combat model, a digital twin virtual battlefield, and a command center:

[0006] The combat model is used to generate combat area environment information based on the combat area; it is also used to generate target information based on the detection information, and to generate various strike plans based on the target information, and then to obtain the optimal strike plan based on the instructions of the command center; wherein the combat area and detection information are provided by the command center;

[0007] The digital twin virtual battlefield is used to reconstruct a virtual combat environment based on the received combat area environment information; it is also used to reconstruct virtual targets in the virtual combat environment based on the received target information and perform fine-tuning of the local combat environment; then, according to the received optimal strike plan, it reconstructs virtual weapons in the virtual combat environment and strikes virtual targets; and collects a set of images after the virtual targets are struck;

[0008] The large combat model is also used to obtain a simulated damage assessment result based on the received post-strike image set; the command center is also used to carry out an actual strike based on the simulated damage assessment result.

[0009] Furthermore, the command center monitors the combat area through a satellite monitoring system. If a suspicious target is found, a reconnaissance drone is assigned to perform real-time detection and original image acquisition, and the original image is recognized based on the built-in fine-tuned YOLOv8 target recognition model to obtain a detection image, and the target environment information is recognized based on the built-in sensor module; the detection image and target environment information are sent to the combat large model in sequence as detection information; wherein the detection image is an annotated original image, the annotation includes a bounding box, a type label and a recognition probability, and the target environment information includes the target position and local environment information.

[0010] Furthermore, the combat model integrates a fine-tuned large language model and multiple functional modules, wherein the functional modules include an intelligent agent model, a strike assessment knowledge base, a military equipment database, and a firepower planning algorithm library;

[0011] The large combat model receives input from the command center, identifies the intent of the input through the fine-tuned large language model, and calls the functional module that matches the intent to obtain the result for output.

[0012] Furthermore, if the fine-tuned large language model recognizes the input content of the command center as the intention to build a virtual battlefield, it extracts the combat area in the input content, calls the military equipment database to obtain the combat area environment information, and sends the combat area environment information to the digital twin virtual battlefield; wherein, the combat area environment information includes the military map and data of the combat area.

[0013] Furthermore, if the fine-tuned large language model recognizes the input content of the command center as target detection intention, it extracts the detection information received by the combat large model, calls the military equipment database to obtain target information, and the fine-tuned large language model returns the target information, and then starts the target monitoring mode; among them, the target information includes target type, target detailed information, and target environment information.

[0014] Furthermore, the target monitoring mode is that the fine-tuned large language model extracts the detection information received by the combat large model in sequence, obtains the corresponding target information, and sends it to the digital twin virtual battlefield in sequence.

[0015] Furthermore, the digital twin virtual battlefield includes a three-dimensional modeling module and an Unreal Engine rendering module;

[0016] The three-dimensional modeling module is used to save and create three-dimensional models of various weapon types;

[0017] The Unreal Engine rendering module is used to initialize environmental parameters in the Unreal Engine based on the received combat area environment information to render a virtual combat environment; it is also used to call the corresponding type of three-dimensional model in the three-dimensional modeling module in the Unreal Engine based on the received target information to render a virtual target and perform local environment fine-tuning, and then start the virtual target fine-tuning mode.

[0018] Furthermore, the virtual target fine-tuning mode is to control the movement of the virtual target and perform local environment fine-tuning according to the target environment information in the target information received sequentially by the digital twin virtual battlefield.

[0019] Furthermore, the Unreal Engine rendering module is also used to call the three-dimensional model of the corresponding available weapon in the three-dimensional modeling module according to the received optimal attack plan, render the virtual weapon in the Unreal Engine, load Chaos fragmentation effect parameters for the virtual target, and load Niagara particle effect parameters for the virtual weapon, and then control the virtual weapon to attack the virtual target based on the optimal attack plan.

[0020] Furthermore, the Unreal Engine rendering module is also used to call the UAV three-dimensional model in the three-dimensional modeling module, render a virtual UAV in the Unreal Engine, and control it to collect post-strike images of the virtual target in different directions to obtain a post-strike image set, and transmit it to the large combat model.

[0021] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:

[0022] The present invention provides an unmanned system intelligent combat platform based on a large model and digital twins, which includes a combat large model, a digital twin virtual battlefield and a command center. The command center inputs instructions to the combat large model, and the combat large model generates combat area environment information and target information based on the instructions of the command center, and then generates various strike plans. The digital twin virtual battlefield reconstructs the virtual combat environment, virtual targets and virtual weapons in the Unreal Engine according to the instructions of the combat large model, strikes the virtual targets in the virtual combat environment according to the optimal strike plan selected by the command center, and collects a set of images of the virtual targets after the strike. Then, the combat large model is used to obtain simulated damage assessment results, and finally the command center issues the actual strike order. It solves the problems of low autonomous decision-making ability of existing unmanned systems, single visualization form of combat scenes, and poor level of strike effect evaluation, improves combat assistance effect and terminal effect evaluation, and improves the intelligent decision-making and subjective initiative of combat unmanned systems; creates real and realistic combat dimension scenes based on Unreal Engine, creates virtual scenes and renders materials based on real geographic data, and realizes dynamic loading and control of three-dimensional models in the scene. At the same time, it displays the dynamic display of weapons and targets in three-dimensional scenes in real time, realizes the visualization of data statistics and data analysis, supports dynamic changes of weapon and target models in time and space, and supports pause and replay functions during strikes, so that users can analyze and schedule.

[0023] In the present invention, the above-mentioned technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of the present invention will be described in the following description, and some advantages will become apparent from the description or be learned through practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the contents particularly pointed out in the description and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings are only for the purpose of illustrating particular embodiments and are not to be considered limiting of the present invention. Like reference symbols denote like parts throughout the drawings.

[0025] Figure 1 A schematic diagram of the structure of an unmanned system intelligent combat platform based on a large model and digital twins provided in an embodiment of the present invention;

[0026] Figure 2 A schematic diagram of a detection image provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The preferred embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, and are not used to limit the scope of the present invention.

[0028] A specific embodiment of the present invention discloses an unmanned system intelligent combat platform based on a large model and digital twins, such as Figure 1 As shown, it includes a large combat model, a digital twin virtual battlefield, and a command center:

[0029] The combat model is used to generate combat area environment information based on the combat area; it is also used to generate target information based on the detection information, and to generate various strike plans based on the target information, and then to obtain the optimal strike plan based on the instructions of the command center; wherein the combat area and detection information are provided by the command center;

[0030] The digital twin virtual battlefield is used to reconstruct a virtual combat environment based on the received combat area environment information; it is also used to reconstruct virtual targets in the virtual combat environment based on the received target information and perform fine-tuning of the local combat environment; then, according to the received optimal strike plan, it reconstructs virtual weapons in the virtual combat environment and strikes virtual targets; and collects a set of images after the virtual targets are struck;

[0031] The large combat model is also used to obtain a simulated damage assessment result based on the received post-strike image set; the command center is also used to carry out an actual strike based on the simulated damage assessment result.

[0032] During implementation, the command center monitors the combat area through a satellite monitoring system. If a suspicious target is found, a reconnaissance drone is assigned to perform real-time detection and original image acquisition. The original image is recognized based on the built-in fine-tuned YOLOv8 target recognition model to obtain a detection image, and the target environment information is recognized based on the built-in sensor module. The detection image and target environment information are sequentially sent to the combat model as detection information. Figure 2 As shown, the detection image is an annotated original image. The annotations include a bounding box, a type label, and a recognition probability. The target environment information includes the target location and local environment information, including local terrain and weather. It should be noted that when determining the local environment information, the local size is determined according to specific needs.

[0033] Specifically, the sensor module includes a positioning module, a meteorological sensor, and a geographic sensor; the positioning module is a GPS, a lidar sensor, or a radar sensor.

[0034] Specifically, the reconnaissance drone uses the detection images and target environment information whose recognition probability exceeds a set probability threshold as detection information.

[0035] Specifically, the YOLOv8 target detection model uses the pre-trained weight loading method to perform model fine-tuning, including:

[0036] If the command center does not provide the type of suspicious target, the YOLOv8 target detection model with pre-trained weights will be used as the fine-tuned YOLOv8 target detection model;

[0037] If the command center gives the type of suspicious target, then based on the type of the current suspicious target, a set of images of that type is obtained as a training set, and the YOLOv8 target detection model with pre-trained weights is trained. The weight parameters are updated to obtain a fine-tuned YOLOv8 target detection model.

[0038] The pre-training weights are obtained by preliminarily training the YOLOv8 target detection model using pictures of each target type as a pre-training set.

[0039] Preferably, pictures of each weapon type in the military equipment database are used as pictures of each target type to form a pre-training set.

[0040] Preferably, a virtual target is rendered in the digital twin virtual battlefield according to the target type, and then multiple pictures under different angles and lighting are collected as a training set for the target type.

[0041] During implementation, the combat model integrates a fine-tuned large language model and multiple functional modules, wherein the functional modules include an intelligent agent model, a strike assessment knowledge base, a military equipment database, and a firepower planning algorithm library;

[0042] The large combat model receives input from the command center, identifies the intent of the input through the fine-tuned large language model, and calls the functional module that matches the intent to obtain the result for output.

[0043] Specifically, the Lang Chain open-source large model framework is initialized and fine-tuned using self-constructed fine-tuning data to form a fine-tuned large language model. The fine-tuned large language model incorporates natural language processing, image recognition, agent technology, RAG (Retrieval Augmented Generation), and vectorized retrieval technology. The self-constructed fine-tuning data includes target vulnerability data, ammunition depot data, explosion mechanics data, fuse data, and warhead data.

[0044] Specifically, Lora is used for fine-tuning.

[0045] Specifically, the fine-tuned large language model uses a "context window" to enable multi-round conversations. This window retains the previous conversation content (a certain number of words or tokens) in each round and feeds it into the model, allowing it to understand the current topic. Larger context windows can accommodate more information, allowing the model to maintain continuity over longer conversations. The model converts each user's sentence into a vector representation. Embeddings help the model understand the semantic relationships between words and use these relationships to generate appropriate responses. By accumulating and analyzing these embeddings, the model maintains the consistency of the conversation.

[0046] Retrieval-augmented generation (RAG) synergistically integrates the inherent knowledge of a large language model with the vast, dynamic repository of external databases. When updating knowledge, RAG technology simply uploads the knowledge to the strike assessment knowledge base and reconstructs the vectorized model, eliminating the need to build a large dataset and then fine-tune the large language model.

[0047] Specifically, the military equipment database includes information on weapons and equipment, military units and personnel, military maps and data, and strategic resource information. Weapon and equipment information includes detailed information on the technical parameters, performance characteristics, and usage of various types of military equipment, including firearms, artillery, tanks, armored vehicles, fighter jets, and bombers. Military unit and personnel information includes information on the military's organizational structure, troop deployment, staffing, and the personal information of key personnel. Military maps and data include various topographic maps, administrative maps, satellite remote sensing imagery, and relevant meteorological and hydrological data. Strategic resource information includes information on strategic resource reserves, such as the reserves, distribution, and extraction of energy sources like oil and natural gas, as well as information on nuclear materials. It is understood that the military equipment database contains comprehensive data on various types of military equipment, providing support for detailed information on targets and weapons.

[0048] Specifically, the strike assessment knowledge base includes the strike effects of various weapons and targets under various missile-target interaction conditions, and is capable of performing target integrity assessments, target durability assessments, and target functionality assessments. Target integrity assessments assess whether a target has been destroyed or damaged, and the severity of the damage, after a strike. Target durability assessments assess whether a target can still perform its basic functions, such as whether a building remains usable or equipment remains operational. Target functionality assessments assess whether a target can still perform its required functions, such as whether a military facility can continue to provide support to troops. It is understood that the strike assessment knowledge base serves as data for the fine-tuned large language model, supplementing its capabilities in strike assessment. When the fine-tuned large language model struggles to answer strike assessment questions based on its own capabilities, relevant content can be retrieved from the knowledge base.

[0049] Specifically, the fire planning algorithm library includes fire equipment information, tactical decision support information, shooting advantage analysis algorithm, fire planning cases, ammunition power field analysis algorithm, target vulnerability analysis algorithm, strike probability matrix calculation algorithm, strike benefit calculation algorithm, and image-based damage assessment algorithm; among them, fire equipment information includes technical parameters, performance characteristics, usage restrictions and combat effectiveness of various types of fire equipment, such as artillery, missile systems, fighters, bombers, etc.; tactical decision support is the tactical strategy of firepower application, including target selection, firepower allocation, shooting command, etc., auxiliary tools and methods for tactical decision-making, such as simulation exercises, data analysis, etc.; shooting advantage analysis is to analyze the shooting advantage of the firepower system to the enemy target, including shooting distance, angle, speed, etc., according to the shooting The results of the strike advantage analysis are used to adjust the deployment and combat posture of the firepower system; firepower planning cases include firepower planning cases in history or modern wars, including successful cases and failed cases; warhead power assessment algorithm, the warhead is the core part of the weapon, and its power directly determines the strike effect of the weapon. The algorithm library contains a variety of algorithms and models for evaluating the power of the warhead, which are based on parameters such as the type, mass, shape, and charge of the warhead, as well as the material, thickness, structure and other characteristics of the target; target vulnerability analysis algorithm, target vulnerability analysis is a key step in evaluating the strike degree of a target under a specific attack. The algorithm library contains vulnerability analysis algorithms for different types of targets. Such algorithms are based on the physical structure and functional characteristics of the target, as well as the performance parameters of the attack weapon.

[0050] During specific implementation, if the fine-tuned large language model recognizes the input content of the command center as the intention to build a virtual battlefield, it will extract the combat area in the input content, call the military equipment database to obtain the combat area environment information, and send the combat area environment information to the digital twin virtual battlefield; wherein, the combat area environment information includes the military map and data of the combat area.

[0051] Specifically, after sending the combat area environment information to the digital twin virtual battlefield, the fine-tuned large language model returns "virtual battlefield construction successful."

[0052] In specific implementation, if the fine-tuned large language model identifies the command center's input as a target detection intent, it extracts the detection information received by the combat large model, calls the military equipment database to obtain target information, and the fine-tuned large language model returns the target information, then activates the target monitoring mode. The target information includes target type, target detailed information, and target environment information.

[0053] Specifically, the target monitoring mode is that the fine-tuned large language model extracts the detection information received by the combat large model in sequence, and after obtaining the corresponding target information, it sends it to the digital twin virtual battlefield in sequence.

[0054] In specific implementation, if the fine-tuned large language model identifies the command center's input as an intention to generate a strike plan, it extracts the latest target information, calls the intelligent agent model and the firepower planning algorithm library to generate each strike plan, and the fine-tuned large language model returns each strike plan.

[0055] Specifically, the combat model generates various attack plans in the following ways:

[0056] The intelligent agent model calls the target vulnerability analysis algorithm based on the target information to obtain the strike tree of the target damage level;

[0057] Based on the strike tree of the target damage level, the intelligent model uses firepower equipment information, tactical decision support information, and firepower planning cases to obtain applicable weapons and corresponding ammunition information. It then uses the ammunition power field analysis algorithm to perform power field analysis on each applicable weapon and corresponding ammunition information to obtain available weapons and corresponding available ammunition that meet the target damage level.

[0058] Based on the available weapons and their corresponding available ammunition, the agent model uses the shooting advantage analysis algorithm to obtain the corresponding available weapon's position information, the explosive ammunition quantity algorithm to obtain the corresponding available ammunition quantity, and the aiming point optimization algorithm to obtain the target aiming point.

[0059] The agent obtains various strike plans based on the available weapons, the corresponding available ammunition, the position information of the available weapons, the amount of available ammunition and the target aiming point.

[0060] Specifically, if the command center gives the target damage level, the intelligent agent model generates different strike plans for the target damage level; if the command center does not give the target damage level, the intelligent agent model generates strike plans for different damage levels, such as light damage, moderate damage, and heavy damage.

[0061] Specifically, the number of attack plans is set according to demand and is 3 by default.

[0062] In specific implementation, if the fine-tuned large language model recognizes the input content of the command center as the intention of attack plan evaluation, it will extract each attack plan, call the attack probability matrix calculation algorithm and the attack benefit calculation algorithm to obtain the attack benefit and attack probability matrix of each attack plan, and the fine-tuned large language model will return the attack benefit and attack probability matrix of each attack plan.

[0063] Preferably, if the fine-tuned large language model cannot obtain the strike benefits and strike probability matrices of each strike plan, the strike evaluation knowledge base is called for evaluation, and the fine-tuned large language model returns the evaluation result.

[0064] It should be noted that strike payoff calculations primarily rely on target impact severity, operational effectiveness, and cost-effectiveness. Target impact severity reflects the effectiveness of a weapon system by quantifying the extent of the weapon system's impact on the target, such as the target's damage area, penetration depth, and degree of functional loss. Furthermore, the weapon's operational effectiveness under specific combat conditions is assessed, including accuracy, range, and sustained combat capability. The strike payoff is assessed by comprehensively considering the development, production, and operational costs of the weapon system. A mathematical model is then established to calculate strike payoff based on factors such as the weapon system's performance parameters, target characteristics, and the combat environment. Based on information such as impact severity, altitude, azimuth, elevation, and projectile velocity, a strike probability matrix is ​​calculated for targets within a 100-square-meter radius of the explosion center. The mathematical models and calculation methods involved are stored in the firepower planning algorithm library.

[0065] During specific implementation, if the fine-tuned large language model recognizes the input content of the command center as the intention of attack plan simulation, the optimal attack plan will be extracted and sent to the digital twin virtual battlefield.

[0066] In specific implementation, if the fine-tuned large language model recognizes the input content of the command center as the intention of simulated post-strike assessment, it extracts the set of images of the virtual target after the strike received by the combat large model, calls the image-based damage assessment algorithm to generate a simulated damage assessment result, and determines whether a second strike is needed based on the simulated damage assessment result and the expected damage level of the strike plan. The fine-tuned large language model returns the simulated damage assessment result and the expected damage level of the optimal strike plan, as well as whether a second strike is needed.

[0067] Preferably, if the fine-tuned large language model cannot give a simulated damage assessment result or a judgment result on whether a second strike is needed, the strike assessment knowledge base is called for retrieval to obtain an assessment result or a judgment result, and the fine-tuned large language model returns the assessment result or the judgment result.

[0068] It should be noted that the target damage assessment, including damage level assessment, number of fragments penetrated, and whether the various functions of the target are normal, can all be obtained through image-based damage assessment algorithms.

[0069] It is understandable that the large combat model trained with massive strike assessment data and algorithms serves as a "smart brain", providing the command center with a convenient interactive platform and intuitive visual effects. On this basis, it realizes data fusion and interaction between text and three-dimensional models, and creates an unmanned system combat platform that integrates intelligence, visualization and digitization.

[0070] During implementation, the digital twin virtual battlefield includes a three-dimensional modeling module and an Unreal Engine rendering module;

[0071] The three-dimensional modeling module is used to save and create three-dimensional models of various weapon types;

[0072] The Unreal Engine rendering module is used to initialize environmental parameters in the Unreal Engine based on the received combat area environment information to render a virtual combat environment; it is also used to call the corresponding type of three-dimensional model in the three-dimensional modeling module in the Unreal Engine based on the received target information to render a virtual target and perform local environment fine-tuning, and then start the virtual target fine-tuning mode.

[0073] Specifically, the virtual target fine-tuning mode controls the movement of the virtual target and performs local environment fine-tuning according to the target environment information in the target information received sequentially by the digital twin virtual battlefield.

[0074] Specifically, the local environment in the Unreal Engine is fine-tuned according to the local environment information in the target environment information; and the virtual target in the Unreal Engine is controlled to move according to the target position in the target environment information.

[0075] Specifically, the initialized environment parameters include lighting, materials, textures, terrain, vegetation, and buildings.

[0076] It should be noted that if the corresponding weapon type does not exist in the three-dimensional modeling module, modeling will be performed using 3dmax modeling software.

[0077] Furthermore, the Unreal Engine rendering module is also used to call the three-dimensional model of the corresponding available weapon in the three-dimensional modeling module according to the received optimal attack plan, render the virtual weapon in the Unreal Engine, load Chaos fragmentation effect parameters for the virtual target, and load Niagara particle effect parameters for the virtual weapon, and then control the virtual weapon to attack the virtual target based on the optimal attack plan.

[0078] Specifically, the Niagara particle effect parameters include particle initial attribute parameters, particle force parameters, and particle rendering parameters. Among them, the particle initial attributes include position, color, and size, and the particle forces include acceleration, resistance, and gravity; the Chaos shattering effect parameters include shattering level, shattering shape, and shattering force threshold.

[0079] Specifically, the Niagara particle effect parameters and Chaos crushing effect parameters are set according to the target information and available weapon information.

[0080] Specifically, the Unreal Engine converts the three-dimensional models of available weapons and targets into FBX files, creates the corresponding Actor base class, loads the static mesh models of such weapons or targets, instantiates the corresponding weapons and targets, and thus renders the three-dimensional models of virtual weapons and virtual targets.

[0081] It should be noted that the optimal strike plan is executed in the digital twin virtual battlefield, and the virtual weapons and virtual targets in the plan are observed in the order of strike, which presents the battlefield situation intuitively, flexibly and realistically, and reproduces the strike effect of the real battlefield; it can display the dynamic display of weapons and targets in the three-dimensional scene in real time, realize the visualization of data statistics and data analysis, support the dynamic changes of weapon and target models in time and space, and support pause and replay functions during the strike, so that users can analyze and schedule.

[0082] Furthermore, the Unreal Engine rendering module is also used to call the UAV three-dimensional model in the three-dimensional modeling module, render a virtual UAV in the Unreal Engine, and control it to collect post-strike images of the virtual target in different directions to obtain a post-strike image set, and transmit it to the large combat model.

[0083] Specifically, the post-strike image set includes post-strike images in at least four directions: east, south, west, and north.

[0084] Specifically, a Pawn base class of a drone is created in the Unreal Engine to implement drone control.

[0085] During implementation, the combat model, digital twin virtual battlefield, and command center communicate through real-time communication modules, specifically including:

[0086] In the Unreal Engine, the Blueprint HTTP Server plug-in is used to listen to the command center's requests, and a Listen blueprint node is created to listen to port 8081 with an IP of 127.0.0.1. In the Python-based combat model, the requests and Json packages are used to send HTTP requests in Json data format and receive HTTP responses in Json data format.

[0087] It is understandable that the combat model, the digital twin virtual battlefield and the command center use the HTTP protocol for three-party real-time communication to jointly simulate the execution process of real battlefield commands. The combat model provides macro analysis and prediction of the battlefield situation, and the digital twin virtual shooting range simulates the battlefield environment to verify the feasibility of the combat plan. The command center is responsible for receiving and analyzing information from the former two and issuing specific combat commands.

[0088] Compared with the existing technology, this embodiment provides an unmanned system intelligent combat platform based on large models and digital twins, including a combat large model, a digital twin virtual battlefield and a command center. The command center inputs instructions to the combat large model, and the combat large model generates combat area environment information and target information based on the instructions of the command center, and then generates various strike plans. The digital twin virtual battlefield reconstructs the virtual combat environment, virtual targets and virtual weapons in the Unreal Engine according to the instructions of the combat large model, strikes the virtual targets in the virtual combat environment according to the optimal strike plan selected by the command center, and collects a set of images of the virtual targets after the strike, and then obtains the simulated damage assessment results through the combat large model. Finally, the command center issues an actual The command to strike solves the problems of low autonomous decision-making ability of existing unmanned systems, single visual expression of combat scenes, and poor level of strike effect evaluation, improves combat assistance effect and terminal effect evaluation, and improves the intelligent decision-making and subjective initiative of combat unmanned systems; it creates real and realistic combat dimension scenes based on Unreal Engine, creates virtual scenes and renders materials based on real geographic data, and realizes dynamic loading and control of three-dimensional models in the scene. At the same time, it displays the dynamic display of weapons and targets in three-dimensional scenes in real time, realizes the visualization of data statistics and data analysis, supports dynamic changes of weapon and target models in time and space, and supports pause and replay functions during strikes, so that users can analyze and schedule.

[0089] Those skilled in the art will appreciate that all or part of the process steps of the above-described embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, such as a magnetic disk, an optical disk, a read-only memory, or a random access memory.

[0090] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by any technician familiar with this technical field within the technical scope disclosed by the present invention should be covered by the scope of protection of the present invention.

Claims

1. An unmanned system intelligent combat platform based on large models and digital twins, characterized by: Including large combat models, digital twin virtual battlefields and command centers: The combat model is used to generate combat area environment information based on the combat area; it is also used to generate target information based on the detection information, and to generate various strike plans based on the target information, and then to obtain the optimal strike plan based on the instructions of the command center; wherein the combat area and detection information are provided by the command center; The digital twin virtual battlefield is used to reconstruct a virtual combat environment based on the received combat area environment information; it is also used to reconstruct virtual targets in the virtual combat environment based on the received target information and perform fine-tuning of the local combat environment; then, according to the received optimal strike plan, it reconstructs virtual weapons in the virtual combat environment and strikes virtual targets; and collects a set of images after the virtual targets are struck; The large combat model is also used to obtain a simulated damage assessment result based on the received post-strike image set; the command center is also used to carry out an actual strike based on the simulated damage assessment result.

2. The unmanned system intelligent combat platform based on large models and digital twins according to claim 1 is characterized in that: The command center monitors the combat area through a satellite monitoring system. If a suspicious target is found, a reconnaissance drone is assigned to perform real-time detection and original image acquisition. The original image is recognized based on the built-in fine-tuned YOLOv8 target recognition model to obtain a detection image, and the target environment information is recognized based on the built-in sensor module. The detection image and target environment information are sent to the combat large model in sequence as detection information. The detection image is an annotated original image, and the annotations include a bounding box, a type label, and a recognition probability. The target environment information includes the target position and local environment information.

3. The unmanned system intelligent combat platform based on large models and digital twins according to claim 1 is characterized in that: The combat model integrates a fine-tuned large language model and multiple functional modules, wherein the functional modules include an intelligent agent model, a strike assessment knowledge base, a military equipment database, and a firepower planning algorithm library; The large combat model receives input from the command center, identifies the intent of the input through the fine-tuned large language model, and calls the functional module that matches the intent to obtain the result for output.

4. The unmanned system intelligent combat platform based on large models and digital twins according to claim 3 is characterized in that: If the fine-tuned large language model recognizes the command center's input content as an intention to build a virtual battlefield, it extracts the combat area in the input content, calls the military equipment database to obtain combat area environmental information, and sends the combat area environmental information to the digital twin virtual battlefield; wherein, the combat area environmental information includes the military map and data of the combat area.

5. The unmanned system intelligent combat platform based on large models and digital twins according to claim 4 is characterized in that: If the fine-tuned large language model recognizes the input content of the command center as target detection intention, it extracts the detection information received by the combat large model, calls the military equipment database to obtain target information, and the fine-tuned large language model returns the target information, and then starts the target monitoring mode; among them, the target information includes target type, target detailed information, and target environment information.

6. The unmanned system intelligent combat platform based on large models and digital twins according to claim 5 is characterized in that: The target monitoring mode is that the fine-tuned large language model extracts the detection information received by the combat large model in sequence, and after obtaining the corresponding target information, it sends it to the digital twin virtual battlefield in sequence.

7. The unmanned system intelligent combat platform based on large models and digital twins according to claim 6 is characterized in that: The digital twin virtual battlefield includes a three-dimensional modeling module and an Unreal Engine rendering module; The three-dimensional modeling module is used to save and create three-dimensional models of various weapon types; The Unreal Engine rendering module is used to initialize environmental parameters in the Unreal Engine based on the received combat area environment information to render a virtual combat environment; it is also used to call the corresponding type of three-dimensional model in the three-dimensional modeling module in the Unreal Engine based on the received target information to render a virtual target and perform local environment fine-tuning, and then start the virtual target fine-tuning mode.

8. The unmanned system intelligent combat platform based on large models and digital twins according to claim 7 is characterized in that: The virtual target fine-tuning mode controls the movement of the virtual target and performs local environment fine-tuning based on the target environment information in the target information received sequentially by the digital twin virtual battlefield.

9. The unmanned system intelligent combat platform based on large models and digital twins according to claim 8 is characterized in that: The Unreal Engine rendering module is further used to call the three-dimensional model of the corresponding available weapon in the three-dimensional modeling module according to the received optimal attack plan, render the virtual weapon in the Unreal Engine, load Chaos fragmentation effect parameters for the virtual target, and load Niagara particle effect parameters for the virtual weapon, and then control the virtual weapon to attack the virtual target based on the optimal attack plan.

10. The unmanned system intelligent combat platform based on large models and digital twins according to claim 9 is characterized in that: The Unreal Engine rendering module is also used to call the UAV three-dimensional model in the three-dimensional modeling module, render a virtual UAV in the Unreal Engine, and control it to collect post-strike images of the virtual target in different directions to obtain a post-strike image set, and transmit it to the combat large model.

Images