System and method for artificial intelligence-based target association

Abstract

An artificial intelligence-based target association system for associating a plurality of target tracks generated from multiple sensors according to the present invention comprises: a track collection module for collecting target track information generated from a plurality of sensors; an association determination module for inputting the target track information into a pre-trained artificial intelligence model to calculate the probability of identical targets between tracks; an association determination module for determining whether the target tracks are associated according to the probability of identical targets calculated by the artificial intelligence model; and a target management module for displaying and managing target information based on whether the target tracks are associated.

Description

Technology Field

[0001] The present invention relates to an artificial intelligence-based target association system and method, and more particularly to an artificial intelligence-based target association system and method for determining the same target between target tracks generated from multiple sensors. Background Technology

[0003] Modern defense systems and tactical command and control systems process vast amounts of target information acquired from multiple sensors in real time and provide situational awareness based on this. In particular, in battlefield environments, sensors with distinct characteristics, such as radar, electro-optical sensors, and acoustic sensors, are operated simultaneously, requiring technology to efficiently integrate and manage the target information collected from them.

[0004] In response to these requirements, target management systems, such as naval combat systems, perform the functions of detecting, tracking, fusing, and displaying targets based on target information received from multiple sensors. During this process, the same actual target frequently generates different tracks due to differences in sensor characteristics, discrepancies in detection timing, and error characteristics; consequently, it is essential to determine whether these tracks correspond to the same target and associate them as a single target.

[0005] Conventional target association technologies have primarily utilized methods to determine whether targets are identical by using thresholds set based on differences in physical characteristics such as distance, bearing, speed, and altitude between tracks, or by applying probabilistic data association techniques. In these methods, various parameters are set according to sensor detection performance, error models, environmental conditions, and operational policies, and target association is performed based on predefined rules or thresholds.

[0006] However, the target association method according to the conventional technology described above has the following problems.

[0007] First, there is a problem in that it is difficult to uniformly apply predefined thresholds or rules to all situations, even though optimal association parameters vary depending on sensor performance, deployment type, operating environment, and tactical situation. As a result, excessive mis-association or association failure may occur in specific environments.

[0008] Second, some systems perform target association and display by assuming a track generated by a specific sensor or a specific track as a reference track; however, the reliability of such reference tracks can degrade depending on sensor status or environmental changes, which leads to a problem where the accuracy of overall target management also decreases.

[0009] Third, in actual operation, operators frequently modify automatic target association results or manually determine whether a target is associated; however, there is a limitation in that the results of these operator judgments are not systematically reflected in the subsequent target association process or utilized as training data. The problem to be solved

[0011] Accordingly, the present invention aims to solve the aforementioned problems, and the objective of the present invention is to provide an artificial intelligence-based target association system and method that can be adapted to each vessel and operational environment by utilizing the results of target association judgments performed by an operator during actual operation in a multi-sensor environment as training data.

[0012] However, the problems that the present invention aims to solve are not limited to those described above, and other problems may exist. means of solving the problem

[0014] An artificial intelligence-based target association system for associating a plurality of target tracks generated from multiple sensors according to a first aspect of the present invention for achieving the above-mentioned purpose comprises: a track collection module for collecting target track information generated from a plurality of sensors; an association determination module for inputting the target track information into a pre-trained artificial intelligence model to calculate the probability of identical targeting between tracks; an association determination module for determining whether the target tracks are associated according to the probability of identical targeting calculated by the artificial intelligence model; and a target management module for displaying and managing target information based on whether the target tracks are associated.

[0015] In some embodiments of the present invention, the association determination module can calculate the likelihood of identical targets using the artificial intelligence model that takes as input at least one of position, velocity, bearing, and time information between the target tracks.

[0016] In some embodiments of the present invention, the association determination module can generate a plurality of target track pairs with association potential from the target track information and calculate the same target probability for each target track pair using the artificial intelligence model.

[0017] Some embodiments of the present invention may further include a track normalization module that performs at least one of coordinate system transformation, time synchronization correction, and sensor alignment error correction on target track information collected through the track collection module to convert it to fit the input format of the artificial intelligence model.

[0018] In some embodiments of the present invention, the association determination module may classify a target track as an uncertain track if the same target probability calculated by the artificial intelligence model is less than a predetermined standard.

[0019] Some embodiments of the present invention may further include an operator interface module that receives target association or unassociation input from an operator for a target track whose association status has not been determined by the artificial intelligence model.

[0020] Some embodiments of the present invention may further include a learning data management module that manages the target association results and the operator's judgment results as learning data to be used for learning the artificial intelligence model.

[0021] Some embodiments of the present invention may further include a model training and verification module that retrains and verifies the artificial intelligence model based on training data managed by the training data management module.

[0022] In some embodiments of the present invention, the plurality of sensors may include at least one of a radar, an electro-optical sensor, an infrared sensor (EO and IR sensor), and an electronic support means (ESM) sensor.

[0024] Meanwhile, an artificial intelligence-based target association method for associating a plurality of target tracks generated from multiple sensors according to a second aspect of the present invention comprises: a track collection step for collecting target track information generated from a plurality of sensors; an association determination step for inputting the target track information into a pre-trained artificial intelligence model to calculate the probability of identical targets between tracks; an association determination step for determining whether the target tracks are associated according to the probability of identical targets calculated by the artificial intelligence model; and a target management step for displaying and managing target information based on whether the target tracks are associated.

[0025] In some embodiments of the present invention, the association determination step can calculate the likelihood of identical targets using the artificial intelligence model that takes as input at least one of the position, velocity, bearing, and time information between the target tracks.

[0026] In some embodiments of the present invention, the association determination step may generate a plurality of target track pairs with association potential from the target track information and calculate the same target probability for each target track pair.

[0027] Some embodiments of the present invention may further include a track normalization step after the track collection step, wherein at least one of coordinate system transformation, time synchronization correction, and sensor alignment error correction is performed on the collected target track information to convert it to fit the input format of the artificial intelligence model.

[0028] In some embodiments of the present invention, the association determination step may classify the corresponding target track as an uncertain track if the same target probability is below a predetermined criterion.

[0029] Some embodiments of the present invention may further include an operator input receiving step, after the association determination step, of receiving target association or unassociation input from an operator for a target track whose association status has not been determined.

[0030] Some embodiments of the present invention may further include a learning data management step for managing the target association result and the operator's judgment result as learning data.

[0031] Some embodiments of the present invention may further include a model training and verification step of retraining and verifying the artificial intelligence model based on the training data.

[0032] A computer program according to another aspect of the present invention for solving the above-mentioned problem is combined with a computer, which is hardware, to execute the artificial intelligence-based target association system and method, and is stored in a computer-readable recording medium.

[0033] Other specific details of the present invention are included in the detailed description and drawings. Effects of the invention

[0035] According to one embodiment of the present invention, target association performance capable of adapting to changes in sensor configuration or operating environment can be provided by associating target tracks generated from multiple sensors based on an artificial intelligence model. In particular, by retraining the artificial intelligence model using the actual judgment results of operators accumulated during operation as training data, learning-based target association reflecting actual operational characteristics is possible.

[0036] Furthermore, unlike conventional rule-based or fixed-parameter-based association methods, the AI ​​model calculates the same target probability, thereby reducing the burden of manual adjustment and tuning of parameters for target association. Moreover, since the model can be adaptively improved to suit the sensor configuration and operational characteristics of each vessel, a customized target management system for each ship can be implemented.

[0037] In addition, by effectively analyzing the complex relationships between target tracks occurring in heterogeneous sensor environments, such as radar, electro-optical sensors, infrared sensors, and electronic support means sensors, target association accuracy in multi-sensor environments can be improved.

[0038] The effects of the present invention are not limited to those mentioned above, and other unmentioned effects will be clearly understood by a person skilled in the art from the description below. Brief explanation of the drawing

[0040] The drawings attached below are intended to aid in understanding the embodiments thereof and provide embodiments together with a detailed description. However, the technical features of the embodiments thereof are not limited to specific drawings, and the features disclosed in each drawing may be combined with one another to form new embodiments. FIG. 1 is a block diagram of an artificial intelligence-based target association system according to one embodiment of the present invention. FIG. 2 is a flowchart of an artificial intelligence-based target association method according to one embodiment of the present invention. Specific details for implementing the invention

[0041] The advantages and features of the present invention and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below but may be implemented in various different forms. These embodiments are provided merely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the present invention, and the present invention is defined only by the scope of the claims.

[0042] The terms used in this specification are for describing embodiments and are not intended to limit the invention. In this specification, the singular form includes the plural form unless specifically stated otherwise in the text. The terms "comprises" and / or "comprising" used in this specification do not exclude the presence or addition of one or more other components in addition to the components mentioned. Throughout the specification, the same reference numerals refer to the same components, and "and / or" includes each of the mentioned components and all combinations of one or more. Although terms such as "first," "second," etc., are used to describe various components, these components are not limited by these terms. These terms are used merely to distinguish one component from another. Therefore, the first component mentioned below may be the second component within the technical scope of the invention.

[0043] Unless otherwise defined, all terms used herein (including technical and scientific terms) may be used in a meaning commonly understood by those skilled in the art to which the present invention pertains. Additionally, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively unless explicitly and specifically defined otherwise.

[0044] The present invention relates to an artificial intelligence-based target association system (100) and a method.

[0045] One embodiment of the present invention aims to solve the problem of reduced target association accuracy depending on the type of sensor, sensor performance, and operating environment in the process of determining whether target tracks generated from a plurality of sensors, such as radar, EO / IR, and ESM, are the same target. Here, the radar may include radars having different characteristics, such as search radar, tracking radar, and fire control radar.

[0046] To this end, one embodiment of the present invention can provide an artificial intelligence-based target association system (100) and a method that adapt to multi-sensor environments such as radar, EO / IR, and ESM, and the operating conditions of each vessel by utilizing the target association judgment results performed by an operator during actual operation as training data and reflecting them in the training of an artificial intelligence model.

[0047] Hereinafter, an artificial intelligence-based target association system (100) according to an embodiment of the present invention will be described with reference to FIG. 1.

[0048] FIG. 1 is a block diagram of an artificial intelligence-based target association system (100) according to one embodiment of the present invention.

[0049] An artificial intelligence-based target association system (100) according to one embodiment of the present invention includes a track collection module (110), a track normalization module (120), an association judgment module (130), an association determination module (140), a target management module (150), and an operator interface module (160).

[0050] First, the track collection module (110) collects target track information generated from multiple sensors, such as radar, electro-optical sensors, infrared sensors (EO and IR sensors), and electronic support means (ESM) sensors. The track information received from each sensor may include location, speed, bearing, time information, and sensor identification information, and the track collection module (110) receives and sorts the tracks generated by each sensor into a format that allows for integrated management.

[0051] The track normalization module (120) performs at least one of coordinate system transformation, time synchronization correction, and sensor alignment error correction on target track information collected through the track collection module (110). Through this, track information generated from different sensor references is aligned to the same reference coordinate system and time axis, and the data structure is transformed to be suitable for the input format of an artificial intelligence model. Through this, the present invention can improve association accuracy by minimizing data inconsistency between heterogeneous sensors.

[0052] The association determination module (130) receives normalized target track information and calculates the probability of the same target between tracks using a pre-trained artificial intelligence model. At this time, the association determination module (130) generates track pairs with potential association from a plurality of target tracks and inputs at least one of position, velocity, bearing, and time information as a feature for each track pair into the artificial intelligence model. Based on the input features, the artificial intelligence model calculates the probability or probability that each track pair is the same target.

[0053] As an example of the above artificial intelligence model, a neural network-based classification model that receives feature vectors extracted from multiple pairs of target tracks and binary classifies whether they are the same target may be used. For example, a fully connected neural network with a multilayer perceptron (MLP) structure, a feature extraction-based model using a convolutional neural network (CNN), a time series learning model such as a recurrent neural network (RNN) or LSTM (Long Short-Term Memory), a Transformer-based model, or a combination thereof may be applied. At this time, the type of artificial intelligence model is not necessarily limited to the above examples, and it goes without saying that various model formats for calculating the likelihood of being the same target in the present invention, such as gradient methods and ensemble methods, may be applied.

[0054] The association determination module (140) makes a final determination of whether target tracks are associated based on the same target probability calculated by an artificial intelligence model. In one embodiment, the association determination module (140) associates the corresponding pair of tracks as the same target if the probability is above a predetermined standard (threshold), and classifies them as unassociated or uncertain tracks if the probability is below the threshold. If classified as an uncertain track, it may be configured to request a judgment from the operator without performing automatic association.

[0055] The operator interface module (160) receives target association or unassociation input for target tracks for which association status has not been determined by the artificial intelligence model or for which the operator determines that further verification is required. The operator can check the association status through the display screen and manually establish or unassociate the relationship, and such input is recorded by the system.

[0056] In one embodiment, the operator interface module (160) is implemented in the form of a graphical user interface to visually display target tracks based on a tactical situation map, and tracks associated with the same target can be displayed with the same symbol or connecting line. Uncertain tracks can be distinguished and displayed with a separate color or warning mark. The operator can select a specific pair of tracks via mouse selection, touch input, or shortcut commands to input association or unassociation commands, and the operator interface module (160) can support judgment by displaying association history, same target probability values, and track information by sensor together.

[0057] The target management module (150) displays and manages target information based on the association results determined by the association determination module (140). As described above, tracks associated with the same target are displayed as a single integrated target, while uncertain tracks may be displayed separately. This allows the operator to intuitively recognize the target situation even in a multi-sensor environment.

[0058] Meanwhile, one embodiment of the present invention may further include a training data management module (170) and a model training and verification module (180) for training an artificial intelligence model.

[0059] The learning data management module (170) stores and manages target association results and operator judgment results as learning data. In particular, the operator's manual association or association de-inclusion input can be used as correct answer data for learning the artificial intelligence model and is accumulated and stored along with sensor configuration, operating environment, and time information.

[0060] For example, the training dataset may be structured to include a track pair identifier, a feature vector consisting of position, velocity, bearing, and time information for each track, sensor type and sensor identification information, operational environment information, and a ground truth label indicating whether the target is the same.

[0061] As such, one embodiment of the present invention can provide target association performance that continuously improves based on actual data collected during operation and adapts to changes in sensor configuration or operating environment.

[0062] The model training and verification module (180) retrains and verifies the artificial intelligence model based on the training data accumulated by the training data management module (170). Retraining can be performed in an offline environment or a separate verification environment, and the model that passes the verification procedure is reflected in the association determination module (130) and subsequently applied to target associations.

[0063] For example, if track characteristics change due to the addition of a new radar to a specific vessel or a change in the placement of existing sensors, or if false alarms or miscorrelation cases repeatedly occur at a specific location, the artificial intelligence model can be retrained by reflecting the accumulated operator judgment results. Through this, in one embodiment of the present invention, the criteria for calculating the same target probability can be adjusted to suit the changed sensor configuration or operating environment.

[0064] As such, the artificial intelligence-based target association system (100) according to the present invention can improve target association accuracy and reduce the operator's parameter adjustment burden through a closed-loop learning structure that combines automatic association and operator judgment in a multi-sensor environment, and can implement a customized target management system for each vessel.

[0065] Hereinafter, with reference to FIG. 2, a method performed by an artificial intelligence-based target association system (100) according to an embodiment of the present invention will be described.

[0066] FIG. 2 is a flowchart of an artificial intelligence-based target association method according to one embodiment of the present invention.

[0067] First, target track information generated from a plurality of sensors is collected (S111). At this time, the plurality of sensors may include radar, electro-optical sensors, infrared sensors (EO and IR sensors), and electronic support means (ESM) sensors, and track data including position, speed, bearing, time information and sensor identification information is received from each sensor.

[0068] Next, at least one of coordinate system transformation, time synchronization correction, and sensor alignment error correction is performed on the collected target track information to normalize data generated from different sensor references to the same reference (S112). This converts the data into a structure suitable for the input format of an artificial intelligence model and minimizes data inconsistency between heterogeneous sensors.

[0069] Once normalization is complete, multiple target track pairs with potential association are generated next (S113). At this time, track pairs that are candidates for association can be configured by considering temporal proximity, spatial distance, sensor combination information, etc.

[0070] Next, for each generated track pair, at least one of position, velocity, bearing, and time information is configured as a feature and input into a pre-trained artificial intelligence model, and the same target probability for each track pair is calculated (S114, S115).

[0071] Next, the association between target tracks is determined based on the calculated identical target probability (S116). At this time, if the identical target probability is greater than or equal to a predetermined threshold (S116-Y), the corresponding pair of tracks is automatically associated as the same target (S117). Conversely, if it is less than or equal to the threshold (S116-N), it is classified as unassociated or uncertain tracks (S118).

[0072] If automatic association is uncertain or not performed, a decision is requested from the operator, and input for target association or association release is received from the operator (S119~S120). After checking the association status and possibility information through the display screen, the operator can manually establish or release the association relationship, and the result of the decision is recorded by the system (S121).

[0073] Next, target information is displayed and managed based on the association decision results (S122). Tracks associated with the same target are displayed as integrated targets, and uncertain tracks are displayed separately to support the operator's situational awareness.

[0074] In addition, automatic association results and operator judgment results can be stored and managed as training data (S123). The training data may include track pair feature information, sensor information, operating environment information, and correct answer labels indicating whether the targets are the same.

[0075] Then, the artificial intelligence model is retrained and verified based on the accumulated training data (S124). Retraining can be performed in an offline environment or a separate verification environment, and the model that passes verification is applied again to the association judgment step. Accordingly, the method of the present invention continuously improves performance based on actual data collected during operation and can provide target association performance that adapts to changes in sensor configuration or operating environment.

[0076] Meanwhile, in the above description, steps S111 to S124 may be further divided into additional steps or combined into fewer steps according to an embodiment of the present invention. Also, some steps may be omitted as necessary, and the order between steps may be changed. Furthermore, even if other omitted details are omitted, the details described in FIG. 1 and FIG. 2 may be mutually applicable.

[0078] The artificial intelligence-based target association system (100) and method according to one embodiment of the present invention described above may be implemented as a program (or application) and stored on a medium to be executed in combination with a computer, which is hardware.

[0079] The aforementioned program may include code encoded in computer languages ​​such as C, C++, JAVA, Ruby, and machine language, which can be read by the computer's processor (CPU) through the computer's device interface, in order for the computer to read the program and execute the methods implemented in the program. Such code may include functional code related to functions that define the necessary functions for executing the methods, and may include control code related to execution procedures necessary for the computer's processor to execute the functions according to a predetermined procedure. Additionally, such code may further include memory reference code regarding where (address) additional information or media necessary for the computer's processor to execute the functions should be referenced in the computer's internal or external memory. In addition, if the processor of the computer needs to communicate with any other computer or server located remotely in order to execute the above functions, the code may further include communication-related code regarding how to communicate with any other computer or server located remotely using the communication module of the computer, and what information or media to transmit or receive during communication.

[0080] The above-mentioned storage medium refers to a medium that stores data semi-permanently and is readable by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specifically, examples of the above-mentioned storage medium include, but are not limited to, ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device. That is, the above-mentioned program may be stored on various recording media on various servers that the computer can access, or on various recording media on the user's computer. Additionally, the above-mentioned medium may be distributed across networked computer systems, and computer-readable code may be stored in a distributed manner.

[0081] The foregoing description of the present invention is for illustrative purposes only, and those skilled in the art will understand that other specific forms can be easily modified without altering the technical spirit or essential features of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single unit may be implemented in a distributed manner, and components described as distributed may likewise be implemented in a combined form.

[0082] The scope of the present invention is defined by the claims set forth below rather than by the detailed description above, and all modifications or variations derived from the meaning and scope of the claims and equivalent concepts thereof should be interpreted as being included within the scope of the present invention. Explanation of the symbols

[0084] 100: AI-based target association system 110: Track Collection Module 120: Track Normalization Module 130: Association Judgment Module 140: Association Decision Module 150: Target Management Module 160: Operator Interface Module 170: Training Data Management Module 180: Model Training and Validation Module

Claims

Claim 1 An artificial intelligence-based target association system for associating multiple target tracks generated from multiple sensors comprises: a track collection module for collecting target track information generated from multiple sensors; an association determination module for inputting the target track information into a pre-trained artificial intelligence model to calculate the probability of identical targeting between tracks; an association determination module for determining whether the target tracks are associated according to the probability of identical targeting calculated by the artificial intelligence model; and a target management module for displaying and managing target information based on the probability of identical targeting between the target tracks. The association determination module calculates the probability of identical targeting using the artificial intelligence model, which takes as input at least one of position, velocity, bearing, and time information between the target tracks. The association determination module further comprises a track normalization module that generates multiple pairs of target tracks with potential association from the target track information, calculates the probability of identical targeting for each pair of target tracks using the artificial intelligence model, and performs at least one of coordinate system transformation, time synchronization correction, and sensor alignment error correction on the target track information collected through the track collection module to convert it to fit the input format of the artificial intelligence model. If the probability of identical targeting calculated by the artificial intelligence model is less than a predetermined standard, the association determination module [represents] the corresponding target track It further includes an operator interface module that classifies a target track as an uncertain track and receives target association or disassociation input from an operator for a target track whose association status has not been determined by the artificial intelligence model, and further includes a training data management module that manages the target association results and the operator's judgment results as training data to be used for training the artificial intelligence model, and further includes a model training and verification module that retrains and verifies the artificial intelligence model based on the training data managed by the training data management module, and the plurality of sensors are radars,An artificial intelligence-based target association system comprising at least one of an electro-optical sensor, an infrared sensor (EO and IR sensor), and an electronic support means (ESM) sensor. Claim 2 delete Claim 3 delete Claim 4 delete Claim 5 delete Claim 6 delete Claim 7 delete Claim 8 delete Claim 9 delete Claim 10 An artificial intelligence-based target association method for associating multiple target tracks generated from multiple sensors, comprising: a track collection step for collecting target track information generated from multiple sensors; an association determination step for inputting the target track information into a pre-trained artificial intelligence model to calculate the probability of identical targets between tracks; and an association determination step for determining whether the target tracks are associated according to the probability of identical targets calculated by the artificial intelligence model. The AI-based target association method includes a target management step for displaying and managing target information based on whether there is an association between the target tracks, wherein the association determination step calculates the same target probability using an AI model that takes at least one of position, velocity, bearing, and time information between the target tracks as input, wherein the association determination step further includes a track normalization step for generating a plurality of target track pairs with potential association from the target track information, calculating the same target probability for each target track pair, and, after the track collection step, performing at least one of coordinate system transformation, time synchronization correction, and sensor alignment error correction on the collected target track information to convert it to fit the input format of the AI ​​model, wherein the association determination step classifies the corresponding target track as an uncertain track if the same target probability is below a predetermined standard, and, after the association determination step, further includes an operator input receiving step for receiving target association or unassociation input from an operator regarding target tracks whose association status has not been determined, wherein the method further includes a learning data management step for managing the target association results and the operator's judgment results as learning data, and further includes a model learning and verification step for retraining and verifying the AI ​​model based on the learning data. Claim 11 delete Claim 12 delete Claim 13 delete Claim 14 delete Claim 15 delete Claim 16 delete Claim 17 delete

Images