Battlefield target recognition method driven by knowledge and data

Abstract

The present invention proposes a battlefield target recognition method driven by both knowledge and data. The target data simulation and data set construction phase of the method: construct structured data; the knowledge graph construction phase: perform ontology modeling, extract targets and their correlation values, obtain target correlation data, and store them in a graph database; the target recognition phase based on correlation: distinguish the category of each target by calculating the overlap and correlation information between targets of different categories in the image, thereby improving the category accuracy of target recognition. Compared with the prior art, the present invention realizes the association organization and unified integration of data by constructing a battlefield target knowledge graph, creates a knowledge base for battlefield target recognition, and then completes battlefield target recognition based on a target recognition algorithm driven by a combination of data and knowledge, thereby improving the accuracy of target recognition. It has certain practical value in multiple application fields such as detection and early warning, intelligence reconnaissance, situational awareness, and precision guidance.

Description

Technical Field

[0001] The present invention relates to the field of situational awareness, and in particular to a battlefield target recognition method driven by both knowledge and data. Background Art

[0002] Battlefield target identification plays a very critical role in the combat process. It is the basis for commanders to make combat decisions, affecting the allocation of forces, timing and combat effectiveness. In the future, there will be many types of battlefield targets and they will be widely distributed. Complex and diverse reconnaissance sensors will be spread across land, sea, air, outer space and electromagnetic network space. The various types of intelligence data generated will grow "explosively". The real information of battlefield targets is hidden in them, and commanders will be trapped in the "battlefield fog". Clearing the "battlefield fog", accurately determining the enemy and friendly attributes of the target, distinguishing the target type, identifying the authenticity of the target, realizing intelligent target identification, and improving the accuracy of target identification are the key links to winning intelligent warfare.

[0003] Battlefield target recognition is a key technology for battlefield intelligence acquisition. In future unmanned warfare, improving battlefield intelligence acquisition capabilities can effectively improve the overall control of war. Different military powers are enhancing the research and development of related technologies. Current battlefield target recognition technology has problems such as weak target recognition capability, low recognition accuracy, and insufficient intelligent capabilities, making it difficult to meet the combat needs of a multi-domain integrated intelligent system with cognitive decision-making center warfare as the core. Summary of the invention

[0004] Purpose of the invention: In view of the problems existing in the prior art, the present invention provides a battlefield target recognition method driven by knowledge and data. The method first constructs a battlefield target knowledge graph to realize the association organization and unified integration of data, creates a knowledge base for battlefield target recognition, and then completes battlefield target recognition based on a target recognition algorithm driven by data and knowledge.

[0005] The technical solution adopted by the present invention is: a battlefield target recognition method driven by knowledge and data, which comprehensively utilizes target data simulation and data set construction technology, knowledge graph construction technology, and target recognition technology based on correlation. The specific steps are as follows:

[0006] S1: Target data simulation and data set construction phase: construct structured data by crawling open source data;

[0007] S2: Knowledge graph construction stage: Ontology modeling is performed based on Protege, and domain ontologies related to battlefield target recognition, including helicopters, drones, tanks, infantry fighting vehicles, civilian vehicles, ships, etc., are constructed. Targets and their correlation values ​​are selected to obtain target correlation data, which is stored in the graph database;

[0008] S3: Target recognition stage based on correlation: By calculating the overlap, correlation and other information between targets of different categories in an image, the category of each target is judged more carefully to improve the category accuracy of target recognition.

[0009] Among them, the target data simulation and data set construction stages in step 1 are as follows:

[0010] S1.1: Use the “Octopus Collector” to crawl data from tables such as Wikipedia pages to obtain structured data for building knowledge graphs.

[0011] S1.2: Manual adjustment and proofreading are performed for the misalignment or inability to crawl in 1.1.

[0012] Among them, the knowledge graph construction phase of step 2 is as follows:

[0013] S2.1: For structured data, such as general expert knowledge, directly perform entity extraction, relationship extraction, and attribute extraction.

[0014] S2.2: Extract entities, attributes and relationships from unstructured, semi-structured and structured data sources, integrate, disambiguate and eliminate conflicts between data from different sources, and adopt a joint entity and relationship extraction scheme based on LSTM-RNN scheme.

[0015] S2.3: In the annotation files of the dataset, multiple targets including tanks, infantry fighting vehicles, drones, civilian vehicles, and helicopters are recorded in detail in the label files of a picture, including the coordinate information, confidence level, and category of each target. Through these annotation files, the label information in all annotation files is extracted. In this way, the category of all detection boxes appearing in each picture can be obtained.

[0016] S2.4: Randomly extract a certain number of images and label files from the data set, traverse the label files corresponding to each image line by line, count the number of co-occurrences of 23 target categories in these randomly extracted data set files, and calculate the correlation between target categories based on the statistical results. Select two targets with a correlation greater than 0.1 and their correlation values ​​to obtain target correlation data, which are stored in the graph database.

[0017] The correlation degree is the ratio of the number of times two targets appear together in the same image to the total number of times the two targets appear, as shown in the formula:

[0018]

[0019] Among them, i, j are target categories, count m( ) function is the number of times the target appears in image m, and n is the total number of images. The target association weight is related to the number of times targets i and j appear alone and together.

[0020] Among them, the target recognition stage based on relevance in step 3 is as follows:

[0021] S3.1: Based on the YOLOv5 algorithm, a multi-candidate target discrimination module is added to first distinguish multiple candidate targets from single candidate targets.

[0022] S3.2: Using the single candidate target in the image as the associated object, call the target semantic feature knowledge base to calculate the association degree of multiple candidate targets, and take the target category corresponding to the detection box with the largest association score as the discrimination result of multiple candidate targets.

[0023] S3.3: Finally, the single candidate target and the identified multiple candidate results are output.

[0024] S3.4: After recognizing a certain number of images, the number of times each category of targets appears simultaneously on an image in all current recognition results will be counted. When the number of simultaneous appearances reaches a certain number, the knowledge graph will be adjusted to correct the association between the stored targets.

[0025] Among them, in step 3.2, the single candidate target category that has been screened out in the image is used as the target association object to calculate the association degree of multiple candidate targets. The specific steps are as follows:

[0026] S3.2.1: Search each multi-candidate target category in the knowledge graph to find the single target category associated with it, accumulate the association degree between the multi-candidate targets and each associated single candidate target, and obtain the weight size of each multi-candidate target detection box.

[0027] S3.2.2: By sorting the weights, the target category corresponding to the detection box with the largest weight is used as the discrimination result of multiple candidate targets.

[0028] S3.2.3: At this time, the multiple candidate targets whose target types have been determined can be added as single candidate targets to the discrimination of the next multiple candidate targets, and as feedback within the loop, play a role in iterative discrimination within the image range.

[0029] S3.2.4: Repeat the above process until the multiple candidate target groups are traversed. At this time, the multiple candidate targets with overlap greater than the threshold in the image have all generated corresponding target detection results.

[0030] Compared with the prior art, the present invention has the following beneficial effects:

[0031] (1) The present invention integrates multiple data sources together by constructing a battlefield target knowledge graph to form a unified and structured database, which not only includes information such as text descriptions and image features, but also covers information such as the relationship and attributes between targets, providing more comprehensive data support for target identification, realizing the associative organization and unified integration of data, and creating a knowledge base for battlefield target identification.

[0032] (2) The present invention performs target recognition based on a hybrid drive of data and knowledge. By integrating data and knowledge graphs, it can more accurately capture the essential characteristics of the target, thereby improving the accuracy and credibility of recognition. It provides new ideas and methods for solving various complex recognition problems and has certain practical value in multiple application fields such as detection and early warning, intelligence reconnaissance, situational awareness, and precision guidance. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A flow chart of battlefield target recognition method driven by knowledge and data;

[0034] Figure 2 Take screenshots of Wikipedia crawl results;

[0035] Figure 3 This is the image dataset after crawling;

[0036] Figure 4 It is the battlefield target ontology knowledge graph;

[0037] Figure 5 It is a joint extraction scheme of entities and relations based on LSTM-RNN;

[0038] Figure 6 It is the battlefield target knowledge graph;

[0039] Figure 7 The result graph of target correlation is selected;

[0040] Figure 8 Flowchart for multiple candidate target identification;

[0041] Fig. 9 This is a comparison chart of multiple candidate recognition effects. DETAILED DESCRIPTION

[0042] In order to make the technical means and creative features of the present invention easy to understand, the following is a brief description of the present invention in conjunction with the attached Figure 1-9 The technical scheme of the present invention is further illustrated with embodiments.

[0043] S1: Target data simulation and data set construction phase: construct structured data by crawling open source data;

[0044] S1.1: Use "Octopus Collector" to crawl Wikipedia web data, focusing on the data in tables such as Wikipedia web pages. Octopus Collector can browse web pages like a browser. Click on the element you want to crawl. The software will automatically generate the crawled data. Save the task to complete the crawl. First, open the software and log in successfully. Select "New" in the left menu bar, click "Custom Task", enter the website you want to crawl, click Save Settings, and click the desired element with the mouse. The software will automatically generate the crawled field. After the selection is completed, click Collect in the upper right corner. After the collection is completed, the collected data can be exported as a file in Jason or csv format.

[0045] S1.2: Due to the flexible layout of Wikipedia pages, some data may be misplaced or cannot be crawled, which requires manual adjustment and proofreading. Figure 2 As shown, the image dataset is formed as Figure 3 shown.

[0046] S2: Knowledge graph construction stage: Based on the Protege ontology modeling tool, ontology modeling was performed to build domain ontologies related to target recognition, including helicopters, drones, tanks, infantry fighting vehicles, civilian vehicles, ships, etc., forming Figure 4 The objects and their association values ​​are extracted to obtain the object association data, which are stored in the graph database.

[0047] S2.1: For structured data, such as general expert knowledge, formulate rules and patterns to directly extract entities, relationships, and attributes.

[0048] S2.2: Establish several knowledge integration rules from unstructured, semi-structured and structured data sources, perform knowledge integration under human participation, perform entity extraction, attribute extraction and relationship extraction, integrate, disambiguate and eliminate conflicts of data from different sources, and adopt an entity and relationship joint extraction scheme based on the LSTM-RNN long short-term memory neural network scheme. Figure 5 As shown in the figure, the first type is the embedding layer, which is used to represent words, part-of-speech tags, dependencies, and entity tags; the second type is the sequence layer, which is used to represent word sequences. This layer is a bidirectional LSTM-RNN; the third type is the dependency layer, which represents the relationship between two target words (entities) on the dependency tree. In the decoding process, the sequence layer is responsible for entity recognition, and the dependency layer is responsible for classifying the relationship between the two entities. After the extraction is completed, a knowledge graph is formed. Figure 6 shown.

[0049] S2.3: In the annotation files of the dataset, multiple targets including tanks, infantry fighting vehicles, drones, civilian vehicles, and helicopters are recorded in detail in the label files of a picture, including the coordinate information, confidence level, and category of each target. Through these annotation files, the label information in all annotation files is extracted. In this way, the category of all detection boxes appearing in each picture can be obtained.

[0050] S2.4: By randomly extracting a certain number of images and label files from the data set, traversing the label files corresponding to each image line by line, counting the number of co-occurrences of the 23 target categories in these randomly extracted data set files, and calculating the correlation between the target categories based on the statistical results, two targets with a correlation greater than 0.1 and their correlation values ​​are selected to obtain the target correlation data, which is stored in the graph database, such as Figure 7 shown.

[0051] The correlation is the ratio of the number of times two targets appear together in the same image to the total number of times the two targets appear, as shown in the formula:

[0052]

[0053] Among them, i and j are the two target categories to be sought, count m ( ) function counts the number of times the target category appears in the mth image, weight n () function is the target association weight, n is the total number of images; the target association weight weight is related to the number of times targets i, j appear alone and together.

[0054] S3: Target recognition stage based on correlation: By calculating the overlap, correlation and other information between targets of different categories in an image, the category of each target is judged more carefully to improve the category accuracy of target recognition.

[0055] S3.1: Based on the YOLOv5 algorithm, a multi-candidate target discrimination module is added. The theoretical framework of the image target recognition system and the calling relationship between each module are as follows: Figure 8 As shown. Specifically, through the multiple candidate target screening module, multiple candidate targets are distinguished from single candidate targets. Targets are divided into single candidate targets and multiple candidate targets according to the number of detection frames in the area. In the small range covered by the detection frame, it is assumed that the confidence threshold is set to conf_thres. When the detection frame with a confidence greater than the threshold is unique, the target corresponding to the detection frame is considered to be a single candidate target. On the contrary, if there are multiple detection frames with a confidence greater than the threshold, it is considered that there are multiple candidate targets in the small range. The small range here is determined by the intersection and union ratio between the detection frames.

[0056] S3.2: Using the single candidate target category that has been screened out in the image as the target association object, the association degree of multiple candidate targets is calculated.

[0057] S3.2.1: Search each multi-candidate target category in the knowledge graph to find the single target category associated with it, accumulate the association degree between the multi-candidate targets and each associated single candidate target, and obtain the weight size of each multi-candidate target detection box.

[0058] S3.2.2: By sorting the weights, the target category corresponding to the detection box with the largest weight is used as the discrimination result of multiple candidate targets.

[0059] S3.2.3: At this time, the multiple candidate targets whose target types have been determined can be added as single candidate targets to the discrimination of the next multiple candidate targets, and as feedback within the loop, play a role in iterative discrimination within the image range.

[0060] S3.2.4: Repeat the above process until the multiple candidate target groups are traversed. At this time, the multiple candidate targets with overlap greater than the threshold in the image have all generated corresponding target detection results.

[0061] S3.3: Finally, the single candidate target and the multiple candidate results after discrimination are output, including target category, coordinate position, confidence and other data information. After adding the multiple candidate target discrimination module based on target correlation, the overall target recognition accuracy is improved, especially for target categories such as helicopters, ships, containers, etc., which has the effect of significantly improving the recognition performance. Taking ships as an example, the recognition effect is as follows: Fig. 9 As shown, the left side of the picture shows the target recognition effect of Yolo V5, and the right side of the picture shows the target recognition effect of the algorithm of the present invention. The algorithm of the present invention adds a multi-candidate target discrimination module, which can better handle multiple similar or overlapping targets, and the target detection accuracy has been significantly improved.

[0062] S3.4: After recognizing a certain number of images, the number of times each category of targets appears simultaneously on an image in all current recognition results will be counted. When the number of simultaneous appearances reaches a certain number, the knowledge graph will be adjusted to correct the association between the stored targets.

[0063] The above is only an implementation mode of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A battlefield target recognition method driven by knowledge and data, characterized in that: The following steps are involved: S1: Target data simulation and data set construction phase: construct structured data by crawling open source data; S2: Knowledge graph construction stage: Ontology modeling is performed based on Protege to build domain ontology related to battlefield target recognition, and targets and correlation values ​​are extracted to obtain target correlation data, which is stored in the graph database; S3: Target recognition stage based on correlation: By calculating the overlap and correlation information between targets of different categories in an image, the category of each target is determined.

2. The knowledge and data dual-driven battlefield target recognition method according to claim 1 is characterized in that: The target data simulation and data set construction phase in step 1 includes the following steps: S1.1: Crawl data from tables on Wikipedia web pages to obtain structured data for building knowledge graphs; S1.2: Manual adjustment and proofreading are performed for misalignment or failure to crawl.

3. The knowledge and data dual-driven battlefield target recognition method according to claim 1 is characterized in that: Step 2: The knowledge graph construction phase includes the following steps: S2.1: For structured data, perform entity extraction, relationship extraction, and attribute extraction; S2.2: Extract entities, attributes and relationships from unstructured, semi-structured and structured data sources, integrate, disambiguate and eliminate conflicts between data from different sources, and adopt a joint entity and relationship extraction scheme based on LSTM-RNN scheme; S2.3: For the annotation files of the data set, extract the label information in all the annotation files, thereby obtaining the categories to which all the detection boxes appearing in each image belong; S2.4: By randomly extracting images and label files from the data set, the label files corresponding to each image are traversed line by line, the number of co-occurrences of 23 target categories in these randomly extracted data set files is counted, and the correlation between target categories is calculated based on the statistical results. Two targets with a correlation greater than 0.1 and their correlation values ​​are selected to obtain target correlation data, which are stored in the graph database.

4. The knowledge and data dual-driven battlefield target recognition method according to claim 3 is characterized in that: The LSTM-RNN scheme in step 2.2 is as follows: the first type is the embedding layer, which is used to represent words, part-of-speech tags, dependencies, and entity tags; the second type is the sequence layer, which is used to represent word sequences. This layer is a bidirectional LSTM-RNN. The third category is the dependency layer, which represents the relationship between two target words on the dependency tree; during the decoding process, the sequence layer is responsible for entity recognition, and the dependency layer is responsible for classifying the relationship between the two entities after the extraction is completed.

5. The knowledge and data dual-driven battlefield target recognition method according to claim 3 is characterized in that: The correlation formula of the target category in step 2.4 is: Among them, i and j are the two target categories to be sought, count m ( ) function is the number of times the target appears in image m, weight n () function is the target association weight, and n is the total number of images.

6. The knowledge and data dual-driven battlefield target recognition method according to claim 1 is characterized in that: In step 3, in the target recognition stage based on relevance, the specific steps include the following: S3.1: Based on the YOLOv5 algorithm, a multi-candidate target discrimination module is added to distinguish multiple candidate targets from single candidate targets; S3.2: Using a single candidate target in the image as the associated object, calling the target semantic feature knowledge base to calculate the association degree of multiple candidate targets, and taking the target category corresponding to the detection box with the largest association score as the discrimination result of the multiple candidate targets; S3.3: Output the single candidate target and the multiple candidate results after discrimination; S3.4: After the image is recognized, the number of times each category of targets appears simultaneously on an image in all current recognition results is counted. When the number of simultaneous appearances reaches the set number, the knowledge graph will be adjusted to correct the association between the stored targets.

7. The knowledge and data dual-driven battlefield target recognition method according to claim 6 is characterized in that: In step 3.1, the targets are divided into single candidate targets and multiple candidate targets according to the number of detection boxes in the area; within the range area, assuming a confidence threshold is set, when there is only one detection box with a confidence greater than the confidence threshold, the target corresponding to the detection box is considered to be a single candidate target. Otherwise, if there are multiple detection boxes with a confidence greater than the threshold, it is considered that there are multiple candidate targets in the range area.

8. The knowledge and data dual-driven battlefield target recognition method according to claim 6 is characterized in that: Step 3.2 specifically includes: S3.2.1: Search each multi-candidate target category in the knowledge graph to find the single target category associated with it, accumulate the association degree between the multi-candidate targets and each associated single candidate target, and obtain the weight value of each multi-candidate target detection box; S3.2.2: By sorting the weight values, the target category corresponding to the detection box with the largest weight is taken as the discrimination result of multiple candidate targets; S3.2.3: At this time, the multiple candidate targets whose target types have been determined are added as single candidate targets to the discrimination of the next multiple candidate targets, and as feedback within the loop, play the role of iterative discrimination within the image range; S3.2.4: Repeat the above process until the multiple candidate target groups are traversed. At this time, the multiple candidate targets with overlap greater than the threshold in the image have all generated corresponding target detection results.

Images