Battlefield situation awareness method based on artificial intelligence

Abstract

The invention discloses a battlefield situation awareness method based on artificial intelligence, and belongs to the field of artificial intelligence, and the awareness method specifically comprises the following steps: I, collecting camera data, and preprocessing the collected original image data; iI, positioning a target in the scene and identifying a target model; iII, detecting key points of a battlefield target to identify the direction of a main weapon; iV, converting the captured coordinates of the key points into a world coordinate system of a simulation environment; v, collecting each group of collected data and transmitting the collected data to an upper computer; according to the method, the multi-target feature recognition capability of the model in a low-resolution scene is enhanced, the model parameter quantity and the calculation requirement can be greatly reduced on the premise of ensuring the recognition performance, the potential threat direction of the target can be visually analyzed, a basis is provided for tactical planning, and the method is suitable for popularization and application. And the direction identification of the equipment target main weapon in a complex battlefield environment is realized.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a battlefield situation awareness method based on artificial intelligence. Background Art

[0002] With the rapid development of information and intelligence in modern warfare, battlefield situation awareness systems have become an important part of improving the combat capability of troops. Battlefield situation awareness systems can obtain battlefield information in real time through various sensors, identify, track and analyze enemy targets, and assist in command decision-making. However, the battlefield environment is complex and changeable. Existing battlefield situation awareness methods mostly rely on traditional signal processing and data fusion technologies, such as Kalman filtering and particle filtering. These methods have the following main disadvantages when processing massive and heterogeneous data in complex battlefield environments: First, traditional methods have limited data processing capabilities and it is difficult to analyze and integrate a large amount of information from multi-source sensors in real time, resulting in insufficient real-time and accuracy of situation awareness. In addition, traditional situation awareness systems lack self-learning and adaptability, making it difficult to self-optimize according to changes in the battlefield environment, reducing the flexibility and robustness of the system; therefore, it is particularly important to invent a battlefield situation awareness method based on artificial intelligence. Summary of the invention

[0003] The purpose of the present invention is to solve the defects in the prior art and to propose a battlefield situation awareness method based on artificial intelligence.

[0004] In order to achieve the above object, the present invention adopts the following technical solutions:

[0005] A battlefield situation awareness method based on artificial intelligence, the specific steps of the awareness method are as follows:

[0006] Ⅰ. Collect camera data and pre-process the collected raw image data;

[0007] II. Locate the target in the scene and identify the target model;

[0008] III. Detect key points of battlefield targets to identify their main weapon direction;

[0009] IV. Convert the captured key point coordinates into the world coordinate system of the simulation environment;

[0010] V. Summarize the collected data and transmit them to the host computer.

[0011] As a further solution of the present invention, the specific steps of preprocessing the collected raw image data in step I are as follows:

[0012] S1.1: Generate a set of N×N filter windows, set the initial position of the filter window on the original image, and move the filter window from the upper left corner of the image pixel by pixel. At the same time, process each pixel in the window through bilateral filtering to remove noise information in the image data.

[0013] S1.2: Calculate the grayscale histogram of the image, count the number of pixels at each grayscale level, calculate the new grayscale level of each pixel through the cumulative distribution function of the histogram, map the original image pixel value to the new grayscale value, and then use gamma correction to adjust the brightness and contrast of the image;

[0014] S1.3: Correct the color distortion and color cast of each group of image data, then calculate the image gradient and use non-maximum suppression technology to accurately locate the edge, then filter out strong edges and weak edges through double threshold detection, and connect the weak edges to form a complete edge, and then use the bilinear interpolation algorithm to uniformly scale the image data of different resolutions to the target size.

[0015] As a further solution of the present invention, the specific steps of identifying the target model in step II are as follows:

[0016] S2.1: Input the preprocessed image data into the YOLO target detection model, extract the feature map in each group of image data through the convolution layer of the YOLO target detection model, and predict the bounding box of the target in the image based on the extracted features, and return the coordinates, target category and confidence of the predicted bounding box;

[0017] S2.2: Sliding window operations are performed on the predicted target area through different convolutional layers in the ResNet model to extract features at different levels. The extracted local features are then processed by the maximum pooling layer to reduce the dimension of the feature map while retaining key feature information. Features at different levels are fused through the feature fusion network.

[0018] S2.3: After the fusion is completed, the inference model is pre-trained on a large-scale general dataset, and then the model weights are fine-tuned using the prepared target model dataset to adjust the model's recognition ability for each category. The fused features are then used to classify the target features through the fully connected layer and softmax layer of the inference model, and the target model category is inferred. The target model inferred by the model is then integrated with the location information of the target detection box and output;

[0019] S2.4: Collect information from each network model, and use deep model pruning technology to delete redundant convolution kernels or neurons, reduce the number of model parameters and calculations, and then use the weight quantization method to convert the model's floating-point calculations into low-precision fixed-point calculations, thereby reducing the memory space occupied by the model and accelerating the inference speed.

[0020] As a further solution of the present invention, the specific steps of main weapon direction identification in step III are as follows:

[0021] S3.1: Obtain images of battlefield targets from various angles, lighting conditions, and environmental backgrounds from the battlefield or simulation environment, annotate the bounding boxes of the target objects and the key points of the main weapons using manual or semi-automatic tools, and then divide the annotated data into training, validation, and test sets;

[0022] S3.2: Generate a confidence map for each group of pixels in each image in each data set to indicate the probability that each pixel is a key point of the main weapon. Then, use the PAFs detection algorithm to capture the structural constraints between different parts of the main weapon and obtain the spatial connection relationship between the target key points. Use data enhancement technology to expand the training data.

[0023] S3.3: Input the training set data into a two-branch convolutional neural network, and then use the convolutional neural network to predict the confidence map of the key points and the associated vector field between the key points, respectively, and measure the difference between the predicted confidence map and the actual key point position, as well as the difference between the vector field between the key points predicted by the model and the actual vector field;

[0024] S3.4: Based on the obtained difference values, update the parameters of each network layer of the model, use GPU or TPU to accelerate the training of the deep learning model during the training process, then evaluate the performance of the model through the validation set, and adjust the model data based on the evaluation results. After the preset number of training and validation, verify the accuracy of the model with the test set;

[0025] S3.5: Input the collected battlefield image into the trained convolutional neural network, and the convolutional neural network generates a confidence map of the key points of the target main weapon and the corresponding key point association vector field through forward propagation;

[0026] S3.6: Based on the confidence map, extract the highest confidence position of each key point, infer multiple key points belonging to the same main weapon through PAFs, and connect these key points in sequence to form the complete outline of the main weapon, and infer its direction based on the starting and ending points of the main weapon.

[0027] As a further solution of the present invention, the specific steps of converting the key point coordinates in step IV to the world coordinate system of the simulation environment are as follows:

[0028] S4.1: Based on the identified target information and the main weapon direction, the relationship between the pixel coordinates and the camera coordinate system is obtained through the camera internal parameters, and the pixel coordinates are converted to the camera coordinate system using the inverse matrix of the camera's internal parameter matrix. At the same time, during the conversion process, the actual distance of the target from the camera is determined by the depth sensor or binocular stereo vision;

[0029] S4.2: Use the rotation matrix to rotate the point in the camera coordinate system to the direction of the world coordinate system, then use the translation vector to adjust the displacement of the camera relative to the world coordinate system, obtain the position of the target in the world coordinate system through homogeneous coordinate transformation, and then use the radial distortion model to correct the barrel or pincushion distortion of the lens;

[0030] S4.3: Reconstruct the three-dimensional model of the target object in the simulation environment through the key point data of the world coordinate system, perform simulation deduction, and plan the optimal travel path for the automation system based on the position information of the key points.

[0031] As a further solution of the present invention, the specific steps of summarizing the collected groups of data and transmitting them to the host computer in step V are as follows:

[0032] S5.1: Remove redundant information between groups of adjacent picture frames, convert single-frame image data into frequency domain for compression through block processing and transform coding, and then detect the availability of network bandwidth in real time to determine whether the bit rate needs to be adjusted. When the bandwidth is sufficient, use a high bit rate, and when the bandwidth is limited, reduce the bit rate;

[0033] S5.2: Encapsulate the data of the recognition result into a standard format, and select the corresponding compression rate according to the data volume and real-time requirements. Then, use the compression algorithm to remove redundant information, reduce the data volume, and then generate a random symmetric key. Use the AES algorithm to encrypt the compressed data to generate ciphertext, and transmit the key through pre-shared keys or using a secure key exchange protocol. After the transmission is completed, use the recipient's RSA public key to encrypt the AES key;

[0034] S5.3: The sender and receiver perform a three-way handshake to establish a connection through the TCP protocol, and then disperse the large data packet to multiple paths for transmission. Each segment needs to be confirmed by the receiver after transmission to ensure that the data is reassembled in time after arrival. The receiver then decrypts the received data using the RSA public key.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] This battlefield situation awareness method based on artificial intelligence uses the YOLO target detection model to quickly locate targets in the scene, and then uses deep model pruning technology to delete redundant convolution kernels or neurons in each group of models used, reducing the number of parameters and calculations of the model. The weight quantization method is then used to convert the model's floating-point calculations into low-precision fixed-point calculations, reducing the memory space occupied by the model and accelerating the reasoning speed. The ResNet model and feature fusion network are used to extract and fuse each group of feature data. The inference model is then pre-trained on a large-scale general data set, and the model's weights are fine-tuned using the prepared target model data set to adjust the model's recognition ability for each category. The fused features are then used to classify the target's features through the fully connected layer and softmax layer of the inference model. The trained two-branch structure is then The convolutional neural network with this structure generates the confidence map of the key points of the target main weapon and the corresponding key point association vector field through forward propagation. According to the confidence map, the highest confidence position of each key point is extracted, and multiple key points belonging to the same main weapon are inferred through PAFs. These key points are connected in sequence to form a complete outline of the main weapon, and its direction is inferred according to the starting and ending points of the main weapon. The acquired groups of data are summarized and transmitted to the host computer, and the captured key point coordinates are converted into the world coordinate system of the simulation environment. The model's multi-target feature recognition capability in low-resolution scenarios is enhanced, and the model parameters and calculation requirements can be greatly reduced while ensuring the recognition performance. The potential threat direction of the target can be intuitively analyzed, providing a basis for tactical planning, and realizing the direction recognition of the main weapon of equipment targets in complex battlefield environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0038] Figure 1 This is a flowchart of a battlefield situation awareness method based on artificial intelligence proposed by the present invention. DETAILED DESCRIPTION

[0039] Reference Figure 1 , a battlefield situation awareness method based on artificial intelligence, the specific steps of the awareness method are as follows:

[0040] Collect camera data and preprocess the collected raw image data.

[0041] Specifically, a set of N×N filter windows is generated, and the initial position of the filter window on the original image is set. The filter window is moved pixel by pixel starting from the upper left corner of the image, and each pixel in the window is processed by bilateral filtering to remove noise information in the image data. The image grayscale histogram is calculated, the number of pixels at each grayscale level is counted, and the new grayscale level of each pixel is calculated by the cumulative distribution function of the histogram. The original image pixel value is mapped to the new grayscale value, and then the brightness and contrast of the image are adjusted by gamma correction, and the color distortion and color shift of each group of image data are corrected. Then, the image gradient is calculated and the non-maximum suppression technology is used to accurately locate the edge. Then, the strong edge and the weak edge are filtered out by double threshold detection, and the weak edges are connected to form a complete edge. Then, the image data of different resolutions are uniformly scaled to the target size by a bilinear interpolation algorithm.

[0042] Locate the objects in the scene and identify the object models.

[0043] Specifically, the preprocessed image data is input into the YOLO target detection model, and the feature map in each group of image data is extracted through the convolution layer of the YOLO target detection model. The bounding box of the target in the image is predicted based on the extracted features, and the coordinates, target category and confidence of the predicted bounding box are returned. The predicted target area is subjected to sliding window operation through different convolution layers in the ResNet model to extract features at different levels, and then the extracted local features are subjected to the maximum pooling processing of the pooling layer to reduce the dimension of the feature map while retaining the key feature information. The features at different levels are fused through the feature fusion network. After the fusion is completed, the reasoning is performed on a large-scale general data set. The model is pre-trained, and then the prepared target model data set is used to fine-tune the model's weights to adjust the model's recognition ability for each category. The fused features are then classified through the fully connected layer and softmax layer of the inference model to infer the target's model category. The target model inferred by the model and the position information of the target detection box are then integrated and output. The information of each network model is collected, and redundant convolution kernels or neurons are deleted through deep model pruning technology to reduce the number of model parameters and calculations. The weight quantization method is then used to convert the model's floating-point calculations into low-precision fixed-point calculations, reducing the memory space occupied by the model and accelerating the inference speed.

[0044] Detect key points of battlefield targets to identify their primary weapon directions.

[0045] Specifically, pictures of battlefield targets with various angles, lighting conditions and environmental backgrounds are obtained from the battlefield or simulation environment, and the bounding boxes of the target objects and the key points of the main weapon are annotated manually or semi-automatically. The annotated data are then divided into a training set, a validation set and a test set. A confidence map is generated for each group of pixels in each picture in each data set to indicate the probability that each pixel is a key point of the main weapon. The structural constraints between different parts of the main weapon are captured by the PAFs detection algorithm, and the spatial connection relationship between the target key points is obtained. The training data is expanded by data enhancement technology, and the training set data is input into a convolutional neural network with a two-branch structure. The convolutional neural network is then used to predict the confidence map of the key points and the associated vector field between the key points, respectively, and the difference between the predicted confidence map and the actual key point position and the model are measured. The difference between the predicted vector field between key points and the actual vector field is calculated, and the parameters of each network layer of the model are updated based on the obtained difference value. During the training process, GPU or TPU is used to accelerate the training of the deep learning model. After that, the performance of the model is evaluated through the validation set, and the model data is adjusted based on the evaluation results. After the preset number of training and validation times, the accuracy of the model is verified with the test set, and the collected battlefield images are input into the trained convolutional neural network. The convolutional neural network generates the confidence map of the key points of the target main weapon and the corresponding key point associated vector field through forward propagation. According to the confidence map, the highest confidence position of each key point is extracted, and multiple key points belonging to the same main weapon are inferred through PAFs. These key points are connected in sequence to form a complete outline of the main weapon, and its direction is inferred based on the starting and end points of the main weapon.

[0046] Convert the captured key point coordinates to the world coordinate system of the simulation environment.

[0047] Specifically, according to the identified target information and the direction of the main weapon, the relationship between the pixel coordinates and the camera coordinate system is obtained through the camera internal parameters, and the pixel coordinates are converted to the camera coordinate system using the inverse matrix of the camera's internal parameter matrix. During the conversion process, the actual distance of the target from the camera is determined by a depth sensor or binocular stereo vision, and the point in the camera coordinate system is rotated to the direction of the world coordinate system through a rotation matrix. The displacement of the camera relative to the world coordinate system is adjusted using a translation vector, and the position of the target in the world coordinate system is obtained through homogeneous coordinate transformation. The barrel or pincushion distortion of the lens is corrected using a radial distortion model, and the three-dimensional model of the target object is reconstructed in a simulation environment through the key point data of the world coordinate system, and simulation deduction is performed. Based on the position information of the key points, the optimal travel path is planned for the automation system.

[0048] Summarize the collected data and transmit them to the host computer.

[0049] Specifically, the redundant information between each group of adjacent picture frames is removed, and the single-frame image data is converted into the frequency domain for compression through block processing and transform coding. Then, the availability of the network bandwidth is detected in real time to determine whether the bit rate needs to be adjusted. When the bandwidth is sufficient, a high bit rate is used, and when the bandwidth is limited, the bit rate is reduced. The data of the recognition result is encapsulated into a standard format, and the corresponding compression rate is selected according to the data volume and real-time requirements. Then, the redundant information is removed by the compression algorithm to reduce the data volume, and then a random symmetric key is generated. The compressed data is encrypted using the AES algorithm to generate a ciphertext, and the key is transmitted by pre-sharing the key or using a secure key exchange protocol. After the transmission is completed, the AES key is encrypted using the receiver's RSA public key. The sender and the receiver perform a three-way handshake to establish a connection through the TCP protocol, and then the large data packet is dispersed to multiple paths for transmission, and each segment needs to be confirmed by the receiver after transmission to ensure that the data is reassembled in time after arrival. Then the receiver decrypts the received data through the RSA public key.

Claims

1. A battlefield situation awareness method based on artificial intelligence, characterized in that: The specific steps of the perception method are as follows: Ⅰ. Collect camera data and pre-process the collected raw image data; II. Locate the target in the scene and identify the target model; III. Detect key points of battlefield targets to identify their main weapon direction; IV. Convert the captured key point coordinates into the world coordinate system of the simulation environment; V. Summarize the collected data and transmit them to the host computer.

2. The battlefield situation awareness method based on artificial intelligence according to claim 1 is characterized in that: The specific steps of preprocessing the collected raw image data in step I are as follows: S1.1: Generate a set of N×N filter windows, set the initial position of the filter window on the original image, and move the filter window from the upper left corner of the image pixel by pixel. At the same time, process each pixel in the window through bilateral filtering to remove noise information in the image data. S1.2: Calculate the grayscale histogram of the image, count the number of pixels at each grayscale level, calculate the new grayscale level of each pixel through the cumulative distribution function of the histogram, map the original image pixel value to the new grayscale value, and then use gamma correction to adjust the brightness and contrast of the image; S1.3: Correct the color distortion and color cast of each group of image data, then calculate the image gradient and use non-maximum suppression technology to accurately locate the edge, then filter out strong edges and weak edges through double threshold detection, and connect the weak edges to form a complete edge, and then use the bilinear interpolation algorithm to uniformly scale the image data of different resolutions to the target size.

3. The battlefield situation awareness method based on artificial intelligence according to claim 2 is characterized in that: The specific steps for identifying the target model described in step II are as follows: S2.1: Input the preprocessed image data into the YOLO target detection model, extract the feature map in each group of image data through the convolution layer of the YOLO target detection model, and predict the bounding box of the target in the image based on the extracted features, and return the coordinates, target category and confidence of the predicted bounding box; S2.2: Sliding window operations are performed on the predicted target area through different convolutional layers in the ResNet model to extract features at different levels. The extracted local features are then processed by the maximum pooling layer to reduce the dimension of the feature map while retaining key feature information. Features at different levels are fused through the feature fusion network. S2.3: After the fusion is completed, the inference model is pre-trained on a large-scale general dataset, and then the model weights are fine-tuned using the prepared target model dataset to adjust the model's recognition ability for each category. The fused features are then used to classify the target features through the fully connected layer and softmax layer of the inference model, and the target model category is inferred. The target model inferred by the model is then integrated with the location information of the target detection box and output; S2.4: Collect information from each network model, and use deep model pruning technology to delete redundant convolution kernels or neurons, reduce the number of model parameters and calculations, and then use the weight quantization method to convert the model's floating-point calculations into low-precision fixed-point calculations, thereby reducing the memory space occupied by the model and accelerating the inference speed.

4. The battlefield situation awareness method based on artificial intelligence according to claim 3 is characterized in that: The specific steps of main weapon direction identification in step III are as follows: S3.1: Obtain images of battlefield targets from various angles, lighting conditions, and environmental backgrounds from the battlefield or simulation environment, annotate the bounding boxes of the target objects and the key points of the main weapons using manual or semi-automatic tools, and then divide the annotated data into training, validation, and test sets; S3.2: Generate a confidence map for each group of pixels in each image in each data set to indicate the probability that each pixel is a key point of the main weapon. Then, use the PAFs detection algorithm to capture the structural constraints between different parts of the main weapon and obtain the spatial connection relationship between the target key points. Use data enhancement technology to expand the training data. S3.3: Input the training set data into a two-branch convolutional neural network, and then use the convolutional neural network to predict the confidence map of the key points and the associated vector field between the key points, respectively, and measure the difference between the predicted confidence map and the actual key point position, as well as the difference between the vector field between the key points predicted by the model and the actual vector field; S3.4: Based on the obtained difference values, update the parameters of each network layer of the model, use GPU or TPU to accelerate the training of the deep learning model during the training process, then evaluate the performance of the model through the validation set, and adjust the model data based on the evaluation results. After the preset number of training and validation, verify the accuracy of the model with the test set; S3.5: Input the collected battlefield image into the trained convolutional neural network, and the convolutional neural network generates a confidence map of the key points of the target main weapon and the corresponding key point association vector field through forward propagation; S3.6: Based on the confidence map, extract the highest confidence position of each key point, infer multiple key points belonging to the same main weapon through PAFs, and connect these key points in sequence to form the complete outline of the main weapon, and infer its direction based on the starting and ending points of the main weapon.

5. The battlefield situation awareness method based on artificial intelligence according to claim 4 is characterized in that: The specific steps of converting the key point coordinates described in step IV to the world coordinate system of the simulation environment are as follows: S4.1: Based on the identified target information and the main weapon direction, the relationship between the pixel coordinates and the camera coordinate system is obtained through the camera internal parameters, and the pixel coordinates are converted to the camera coordinate system using the inverse matrix of the camera's internal parameter matrix. At the same time, during the conversion process, the actual distance of the target from the camera is determined by the depth sensor or binocular stereo vision; S4.2: Use the rotation matrix to rotate the point in the camera coordinate system to the direction of the world coordinate system, then use the translation vector to adjust the displacement of the camera relative to the world coordinate system, obtain the position of the target in the world coordinate system through homogeneous coordinate transformation, and then use the radial distortion model to correct the barrel or pincushion distortion of the lens; S4.3: Reconstruct the three-dimensional model of the target object in the simulation environment through the key point data of the world coordinate system, perform simulation deduction, and plan the optimal travel path for the automation system based on the position information of the key points.

6. The battlefield situation awareness method based on artificial intelligence according to claim 4 is characterized in that: The specific steps of summarizing the collected data sets and transmitting them to the host computer in step V are as follows: S5.1: Remove redundant information between groups of adjacent picture frames, convert single-frame image data into frequency domain for compression through block processing and transform coding, and then detect the availability of network bandwidth in real time to determine whether the bit rate needs to be adjusted. When the bandwidth is sufficient, use a high bit rate, and when the bandwidth is limited, reduce the bit rate; S5.2: Encapsulate the data of the recognition result into a standard format, and select the corresponding compression rate according to the data volume and real-time requirements. Then, use the compression algorithm to remove redundant information, reduce the data volume, and then generate a random symmetric key. Use the AES algorithm to encrypt the compressed data to generate ciphertext, and transmit the key through pre-shared keys or using a secure key exchange protocol. After the transmission is completed, use the recipient's RSA public key to encrypt the AES key; S5.3: The sender and receiver perform a three-way handshake to establish a connection through the TCP protocol, and then disperse the large data packet to multiple paths for transmission. Each segment needs to be confirmed by the receiver after transmission to ensure that the data is reassembled in time after arrival. The receiver then decrypts the received data using the RSA public key.

Images