Battlefield target intelligent identification plotting system and method based on deep learning

Abstract

The invention belongs to the technical field of deep learning, and discloses a battlefield target intelligent identification plotting system and method based on deep learning. Comprising the steps of collecting data and performing data cleaning to obtain battlefield enhanced image data; performing scene recognition on the battlefield enhanced image data, and outputting a battlefield type label; performing data scheduling on the battlefield enhanced image data to obtain scheduled image data; performing dynamic target intelligent identification on the scheduling image data in sequence, and outputting dynamic target moving trajectory data; carrying out dense target identification, and outputting high-density battlefield target detection data; executing auxiliary judgment and updating a preset database; acquiring battlefield space information corresponding to the battlefield enhanced image data, and constructing a battlefield simulation scene; accurately marking a battlefield target of the battlefield simulation scene to obtain battlefield target visual plotting data; updating the intelligent identification path and sending the path to a preset database for storage; therefore, accurate recognition and plotting of the battlefield target are realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of deep learning, more particularly, to a battlefield target intelligent identification and mapping system and method based on deep learning. BACKGROUND

[0002] With the rapid development of battlefield perception intelligence and command and control system, in the modern combat scene, analyzing multi-source battlefield data has become one of the core capabilities of information-based operations; especially in the complex tactical scene with dynamic target intensive and variable environment, how to quickly complete target detection and real-time labeling has become an important direction to improve the real-time response capability and battlefield situation understanding capability of combat, but it is still challenging to achieve high-precision battlefield target identification in the traditional target identification process.

[0003] In the prior art, battlefield target identification is mostly dependent on fixed rules or single model architecture, lacking detailed identification of battlefield scene types, and being difficult to adapt to complex battlefield environments with diverse terrain, diverse targets and dynamic changes, such as urban street fighting, jungle operations, high-altitude maneuvers and other scenes, resulting in significant decline in target identification efficiency and accuracy; most of the existing identification methods use static deep learning structure, which cannot dynamically adjust the identification direction and adapt the model structure according to the battlefield type label, resulting in low identification efficiency and uneven task load, etc., which cannot meet the identification needs under multi-task concurrency; most of the existing methods are limited to static calibration result output, lacking continuous modeling capability of target motion trajectory and coordinate mapping in spatial simulation environment, causing disconnection between target information and spatial layout; in addition, when the target in the battlefield image is blocked or overlapped due to excessive density, there is a lack of fine target area analysis, so the existing identification method often has detection errors or target loss, etc., which cannot meet the high-precision demand of tactical operation; at the same time, for the target types of new equipment in part of the battlefield that are not included in the database, the traditional identification method cannot give effective discrimination, and there is a risk of missing typical targets; and the existing target identification method lacks effective feedback mechanism, and cannot adaptively optimize the target identification structure according to the identification effect, resulting in long-term lag of performance, and the overall identification system is difficult to evolve and update.

[0004] In view of this, the present application provides a battlefield target intelligent identification and mapping system and method based on deep learning to solve the above problems. SUMMARY

[0005] In order to overcome the above-mentioned defects of the prior art, in order to achieve the above-mentioned purpose, the present application provides the following technical scheme: a battlefield target intelligent identification and mapping method based on deep learning, comprising:

[0006] S1. Collect battlefield image data and perform data cleaning to obtain battlefield enhanced image data;

[0007] S2. Multi-modal feature extraction is performed on the battlefield enhanced image data to obtain a set of scene feature vectors; scene recognition is performed on the battlefield enhanced image data based on the set of scene feature vectors, and a battlefield type label is output;

[0008] S3. Data scheduling is performed on the battlefield enhanced image data based on the battlefield type label, and scheduled image data is obtained and sent to an intelligent recognition path constructed based on a deep learning network to perform target recognition;

[0009] S4. Dynamic target intelligent recognition is performed on the scheduled image data, and dynamic target running track data is output; dense target recognition is performed on the scheduled image data, and high-density battlefield target detection data is output; auxiliary discrimination is performed on the scheduled image data, and a preset database is updated based on the discrimination result;

[0010] S5. Battlefield space information corresponding to the battlefield enhanced image data is obtained, and a battlefield simulation scene is constructed based on the battlefield space information; battlefield targets in the battlefield simulation scene are accurately labeled to obtain battlefield target visualization plot data;

[0011] S6. The intelligent recognition path is updated to an optimized processing path and sent to a preset database for storage.

[0012] Further, the way of performing scene recognition on the battlefield enhanced image data comprises:

[0013] extracting a discrimination feature vector and performing battlefield environment discrimination on the battlefield enhanced image data to output a battlefield type label.

[0014] Further, the way of performing data scheduling on the battlefield enhanced image data based on the battlefield type label comprises:

[0015] generating a scene identifier corresponding to the battlefield type based on the battlefield type label, and establishing a mapping relationship between the scene identifier and the battlefield type label; dividing an intelligent recognition path of a preset control terminal based on the battlefield type label; deploying a deep learning network model in each intelligent recognition path, and collecting historical battlefield enhanced image data corresponding to the battlefield type label of the intelligent recognition path to train the deep learning network model; activating the corresponding intelligent recognition path using the scene identifier, and inputting the battlefield enhanced image data as input data of the intelligent recognition path.

[0016] Further, the way of performing dynamic target intelligent recognition on the scheduled image data comprises:

[0017] transforming the scheduled image data into a battlefield image frame sequence; obtaining basic feature information of a battlefield target; performing preliminary screening on the battlefield image frame sequence to output a target candidate set; performing bounding on the battlefield target in the target candidate set, and outputting a target category probability;

[0018] The battlefield target frame sequence is boxed, and the position change information of the battlefield target is obtained; the future trajectory of the battlefield target is predicted based on the position change information, and the future frame prediction path of the battlefield target is obtained; the identification position change of the current battlefield target is corrected based on the future frame prediction path, and the dynamic target running trajectory data of the battlefield target is output.

[0019] Further, the way of performing dense target identification on the scheduling image data comprises:

[0020] Deep feature extraction is performed on each frame image in the battlefield image frame sequence to obtain frame image deep features; target distribution density of each frame image is calculated based on the frame image deep features; high-density target area is determined based on the target distribution density; candidate target initial detection points are set and feature perception is performed on the candidate target initial detection points to output feature perception data; the feature perception data is matched and discriminated with the basic feature information of the battlefield target, and the battlefield target is boxed;

[0021] The battlefield target that each detection box continuously appears in the adjacent frame is retained, and the continuously frame identification of the occluded target is performed; the position change of the occluded target is obtained; the complete contour of the occluded target is determined based on the position change and the basic feature information of the battlefield target, the battlefield target is boxed, and the target category probability is generated; the high-density battlefield target detection data is formed based on the identified battlefield image frame sequence.

[0022] Further, the way of performing auxiliary discrimination on the scheduling image data comprises:

[0023] The unknown target position is determined, and the pixel feature is extracted to generate an unknown target feature vector; the known target feature vector corresponding to the known battlefield target is obtained, and the feature distance between the unknown target feature vector and the known target feature vector is calculated; the confidence is calculated, the confidence matching threshold is set, when the confidence is greater than or equal to the confidence matching threshold, the unknown target is marked as a related target, otherwise it is marked as a typical unknown target; the feature information of the typical unknown target is extracted as a new entry of the preset database, and the feature information of the related target is combined with the existing entries in the preset database.

[0024] Further, the way of constructing the battlefield simulation scene comprises:

[0025] The pixel change feature is extracted, and the change trajectory of the area is constructed based on the pixel change feature; the change amplitude of the change trajectory is quantized, the change amplitude threshold is set, and the space reconstruction area is screened; the time stamp and the space reconstruction area are fused to output the battlefield environment change layer; the battlefield environment space model is constructed based on the battlefield space information and is time-sequentially matched with the battlefield environment change layer to generate the battlefield simulation scene.

[0026] Further, the way of accurately labeling the battlefield targets of the battlefield simulation scene comprises:

[0027] Obtaining the image frame index of each battlefield target, constructing a top-level battlefield target mapping layer, mapping the image coordinates corresponding to the latest timestamp and the initial timestamp of each battlefield target to the top-level battlefield target mapping layer based on the image frame index; constructing a battlefield target motion layer and dividing the number of layers to generate a battlefield target motion sub-layer; mapping the dynamic target running track data to the corresponding battlefield target motion sub-layer; constructing a battlefield dense area layer, labeling the dense degree of the battlefield dense area layer based on the high-density battlefield target detection data; generating a structured plotting layer, nesting the structured plotting layer and the battlefield simulation scene to generate battlefield target visual plotting data.

[0028] Further, the way of obtaining the optimized processing path comprises:

[0029] Real-time acquisition of feedback data, parameter optimization of the intelligent recognition path based on the feedback data, and obtaining of the optimized processing path.

[0030] A battlefield target intelligent identification and plotting system based on deep learning, which is used to realize a battlefield target intelligent identification and plotting method based on deep learning, comprises:

[0031] A data acquisition module is configured to acquire battlefield image data and perform data cleaning to obtain battlefield enhanced image data.

[0032] A scene recognition module is configured to perform multi-modal feature extraction on the battlefield enhanced image data to obtain a scene feature vector set, and perform scene recognition on the battlefield enhanced image data based on the scene feature vector set to output a battlefield type label.

[0033] A path construction module is configured to perform data scheduling on the battlefield enhanced image data based on the battlefield type label to obtain scheduling image data and send the scheduling image data to an intelligent recognition path constructed based on a deep learning network to execute a target identification process.

[0034] A fine identification module is configured to perform dynamic target intelligent identification on the scheduling image data to output dynamic target running track data, perform dense target identification on the scheduling image data to output high-density battlefield target detection data, and perform auxiliary discrimination on the scheduling image data and update a preset database based on the discrimination result.

[0035] A visual plotting module is configured to obtain battlefield space information corresponding to the battlefield enhanced image data, construct a battlefield simulation scene based on the battlefield space information, accurately label battlefield targets of the battlefield simulation scene, and obtain battlefield target visual plotting data.

[0036] A parameter optimization module is configured to update the intelligent identification path to an optimized processing path and send the optimized processing path to a preset database for storage.

[0037] The battlefield target intelligent identification and mapping system and method based on deep learning have the following technical effects and advantages:

[0038] The battlefield target intelligent identification and mapping system and method based on deep learning have the following technical effects and advantages: The battlefield target intelligent identification and mapping system and method based on deep learning have the following technical effects and advantages:

[0039] Figure 1 The battlefield target intelligent identification and mapping system and method based on deep learning have the following technical effects and advantages:

[0040] The battlefield target intelligent identification and mapping system and method based on deep learning have the following technical effects and advantages: Figure 2 The battlefield target intelligent identification and mapping system and method based on deep learning have the following technical effects and advantages:

[0041] The battlefield target intelligent identification and mapping system and method based on deep learning have the following technical effects and advantages:

[0042] Embodiment 1

[0043] Please refer to Figure 1 As shown in the embodiment, the battlefield target intelligent recognition and mapping method based on deep learning comprises the following steps:

[0044] S1. Collect battlefield image data and video data to obtain battlefield video data; perform data cleaning and clarity enhancement on the battlefield video data to obtain enhanced battlefield video data;

[0045] S2. Perform multi-modal feature extraction on the enhanced battlefield video data to obtain a set of scene feature vectors; perform scene recognition on the enhanced battlefield video data based on the set of scene feature vectors, and output a battlefield type label;

[0046] S3. Perform data scheduling on the enhanced battlefield video data based on the battlefield type label, obtain scheduled video data, and send the scheduled video data to an intelligent recognition path constructed based on a deep learning network corresponding to the battlefield type to perform target recognition process;

[0047] S4. Perform dynamic target intelligent recognition on the scheduled video data to output dynamic target trajectory data; perform dense target recognition on the scheduled video data to output high-density battlefield target detection data; perform auxiliary discrimination on the scheduled video data and update a preset database based on the discrimination result;

[0048] S5. Obtain battlefield space information corresponding to the enhanced battlefield video data, construct a battlefield simulation scene based on the battlefield space information, and accurately label battlefield targets in the battlefield simulation scene to obtain battlefield target visualization mapping data;

[0049] S6. Update the intelligent recognition path to an optimized processing path and send it to the preset database for storage.

[0050] The battlefield video data is obtained by taking battlefield images and videos from multiple angles by a UAV, and the data cleaning is completed by image denoising and invalid frame elimination on the battlefield video data in sequence. The clarity enhancement is completed by enhancing image contrast, enhancing edge clarity, and super-resolution reconstruction, and the clear enhanced battlefield video data is obtained, which is convenient for subsequent processing of the data.

[0051] The scene recognition of the enhanced battlefield video data comprises the following steps:

[0052] The discrimination feature vector is extracted, and the battlefield environment of the enhanced battlefield video data is discriminated to output the battlefield type label.

[0053] The scene feature vector set is uniformly formatted and coded to obtain a scene feature coding data set. In this embodiment, the battlefield enhanced image data is subjected to feature extraction through a multi-modal feature extraction channel, and the obtained scene feature vector set includes color features, texture features, spatial features and other features of the current scene. The obtained scene feature vectors are subjected to normalization processing and coded using a principal component analysis algorithm, and these codes are mapped to a uniform vector space, eliminating inconsistencies in structural dimensions and numerical distribution, and laying a data foundation for subsequent processing.

[0054] A feature correlation relationship is established for all scene feature codes in the scene feature coding data set to obtain a feature correlation coding matrix. The cosine similarity between the scene feature codes is calculated and used as a correlation discrimination index, a similarity threshold is set, scene feature codes with a cosine similarity greater than the similarity threshold are combined to form a correlation unit, and all correlation units form the feature correlation coding matrix. The feature correlation coding matrix is used to represent the semantic feature relationship between the scene feature codes therein.

[0055] A feature correlation topology is constructed based on the feature correlation coding matrix, and a neighborhood aggregation is performed on the feature correlation topology to generate a feature correlation coding data set. In the feature correlation topology, each scene feature code in each feature correlation coding matrix is taken as a node, and the semantic correlation between two scene feature codes is taken as an edge. The nodes with correlation are aggregated based on a graph neural network, so that each aggregated node set includes the semantic features of each type of related node, for example, the scene features of the same building group in the data are aggregated in a node set. Multi-scale feature extraction is performed on the feature correlation coding data set to obtain multi-scale features. In this embodiment, a CNN convolutional neural network is used to extract global and local semantic features from the feature correlation coding data set, and the output results are channel spliced to generate multi-scale features. The multi-scale features are used to capture regional difference information at different scales in the image data corresponding to the feature correlation coding data set.

[0056] The battlefield environment dynamic change mode is modeled in combination with multi-scale features, and a discriminative feature vector is output, wherein a pre-trained recurrent neural network model, that is, a GRU network model, is used for modeling. In order to enhance the discriminative ability of the battlefield type, scene feature change modes between frames or images at different times in the image data are learned by receiving multi-scale features, wherein the output discriminative feature vector includes information such as spatial features, semantic association, dynamic change mode, and change trend between previous and subsequent frames of the image or video. Based on the discriminative feature vector, the battlefield enhanced image data is subjected to battlefield environment discrimination, and a battlefield type label is output. In this embodiment, the discriminative feature vector is specifically classified by a Softmax classifier, and the obtained battlefield type label includes various battlefield types such as jungle battlefield, urban battlefield, sea battlefield, and air battlefield.

[0057] The battlefield type label is used to schedule the battlefield enhanced image data in the following manner:

[0058] A scene identifier corresponding to the battlefield type is generated based on the battlefield type label, and a mapping relationship between the scene identifier and the battlefield type label is established. A label matching condition is constructed in combination with expert experience and military field literature, including feature modes, terrain structures, and tactical performance elements of each label corresponding to the battlefield type, for example, in an air battlefield scene, the proportion of the sky region is high, the boundary texture is sparse, the frequency of buildings is low, and the background changes slowly. For each label matching condition and corresponding battlefield type label, a unique scene identifier is generated, and the scene identifier is constructed as a code structure that can be recognized by the system.

[0059] The intelligent recognition path of the preset control end is divided based on the battlefield type label. In this embodiment, the control end constructs an intelligent recognition path most suitable for each battlefield type label. For example, when the battlefield type label is urban street battle, the intelligent recognition path adopts high-precision edge detection and geometric structure recognition, and is biased towards small target tracking. When the battlefield type label is an air battlefield, the intelligent recognition path pays more attention to tracking high-speed targets.

[0060] A deep learning network model is deployed in each intelligent recognition path, and historical battlefield enhanced image data corresponding to the battlefield type label to which the intelligent recognition path belongs is collected to train the deep learning network model. The types of deep learning network models deployed in each intelligent recognition path are different. For example, a YOLO model is deployed in the air battlefield path, which has a time series modeling and attention mechanism, and is easy to capture high-speed moving targets. An HRNet model is deployed in the sea battlefield path, which has strong background modeling capability and excellent adaptability to water surface shaking. The model is trained by using historical battlefield enhanced image data corresponding to the battlefield type label, so as to ensure that each intelligent recognition path focuses on recognizing one battlefield environment and improves the execution efficiency.

[0061] The corresponding intelligent recognition path is activated by using the scene identifier, and the battlefield enhancement image data is taken as input data of the intelligent recognition path. The corresponding intelligent recognition path is activated by using the scene identifier, and the battlefield enhancement image data is taken as input data of the intelligent recognition path. The function of adaptive switching of the intelligent recognition path based on the input data is realized. Whenever the scene described by the input data meets a certain scene identifier, the corresponding intelligent recognition path is enabled and the deep learning network model is loaded to prepare for target recognition. However, due to the fact that the models deployed in each path may not be able to perform all recognition tasks in different battlefield environments, these factors often include displacement deviation, target distribution density and new equipment identification, three common problems, so in the subsequent steps, the scheduling image data allocated to each intelligent recognition path is processed for general function, solving the three common problems, as an extension of each intelligent recognition path. Supplementary means to improve the overall recognition accuracy.

[0062] The way of dynamically identifying the scheduling image data includes:

[0063] The scheduling image data is converted into image frames and sorted based on timestamps to obtain a battlefield image frame sequence. In order to facilitate the capture of motion information in the time dimension from continuous scenes, the scheduling image data is processed by frame-by-frame disassembly, and video type data and image sequences are converted into battlefield image frame sequences sorted based on timestamps.

[0064] The basic feature information of the battlefield target is obtained, and the battlefield image frame sequence is preliminarily screened to output a target candidate set. The basic feature information of the battlefield target includes the basic information such as the size, color texture distribution and shape of the battlefield target to be identified. The basic feature information of the battlefield target is matched with each frame of image by frame-by-frame scanning of the battlefield image frame sequence. The region containing the feature information of the battlefield target to be identified is set as a target candidate region, and all target candidate regions are integrated to form a target candidate set. For example, if the battlefield target to be identified is a certain type of tank, the region where the rectangular, low and edge contour obvious object is located is set as the target candidate region.

[0065] A target recognition window is constructed to frame the battlefield target in the target candidate set, and a target category probability is output. In this embodiment, a detection frame is constructed by using the deep learning network model deployed in the corresponding intelligent recognition path. The detection frame is used to frame the battlefield target, and the target category probability is output around the detection frame. The target category probability refers to the recognition probability of the target category in the detection frame.

[0066] The frame images of the continuous N frames in the battlefield image frame sequence are boxed, and the position change information of the battlefield target is obtained. In this embodiment, the model is called to box the battlefield target in the continuous N frames of frame images of the battlefield target to be recognized, wherein N can be dynamically adjusted based on the specific number of frame images; wherein the position change information includes the coordinate position of the battlefield target in each frame image, and the speed and displacement direction of the battlefield target, and the time stamp corresponding to the frame image is combined to obtain the position change information of the battlefield target in multiple frames of images over time.

[0067] The future trajectory of the battlefield target is predicted based on the position change information, and the future frame prediction path of the battlefield target is obtained. In this embodiment, the pre-trained LSTM model is used to capture the trajectory change trend in the position change information of the battlefield target, so as to realize the prediction of the position in the future several frames of images of the battlefield target.

[0068] The error of the recognition position of the current battlefield target is corrected based on the future frame prediction path, and the dynamic target running trajectory data of the battlefield target is output, the position information of the battlefield target at a certain frame at the current time and the position change information of the past several frames of the frame are obtained, and the future frame prediction path is predicted; and the future frame prediction path is compared with the detection motion path of the battlefield target in the image sequence of the same frame number after the frame and the future frame prediction path, and the error between the two paths is calculated; wherein the error calculation formula is: Δdis=(a1-a, b1-b, c1-c, d1-d); wherein, Δdis represents the error compensation value of the detection box of the battlefield target in each frame image between the two paths; a1 represents the horizontal coordinate of the center point of the detection box in the prediction path; a represents the horizontal coordinate of the center point of the detection box in the detection motion path; b1 represents the vertical coordinate of the center point of the detection box in the prediction path; b represents the vertical coordinate of the center point of the detection box in the detection motion path; c1 represents the width of the detection box in the prediction path; c represents the width of the detection box in the detection motion path; d1 represents the height of the detection box in the prediction path; d represents the height of the detection box in the detection motion path; The running trajectory data can be used to adjust the error of the recognition position, and can also be used to complete the motion trajectory when the battlefield target is temporarily lost.

[0069] The way of dense target recognition for dispatch video data includes:

[0070] Deep feature of each frame image in the battlefield image frame sequence is extracted, to prevent some frame images in the battlefield image frame sequence from changing dramatically in the image environment, resulting in that normal feature extraction cannot fully extract features, in the embodiment, the feature extraction is combined with a spatial pyramid pooling strategy to obtain the deep feature of the frame image.

[0071] Target distribution density of each frame image is calculated based on the deep feature of the frame image, in the embodiment, the response intensity of each pixel point in the frame image is calculated by using a method of superimposing Gaussian kernel functions, a sliding window with adjustable size is constructed to traverse the frame image, and the average value of the sum of the response intensities of all pixel points in each sliding window is the target distribution density of the region corresponding to the window; wherein the calculation formula of the response intensity is: ρ(x,y) represents the response intensity of the pixel point with coordinates (x,y) in the frame image; Num represents the total number of pixel points; G σ1 represents the Gaussian kernel function with parameter σ1, which is used to measure the influence degree of the pixel point with coordinates (x k ,y k ) on the pixel point with coordinates (x,y).

[0072] Density heat maps corresponding to each frame image are drawn based on the target distribution density, the density heat maps are divided into density levels by setting a distribution density threshold, and high-density target regions are determined, wherein the higher the target distribution density of a region in the density heat map, the more and denser the potential targets in the region; the region with a target distribution density greater than a preset distribution density threshold is determined as a high-density target region, which facilitates subsequent identification of potential targets in the region.

[0073] Candidate target initial detection points are set in the high-density target region, and feature perception is performed on the candidate target initial detection points to output feature perception data, candidate target initial detection points are set in each local response maximum region in the high-density target region, Top-k algorithm is used to select pixel points in the neighborhood of the candidate target initial detection points and perform feature perception, the pixel points are introduced into the middle perception layer of the CNN model, and the feature perception data is obtained by fusing the attribute information such as texture, illumination and semantics in the region formed by the pixel points.

[0074] The feature perception data is matched and distinguished with the basic feature information of the battlefield target, and the battlefield target is framed. In the embodiment, the feature perception data and the basic feature information of the battlefield target are both converted into vector forms, the Euclidean distance between the two vectors is calculated, and a Euclidean distance threshold is set. If the Euclidean distance is less than the preset Euclidean distance threshold, it is considered that the feature perception data can be matched with the basic feature information of the battlefield target, and the battlefield target can be framed by using the detection frame. However, at this time, a region is framed, instead of each single target, which provides a basis for subsequent elimination of overlapping conflicts.

[0075] When the contours overlap in the detection frame, the battlefield target that each detection frame continuously appears in the adjacent frame is reserved, and the continuously frame recognition is performed on the occluded target. The battlefield target contour overlap includes two cases, one is that multiple battlefield targets overlap together, and the other is that the battlefield target to be recognized is partially occluded by other non-target objects in the environment. In the embodiment, the battlefield target that completely appears in continuous frames is extracted and reserved in the detection frame. If multiple moving targets appear in the continuous frames, but are not completely displayed in some frame images, the moving targets are considered as occluded targets and are marked. The position change of the occluded target is obtained by gradually expanding the continuous frame recognition range. In the embodiment, a time sequence sliding window combined with backward tracking and forward expansion is constructed, and the frame image in which the occluded target first appears is taken as the starting point, and the forward and backward traversals are performed respectively. The moving track of the occluded target in the continuous frames, that is, the position change, is determined by extracting the coordinate position, displacement direction and speed of the occluded target in other frame images.

[0076] The complete contour of the occluded target is determined based on the position change and the basic feature information of the battlefield target, the battlefield target is framed, and the target category probability is generated. Since the occluded target does not completely disappear, part of the feature information about the occluded target can be extracted from other frame images. Meanwhile, there may be a frame image in which the contour of the occluded target is completely displayed in other frame images, but it cannot be confirmed whether the occluded target in the image is the target to be recognized. Therefore, in the embodiment, the possible complete contour of each occluded target is restored based on the basic feature information of the battlefield target, the contour in other frame images that meets the position change is matched with the possible complete contour, and finally the frame image that can reflect the complete contour of the corresponding occluded target is found, and the battlefield target is framed in the image and the target category probability is generated. The high-density battlefield target detection data is constituted based on the recognized battlefield image frame sequence. The recognized battlefield image frame sequence realizes the recognition of each single target in the high-density region. In order to prevent too many detection frames from appearing in any frame image at any time, each frame image is layered, and each layer corresponds to a battlefield target.

[0077] The manner of performing auxiliary discrimination on the scheduled video data comprises:

[0078] An unknown target position is determined and pixel features are extracted to generate an unknown target feature vector; wherein a target that is framed by a detection frame but cannot be matched with the basic feature information of a known battlefield target in any frame image of the battlefield video frame sequence is regarded as an unknown target, in this embodiment, the unknown target refers to new equipment or weapons appearing in the battlefield that are not included in the preset database, such as new tanks or new warplanes, and cannot be identified if not included in the database; unknown target feature vectors of the unknown target are obtained by performing feature extraction on all pixel blocks in the region where the unknown target is located, including texture analysis and other technical means.

[0079] Known target feature vectors corresponding to the known battlefield targets are obtained, and a feature distance between the unknown target feature vector and the known target feature vector is calculated, in this embodiment, the feature distance is the Mahalanobis distance.

[0080] A confidence degree is calculated based on the feature distance, a confidence degree matching threshold is set, when the confidence degree is greater than or equal to the confidence degree matching threshold, the unknown target is marked as a related target, otherwise it is marked as a typical unknown target; wherein the related target indicates that there is a correlation with the known battlefield target, for example, the related target belongs to the extended type of the known battlefield target; the feature information of the typical unknown target is added as a new entry of the preset database, the feature information of the related target is merged with the related known battlefield target entry in the preset database, and the self-updating of the database is realized by adding the typical unknown target and the related target to the preset database, so as to facilitate the future calling of the database for target identification.

[0081] The manner of constructing a battlefield simulation scene comprises:

[0082] Pixel change features of any region in the battlefield enhanced video data are extracted with respect to time stamps, and a change trajectory of the region is constructed based on the pixel change features, wherein the pixel change features refer to information such as brightness values, color channels and edge gradients of pixel blocks in the selected region; the pixel change features are combined with the time stamps to obtain the change trajectory of the region in a period of time; the change amplitude of the change trajectory is quantified, a change amplitude threshold is set, and a spatial reconstruction region is selected based on the change amplitude and the change amplitude threshold, wherein the calculation formula of the change amplitude is: Wherein, ΔTR represents the change amplitude of any change trajectory; M represents the total number of pixel points of the region corresponding to the change trajectory; R represents the region corresponding to the change trajectory; I t (X0,Y0) represents the pixel value of the coordinate (X0,Y0) at time t; I t-1(X0, Y0) represents the pixel value of the coordinate (X0, Y0) at the time t-1; the change trajectory with a change amplitude greater than a preset change amplitude threshold is determined as a large change region, which needs to be taken as a spatial reconstruction region.

[0083] The timestamp and the spatial reconstruction region are fused to output a battlefield environment change layer, and the spatial reconstruction region is combined with the frame image corresponding to each timestamp to form an environment change layer of the region changing over time, that is, the battlefield environment change layer; the layer can fully reflect the battlefield environment change caused by the movement of obstacles or the destruction of the terrain in the battlefield; a battlefield environment space model is constructed based on the battlefield environment three-dimensional coordinates, and the battlefield environment change layer is time-sequentially matched with the battlefield environment space model to generate a battlefield simulation scene; in the embodiment, a modeling technology is used to construct a battlefield environment space model based on battlefield space information, wherein the battlefield space information includes information reflecting spatial properties such as GPS coordinates of an actual battlefield corresponding to the battlefield augmented image data, battlefield environment height difference data, and battlefield area, and the battlefield environment change layer is synchronously nested into the battlefield environment space model according to the timestamp, so that the model not only has a static structure, but also can dynamically reflect the active battle area, the battlefield destruction area, and the change of the shelter, thereby improving the readability of the generated battlefield simulation scene.

[0084] The precise labeling manner of the battlefield target of the battlefield simulation scene includes:

[0085] An image frame index of each battlefield target is acquired, a top-level battlefield target mapping layer is constructed, and the image coordinates corresponding to the latest timestamp and the initial timestamp of each battlefield target are mapped to the top-level battlefield target mapping layer based on the image frame index; in the embodiment, the earliest appearing coordinate and the latest appearing coordinate of the battlefield target are extracted to obtain the first frame coordinate point and the last frame coordinate point of the life cycle of the battlefield target in the data of the embodiment; the first frame coordinate point and the last frame coordinate point of each battlefield target are projected to the top-level battlefield target mapping layer, which is used to reflect the starting point and the ending point of the movement trajectory of the battlefield target; the first frame coordinate point and the last frame coordinate point of each battlefield target are labeled in the top-level battlefield target mapping layer, so that the performance form is more intuitive, and the situation that the coordinate points of the battlefield target at each time point are all present in one layer does not occur.

[0086] The battlefield target motion layer is constructed and layer division is performed, to generate a battlefield target motion sub-layer, wherein the number of sub-layers of the battlefield target motion layer is equal to the number of battlefield targets, and a category label is used for marking, so that each battlefield target motion sub-layer corresponds to a unique battlefield target; the dynamic target running track data is mapped to the corresponding battlefield target motion sub-layer, and in this embodiment, the dynamic target running track data is converted into a coordinate expression, to form structured track point data composed of a plurality of coordinate points, wherein the path of the obscured tactical target is simulated by continuous frame recognition; the real motion track of each battlefield target is restored by projecting the dynamic target running track data of each battlefield target to the corresponding battlefield target motion sub-layer, and the user can directly search for the category label of the battlefield target to view the corresponding motion track.

[0087] A battlefield dense area layer is constructed, and the battlefield dense area layer is marked based on high-density battlefield target detection data, the density heat map of each frame image in the high-density battlefield target detection data is superimposed into the battlefield density area layer, and in this embodiment, color is used to distinguish the density levels, red represents a high-density area, yellow represents a medium-density area, blue represents a sparse area, and green represents an open area; the battlefield dense area layer can be used for important decisions such as fire deployment and route design of the battlefield.

[0088] A structured plotting layer is generated, the structured plotting layer and the battlefield simulation scene are layered, to generate battlefield target visual plotting data, wherein the battlefield target motion layer, the battlefield dense area layer and the top battlefield target mapping layer are combined; the structured plotting layer includes a tactical target motion track, a first frame coordinate point and a high-density area identifier; the user can adjust the layer structure as needed, for example, if the motion track of a certain battlefield target is needed, the battlefield target motion sub-layer corresponding to the target can be adjusted to the top layer, and other layers can be hidden; the battlefield target visual plotting data is obtained by binding the structured plotting layer and the coordinates of the battlefield simulation scene, and designing functions such as layer transparency adjustment and layer dynamic state opening.

[0089] The acquisition method of the optimized processing path includes:

[0090] Real-time feedback data is collected, and the intelligent recognition path is optimized based on the feedback data to obtain an optimized processing path; wherein the feedback data is time-divisionally divided and feature-extracted to obtain an intelligent recognition difference vector, wherein the intelligent recognition difference vector is a vector corresponding to each intelligent recognition path in any continuous time period; in this embodiment, the feedback data is divided into several time periods based on timestamps, and the intelligent recognition difference vector is constructed by extracting the response efficiency, misjudgment probability and resource occupancy rate of the intelligent recognition path in each time period, which is used to reflect the running performance of the intelligent recognition path in the corresponding time period.

[0091] Integrate all intelligent recognition difference vectors to obtain a path difference feedback set, and establish a corresponding relationship between a path parameter of each intelligent recognition path and the intelligent recognition difference vector, wherein the path parameter includes a model parameter of a deep learning network model deployed in the intelligent recognition path and a discrimination threshold and the like. The corresponding relationship between the path parameter and the intelligent recognition difference vector is established so that the intelligent recognition difference vector can be used as a judgment basis in subsequent operations to optimize the relevant path parameter.

[0092] An effect evaluation model is constructed to evaluate the path recognition effect of the path difference feedback set, and output a path recognition effect score of each intelligent recognition path. In this embodiment, the effect evaluation model is a multi-layer perception network model. The historical path difference feedback set is collected and labeled with a time period label. The historical path difference feedback set is used as a training set of the multi-layer perception network model. The path recognition effect score of each intelligent recognition path in each time period is calculated by the loss function of the model.

[0093] An identification effect score threshold is set. The intelligent recognition path with a path recognition effect score less than the identification effect score threshold is determined as an optimization target path. The path parameter of the optimization target path is analyzed, and the path parameter is reconstructed based on the parameter analysis result. The optimized parameter is applied to the optimization target path. In this embodiment, the path parameter of the optimization target path is extracted, and the parameter set is reconstructed by using the Bayesian optimization algorithm. The optimized parameter is obtained and applied to the corresponding optimization target path to obtain the optimized optimization target path.

[0094] The optimized optimization target path and the intelligent recognition path with a path recognition effect score greater than or equal to the identification effect score threshold are integrated to obtain an optimized intelligent recognition path. The optimized intelligent recognition path is sent to a preset database for storage, and the optimized optimization target path is used to directly replace the original intelligent recognition path to realize the closed-loop parameter optimization of the system.

[0095] The embodiment realizes accurate labeling of battlefield targets by performing scene recognition, recognition path scheduling, trajectory prediction, dense target processing, unknown target auxiliary recognition and scene construction on collected battlefield video data, and improves the performance of target recognition by optimizing the recognition path parameters; By constructing an intelligent recognition path based on scene label division, dynamically loading a dedicated recognition model according to the battlefield type, the task adaptability and recognition accuracy of target recognition in complex battlefield environments are significantly improved; By fusing the inter-frame detection results and the pre-training time sequence model, the multi-target trajectory real-time prediction is realized, and the target positioning accuracy in continuous time is strengthened through error correction; In the high-density area recognition, the candidate detection point mechanism and the occlusion completion strategy are fused to ensure the recognition integrity and stability in the target dense and occlusion interference scenes; At the same time, unknown targets can be identified and the database can be dynamically expanded and updated, which has good self-learning ability and equipment recognition expansion ability; In addition, in the simulation scene construction and target labeling stage, multi-layer target nesting is realized based on the three-dimensional coordinate system, which improves the visualization clarity and multi-layer controllability; In the feedback optimization path scheduling, the recognition path parameters are reconstructed based on the time segmentation error feedback, which effectively improves the recognition efficiency and closed-loop control ability; It has very high practical deployment value and is suitable for multi-type battlefield target recognition tasks, which helps to improve the execution efficiency of dynamic target management and intelligent command system.

[0096] Embodiment 2

[0097] Please refer to Figure 2 The embodiment does not describe some parts in detail, see the description of embodiment 1, and provides a battlefield target intelligent recognition and labeling system based on deep learning, comprising:

[0098] A data acquisition module is used to acquire battlefield video data and perform data cleaning to obtain battlefield enhanced video data;

[0099] A scene recognition module is used to extract multi-modal features from the battlefield enhanced video data to obtain a scene feature vector set; The scene recognition module performs scene recognition on the battlefield enhanced video data based on the scene feature vector set, and outputs a battlefield type label;

[0100] A path construction module is used to schedule the battlefield enhanced video data based on the battlefield type label to obtain scheduling video data and send the scheduling video data to an intelligent recognition path constructed based on a deep learning network to perform a target recognition process;

[0101] A fine recognition module is used to perform dynamic target intelligent recognition on the scheduling video data and output dynamic target running trajectory data; The fine recognition module performs dense target recognition on the scheduling video data and outputs high-density battlefield target detection data; The fine recognition module performs auxiliary discrimination on the scheduling video data and updates a preset database based on the discrimination result;

[0102] The visualization mapping module is configured to acquire battlefield space information corresponding to the battlefield enhanced image data, construct a battlefield simulation scene based on the battlefield space information, and accurately label a battlefield target of the battlefield simulation scene to obtain battlefield target visualization mapping data.

[0103] The parameter optimization module is configured to update the intelligent identification path to an optimized processing path and send the optimized processing path to the preset database for storage.

[0104] The above merely describes preferred embodiments of the present application and is not intended to limit the present application. Although the foregoing embodiments of the present application have been described in detail, those skilled in the art can modify the technical solutions described in the foregoing embodiments or equivalently replace some technical features thereof. Any modification, equivalent replacement or improvement made within the spirit and principle of the present application shall fall within the protection scope of the present application.

[0105] It should be noted that in this document, the terms "comprise", "comprising", or any other variant thereof are intended to cover non-exclusive inclusions, so that processes, methods, articles or devices that include a series of elements not only include those elements, but also include other elements not explicitly listed, or include elements inherent to such processes, methods, articles or devices. Without more limitations, the element defined by the statement "comprises a" does not exclude the presence of another identical element in the process, method, article or device that includes the element.

[0106] In the description of the present application, it should be understood that the terms "first", "second" and the like are only used to distinguish descriptions, and cannot be understood as indicating or implying relative importance.

[0107] In the description of the present application, unless otherwise specified, the meaning of "a plurality of" is two or more.

[0108] In the description of the present application, the meaning of "several" is one or more, and the meaning of "a large number" is two or more.

[0109] In the description of the present application, the description of the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0110] The formula of the present specification is a value calculated by de-dimensioning, the formula is obtained by collecting a large amount of data to simulate a formula of the most recent real situation, and preset parameters and threshold values in the formula are set by a person skilled in the art according to actual conditions.

[0111] Although the embodiments of the present application have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the present application, and the scope of the present application is defined by the claims and their equivalents.

Claims

1. A battlefield target intelligent identification and mapping method based on deep learning, characterized in that, The method comprises the following steps: S1. Collect battlefield image data and perform data cleaning to obtain enhanced battlefield image data; S2. Perform multi-modal feature extraction on the enhanced battlefield image data to obtain a set of scene feature vectors; Perform scene recognition on the enhanced battlefield image data based on the set of scene feature vectors, and output a battlefield type label; S3. Perform data scheduling on the enhanced battlefield image data based on the battlefield type label, obtain scheduled image data, and send the scheduled image data to an intelligent recognition path constructed based on a deep learning network to perform target recognition; S4. Perform dynamic target intelligent recognition on the scheduled image data, and output dynamic target running track data; Perform dense target recognition on the scheduled image data, and output high-density battlefield target detection data; Perform auxiliary discrimination on the scheduled image data, and update a preset database based on the discrimination result; S5. Obtain battlefield space information corresponding to the enhanced battlefield image data, and construct a battlefield simulation scene based on the battlefield space information; Perform accurate labeling on battlefield targets in the battlefield simulation scene to obtain battlefield target visualization plot data; S6. Update the intelligent recognition path to an optimized processing path and send the optimized processing path to the preset database for storage.

2. The battlefield target intelligent identification and mapping method based on deep learning according to claim 1, characterized in that, The scene recognition method for the enhanced battlefield image data comprises: Extracting a discrimination feature vector and discriminating the enhanced battlefield image data to output a battlefield type label.

3. The battlefield target intelligent identification and mapping method based on deep learning according to claim 2, characterized in that, The data scheduling method for the enhanced battlefield image data based on the battlefield type label comprises: Generating a scene identifier corresponding to the battlefield type based on the battlefield type label, establishing a mapping relationship between the scene identifier and the battlefield type label, dividing an intelligent recognition path of a preset control terminal based on the battlefield type label, deploying a deep learning network model in each intelligent recognition path, collecting historical enhanced battlefield image data corresponding to the battlefield type label of the intelligent recognition path to train the deep learning network model, and activating the corresponding intelligent recognition path by using the scene identifier, and inputting the enhanced battlefield image data as input data of the intelligent recognition path.

4. The battlefield target intelligent identification and mapping method based on deep learning according to claim 3, characterized in that, The dynamic target intelligent recognition method for the scheduled image data comprises: Converting the scheduled image data into a sequence of battlefield image frames, obtaining basic feature information of a battlefield target, performing preliminary screening on the sequence of battlefield image frames to output a target candidate set, and performing bounding on the battlefield target in the target candidate set while outputting a target category probability; Performing bounding on the sequence of battlefield image frames to obtain position change information of the battlefield target, predicting a future track of the battlefield target based on the position change information to obtain a future frame prediction path of the battlefield target, and correcting an error in the recognition position change of the current battlefield target based on the future frame prediction path to output dynamic target running track data of the battlefield target.

5. The battlefield target intelligent identification and mapping method based on deep learning according to claim 4, characterized in that, The dense target recognition method for the scheduled image data comprises: Deep feature extraction is performed on each frame image in the battlefield image frame sequence to obtain frame image deep features; target distribution density of each frame image is calculated based on the frame image deep features; high-density target regions are determined based on the target distribution density; candidate target initial detection points are set and feature perception is performed on the candidate target initial detection points to output feature perception data; the feature perception data is matched and discriminated with basic feature information of battlefield targets, and a battlefield target frame is selected; The battlefield targets that continuously appear in the adjacent frames of each detection frame are retained, and the continuously frame recognition is performed on the occluded targets; the position change of the occluded targets is obtained; the complete contour of the occluded targets is determined based on the position change and the basic feature information of the battlefield targets, the battlefield target frame is selected, and the target category probability is generated; the high-density battlefield target detection data is formed based on the recognized battlefield image frame sequence.

6. The battlefield target intelligent identification and mapping method based on deep learning according to claim 5, characterized in that, The auxiliary discrimination manner for the scheduling image data comprises: An unknown target position is determined, and pixel features are extracted to generate an unknown target feature vector; a known target feature vector corresponding to a known battlefield target is obtained, and a feature distance between the unknown target feature vector and the known target feature vector is calculated; a confidence degree is calculated, a confidence degree matching threshold is set, when the confidence degree is greater than or equal to the confidence degree matching threshold, the unknown target is marked as a related target, otherwise, it is marked as a typical unknown target; the feature information of the typical unknown target is extracted as a new entry of a preset database, and the feature information of the related target is combined with the existing entries in the preset database.

7. The battlefield target intelligent identification and mapping method based on deep learning according to claim 6, characterized in that, The battlefield simulation scene construction manner comprises: Pixel change features are extracted, and a change trajectory of the region is constructed based on the pixel change features; the change amplitude of the change trajectory is quantified, a change amplitude threshold is set, and a spatial reconstruction region is screened; a timestamp and the spatial reconstruction region are fused to output a battlefield environment change layer; a battlefield environment space model is constructed based on battlefield space information, and time sequence matching is performed with the battlefield environment change layer to generate a battlefield simulation scene.

8. The battlefield target intelligent identification and mapping method based on deep learning according to claim 7, characterized in that, The precise labeling manner for the battlefield targets in the battlefield simulation scene comprises: Image frame indexes of each battlefield target are obtained, a top-level battlefield target mapping layer is constructed, and image coordinates corresponding to the latest timestamp and the initial timestamp of each battlefield target are mapped to the top-level battlefield target mapping layer based on the image frame indexes; a battlefield target motion layer is constructed and is divided into layers to generate battlefield target motion sub-layers; dynamic target running track data is mapped to the corresponding battlefield target motion sub-layers; a battlefield dense region layer is constructed, and the battlefield dense region layer is annotated for dense degree based on high-density battlefield target detection data; a structured plotting layer is generated, the structured plotting layer and the battlefield simulation scene are layered, and battlefield target visual plotting data is generated.

9. The battlefield target intelligent identification and mapping method based on deep learning according to claim 8, characterized in that, The acquisition manner of the optimized processing path comprises: Real-time feedback data is collected, and the intelligent recognition path is parameter-optimized based on the feedback data to obtain the optimized processing path.

10. A battlefield target intelligent recognition and mapping system based on deep learning, used to implement the battlefield target intelligent recognition and mapping method based on deep learning in any one of claims 1-9, characterized in that, Comprise: A data acquisition module is configured to collect battlefield image data and perform data cleaning to obtain battlefield enhanced image data; The scene recognition module is configured to perform multi-modal feature extraction on the battlefield augmented image data to obtain a set of scene feature vectors; The scene recognition module is configured to perform multi-modal feature extraction on the battlefield augmented image data to obtain a set of scene feature vectors; The path construction module is configured to perform data scheduling on the battlefield augmented image data based on the battlefield type label to obtain scheduled image data and send the scheduled image data to an intelligent recognition path to perform target recognition process based on a deep learning network; The fine recognition module is configured to perform dynamic target intelligent recognition on the scheduled image data to output dynamic target running track data; The fine recognition module is configured to perform dynamic target intelligent recognition on the scheduled image data to output dynamic target running track data; The fine recognition module is configured to perform dynamic target intelligent recognition on the scheduled image data to output dynamic target running track data; The visualization mapping module is configured to obtain battlefield space information corresponding to the battlefield augmented image data, and construct a battlefield simulation scene based on the battlefield space information; The visualization mapping module is configured to obtain battlefield space information corresponding to the battlefield augmented image data, and construct a battlefield simulation scene based on the battlefield space information; The parameter optimization module is configured to update the intelligent recognition path to an optimized processing path and send the optimized processing path to the preset database for storage. The modules are connected through wired and / or wireless means.

Images