An electromagnetic radiation source individual identification method based on a neural network and a knowledge graph double-channel system

Abstract

The application discloses an electromagnetic radiation source individual identification method based on a neural network and a knowledge graph double-channel system, and comprises the following steps: acquiring initial electromagnetic radiation source signals and intelligence information by using a signal acquisition device, filtering and cleaning the data; performing envelope analysis and bispectrum analysis on the filtered signals to obtain envelope data and bispectrum data respectively; giving the same label to the original signal data of the same individual after filtering and cleaning, and corresponding envelope data and bispectrum data, and associating the intelligence information with the target individual; using an open set identification method to determine the type of all the target individuals; using a collaborative reasoning method to complete the individual identification task of the electromagnetic radiation source signals for the data determined as the known class; using a knowledge dynamic updating method to update the electromagnetic radiation source identification network and the knowledge graph for the data determined as the unknown class, and then using the collaborative reasoning method to complete the individual identification task of the signals.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of deep learning and electromagnetic radiation source individual identification, and relates to an electromagnetic radiation source individual identification method based on a neural network and a knowledge graph double-channel system. BACKGROUND

[0002] In the modern electromagnetic environment, due to the increasing signal density of electromagnetic radiation sources, and the factors such as time-frequency domain overlap, complex signal modulation and variable parameters, the electromagnetic radiation source feature mining, knowledge updating and reasoning utilization all encounter great difficulties, which brings great challenges to electronic reconnaissance. In terms of the complexity of the modern electromagnetic environment, it mainly reflects in three aspects: 1) the number of signal sources is large, the distribution density is large, the distribution range is wide and the signal overlap is serious. 2) The signal modulation is complex, the parameters are variable and agile. 3) The data cannot be effectively utilized, the intelligence analysis process lacks knowledge assistance, the automation level is not high, and the electronic reconnaissance equipment cannot effectively provide intelligence assistance for command and control.

[0003] The reason is that, on the one hand, the existing massive data contains rich valuable information, but lacks compact and effective organizational structure and intuitive query method, and has not formed a corresponding knowledge system, which cannot be expressed as computer rules by manpower, so it is necessary to let the computer learn to utilize the existing knowledge to carry out systematic reasoning through various technical means. On the other hand, due to the continuous updating and change of the actual electromagnetic environment, if the reasoning system cannot update the knowledge in real time, the accuracy and reliability of the identification will gradually be lost.

[0004] Based on deep learning method can extract the fingerprint characteristics in electromagnetic radiation source signal, and then effectively complete the electromagnetic radiation source individual identification task (such as: [1] Wang X, Huang G, Ma C, et al. Convolutional neural network applied to specific emitter identification based on pulse waveform images [J]. Iet Radar Sonar and Navigation, 2020, 14(5):728-735. [2] Si W, Wan C, Zhang C. Towards an accurate radar waveform recognition algorithm based on dense CNN [J]. Multimedia Tools and Applications, 2021, 80(2):1779-1792. [3] Liu Y, Tian R, Wang X. Radar signal recognition method based on deep convolutional neural network and bispectrum characteristics [J]. System Engineering and Electronic Technology, 2019(9).). The wide application of knowledge graph provides a more intelligent storage and utilization way for the knowledge of intelligence information and other knowledge in the field of electromagnetic radiation source. The way to solve the accurate identification of electromagnetic radiation source individual is to integrate the advantages of deep neural network and knowledge graph, use network to extract the implicit features in electromagnetic radiation source signal, and use knowledge graph to generate auxiliary reasoning information, so that the two can cooperate to complete the individual identification task, break through the precision limit of single identification model, and finally significantly improve the individual identification accuracy.

[0005] To ensure the credibility of collaborative reasoning, it is necessary to ensure the recognition ability of the recognition network to all known categories and the timeliness of the intelligence information in the knowledge graph. Specifically, in order to enable the recognition network to have the ability to recognize new categories of electromagnetic radiation source individuals that are constantly emerging, the model needs to have the ability of incremental learning (such as [1] Hu X, Tang K, Miao C, et al. Distilling Causal Effect of Data in Class-Incremental Learning [C]. proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition (CVPR). Electr Network, 2021: 3956-3965. [2] Cha H, Lee J, Shin J. Co2L: Contrastive Continual Learning [C]. proceedings of the 2022 IEEE / CVF Conference on Computer Vision and Pattern Recognition. 2022. [3] Ahn H, Kwak J, Lim S, et al. SS-IL: Separated Softmax for Incremental Learning [C]. proceedings of the 2021 IEEE International Conference on Computer Vision (ICCV). 2021), that is, after learning new category samples, the model still maintains the recognition ability to old categories, in this process only a small amount of old category samples need to be saved, and the incremental training time of the model is greatly reduced compared with general learning models. In order to ensure that the knowledge in the knowledge graph remains the latest, it is necessary to update the knowledge graph for the newly obtained external intelligence information. SUMMARY

[0006] In order to overcome the deficiencies of the prior art, the present application proposes an electromagnetic radiation source individual identification method based on a neural network and a knowledge graph dual-channel system. Specifically, it is divided into two parts: a collaborative reasoning method based on a deep neural network and a knowledge graph, and a knowledge dynamic updating method. For the electromagnetic radiation source signal obtained by the receiver and the corresponding intelligence information (signal parameters, geographic location) and other information, first, use the open set identification technology to determine the known class / unknown class, and use the collaborative reasoning method in the present application to predict the signal class: first, use the multi-modal feature extraction network to extract the implicit features of the signal, and second, query the intelligence information in the knowledge graph to obtain auxiliary reasoning information and encode it into a feature vector, then fuse these features and classify them to obtain the final reasoning result. For signal data determined as unknown class, use the knowledge updating method in the present application to update the identification network and the knowledge graph: first, use the class incremental learning based method to update the identification network, so that it can learn new classes while maintaining the identification ability of old classes, and second, update the knowledge graph based on the intelligence information of the electromagnetic radiation source, generate corresponding nodes in the graph and produce new links. Knowledge dynamic updating is the basis of collaborative reasoning, which guarantees the accuracy and credibility of collaborative reasoning; collaborative reasoning is the technical core of realizing precise identification of electromagnetic radiation source individuals.

[0007] The specific steps of the electromagnetic radiation source individual identification method based on the neural network and the knowledge graph dual-channel system of the present application are as follows:

[0008] Step 1: Use the electromagnetic radiation source signal acquisition device to obtain the initial signal of the electromagnetic radiation source from the target individual, and also obtain the signal parameters, geographic location and other intelligence information corresponding to these data for subsequent use. These initial radiation source signals are periodic, showing a process of "voltage amplitude suddenly increasing in a short duration (millisecond level), and then quickly returning to its initial value", and the signal formed by completing one cycle is called a pulse. The initial electromagnetic radiation source signal is cropped according to a single pulse period to obtain several pulse signals, and the pulse signals with a signal-to-noise ratio lower than a preset threshold are filtered out, and the threshold is determined according to the environmental interference factors in the signal acquisition process, and the two are negatively correlated, that is, the greater the interference, the lower the threshold setting, and the smaller the interference, the higher the threshold setting, which is generally set to 3.

[0009] Step 2: Use the Hilbert transform method to extract the envelope features of the filtered pulse signals to obtain envelope signal data, the specific method is as follows:

[0010] Convolve the pulse signal x(t) with 1 / πt to obtain a signal H[x(t)] with a phase shift of -π / 2, where H[x(t)] is the Hilbert transform of x(t), and the analytical signal calculation method is as follows:

[0011] S(t) = x(t) + j H [x(t)] # (1)

[0012] where j is the imaginary unit.

[0013] The envelope E[x(t)] is the modulus of the analytic signal, and the calculation method is:

[0014]

[0015] Step 3: Perform bispectrum analysis on the filtered and cleaned pulse signal to obtain bispectrum data (bispectrum matrix). This step is independent of step 2 and can be performed simultaneously. The specific method is:

[0016] For the filtered and cleaned data {x(n), x(n+τ1),…x(n+τ k-1 )}, if its high-order cumulant c kx (τ1,τ2,…,τ k-1 ) satisfies: Then the K-order spectrum is defined as the (k-1)-dimensional discrete Fourier transform of the K-order cumulant, that is:

[0017]

[0018] where j is the imaginary unit; ω i satisfies the condition ω i ≤ π, (i = 1, 2, …, k-1) and ω1+ω2+…+ω k-1 ≤ π; Bispectrum is the third-order spectrum, which is the two-dimensional discrete Fourier transform of the third-order cumulant, defined as:

[0019]

[0020] Step 4: For signals from the same target individual, assign the same label to the filtered and cleaned original data, as well as the corresponding envelope data and bispectrum data, i.e. the multi-modal representation data of the same individual. The original pulse signal data and envelope signal data are considered as one-dimensional time series signal data, and the bispectrum data are considered as two-dimensional image-like data. The multi-modal representation data of different individuals form a data set for subsequent training, and the intelligence information is associated with the target individual. Each target individual is also referred to as a class.

[0021] The process of assigning labels refers to distinguishing the processed signals of different target individuals using labels for training of deep neural networks. The labels can be represented by non-repeating labels. The association of intelligence information with target individuals means that intelligence information is one-to-one corresponding to the original radiation source individual.

[0022] Step 5: Using the open set recognition method, all target individuals are sequentially determined as known classes or unknown classes. The known class refers to the class of data obtained before the current time node, and the unknown class refers to a new class that has never been seen before. For data of known classes, the collaborative reasoning method proposed in the application is used to complete the accurate identification of individuals; for data of unknown classes, the knowledge dynamic updating method proposed in the application is used to complete the real-time updating of the identification network and the knowledge graph.

[0023] Specifically, each signal is identified using the open set recognition technology to determine whether the signal belongs to a class that has appeared before. If so, the current identification network can effectively classify the signal, and the electromagnetic radiation source knowledge graph has knowledge including node and attribute information corresponding to the class. The collaborative reasoning method is used for identification; if not, the identification network and the electromagnetic radiation source knowledge graph need to be updated first to generate recognition ability for new classes; wherein, the knowledge dynamic updating is the basis of collaborative reasoning, and the collaborative reasoning is the technical core of individual identification.

[0024] Step 6: For data determined as known classes, the collaborative reasoning method is used to complete the individual identification task of the electromagnetic radiation source signal; the collaborative reasoning method is used to complete the accurate identification of individuals, and the method includes the following parts: multi-modal recognition network building; training of knowledge graph attribute encoder and structure encoder; intelligence information correlation query and auxiliary reasoning information generation; recognition result generation and integration based on feature layer fusion strategy and decision layer fusion strategy.

[0025] Specifically,

[0026] The multi-modal recognition network building process is an initialization task before the first use of the collaborative reasoning method, and this step is no longer executed after the building is completed; for data determined as known classes (including received original signal data, envelope data, bispectrum data, etc.), input them into the multi-modal recognition network to extract the multi-modal implicit features of the data (i.e. multi-modal representation of electromagnetic radiation source signal fingerprint features). The multi-modal recognition network (multi-modal feature extractor) includes a single-modal image encoder, a single-modal text encoder, and a multi-modal encoder for aligning and fusing image representation and text representation. All encoder structures are based on Transformer (Vision Transformer).

[0027] The training of the knowledge graph attribute encoder and the structure encoder is an initialization task before the first use of the collaborative reasoning method, and this step is no longer performed after the training is completed; the attribute encoder and the structure encoder of the electromagnetic radiation source knowledge graph are trained, so as to find the knowledge related to the to-be-identified signal from the knowledge graph and encode it into a feature vector for auxiliary reasoning. The training process of the attribute encoder is described as follows: the attribute encoder H a uses the attribute one-hot vector x i of the node v i as input, and outputs the attribute vector i of the node v a . H i is a multi-layer perception model with a nonlinear mapping, and its expression is as follows:

[0028]

[0029] wherein and are learnable parameters, and σ is a nonlinear activation function.

[0030] During the training of the attribute encoder, a data set containing K pairs of positive and negative node pairs is given, which is sampled from the radiation source graph , wherein

[0031] is the positive and negative node pair.

[0032]

[0033] In order to make nodes with the same attribute close to each other in the attribute vector space and nodes with different attributes far away from each other, the optimization loss function is as follows: wherein if the nodes have the same attribute (corresponding to the parameter information, geographical position, etc. of the electromagnetic radiation source), y i = 1, otherwise y i = 0; s(·) is a vector cosine distance, denotes the attribute vector. α(·,·) is a weight function, which measures the importance of the negative sample according to the shortest path of the two nodes of the negative sample in the graph, and the closer the distance, the greater the weight.

[0034] The structure encoder H s uses the one-hot encoding I i of the node sequence number v i as input, and maps v i to a vector space to obtain the structure vector of the node. y i = 1, otherwise y i = 0, then the loss function is:

[0035]

[0036] By optimizing the above loss function, the nodes with relationships on the graph are closer in the structure vector space, and the nodes without relationships are far away from each other.

[0037] The two encoders of attributes and structure coded and trained separately cannot share information, and the obtained vectors cannot well represent the relationship between attributes and graph structure. In order to better fuse the two information, the alignment loss function is as follows:

[0038]

[0039] Where, V represents the set of all nodes, s(·) is the cosine distance of the vector, represents the structure vector, represents the attribute vector; by jointly optimizing the three loss functions shown in formulas (6-8), the attribute vector and the structure vector belonging to a node can learn the information between each other, and the purpose of attribute and structure information fusion is achieved.

[0040] The information relevance query and auxiliary reasoning information generation are specifically as follows: the obtained electromagnetic radiation source signal radiation parameter, collection location and other information intelligence are sorted and input into the electromagnetic radiation source knowledge graph for relevance query. The relevance query is specifically described as follows: the attribute encoder is used to encode the attributes of the new individual v q , and the attribute vector is obtained. The attribute vector and the structure vector of the entity v p existing in the radiation source graph are linked for prediction. The link score calculation method of the new individual and the existing entity in the electromagnetic radiation source knowledge graph is as follows:

[0041]

[0042] Where, λ1 and λ2 are weight coefficients of the attribute-attribute similarity and the attribute-structure similarity scores of v p and v q . The more the attribute information of the electromagnetic radiation source knowledge graph, the greater the value of λ1; the more complex the electromagnetic radiation source knowledge graph, the greater the value of λ2.

[0043] The nodes with high relevance are used as query results to generate inference information to assist in the individual identification task. Specifically, generating this inference information involves selecting the top K entities with the highest scores as sources of auxiliary information to provide more generalized prior knowledge to the graph. For the top K representation vectors most likely associated with the radiation source, the attribute vectors of each entity are first... and structure vectors The link score P between the new individual and the corresponding entity is calculated by concatenating the elements. i =score(v p ,v q Using these as weights, we obtain the weighted vector representation:

[0044]

[0045] The specific steps for generating and integrating the recognition results based on the feature layer fusion strategy and the decision layer fusion strategy are as follows: The feature vectors obtained by the multimodal feature extractor and the auxiliary reasoning feature vectors generated by the electromagnetic radiation source knowledge graph are concatenated to obtain a multimodal feature matrix, or weighted fusion is performed using Dempster-Shaferevidence theory to obtain a better decision. After batch normalization, a classifier composed of several fully connected layers and a SoftMax activation function is used for classification to obtain the recognition result based on the feature layer fusion strategy. This recognition result serves as an intermediate result for collaborative reasoning.

[0046] The general form of the DS evidence theory is as follows: Let m1 and m2 be two independent mass functions, m(F) = 0, then m1 and m2 can be fused using the following combination rule:

[0047]

[0048] in, And N>0.

[0049] The batch normalization operation refers to transforming the data into a distribution with a mean of 0 and a standard deviation of 1, or a distribution within the range of [0, 1].

[0050] Since different modalities contain varying amounts of information, directly concatenating feature matrices may result in the inclusion of redundant information and random noise when multimodal features are superimposed. Therefore, based on DS evidence theory, a hyperparameter for model feature selection is added to control the weights of feature matrices between different modalities, prioritizing the learning of more discriminative feature weights to improve the significance of multimodal fusion features.

[0051] The feature vectors obtained by the multi-modal feature extractor and the auxiliary reasoning feature vectors generated by the electromagnetic radiation source knowledge graph are respectively input into multiple classifiers, so that classification probability results based on original data, classification probability results based on envelope data, classification probability results based on bispectrum data, and classification probability results based on radiation source knowledge graph auxiliary reasoning information can be obtained, and then according to the certainty (confidence) of each classification result, a recognition result based on a decision layer fusion strategy is obtained, which is an intermediate result of collaborative reasoning.

[0052] For the recognition result based on the feature layer fusion strategy and the recognition result based on the decision layer fusion strategy, according to the certainty (confidence) of the two recognition results, combined with the actual situation and expert opinions, the final model recognition result is obtained, so as to complete the collaborative reasoning.

[0053] Step 7: For data determined as unknown class, use the knowledge dynamic updating method to update the electromagnetic radiation source recognition network and the knowledge graph, and when the signal of the unknown class individual is intercepted again, use the collaborative reasoning method to complete the individual recognition task of the signal;

[0054] Specifically,

[0055] For data determined as unknown class (including received original signal data, envelope data, bispectrum data, etc.), the same individual label is given to the multi-modal data of the same individual, and the parameters of various encoder structures and other network parts in the multi-modal individual recognition network are updated in the class incremental learning mode. The goal is to make the updated recognition network have good recognition ability for the current new class while maintaining the recognition ability for the old class. Among them, the class incremental learning mode specifically requires: first, it can learn new class data that appears in any order at any time from dynamic data flow; second, it can effectively classify data of all classes that have appeared so far; third, only limited storage space and computing resources are used, or the storage space and computing resource requirements grow very slowly during the entire class incremental learning process. The specific method is not to save old class samples for incremental training or only to save a small part.

[0056] The acquired electromagnetic radiation source signal radiation parameters, collection location and other information are sorted out, and the electromagnetic radiation source knowledge graph is dynamically updated. The electromagnetic radiation source knowledge graph update generally refers to accurately adding a new entity node to the knowledge graph and connecting it with other entities under the condition that only part of the attributes of the new radiation source individual is known. Due to the lack of clear entity reference, the attribute information attached to the entity is needed to make up for it, that is, the link prediction of the new isolated node is used to realize it. Specifically, the new individual is encoded to obtain the entity vector of the new individual in the knowledge graph, the vector is linked to the vector of the existing individual in the graph for link prediction calculation, a relationship is added between the node with the highest link score and the new node, and the record is submitted to the electromagnetic radiation source knowledge graph for review by a domain expert, and the change is submitted to the electromagnetic radiation source knowledge graph after the review is passed, and the updating process of the knowledge graph is completed.

[0057] The updated multi-modal recognition network is used to re-calculate the prototypes of all known categories, and the prototypes of each category are used as attributes to update or add to the corresponding entity node in the electromagnetic radiation source knowledge graph, further improving the updating of the electromagnetic radiation source knowledge graph, and completing the dynamic updating of the knowledge.

[0058] The collaborative reasoning method for electromagnetic radiation source individual identification in the application includes the following processes:

[0059] Before using the collaborative reasoning method for the first time, the initialization task needs to be completed, that is, the multi-modal recognition network is built and the knowledge graph attribute encoder and structure encoder are trained, and the two initialization tasks are independent of each other, and the subsequent step is not executed;

[0060] In the case of not using the collaborative reasoning method for the first time, when receiving data judged as a known category, firstly, on the one hand, the information of the electromagnetic radiation source signal is input into the electromagnetic radiation source knowledge graph for correlation query, and the obtained auxiliary reasoning information is encoded into a feature vector, and on the other hand, the multi-modal signal data is input into the multi-modal recognition network for feature extraction to obtain the data implicit feature vector; secondly, on the one hand, the auxiliary reasoning information feature vector and the data implicit feature vector are input into independent classifiers for decision and two classification results are obtained, and a multi-modal feature fusion method based on the decision layer is used to determine the final classification result of this part according to the certainty of the classifier, and on the other hand, the auxiliary reasoning information feature vector and the data implicit feature vector are fused based on the feature layer, and the fused feature vector is input into an independent classifier to obtain the final classification result of this part; finally, according to the certainty of the classification results obtained by the two multi-modal feature fusion strategies, the higher one is taken as the final classification result, and the collaborative reasoning task is completed.

[0061] The dynamic updating method for the electromagnetic radiation source identification network and knowledge graph in the application comprises the following processes:

[0062] When receiving data determined as an unknown class, on one hand, relevant information corresponding to the electromagnetic radiation source signal is collected and structured, the information is converted into a triple form of "entity-relation-entity" and submitted to experts for review, the structured information passing the review is used to update the electromagnetic radiation source knowledge graph, so that the unknown class data information is integrated into the electromagnetic radiation source knowledge graph; on the other hand, the unknown class data is assigned an individual label, the individual identification network is updated and trained in a class incremental learning mode, so that the identification network has the identification ability for the class signal, then the updated identification network is used to recalculate or update the class prototype of all classes, and the class prototype is integrated or updated into the electromagnetic radiation source knowledge graph as attribute information.

[0063] After the above updating process is completed, the current received data determined as an unknown class is changed to a known class, and when the signals of the classes are intercepted again, the signals are classified and identified according to the cooperative reasoning method.

[0064] The application has the following beneficial effects:

[0065] For the individual identification task in the field of electromagnetic radiation sources, the application designs an electromagnetic radiation source individual identification method based on a neural network and knowledge graph dual-channel system, including cooperative reasoning and knowledge dynamic updating of the electromagnetic radiation source identification network and knowledge graph. Starting from the multi-modal preprocessing of the electromagnetic radiation source signal, the multi-modal fusion features with rich information are obtained by using the partial intersection and complementarity between the information of different modalities. Starting from the cooperative reasoning of the identification network and the knowledge graph, the reasoning information of the knowledge graph and the neural network is systematically combined through knowledge graph auxiliary information generation and multi-modal feature fusion, so that the identification accuracy limit of using a single identification network as a classification model is broken through. Starting from the dynamic updating of the identification network and the knowledge graph, when a new sample of an unknown class is identified, the identification network is trained in a class incremental learning mode, and the electromagnetic radiation source knowledge graph is updated in real time according to the information, so that the identification network and the knowledge in the knowledge graph used for reasoning are kept up-to-date, and the accuracy and reliability of the cooperative reasoning are ensured.

[0066] The electromagnetic radiation source individual identification method based on the neural network and knowledge graph dual-channel system designed by the application cooperates the identification network and the knowledge graph to complete the identification task, and can update the two in real time to ensure the reliability and accuracy of the identification. The application has enlightening significance for the electromagnetic radiation source individual identification task and individual identification tasks in other fields. BRIEF DESCRIPTION OF DRAWINGS

[0067] Figure 1 is a flow chart of the electromagnetic radiation source individual identification method based on the neural network and knowledge graph dual-channel system proposed by the present application.

[0068] Figure 2 is a radar radiation source envelope signal example diagram extracted by the present application.

[0069] Figure 3 is a radar radiation source bispectrum signal example diagram extracted by the present application.

[0070] Figure 4 is an electromagnetic radiation source knowledge graph instance diagram proposed by the present application, containing several entity types and parameter information of military ports, aircraft carriers, radiation source models, and radiation source individuals.

[0071] Figure 5 is a schematic diagram of the attribute encoder and structure encoder training process of the knowledge graph proposed by the present application. DETAILED DESCRIPTION

[0072] The application will be further described in detail in combination with the following specific embodiments and drawings. The process, condition, experimental method, etc. for implementing the present application are the general knowledge and common sense in the art, and the present application does not have special restrictions.

[0073] The present application proposes an electromagnetic radiation source individual identification method based on a neural network and knowledge graph dual-channel system, which is suitable for individual identification tasks in the field of electromagnetic radiation sources. The method consists of two parts: a collaborative reasoning method based on a deep neural network and a knowledge graph, and a knowledge dynamic updating method. For the signal sample to be identified, the open set identification technology is used to determine the known class / unknown class. If it belongs to the known class, the collaborative reasoning method is used for class prediction; if it belongs to the unknown class, the knowledge of the identification network and the knowledge graph is updated. The collaborative reasoning method specifically includes: querying the radiation source knowledge graph according to the related parameter information in the data, generating auxiliary reasoning information and encoding it into a feature vector; at the same time, the neural network is used to extract features from the multi-modal signal data, and finally the extracted features are used for classification to generate a reasoning result. The knowledge dynamic updating method specifically includes: first, using link prediction of new nodes as the main technical means to complete the knowledge update of out-of-source information of new intelligence data, and second, using incremental update of the identification network as the main technical means to complete the knowledge update of new class signals. Knowledge dynamic updating is the basis of collaborative reasoning, which guarantees the accuracy and credibility of collaborative reasoning; collaborative reasoning is the technical core of realizing precise identification of electromagnetic radiation sources.

[0074] As Figure 1As shown, a kind of electromagnetic radiation source individual identification method based on neural network and knowledge graph double channel system, specifically includes the following steps:

[0075] Step 1: using electromagnetic radiation source signal acquisition equipment to obtain initial signal, and the corresponding signal parameters, geographical position and other intelligence. The initial signal is cut according to pulse period, and several pulse signals are obtained. The pulse signals with signal-to-noise ratio lower than the preset threshold are filtered out, and the threshold is generally set to 3;

[0076] Step 2: envelope analysis is carried out on the filtered pulse signal using Hilbert transform method, and envelope data is obtained;

[0077] Step 3: double spectrum analysis is carried out on the filtered pulse signal, and double spectrum data is obtained;

[0078] Step 4: for the signals from the same target individual, the original data and the corresponding envelope data and double spectrum data are assigned with the same label;

[0079] Step 5: using open set recognition method, the known class / unknown class of each category of data is determined.

[0080] Step 6: for the data determined as known class, the individual identification task of the electromagnetic radiation source signal is completed using the collaborative reasoning method proposed in the application.

[0081] Step 7: for the data determined as unknown class, the electromagnetic radiation source recognition network and knowledge graph are updated using the knowledge dynamic updating method proposed in the application. When the signal of the unknown class individual is intercepted again, the individual identification task of the signal is completed using the collaborative reasoning method.

[0082] The above is only a simple statement of the flowchart.

[0083] Embodiment

[0084] The electromagnetic radiation source individual identification method based on neural network and knowledge graph double channel system in the embodiment of the application is described in detail as follows:

[0085] Step 1: several electromagnetic radiation source signals from two individuals (category a and category b) are collected, the signals are cut according to pulse, and the pulse samples with signal-to-noise ratio lower than 3 are filtered out. Each pulse sample is assigned with an individual label according to the individual it comes from.

[0086] Step 2: the Hilbert transform method is used to obtain the envelope data of the remaining pulse signals after filtering as Figure 2The envelope data is shown in the following steps: the pulse signal x(t) is convolved with 1 / πt to obtain a signal H[x(t)] with a phase shift of-π / 2, where H[x(t)] is the Hilbert transform of x(t), and the analytical signal is calculated as follows:

[0087] S(t)=x(t)+j·H[x(t)]#(1)

[0088] where j is the imaginary unit.

[0089] The envelope E[x(t)] is the modulus of the analytical signal, and the calculation method is as follows:

[0090]

[0091] The envelope data and the original data have the same individual label.

[0092] Step 3: Perform bispectrum analysis on the filtered remaining pulse signal to obtain bispectrum data as shown in the following steps: Figure 3

[0093] For data {x(n), x(n+τ1),…x(n+τ k-1 )}, if its high-order cumulant c kx (τ1,τ2,…,τ k-1 ) satisfies: then the K-order spectrum is defined as the (k-1)-dimensional discrete Fourier transform of the K-order cumulant, that is:

[0094]

[0095] where j is the imaginary unit; ω i satisfies the condition ω i ≤π,(i=1,2,…,k-1) and ω1+ω2+…+ω k-1 ≤π; the bispectrum is the third-order spectrum, that is, the two-dimensional discrete Fourier transform of the third-order cumulant, and is defined as:

[0096]

[0097] The bispectrum data and the original data have the same individual label.

[0098] Step 4: Use the open set recognition method to sequentially perform known class / unknown class judgment on the data of class a and class b, and the judgment result is that class a is a known class and class b is an unknown class. There are currently 10 kinds of known classes (excluding class a).

[0099] Step 5: For the data of class a (including the original signal data, envelope data, bispectrum data, etc. of class a), input it into the multi-modal recognition network to extract the multi-modal implicit features of the data.​

[0100] Step 6: Train the attribute encoder and structure encoder of the electromagnetic radiation source knowledge graph. Specifically, the attribute encoder H a uses the attribute one-hot vector x i of node v i as input, and outputs the attribute vector of node v i . H a is a multi-layer perceptron model with nonlinear mapping, and its expression is as follows:

[0101]

[0102] where and are learnable parameters, and σ is a nonlinear activation function.

[0103] During the training of the encoder, a data set containing K pairs of positive and negative node pairs is given, which is sampled from the radiation source graph The optimization loss function is as follows:

[0104]

[0105] where y i = 1 if node i and s have the same attribute, otherwise y i = 0; s(·) is the vector cosine distance. α(·,·) is a weight function that measures the importance of negative samples according to the shortest path of the two nodes in the graph. The closer the distance, the greater the weight.

[0106] The structure encoder H i uses the one-hot encoding I i of node sequence number v i as input, and maps v i to a vector space to obtain the structure vector of the node Change the loss function in the attribute encoder, so that when node and have a direct edge in the graph, y a = 1, otherwise y s = 0, and the loss function is as follows:

[0107]

[0108] In order to better integrate the information of H q and H p , the alignment loss function is as follows:

[0109]

[0110] The encoder has been trained by jointly optimizing the three loss functions shown in formula (6-8).

[0111] Step 7: Organize the information such as electromagnetic radiation source signal radiation parameters and collection location for category a, and input it into the electromagnetic radiation source knowledge graph for relevance query. Specifically, use an attribute encoder to perform a relevance query on the new individual v. q The attributes are encoded to obtain an attribute vector. Compare it with entity v in the radiation source map p attribute vector and structure vector Link prediction is performed. The link score between the new individual and existing entities in the electromagnetic radiation source knowledge graph is calculated as follows:

[0112]

[0113] Where λ1 and λ2 are v p With v q Weights of attribute-attribute similarity and attribute-structure similarity scores.

[0114] Step 8: Use the nodes with high relevance found in the query as the query results to generate inference information to assist in the individual identification task. Specifically, select the top 3 entities with the highest scores as the source of auxiliary information. For the top 3 representation vectors that are most likely to be associated with the radiation source, first convert the attribute vectors of each entity... and structure vectors The link score P between the new individual and the corresponding entity is calculated by concatenating the elements. i As the weights, we obtain the weighted vector representation:

[0115]

[0116] Step 9: Concatenate the feature vectors obtained from the multimodal feature extractor in Step 5 and the auxiliary reasoning feature vectors generated from the electromagnetic radiation source knowledge graph in Step 8 to obtain a multimodal feature matrix. Alternatively, use DS evidence theory for weighted fusion. After batch normalization, use a classifier composed of several fully connected layers and a SoftMax activation function to classify the sample a. i The recognition result p(a) based on the feature layer fusion strategy i Given the probability distribution p(a) = {0.03, 0.01, 0.01, 0.57, 0.15, 0.06, 0.12, 0.02, 0.02, 0.1}, the probability distribution p(a) = {0.03, 0.01, 0.01, 0.57, 0.15, 0.06, 0.12, 0.02, 0 i It can be concluded that the signal is most likely category 4 among the known classes.

[0117] Step 10: The feature vectors obtained from the multi-modal feature extractor in step 5 and the auxiliary reasoning feature vectors generated from the electromagnetic radiation source knowledge graph in step 8 are respectively input into multiple classifiers to obtain classification probability results based on original data, classification probability results based on envelope data, classification probability results based on bispectrum data, and classification probability results based on radiation source knowledge graph auxiliary reasoning information. Then, according to the certainty (confidence) of each classification result, the voting mechanism of D-S evidence theory is used to obtain the recognition result q(a i ) of sample a i based on the decision layer fusion strategy, which is {0.03, 0.01, 0.01, 0.73, 0.05, 0.06, 0.06, 0.02, 0.02, 0.1}. According to the probability distribution q(a i ), it is known that the signal is most likely to be class 4 in the known classes.

[0118] Step 11: The probability distribution of p(a i ) and q(a i ) both indicate that sample a i is most likely to belong to class 4 in the known classes, and the reasoning results of the two methods are consistent. Therefore, the output result of the final collaborative reasoning is that sample a i belongs to class 4, and the certainty of the recognition result q(a i ) based on the decision layer fusion strategy is higher. This round of collaborative reasoning is completed.

[0119] Step 12: Use the data of class b (including the received original signal data, envelope data, bispectrum data, etc.) to train the multi-modal individual recognition network in the class incremental learning mode, and update the parameters of various encoder structures and other network parts in the network. The trained individual recognition network can produce good prediction ability for the known 10 old classes and class b.

[0120] Step 13: After sorting the signal radiation parameters, collection location information, etc. of class b, the electromagnetic radiation source knowledge graph is dynamically updated. Specifically: encode class b to obtain the entity vector of the new individual in the knowledge graph, calculate the link prediction between the vector and the vector of the existing individual in the graph, add the relationship between the node with the highest link score and the new node, and submit the change to the electromagnetic radiation source knowledge graph after the record is reviewed by experts. The update process of the knowledge graph is completed.

[0121] Step 14: Recalculate the class prototypes of all known classes using the updated multi-modal recognition network in step 12, and update or add the class prototypes as attributes to the corresponding entity nodes in the electromagnetic radiation source knowledge graph, further improving the update of the electromagnetic radiation source knowledge graph. From the current time, class b becomes the 11th known class, and the current knowledge update is completed.

[0122] The protection scope of the present application is not limited to the above embodiments. Changes and advantages that can be thought of by those skilled in the art without departing from the spirit and scope of the present application are included in the present application, and are protected by the appended claims.

Claims

1. A method for individual identification of electromagnetic radiation sources based on a dual-channel system of neural networks and knowledge graphs, characterized in that, The method includes: Step (1): Use an electromagnetic radiation source signal acquisition device to acquire the initial electromagnetic radiation source signal and corresponding intelligence information of the target individual, and filter and clean the acquired electromagnetic radiation source signal data; Step (2): Perform envelope analysis on the filtered pulse signal from step (1) to obtain the corresponding envelope data; Step (3): Perform bispectral analysis on the filtered pulse signal in step (1) to obtain the corresponding bispectral data; Step (4): For signals from the same target individual, assign the same label to the original signal data after filtering and cleaning, as well as the corresponding envelope data and bispectral data, to obtain a multimodal electromagnetic radiation source signal dataset for subsequent training. At the same time, associate the intelligence information with the target individual; each target individual is called a category. Step (5): Use the open set recognition method to determine the type of all the target individuals; Step (6): For data that is determined to be of a known class, use collaborative reasoning to complete the individual identification task of the electromagnetic radiation source signal; In step (6), the collaborative reasoning method is used to complete accurate individual identification. The method includes the following parts: construction of a multimodal identification network; training of knowledge graph attribute encoder and structural encoder; intelligence information relevance query and auxiliary reasoning information generation; generation and integration of identification results based on feature layer fusion strategy and decision layer fusion strategy; The multimodal recognition network construction process is an initialization task before the first use of the collaborative reasoning method. This step is not executed after the network is built. The specific process is as follows: For data identified as belonging to a known class, it is input into a multimodal recognition network to extract the multimodal implicit features of the data. The multimodal recognition network includes a unimodal image encoder, a unimodal text encoder, and a multimodal encoder to align and fuse image representations and text representations. All encoder structures are based on Transformer. Step (7): For data that is determined to be of the unknown class, the knowledge dynamic update method is used to update the electromagnetic radiation source identification network and knowledge graph. When the signal of the unknown individual is intercepted again, the collaborative reasoning method is used to complete the individual identification task of the signal.

2. The method for individual identification of electromagnetic radiation sources based on a dual-channel system of neural networks and knowledge graphs according to claim 1, characterized in that, In step (1), the initial electromagnetic radiation source signal is periodic, exhibiting a repetitive process: "the voltage amplitude suddenly increases within a short duration, and then quickly returns to its initial value"; wherein, the signal formed after each cycle is called a pulse; the intelligence information includes signal parameters and geographical location.

3. The method for individual identification of electromagnetic radiation sources based on a dual-channel system of neural networks and knowledge graphs according to claim 1, characterized in that, In step (1), the data filtering and cleaning operation includes trimming the initial electromagnetic radiation source signal according to a single pulse period and filtering pulse signals with a signal-to-noise ratio lower than a preset threshold. The preset threshold is determined according to the magnitude of interference in the signal acquisition process. The two are negatively correlated, that is, the greater the interference, the lower the threshold setting, and the smaller the interference, the higher the threshold setting.

4. The method for individual identification of electromagnetic radiation sources based on a dual-channel system of neural networks and knowledge graphs according to claim 1, characterized in that, In step (2), the specific method for envelope analysis is as follows: Convolving the pulse signal x(t) with 1 / πt yields a signal H[x(t)] with a phase shift of -π / 2, where H[x(t)] is the Hilbert transform of x(t). The analytic signal is calculated as follows: S(t) = x(t) + j·H[x(t)], Where j is the imaginary unit; The envelope E[x(t)] is the magnitude of the analytic signal, and its calculation method is as follows:

5. The method for individual identification of electromagnetic radiation sources based on a dual-channel system of neural networks and knowledge graphs according to claim 1, characterized in that, In step (3), the specific method for bispectral analysis is as follows: For the filtered and cleaned signal data {x(n),x(n+τ1),…x(n+τ1),…x(n+τ1)} in step (1) k-1 If its higher-order cumulant c kx (τ1,τ2,…,τ k-1 )satisfy The K-order spectrum is defined as the (k-1)-dimensional discrete Fourier transform of the K-order cumulants, and the calculation formula is: Where j is the imaginary unit; ω i Condition satisfied: ω i ≤π, i=1,2,…,k-1 and ω1+ω2+…+ω k-1 ≤π; Bispectrum is the third-order spectrum, which is the two-dimensional discrete Fourier transform of the third-order cumulant. The calculation formula is:

6. The method for individual identification of electromagnetic radiation sources based on a dual-channel system of neural networks and knowledge graphs according to claim 1, characterized in that, In step (4), the process of assigning labels refers to using labels to distinguish the processed signals of different target individuals for training of deep neural networks; the process of associating intelligence information with target individuals refers to matching intelligence with its corresponding original radiation source individuals one by one.

7. The method for individual identification of electromagnetic radiation sources based on a dual-channel system of neural networks and knowledge graphs according to claim 1, characterized in that, In step (5), the type includes known class or unknown class, and the step of determining known class or unknown class using the open set recognition method is as follows: Each signal is identified using open set recognition technology. It is then determined whether the signal belongs to a category that has already appeared. If so, the current recognition network can effectively classify the signal, and the electromagnetic radiation source knowledge graph contains knowledge of the category, including node and attribute information. Collaborative reasoning is then used for recognition. If not, the recognition network and the electromagnetic radiation source knowledge graph need to be updated first to enable the recognition of new categories. Dynamic knowledge updating is the foundation of collaborative reasoning, and collaborative reasoning is the core technology for achieving individual recognition.

8. The method for individual identification of electromagnetic radiation sources based on a dual-channel system of neural networks and knowledge graphs according to claim 7, characterized in that, The collaborative reasoning method for identifying individual electromagnetic radiation sources includes the following process: Before using the collaborative reasoning method for the first time, an initialization task needs to be completed, namely the construction of the multimodal recognition network and the training of the knowledge graph attribute encoder and structure encoder. This step will not be performed again, as these two initialization tasks are independent of each other. In cases where collaborative reasoning is not used for the first time, when data classified as a known class is received, the process is as follows: First, relevant information about the electromagnetic radiation source signal is input into an electromagnetic radiation source knowledge graph for relevance querying, and the obtained auxiliary reasoning information is encoded into feature vectors. Simultaneously, the multimodal signal data is input into a multimodal recognition network for feature extraction, obtaining the implicit feature vectors of the data. Second, the obtained auxiliary reasoning information feature vectors and the implicit feature vectors are input into independent classifiers for decision-making, yielding two classification results. A multimodal feature fusion method based on the decision layer is then used, determining the final classification result for this part based on the classifier's certainty. Alternatively, the obtained auxiliary reasoning information feature vectors and the implicit feature vectors are fused using a feature layer, and the fused feature vector is input into an independent classifier to obtain the final classification result for this part. Finally, the classification result with the higher certainty obtained from the two multimodal feature fusion strategies is selected as the final classification result, completing the collaborative reasoning task.

9. The method for individual identification of electromagnetic radiation sources based on a dual-channel system of neural networks and knowledge graphs according to claim 7, characterized in that, The dynamic update method for the electromagnetic radiation source identification network and knowledge graph includes the following steps: When data classified as unknown is received, on the one hand, relevant intelligence information corresponding to the electromagnetic radiation source signal is collected and structured, and the intelligence information is transformed into a "entity-relationship-entity" triple and submitted to experts for review. The approved structured intelligence information is used to update the electromagnetic radiation source knowledge graph, so that the unknown data intelligence is integrated into the electromagnetic radiation source knowledge graph. On the other hand, these unknown data are assigned individual labels, and the individual identification network is updated and trained in a class incremental learning mode, so that the identification network can identify signals of this category. Then, the updated identification network is used to recalculate or update the class prototypes of all current categories, and the class prototypes are integrated or updated into the electromagnetic radiation source knowledge graph as attribute information. Once the above update process is complete, the data currently received that was judged as an unknown class will be changed to a known class. When signals of these classes are intercepted again, they will be classified and identified according to the collaborative reasoning method.

10. The method for individual identification of electromagnetic radiation sources based on a dual-channel system of neural networks and knowledge graphs according to claim 1, characterized in that, The training of the knowledge graph attribute encoder and structure encoder is an initialization task before the first use of the collaborative reasoning method. This step is not performed after the training is completed. The specific process is as follows: The training process of the attribute encoder is as follows: Attribute encoder H a Use node v i The properties of the one-hot vector x i As input, output node v i attribute vector Among them, H a It is a multilayer perceptron model with nonlinear mapping, and its expression is: in, and Here, σ is a learnable parameter, and σ is a non-linear activation function. During the training of the attribute encoder, a dataset containing K pairs of positive and negative nodes is sampled from the radiation source map. The positive and negative node pairs mentioned above; To ensure that nodes with the same attributes are close to each other in the attribute vector space, and nodes with different attributes are far apart, the loss function is defined as: Among them, if node and If there is the same intelligence information corresponding to the electromagnetic radiation source, then y i =1, otherwise y i =0; s(·) is the cosine distance between vectors. Represents an attribute vector; α(·,·) is a weighting function that measures the importance of a negative sample based on the shortest path between two nodes in the graph, with a higher weight for closer nodes. Structure encoder H s Use node number v i One-hot encoding I i As input, v i Mapping to a vector space yields the structure vector of that node. Change the loss function in the attribute encoder so that when the node and When there are directly connected edges in the graph, y i =1, otherwise y i If = 0, then the loss function is: By optimizing the above loss function, nodes that are related in the graph are closer in the structure vector space, while nodes that are not related are further apart. Attribute encoders and structural encoders, which are encoded separately and trained in isolation, cannot share information. The alignment loss function fuses the information from both, as expressed below: Where V represents the set of all nodes, and s(·) is the cosine distance between vectors. Represents a structure vector. The attribute vector is represented by the above three loss functions, and the fusion of attribute and structural information is achieved by jointly optimizing them.

11. The method for individual identification of electromagnetic radiation sources based on a dual-channel system of neural networks and knowledge graphs according to claim 1, characterized in that, The specific process for intelligence information relevance query and auxiliary reasoning information generation is as follows: The acquired electromagnetic radiation source signals, including radiation parameters and collection location information, are processed and input into an electromagnetic radiation source knowledge graph for relevance queries. This relevance query involves using an attribute encoder to perform a relevance lookup on a new individual (v). q The attributes are encoded to obtain an attribute vector. Compare it with entity v in the radiation source map p attribute vector and structure vector Link prediction is performed; the link score between the new individual and existing entities in the electromagnetic radiation source knowledge graph is calculated as follows: Where λ1 and λ2 are v p With v q The weighting coefficients for attribute-attribute similarity and attribute-structure similarity scores are as follows: the more attribute information the electromagnetic radiation source knowledge graph has, the larger the value of λ1; the more complex the electromagnetic radiation source knowledge graph, the larger the value of λ2. The highly relevant nodes are used as query results to generate inference information to assist in the individual identification task. Specifically, the top K highest-scoring entity query results are selected, which are the sources of auxiliary information. For the top K most likely associated representation vectors with the radiation source, the attribute vectors of each entity are first... and structure vectors The link score P between the new individual and the corresponding entity is calculated by concatenating the elements. i =score(v p ,v q Using these values ​​as weights, we obtain the weighted vector representation:

12. The method for individual identification of electromagnetic radiation sources based on a dual-channel system of neural networks and knowledge graphs according to claim 1, characterized in that, The specific steps for generating and integrating the recognition results based on the feature layer fusion strategy and the decision layer fusion strategy are as follows: First, the feature vectors obtained from the multimodal recognition network and the auxiliary reasoning feature vectors generated from the electromagnetic radiation source knowledge graph are concatenated to obtain a multimodal feature matrix, or weighted fusion is performed using DS evidence theory to obtain a better decision. After batch normalization, a classifier consisting of several fully connected layers and a SoftMax activation function is used for classification to obtain a recognition result based on a feature layer fusion strategy. This recognition result serves as an intermediate result for collaborative inference. Secondly, the feature vectors obtained from the multimodal recognition network and the auxiliary reasoning feature vectors generated from the electromagnetic radiation source knowledge graph are respectively input into multiple classifiers to obtain classification probability results based on the original data, classification probability results based on the envelope data, classification probability results based on the bispectral data, and classification probability results based on the auxiliary reasoning information of the radiation source knowledge graph. Then, based on the certainty of each classification result, the recognition result based on the decision layer fusion strategy is obtained, and this recognition result serves as an intermediate result of collaborative reasoning. Finally, based on the recognition results of the feature layer fusion strategy and the recognition results of the decision layer fusion strategy, the final model recognition result is obtained by combining the certainty of the two with the actual situation and expert opinions, thereby completing collaborative reasoning.

13. The method for individual identification of electromagnetic radiation sources based on a dual-channel system of neural networks and knowledge graphs according to claim 1, characterized in that, In step (7), the knowledge dynamic update method specifically involves the following process: First, for data classified as unknown, the same individual label is assigned to multiple modal data of the same individual. The parameters of various encoder structures and other network parts in the multimodal individual recognition network are updated in the mode of class incremental learning, so that the updated recognition network can maintain the ability to recognize the old category while also having a good recognition ability for the current new category. Secondly, after organizing the acquired electromagnetic radiation source signals, including radiation parameters and collection location information, the electromagnetic radiation source knowledge graph is dynamically updated. That is, even with only partial attributes of a new radiation source individual, the new entity node is accurately added to the knowledge graph and connected with other entities. The dynamic update is achieved through link prediction of new isolated nodes. Specifically, the new individual is encoded to obtain the entity vector of the new individual in the knowledge graph. The vector is then used to perform link prediction calculation with the vectors of existing individuals in the graph. A relationship is added between the node with the highest link score and the new node. This record is then submitted to domain experts for review. After the review is approved, the change is submitted to the electromagnetic radiation source knowledge graph, thus completing the knowledge graph update process. Finally, the updated multimodal recognition network is used to recalculate the prototypes of all known categories, and the prototypes are used as attributes to update or add to the corresponding entity nodes in the electromagnetic radiation source knowledge graph, thereby improving the update of the electromagnetic radiation source knowledge graph and completing the dynamic update of knowledge.

14. The method for individual identification of electromagnetic radiation sources based on a dual-channel system of neural networks and knowledge graphs according to claim 13, characterized in that, The specific requirements of the class incremental learning mode are as follows: First, it is able to learn new class data that appears at any time and in any order from a dynamic data stream; second, it is able to effectively classify all classes of data that have appeared so far; third, it uses only limited storage space and computing resources throughout the entire class incremental learning process, or the storage space and computing resource requirements grow very slowly, that is, it is achieved by not saving samples of old classes for incremental training or saving only a very small portion.

Images