Electromagnetic feature mapping and iterative reconstruction method and device based on comparative learning

Abstract

The invention relates to the technical field of electromagnetic inverse scattering, and provides an electromagnetic feature mapping and iterative reconstruction method and device based on comparative learning. The electromagnetic characteristic mapping method comprises the following steps: acquiring scattered field data and electromagnetic characteristic data of a set scatterer; converting the scattered field data into graph structure data and generating a graph data set, converting the electromagnetic characteristic data into image structure data and generating an image data set, generating a positive sample pair and a negative sample pair according to the graph data set and the image data set, and utilizing the positive sample pair and the negative sample pair to obtain a positive sample pair and a negative sample pair; and training the double-flow network model comprising the image encoder and the image encoder to obtain a feature mapping network. The invention provides a new electromagnetic reconstruction thought, at least partially solves the problems of high electromagnetic reconstruction difficulty and poor effect in the prior art, does not need to construct a complex forward model, and can reconstruct the electromagnetic characteristic data of various types of scatterers with a better effect.

Description

Technical Field

[0001] The present invention relates to the field of electromagnetic inverse scattering technology, and in particular to an electromagnetic feature mapping and iterative reconstruction method and device based on contrast learning. Background Art

[0002] Electromagnetic inverse scattering is to reconstruct the electromagnetic characteristics of unknown scatterers in the detection area by using scattering field data and forward models. Electromagnetic inverse scattering technology has the advantages of non-contact and non-destructive, which makes it have a wide range of application prospects, such as military detection, medical imaging, geological exploration, etc.

[0003] However, the forward model needs to be constructed based on known laws and conditions, which is difficult and directly affects the effect of electromagnetic reconstruction. Especially for some special cavities (such as cavities filled with overflow-type media), the mathematical model of the forward model will be more complicated. Coupled with the original highly nonlinear and pathological characteristics of the inverse scattering problem, the inverse scattering problem of special cavities is difficult to be effectively solved. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides an electromagnetic feature mapping and iterative reconstruction method and device based on contrastive learning, which at least partially solves the problem of great difficulty and poor effect of electromagnetic reconstruction in the prior art.

[0005] In a first aspect, an embodiment of the present invention provides an electromagnetic feature mapping method based on contrastive learning, comprising: Obtaining scattered field data and electromagnetic characteristic data of a set scatterer; Converting the scattered field data into graph structure data to generate a graph data set, and converting the electromagnetic characteristic data into image structure data to generate an image data set, wherein the graphs in the graph data set correspond one-to-one to the images in the image data set; Generate positive sample pairs and negative sample pairs according to the graph dataset and the image dataset, wherein the positive sample pairs include corresponding graphs and images, and the negative sample pairs include non-corresponding graphs and images; The positive sample pairs and the negative sample pairs are used to train a two-stream network model including a graph encoder and an image encoder to obtain a feature mapping network.

[0006] Optionally, the using the positive sample pairs and the negative sample pairs to train a two-stream network model including a graph encoder and an image encoder to obtain a feature mapping network includes: Inputting the graph and the image in the positive sample pair or the negative sample pair into a graph encoder and an image encoder respectively to obtain a graph feature vector and an image feature vector; Calculating a loss value of a contrast loss function based on the graph feature vector and the image feature vector; The parameters of the two-stream network model are updated according to the loss value until a convergence condition is met to obtain the feature mapping network.

[0007] Optionally, the calculating the loss value of the contrast loss function based on the graph feature vector and the image feature vector includes: Generate a similarity matrix based on the graph feature vector and the image feature vector; Calculating the cross entropy loss in the row direction and the cross entropy loss in the column direction of the similarity matrix; The cross entropy loss in the row direction and the cross entropy loss in the column direction are averaged to obtain the loss value.

[0008] Optionally, the electromagnetic characteristic data is a dielectric constant contrast matrix.

[0009] In a second aspect, an embodiment of the present invention provides an electromagnetic iterative reconstruction method, comprising: Obtain scattering field data of the scatterer to be measured and convert it into graph structure data; Inputting the graph structure data and the image structure data representing the electromagnetic characteristic data into a feature mapping network respectively to obtain a graph data feature vector and an image data feature vector; Calculate the loss value of the reconstruction loss function according to the graph data feature vector and the image data feature vector; Iteratively update the image structure data according to the loss value to obtain electromagnetic characteristic data of the scatterer to be measured; Wherein, the feature mapping network is obtained according to the method described in the first aspect.

[0010] Optionally, the calculating the loss value of the reconstruction loss function according to the graph data feature vector and the image data feature vector comprises: Inputting the graph data feature vector into a similarity prediction network to obtain a similarity threshold; Obtaining an estimated feature vector according to the image data feature vector and the similarity threshold; The loss value of the reconstruction loss function is calculated according to the graph data feature vector and the estimated feature vector.

[0011] Optionally, the reconstruction loss function is: Among them, g represents the feature vector of graph data, g * Represents the estimated feature vector, cos_sim() represents the cosine similarity, i represents the dimension, and is the set weight value.

[0012] Optionally, the similarity prediction network is trained based on the similarity between the graph and the image in the sample pair.

[0013] In a third aspect, an embodiment of the present invention further provides an electronic device, the electronic device comprising: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the electromagnetic feature mapping method according to the first aspect or the electromagnetic iterative reconstruction method according to the second aspect.

[0014] The method and device proposed in the embodiment of the present invention converts the scattered field data and electromagnetic characteristic data of the scatterer into data sets of graph structure and image structure respectively, generates positive and negative sample pairs, and constructs a two-stream network model based on the idea of ​​contrastive learning to extract the features of the scattered field data and electromagnetic characteristic data respectively. By training the matching relationship between the two by contrastive learning on a large amount of data, a feature mapping network for electromagnetic feature mapping is obtained, and then the feature mapping network can be used to complete the electromagnetic feature mapping, and the image features initialized based on the scattering field data are continuously iterated to finally realize electromagnetic reconstruction. The scheme of the embodiment of the present invention provides a new electromagnetic reconstruction idea, which does not require the construction of a complex forward model, is suitable for electromagnetic reconstruction of various types of scatterers, and can obtain better reconstruction effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0016] Figure 1 A schematic diagram of a flow chart of an electromagnetic feature mapping method based on contrastive learning provided in an embodiment of the present invention; Figure 2 A schematic diagram of a flow chart of an electromagnetic iterative reconstruction method provided by an embodiment of the present invention; Figure 3 A schematic diagram of a method for generating graph structure data in a method provided in an embodiment of the present invention; Figure 4 A schematic diagram of the architecture of a feature mapping network provided by an embodiment of the present invention; Figure 5 A schematic diagram of a similarity matrix generation method in a method provided in an embodiment of the present invention; Figure 6A schematic diagram of an electromagnetic iterative reconstruction process provided by an embodiment of the present invention; Figure 7 A schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0017] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0018] The following describes the embodiments of the present invention through specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. The present invention is also implemented or applied through other different specific embodiments, and the details in this specification are also based on different viewpoints and applications, and various modifications or changes are made without departing from the spirit of the present invention. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments are combined with each other. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work belong to the scope of protection of the present invention.

[0019] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described in the present invention may be embodied in a wide variety of forms, and any specific structure and / or function described in the present invention is merely illustrative. Based on the present invention, it should be understood by those skilled in the art that an aspect described in the present invention may be implemented independently of any other aspect, and various ways may be combined with two or more of these aspects. For example, any number of aspects described in the present invention may be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described in the present invention may be used to implement this device and / or practice this method.

[0020] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention. The drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0021] Additionally, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects described may be practiced without these specific details.

[0022] See also Figure 1 , an embodiment of the present invention provides an electromagnetic feature mapping method based on contrastive learning, comprising: S110, obtaining scattering field data and electromagnetic characteristic data of a set scatterer; The scattered field data reflects the characteristics of scattered fields with different cavity shapes, different medium interface shapes and different overflow-type boundary shapes. For example, the scatterer is a cavity filled with overflowing inhomogeneous medium, and the scattered field data is solved using the Petrov-Galerkin finite element interface method on a non-body-fitting grid based on the derived forward model of the scatterer.

[0023] In some possible embodiments, the electromagnetic characteristic data is a dielectric constant contrast matrix.

[0024] S120, converting the scattering field data into graph structure data to generate a graph data set, and converting the electromagnetic characteristic data into image structure data to generate an image data set, wherein the graphs in the graph data set correspond one to one to the images in the image data set; Due to the correspondence between the scattered field data and the electromagnetic characteristic data, for each graph in the graph data set, there is an image in the image data set that corresponds to it one-to-one, and the two can be set to have the same index.

[0025] S130, generating positive sample pairs and negative sample pairs according to the graph dataset and the image dataset, wherein the positive sample pairs include corresponding graphs and images, and the negative sample pairs include non-corresponding graphs and images; Specifically, since the graph and image with the same index are corresponding, elements with the same index can be obtained from the graph dataset and the image dataset to form a positive sample pair. The graph and image in the negative sample pair are not corresponding. In order to construct a negative sample pair, it is necessary to randomly select an image for each graph (except the image corresponding to the graph). An index different from the index of the current graph can be randomly generated, and the image corresponding to this different index can be selected from the image dataset to form a negative sample pair with the current graph.

[0026] In some possible embodiments, positive sample pairs and negative sample pairs are generated before training begins.

[0027] In some possible embodiments, positive sample pairs are generated before training begins, negative sample pairs are dynamically generated during training, or positive sample pairs and negative sample pairs used in each training cycle are generated at the beginning of each training cycle. In this way, the generalization ability of the feature mapping network obtained through training can be improved.

[0028] S140. Using the positive sample pairs and the negative sample pairs, a two-stream network model including a graph encoder and an image encoder is trained to obtain a feature mapping network.

[0029] The graph encoder is used to process the scattered field data of the graph structure, and the image encoder is used to process the electromagnetic characteristic data of the image structure.

[0030] In some possible embodiments, the graph encoder includes an input layer, a graph convolution layer (GCNConv) and a global average pooling layer, wherein the graph convolution layer is used to convert the features of the graph nodes into an abstract graph representation, and the global average pooling layer is used to perform global average pooling processing on the output of the graph convolution layer to obtain a fixed-size graph feature vector representation. Optionally, there are two graph convolution layers.

[0031] In some possible embodiments, the image encoder uses a pre-trained ResNet model, and adjusts the output image feature vector and the graph feature vector to have consistent dimensions by modifying its fully connected layer.

[0032] In some possible embodiments, the two-stream network model further includes a feature projection layer, which is used to project the graph feature vector and the image feature vector into a common dimensional space to perform effective similarity calculation.

[0033] In some possible embodiments, the step includes: Inputting the graph and the image in the positive sample pair or the negative sample pair into a graph encoder and an image encoder respectively to obtain a graph feature vector and an image feature vector; Calculating a loss value of a contrast loss function based on the graph feature vector and the image feature vector; The parameters of the two-stream network model are updated according to the loss value until a convergence condition is met to obtain the feature mapping network.

[0034] The convergence condition includes that the network model maps the graph structure data and the image structure data to the same feature space, in which the similarity between positive sample pairs is the highest and the similarity between negative sample pairs is as low as possible.

[0035] In some possible embodiments, the loss value of the contrast loss function is calculated as follows: Generate a similarity matrix based on the graph feature vector and the image feature vector; Calculating the cross entropy loss in the row direction and the cross entropy loss in the column direction of the similarity matrix; The cross entropy loss in the row direction and the cross entropy loss in the column direction are averaged to obtain the loss value.

[0036] In some possible embodiments, after the convergence condition is met, the trained two-stream network model is output, and then the trained two-stream network model is verified using a validation set to obtain a feature mapping network. The validation set is a data set other than the training set and the test set data in the data set.

[0037] Specifically, graph structure data with a first index is randomly extracted from the graph data set in the validation set, and input into the trained two-stream network model to obtain a graph feature vector; Input the image data set of the validation set into the trained two-stream network model to obtain the image feature vector; By calculating the cosine similarity between the graph feature vector and the image feature vector, the image structure data with the highest similarity is found; If the index of the found image structure data is the same as the first index or the deviation is less than a preset threshold, the verification passes; otherwise, the verification fails.

[0038] The resulting feature mapping network is used to map electromagnetic features. As a feature extractor in the electromagnetic iterative reconstruction process, it can map the scattered field data and electromagnetic characteristic data into a shared feature space.

[0039] The above-mentioned embodiment of the present invention utilizes the idea of ​​contrastive learning to construct a dual-stream network model to respectively extract the features of the scattering field data and the electromagnetic properties of the scatterer, and contrastively learns the matching relationship between the two on a large amount of data to find a specific feature space, and trains to obtain a feature mapping network. Then, for various types of scatterers including special cavities, the feature mapping network can be used to map the scattering field data and electromagnetic features to a shared feature space, and the image features initialized based on the scattering field data are continuously iterated, so as to finally achieve more effective electromagnetic reconstruction; and this learning method is more transparent, and the correlation between the data learned by the model has been clearly displayed in the loss function of the contrastive learning model.

[0040] See also Figure 2 , an embodiment of the present invention provides an electromagnetic iterative reconstruction method, comprising: S210, obtaining scattering field data of the scatterer to be measured and converting it into graph structure data; S220, inputting the graph structure data and the image structure data representing the electromagnetic characteristic data into a feature mapping network respectively to obtain a graph data feature vector and an image data feature vector; The input initialized image structure data is, for example, selected from an existing image feature library of electromagnetic characteristic data.

[0041] Among them, the feature mapping network is based on Figure 1 The electromagnetic feature mapping method is trained to map the graph structure data representing the scattered field data and the image structure data representing the electromagnetic characteristic data into a shared feature space.

[0042] S230, calculating a loss value of a reconstruction loss function according to the graph data feature vector and the image data feature vector; In some possible embodiments, this step specifically includes: Inputting the graph data feature vector into a similarity prediction network to obtain a similarity threshold; Obtaining an estimated feature vector according to the image data feature vector and the similarity threshold; A loss value of a reconstruction loss function is calculated based on the graph data feature vector and the estimated feature vector.

[0043] Optionally, the reconstruction loss function is: Among them, g represents the feature vector of graph data, g * Represents the estimated feature vector, cos_sim() represents the cosine similarity, i represents the dimension, and is the set weight value.

[0044] Optionally, the similarity prediction network is obtained by training based on the similarity of the graph and the image in the sample pair. After the graph and the image in each sample pair are respectively passed through the graph encoder and the image encoder of the feature mapping network, the similarity between the graph and the image is obtained, and the similarity is used as a label to associate with the feature vector of the graph data, and the regression network is trained to obtain the similarity prediction network.

[0045] S240: Iteratively update the image structure data according to the loss value to obtain electromagnetic characteristic data of the scatterer to be measured.

[0046] The above-mentioned embodiment of the present invention utilizes the feature mapping network obtained by pre-training, initializes the electromagnetic characteristic image as another input of the electromagnetic reconstruction model, calculates the loss value according to the scattered field data and the electromagnetic characteristic image of the graph structure, iteratively updates the electromagnetic characteristic image features, and reconstructs the electromagnetic characteristic data matching the scattered field data. It does not need to build a complex forward model, is suitable for the electromagnetic reconstruction of various types of scatterers, and can obtain better reconstruction effects.

[0047] Figure 1 and Figure 2 The method provided in the embodiment is executed by a computing device, which is implemented as software, or as a combination of software and hardware, and the computing device is integrated in a server, terminal device, etc.

[0048] To facilitate understanding, the electromagnetic iterative reconstruction method based on contrast learning provided by an embodiment of the present invention is explained below with a more specific example. For example, the scatterer is set to be a cavity filled with an overflow-type medium, the electromagnetic property of the scatterer refers to the dielectric constant distribution in the cavity, and the electromagnetic property data is a dielectric constant contrast matrix.

[0049] First, obtain the scattered field data and electromagnetic characteristic data of the set scatterer: See also Figure 3, for a cavity filled with overflow-type medium, assume a separate emitter and receiver scattering model (e.g., emitter with 64 different incident angles, 255 receiver positions), with a virtual semicircular boundary B R , B R A spill-over cavity that can completely cover the embedded ground plane; According to Maxwell's equations and boundary conditions, the Helmholtz equation satisfied by the scattering model is given; The overflow cavity aperture is surrounded by a virtual semicircle, and the artificial boundary conditions on the virtual semicircle in TM and TE polarization modes are derived using the Fourier technique. The computational domain is restricted to the area enclosed by the virtual semicircle, the cavity boundary and the ground plane. Finally, the overflow cavity model in the unbounded region is simplified to the Helmholtz problem in the bounded region. For the above model, the Petrov-Galerkin finite element interface method is introduced into the scattering problem of the overflow cavity filled with inhomogeneous medium. First, the cavity is placed in a finite rectangular domain and divided into 64*64 grids using a uniform Cartesian grid. Each grid is divided into four triangles by two diagonals. Each triangle is marked as a unit for calculation, and the scattered field data and dielectric constant contrast matrix are collected.

[0050] Then, a graph mode of the scattered field data and an image mode of the dielectric constant contrast matrix are constructed, that is, the two types of data are converted into graph structure data and image structure data respectively.

[0051] Among them, the far-field scattered field data is used to construct the graphical mode, aiming to express the spatial characteristics and dynamic relationships of the scattered field through the graph structure. In the context of electromagnetic scattering, the sequences of far-field electric field values ​​received at different angles may have potential connections, which originate from the physical process of the interaction between electromagnetic waves and scatterers. These interactions usually depend on the shape, size, and material properties (such as dielectric constant) of the scatterer. Considering the physical and geometric associations between far-field data, the far-field electric field value at each angle is regarded as a node in the graph, and the edges between nodes are established based on the nearest neighbor relationship of the angle or the similarity of the scattering intensity. The construction process of the graphical mode of the scattered field data is as follows: First, for each collected scattered field data point, the four basic features of its complex form, real part, imaginary part, amplitude and phase are extracted, and these features are merged into a vector with a dimension of 1020 (each feature has 255 dimensions, representing 255 different receiver positions at each incident angle) to fully describe the characteristics of each data point. Calculate the full feature similarity matrix and normalize each feature (real part, imaginary part, amplitude, phase) to eliminate the impact of different magnitudes. Calculate the cosine similarity of each feature separately. The similarities of these four features are averaged to obtain the final similarity matrix. Based on the calculated similarity matrix, an undirected graph is constructed according to the preset threshold (for example, 0.43). Each node represents a data point, and the node feature is the aforementioned 1020-dimensional vector. If the similarity between two nodes exceeds the threshold, an edge is established between the two nodes, and the weight of the edge is the corresponding similarity value. The constructed graph is converted into a format suitable for PyTorch Geometric processing, including: node feature matrix, edge index list, and edge attributes (weights). Finally, each graph structure data is saved as a separate instance for subsequent graph neural network training and analysis.

[0052] Based on the obtained graph structure data and image structure data, a graph data set representing the scattering field data and an image data set representing the dielectric constant contrast matrix are generated. The data in these two data sets correspond one to one, that is, each graph structure has a corresponding image structure, representing the corresponding scattering field data and dielectric constant contrast matrix.

[0053] At the beginning of each training cycle, the positive and negative sample pairs used in the cycle are generated. All generated positive and negative sample pairs are encapsulated in a data structure for efficient processing by the data loader. This encapsulation not only contains the sample data itself, but also covers the necessary index information to ensure correct access during training.

[0054] Then, use the data loader to load the graph data and image data pairs. Each data pair contains a graph structure data and its corresponding image data. During the training process, the contrastive learning method is used to dynamically generate positive sample pairs and negative sample pairs through the network structure and loss function to optimize the effect of feature learning. The data loader batches the graph structure data to ensure that the image and graph structure data can be correctly processed during the batch processing process, which is convenient for the subsequent contrastive learning process. During the training process, various parameters of the model are first configured, including input dimensions, hidden layer dimensions, and output dimensions. These configurations are intended to provide necessary information for the construction of the two-stream network (graph encoder and image encoder). Then, the network model is initialized and deployed on available computing devices. At the same time, an optimizer is initialized to adjust the network parameters, and a loss function is initialized to evaluate the performance of the model. Training is performed through multiple cycles, and each cycle processes all batches in the training dataset. In each batch, the model feeds the input graph data and the corresponding positive sample pairs and negative sample pairs into the network, obtains the feature representation of the graph and the feature representation of the image through forward propagation, and calculates the loss value of the contrastive loss function. This loss function aims to make the feature representations of positive pairs more similar and the feature representations of negative pairs more different. After the loss value is calculated, backpropagation is performed to update the network weights in order to reduce the error of future predictions until the model converges.

[0055] The dual-stream network aims to achieve accurate feature extraction and data matching by combining deep learning techniques of graph structure data and image data. This network architecture is also called a dual-tower network and consists of two main parts: Graph Encoder and Image Encoder, which are described in detail as follows (see Figure 4 ): The specific implementation of the graph encoder includes: (1) Input layer: dimension input_dim=1020, corresponding to the feature dimension of the graph nodes; (2) The first layer of graph convolution: using the GCN graph convolutional neural network layer, the input feature dimension input_dim=1020 is converted to the hidden layer dimension hidden_dim=512, and the ReLU activation function is used to enhance the nonlinear feature extraction capability; (3) The second layer of graph convolution: continuing to use the GCNConv layer, the hidden layer features are converted from hidden_dim=512 to the final output dimension 256; (4) Global average pooling layer: the output features of all nodes are aggregated into a fixed-size graph-level feature vector, namely the graph feature vector, for subsequent feature matching.

[0056] The image encoder uses a pre-trained ResNet-18 model. The key configurations include: (1) Output dimension setting: By modifying the fully connected layer fc in ResNet-18, the dimension of the output feature is adjusted to ensure that the image feature vector is consistent with the image feature vector dimension 256, which is convenient for feature comparison learning. (2) Forward propagation: The image data is propagated omnidirectionally through the modified ResNet-18 network to output the feature vector of each image, that is, the image feature vector. In the figure, Conv block and Basicblock are two residual blocks of the ResNet-18 network, which are implemented using the existing structure.

[0057] Through this dual-stream design, the feature mapping network can not only capture the deep features in the graph structure data and image data at the same time, but also use contrastive learning to accurately align the feature expressions of the two. With the help of deep feature learning, the model performs well in capturing complex similarities and subtle differences, making it have excellent accuracy and adaptability in complex data matching tasks such as electromagnetic imaging reconstruction.

[0058] The network architecture also includes a feature projection layer: a linear transformation layer (without bias) is set for each graph feature vector and image feature vector to project them into a shared space with common dimensions for efficient similarity calculation.

[0059] In order to optimize the learning process of the two-stream network model, this embodiment designs a loss function to strengthen the correlation between graph data and image data. Specifically, the loss function uses the similarity information between graph embedding and image embedding to enhance the feature expression ability of the model through a two-way contrast learning strategy. The calculation process of the loss function is as follows: (1) Feature Normalization First, the graph feature vector g and the image feature vector i are L2 normalized so that their modulus is 1, so that the cosine similarity can be used as a metric: (2) Similarity calculation Use the normalized feature vector to calculate the similarity between the graph and the image. Specifically, refer to Figure 5 , a similarity matrix is ​​generated based on the graph feature vector and the image feature vector, and the two directions of the matrix correspond to graph to image (column direction) and image to graph (row direction).

[0060] Calculate the cosine similarity in two directions: Graph-to-image similarity: Image to graph similarity: The exponential form of logit_scale is a factor used to adjust the scale of similarity calculation. By scaling the similarity value, the sensitivity of the contrast loss can be effectively adjusted, so that the model can better fit the data during training.

[0061] (3) Contrast loss calculation In order to strengthen the matching relationship between graph embedding and image embedding, the cross entropy loss is used to minimize the matching error of positive sample pairs and increase the discrimination of negative sample pairs. In this process, the goal of each sample is to be most similar to itself, and its label is generated by the index in the batch. Define the contrast loss in two directions: Image-to-image contrast loss: Image-to-image contrast loss: The contrast loss function is the average of the image-to-image and image-to-image contrast losses: The comprehensive loss calculation process ensures that the model is optimized evenly in both directions.

[0062] The trained two-stream network model still needs to be verified. Randomly extract the graph structure data in the verification set and input it into the trained two-stream network model. The graph feature vector is obtained through the graph encoder in the model. The current graph feature vector is allowed to access the N feature vectors in the image feature library. The access method is to calculate the cosine similarity. If the image feature index with the highest similarity to the graph feature vector is the same or similar to the index of the input graph structure data, it means that the verification is passed, and the two-stream network model is output as the feature mapping network in the electromagnetic reconstruction model.

[0063] In addition, after each training sample pair passes through the dual encoder, each sample pair will have a similarity in the feature space. The embedding vector of the graph data representing the scattering field information in the feature space is denoted as g i , the similarity between sample pairs is sim i . Use sim i As g i , train a small regression network as a similarity prediction network, associate this similarity label with the graph feature vector, and use the regression network to learn the mapping relationship to facilitate the selection of the similarity threshold when the sample to be tested is iteratively reconstructed.

[0064] Based on the trained feature mapping network and similarity prediction network, an electromagnetic reconstruction model is obtained. Figure 6 The process of electromagnetic reconstruction based on the electromagnetic reconstruction model is shown. First, the Figure 3In the same manner as shown, the scattering field data of the scatterer to be measured is obtained and converted into image structure data, and a trainable image feature tensor is created to initialize the image structure data representing the dielectric constant contrast matrix.

[0065] Then, the trained electromagnetic reconstruction model and model parameters are loaded, and the graph structure data is input into the graph encoder to obtain the graph data feature vector g i , input the image structure data into the image encoder to obtain the image data feature vector I*. Perform the following steps: 1) Similarity prediction: Send the graph data feature vector g into the similarity prediction network to obtain the similarity threshold sim; 2) Similarity constraint: Get the estimated feature vector g based on I* and sim * , by constraining I * and g * The cosine similarity of is the predicted sim value to maintain the overall similarity; 3) Dimensional matching loss: further optimization to make I * The estimated eigenvector g * The value of g in each dimension is close to that of g. * ) contains two main parts: Feature dimension-by-dimension matching loss: Generate loss by comparing the feature values ​​of g and g* dimension by dimension. The loss value will be minimized only when the values ​​in each dimension are almost consistent; Similarity matching loss: 1-cos_sim(g*,g) is used to ensure that g and g* are similar in direction, further reducing the error.

[0066] If the loss value does not meet the termination condition, the loss value is back-propagated in the shared feature space, and the image features of the image structure data are iteratively updated using an optimizer (such as Adam) until the termination condition is met, and the image structure data at this time is output to complete the electromagnetic reconstruction. The termination condition is, for example, that the loss value is lower than a preset threshold, or that the preset maximum number of iterations is reached.

[0067] See also Figure 7 The embodiment of the present invention further provides an electronic device 80, the electronic device comprising: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the electromagnetic feature mapping method and electromagnetic iterative reconstruction method based on contrast learning in the aforementioned method embodiment.

[0068] An embodiment of the present invention also provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable the computer to execute the electromagnetic feature mapping method and electromagnetic iterative reconstruction method based on contrast learning in the aforementioned method embodiment.

[0069] An embodiment of the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer executes the electromagnetic feature mapping method based on contrast learning and the electromagnetic iterative reconstruction method in the aforementioned method embodiment.

[0070] Reference below Figure 7 , which shows a schematic diagram of the structure of an electronic device 80 suitable for implementing an embodiment of the present invention. The electronic devices in the embodiment of the present invention include, but are not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0071] like Figure 7 As shown, the electronic device 80 includes a processing device (such as a central processing unit, a graphics processing unit, etc.) 801, which performs various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage device 808 to a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the electronic device 80 are also stored. The processing device 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0072] Typically, the following devices are connected to the I / O interface 805: input devices 806 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 807 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 808 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 809. The communication device 809 allows the electronic device 80 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 7 The electronic device 80 is shown with various devices, but it should be understood that it is not required to implement or have all the devices shown. More or fewer devices may be implemented or have instead.

[0073] In particular, according to an embodiment of the present invention, the process described above with reference to the flowchart is implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program is downloaded and installed from the network through the communication device 809, or installed from the storage device 808, or installed from the ROM 802. When the computer program is executed by the processing device 801, the above-mentioned functions defined in the method of the embodiment of the present invention are executed.

[0074] It should be noted that the computer-readable medium of the present invention is a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium is, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium is any tangible medium containing or storing a program that is used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium includes a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal takes a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium is any computer-readable medium other than a computer-readable storage medium that sends, propagates or transmits a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium is transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0075] The computer-readable medium is included in the electronic device; it also exists independently without being assembled into the electronic device.

[0076] Computer program code for performing the operations of the present invention is written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code is executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer is connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or is connected to an external computer (e.g., through the Internet using an Internet service provider).

[0077] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram represents a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box also occur in a sequence different from that marked in the accompanying drawings. For example, two boxes represented in succession are actually executed substantially in parallel, and they are sometimes executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart, are implemented by a dedicated hardware-based system that performs the specified function or operation, or are implemented by a combination of dedicated hardware and computer instructions.

[0078] The units involved in the embodiments of the present invention are implemented by software or hardware, wherein the name of a unit does not limit the unit itself in some cases.

[0079] It should be understood that various parts of the present invention are implemented by hardware, software, firmware or a combination thereof.

[0080] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. An electromagnetic feature mapping method based on contrastive learning, characterized in that: include: Obtaining scattered field data and electromagnetic characteristic data of a set scatterer; Converting the scattered field data into graph structure data to generate a graph data set, and converting the electromagnetic characteristic data into image structure data to generate an image data set, wherein the graphs in the graph data set correspond one-to-one to the images in the image data set; Generate positive sample pairs and negative sample pairs according to the graph dataset and the image dataset, wherein the positive sample pairs include corresponding graphs and images, and the negative sample pairs include non-corresponding graphs and images; The positive sample pairs and the negative sample pairs are used to train a two-stream network model including a graph encoder and an image encoder to obtain a feature mapping network.

2. The electromagnetic signature mapping method according to claim 1, characterized in that: The two-stream network model including a graph encoder and an image encoder is trained by using the positive sample pairs and the negative sample pairs to obtain a feature mapping network including: Inputting the graph and the image in the positive sample pair or the negative sample pair into a graph encoder and an image encoder respectively to obtain a graph feature vector and an image feature vector; Calculating a loss value of a contrast loss function based on the graph feature vector and the image feature vector; The parameters of the two-stream network model are updated according to the loss value until a convergence condition is met to obtain the feature mapping network.

3. The electromagnetic signature mapping method according to claim 2, characterized in that: The loss value of the contrast loss function calculated based on the graph feature vector and the image feature vector includes: Generate a similarity matrix based on the graph feature vector and the image feature vector; Calculating the cross entropy loss in the row direction and the cross entropy loss in the column direction of the similarity matrix; The cross entropy loss in the row direction and the cross entropy loss in the column direction are averaged to obtain the loss value.

4. The electromagnetic signature mapping method according to any one of claims 1 to 3, characterized in that: The electromagnetic characteristic data is a dielectric constant contrast matrix.

5. An electromagnetic iterative reconstruction method, characterized in that: include: Obtain scattering field data of the scatterer to be measured and convert it into graph structure data; Inputting the graph structure data and the image structure data representing the electromagnetic characteristic data into a feature mapping network respectively to obtain a graph data feature vector and an image data feature vector; Calculate the loss value of the reconstruction loss function according to the graph data feature vector and the image data feature vector; Iteratively update the image structure data according to the loss value to obtain electromagnetic characteristic data of the scatterer to be measured; Wherein, the feature mapping network is obtained according to any one of the methods of claims 1-4.

6. The electromagnetic iterative reconstruction method according to claim 5, characterized in that: The loss value of the reconstruction loss function is calculated according to the graph data feature vector and the image data feature vector, comprising: Inputting the graph data feature vector into a similarity prediction network to obtain a similarity threshold; Obtaining an estimated feature vector according to the image data feature vector and the similarity threshold; A loss value of a reconstruction loss function is calculated based on the graph data feature vector and the estimated feature vector.

7. The electromagnetic iterative reconstruction method according to claim 6, characterized in that: The reconstruction loss function is: Among them, g represents the feature vector of graph data, g* represents the estimated feature vector, cos_sim() represents the cosine similarity, and i represents the dimension. and is the set weight value.

8. The electromagnetic iterative reconstruction method according to claim 6, characterized in that: The similarity prediction network is trained based on the similarity between the graph and the image in the sample pair.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively coupled to the at least one processor; Wherein, the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the electromagnetic feature mapping method according to any one of claims 1-4 or the electromagnetic iterative reconstruction method according to any one of claims 5-8.

Images