Dual-stage prior-guided micro-Doppler spectrum interference suppression method, device and electronic equipment

Abstract

The present invention provides a two-stage prior-guided micro-Doppler spectrum interference suppression method, device and electronic device, which can be applied to the fields of image processing, radar signal processing and computer vision technology. The method includes: using a generator to perform a two-dimensional discrete wavelet transform on input data and mask data to obtain multiple feature maps in different frequency domains, wherein the input data includes a micro-Doppler spectrum, and the mask data is obtained by binarizing the input data; performing multi-band signal recovery on the feature maps in multiple frequency domains to obtain multiple output feature maps in different frequency domains, wherein the multi-band signal recovery includes feature extraction and feature compensation; performing an inverse two-dimensional discrete wavelet transform on the output feature maps in multiple frequency domains to obtain a structural repair map of the input data; and using an image processing network to process the structural repair map and mask data to obtain an interference suppression result of the input data.

Description

Technical Field

[0001] The present invention relates to the technical fields of image processing, radar signal processing and computer vision, and in particular to a method, device and electronic equipment for suppressing interference of micro-Doppler spectrogram guided by a dual-stage priori. Background Art

[0002] Computer vision technology can efficiently process various types of information and improve computing and production efficiency. Therefore, it is widely used in various fields. Compared with optical sensors, radar has a longer detection distance, can capture small movements, and has no requirements for light conditions and occlusion. It can be used in daily scenes and important places such as airports and disaster areas. Micro-Doppler radar can capture micro-Doppler signals generated by human movement. The micro-Doppler signal spectrum can be obtained through time-frequency transformation. The human behavior information and identity information contained in the signal can be interpreted by analyzing the spectrum. However, in the process of radar receiving echo signals, interference signals from the scene will also be recorded together. These interferences will damage or cover up the final signal spectrum, thereby contaminating useful information and affecting the accuracy of subsequent detection and recognition. Therefore, eliminating the interference signal in the spectrum, obtaining the compensation signal and restoring the original Doppler signal spectrum are of great research significance for the accurate detection and recognition of targets in the scene.

[0003] In the process of implementing the present invention, the prior art has at least the following problems: in the spectrum de-interference task, the interference signal is confused and superimposed with the original micro-Doppler signal, and it is difficult to ensure the integrity of the original spectrum signal when the interference is completely eliminated. For example, the use of gamma correction can enhance the contrast effect of the image, but some feature information will be eliminated during the signal processing process, resulting in the restored spectrum signal being less accurate than the original spectrum signal. Summary of the invention

[0004] In view of the above problems, the present invention provides a dual-stage a priori-guided micro-Doppler spectrum interference suppression method, device and electronic device.

[0005] According to a first aspect of the present invention, a two-stage prior-guided micro-Doppler spectrum interference suppression method is provided, comprising: using a generator to perform a two-dimensional discrete wavelet transform on input data and mask data to obtain a plurality of feature maps in different frequency domains, wherein the input data includes a micro-Doppler spectrum, and the mask data is obtained by binarizing the input data; performing multi-band signal recovery on the feature maps in the plurality of different frequency domains to obtain a plurality of output feature maps in different frequency domains, wherein the multi-band signal recovery includes feature extraction and feature compensation; performing an inverse two-dimensional discrete wavelet transform on the output feature maps in the plurality of different frequency domains to obtain a structure repair map of the input data; and using an image processing network to process the structure repair map and the mask data to obtain an interference suppression result of the input data.

[0006] According to an embodiment of the present invention, multi-band signal recovery is performed on feature maps of multiple different frequency domains to obtain output feature maps of multiple different frequency domains, including: using a residual network and a gated convolution layer to perform feature extraction and feature compensation on the feature map of the first frequency domain to obtain an output feature map of the first frequency domain; fusing the output feature map of the first frequency domain with the feature map of the second frequency domain to obtain an intermediate feature map; and using a residual network and a gated convolution layer to perform feature extraction and feature compensation on the intermediate feature map to obtain an output feature map of the second frequency domain.

[0007] According to an embodiment of the present invention, an inverse two-dimensional discrete wavelet transform is performed on a plurality of output feature maps in different frequency domains to obtain a structure repair map of the input data, including: fusing a plurality of output feature maps in different frequency domains to obtain a fused feature map; and performing an inverse two-dimensional discrete wavelet transform on the fused feature map to obtain a structure repair map of the input data.

[0008] According to an embodiment of the present invention, a generator is used to perform a two-dimensional discrete wavelet transform on input data and mask data to obtain multiple feature maps in different frequency domains, including: determining a spectrum to be repaired based on the input data and mask data; and performing a two-dimensional discrete wavelet transform on the spectrum to be repaired using a generator to obtain multiple feature maps in different frequency domains.

[0009] According to an embodiment of the present invention, an image processing network includes N encoders and N decoders; the image processing network is used to process a structure repair image and mask data to obtain an interference suppression result of the input data, including: inputting the Pth encoding result to the P+1th encoder to obtain the P+1th encoding result, wherein P is a positive integer less than N, and the first encoding result when P is 1 is obtained by inputting the structure repair image and the mask data to the first encoder; inputting the Qth decoding result and the NQth encoding result to the Q+1th decoder to obtain the Q+1th decoding result, wherein Q is a positive integer less than N, and the first decoding result when Q is 1 is obtained by decoding the Nth encoding result using the first decoder; and sequentially increasing Q until Q is N-1, inputting the N-1th decoding result and the 1st encoding result to the Nth decoder to obtain the Nth decoding result; and determining the Nth decoding result as the interference suppression result of the input data.

[0010] According to an embodiment of the present invention, the generator and the image processing network are obtained by pre-training, and the pre-training method includes: using the initial generator to perform a two-dimensional discrete wavelet transform on the sample input data and the sample mask data to obtain a plurality of sample feature maps in different frequency domains, wherein the sample input data includes a micro-Doppler spectrum map, and the sample mask data is obtained by binarizing the sample input data; performing multi-band signal recovery on the sample feature maps in the plurality of different frequency domains to obtain a plurality of sample output feature maps in different frequency domains, wherein the multi-band signal recovery includes feature extraction and feature compensation; performing an inverse two-dimensional discrete wavelet transform on the sample output feature maps in the plurality of different frequency domains to obtain a sample structure repair map of the sample input data; using the initial image processing network to process the sample structure repair map and the sample mask data to obtain a sample interference suppression result of the sample input data; determining the model loss based on the true value feature map corresponding to the sample output feature maps in the plurality of different frequency domains, the sample output feature maps in the plurality of different frequency domains, the sample true value of the sample input data, and the sample interference suppression result; based on the model loss, adjusting the model parameters of the initial generator and the initial image processing network to obtain the generator and the image processing network.

[0011] According to an embodiment of the present invention, the model loss includes semantic consistency loss, absolute value loss, envelope loss, strength loss and adversarial loss; based on the true value feature map corresponding to the sample output feature maps of multiple different frequency domains, the sample output feature maps of multiple different frequency domains, the sample true value of the sample input data and the sample interference suppression result, the model loss is determined, including: based on the true value feature map corresponding to the sample output feature maps of multiple different frequency domains, the sample mask data and the sample output feature maps of multiple different frequency domains, respectively, the semantic consistency losses of multiple different frequency domains are determined; based on the sample output feature map of the first frequency domain and the true value feature map corresponding to the sample output feature map of the first frequency domain, the first absolute value loss is determined; based on the sample interference suppression result and the sample input data According to the true value of the sample, determine the second absolute value loss; based on the sample output feature map of the second frequency domain, determine the first adversarial loss; based on the sample output feature map of the second frequency domain and the true value feature map corresponding to the sample output feature map of the second frequency domain, determine the second adversarial loss; based on the envelope of the true value of the sample and the envelope of the sample interference suppression result, determine the envelope loss; based on the pixel value probability distribution of the sample interference suppression result and the pixel value probability distribution of the true value of the sample, determine the intensity loss; based on the semantic consistency loss, absolute value loss, envelope loss, intensity loss and adversarial loss in multiple different frequency domains, determine the model loss, wherein the absolute value loss includes the first absolute value loss and the second absolute value loss, and the adversarial loss includes the first adversarial loss and the second adversarial loss.

[0012] According to an embodiment of the present invention, the dual-stage prior-guided micro-Doppler spectrum interference suppression method also includes: using an edge detection algorithm to process the true value of the sample to obtain the envelope of the true value of the sample; and using an edge detection algorithm to process the sample interference suppression result to obtain the envelope of the sample interference suppression result.

[0013] The second aspect of the present invention provides a two-stage prior-guided micro-Doppler spectrum interference suppression device, including: a discrete wavelet transform module, which is used to use a generator to perform a two-dimensional discrete wavelet transform on input data and mask data to obtain multiple feature maps in different frequency domains, wherein the input data includes a micro-Doppler spectrum and the mask data is obtained by binarizing the input data; a signal recovery module, which is used to perform multi-band signal recovery on the feature maps in multiple frequency domains to obtain multiple output feature maps in different frequency domains, wherein the multi-band signal recovery includes feature extraction and feature compensation; an inverse discrete wavelet transform module, which is used to perform an inverse two-dimensional discrete wavelet transform on the output feature maps in multiple frequency domains to obtain a structure repair map of the input data; and a result determination module, which is used to use an image processing network to process the structure repair map and the mask data to obtain an interference suppression result of the input data.

[0014] A third aspect of the present invention provides an electronic device, comprising: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0015] The fourth aspect of the present invention further provides a computer-readable storage medium on which a computer program or instruction is stored, and the steps of the above method are implemented when the above computer program or instruction is executed by a processor.

[0016] The fifth aspect of the present invention also provides a computer program product, including a computer program or instructions, which implement the steps of the above method when executed by a processor.

[0017] According to an embodiment of the present invention, the input data and mask data are decomposed according to the frequency domain through a two-dimensional discrete wavelet transform to obtain feature maps of multiple frequency domains. The semantic structure of the feature maps of multiple frequency domains is restored by the constraint of true value features and the extraction of context information in different frequency domains to avoid confusion of the semantic structure. Multi-band signal recovery is performed on the feature maps of multiple different frequency domains to obtain output feature maps of multiple different frequency domains. The output feature map is inversely transformed with a two-dimensional discrete wavelet transform to obtain a structural repair map of the input data, so that the structural repair map includes features of multiple frequency domains, making the features more comprehensive and the interference suppression effect better. The image processing network is used to process the structural repair map and the mask data to obtain the interference suppression result of the input data, so that the network is used to accurately restore the detailed texture and intensity information of the input data, further improving the accuracy of the interference suppression result. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above contents and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings.

[0019] Figure 1 An application scenario diagram of a dual-stage a priori-guided micro-Doppler spectrum interference suppression method, apparatus, device, medium, and program product according to an embodiment of the present invention is shown.

[0020] Figure 2 A flow chart of a dual-stage a priori-guided micro-Doppler spectrum interference suppression method according to an embodiment of the present invention is shown.

[0021] Figure 3 A model structure diagram of a dual-stage a priori-guided micro-Doppler spectrum interference suppression model according to an embodiment of the present invention is shown.

[0022] Figure 4 A comparison diagram of interference suppression results using a dual-stage a priori-guided micro-Doppler spectrum interference suppression method according to an embodiment of the present invention is shown.

[0023] Figure 5 A structural block diagram of a dual-stage a priori-guided micro-Doppler spectrum interference suppression device according to an embodiment of the present invention is shown.

[0024] Figure 6 A block diagram of an electronic device suitable for implementing a dual-stage a priori-guided micro-Doppler spectrum interference suppression method according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0025] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0026] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0027] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0028] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0029] In the technical solution of the present invention, the user information (including but not limited to user personal information, user image information, user device information, such as location information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, invention and application of the relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0030] In the scenario of using personal information for automated decision-making, the methods, devices, and systems provided by the embodiments of the present invention provide users with corresponding operation portals for users to choose to agree or reject the automated decision-making results; if the user chooses to reject, the expert decision-making process will be entered. The expression "automated decision-making" here refers to the activity of automatically analyzing and evaluating an individual's behavioral habits, interests and hobbies, or economic, health, credit status, etc. through computer programs, and making decisions. The expression "expert decision-making" here refers to the activity of making decisions by people who specialize in a certain field, have specialized experience, knowledge and skills, and have reached a certain level of professionalism.

[0031] An embodiment of the present invention provides a two-stage prior-guided micro-Doppler spectrum interference suppression method, which uses a generator to perform a two-dimensional discrete wavelet transform on input data and mask data to obtain multiple feature maps in different frequency domains, wherein the input data includes a micro-Doppler spectrum, and the mask data is obtained by binarizing the input data; multi-band signal recovery is performed on the feature maps in multiple frequency domains to obtain multiple output feature maps in different frequency domains, wherein the multi-band signal recovery includes feature extraction and feature compensation; an inverse two-dimensional discrete wavelet transform is performed on the output feature maps in multiple frequency domains to obtain a structure repair map of the input data; and the structure repair map and the mask data are processed by an image processing network to obtain an interference suppression result of the input data.

[0032] Figure 1 An application scenario diagram of a dual-stage a priori-guided micro-Doppler spectrum interference suppression method, apparatus, device, medium, and program product according to an embodiment of the present invention is shown.

[0033] like Figure 1 As shown, the application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0034] The user can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only for example).

[0035] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0036] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0037] It should be noted that the dual-stage a priori guided micro-Doppler spectrum interference suppression method provided in the embodiment of the present invention can generally be executed by the server 105. Accordingly, the dual-stage a priori guided micro-Doppler spectrum interference suppression device provided in the embodiment of the present invention can generally be set in the server 105. The dual-stage a priori guided micro-Doppler spectrum interference suppression method provided in the embodiment of the present invention can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Correspondingly, the dual-stage a priori guided micro-Doppler spectrum interference suppression device provided in the embodiment of the present invention can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105.

[0038] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0039] The following will be based on Figure 1 The scene described by Figure 2~Figure 4 The dual-stage a priori-guided micro-Doppler spectrum interference suppression method of an embodiment of the invention is described in detail.

[0040] Figure 2 A flow chart of a dual-stage a priori-guided micro-Doppler spectrum interference suppression method according to an embodiment of the present invention is shown.

[0041] like Figure 2 As shown, the dual-stage a priori-guided micro-Doppler spectrum interference suppression method of this embodiment includes operations S210 to S240.

[0042] In operation S210, a generator is used to perform a two-dimensional discrete wavelet transform on the input data and the mask data to obtain a plurality of feature maps in different frequency domains.

[0043] In operation S220, multi-band signal recovery is performed on the feature maps in the multiple frequency domains to obtain the output feature maps in the multiple frequency domains.

[0044] In operation S230, an inverse two-dimensional discrete wavelet transform is performed on a plurality of output feature maps in different frequency domains to obtain a structural restoration map of the input data.

[0045] In operation S240, the structure restoration image and the mask data are processed using an image processing network to obtain an interference suppression result of the input data.

[0046] According to an embodiment of the present invention, a generator is used to perform a two-dimensional discrete wavelet transform on input data and mask data to obtain multiple feature maps in different frequency domains, wherein the generator includes a two-dimensional discrete wavelet transform module, a multi-band signal recovery module and an inverse two-dimensional discrete wavelet transform module, the input data includes a micro-Doppler spectrum, and the mask data is obtained by binarizing the input data.

[0047] According to an embodiment of the present invention, a multi-band signal recovery module is used to perform multi-band signal recovery on feature maps in multiple frequency domains to obtain multiple output feature maps in different frequency domains, wherein the multi-band signal recovery includes feature extraction and feature compensation, and each output feature map is obtained by performing multi-band signal recovery on a feature map.

[0048] According to an embodiment of the present invention, an inverse two-dimensional discrete wavelet transform module is used to perform inverse two-dimensional discrete wavelet transform on output feature maps of multiple different frequency domains to obtain a structural repair map of the input data, wherein the inverse two-dimensional discrete wavelet transform and the two-dimensional discrete wavelet transform are inverse processes of each other.

[0049] According to an embodiment of the present invention, an image processing network is used to process a structure restoration image and mask data to obtain an interference suppression result of the input data.

[0050] According to an embodiment of the present invention, the input data and mask data are decomposed according to the frequency domain through a two-dimensional discrete wavelet transform to obtain feature maps of multiple frequency domains. The semantic structure of the feature maps of multiple frequency domains is restored by the constraint of true value features and the extraction of context information in different frequency domains to avoid confusion of the semantic structure. Multi-band signal recovery is performed on the feature maps of multiple different frequency domains to obtain output feature maps of multiple different frequency domains. The output feature map is inversely transformed with a two-dimensional discrete wavelet transform to obtain a structural repair map of the input data, so that the structural repair map includes features of multiple frequency domains, making the features more comprehensive and the interference suppression effect better. The image processing network is used to process the structural repair map and the mask data to obtain the interference suppression result of the input data, so that the network is used to accurately restore the detailed texture and intensity information of the input data, further improving the accuracy of the interference suppression result.

[0051] According to an embodiment of the present invention, multi-band signal recovery is performed on feature maps of multiple different frequency domains to obtain output feature maps of multiple different frequency domains, including: using a residual network and a gated convolution layer to perform feature extraction and feature compensation on the feature map of the first frequency domain to obtain an output feature map of the first frequency domain; fusing the output feature map of the first frequency domain with the feature map of the second frequency domain to obtain an intermediate feature map; and using a residual network and a gated convolution layer to perform feature extraction and feature compensation on the intermediate feature map to obtain an output feature map of the second frequency domain.

[0052] According to an embodiment of the present invention, the plurality of feature maps in different frequency domains may include an output feature map of a first frequency domain and an output feature map of a second frequency domain, wherein the second frequency domain is higher than the first frequency domain.

[0053] According to an embodiment of the present invention, a residual network and a gated convolution layer are used to perform feature extraction and feature compensation on a feature map of a first frequency domain to obtain an output feature map of the first frequency domain, wherein the residual network is used to perform feature extraction on the first frequency domain feature map to capture the features of different frequency components. The residual network may include a plurality of residual blocks, each of which is used to learn useful information in a feature map and pass it to the next residual block of the network. The gated convolution layer is used to compensate for the extracted features. For example, if certain frequency components are of high importance in the original signal but are lost or suppressed during the feature extraction process, the gating mechanism can be used to help the network re-emphasize these components, thereby achieving feature compensation.

[0054] According to an embodiment of the present invention, the output feature map of the first frequency domain is fused with the feature map of the second frequency domain to obtain an intermediate feature map, and the intermediate feature map is subjected to feature extraction and feature compensation using a residual network and a gated convolutional layer to obtain an output feature map of the second frequency domain.

[0055] According to an embodiment of the present invention, the feature maps of multiple different frequency domains may also include an output feature map of a third frequency domain, wherein the third frequency domain is higher than the second frequency domain, and the output feature map of the third frequency domain is similar to the output feature map of the second frequency domain in the solution process. The output feature map of the second frequency domain is fused with the feature map of the third frequency domain to obtain a second intermediate feature map, and the residual network and the gated convolution layer are used to perform feature extraction and feature compensation on the second intermediate feature map to obtain an output feature map of the third frequency domain.

[0056] According to an embodiment of the present invention, a feature map in a higher frequency domain is fused with an output feature map in a lower frequency domain, and feature extraction and feature compensation are performed to obtain an output feature map in a higher frequency domain, thereby achieving information transfer between feature maps in different frequency domains, enhancing and enriching feature representation, reducing the problem of gradient vanishing, improving the generalization ability of the model, and improving the robustness of the model.

[0057] According to an embodiment of the present invention, an inverse two-dimensional discrete wavelet transform is performed on a plurality of output feature maps in different frequency domains to obtain a structure repair map of the input data, including: fusing a plurality of output feature maps in different frequency domains to obtain a fused feature map; and performing an inverse two-dimensional discrete wavelet transform on the fused feature map to obtain a structure repair map of the input data.

[0058] According to an embodiment of the present invention, multiple output feature maps in different frequency domains are fused to obtain a fused feature map, wherein the fused feature map includes the respective features of the output feature maps in different frequency domains. The fused feature map is inversely transformed into a two-dimensional discrete wavelet transform to obtain a structural restoration map of the input data.

[0059] According to an embodiment of the present invention, multiple output feature maps in different frequency domains are fused to obtain a fused feature map, and the fused feature map is inversely transformed by two-dimensional discrete wavelet transform to obtain a structure repair map of the input data, so that the structure repair map includes features of all frequency domains, thereby ensuring the accuracy of the interference suppression result obtained after image processing based on the structure repair map.

[0060] According to an embodiment of the present invention, a generator is used to perform a two-dimensional discrete wavelet transform on input data and mask data to obtain multiple feature maps in different frequency domains, including: determining a spectrum to be repaired based on the input data and mask data; and performing a two-dimensional discrete wavelet transform on the spectrum to be repaired using a generator to obtain multiple feature maps in different frequency domains.

[0061] According to an embodiment of the present invention, the input data and the mask data are fused to obtain a spectrum to be repaired, and a two-dimensional discrete wavelet transform module in the generator is used to perform a two-dimensional discrete wavelet transform on the spectrum to be repaired to obtain feature maps of multiple sub-frequency domains, and the feature maps of the multiple sub-frequency domains are combined to obtain feature maps of multiple different frequency domains. Taking the feature maps of three different frequency domains as an example, a two-dimensional discrete wavelet transform is performed on the spectrum to be repaired to obtain a feature map of the H sub-frequency domain and a feature map of the L sub-frequency domain, and the feature map of the H sub-frequency domain and the feature map of the L sub-frequency domain are respectively subjected to a two-dimensional discrete wavelet transform to obtain a feature map of the HH1 sub-frequency domain and a feature map of the HL1 sub-frequency domain, and a feature map of the LH1 sub-frequency domain and a feature map of the LL1 sub-frequency domain, and the feature maps of the HH1, HL1 and LH1 sub-frequency domains are combined to obtain a feature map of the first frequency domain, and the feature map of the sub-frequency domain corresponding to the frequency band with the lowest frequency is determined as the feature map of the third frequency domain, and the feature maps of the other sub-frequency domains are combined to obtain a feature map of the second frequency domain.

[0062] According to an embodiment of the present invention, a two-dimensional discrete wavelet transform is used to process the spectrum to be repaired multiple times to obtain feature maps of multiple sub-frequency domains, and the feature maps of multiple sub-frequency domains within a certain frequency band are combined to obtain feature maps of multiple frequency domains. The feature maps of multiple frequency domains are processed separately, so as to restore the image more comprehensively and improve the interference suppression effect.

[0063] According to an embodiment of the present invention, an image processing network includes N encoders and N decoders; the image processing network is used to process a structure repair image and mask data to obtain an interference suppression result of the input data, including: inputting the Pth encoding result to the P+1th encoder to obtain the P+1th encoding result, wherein P is a positive integer less than N, and the first encoding result when P is 1 is obtained by inputting the structure repair image and the mask data to the first encoder; inputting the Qth decoding result and the NQth encoding result to the Q+1th decoder to obtain the Q+1th decoding result, wherein Q is a positive integer less than N, and the first decoding result when Q is 1 is obtained by decoding the Nth encoding result using the first decoder; and sequentially increasing Q until Q is N-1, inputting the N-1th decoding result and the 1st encoding result to the Nth decoder to obtain the Nth decoding result; and determining the Nth decoding result as the interference suppression result of the input data.

[0064] According to an embodiment of the present invention, an image processing network includes N encoders and N decoders, wherein the image processing network may include a U-type network (U-Net), and the U-Net includes a U-shaped structure consisting of a downsampling path and an upsampling path, wherein the downsampling path includes N encoders, and the upsampling path includes N decoders. A skip connection is used between the Rth encoder and the NRth decoder to retain more detail information during the upsampling process. Preferably, N can be 8.

[0065] According to an embodiment of the present invention, the structure repair map and mask data are input into the first encoder to obtain the first encoding result, and the Pth encoding result is input into the P+1th encoder to obtain the P+1th encoding result, until P is N-1, the N-1th encoding result is input into the Nth encoder to obtain the Nth encoding result.

[0066] According to an embodiment of the present invention, based on the Nth encoding result, a first decoding result is determined, the Qth decoding result and the NQth encoding result are jump-connected and input to the Q+1th decoder to obtain the Q+1th decoding result, until Q takes N-1, the N-1th decoding result and the first encoding result are jump-connected and input to the Nth decoder to obtain the Nth decoding result, and the Nth decoding result is determined as the interference suppression result of the input data.

[0067] According to the embodiment of the present invention, since the image processing network is constructed using the downsampling path and the upsampling path, the receptive field of the image processing network is large, and all pixels of the structure repair map and the mask data can be covered, thereby making full use of the detail information in each area. Since the feature map obtained by downsampling is connected to the upsampling layer by a skip connection method, the information can be fully utilized, the weight of the global details in the network is enhanced, and the effect of the interference suppression of the micro-Doppler spectrum guided by the dual-stage prior is improved.

[0068] According to an embodiment of the present invention, the generator and the image processing network are obtained by pre-training, and the pre-training method includes: using the initial generator to perform a two-dimensional discrete wavelet transform on the sample input data and the sample mask data to obtain a plurality of sample feature maps in different frequency domains, wherein the sample input data includes a micro-Doppler spectrum map, and the sample mask data is obtained by binarizing the sample input data; performing multi-band signal recovery on the sample feature maps in the plurality of different frequency domains to obtain a plurality of sample output feature maps in different frequency domains, wherein the multi-band signal recovery includes feature extraction and feature compensation; performing an inverse two-dimensional discrete wavelet transform on the sample output feature maps in the plurality of different frequency domains to obtain a sample structure repair map of the sample input data; using the initial image processing network to process the sample structure repair map and the sample mask data to obtain a sample interference suppression result of the sample input data; determining the model loss based on the true value feature map corresponding to the sample output feature maps in the plurality of different frequency domains, the sample output feature maps in the plurality of different frequency domains, the sample true value of the sample input data, and the sample interference suppression result; based on the model loss, adjusting the model parameters of the initial generator and the initial image processing network to obtain the generator and the image processing network.

[0069] According to an embodiment of the present invention, the sample input data and the sample mask data are fused to obtain a sample spectrum to be repaired, and the initial two-dimensional discrete wavelet transform module in the initial generator is used to perform a two-dimensional discrete wavelet transform on the sample spectrum to be repaired to obtain sample feature maps of multiple sub-frequency domains, and the sample feature maps of the multiple sub-frequency domains are combined to obtain sample feature maps of multiple different frequency domains, wherein the sample input data includes a micro-Doppler spectrum, and the sample mask data is obtained by binarizing the sample input data.

[0070] According to an embodiment of the present invention, multi-band signal recovery is performed on a plurality of sample feature maps in different frequency domains to obtain a plurality of sample output feature maps in different frequency domains, wherein the multi-band signal recovery includes feature extraction and feature compensation.

[0071] According to an embodiment of the present invention, an initial residual network and an initial gated convolutional layer are used to perform feature extraction and feature compensation on a sample feature map of a first frequency domain to obtain a sample output feature map of the first frequency domain; the sample output feature map of the first frequency domain is fused with the sample feature map of the second frequency domain to obtain a sample intermediate feature map; an initial residual network and an initial gated convolutional layer are used to perform feature extraction and feature compensation on the sample intermediate feature map to obtain a sample output feature map of the second frequency domain.

[0072] According to an embodiment of the present invention, a plurality of sample output feature maps in different frequency domains are fused to obtain a fused sample feature map, and the fused sample feature map is inversely transformed with a two-dimensional discrete wavelet transform to obtain a sample structure repair map of the sample input data; the sample structure repair map and the sample mask data are processed using an initial image processing network to obtain a sample interference suppression result of the sample input data.

[0073] According to an embodiment of the present invention, based on true value feature maps corresponding to sample output feature maps in multiple frequency domains, sample output feature maps in multiple frequency domains, sample true values ​​of sample input data, and sample interference suppression results, a model loss is determined, and based on the model loss, model parameters of an initial generator and an initial image processing network are adjusted to obtain a generator and an image processing network, wherein the model loss includes a model loss of a discriminator and a model loss of a generator, the model loss of the discriminator is used to adjust the parameters of the discriminator, and the model loss of the generator is used to adjust the model parameters of the generator and the initial image processing network.

[0074] According to an embodiment of the present invention, the initial generator and the initial image processing network are trained using sample input data, sample mask data and sample true values ​​corresponding to the sample input data, thereby adjusting parameters to obtain a generator and an image processing network. Since the channel degree and dimension of the sample input data and the sample true value are the same or similar to the input data and the true value corresponding to the input data, the input data can be effectively interfered with by adjusting the model parameters after training using the sample data and the sample true value, so that the interference suppression result is closer to the true value corresponding to the input data.

[0075] According to an embodiment of the present invention, the model loss includes semantic consistency loss, absolute value loss, envelope loss, strength loss and adversarial loss; based on the true value feature map corresponding to the sample output feature maps of multiple different frequency domains, the sample output feature maps of multiple different frequency domains, the sample true value of the sample input data and the sample interference suppression result, the model loss is determined, including: based on the true value feature map corresponding to the sample output feature maps of multiple different frequency domains, the sample mask data and the sample output feature maps of multiple different frequency domains, respectively, the semantic consistency losses of multiple different frequency domains are determined; based on the sample output feature map of the first frequency domain and the true value feature map corresponding to the sample output feature map of the first frequency domain, the first absolute value loss is determined; based on the sample interference suppression result and the sample input data According to the true value of the sample, determine the second absolute value loss; based on the sample output feature map of the second frequency domain, determine the first adversarial loss; based on the sample output feature map of the second frequency domain and the true value feature map corresponding to the sample output feature map of the second frequency domain, determine the second adversarial loss; based on the envelope of the true value of the sample and the envelope of the sample interference suppression result, determine the envelope loss; based on the pixel value probability distribution of the sample interference suppression result and the pixel value probability distribution of the true value of the sample, determine the intensity loss; based on the semantic consistency loss, absolute value loss, envelope loss, intensity loss and adversarial loss in multiple different frequency domains, determine the model loss, wherein the absolute value loss includes the first absolute value loss and the second absolute value loss, and the adversarial loss includes the first adversarial loss and the second adversarial loss.

[0076] According to an embodiment of the present invention, based on the true value feature maps corresponding to the sample output feature maps of multiple different frequency domains, the sample mask data and the sample output feature maps of multiple different frequency domains, the semantic consistency loss of multiple different frequency domains is determined respectively. .

[0077] According to an embodiment of the present invention, based on the sample output feature map of the first frequency domain and the true value feature map corresponding to the sample output feature map of the first frequency domain, the first absolute value loss is determined, as shown in formula (1):

[0078] (1)

[0079] Among them, ||·|| 1 is the L1 loss, I Lout is the sample output feature map of the first frequency domain, I Lgt is a true feature map corresponding to the sample output feature map in the first frequency domain.

[0080] According to an embodiment of the present invention, based on the sample interference suppression result and the sample true value of the sample input data, the second absolute value loss is determined, as shown in formula (2):

[0081] (2)

[0082] Among them, I out is the sample interference suppression result, I gt is the sample true value of the sample input data. The absolute value loss is calculated using formula (3) :

[0083] (3)

[0084] Among them, λ 1 is a hyperparameter, preferably, λ 1 You can choose 2.

[0085] According to an embodiment of the present invention, based on the sample output feature map in the second frequency domain, a first adversarial loss is determined, wherein the first adversarial loss represents the adversarial loss of the generator, as shown in formula (4):

[0086] (4)

[0087] Among them, D 1 and D 2 are the first discriminator and the second discriminator respectively, I H1out is the output feature map of the second frequency domain, I H2out is the output feature map of the third frequency domain, E IH1out is the expectation of the second frequency domain, E I2out is the expectation in the third frequency domain.

[0088] According to an embodiment of the present invention, based on the sample output feature map of the second frequency domain and the true value feature map corresponding to the sample output feature map of the second frequency domain, a second adversarial loss is determined, wherein the second adversarial loss represents the adversarial loss of the discriminator, and the adversarial losses of the first discriminator and the second discriminator are respectively shown in formula (5) and formula (6):

[0089] (5)

[0090] (6)

[0091] Among them, I H1gt and I H2gt are the true feature maps of the second and third frequency domains, respectively, and Relu (·) is the Relu activation function. The second adversarial loss is shown in formula (7):

[0092] (7)

[0093] According to an embodiment of the present invention, based on the envelope of the sample true value and the envelope of the sample interference suppression result, the envelope loss is determined, as shown in formula (8):

[0094] (8)

[0095] Among them, C i (·) is the extraction of the envelope of the image, and n is the number of spectrograms.

[0096] According to an embodiment of the present invention, the spectrograms of different actions have different frequency distributions, and therefore the image intensities of the spectrograms of different actions are different. The difference in intensity distribution information between the spectrograms can be obtained by calculating the KL divergence between the true value of the sample and the sample interference suppression result, so that the distribution of intensity information is closer to the true spectrum, thereby avoiding intensity distortion of the spectrum.

[0097] According to an embodiment of the present invention, the intensity loss is determined based on the pixel value probability distribution of the sample interference suppression result and the pixel value probability distribution of the sample true value, as shown in formula (9):

[0098] (9)

[0099] Among them, P i (·) and Q i (·) is the probability distribution of pixel values ​​of the image.

[0100] According to an embodiment of the present invention, the model loss is determined based on the semantic consistency loss, absolute value loss, envelope loss, intensity loss and adversarial loss in multiple different frequency domains. The model loss includes the model loss of the discriminator and the model loss of the generator, the absolute value loss includes the first absolute value loss and the second absolute value loss, and the adversarial loss includes the first adversarial loss and the second adversarial loss.

[0101] (10)

[0102] (11)

[0103] in, is the model loss of the discriminator, is the model loss of the generator, λ 2 is a hyperparameter, preferably, λ2 It can be taken as 0.1.

[0104] According to an embodiment of the present invention, the model loss is determined based on the semantic consistency loss, absolute value loss, envelope loss, intensity loss and adversarial loss in multiple frequency domains, and the model parameters are adjusted using the model loss of the discriminator and the model loss of the generator respectively to improve the interference suppression effect of the model on the Doppler spectrum.

[0105] According to an embodiment of the present invention, the L1 loss can be used to determine the semantic consistency loss of multiple frequency domains. However, since the L1 loss is a pixel-level loss function, it focuses more on measuring the difference between the pixel values ​​of the spectrogram and tends to ignore the overall similarity of the spectrogram. Therefore, the spatial similarity of the spectrogram is taken into account in the generator stage so that the overall semantic structure of the image tends to be the same. Taking the feature graphs L, H1, and H2 of three different frequency domains as examples, where L represents the feature graph of the first frequency domain, H1 represents the feature graph of the second frequency domain, and H2 represents the feature graph of the third frequency domain, the semantic consistency loss of each of the three different frequency domains can be calculated using formulas (12) to (14), and the semantic consistency loss of the model can be calculated using formula (15):

[0106] (12)

[0107] (13)

[0108] (14)

[0109] (15)

[0110] In formula (12) to formula (14), λ 3 The guided polynomial is used to characterize the disturbed contaminated area, λ 4 The guided polynomial is used to characterize the clean region. is the spatial distance between A and B, M is the mask image, (1-M) is the image obtained by inverting the mask image, λ 3 , 4 and λ 5 is a hyperparameter, preferably, λ 3 We can take 2, λ 4 and λ 5, You can take 4. It can be calculated using formula (16):

[0111] (16)

[0112] in, To expand the spectrum feature map into a one-dimensional vector in space, m is the dimension of the spectrum feature map.

[0113] Figure 3 A model structure diagram of a dual-stage a priori-guided micro-Doppler spectrum interference suppression model according to an embodiment of the present invention is shown.

[0114] like Figure 3 As shown, the model structure of the model includes two stages, wherein the first stage 301 includes a generator and a discriminator, and the second stage 302 includes an image processing network. The spectrum to be repaired is determined based on the input data and the mask data, and a two-dimensional discrete wavelet transform module is used to perform a two-dimensional discrete wavelet transform on the spectrum to be repaired to obtain a sample feature map I in the first frequency domain. Lin , the sample feature map I of the second frequency domain H1in And the sample feature map I of the third frequency domain H2in , respectively, through the multi-band signal recovery module, the sample output feature map I of the first frequency domain is obtained Lout , the sample output feature map I of the second frequency domain H1out And the sample output feature map I of the third frequency domain H2out , and processed by the inverse two-dimensional discrete wavelet transform module to obtain a structure repair map, and the discriminator is used to determine the adversarial loss of the discriminator based on the true feature maps of the second frequency domain and the third frequency domain and the output feature maps of the second frequency domain and the third frequency domain.

[0115] The image processing network fuses the structure repair image and the mask data, and obtains the interference suppression result through the image processing network, determines the intensity distribution and envelope diagram of the interference suppression result, and determines the envelope loss and intensity loss based on the intensity distribution of the interference suppression result, the true value envelope diagram and the envelope diagram of the interference suppression result, and the true value intensity distribution, respectively, thereby determining the model loss and adjusting the model parameters of the image processing network.

[0116] According to an embodiment of the present invention, the dual-stage prior-guided micro-Doppler spectrum interference suppression method also includes: using an edge detection algorithm to process the true value of the sample to obtain the envelope of the true value of the sample; and using an edge detection algorithm to process the sample interference suppression result to obtain the envelope of the sample interference suppression result.

[0117] Figure 4 A comparison diagram of interference suppression results using a dual-stage a priori-guided micro-Doppler spectrum interference suppression method according to an embodiment of the present invention is shown.

[0118] like Figure 4As shown, the result comparison diagram includes 4 rows and 8 columns of images, wherein each column is, from left to right, an interference graph, the processing result of the input data using the Ensemble Clustering (EC) algorithm, the processing result of the input data using the Multi-scale Detail Feature Extractor (MEDFE) algorithm, the processing result of the input data using the Light Gradient Boosting Machine (LG) algorithm, the processing result of the input data using the Conditional Random Field Fully Convolutional Network (CV-FCN), the processing result of the input data using the Deep Fusion Clustering Network (DFCN), the interference suppression result obtained by the two-stage prior-guided micro-Doppler spectrum interference suppression method of the present invention, and the true value image corresponding to the input data. Each row corresponds to a different micro-Doppler spectrum input data, the processing result obtained by processing the input data using multiple methods, and the true value image corresponding to the input data. Among them, the interference suppression result obtained by the micro-Doppler spectrum interference suppression method guided by the dual-stage prior is obviously closer to the true image than the processing results of the input data using other algorithms. Therefore, the restoration effect of the micro-Doppler spectrum interference suppression method guided by the dual-stage prior is good.

[0119] According to an embodiment of the present invention, for a spectrogram of a micro-Doppler signal reflecting human motion, the frequency can reflect the speed of motion of each part of the body, and the envelope can reflect motion information, wherein the upper envelope represents the motion of the arm during human motion, the lower envelope represents the motion of the leg, and the envelope in the middle part of the spectrogram represents the motion of the trunk. In the image processing network, the gradient change information of the spectrogram can be used to extract the upper envelope, the lower envelope, and the middle part of the spectrogram.

[0120] According to an embodiment of the present invention, the sample true value is processed by an edge detection algorithm to obtain an envelope of the sample true value, wherein the edge detection algorithm may include a Canny algorithm for detecting edges from an image. The sample interference suppression result is processed by an edge detection algorithm to obtain an envelope of the sample interference suppression result, wherein the Gaussian filter standard deviation σ in the Canny algorithm represents the sensitivity of edge detection, and preferably, σ can be 5.

[0121] According to an embodiment of the present invention, an edge detection algorithm is used to determine the envelope of the true value of the sample and the envelope of the sample interference suppression result, so that the envelope loss is determined by using the envelope, and the model parameters are adjusted, thereby improving the effect of dual-stage prior-guided micro-Doppler spectrum interference suppression.

[0122] According to an embodiment of the present invention, the quality of the interference suppression result of the dual-stage prior-guided micro-Doppler spectrogram interference suppression method can be judged by indicators such as structural similarity index measure (SSIM), peak signal to noise ratio (PSNR), Fréchet distance (FID), and mean absolute error (MAE). The higher the values ​​of SSIM and PSNR, the better the performance of the spectrogram repair, and the lower the values ​​of FID and MAE, the better the performance of the spectrogram repair, as shown in Table 1.

[0123] Table 1

[0124]

[0125] According to the multiple index values ​​of the two-stage prior-guided micro-Doppler spectrum interference suppression method and multiple comparison algorithms in Table 1, it can be determined that the SSIM, PSNR and MAE of the two-stage prior-guided micro-Doppler spectrum interference suppression method characterize that the two-stage prior-guided micro-Doppler spectrum interference suppression method is better than all comparison algorithms, and the FID characterizes that the two-stage prior-guided micro-Doppler spectrum interference suppression method is second only to the LG algorithm. Therefore, the interference suppression performance of the two-stage prior-guided micro-Doppler spectrum interference suppression method is the best among all comparison algorithms.

[0126] Based on the above two-stage prior-guided micro-Doppler spectrum interference suppression method, the present invention also provides a two-stage prior-guided micro-Doppler spectrum interference suppression device. Figure 5 The device is described in detail.

[0127] Figure 5 A structural block diagram of a dual-stage a priori-guided micro-Doppler spectrum interference suppression device according to an embodiment of the present invention is shown.

[0128] like Figure 5 As shown, the dual-stage a priori-guided micro-Doppler spectrum interference suppression device 500 of this embodiment includes a discrete wavelet transform module 510 , a signal recovery module 520 , an inverse discrete wavelet transform module 530 and a result determination module 540 .

[0129] The discrete wavelet transform module 510 is used to perform a two-dimensional discrete wavelet transform on the input data and the mask data using a generator to obtain a plurality of feature maps in different frequency domains, wherein the input data includes a micro-Doppler spectrum and the mask data is obtained by binarizing the input data. In one embodiment, the discrete wavelet transform module 510 can be used to perform the operation S210 described above, which will not be described in detail here.

[0130] The signal recovery module 520 is used to perform multi-band signal recovery on the feature maps of multiple different frequency domains to obtain output feature maps of multiple different frequency domains, wherein the multi-band signal recovery includes feature extraction and feature compensation. In one embodiment, the signal recovery module 520 can be used to perform the operation S220 described above, which will not be repeated here.

[0131] The inverse discrete wavelet transform module 530 is used to perform inverse two-dimensional discrete wavelet transform on multiple output feature maps in different frequency domains to obtain a structural restoration map of the input data. In one embodiment, the inverse discrete wavelet transform module 530 can be used to perform the operation S230 described above, which will not be repeated here.

[0132] The result determination module 540 is used to process the structural restoration image and the mask data using the image processing network to obtain the interference suppression result of the input data. In one embodiment, the result determination module 540 can be used to perform the operation S240 described above, which will not be repeated here.

[0133] According to an embodiment of the present invention, the signal recovery module 520 includes a feature map determination submodule, a first feature fusion submodule and a feature compensation submodule.

[0134] The feature map determination submodule is used to perform feature extraction and feature compensation on the feature map of the first frequency domain by using a residual network and a gated convolutional layer to obtain an output feature map of the first frequency domain.

[0135] The first feature fusion submodule is used to fuse the output feature map of the first frequency domain with the feature map of the second frequency domain to obtain an intermediate feature map.

[0136] The feature compensation submodule is used to extract and compensate the intermediate feature map using the residual network and the gated convolution layer to obtain the output feature map of the second frequency domain.

[0137] According to an embodiment of the present invention, the inverse discrete wavelet transform module 530 includes a second feature fusion submodule and an inverse discrete wavelet transform submodule.

[0138] The second feature fusion submodule is used to fuse multiple output feature maps in different frequency domains to obtain a fused feature map.

[0139] The inverse discrete wavelet transform submodule is used to perform an inverse two-dimensional discrete wavelet transform on the fused feature map to obtain a structural restoration map of the input data.

[0140] According to an embodiment of the present invention, the discrete wavelet transform module 510 includes a spectrogram determination submodule and a discrete wavelet transform submodule.

[0141] The graph determination submodule is used to determine the spectrum graph to be repaired based on the input data and the mask data.

[0142] The discrete wavelet transform submodule is used to perform a two-dimensional discrete wavelet transform on the spectrum to be repaired using the generator to obtain multiple feature maps in different frequency domains.

[0143] According to an embodiment of the present invention, the result determination module 540 includes an encoder input submodule, a decoder input submodule, a residual connection submodule and a result determination submodule.

[0144] The encoder input submodule is used to input the Pth encoding result to the P+1th encoder to obtain the P+1th encoding result, wherein P is a positive integer less than N, and the first encoding result when P is 1 is obtained by inputting the structure repair map and mask data into the first encoder.

[0145] The decoder input submodule is used to input the Qth decoding result and the NQth encoding result into the Q+1th decoder to obtain the Q+1th decoding result, wherein Q is a positive integer less than N, and the first decoding result when Q is 1 is obtained by decoding the Nth encoding result using the first decoder.

[0146] The residual connection submodule is used to increase Q in sequence until Q is N-1, and then input the N-1th decoding result and the 1st encoding result to the Nth decoder to obtain the Nth decoding result.

[0147] The result determination submodule is used to determine the Nth decoding result as the interference suppression result of the input data.

[0148] According to an embodiment of the present invention, the dual-stage a priori guided micro-Doppler spectrum interference suppression device 500 also includes a sample discrete wavelet transform module, a sample signal recovery module, a sample inverse discrete wavelet transform module, a sample result determination module, a loss determination module and a parameter adjustment module.

[0149] The sample discrete wavelet transform module is used to perform a two-dimensional discrete wavelet transform on the sample input data and the sample mask data using the initial generator to obtain a plurality of sample feature maps in different frequency domains, wherein the sample input data includes a micro-Doppler spectrum and the sample mask data is obtained by binarizing the sample input data.

[0150] The sample signal recovery module is used to perform multi-band signal recovery on a plurality of sample feature maps in different frequency domains to obtain a plurality of sample output feature maps in different frequency domains, wherein the multi-band signal recovery includes feature extraction and feature compensation.

[0151] The sample inverse discrete wavelet transform module is used to perform inverse two-dimensional discrete wavelet transform on multiple sample output feature maps in different frequency domains to obtain a sample structure restoration map of the sample input data.

[0152] The sample result determination module is used to process the sample structure restoration image and the sample mask data using the initial image processing network to obtain the sample interference suppression result of the sample input data.

[0153] The loss determination module is used to determine the model loss based on the true value feature map corresponding to the sample output feature map of multiple different frequency domains, the sample output feature map of multiple different frequency domains, the sample true value of the sample input data and the sample interference suppression result.

[0154] The parameter adjustment module is used to adjust the model parameters of the initial generator and the initial image processing network based on the model loss to obtain the generator and the image processing network.

[0155] According to an embodiment of the present invention, the loss determination module includes a first loss determination submodule, a second loss determination submodule, a third loss determination submodule, a fourth loss determination submodule, a fifth loss determination submodule, a sixth loss determination submodule, a seventh loss determination submodule and a loss determination submodule.

[0156] The first loss determination submodule is used to determine the semantic consistency losses of multiple different frequency domains based on the true value feature maps, sample mask data and the sample output feature maps of multiple different frequency domains respectively corresponding to the sample output feature maps of multiple different frequency domains.

[0157] The second loss determination submodule is used to determine the first absolute value loss based on the sample output feature map of the first frequency domain and the true value feature map corresponding to the sample output feature map of the first frequency domain.

[0158] The third loss determination submodule is used to determine a second absolute value loss based on the sample interference suppression result and the sample true value of the sample input data.

[0159] The fourth loss determination submodule is used to determine the first adversarial loss based on the sample output feature map in the second frequency domain.

[0160] The fifth loss determination submodule is used to determine the second adversarial loss based on the sample output feature map of the second frequency domain and the true value feature map corresponding to the sample output feature map of the second frequency domain.

[0161] The sixth loss determination submodule is used to determine the envelope loss based on the envelope of the sample true value and the envelope of the sample interference suppression result.

[0162] The seventh loss determination submodule is used to determine the intensity loss based on the pixel value probability distribution of the sample interference suppression result and the pixel value probability distribution of the sample true value.

[0163] The loss determination submodule is used to determine the model loss based on semantic consistency loss, absolute value loss, envelope loss, intensity loss and adversarial loss in multiple different frequency domains, wherein the absolute value loss includes a first absolute value loss and a second absolute value loss, and the adversarial loss includes a first adversarial loss and a second adversarial loss.

[0164] According to an embodiment of the present invention, the dual-stage a priori-guided micro-Doppler spectrum interference suppression device 500 further includes a first envelope determination module and a second envelope determination module.

[0165] The first envelope determination module is used to process the true value of the sample using an edge detection algorithm to obtain the envelope of the true value of the sample.

[0166] The second envelope determination module is used to process the sample interference suppression result by using an edge detection algorithm to obtain the envelope of the sample interference suppression result.

[0167] According to an embodiment of the present invention, any multiple modules of the discrete wavelet transform module 510, the signal recovery module 520, the inverse discrete wavelet transform module 530 and the result determination module 540 can be combined in one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the discrete wavelet transform module 510, the signal recovery module 520, the inverse discrete wavelet transform module 530 and the result determination module 540 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in a proper combination of any of them. Alternatively, at least one of the discrete wavelet transform module 510, the signal recovery module 520, the inverse discrete wavelet transform module 530 and the result determination module 540 may be at least partially implemented as a computer program module, which may perform corresponding functions when executed.

[0168] Figure 6A block diagram of an electronic device suitable for implementing a dual-stage a priori-guided micro-Doppler spectrum interference suppression method according to an embodiment of the present invention is shown.

[0169] like Figure 6 As shown, the electronic device 600 according to an embodiment of the present invention includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 608 to a random access memory (RAM) 603. The processor 601 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include an onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0170] In RAM 603, various programs and data required for the operation of electronic device 600 are stored. Processor 601, ROM 602 and RAM 603 are connected to each other via bus 604. Processor 601 performs various operations of the method flow according to the embodiment of the present invention by executing the program in ROM 602 and / or RAM 603. It should be noted that the program can also be stored in one or more memories other than ROM 602 and RAM 603. Processor 601 can also perform various operations of the method flow according to the embodiment of the present invention by executing the program stored in the one or more memories.

[0171] According to an embodiment of the present invention, the electronic device 600 may further include an input / output (I / O) interface 605, which is also connected to the bus 604. The electronic device 600 may further include one or more of the following components connected to the input / output (I / O) interface 605: an input portion 606 including a keyboard, a mouse, etc.; an output portion 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 608 including a hard disk, etc.; and a communication portion 609 including a network interface card such as a LAN card, a modem, etc. The communication portion 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output (I / O) interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage portion 608 as needed.

[0172] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method of the embodiment of the present invention is implemented.

[0173] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 602 and / or RAM 603 described above and / or one or more memories other than ROM 602 and RAM 603.

[0174] The embodiment of the present invention also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the method provided by the embodiment of the present invention.

[0175] The computer program executes the above functions defined in the system / device of the embodiment of the present invention when it is executed by the processor 601. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0176] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 609, and / or installed from a removable medium 611. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0177] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the processor 601, the above functions defined in the system of the embodiment of the present invention are performed. According to the embodiment of the present invention, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0178] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0179] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0180] It will be appreciated by those skilled in the art that the features described in the various embodiments of the present invention may be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features described in the various embodiments of the present invention may be combined and / or combined in various ways. All of these combinations and / or combinations fall within the scope of the present invention.

[0181] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination advantageously. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A two-stage prior-guided micro-Doppler spectrum interference suppression method, characterized in that: The method comprises: Using a generator to perform a two-dimensional discrete wavelet transform on input data and mask data to obtain a plurality of feature maps in different frequency domains, wherein the input data includes a micro-Doppler spectrum and the mask data is obtained by binarizing the input data; Performing multi-band signal recovery on the feature maps in the multiple frequency domains to obtain multiple output feature maps in the multiple frequency domains, wherein the multi-band signal recovery includes feature extraction and feature compensation; Performing an inverse two-dimensional discrete wavelet transform on the output feature maps of the multiple different frequency domains to obtain a structural restoration map of the input data; and Processing the structure repair image and the mask data using an image processing network to obtain an interference suppression result of the input data; The performing multi-band signal recovery on the feature maps in different frequency domains to obtain output feature maps in different frequency domains includes: Using a residual network and a gated convolutional layer to perform feature extraction and feature compensation on the feature map of the first frequency domain, to obtain an output feature map of the first frequency domain; Fusing the output feature map of the first frequency domain with the feature map of the second frequency domain to obtain an intermediate feature map; Using the residual network and the gated convolutional layer to perform feature extraction and feature compensation on the intermediate feature map to obtain an output feature map in the second frequency domain; The method further comprises: The output feature map of the second frequency domain is fused with the feature map of the third frequency domain to obtain a second intermediate feature map; Using the residual network and the gated convolutional layer to perform feature extraction and feature compensation on the second intermediate feature map to obtain an output feature map in a third frequency domain; Among them, the first frequency domain, the second frequency domain and the third frequency domain increase in sequence.

2. The method according to claim 1, characterized in that The step of performing an inverse two-dimensional discrete wavelet transform on the output feature maps of the multiple different frequency domains to obtain a structural restoration map of the input data includes: Fusing the multiple output feature maps in different frequency domains to obtain a fused feature map; and An inverse two-dimensional discrete wavelet transform is performed on the fused feature map to obtain a structural restoration map of the input data.

3. The method according to claim 1, characterized in that The generator is used to perform a two-dimensional discrete wavelet transform on the input data and the mask data to obtain a plurality of feature maps in different frequency domains, including: Determining a spectrum to be repaired based on the input data and the mask data; and The generator is used to perform a two-dimensional discrete wavelet transform on the spectrum to be repaired to obtain the feature maps in the multiple different frequency domains.

4. The method according to claim 1, characterized in that: The image processing network includes N encoders and N decoders; The step of processing the structure repair image and the mask data using an image processing network to obtain an interference suppression result of the input data includes: Inputting the Pth encoding result to the P+1th encoder to obtain the P+1th encoding result, wherein P is a positive integer less than N, and the first encoding result when P is 1 is obtained by inputting the structure repair image and the mask data to the first encoder; Inputting the Qth decoding result and the NQth encoding result into the Q+1th decoder to obtain the Q+1th decoding result, wherein Q is a positive integer less than N, and the first decoding result when Q is 1 is obtained by decoding the Nth encoding result using the first decoder; and Increasing Q successively until Q is N-1, inputting the N-1th decoding result and the 1st encoding result into the Nth decoder to obtain the Nth decoding result; The Nth decoding result is determined as an interference suppression result of the input data.

5. The method according to claim 1, characterized in that The generator and the image processing network are pre-trained, and the pre-training method includes: Using an initial generator to perform a two-dimensional discrete wavelet transform on sample input data and sample mask data to obtain a plurality of sample feature maps in different frequency domains, wherein the sample input data includes a micro-Doppler spectrum, and the sample mask data is obtained by binarizing the sample input data; Performing multi-band signal recovery on the sample feature maps of the multiple different frequency domains to obtain sample output feature maps of the multiple different frequency domains, wherein the multi-band signal recovery includes feature extraction and feature compensation; Performing an inverse two-dimensional discrete wavelet transform on the plurality of sample output feature maps in different frequency domains to obtain a sample structure restoration map of the sample input data; Processing the sample structure restoration image and the sample mask data using an initial image processing network to obtain a sample interference suppression result of the sample input data; Determine a model loss based on a true value feature map corresponding to the plurality of sample output feature maps in different frequency domains, the plurality of sample output feature maps in different frequency domains, a sample true value of the sample input data, and the sample interference suppression result; Based on the model loss, the model parameters of the initial generator and the initial image processing network are adjusted to obtain the generator and the image processing network.

6. The method according to claim 5, characterized in that The determining of the model loss based on the true value feature graph corresponding to the plurality of sample output feature graphs in different frequency domains, the plurality of sample output feature graphs in different frequency domains, the sample true value of the sample input data and the sample interference suppression result comprises: Determine semantic consistency losses of the multiple different frequency domains respectively based on the true value feature maps respectively corresponding to the sample output feature maps of the multiple different frequency domains, the sample mask data and the sample output feature maps of the multiple different frequency domains; Determine a first absolute value loss based on a sample output feature map in a first frequency domain and a true feature map corresponding to the sample output feature map in the first frequency domain; determining a second absolute value loss based on the sample interference suppression result and the sample true value of the sample input data; Determine a first adversarial loss based on the sample output feature map of the second frequency domain; Determining a second adversarial loss based on the sample output feature map of the second frequency domain and a true feature map corresponding to the sample output feature map of the second frequency domain; Determining envelope loss based on an envelope of the sample true value and an envelope of the sample interference suppression result; Determine the intensity loss based on the pixel value probability distribution of the sample interference suppression result and the pixel value probability distribution of the sample true value; Based on the semantic consistency loss, absolute value loss, envelope loss, intensity loss and adversarial loss of the multiple different frequency domains, the model loss is determined, wherein the absolute value loss includes the first absolute value loss and the second absolute value loss, and the adversarial loss includes the first adversarial loss and the second adversarial loss.

7. The method according to claim 6, characterized in that The method further comprises: Processing the sample true value using an edge detection algorithm to obtain an envelope of the sample true value; and The sample interference suppression result is processed by using the edge detection algorithm to obtain an envelope of the sample interference suppression result.

8. A dual-stage prior-guided micro-Doppler spectrum interference suppression device, characterized in that: The device comprises: A discrete wavelet transform module, used for performing a two-dimensional discrete wavelet transform on input data and mask data using a generator to obtain a plurality of feature maps in different frequency domains, wherein the input data includes a micro-Doppler spectrum, and the mask data is obtained by binarizing the input data; A signal recovery module, used for performing multi-band signal recovery on the feature maps of the multiple different frequency domains to obtain multiple output feature maps of the multiple different frequency domains, wherein the multi-band signal recovery includes feature extraction and feature compensation; an inverse discrete wavelet transform module, used for performing an inverse two-dimensional discrete wavelet transform on the output feature maps of the plurality of different frequency domains to obtain a structural restoration map of the input data; and A result determination module, used for processing the structure restoration image and the mask data by using an image processing network to obtain an interference suppression result of the input data; The signal recovery module includes a feature map determination submodule, a first feature fusion submodule and a feature compensation submodule. The feature map determination submodule is used to perform feature extraction and feature compensation on the feature map of the first frequency domain by using a residual network and a gated convolutional layer to obtain an output feature map of the first frequency domain; The first feature fusion submodule is used to fuse the output feature map of the first frequency domain with the feature map of the second frequency domain to obtain an intermediate feature map; The feature compensation submodule is used to perform feature extraction and feature compensation on the intermediate feature map by using the residual network and the gated convolution layer to obtain an output feature map of the second frequency domain; The device also includes: A feature fusion module, used to fuse the output feature map of the second frequency domain with the feature map of the third frequency domain to obtain a second intermediate feature map; A feature map processing module, used to perform feature extraction and feature compensation on the second intermediate feature map by using the residual network and the gated convolution layer to obtain an output feature map in a third frequency domain; Among them, the first frequency domain, the second frequency domain and the third frequency domain increase in sequence.

9. An electronic device, characterized in that: The electronic device comprises: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 7.

Images