SYSTEM AND METHOD FOR TIME-FREQUENCY SEPARATION OF MULTIPLE WIRELESS SIGNALS - Patent application

Abstract

1. A system for time-frequency separation based on image analysis of one or more spectrograms of a plurality of wireless signals received over time from one or more emitters, the system comprising: a processing circuit configured to: determine, using the image analysis, a region of at least one of the spectrograms that contains a representation of a superposition of two or more overlapping wireless signals of the wireless signals, the overlapping wireless signals being representations of wireless signals that share a common time frequency in at least one of the spectrograms; decompose the superposition of the two or more overlapping wireless signals in the region into separate signal representations, each given one of the separate signal representations being represented by a corresponding bounding box around the given separate signal representation; and perform one or more actions using the separate signal representations.

Description

[Technical field]

[0001] The present invention relates to a system and method for time-frequency separation of multiple radio signals received over time from one or more emitters. [Background technology]

[0002] Wideband RF systems that receive and analyze Radio Frequency (RF) communications at multiple frequencies are important in many domains, such as automotive radar solutions, cellular and Wi-Fi coverage analysis, and communications intelligence systems. Current wideband RF analysis utilizes one-dimensional signal processing methods to analyze the energy levels of intercepted RF communications emitted at multiple frequencies from multiple sources to detect areas within the intercepted RF communications that contain signals of interest to the user of the wideband RF system. Current wideband RF systems are designed to process communications emitted from a limited number of communication sources that are narrow band, have static or cyclically varying frequencies, have known modulations (e.g., Amplitude Modulation (AM), Frequency Modulation (FM), Frequency Shift Keying (FSK), etc.), and are on a standard grid. Current wideband RF systems are not well suited to analyze complex RF communications emanating from multiple communication sources that are wideband and may have adaptive frequency hopping spread spectrum communication schemes, such as many modern RF transceivers used in ad hoc networks. These current wideband RF analysis systems are not suited to handle bursty communication sources, wideband messages (such as video feeds, multimedia feeds), multiple communication sources, sources with a high degree of modulation, and rich communication protocols.

[0003] In particular, current wideband RF systems receive multiple signals from multiple communication sources. These incoming signals may interfere with one another, causing the wideband RF system to receive superimposed signals. Current wideband RF analysis systems that utilize one-dimensional signal processing methods to separate the superimposed signals into multiple incoming signals have limited performance and separation capabilities.

[0004] Thus, there is a need in the art for new methods and systems for wideband RF analysis, and in particular for time-frequency separation of one or more spectrograms of multiple wireless signals received over time from one or more emitters that may be unknown, wideband, use burst communications, use complex modulation, or have no standard / pre-identified grid.

[0005] The following references are believed to be relevant as background to the presently disclosed subject matter. The admission of references herein should not be inferred as meaning that they are in any way relevant to the patentability of the presently disclosed subject matter.

[0006] Patent Document 1 discloses a signal identification method based on a wireless signal spectrum feature template. The method includes: constructing a signal template library by extracting the spectrum features of a plurality of wireless signals; preprocessing the signal to be identified according to the expression format of the signal template in the signal template library; and comparing and matching the preprocessed signal to be identified with the signal spectrum template library to obtain the type and the spectrum feature parameters of the signal to be identified. The present invention also provides a signal identification system based on a wireless signal spectrum feature template, which includes a template construction unit, a preprocessing unit, and a matching unit. In the present invention, the spectrum template can represent various types of signals, and the signal identification algorithm is further simplified while ensuring the accuracy of the signal identification.

[0007] Non-Patent Document 1 discloses that modulation identification is a key function for intelligent receivers. There are numerous applications in cognitive radar, software-defined radio (SDR), and efficient spectrum management. To identify communication and radar waveforms, their modulation types need to be classified. DARPA's Spectrum Coordination Challenge highlights the need to manage the demand for shared RF spectrum. Here we show how learning techniques can be leveraged in these types of applications to effectively identify modulation schemes.

[0008] In Non-Patent Document 2, a novel framework based on image processing techniques for radio environment characterization is disclosed. In particular, after digitizing a given scenario according to the frequencies detected at each point, an image can be constructed in which the pixel intensities of the image capture the radio-electrical conditions. Through non-linear filtering and object detection operations, the proposed framework makes it possible to identify homogeneous regions in which several frequencies can be detected. The method can be used in various contexts, such as building a database to give the terminals spectrum awareness in switch-on procedures or flexible spectrum scenarios to facilitate secondary utilization of the spectrum, support for devising appropriate strategies for cell and RAT selection.

[0009] Non-Patent Document 3 discloses that cognitive radio is a new paradigm for wireless communications that offers a solution to reconcile the growing demand and under-utilization of the current spectrum without modifications to the existing legacy wireless systems. Secondary users should be able to identify spatial and temporal spectrum holes not occupied by primary users and use them opportunistically without causing interference to the primary receiver. For that purpose, it is required to have knowledge of the primary network to ensure proper secondary user operation. In this context, assuming no collaboration between the primary and secondary networks, this document proposes a new framework, based on image processing techniques, that aims to combine several sensed samples at different geographical locations collected by secondary sensors in order to estimate the location of different primary transmitters. The results can be used to discover frequencies that the secondary network can use without disturbing the primary receiver. Our results, performed in a realistic scenario, show the effectiveness of the proposed framework in estimating transmitter locations.

[0010] Patent Document 2 discloses a flutter signal analysis method based on a convolutional neural network and a short-time Fourier transform, which includes the steps of performing a time-frequency analysis on an actually measured flutter signal by using a short-time Fourier transform to obtain a time-frequency graph of the flutter signal, and then mining image characteristics by using the powerful image processing ability of the convolutional neural network, and realizing the extraction of flutter characteristics and the subsequent analysis of the signal by calculating through sufficient connection layers and loss functions. The present invention combines a convolutional neural network with a short-time Fourier transform of a flutter signal, which has good reliability and accuracy for the analysis of actually measured flutter data, lays a solid foundation for further development of the research of combining artificial intelligence with aeroelasticity, and has practical engineering application value. [Prior art documents] [Patent documents]

[0011] [Patent Document 1] International Publication No. 2016 / 082562 [Patent Document 2] China Patent Application Publication No. 110866448 [Non-patent literature]

[0012] [Non-Patent Document 1] Algorithms to antenna: train deep-learning networks with synthesized radar and comms signals (Gentile et al.) published on November 20, 2019 [Non-Patent Document 2] On the applicability of image processing techniques in the radio environment characterization (Perez-Romero et al.) published on April 2009 [Non-Patent Document 3] Image processing techniques as a support to transmitter positioning determination in cognitive radio networks (Bolea et al.) published on May 2010 Summary of the Invention [Means for solving the problem]

[0013] According to a first aspect of the presently disclosed subject matter, there is provided a system for time-frequency separation based on image analysis of one or more spectrograms of a plurality of wireless signals received over time from one or more emitters, the system comprising: processing circuitry configured to: determine, using the image analysis, a region of at least one of the spectrograms containing a representation of a superposition of two or more overlapping wireless signals of the wireless signals, where the overlapping wireless signals are representations of wireless signals that share a common time frequency in at least one of the spectrograms; decompose the superposition of the two or more overlapping wireless signals in the region into separate signal representations, where each given one of the separate signal representations is represented by a corresponding bounding box around the given separate signal representation; and perform one or more actions utilizing the separate signal representations.

[0014] In some cases, the processing circuitry is further configured to determine, for at least one of the distinct signal representations, one or more parameters based on a corresponding bounding box, and the one or more actions are also performed based on the parameters.

[0015] In some cases, the parameters of the distinct signal representations include one or more of: (a) wavelength, (b) frequency, (c) rise time, (d) fall time, (e) duration, (f) power, (g) occupied bandwidth, (h) spectral density, (i) activity profile, or (j) polarization.

[0016] In some cases, the processing circuitry is further configured to: generate, upon decomposing the superposition of two or more overlapping wireless signals, a plurality of additional spectrograms of the plurality of wireless signals, where (a) a first additional spectrogram of the additional spectrograms has a first resolution, (b) a second additional spectrogram of the additional spectrograms has a second resolution, and (c) the first resolution is different from the second resolution; and, for at least one given distinct signal representation of the distinct signal representations, utilize the additional spectrogram and the corresponding parameters to determine one or more fine-tuned parameters of the given distinct signal representation having a higher accuracy than the accuracy of the corresponding parameters.

[0017] In some cases, the image analysis of spectrograms of multiple radio signals received over time is performed continuously or periodically.

[0018] In some cases, the processing circuitry is further configured to, upon resolving the superposition of two or more overlapping wireless signals, determine one or more statistical parameters associated with at least one given distinct signal representation of the distinct signal representations based on prior occurrences of the given distinct signal representation in the spectrogram.

[0019] In some cases, the statistical parameters include one or more of (a) signal activity level, (b) probability of occurrence, (c) probability of interception, (d) power distribution, (e) frequency stability, or (f) average duration.

[0020] In some cases, the actions include one or more of: (a) identifying emitters that emitted the overlapping wireless signals; (b) classifying the emitters that emitted the overlapping wireless signals; (c) generating a condensed representation of the multiple wireless signals received over time; (d) extracting samples of the overlapping wireless signals, thereby enabling wireless signal analysis; (e) providing the overlapping wireless signals to one or more external systems; or (f) providing the overlapping wireless signals to a user of the system.

[0021] In some cases, the decomposition of the superposition of overlapping wireless signals is based on Rectilinear Polygon Decomposition (RPD).

[0022] In some cases, the decomposition of the superposition of overlapping radio signals is based on artificial neural networks (ANN).

[0023] In some cases, the determination of the region is also based on bands of multiple radio signals received over time.

[0024] In some cases, the region is determined by identifying a bounding box around the region.

[0025] In some cases, each of the separate signal representations is represented by a corresponding bounding box around the separate signal representation.

[0026] In some cases, the spectrogram is used to optimize the signal-to-noise ratio (SNR) of multiple wireless signals.

[0027] According to a second aspect of the presently disclosed subject matter, there is provided a method for time-frequency separation based on image analysis of one or more spectrograms of multiple radio signals received over time from one or more emitters, the method including: determining, by a processing circuitry using the image analysis, a region of at least one of the spectrograms containing a representation of a superposition of two or more overlapping radio signals of the radio signals, where the overlapping radio signals are representations of radio signals that share a common time frequency in at least one of the spectrograms; decomposing, by the processing circuitry, the superposition of the two or more overlapping radio signals in the region into separate signal representations, where each given one of the separate signal representations is represented by a corresponding bounding box around the given separate signal representation; and performing, by the processing circuitry, one or more actions utilizing the separate signal representations.

[0028] In some cases, the method further includes determining, by the processing circuitry, for at least one of the distinct signal representations, one or more parameters based on a corresponding bounding box, wherein one or more actions are also performed based on the parameters.

[0029] In some cases, the parameters of the distinct signal representations include one or more of: (a) wavelength, (b) frequency, (c) rise time, (d) fall time, (e) duration, (f) power, (g) occupied bandwidth, (h) spectral density, (i) activity profile, or (j) polarization.

[0030] 16. The method of claim 15, further comprising: optionally generating, by the processing circuitry, a plurality of additional spectrograms of the plurality of wireless signals upon resolving the superposition of two or more overlapping wireless signals, where (a) a first additional spectrogram of the additional spectrograms has a first resolution, (b) a second additional spectrogram of the additional spectrograms has a second resolution, and (c) the first resolution is different from the second resolution; and for at least one given distinct signal representation of the distinct signal representations, determining, by the processing circuitry, one or more fine-tuned parameters of the given distinct signal representation having a precision greater than a precision of the corresponding parameter, utilizing the additional spectrogram and the corresponding parameter.

[0031] In some cases, the image analysis of spectrograms of multiple radio signals received over time is performed continuously or periodically.

[0032] In some cases, upon resolving the superposition of two or more overlapping wireless signals, the method further includes determining one or more statistical parameters associated with at least one of the distinct signal representations based on prior occurrences of the distinct signal representation in the spectrogram.

[0033] In some cases, the statistical parameters include one or more of (a) signal activity level, (b) probability of occurrence, (c) probability of interception, (d) power distribution, (e) frequency stability, or (f) average duration.

[0034] In some cases, the actions include one or more of: (a) identifying emitters that emitted the overlapping wireless signals; (b) classifying the emitters that emitted the overlapping wireless signals; (c) generating a condensed representation of the multiple wireless signals received over time; (d) extracting samples of the overlapping wireless signals, thereby enabling wireless signal analysis; (e) providing the overlapping wireless signals to one or more external systems; or (f) providing the overlapping wireless signals to a user of the system.

[0035] In some cases, the decomposition of the superposition of overlapping radio signals is based on rectilinear polygon decomposition (RPD).

[0036] In some cases, the decomposition of the superposition of overlapping radio signals is based on artificial neural networks (ANN).

[0037] In some cases, the determination of the region is also based on bands of multiple radio signals received over time.

[0038] In some cases, the region is determined by identifying a bounding box around the region.

[0039] In some cases, each of the separate signal representations is represented by a corresponding bounding box around the separate signal representation.

[0040] In some cases, the spectrogram is used to optimize the signal-to-noise ratio (SNR) of multiple wireless signals.

[0041] In accordance with a third aspect of the presently disclosed subject matter, there is provided a non-transitory computer readable storage medium having computer readable program code embodied therewith, the computer readable program code being executable by at least one processing circuit of a computer to perform a method for time-frequency separation based on image analysis of one or more spectrograms of a plurality of wireless signals received over time from one or more emitters, the method including: determining, by the processing circuitry using the image analysis, a region of at least one of the spectrograms containing a representation of a superposition of two or more overlapping wireless signals of the wireless signals, where the overlapping wireless signals are representations of wireless signals sharing a common time frequency in at least one of the spectrograms; decomposing, by the processing circuitry, the superposition of the two or more overlapping wireless signals in the region into separate signal representations, where each given one of the separate signal representations is represented by a corresponding bounding box around the given separate signal representation; and performing, by the processing circuitry, one or more actions utilizing the separate signal representations.

[0042] In order to understand the presently disclosed subject matter and to see how it may be carried out in practice, the subject matter will now be described, by way of non-limiting example only, with reference to the accompanying drawings, in which: [Brief description of the drawings]

[0043] [Figure 1A] 2 is a schematic diagram of an example input spectrogram having a representation of one or more input signals in accordance with the presently disclosed subject matter. [Figure 1B] 2 is a schematic diagram of an example spectrogram with a representation of one or more of the input signals above a threshold in accordance with the presently disclosed subject matter. [Figure 1C] 1 is a schematic diagram of an example spectrogram having a representation of one or more identified groups of pixels in accordance with the presently disclosed subject matter. [Figure 1D] 1 is a schematic diagram of an example spectrogram with a representation of one or more groups with marked boundaries in accordance with the presently disclosed subject matter; [Diagram 2] FIG. 1 is a block diagram that illustrates generally one example of a system for time-frequency separation based on image analysis of one or more spectrograms of multiple radio signals received over time from one or more emitters, in accordance with the presently disclosed subject matter. [Diagram 3] 1 is a flowchart illustrating one example sequence of operations performed for time-frequency separation based on image analysis of one or more spectrograms of multiple radio signals received over time from one or more emitters in accordance with the presently disclosed subject matter. [Figure 4A] 1 is a schematic diagram of an example region within a spectrogram having representations of one or more overlapping signals in accordance with the presently disclosed subject matter; [Figure 4B] 1 is a schematic diagram of an example region within a spectrogram having representations of one or more overlapping signals in accordance with the presently disclosed subject matter; [Figure 4C] 1 is a schematic diagram of an example region within a spectrogram having representations of one or more overlapping signals in accordance with the presently disclosed subject matter; [Figure 4D] 1 is a schematic diagram of an example region within a spectrogram having representations of one or more overlapping signals in accordance with the presently disclosed subject matter; [Figure 4E] 1 is a schematic diagram of an example region within a spectrogram having representations of one or more overlapping signals in accordance with the presently disclosed subject matter; [Figure 4F] 1 is a schematic diagram of an example region within a spectrogram having representations of one or more overlapping signals in accordance with the presently disclosed subject matter; [Figure 4G] 1 is a schematic diagram of an example region within a spectrogram having representations of one or more overlapping signals in accordance with the presently disclosed subject matter; [Figure 4H] 1 is a schematic diagram of an example region within a spectrogram having representations of one or more overlapping signals in accordance with the presently disclosed subject matter; [Diagram 5] 1 is a schematic diagram of an example spectrogram having a representation of one or more separated signals in accordance with the presently disclosed subject matter. [Figure 6] 1 is a schematic diagram of an exemplary spectrogram with good time resolution having a representation of two exemplary separated signals, and an exemplary spectrogram with good frequency resolution having a representation of the two exemplary separated signals, in accordance with the presently disclosed subject matter. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0044] In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the presently disclosed subject matter. However, it will be understood by those skilled in the art that the presently disclosed subject matter may be practiced without these specific details. In other instances, well-known methods, procedures and components have not been described in detail so as not to obscure the presently disclosed subject matter.

[0045] In the drawings and description set forth, like reference numbers designate components that are common to different embodiments or configurations.

[0046] Unless otherwise indicated, and as will become apparent from the following description, it will be appreciated that use of terms such as "determine," "resolve," "perform," "update," "isolate," and the like throughout the description of this specification includes computational acts and / or processes that manipulate and / or transform data into other data, where said data is represented as physical quantities, e.g., electronic quantities, and / or where said data represents physical objects. The terms “computer,” “processor,” “processing resource,” “processing circuitry,” and “controller” should be interpreted broadly to cover any type of electronic device with data processing capability, including, by way of non-limiting examples, personal desktop computers / personal laptop computers, servers, computing systems, communications devices, smartphones, tablet computers, smart televisions, processors (e.g., digital signal processors (DSPs), microcontrollers, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), etc.), groups of multiple physical machines that share performance of various tasks, virtual servers co-resident on a single physical machine, other electronic computing devices, and / or any combination thereof.

[0047] Operations according to the teachings herein may be performed by a specially constructed computer for the desired purpose, or may be performed by a general-purpose computer specially configured for the desired purpose by a computer program stored on a non-transitory computer-readable storage medium. The term "non-transitory" is used herein to exclude transitory propagating signals, but is otherwise used to include volatile or non-volatile computer memory technology suitable for the application.

[0048] As used herein, the phrases "for example," "such as," "for example," and variations thereof refer to non-limiting examples of the presently disclosed subject matter. Reference herein to "in one instance," "in some instances," "in other instances," or variations thereof means that a particular feature, structure, or characteristic described in connection with an embodiment is included in at least one embodiment of the presently disclosed subject matter. Thus, appearances of the phrases "in one instance," "in some instances," "in other instances," or variations thereof do not necessarily refer to the same embodiment.

[0049] It will be appreciated that, unless otherwise stated, certain features of the presently disclosed subject matter that are, for clarity, described in the context of separate embodiments, can also be provided in combination in a single embodiment. Conversely, various features of the presently disclosed subject matter that are, for brevity, described in the context of a single embodiment, can also be provided separately or in any suitable subcombination.

[0050] In embodiments of the presently disclosed subject matter, fewer, more, and / or different steps than those shown in FIG. 3 may be performed. In embodiments of the presently disclosed subject matter, one or more steps shown in FIG. 3 may be performed in a different order and / or one or more groups of steps may be performed simultaneously. FIGS. 1A-1D, 2, 4A-4H, and 5 show a general schematic diagram of system components and flows according to one embodiment of the presently disclosed subject matter. Each module and result shown in FIGS. 1A-1D, 2, 4A-4H, and 5 may be composed of any combination of software, hardware, and / or firmware that performs the functions defined and described herein. The modules and results shown in FIGS. 1A-1D, 2, 4A-4H, and 5 may be centralized in one location or distributed across two or more locations. In other embodiments of the presently disclosed subject matter, the system may include fewer, more, and / or different modules than those shown in FIGS. 1A-1D, 2, 4A-4H, and 5.

[0051] References herein to a method should be applied mutatis mutandis to a system capable of performing the method, and also to a non-transitory computer-readable medium storing instructions that, when executed by a computer, result in the method being performed.

[0052] References herein to a system should also be applied mutatis mutandis to a method that may be performed by that system, and should also be applied mutatis mutandis to a non-transitory computer-readable medium having instructions stored thereon that may be executed by that system.

[0053] References herein to a non-transitory computer readable medium should be applied mutatis mutandis to a system capable of executing instructions stored on the non-transitory computer readable medium, and should also be applied mutatis mutandis to a method that can be executed by a computer reading instructions stored on the non-transitory computer readable medium.

[0054] With this in mind, attention is directed to FIG. 1A, which is a schematic diagram of an exemplary input spectrogram having a representation of one or more input signals in accordance with the presently disclosed subject matter.

[0055] An input spectrogram is a two-dimensional image representing one or more input signals 110 of an intercepted communication. An input spectrogram may be collected by a wideband receiver, such as a wideband RF receiver (e.g., a spectrum analyzer) that intercepts various RF communications (e.g., wireless communications, cellular communications, Wi-Fi communications, radar communications, etc.) emitted by one or more emitters over a period of time. The input spectrogram may be analyzed utilizing image analysis methods to detect areas within the intercepted communication that contain signals of interest for the wideband RF analysis system.

[0056] An input spectrogram is a graph where the X-axis represents the time of interception (e.g., in milliseconds) and the Y-axis represents the frequency of interception (e.g., in MegaHertz (MHz)). The color (or gray scale) of each pixel located at a given X, Y location in the graph relates to the energy level (e.g., in deciBels (dB)) intercepted by the wideband RF receiver at time X and frequency Y. A sequence of colored pixels may represent an input signal 110. For example, FIG. 1A shows a recurrent sequence of light-colored pixel groups located between pixels 1500 and 2000 on the Y-axis of the graph. These pixel groups represent recurrent input signals 110 with a given energy level (represented by the luminance level of these pixels) intercepted between pixels 1500 and 2000.

[0057] The wideband RF analysis system can optionally process the input spectrogram to generate a filtered intercepted communication image, a non-limiting example of which is provided in FIG. 1B. A pixel of the filtered intercepted communication image located at a given X, Y location is in an "on state" if the color of the corresponding given pixel located at a given X, Y location of the input spectrogram is above a color threshold (e.g., above a color representing -40 dBm). If the corresponding given pixel of the input spectrogram is below the color threshold, the pixel is in an "off state" in the filtered intercepted communication image and does not become part of additional processing steps, as detailed herein. The pixels remaining in an "on state" represent the above-threshold signal 120. In the non-limiting example of FIG. 1B, input signals 110 located at pixel 1500 at the start of the X-axis that are below the color threshold are removed and do not become part of the above-threshold signal 120.

[0058] Similarly, the wideband RF analysis system can optionally process the filtered intercepted communications image to generate a noise filtered intercepted communications image, eliminating pixels above a noise threshold. The noise threshold can be determined for each spectrogram according to a desired sensitivity target. The filtering stage allows the wideband RF analysis system to process only those portions of the input spectrogram that have signals 120 above the threshold signal and are noise filtered signals, thereby providing better results for identifying signals in the input spectrogram.

[0059] The wideband RF analysis system processes the signal communication image above a threshold utilizing a local distance grouping algorithm (e.g., a labeling algorithm that labels connected components in a 2D binary image) to detect groups of pixels in the filtered intercepted communication image. The wideband RF analysis system may also optionally filter out pixels not found by the local distance grouping algorithm that should be part of any group of pixels. Each of the identified groups of pixels 130 may represent an identified signal in the intercepted communication, as described in further detail herein with reference to, inter alia, Figures 4A-4H, and the system may perform one or more actions on such signals, such as identifying overlapping wireless signals represented by groups of pixels that realize two or more overlapping wireless signals represented as two or more subgroups of the identified group. A non-limiting example of the resulting processed image is shown in Figure 1C.

[0060] 1D is a non-limiting example of marking the identified group 130 with a boundary, thereby resulting in a group 140 with a marked boundary, which is a pixel identified as a group with a boundary around the identified group 130, individually marking each of the groups of pixels. Note that this phase of processing can optionally use a filtered intercepted communication image, or the input spectrogram itself. Additionally, the described system can determine one or more parameters of the identified signal, such as wavelength, frequency, polarization, etc.

[0061] Having briefly described an exemplary input spectrogram and the processing of the input spectrogram to identify one or more identified signals 140, attention is now directed to FIG. 2, which is a block diagram that generally illustrates one example of a system for time-frequency separation based on image analysis of one or more spectrograms of multiple wireless signals received over time from one or more emitters, in accordance with the presently disclosed subject matter.

[0062] The system 200 may comprise or otherwise be associated with a data repository 210 (e.g., a database, a storage system, a memory including a read only memory ROM, a random access memory RAM, or any other type of memory, etc.) configured to store data including, among other things, an input spectrogram, a filtered intercepted communication image, a noise filtered intercepted communication image, a processed image, a threshold, characteristics of the identified signal 140, etc. In some cases, the data repository 210 may be further configured to allow retrieval and / or updating and / or deletion of data stored thereon. It is noted that in some cases, the data repository 210 may be distributed. It is noted that in some cases, the data repository 210 may be stored in a cloud-based storage.

[0063] System 200 further comprises a network interface 220, which allows system 200 to connect to a network and allows system 200 to send data through the network and receive data sent to system 200, including, in some cases, receiving one or more spectrograms representing RF signals collected over time, for example, from a wideband RF receiver. In a non-limiting sample setup, system 200 can analyze in real time a spectrogram representing a 40 MHz spectrum including multiple emitters (some of which may be burst communication emitters and / or frequency hopping emitters) emitting hundreds of signals per second. In some cases, network interface 220 can connect to a local area network (LAN), to a wide area network (WAN), or to the Internet. In some cases, network interface 220 can connect to a wireless network.

[0064] System 200 further comprises processing circuitry 230. Processing circuitry 230 may be one or more processing circuitry units (e.g., a central processing unit), microprocessor, microcontroller (e.g., a MicroController Unit (MCU)), or any other computing device or module, including multiple and / or parallel and / or distributed processing circuitry units, adapted to process data, either alone or in concert, to control associated system 200 resources and to enable operations related to system 200 resources.

[0065] The processing circuit 230 comprises the following modules: a time-frequency separation module 240 .

[0066] The time-frequency separation module 240 may be configured to perform a time-frequency separation process, as described in further detail herein, with particular reference to FIG.

[0067] FIG. 3 is a flowchart illustrating an example of a sequence of operations performed for time-frequency separation based on image analysis of one or more spectrograms of multiple radio signals received over time from one or more emitters in accordance with the presently disclosed subject matter.

[0068] According to some examples of the presently disclosed subject matter, the system 200 can be configured to perform a time-frequency separation process 300, for example, utilizing the time-frequency separation module 240, for identification and resolution of overlapping radio signals represented in an input spectrogram. The overlapping radio signals can be collected by a wideband RF receiver when two or more emitters simultaneously emit radio signals in overlapping frequencies. The overlapping radio signals are represented by regions of the input spectrogram as groups of pixels that realize two or more overlapping radio signals. FIG. 4A is a schematic diagram of an example region (e.g., one of region A 410-a, region B 410-b, or region C 410-c) in an input spectrogram representing one or more overlapping signals according to the presently disclosed subject matter. The system 200 analyzes at least one of the regions to determine whether it contains a superimposed representation of overlapping radio signals and resolves the overlapping signals into separate signal representations. To this end, the system 200 can be configured to use image analysis to determine at least one region of the spectrogram (e.g., one of region A 410-a, region B 410-b, or region C 410-c) that contains a representation of a superposition of two or more overlapping wireless signals of the wireless signal, the overlapping wireless signals being representations of wireless signals that share a common time frequency within the at least one of the spectrograms (Block 310). The system 200 analyzes the input spectrogram to determine at least one region in the input spectrogram (e.g., one of region A 410-a, region B 410-b, or region C 410-c). The determination of the region can be accomplished by using a machine learning model (e.g., an artificial neural network (ANN), random forest, decision tree, etc.) that masks all pixels associated with objects that represent non-overlapping wireless signals, or by any other image analysis technique.The determination of the region (e.g., one of region A 410-a, region B 410-b, or region C 410-c) may also be based on bands (e.g., High Frequency (HF) band, Very High Frequency (VHF) band, Ultra High Frequency (UHF) band, etc.) of multiple radio signals received over time and represented in the input spectrogram.

[0069] It should be noted that a region (e.g., one of region A 410-a, region B 410-b, or region C 410-c) may be one or more groups of pixels from one or more areas in the input spectrogram, and these areas may be discontinuous areas of the input spectrogram. For example, one region (e.g., one of region A 410-a, region B 410-b, or region C 410-c) may include a group of pixels from the upper left corner of the input spectrogram and a group of pixels from the lower right corner of the input spectrogram.

[0070] After determining the region (e.g., one or more of region A 410-a, region B 410-b, or region C 410-c), system 200 may be further configured to decompose a superposition of two or more overlapping wireless signals within the region (e.g., one or more of region A 410-a, region B 410-b, or region C 410-c) into separate signal representations, each given one of the separate signal representations being represented by a corresponding bounding box around the given separate signal representation (block 320). The system 200 decomposes the superposition of two or more overlapping wireless signals in a region (e.g., one or more of region A 410-a, region B 410-b, or region C 410-c) by employing one or more decomposition algorithms (e.g., rectilinear polygon decomposition (RPD), etc.) and / or by using one or more machine learning modules (e.g., artificial neural networks (ANN), random forests, decision trees, etc.) trained to decompose the superposition of two or more overlapping wireless signals into separate signal representations.

[0071] The system 200 determines and draws a bounding box around at least one of the separated signal representations. The bounding box depicts the boundary of the signal in the time and frequency domains. The bounding box marks the location of the separated signal in the spectrogram. These bounding boxes represent the separated signal representations and can be utilized together with the spectrogram to extract additional information about the separated signal to determine one or more parameters of the separated signal. This is performed by analyzing the corresponding bounding box itself, for example, by analyzing the location of the bounding box in the x-axis and y-axis, the shape of the bounding box, the color of the bounding box, the size of the bounding box, etc. Parameters that can be deduced from analyzing the bounding boxes include the wavelength of the corresponding separated signal, the frequency of the corresponding separated signal, the rise time of the corresponding separated signal, the fall time of the corresponding separated signal, the duration of the corresponding separated signal, the power of the corresponding separated signal, the occupied bandwidth of the corresponding separated signal, the spectral density of the corresponding separated signal, the activity profile of the corresponding separated signal, the polarization of the corresponding separated signal, etc.

[0072] In some cases, the system 200 can fine-tune the generated parameters of the separated signals. The system 200 achieves this by applying a multi-resolution time-frequency analysis method. This allows the system 200 to further increase the accuracy and resolution of the bounding boxes around the separated signals and to deduce more accurate estimates of the parameters. In multi-resolution analysis, the system 200 generates multiple additional spectrograms of the same received wireless signals, where the separated signal representations and their bounding boxes are used for the identified spectrograms. These additional spectrograms may have different resolutions. The additional spectrograms are used to deduce accurate signal parameter estimates and to fine-tune those parameters. Because it is not theoretically possible to accurately capture both the time and frequency characteristics of a signal using a single spectrogram due to the limitations imposed by the Gabor uncertainty principle, there is a trade-off between a good time resolution representation of a signal and a good frequency resolution representation of the same signal when analyzing a signal using a single spectrogram. The system 200 generates both additional spectrograms with good time resolution and additional spectrograms with good frequency resolution, and utilizes them to fine-tune the parameters deduced from the bounding box of the separated signal. The additional spectrogram resolution depends on the signal duration and the signal bandwidth and is generated to give an optimal precision alternately in the time domain and in the frequency domain. Thus, the parameters related to the time domain are extracted from the additional spectrogram with high time resolution, and the parameters related to the frequency domain are extracted from the additional spectrogram with high frequency resolution. The estimation and fine-tuning of the signal time parameters and the signal frequency parameters can be performed separately by analyzing the bounding box of the separated signal in the time domain and the spectral power envelope of the signal in the frequency domain. FIG. 6 is a non-limiting example of additional spectrograms generated by the system 200, where the spectrogram A600-a has good time resolution.The separated signals A520-a are decomposed by the system 200 and corresponding bounding boxes are drawn around them. The system 200 also generates an example spectrogram shown in spectrogram B600-b, which shows an additional spectrogram with good frequency resolution for the same received wireless signal. The same separated signal from spectrogram A600 is also shown in spectrogram B600-b as separated signal B520-b. It is noted that the characteristics (e.g., location, shape, etc.) of the separated signals B520-b and their corresponding bounding boxes are different in the good frequency resolution spectrogram because of the different resolution, and thus fine-tuned parameters related to frequency can be deduced from this spectrogram. Similarly, the system 200 can fine-tune time-related parameters by using the additional spectrogram with good time resolution shown in spectrogram A600-a.

[0073] In some cases, the system 200 can detect wireless signal representations in spectrograms by processing a sequence of input spectrograms continuously or periodically on a frame-by-frame basis. As multiple wireless signals are received over a given time period, their representations can appear in one or more spectrograms that are generated continuously or periodically over the given time period. The system 200 can detect the same signal with similar or varying bounding boxes between the sequence of spectrograms. The signal bounding boxes can be processed frame-by-frame to deduce statistical and temporal parameters. The statistical and temporal parameters can be extracted on the detected signals. Such information can include signal activity levels, probability of occurrence and interception, power distribution, frequency stability, average duration, etc. The system 200 can process the signal representations continuously for each incoming spectrogram. The system 200 can perform periodic processing on at least some of the spectrograms.

[0074] A non-limiting example of superposition of two or more overlapping radio signals includes the following steps, although it should be noted that at least some of the steps are optional. Down-sample the input representation. The system 200 processes a region in the input spectrogram (e.g., one or more of region A 410-a, region B 410-b, or region C 410-c) as shown in FIG. 4A to generate a down-sampled spectrogram, an example of which is shown in FIG. 4B. The down-sampling can be achieved by using a sample-based discretization process, such as max-pooling. The down-sampling can improve the signal-to-noise ratio (SNR) in the resulting down-sampled spectrogram. Threshold Filtering. The system 200 processes a region in the downsampled spectrogram (e.g., one or more of region A 410-a, region B 410-b, or region C 410-c) as shown in FIG. 4B to generate a threshold filtered spectrogram, an example of which is shown in FIG. 4C. The threshold filtered spectrogram may include only pixels having a color above a color threshold (e.g., above a color representing −40 dB). If a corresponding given pixel of the input spectrogram is below the color threshold, that pixel becomes “off” in the threshold filtered spectrogram. The system 200 may utilize the pixels below the color threshold in subsequent processing steps. These pixels may be used as “off” pixels. In some cases, the pixels below the color threshold may be used for image closing, for example, by filling holes in the “on” pixel group. Noise Filtering: System 200 processes regions in the threshold filtered spectrogram (e.g., one or more of region A 410-a, region B 410-b, or region C 410-c) as shown in FIG. 4C to generate a noise filtered spectrogram, an example of which is shown in FIG. 4D. Object Recognition. The system 200 processes a region (e.g., one or more of region A 410-a, region B 410-b, or region C 410-c) in the noise-filtered spectrogram as shown in FIG. 4D to generate an object recognition spectrogram, an example of which is shown in FIG. 4E. Object recognition can be achieved by utilizing a labeling algorithm that labels connected components in the noise-filtered spectrogram to detect objects, which are, for example, contiguous groups of pixels that make up the region (e.g., one or more of region A 410-a, region B 410-b, or region C 410-c). Recognizing Overlapping Objects. The system 200 processes a region (e.g., one or more of region A 410-a, region B 410-b, or region C 410-c) in the object recognition spectrogram as shown in FIG. 4E to generate an overlapping object recognition spectrogram, an example of which is shown in FIG. 4F, and the region is determined by identifying a bounding box around the region. This can be accomplished, for example, by drawing a tangential polygon that contains each of the identified objects. In a region with overlapping signals (e.g., one or more of region A 410-a, region B 410-b, or region C 410-c), the tangential polygon separates the overlapping signals into separate signal representations, each of which is represented by a corresponding bounding box around the separate signal representation, as shown, for example, in FIG. 4G and FIG. 4H. In FIG. 4H, the separate signal representations are shown in the background of the input spectrogram representation. In the non-limiting example shown in Figures 4G and 4H, the regions, i.e., region A 410-a, region B 410-b, and region C 410-c, are separated into separate signals, i.e., separate signal A 420-a, separate signal B 420-b, separate signal C 420-c, and separate signal D 420-d. Note that in some cases, overlapping objects have one or more intersections 430, where each intersection 430 is an area of ​​the object that is common to two or more objects. In this example, intersection 430 is common to separate signal A 420-a and separate signal B 420-b. System 200 can identify intersection 430 and associate intersection 430 with each of the objects to which intersection 430 belongs. A bounding box can be placed over all the regions of the input spectrogram, as shown in FIG. 5.

[0075] The system 200 can also use the input spectrogram to optimize the signal-to-noise ratio (SNR) of the separate signal representations.

[0076] Additionally, the system 200 can employ pattern recognition techniques to determine whether the input spectrogram contains a frequency hopping signal.

[0077] System 200 may then be configured to perform one or more actions utilizing the distinct signal representations (e.g., one or more of distinct signal A 420-a, distinct signal B 420-b, distinct signal C 420-c, or distinct signal D 420-d) (block 330). Note that the actions may also be based on parameters deduced from the bounding boxes in block 320 and fine-tuned parameters determined using the multi-resolution spectrograms generated by system 200 in block 320.

[0078] The actions may include one or more of the following: Identifying emitters that emitted overlapping wireless signals. System 200 can have an identifier for each emitter, with each of the separated signals (e.g., one or more of distinct signal A 420-a, distinct signal B 420-b, distinct signal C 420-c, or distinct signal D 420-d) being associated with a different identified emitter. Classifying emitters that emit overlapping wireless signals. The system 200 can utilize parameters of the separated signals (e.g., one or more of separate signal A 420-a, separate signal B 420-b, separate signal C 420-c, or separate signal D 420-d) to classify emitters according to the bands they emit. For example, HF emitter, VHF emitter, UHF emitter, etc. A condensed representation of a plurality of radio signals received over time is generated. Samples of the overlapping wireless signals are extracted, thereby enabling wireless signal analysis of locating one or more of the overlapping wireless signals, detecting one or more of the overlapping wireless signals, or accurately estimating parameters of one or more of the overlapping wireless signals. The separated signals (e.g., one or more of separate signal A 420-a, separate signal B 420-b, separate signal C 420-c, or separate signal D 420-d) are provided to one or more external systems, such as a cellular coverage planner, a radio frequency usage monitoring system, etc. The separated signals (e.g., one or more of separate signal A 420-a, separate signal B 420-b, separate signal C 420-c, or separate signal D 420-d) are provided to a user of system 200, for example, by utilizing a user interface (UI) component of system 200.

[0079] In some cases, the separated signals, their corresponding bounding boxes and parameters may be used by different signal analysis algorithms and applications, internal or external to system 200, including spectrum monitoring, cognitive radio spectrum sensing, signal classification and clustering, communication signal demodulation and data extraction, signal direction finding and geolocation, and other algorithms and applications.

[0080] With reference to Figure 3, it should be noted that some of the blocks may be combined into an integrated block, or may be divided into several blocks, and / or other blocks may be added. Furthermore, in some cases, the blocks may be performed in a different order than that described herein. Furthermore, it should be noted that some of the blocks are optional. Also, it should be noted that while the flow charts are described in terms of the system elements that implement them, this is by no means mandatory and that the blocks may be performed by elements other than those described herein.

[0081] It is to be understood that the presently disclosed subject matter is not limited in its application to the details set forth in the description contained herein or illustrated in the drawings. The presently disclosed subject matter is capable of other embodiments and of being practiced and carried out in various ways. It is therefore to be understood that the phraseology and terminology employed herein is for the purpose of description and should not be regarded as limiting. Those skilled in the art will therefore appreciate that the conception upon which the present disclosure is based may be readily utilized as a basis for the designing of other structures, methods and systems for carrying out some of the purposes of the presently disclosed subject matter.

[0082] It will also be appreciated that systems in accordance with the presently disclosed subject matter may be implemented, at least in part, as a suitably programmed computer. Similarly, the presently disclosed subject matter contemplates a computer program readable by a computer for performing the disclosed methods. The presently disclosed subject matter further contemplates a machine-readable memory tangibly embodying a program of instructions executable by a machine for performing the disclosed methods.

Claims

1. A system for time-frequency separation based on image analysis of one or more spectrograms of a plurality of wireless signals received over time from one or more emitters, the system comprising: Using image analysis, an overlay representation of overlapping wireless signals of two or more of the wireless signals, the overlapping wireless signals being a representation of wireless signals that share a common time-frequency within at least one of the spectrograms, determining at least one region of the spectrograms that includes the overlay representation; Decomposing the overlay of the two or more overlapping wireless signals within the region into separate signal representations, each given separate signal representation of the separate signal representations being represented by a corresponding bounding box around the given separate signal representation; Determining one or more parameters based on the corresponding bounding box for at least one of the separate signal representations of the separate signal representations; Performing one or more actions using the separate signal representations and the parameters A system comprising a processing circuit configured as such.

2. The parameters of the separate signal representations are a) wavelength, b) frequency, c) rise time, d) fall time, e) duration, f) power, g) occupied bandwidth, h) spectral density, i) activity profile, or j) polarization The system according to claim 1, comprising one or more of.

3. The processing circuit When decomposing the overlay of the two or more overlapping wireless signals, generating a plurality of additional spectrograms of the plurality of wireless signals, where (a) the first additional spectrogram of the additional spectrograms has a first resolution, (b) the second additional spectrogram of the additional spectrograms has a second resolution, and (c) the first resolution is different from the second resolution; For at least one given separate signal representation of the separate signal representations, using the additional spectrograms and the corresponding parameters to determine one or more fine-tuned parameters of the given separate signal representation that have a higher accuracy than the accuracy of the corresponding parameters The system according to claim 1, further configured as such.

4. The system of claim 1, wherein the image analysis of the spectrograms of the plurality of wireless signals received over time is performed continuously or periodically.

5. The system of claim 4, wherein when the processing circuit decomposes the superposition of the two or more overlapping wireless signals, it is further configured to determine one or more statistical parameters associated with at least one given distinct signal representation among the distinct signal representations based on the occurrence prior to the given distinct signal representation in the spectrogram.

6. The statistical parameters are (a) signal activity level, (b) probability of occurrence, (c) probability of interception, (d) power distribution, (e) frequency stability, or (f) average duration The system of claim 5, including one or more of.

7. The act is (a) identifying the emitter that emitted the overlapping wireless signals, (b) classifying the emitter that emitted the overlapping wireless signals, (c) generating a compressed representation of the plurality of wireless signals received over time, (d) extracting samples of the overlapping wireless signals, thereby enabling wireless signal analysis, (e) providing the overlapping wireless signals to one or more external systems, or (f) providing the overlapping wireless signals to a user of the system The system of claim 1, including one or more of.

8. The system of claim 1, wherein the decomposition of the superposition of the overlapping wireless signals is based on linear polygon decomposition (RPD).

9. The system of claim 1, wherein the determination of the region is also based on the band of the plurality of wireless signals received over time.

10. The system of claim 1, wherein the region is determined by identifying a bounding box around the region.

11. A method for time-frequency separation based on image analysis of one or more spectrograms of a plurality of wireless signals received over time from one or more emitters, the method comprising A step of determining, by a processing circuit, at least one region of the spectrogram that includes an expression of an overlap of two or more overlapping wireless signals among the wireless signals using image analysis, wherein the overlapping wireless signals are expressions of wireless signals that share a common time-frequency within the at least one of the spectrogram. A step of decomposing, by the processing circuit, the overlap of the two or more overlapping wireless signals in the region into separate signal expressions, wherein each given separate signal expression among the separate signal expressions is represented by a corresponding bounding box around the given separate signal expression. A step of determining, for at least one separate signal expression among the separate signal expressions, one or more parameters based on the corresponding bounding box. A step of performing, by the processing circuit, one or more actions using the separate signal expressions and the parameters. A method comprising the above steps.

12. The parameters of the separate signal expressions are (a) wavelength, (b) frequency, (c) rise time, (d) fall time, (e) duration, (f) power, (g) occupied bandwidth, (h) spectral density, (i) activity profile, or (j) polarization The method according to claim 11, comprising one or more of the above.

13. After decomposing the overlap of the two or more overlapping wireless signals, a step of generating, by the processing circuit, a plurality of additional spectrograms of the plurality of wireless signals, wherein (a) the first additional spectrogram of the additional spectrograms has a first resolution, (b) the second additional spectrogram of the additional spectrograms has a second resolution, and (c) the first resolution is different from the second resolution. A step of determining, for at least one given separate signal expression among the separate signal expressions, one or more fine-tuned parameters of the given separate signal expression that have a higher accuracy than the accuracy of the corresponding parameters, using the additional spectrogram and the corresponding parameters by the processing circuit. The method according to claim 11, further comprising the above steps.

14. The method according to claim 11, wherein the image analysis of the spectrograms of the plurality of wireless signals received over time is performed continuously or periodically.

15. The method according to claim 14, further comprising the step of determining one or more statistical parameters associated with at least one given distinct signal representation among the distinct signal representations, based on the occurrence prior to the given distinct signal representation in the spectrogram, when decomposing the superimposition of the two or more overlapping wireless signals.

16. The statistical parameter is (a) signal activity level, (b) probability of occurrence, (c) probability of interception, (d) power distribution, (e) frequency stability, or (f) average duration The method according to claim 15, comprising one or more of the above.

17. The act is (a) identifying the emitter that emitted the overlapping wireless signals, (b) classifying the emitter that emitted the overlapping wireless signals, (c) generating a compressed representation of the plurality of wireless signals received over time, (d) extracting samples of the overlapping wireless signals, thereby enabling wireless signal analysis, (e) providing the overlapping wireless signals to one or more external systems, or (f) providing the overlapping wireless signals to a user of the system The method according to claim 11, comprising one or more of the above.

18. The method according to claim 11, wherein the determination of the region is also based on the band of the plurality of wireless signals received over time.

19. The method according to claim 11, wherein the region is determined by identifying a bounding box around the region.

20. A non-transitory computer-readable storage medium having computer-readable program code embodied therewith, the computer-readable program code being executable by at least one processing circuit of a computer to perform a method for time-frequency separation based on image analysis of one or more spectrograms of a plurality of wireless signals received over time from one or more emitters, the method comprising A step of determining, by a processing circuit, at least one region of the spectrogram that includes an expression of an overlap of two or more overlapping wireless signals among the wireless signals, wherein the overlapping wireless signals are expressions of wireless signals that share a common time-frequency within the at least one of the spectrogram. A step of decomposing, by the processing circuit, the overlap of the two or more overlapping wireless signals within the region into separate signal expressions, wherein each given separate signal expression among the separate signal expressions is represented by a corresponding bounding box around the given separate signal expression. A step of determining one or more parameters based on the corresponding bounding box for at least one separate signal expression among the separate signal expressions. A step of performing one or more actions by the processing circuit using the separate signal expressions and the parameters. A non-transitory computer-readable storage medium including the above.

Images