Long-range, battery free, wireless sensing with neuromorphic cameras

Abstract

Various examples are provided related to a battery-free platform designed to detect a range of sensor events in rooms and floors. In one example, a neuromorphic camera tag device includes an energy harvesting circuit; an energy management circuit; a wireless communication circuit; and a processing circuit that can utilize a particular modulation type to wirelessly communicate event data to a neuromorphic camera using the wireless communication circuit. Physical stimuli can include temperature, contact, button or key presses, and / or sound.

Description

LONG-RANGE, BATTERY FREE, WIRELESS SENSING WITH NEUROMORPHIC CAMERASCROSS REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to, and the benefit of, U.S. provisional application entitled “Long-Range, Battery Free, Wireless Sensing with Neuromorphic Cameras” having serial no. 63 / 666,389, filed July 1 , 2024, which is hereby incorporated by reference in its entirety.BACKGROUND

[0002] Microgrids featuring a high penetration of inverter-based resources (IBRs) have low physical inertia and are thus vulnerable to disturbances. The low inertia characteristic and fast dynamics of IBR challenge conventional decoupled static economic operation and dynamic control design within microgrids. Existing microgrid scheduling methods typically separate device-level controller design from grid-level economic operation, assuming IBRs as passive devices with constant control parameters. Additionally, dynamic security constraints are often overlooked during the economic scheduling stage.SUMMARY

[0003] Aspects of the present disclosure are related to battery-free platforms designed to detect a range of sensor events in entire rooms and floors. In one aspect, among others, a neuromorphic camera tag device comprises an energy harvesting circuit; an energy management circuit; a wireless communication circuit; and a processing circuit, wherein the processing circuit utilizes a particular modulation type to wirelessly communicate event data to a neuromorphic camera using the wireless communication circuit. In one or more aspects, the neuromorphic camera tag device can further comprise a sensor circuit comprising a sensor device, wherein at least a portion of the event data is based at least in part on data generated by the sensor device. The neuromorphic camera tag device can further comprise an input device circuit comprising an input device, wherein at least a portion of the event data is based at least in part on data generated by the input device. At least a portion of the event data can indicate activity spectrogram data indicating at least one of a duration, a frequency band, and amplitudes for the neuromorphic camera tag device. In various aspects, the neuromorphic camera tag device can further comprise an energy storage device. The particular modulation type can comprise On-Off Keying (OOK) modulation or frequency modulation.

[0004] In another aspect, a neuromorphic camera tag system comprises a neuromorphic camera; and a neuromorphic camera tag device communicatively coupled with the neuromorphic camera, the neuromorphic camera tag device comprising: an energy harvesting circuit; an energy management circuit; a wireless communication circuit; and a processing circuit, wherein the processing circuit utilizes a particular modulation type to wirelessly communicate event data to a neuromorphic camera using the wireless communication circuit. In one or more aspects, the neuromorphic camera tag device can further comprise a sensor circuit comprising a sensor device, wherein at least a portion of the event data is based at least in part on data generated by the sensor device. The neuromorphic camera tag device can further comprise an input device circuit comprising an input device, wherein at least a portion of the event data is based at least in part on data generated by the input device. At least a portion of the event data can indicate activity spectrogram data indicating at least one of a duration, a frequency band, and amplitudes for the neuromorphic camera tag device. In various aspects, the neuromorphic camera tag device can further comprise an energy storage device. The particular modulation type can comprise On-Off Keying (OOK) modulation or frequency modulation.

[0005] Other systems, methods, features, and advantages of the present disclosure will be or become apparent to one with skill in the art upon examination of the following drawings and detailed description. It is intended that all such additional systems, methods, features, and advantages be included within this description, be within the scope of the present disclosure, and be protected by the accompanying claims. In addition, all optional and preferred features and modifications of the described embodiments are usable in all aspects of the disclosure taught herein. Furthermore, the individual features of the dependent claims, as well as all optional and preferred features and modifications of the described embodiments are combinable and interchangeable with one another.BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Many aspects of the present disclosure can be better understood with reference to the following drawings. The components in the drawings are not necessarily to scale, emphasis instead being placed upon clearly illustrating the principles of the present disclosure. Moreover, in the drawings, like reference numerals designate corresponding parts throughout the several views.

[0007] FIG. 1 Illustrates an example of a neuromorphic camera tag system, in accordance with various embodiments of the present disclosure.

[0008] FIG. 2 is a schematic diagram illustrating an example of a digital NeuroCamTag implementation, in accordance with various embodiments of the present disclosure.

[0009] FIG. 3 Illustrates an example of an event processing pipeline, in accordance with various embodiments of the present disclosure.

[0010] FIG. 4 Illustrates an example of an analog NeuroCamTag system, in accordance with various embodiments of the present disclosure.

[0011] FIG. 5 is a schematic diagram illustrating an example of an analog NeuroCamTag implementation, in accordance with various embodiments of the present disclosure.

[0012] FIG. 6 illustrates examples of activity spectrograms from an analog NeuroCamTag system, in accordance with various embodiments of the present disclosure.

[0013] FIG. 7 is a schematic diagram illustrating an example of passive tag MEMS mic circuitry, in accordance with various embodiments of the present disclosure.

[0014] FIG. 8 is a schematic diagram illustrating an example of digital tag Schematic circuitry, in accordance with various embodiments of the present disclosure.

[0015] FIG. 9 a schematic diagram illustrating an example of processing circuitry that can be used for neuromorphic camera tag systems, in accordance with various embodiments of the present disclosure.DETAILED DESCRIPTION

[0016] The present disclosure provides examples related to a battery-free platform designed to detect a range of sensor events in entire rooms and floors without the need for batteries. The sensor events can include detection of a wide range of rich human interactions and activities, which can be predefined for detection using the neuromorphic camera tags. The neuromorphic camera (hardware) tag system, sometimes referred to as NeuroCamTags’ and the ‘NeuroCamTag’ system herein, comprises low-cost tags that harvest ambient light energy and utilize high-frequency modulation of light-emitting diodes (LEDs) for wireless communication. These visual signals can be captured by a neuromorphic camera that boasts an order of magnitude higher temporal resolution and frame rates compared to conventional cameras. The present disclosure describes an event processing pipeline that allows simultaneous localization and identification of multiple unique tags. Neuromorphic camera tags offer a wide range of functionalities, offering battery-free wireless sensing for various physical stimuli, including temperature, contact, button presses, key presses, and even sound cues. Empirical evaluations demonstrate impressive accuracy at long ranges up to 200 feet. In addition to these findings, the disclosure describes consider a range of applications such as battery-free input devices, tracking of human movement, and long-range detection of human activities in various environments such as kitchens, workshops, etc. By reducing the reliance on batteries, the described neuromorphic camera tags promote eco-friendliness and opens doors to exciting possibilities in smart environment technology.

[0017] DESIGN CONSIDERATIONS There are several design considerations achieved using the NeuroCamTag system.

[0018] Self-Powered, Battery-and-Maintenance-Free Operation: In a smart environment, where sensing devices need to be pervasively deployed. The NeuroCamTag can operate for long time without maintenance, including self-powered and battery free operation, as batteries can utilize or require power maintenance.

[0019] Supporting Rich Wireless Input: The sensing devices can go beyond presence detection, buttons and support a rich array of wireless interactions. For example, continuous input events, touch, temperature, pressure, voice based.

[0020] Support for adaptability and no device programming: Lighting conditions in the built environment vary widely; to this end, the design of tags can support different energy levels by allowing modular energy harvesting. Besides lighting, the tags can in some examples require no hardware programming, i.e., tags that operate by simply assembling hardware components.

[0021] FIG. 1 Illustrates an example of a neuromorphic camera tag system. The system comprises two primary elements: the tag hardware and a software pipeline for processing input sensor data and detecting digital (hardware) tags. These tags utilize ambient light for power, they detect user interaction, sense environment, and transmit digital data wirelessly via light signals. The neuromorphic camera, positioned in a secluded, out-of-the way area with a clear view of the tags, such as on the wall or up in the corner of a room, is able to constantly monitor the tag’s activity. Upon receiving messages, the software algorithm first applies a filtering process. Following this, the decoding algorithm takes over, clustering messages from various digital tags to accurately extract and interpret the sensor data.

[0022] FIG. 2 illustrates an example of Hardware Design for a digital version of NeuroCamTag hardware, broken down into energy harvesting, energy management, storage, sensing, computing with ultra-low-power MCU, and wireless communication. FIG. 2 shows the layout of the circuit modules and the components chosen to optimize for self-powered operation. Energy Harvesting: A solar panel such as the IXOLAR SM141 K06L solar panel can be used to harvest ambient solar energy. This panel can have dimensions of 1 .654 inches X 0.906 inches X 0.083 inches and has a weight of 3.4 grams. It can be designed to deliver an open-circuit voltage of 4.15 volts, though actual performance can fluctuate depending on the intensity of available light.

[0023] A range of smaller solar panels, including models SM730K12L and KXOB25- 05X3F, which differ in both size and photocurrent generation can be utilized. Interactive user input necessitates both high-speed responsiveness due to constant sensing and continuous wireless communications capable of handling several messages per second, for example, for playing games like pong, using a battery-free wireless controller with knobs. The chosen solarpanel can meet a predetermined power requirements most effectively, offering an optimal balance of size, weight, power, and cost (SWAP-C) in relation to the predetermined message length.

[0024] Energy Management and Storage-. The power output of solar panels fluctuates due to the varying lighting conditions (e.g. changing sunlight) of the environment, as well as user interaction such as moving a battery-free input device. To provide stable voltage, a buckboost device such as LTC3808, BQ25570, etc. and finally chose AEM 10941 can be used. An ultra-low cold start voltage of 380mV can be used in some examples, start-up current of 7.8 A and a quiescent current draw of <1 A(lq), such as in the AEM10941. Finally, in addition to harvesting, capacitor based energy storage for battery-free operation. In other examples, a battery can be used. The circuits support capacitor banks that can be soldered. Tantalum capacitors (ranging 680pF to OAF) can offer low DC leakage properties and low equivalent series resistance (ESR). The choice of capacitor used depends on the bootup time vs storage capacity desired.

[0025] MCU and Computation: When a sufficient voltage (3.3v) is achieved by energy harvesting, the AEM10941 chip onboard can deliver power to an ultra-low-power 16-bit MCU MSP430G2553 that operates in five low power modes (LPM4, LPM3, LPM2, LPM1 , LPM0). The lowest mode (LPM4) can consume as little as 80pA. The MSP430 microcontroller or another lower power consumption microcontroller can be used along with compatible open- source firmware libraries and toolchains. The MCU can switch between the different LPM modes between input sensing and wireless communication. For example, pressing a button triggers an interrupt from LPM4 (deep sleep), waking up the MCU into its active state for wireless communication, and returning back to deep sleep mode (LPM4) to save power. Although LPM4 is the lowest, not all interactions can operate in that mode due to the clock requirement for an ADC read. Therefore, only button press interactions switch between LPM4, whereas other interactions using ADC read, such as a temperature, moving a slider, rotating a knob, etc., operate between LPM3 and the active state.

[0026] Encoding Input Data and Wireless Communication:

[0027] The MCU can employ a digital On-Off-keying (OOK) to transmit interactive events and sensor data by modulating LEDs wirelessly. The OOK modulation type can be used in part for its power-saving advantage — when transmitting the binary sequence of "1" and "0", the LED remains on for T and off for the representation of ’O’, conserving power. Specifically, the LED’s "on" time for a binary ’1 ’ can be calculated using formula 1 below.

[0028] This can provide significant energy savings, as no power is consumed during the ’0’ intervals. For instance, with a transmission rate of 1000 Bits Per Second (BPS), the LEDlights up for just 500 microseconds for each T bit. In the example depicted in FIG. 2, each digital NeuroCamTag message can include 16 bits divided into four distinct segments: a 4-bit header for synchronization, a 4-bit identifier for tagging up to 16 unique tags, a single-bit parity for error reduction, and a 7-bit payload for data representation. A 7-bit payload can be selected to accommodate a sufficient range of analog inputs (0-127), such as those from dials, sliders, or temperature sensors. A temperature reading of 76° can be transmitted as its 7-bit binary counterpart (1001 100). The 4-bit header can be optimized for message synchronization with the described decoding algorithm for the camera. Other numbers of bits can be provided for each segment in other examples.

[0029] Software Pipeline: With regard to FIG. 3 the camera software for decoding wireless transmission from digital NeuroCamTags is discussed. Neuromorphic cameras can capture encoded light signals as they can operate with high temporal resolution and detect changes within 100s of microseconds.

[0030] Filtering: Before the system begins processing the signal, the system can employ a spatiotemoral contrast (STC) filter to reduce noise. This can be provided in the camera device in some examples. An STC filter works by analyzing camera input contrast across a scene overtime and removes isolated events. This filter can be particularly useful in scenarios where both the spatial relationships between objects and their changes over time are important.

[0031] Decoding: After filtering, when an encoded message arrives, for instance with a LED flash representing a "1" bit, an event can be logged as e1 (p1 ,t1 ,x1 ,y1), with p1 = 1 representing the event polarity (1 is on, 0 is off), t1 the timestamp, and the x1 ,y1 coordinates. The LED turning off generates a second event e2(0,t2,%1 ,y1), where p2 = 0 marks the polarity of the falling edge as ’off at the same coordinate position of x1 ,y 1 . As seen in FIG. 3, the red blocks represent events with polarity p1 = 1 and green blocks represent events with p1 =0

[0032] As seen in FIG. 3, for each XY position (x1 , y1 ,), (x2,y2),(x3,y3) & (x4,y4), the camera software creates a double-ended queue for each coordinate pair, continuously adding new events as they arrive. Although not all scene pixels change simultaneously, significant movements prompt the creation of queues for all affected pixels. For a 1 megapixel camera, this means processing thousands of queues to manage the extensive data from the camera’s view.

[0033] While processing these events, as a first step, the camera software checks for events occurring outside their expected time interval. For example, at a 1000 BPS transmission rate, an event e(1 , t3, x1 , y1) where t3 exceeds a predefined time threshold (77i1) and prompts processing of all events from t1 = 0 to 77i1 at various coordinates, as shownin FIG. 3 at D. However, this processing doesn’t apply to locations like (x3,y3) and (x4,y4) (FIG. 3 at A & C) unless their event timings meet the Th1> threshold.

[0034] Subsequently, as a second step, the camera software inserts "0’s" in queues where significant time lapses follow a polarity shift from 0 to 1 or 1 to 1 , as depicted in FIG. 3 with long gray blocks. After this insertion, as a third step, the camera software pops events from each queue, checking for correct headers. Incorrect headers result in queue dismissal. As a fourth step, the camera software converts bitstreams into messages and verifying the parity bit, tailored to the payload’s evenness or oddness. Messages with incorrect parity are discarded.

[0035] As the fifth and final step, the software compiles message candidates from various coordinates. In some examples, the messages can undergo a k-means (or another type of) clustering process, selecting the most common message as the output. The k-means algorithm can be utilized for its computational efficiency, with the ’k’ value determined by the number of tags present in the scene. Please note that, similar to cluster 1 , event streams at other locations (x3,y3) and (x4,y4) are processed using the steps detailed above at cluster 2 when their event timings exceed a unique Th2> threshold.

[0036] FIG. 4 shows an analog version of the NeuroCamTag System. The analog NeuroCamTag can use a timer circuit such as a 555 timer with expandable slots for solar panels to adapt to varying light conditions. Unlike digital circuits that represent information using binary (0s and 1s), analog circuits represent information through frequency modulation. This approach allows for "programming" the circuits by adjusting passive elements like resistors and capacitors, directly supporting the described D3 design objective. Analog NeuroCamtags can be simple and cost efficient solutions, as some examples of these tags can require only eight components.

[0037] Backscatter analog techniques can be utilized and adapted to the design. In particular, the system can use ultra-low-power oscillators for frequency modulated sensing wireless communication of audio as well as resistance based sensors. While some designs can use this effect for backscatter, the present disclosure adapts the use oscillator circuits for light based frequency modulation.

[0038] On the camera side (event processing), the system can capture the frequency of the LED blinks, which mirror the FM signal. To isolate and extract FM signals from noise, a signal processing step can be employed, filtering based on the intended signal bandwidth. Following this filtering process, the signal can be segmented into discrete windows, allowing for the extraction of key features that can be harnessed to train a Machine Learning (ML) model for activity recognition.

[0039] FIG. 5 shows an Analog NeuroCamTag Schematic.

[0040] Circuit Implementation: Energy Harvesting and Management: Similar to the digital tag implementation, the analog counterpart can use solar panels (SM141 K06L) for self- sustaining power. The analog tag employs frequency modulation (FM), runs continuously, and supports power-intensive sensors like microphones, leading to customizable sizes.

[0041] A microphone-equipped tag can use six solar cells matched for its general power usage and small size, while a light-dependent resistor version requires only two cells and can also be smaller. The analog NeuroCamTag offers flexibility with 2, 4, or 6 solar panels. This design flexibility can enable the creation of the analog NeuroCamTag, featuring break lines on the PCB, allowing for configurations with 2, 4, or 6 solar panels.

[0042] Power from solar panels goes to a Low-Dropout Regulator (LDO) TPS7A0333DBVR with ultra-low quiescent current (IQ) of 200nA, ensuring signal stability. Each LDO port connects to a 1000pF ceramic capacitor for a fixed 3.3-volt output.

[0043] 555 timer oscillator and sensors The output voltage from the Low Dropout (LDO) regulator can be applied to an astable multivibrator circuit using a 555 timer, which acts as a resistance-to-frequency converter. A CSS555 timer can be used, which can be an ultra-low power timer that consumes a minimal 5 A at 1.2 V. This timer can be part of a resistorcapacitor network, consisting of resistors R1 and R2, and capacitor CT. The circuit generates oscillations at a specific carrier frequency and duty cycle, determined by the following relationships:1 44Oscillator Frequency: f = - -:- - (2)J 1((Kl+2Xfi2)XCT )„t„ (R1+R2)Duty Cycle: D — - - (3) J (R1 + 2XR2)v'

[0044] The carrier frequency and duty cycle are inversely proportional to the values of R1 , R2, and CT. Lower values of these components yield higher frequencies and shorter duty cycles, whereas higher values result in lower frequencies and longer duty cycles. These component values (R1 , R2, and CT) can be customized on the PCB, allowing the creation of new tags with distinct carrier frequencies or "IDs." This customization supports the identification of multiple tags operating on different carrier frequencies. In some examples tested, R1=4MQ, R2=470KQ and CT=1nF for a carrier frequencies of 270Hz can be used for the microphone tag.

[0045] In the circuit design, the input signal to the Control Voltage (CV) pin of the timer modulates the output frequency and duty cycle. This modulation depends on the characteristics of the signal applied to the CV pin. Various sensors, such as microphones, light-dependent resistors, or pressure sensors, can be connected to this pin. They produce afrequency-modulated (FM) voltage signal at the output of the timer, which can then be transmitted through an LED with a resistor, sending out the FM signal.

[0046] This section describes the software implementation for the wireless event processing pipeline for analog NeuroCamTags. Similar to the digital NeuroCamTag, an analog NeuroCamTag’s LED blinks generate events with polarity that indicate either ON or OFF states, triggered by changes in brightness from the tag’s LED. The tag’s frequency modulation dictates the frequency of these events, correlating to the LED signal’s frequency. The camera can record these events, capturing their timing and x,y co-ordinates accurately. To extract the events the system can use a camera API, in one implementation a time buffer of 2ms can be used or another timing that ensures all events captured within the predetermined time (e.g., 2ms) can be passed to further processing blocks in the software pipeline.

[0047] Frequency Detection, Clustering and Filtering. To process the events along with XY co-ordinates, each event to be converted can be timestamped into frequencies and cluster the frequencies to reduce redundancies. A series of low-pass and high-pass filters can extract the signal of interest. For instance, the analog microphone NeuroCamTag can use 270 Hz carrier, and a bandwidth from 250 Hz - 2KHz, to modulate and send data; therefore, other frequencies are filtered out. However, if an environment has other sensors, such as resistive sensors (such as potentiometers, sliders, light dependent resistor), they can occupy other parts of the available spectrum such as above 2Khz or below 250Hz, their bandwidth can be customizable depending on the resolution of the device desired.

[0048] Activity Recognition & Machine Learning. Machine learning can classify activities based on the FM signal received from an analog NeuroCamTag. In particular, an analog Microphone using Tag can capture audio and send FM data wirelessly to the machine learning pipeline. Upon receiving and demodulating the FM signal, the processing system can extract features from a sliding window of 4000 data points and compute the time domain and frequency domain features. For spectral features, a 128 point Fast Fourier Transform (FFT) overlapping 50% from the signal data window can be used. TSFEL, a feature extraction toolbox can be utilized create spectral features (e.g. continuous wavelet transform, the quantiles, binned entropy and etc.). In some examples, a predetermined set of features can be fed into the machine learning model along with verified correct classifications associated with the event data, for training and later recognition.

[0049] FIG. 6 shows a series of different activities and their associated retrieved spectrum content. Note the variations in the duration, frequency bands, and amplitudes (intensity of color) for each of the activities. For classification of activities and various user interactions, a Random Forest classifier can be used, among other options in other examples. For example,other models e.g., boosted trees or dense networks are compatible with analog NeuroCam- Tags and the present systems.

[0050] A random forest classifier in some examples can use Gridsearch() to systematically determine the number of estimators and the RF criterion (gini, entropy or log loss). Gridsearch can determine 60 as the optimal number of estimators and ’gini’ as the optimal criterion. These parameters can be cross validated 5 fold by mean to determine the desired parameters. Due to the variations in environmental conditions, and end-use (i.e., applications), the system can use an initial calibration to train the activity classier when first installed.

[0051] The systems can involve several interactive applications, which can include but are not limited to the following applications. Both digital and analog NeuroCamTags can involve at least: 1) Sensors and Interactive Devices 2) Multi-user, Multi-tag applications 3) Object Use Tracking in various environments.

[0052] The NeuroCamTags can include Input Devices including but not limited to: Buttons, Sliders, Knobs and Joystick. A variety of rich input devices can incorporate NeuroCamTags. These include inputs such as pressing a button, rotating a knob, sliding a control, and maneuvering joysticks across XY coordinates. Each device utilizes an ultra-low- power microcontroller unit (MCU) (in the digital NeuroCamTag). This setup operates on an interrupt-driven basis, toggling between low-power states and an active mode. Specifically, the button-tagged device can alternates between LPM4 and active state, whereas other input devices transition between LPM3 and active state. For input mechanisms like knobs and sliders, a basic resistive divider circuit can be employed to monitor voltage changes through the ADC, broadcasting the voltage at a rate of 3000 BPS or another rate. This process can occurs at a frequency of 7 messages per second, or another optimized message rate, ensuring the system remains sufficiently responsive for quick human interactions. In the case of the joystick input device, the tag can be activated from LPM3 to first measure and transmit the ADC value for the ’x’ axis, then pause briefly for 5ms before measuring and transmitting the ’y’ axis ADC value. This approach can effectively track both axes of the joystick, maintaining a practical transmission rate of nearly 4 messages per second.

[0053] Interactive Sensors: The systems can also include a variety of interactive sensors designed for monitoring smart environments. For example, in workshop or kitchen settings, the temperature sensors are capable of detecting temperature changes. Utilizing reed switches, the system can use an application for monitoring supply levels. This system records each interaction, keeps track of inventory, and sends out wireless notifications when restocking is desired. Additionally, some examples of the system can include smart moisture sensors to monitor the soil moisture around plants, using ADC readings from the MCU on the digital tags for wireless communication. The moisture data transmitted can be comparedagainst a pre-defined soil moisture thresh-old and alerts the user depending on needs. Additionally, incorporated pressure sensors can be included in the tags, which can be designed to log and monitor pressure as recorded by users.

[0054] The following electrical designs can also be used with the various versions of the system described. FIG. 7 shows a Passive Tag MEMS Mic Schematic and FIG. 8 shows a Digital Tag Schematic.

[0055] FIG. 9 is a schematic diagram illustrating an example of processing circuitry 1000 that can be used for neuromorphic camera tag systems, in accordance with various embodiments of the present disclosure. The processing circuitry 1000 can include at least one processor circuit having, for example, a processor 1003 and a memory 1006, both of which are coupled to a local interface 1009. The local interface 1009 may comprise, for example, a data bus with an accompanying address / control bus or other bus structure as can be appreciated. The processing circuitry 1000 can comprise one or more computing / processing device such as, e.g., a smartphone, tablet, computer, controller, etc. To this end, each processing circuitry 1000 may comprise, for example, at least one server computer or like device, which can be utilized in a cloud based environment.

[0056] In some embodiments, the processing circuitry 1000 can include one or more network interfaces 1012. The network interface 1012 may comprise, for example, a wireless transmitter, a wireless transceiver, and / or a wireless receiver. The network interface 1012 can communicate to a remote computing / processing device or other components using a Bluetooth, WiFi, or other appropriate wireless protocol. As one skilled in the art can appreciate, other wireless protocols may be used in the various embodiments of the present disclosure. The network interface 1012 can also be configured for communications through wired connections.

[0057] Stored in the memory 1006 are both data and several components that are executable by the processor(s) 1003. In particular, stored in the memory 1006 and executable by the processor 1003 can be a NeuraoCamTag application 1015 which can perform tag processing operations as disclosed herein, and potentially other applications 1018. In this respect, the term "executable" means a program file that can be in a form that can ultimately be run by the processor(s) 1003. Also stored in the memory 1006 may be a data store 1021 and other data. In addition, an operating system may be stored in the memory 1006 and executable by the processor(s) 1003. It is understood that there may be other applications that are stored in the memory 1006 and are executable by the processor(s) 1003 as can be appreciated.

[0058] Examples of executable programs may be, for example, a compiled program that can be translated into machine code in a format that can be loaded into a random access portion of the memory 1006 and run by the processor(s) 1003, source code that may beexpressed in proper format such as object code that is capable of being loaded into a random access portion of the memory 1006 and executed by the processor(s) 1003, or source code that may be interpreted by another executable program to generate instructions in a random access portion of the memory 1006 to be executed by the processor(s) 1003, etc. Where any component discussed herein is implemented in the form of software, any one of a number of programming languages may be employed such as, for example, C, C++, C#, Objective C, Java®, JavaScript®, Perl, PHP, Visual Basic®, Python®, Ruby, Flash®, or other programming languages.

[0059] The memory 1006 is defined herein as including both volatile and nonvolatile memory and data storage components. Volatile components are those that do not retain data values upon loss of power. Nonvolatile components are those that retain data upon a loss of power. Thus, the memory 1006 may comprise, for example, random access memory (RAM), read-only memory (ROM), hard disk drives, solid-state drives, USB flash drives, memory cards accessed via a memory card reader, floppy disks accessed via an associated floppy disk drive, optical discs accessed via an optical disc drive, magnetic tapes accessed via an appropriate tape drive, and / or other memory components, or a combination of any two or more of these memory components. In addition, the RAM may comprise, for example, static random access memory (SRAM), dynamic random access memory (DRAM), or magnetic random access memory (MRAM) and other such devices. The ROM may comprise, for example, a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or other like memory device.

[0060] Also, the processor 1003 may represent multiple processors 1003 and / or multiple processor cores, and the memory 1006 may represent multiple memories 1006 that operate in parallel processing circuits, respectively. In such a case, the local interface 1009 may be an appropriate network that facilitates communication between any two of the multiple processors 1003, between any processor 1003 and any of the memories 1006, or between any two of the memories 1006, etc. The local interface 1009 may comprise additional systems designed to coordinate this communication, including, for example, ultrasound or other devices. The processor 1003 may be of electrical or of some other available construction.

[0061] Although the NeuroCamTag application 1015, and othervarious applications 1018 described herein may be embodied in software or code executed by general purpose hardware as discussed above, as an alternative the same may also be embodied in dedicated hardware or a combination of software / general purpose hardware and dedicated hardware. If embodied in dedicated hardware, each can be implemented as a circuit or state machine that employs any one of or a combination of a number of technologies. These technologies may include, but are not limited to, discrete logic circuits having logic gates for implementingvarious logic functions upon an application of one or more data signals, application specific integrated circuits (ASICs) having appropriate logic gates, field-programmable gate arrays (FPGAs), or other components, etc. Such technologies are generally well known by those skilled in the art and, consequently, are not described in detail herein.

[0062] Also, any logic or application described herein, including the NeuroCamTag application 1015, that comprises software or code can be embodied in any non-transitory computer-readable medium for use by or in connection with an instruction execution system such as, for example, a processor 1003 in a computer system or other system. In this sense, the logic may comprise, for example, statements including instructions and declarations that can be fetched from the computer-readable medium and executed by the instruction execution system. In the context of the present disclosure, a "computer-readable medium" can be any medium that can contain, store, or maintain the logic or application described herein for use by or in connection with the instruction execution system.

[0063] The computer-readable medium can comprise any one of many physical media such as, for example, magnetic, optical, or semiconductor media. More specific examples of a suitable computer-readable medium would include, but are not limited to, magnetic tapes, magnetic floppy diskettes, magnetic hard drives, memory cards, solid-state drives, USB flash drives, or optical discs. Also, the computer-readable medium may be a random access memory (RAM) including, for example, static random access memory (SRAM) and dynamic random access memory (DRAM), or magnetic random access memory (MRAM). In addition, the computer-readable medium may be a read-only memory (ROM), a programmable readonly memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or other type of memory device.

[0064] Further, any logic or application described herein, including the NeuroCamTag application 1015, may be implemented and structured in a variety of ways. For example, one or more applications described may be implemented as modules or components of a single application. For example, the NeuroCamTag application 1015 can include a wide range of modules such as, e.g., an initial model or other modules that can provide specific functionality for the disclosed methodology. Further, one or more applications described herein may be executed in shared or separate computing / processing devices or a combination thereof. For example, a plurality of the applications described herein may execute in the same processing circuitry 1000, or in multiple computing / processing devices in the same computing environment. To this end, each processing circuitry 1000 may comprise, for example, at least one server computer or like device, which can be utilized in a cloud-based environment.

[0065] It should be emphasized that the above-described embodiments of the present disclosure are merely possible examples of implementations set forth for a clearunderstanding of the principles of the disclosure. Many variations and modifications may be made to the above-described embodiment(s) without departing substantially from the spirit and principles of the disclosure. All such modifications and variations are intended to be included herein within the scope of this disclosure and protected by the following claims.

[0066] Although specific terms are employed herein, they are used in a generic and descriptive sense only and not for purposes of limitation.

[0067] As will be apparent to those of skill in the art upon reading this disclosure, each of the individual embodiments described and illustrated herein has discrete components and features which may be readily separated from or combined with the features of any of the other several embodiments without departing from the scope or spirit of the present disclosure.

[0068] Any recited method can be carried out in the order of events recited or in any other order that is logically possible. That is, unless otherwise expressly stated, it is in no way intended that any method or aspect set forth herein be construed as requiring that its steps be performed in a specific order. Accordingly, where a method claim does not specifically state in the claims or descriptions that the steps are to be limited to a specific order, it is no way intended that an order be inferred, in any respect. This holds for any possible non-express basis for interpretation, including matters of logic with respect to arrangement of steps or operational flow, plain meaning derived from grammatical organization or punctuation, or the number or type of aspects described in the specification.

[0069] While aspects of the present disclosure can be described and claimed in a particular statutory class, such as the system statutory class, this is for convenience only and one of skill in the art will understand that each aspect of the present disclosure can be described and claimed in any statutory class.

[0070] It is also to be understood that the terminology used herein is for the purpose of describing particular aspects only and is not intended to be limiting. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the disclosed compositions and methods belong. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the specification and relevant art and should not be interpreted in an idealized or overly formal sense unless expressly defined herein.

[0071] Prior to describing the various aspects of the present disclosure, the following definitions are provided and should be used unless otherwise indicated. Additional terms may be defined elsewhere in the present disclosure.

[0072] As used herein, “comprising” is to be interpreted as specifying the presence of the stated features, integers, steps, or components as referred to, but does not preclude the presence or addition of one or more features, integers, steps, or components, or groupsthereof. Moreover, each of the terms “by”, “comprising,” “comprises”, “comprised of,” “including,” “includes,” “included,” “involving,” “involves,” “involved,” and “such as” are used in their open, non-limiting sense and may be used interchangeably. Further, the term “comprising” is intended to include examples and aspects encompassed by the terms “consisting essentially of’ and “consisting of.” Similarly, the term “consisting essentially of’ is intended to include examples encompassed by the term “consisting of.

[0073] As used in the specification and the appended claims, the singular forms “a,” “an” and “the” include plural referents unless the context clearly dictates otherwise.

[0074] It should be noted that ratios, concentrations, amounts, and other numerical data can be expressed herein in a range format. It will be further understood that the endpoints of each of the ranges are significant both in relation to the other endpoint, and independently of the other endpoint. It is also understood that there are a number of values disclosed herein, and that each value is also herein disclosed as “about” that particular value in addition to the value itself. For example, if the value “10” is disclosed, then “about 10" is also disclosed. Ranges can be expressed herein as from “about” one particular value, and / or to “about” another particular value. Similarly, when values are expressed as approximations, by use of the antecedent “about,” it will be understood that the particular value forms a further aspect. For example, if the value “about 10” is disclosed, then “10” is also disclosed.

[0075] When a range is expressed, a further aspect includes from the one particular value and / or to the other particular value. For example, where the stated range includes one or both of the limits, ranges excluding either or both of those included limits are also included in the disclosure, e.g. the phrase “xto y” includes the range from ‘x’ to ‘y’ as well as the range greater than x’ and less than ‘y’. The range can also be expressed as an upper limit, e.g. ‘about x, y, z, or less’ and should be interpreted to include the specific ranges of ‘about x’, ‘about y’, and ‘about z’ as well as the ranges of ‘less than x’, less than y’, and ‘less than z’. Likewise, the phrase ‘about x, y, z, or greater’ should be interpreted to include the specific ranges of ‘about x’, ‘about y’, and ‘about z’ as well as the ranges of ‘greater than x’, greater than y’, and ‘greater than z’. In addition, the phrase “about ‘x’ to ‘y’”, where ‘x’ and ‘y’ are numerical values, includes “about x’ to about ‘y’”.

[0076] It is to be understood that such a range format is used for convenience and brevity, and thus, should be interpreted in a flexible manner to include not only the numerical values explicitly recited as the limits of the range, but also to include all the individual numerical values or sub-ranges encompassed within that range as if each numerical value and sub-range is explicitly recited. To illustrate, a numerical range of “about 0.1% to 5%” should be interpreted to include not only the explicitly recited values of about 0.1% to about 5%, but also include individual values (e.g., about 1 %, about 2%, about 3%, and about 4%) and the sub-ranges(e.g., about 0.5% to about 1.1%; about 0.5% to about 2.4%; about 0.5% to about 3.2%, and about 0.5% to about 4.4%, and other possible sub-ranges) within the indicated range.

[0077] As used herein, the terms “about,” “approximate,” “at or about,” and “substantially” mean that the amount or value in question can be the exact value or a value that provides equivalent results or effects as recited in the claims or taught herein. That is, it is understood that amounts, sizes, formulations, parameters, and other quantities and characteristics are not and need not be exact, but may be approximate and / or larger or smaller, as desired, reflecting tolerances, conversion factors, rounding off, measurement error and the like, and other factors known to those of skill in the art such that equivalent results or effects are obtained. In some circumstances, the value that provides equivalent results or effects cannot be reasonably determined. In such cases, it is generally understood, as used herein, that “about” and “at or about” mean the nominal value indicated ±10% variation unless otherwise indicated or inferred. In general, an amount, size, formulation, parameter or other quantity or characteristic is “about,” “approximate,” or “at or about” whether or not expressly stated to be such. It is understood that where “about,” “approximate,” or “at or about” is used before a quantitative value, the parameter also includes the specific quantitative value itself, unless specifically stated otherwise.

[0078] As used herein, the terms “optional” or “optionally” means that the subsequently described event or circumstance can or cannot occur, and that the description includes instances where said event or circumstance occurs and instances where it does not.

[0079] Unless otherwise specified, temperatures referred to herein are based on atmospheric pressure (i.e. , one atmosphere).

Claims

CLAIMSTherefore, at least the following is claimed:

1. A neuromorphic camera tag device, comprising: an energy harvesting circuit; an energy management circuit; a wireless communication circuit; and a processing circuit, wherein the processing circuit utilizes a particular modulation type to wirelessly communicate event data to a neuromorphic camera using the wireless communication circuit.

2. The neuromorphic camera tag device of claim 1 , further comprising a sensor circuit comprising a sensor device, wherein at least a portion of the event data is based at least in part on data generated by the sensor device.

3. The neuromorphic camera tag device of any of claims 1 and 2, further comprising an input device circuit comprising an input device, wherein at least a portion of the event data is based at least in part on data generated by the input device.

4. The neuromorphic camera tag device of any of claims 1-3, wherein at least a portion of the event data indicates activity spectrogram data indicating at least one of a duration, a frequency band, and amplitudes for the neuromorphic camera tag device.

5. The neuromorphic camera tag device of any of claims 1-4, further comprising an energy storage device.

6. The neuromorphic camera tag device of any of claims 1-5, wherein the particular modulation type comprises On-Off Keying (OOK) modulation.

7. The neuromorphic camera tag device of any of claims 1-4, wherein the particular modulation type comprises frequency modulation.

8. A neuromorphic camera tag system, comprising: a neuromorphic camera; and a neuromorphic camera tag device communicatively coupled with the neuromorphic camera, the neuromorphic camera tag device comprising:an energy harvesting circuit; an energy management circuit; a wireless communication circuit; and a processing circuit, wherein the processing circuit utilizes a particular modulation type to wirelessly communicate event data to the neuromorphic camera using the wireless communication circuit.

9. The neuromorphic camera tag system of claim 8, wherein the neuromorphic camera tag device comprises a sensor circuit comprising a sensor device, wherein at least a portion of the event data is based at least in part on data generated by the sensor device.

10. The neuromorphic camera tag system of any of claims 8 and 9, wherein the neuromorphic camera tag device comprises an input device circuit comprising an input device, wherein at least a portion of the event data is based at least in part on data generated by the input device.11 . The neuromorphic camera tag system of any of claims 8-10, wherein at least a portion of the event data indicates activity spectrogram data indicating at least one of a duration, a frequency band, and amplitudes for the neuromorphic camera tag device.

12. The neuromorphic camera tag system of any of claims 8-11 , wherein the neuromorphic camera tag device comprises an energy storage device.

13. The neuromorphic camera tag system of any of claims 8-12, wherein the particular modulation type comprises On-Off Keying (OOK) modulation.

14. The neuromorphic camera tag system of any of claims 8-1 1 , wherein the particular modulation type comprises frequency modulation.

Images