A method and apparatus for performing operations based on sensor signal data

Abstract

A method (10) and apparatus (50) for performing an operation based on sensor signal data obtained from at least one sensor. A feature profile is generated (12) from the sensor signal data by a processor (54) of the apparatus (50) that matches a set of predetermined feature profiles. If the generated feature profile matches at least one of the set of predetermined feature profiles, the operation is performed (13) by the apparatus (50) based on the sensor signal data, and if the generated feature profile does not match at least one of the set of predetermined feature profiles, the operation is performed (14) by the apparatus (50) based on remote processing (40) of the sensor signal data (40). With the disclosed method, a balance of processing power, processing time, and reliable operation of the apparatus (50) is achieved without incurring additional cost of upgrading the apparatus (50).

Description

Technical Field

[0001] This disclosure generally relates to controlling operation through a device that enables sensors, and particularly to a processor-controlled method for performing operations based on sensor signal data, as well as processor-controlled devices, systems, and computer program products. Background Technology

[0002] Advances in sensor technology allow more and more devices to perform automated operations adapted to specific events based on sensor signal data obtained from one or more sensors, which are arranged to provide, but are not limited to, at least one of motion data, sound data, image data, video data, temperature data, humidity data, pressure data, brightness data, chemical composition or substance data, olfactory data, and tactile data.

[0003] These sensors are typically housed within the corresponding equipment, but can also be externally connected to it. The acquired sensor signal data is usually processed by processing equipment, such as a properly programmed general-purpose processor or application-specific integrated circuit included in the equipment.

[0004] As an example, motion detection sensors can be used to detect and measure acceleration and rotational forces in order to detect moving objects, thereby enabling the device to perform operations in response to specific types of motion. For instance, if the device is or includes lighting equipment, the operation could include igniting lights via the lighting equipment when someone is detected approaching the device, and avoiding lighting based on detected movement of a nearby waving tree, etc.

[0005] As another example, so-called environmental sensors can be used to measure various environmental parameters, such as ambient air temperature and pressure, lighting and humidity, to trigger alarms, for example, when processing the sensor signal data obtained in this way indicates an emergency.

[0006] Using multiple different sensors in a single device to improve the device’s intended and / or automated operation when a specific event to be detected from the sensor signal data occurs inevitably means that different types of raw sensor signal data will be processed by the device’s processing equipment, which may be heterogeneous in nature compared to single-type sensor signal data and may include a considerable amount of data to be processed.

[0007] For example, compared to detecting intruders in a museum at night based solely on sensor signal data from motion detection sensors, reliable detection of moving people in a garden with numerous trees and bushes presents a fairly complex and detailed data processing task by processing data from motion sensors, humidity sensors, and temperature sensors under rainy and windy conditions.

[0008] In reality, many sensor-equipped networks and / or user equipment or terminals have limited capabilities in terms of available and / or available hardware and software resources for processing collected sensor signal data.

[0009] For example, the increase in both the amount of sensor signal data and the complexity of its processing not only makes implementation difficult in currently available sensor-enabled devices, but may also significantly increase the cost of future devices.

[0010] Compared to processing equipment embedded in user or network devices for processing sensor signal data, the acquired sensor signal data can be transmitted to a remote computing system with computing devices that include substantial processing power and / or more sophisticated processing algorithms, such as desktop computers, workstations, or network servers. However, such a solution is limited in its application for several reasons.

[0011] The continuous exchange of sensor signal data between devices or terminals and remote computing devices requires relatively powerful signaling resources, which increases the power consumption of the devices and necessitates the extensive use of the intermediate telecommunications networks that the devices and remote computing devices need to connect to.

[0012] Furthermore, most operations performed by the device must be delivered promptly, sometimes in real-time or near real-time, which is difficult and / or costly to achieve when remote computing or processing devices are involved in processing the sensed data. This makes collecting sensor signal data through remote processes even more challenging.

[0013] Therefore, in existing and new processor-controlled devices, the efficient use and deployment of processing resources within the devices, and the ability of sensor-enabled devices to respond to and accurately and timely process sensor signal data based on the increased volume of (heterogeneous) sensor signal data to reliably detect specific events that enable the device to perform one or more expected operations, has become a challenging objective. Summary of the Invention

[0014] In a first aspect of this disclosure, the above and other objectives are achieved by a method comprising a device including a processor performing operations based on sensor signal data obtained from at least one sensor, the method comprising the following steps:

[0015] - The processor generates a feature profile from sensor signal data;

[0016] - The processor matches the generated feature contours with a predefined set of feature contours available to the processor;

[0017] - If the generated feature contour matches at least one of the predetermined feature contour sets, the device performs an operation under the control of the processor based on sensor signal data, and

[0018] - If the generated feature contour does not match at least one of the predetermined feature contour sets, the device performs the operation under the control of the processor based on remote processing of sensor signal data.

[0019] This disclosure is based on a hybrid approach to processing available sensor signal data, including a local sensor signal data evaluation mode and a remote sensor signal data evaluation mode. The device's operation based on the corresponding sensor signal data evaluation mode depends on whether the processing resources available locally on the device can reliably determine or detect a specific event.

[0020] Therefore, in the first step, the device's processor generates a so-called feature profile based on sensor signal data obtained from one or more sensors. For the purposes of this application, a feature is a unique technical feature or parameter representing useful and dominant information included or hidden in a sensor signal. A feature profile includes at least one feature. However, in practice, a feature profile will include several features and / or relationships between features. Features can be obtained by processing the sensor signal data according to a specific feature processing algorithm available to the processor.

[0021] In the next step, the feature profiles thus generated are matched with a predetermined set of feature profiles that are locally available to the processor. These predetermined feature profiles all represent information that can be reliably detected from the sensor signal data by processing resources (i.e., the processor and feature processing algorithms) that are locally available at the device.

[0022] In the case of matching, that is, when the generated feature contour corresponds one-to-one with or to a certain extent with at least one feature contour in the predetermined feature contour set, the device performs one or more expected operations based on sensor signal data and under the control of the processor.

[0023] However, if the generated feature profile does not match any feature profile in the predetermined feature profile set, the operation of the device depends on the result of remote processing of the sensor signal data. That is, the acquired sensor signal data is transmitted to a remote computing system, which has computing devices, such as desktop computers, workstations, or network servers, that include a greater amount of processing power and / or more sophisticated feature processing algorithms compared to the processing resources available locally on the device.

[0024] Using this solution, the obtained sensor signal data undergoes an assessment of its complexity or difficulty level, also known as adversary evaluation, to reliably detect or determine events sensed by at least one sensor.

[0025] Typically, specific operations of a device are performed based on an evaluation calculated from the processing resources available locally at or within the device, unless the local processing results indicate that the locally available processing resources are insufficient to determine whether the intended operation should be performed.

[0026] In other words, when the generated feature profile matches the predetermined set of feature profiles, the sensor signal data is considered relatively easy to evaluate with the limited or scarce processing resources available locally on the device, i.e., it is considered an easy counterpart, and the device can perform operations directly based on the sensor signal data.

[0027] Otherwise, if the generated feature contours do not match the predetermined set of feature contours, the evaluation of the sensor signal data is considered difficult and complex, i.e., a formidable adversary. In this case, the limited processing resources available locally on the device are considered insufficient to evaluate the sensor signal data, necessitating processing of the sensor signal data by more powerful processing resources located remotely from the device.

[0028] It is conceivable that sensor-enabled devices will be designed so that most of the operations and tasks the device performs, such as daily routines and procedures, should be processed and controlled based on the processing resources available locally on the device. Only in special circumstances, namely events that result in more complex and extensive sensing data, will the computing power of powerful back-end devices for processing sensor signal data be used.

[0029] In this way, the operation of the device is performed in a balanced manner in terms of processing and data transmission efficiency, operational accuracy, cost, and responsiveness.

[0030] In embodiments of this disclosure, the generation step includes: generating a feature contour by using a predefined data analysis algorithm that includes one of a transformation algorithm and a feature extraction algorithm, wherein the feature extraction algorithm includes a statistical data analysis algorithm.

[0031] Appropriate feature extraction algorithms can be used to advantageously process and evaluate features or dominant information directly corresponding to quantities sensed by the corresponding sensors, such as the measured maximum temperature, the average value of the measured temperature, and the standard deviation, etc. Evaluation of information hidden in the sensor signal data may require the use of data transformation algorithms, such as transforming the sensor signal data from the time domain to the frequency domain. For the purposes of this disclosure, different feature extraction and / or known data transformation algorithms can be applied.

[0032] Therefore, the feature profile according to this disclosure may rely on various features extracted directly from sensor signal data, or features available in a transformed version of the sensor signal data, such as features available in a domain different from the sensing sensor signal data.

[0033] As mentioned above, adversarial evaluation of sensor signal data is performed by limited or scarce processing resources located at or within the sensor-enabled device. However, running transformation algorithms that require a considerably high computational workload is not feasible when local computing and storage resources are limited.

[0034] To achieve the optimal balance between performance and resource economy, according to another embodiment of this disclosure, the device's local processor applies a Fast Fourier Transform (FFT) feature processing algorithm, wherein the generated feature profile is a frequency or spectral feature profile that includes a set of frequency components whose amplitude exceeds a predetermined threshold, and wherein the predetermined feature profile set includes a spectral feature profile defined by the number and frequency of the frequency components exceeding the threshold.

[0035] In terms of computational resource requirements in terms of both processing and storage capacity, FFT is considered a relatively simple data structure that represents useful physical information for evaluating acquired sensor signal data and detecting predetermined events.

[0036] According to embodiments of this disclosure, in the matching step, the generated feature contours are compared with a predetermined set of feature contours available to the processor.

[0037] Contour comparison is a relatively simple form of matching that does not require significant processing power and is therefore suitable for applications with limited processing resources. A predetermined set of feature contours can be obtained, for example, from multiple tests performed on a set of historical sensor signal data associated with one or more specific events, regardless of whether the sensor-enabled device is intended to perform the expected operation on those events.

[0038] The predefined feature profile can distinguish between the profile that is related to the target event on which the device is to perform an operation and the background event that does not indicate the event on which the device is to perform an operation.

[0039] Feature profiles can rely on a variety of features, either features extracted directly from sensor signal data, and / or features available in transformed versions of sensor data in different domains. Therefore, feature profile comparison can be designed to compare various features of the generated feature profile with corresponding features of each feature profile in a predetermined set of feature profiles.

[0040] A match can be determined when the generated feature contour corresponds one-to-one or to a certain extent with at least one feature contour in the predetermined feature contour set.

[0041] The matching step can use different criteria for different features. For example, when the frequency components in both the generated and predetermined spectral feature profiles are at the same frequency value, the matching in the frequency domain can be qualified as positive, while the matching between amplitudes is qualified as positive when one amplitude is higher than the other or within a specific range or percentage of the other amplitude.

[0042] To support the determination of matching feature contours, in another embodiment of this disclosure, the matching step includes determining whether the generated feature contours repeat within a predetermined time period.

[0043] For example, the generated feature profile may not ultimately be determined as a positive match. For such a generated feature profile, the sensor-enabled device will perform an operation (i.e., matching), but the feature profile is repeated for a period of time longer than the duration of the corresponding event.

[0044] Using this additional criterion, for example, the target event can be distinguished from the background event even more reliably.

[0045] In this disclosure, in order to conserve as much scarce or limited processing resources as possible at sensor-enabled devices, it is advantageous for the device to perform specific operations directly based on the outcome or result of the matching step. In the case of positive matching, this corresponds to the matching feature profile. In this case, the predetermined feature profile directly corresponds to one or more specific events that occur when the device needs to perform a specific operation.

[0046] If the generated feature contour does not match the predetermined feature contour set, then two different scenarios can be distinguished.

[0047] In the first scenario, the raw sensor signal data is remotely preprocessed and further evaluated by the device’s local processor to detect or determine specific events in which the device performs an operation, as disclosed above, by generating feature profiles from the preprocessing and matching them with a predetermined set of feature profiles.

[0048] In the second scenario, remote processing of sensor signal data can directly lead to the detection of specific events that may occur during the operation of the device.

[0049] In both scenarios, one or more anticipated operations are executed by the device under the control of the processor.

[0050] In embodiments of this disclosure, the step of the device performing an operation using sensor signal data further includes: applying an operation algorithm to the sensor signal data.

[0051] A relatively simple example of an operating algorithm is measuring the humidity level in a room or space regulated by an air conditioner to set appropriate parameters at the air conditioner before it begins operation, triggered by an event that the room temperature is higher than the set level, and determined, for example, based on a characteristic profile of the room temperature measurement.

[0052] Note that the manipulation algorithm can operate on a larger amount of other sensor signal data compared to the matching step, or vice versa. To conserve scarce resources, the manipulation algorithm used to generate feature contours and the data analysis algorithm can be combined into a single data processing algorithm.

[0053] In order to operate in remote or external sensor signal data evaluation mode, when the sensor signal data is considered a difficult adversary, the transceiver of the device performs the following steps before performing device operations based on remote processing of the sensor signal data:

[0054] - Transmit sensor signal data to a remote computing device, and

[0055] - Receive processed sensor signal data from a remote computing device, obtained by applying at least one processing algorithm to the transmitted sensor signal data by the remote computing device.

[0056] For the purposes of this disclosure, the step of receiving processed sensor signal data includes: receiving processed or preprocessed sensor signal data and / or receiving one or more results of remote processing of the sensor signal data.

[0057] Compared to the operating algorithms applied by the device's own processor, at least one processing algorithm applied by a remote computing device may be more sophisticated. In processing complex sensor signal data, such as data acquired under conditions of strong noise or interference, a more sophisticated algorithm will outperform a simpler one. Furthermore, multiple sophisticated operating algorithms can be used to process complex sensor signal data, and the final result can be based on a vote from several algorithms, such as a majority vote.

[0058] Therefore, this method allows for remote processing of sensor signal data that has become complex for various reasons, such as mixed data obtained from multiple different sensors, and / or irregular data obtained under extreme atmospheric or weather conditions, in order to produce more reliable results by utilizing stronger processing capabilities not available on the device itself.

[0059] The transceiver's transmission or delivery of sensor signal data may depend on the transceiver's available communication resources. Therefore, in embodiments of this disclosure, transmitting sensor signal data by a transceiver includes transmitting a representation of the sensor signal data to a remote computing device.

[0060] In other words, all raw sensor signal data or data samples obtained within a certain time window can be transmitted to remote or back-end computing devices, especially when there are no restrictions on data transmission rates.

[0061] However, raw sensor data samples can be downsampled by a processor locally available within the device to form new data samples representing the raw sensor signal data, ensuring that the transmission of these new data samples does not exceed the data transmission rate limit. For example, upon receiving new data samples, a remote or back-end computing device / multiple remote or back-end computing devices need to reconstruct the original data samples using classic signal reconstruction techniques.

[0062] When the device's processor has the capability to compute data compression, the original data sample is compressed to form a new data sample representing the original sensor signal data, for example, such that the transmission of the new data sample will not exceed the limit of the need to reconstruct the original data sample using classical data decompression techniques.

[0063] The methods disclosed herein can be used to operate a device, or for a sensor-enabled device to perform one or more operations based on, but not limited to, sensor signal data including, at least one of, motion data, sound data, image data, video data, temperature data, humidity data, pressure data, brightness data, chemical composition or substance data, olfactory data, and tactile data. Depending on the sensor and the data available from the sensor, the device can perform different operations.

[0064] In a particular embodiment of this disclosure, the device includes a lighting apparatus, which includes at least one lighting module, particularly a light-emitting diode (LED), and wherein operation performed by the device includes controlling the lighting by the at least one lighting module.

[0065] Lighting equipment can have various sensors connected to or even integrated with it, which can measure a variety of conditions or parameters, such as ambient sound, movement, temperature, luminescence, and so on. As explained above, regardless of whether an operating algorithm is applied, the lighting of the equipment can be controlled based on the measured sensor data.

[0066] In a second aspect of this disclosure, an apparatus is provided for performing operations based on sensor signal data obtained from at least one sensor, the apparatus including a processor and a transceiver, wherein the processor and transceiver are arranged to perform processor-controlled operations according to the method of the first aspect of this disclosure described above.

[0067] The device may include at least one sensor arranged to provide at least one of the following: motion data, sound data, image data, video data, temperature data, humidity data, pressure data, brightness data, chemical composition or substance data, olfactory data, and tactile data.

[0068] In a particular embodiment of this disclosure, the device includes at least one lighting module, particularly a light-emitting diode (LED), wherein the operation performed by the device and controlled by the processor includes controlling the lighting by at least one lighting module.

[0069] Lighting can be controlled by a lighting module by changing or setting light intensity, color, geographical direction of emitted light, and specific operating modes, such as flashing.

[0070] Those skilled in the art will appreciate that the device may also include one or more cameras, sound alarms, etc., to operate based on the acquired sensor signal data.

[0071] In a third aspect of this disclosure, a system is provided, including an apparatus according to a second aspect of this disclosure and a remote or back-end computing device, wherein the remote back-end computing device according to a first aspect of this disclosure is arranged to:

[0072] - Receive sensor signal data from the device, i.e., the device that enables the sensor;

[0073] - The received sensor signal data is preprocessed by applying at least one preprocessing algorithm, and

[0074] - Transmit the pre-processed sensor signal data to the device.

[0075] In a fourth aspect of this disclosure, a computer program product is provided that includes program code stored on a computer-readable medium, the program code being arranged to perform the methods described in the first aspect of this disclosure when executed by at least one processor.

[0076] The above and other aspects of this disclosure will become clear and understood with reference to one or more non-limiting example embodiments described below. Attached Figure Description

[0077] Figure 1The flowchart-type diagram illustrates a method for a device to perform operations based on sensor signal data obtained from at least one sensor, according to the present disclosure.

[0078] Figure 2 The schematic diagram illustrates the steps of a Fast Fourier Transform (FFT) sensor signal data processing algorithm for generating spectral feature profiles according to an embodiment of the present disclosure.

[0079] Figure 3a and Figure 3b The illustration shows two typical easy adversaries in motion detection according to embodiments of the present disclosure, which are obtained from spectral feature profiles generated by applying FFT to motion sensor signal data.

[0080] Figure 4a , Figure 4b and Figure 4c The illustration depicts three difficult opponents in motion detection according to an embodiment of the present disclosure, the three difficult opponents being obtained from spectral feature profiles generated by an FFT applied to motion sensor signal data.

[0081] Figure 5 The schematic diagram illustrates an embodiment of a method for processing sensor signal data on a back-end computing device using various algorithms, according to an embodiment of the present disclosure.

[0082] Figure 6 The schematic diagram illustrates a microwave motion detection system according to an embodiment of the present disclosure, the system including an end-point or node device and a back-end computing device.

[0083] Figure 7 The processing of motion sensor signal data of an easy-to-use hand is illustrated graphically according to an embodiment of the present disclosure.

[0084] Figure 8 The processing of motion sensor signal data of a difficult opponent, according to an embodiment of the present disclosure, is illustrated graphically.

[0085] Figure 9 The processing of motion sensor signal data of a type of difficult opponent, according to an embodiment of the present disclosure, is illustrated graphically. Detailed Implementation

[0086] The present disclosure is described in detail below with reference to methods of operation performed by motion detection devices, and to motion detection devices and systems including motion detection devices and remote or back-end computing devices, wherein the motion detection device is a device for acquiring motion sensor signal data. Those skilled in the art will appreciate that the present disclosure is not limited to operation based on motion detection sensor signal data, but is equally applicable to operation performed by a wide variety of sensor-enabled devices, as indicated in the summary section.

[0087] Throughout this specification, the terms “sensor signal data,” “sensor data,” and “signal” are used interchangeably.

[0088] Typical mainstream motion detection devices currently include one or more microwave motion detection sensors and a microcontroller unit (MCU) and data processing unit integrated into a single motion detection device. The sensors are arranged to collect raw sensor signal data by sensing motion in the environment surrounding the device. The raw data is collected, either wired or wirelessly, for processing and evaluation by one or more data processing algorithms running on the MCU, thereby determining or detecting detection results or events for the device to perform the intended operation.

[0089] A high-quality motion detection device (e.g., for outdoor use) is expected to produce correct detection results, and any operations performed in response to motion detection will not be negatively affected even under adverse or extreme conditions—such as rain and / or the presence of interference or noise caused by moving objects, for example, due to wind.

[0090] Figure 1 The flowchart-type diagram illustrates a method 10 according to the present disclosure for a device, such as a motion detection device, to perform operations based on sensor signal data obtained from at least one sensor, such as a motion sensor.

[0091] In step 11, “Generating feature profiles from sensor signal data”, the device’s processor (such as an MCU or microprocessor) generates feature profiles from sensor signal data collected from at least one sensor.

[0092] Predefined data analysis algorithms, such as statistical feature extraction algorithms or sensor signal data transformation algorithms, are used to generate feature profiles that transform sensor signal data from one domain to another, such as from the time domain to the frequency domain or the spectral domain.

[0093] The Fast Fourier Transform (FFT) algorithm can be considered the first choice for generating spectral feature profiles from motion detection sensor signal data, achieving an optimal balance between performance and economy of processing resources, which may be scarce or limited in end devices.

[0094] The FFT spectrum provides a clear physical meaning for understanding the relationship between the detected event and the sensor signal data. The relatively simple data structure of the FFT is also suitable for computations in subsequent processing to be performed by a processor, such as classification and manipulation algorithms, for further processing of the sensor signal data.

[0095] Spectrum generation using FFT is well known to those skilled in the art, and Figure 2 The diagram illustrates this. The FFT algorithm 15 includes: segmenting the sensor signal data into frames 16 of fixed length, removing offset or DC components from the frame data 17, applying or adding a window function to the data frame 18, and then performing an FFT on the data frame 19. Furthermore, window functions can be used in signal processing or statistics to mitigate spectral leakage effects. Window functions can include one of the following: rectangular window, Hamming window, Parzen window, Blackman window, etc.

[0096] Repeat steps 16-19 for new samples or sets of sensor signal data.

[0097] Back Figure 1 After generating the feature contours, in step 12, the question is asked: "Does the generated feature contour match the predetermined feature contour?" "" is the device's processor checking whether the generated feature profile matches a predefined set of feature profiles available locally to the processor.

[0098] This step performs a so-called adversary assessment or difficulty assessment on the sensor data to establish the level of difficulty for the algorithm running on the MCU, thereby enabling the processing of sensor signal data to provide the correct detection result or event, such as detecting a person moving in a garden at night on a rainy day. An easy adversary exists if the device's processing resources are sufficient to provide the correct output; otherwise, a difficult adversary exists. For this purpose, the generated feature profile is matched against this predetermined set of feature profiles.

[0099] The predetermined set of feature profiles represents information or parameters that can be reliably detected from sensor signal data obtained by the device’s processor using a spectral feature processing algorithm (i.e., FFT operation 15), which is locally available to the processor, meaning the processor is capable of running it.

[0100] The predetermined set of spectral feature profiles can be obtained by analyzing historical sensor data to determine the differences between background features / spectrum and target features / spectrum.

[0101] For motion detection sensors, two main types of events can be distinguished: one or more motion events that are to be targeted, and one or more background events. For example, a motion or target event is an event or occurrence in which the sensor signal data carries distinguishable information or features for detecting the corresponding moving object. Background events or occurrences are characterized by features not found at the moving object to be detected, such as information that is presented sequentially in subsequent sensor signal data samples or data frames.

[0102] Those skilled in the art will appreciate that the detection workload for detecting or determining background events involving steady-state or pseudo-steady-state conditions may differ from the detection workload required for detecting or determining target events based on sensor signal data, depending on whether the characteristics typically indicative of the target event will become readily available from the analysis or evaluation of the sensor signal data.

[0103] Figure 3a and Figure 3b Two typical examples of easy opponents in motion detection are illustrated graphically, using FFT as the feature analysis algorithm. Figure 3a and Figure 3b China, and in Figure 4a , Figure 4b and Figure 4c In the diagram, the horizontal axis represents the frequency in Hz, and the vertical axis represents the normalized amplitude or magnitude of the spectral components.

[0104] Figure 3a It is a typical background spectral feature profile generated by performing FFT on sensor signal data collected within a specific time period, and Figure 3b It is a typical motion spectrum feature profile generated from sensor signal data, representing the target event, i.e., the moving object to be detected.

[0105] exist Figure 3a In the exemplary background spectrum, the spectrum comprises three frequency components 21, 22, and 23, hereinafter referred to as peaks, each with a magnitude significantly higher than the other spectral components. The peaks lie in the frequency range of approximately 25 Hz–80 Hz, encompassing the so-called background frequency bins around 25 Hz, 55 Hz, and 80 Hz. In contrast, in Figure 3b In the motion spectrum, there is only one dominant peak 24 at a frequency of approximately 20 Hz, whose magnitude is overwhelming, significantly exceeding the magnitudes of other spectral components. Peak 24 is located outside these background frequency bins.

[0106] like Figure 3a and Figure 3bAs illustrated in the figure, any feature profile generated from sensor signal data that matches any predetermined feature profile included by two spectra (e.g., the following predetermined feature profile, which includes a total of three peaks at approximately 25 Hz, 55 Hz or 80 Hz for inferring background events, and a peak at approximately 20 Hz for determining target events) is relatively easy to process for identifying target motion events.

[0107] However, spectral / feature profiles may emerge, making it more difficult to determine whether a target event exists based on these profiles. Figure 4a , Figure 4b and Figure 4c An example representing three difficult opponents.

[0108] exist Figure 4a In the motion spectrum feature contour shown, with Figure 3a Compared to the background spectral feature profile, there are three dominant peaks 31, 32, and 33 with similar magnitudes in the background frequency bins, but... Figure 3b Compared to the spectral characteristic profile, there is another dominant peak 34 at around 10 Hz, which exceeds the background frequency bin, and its peak value is not overwhelming, that is, it does not significantly exceed other spectral components in the spectrum.

[0109] The underlying truth results of many tests show that it is difficult to indicate... Figure 4a Does the spectrum represent the target event? It is shown that the ratio between background events and the motion or target event represented by such a spectrum is approximately 4:6. Therefore, using... Figure 3a and Figure 3b The feature contours are used as a predetermined set of feature contours for matching. Figure 4a The characteristic contours will not easily provide correct or reliable output.

[0110] Figure 4b The spectral profile shown has only one dominant peak of 35 Hz, which is overwhelming in magnitude, but it lies within one of the background frequency bins, around 55 Hz. The underlying truth results from numerous experiments show that the ratio between background events and target or moving events with this spectrum is approximately 5:5. According to the Doppler effect, a specific velocity of motion could potentially trigger a peak around 55 Hz, which is consistent with... Figure 3a The background frequency bins are consistent. Therefore, if the algorithm outputs a background event when the spectrum is satisfied during the matching step, it is very likely to obtain an incorrect result.

[0111] Figure 4cThe image shows another type of frequency distribution spectrum representing a difficult opponent from motion sensor data. The test was conducted under adverse weather conditions, with strong winds and raindrops adding significant interference to the raw sensor data. In this case, there is only one dominant peak 36 at approximately 10 Hz, whose magnitude is overwhelming, meaning its value significantly exceeds that of the other spectral components, and this peak is not near any background frequency subdivision.

[0112] Use from Figure 3a and Figure 3b Matching the predefined feature profile derived in the algorithm will very likely lead to erroneous output, specifically targeting the wrong event, because peak 36 is not located in the background event bin. In fact, the underlying ground truth results from the tests show that this spectrum is simply from the background spectrum of the rainstorm weather. However, it is still possible to correct the algorithm output because... Figure 4c The persistent appearance of a characteristic profile may lead to the detection of events with an unusually long duration (e.g., ranging from minutes to hours), which is unlikely to be a moving target event, such as a thief or a person passing by a lamppost.

[0113] From the example above, we can imagine that the predetermined set of feature contours, for example, includes... Figure 3a and Figure 3b The spectrum illustrated in the diagram, or any other contour shown, can be used for relatively simple evaluation algorithms. For example, a matching step may be used, in which the number, location, or frequency of peaks in a generated FFT spectral feature contour is compared with the number, location, or frequency of peaks in a predetermined spectral feature contour (i.e., frequency binning) to determine a target event, which can be used and processed using local processing resources at the device.

[0114] Back to reference Figure 1 If step 12 determines that the generated feature profile matches the predetermined set of feature profiles, i.e., the result is "Yes (Y)," then the device performs the operation "Perform an operation using sensor signal data" at step 13. For example, turning on safety lights or lampposts based on the detection of a moving person. Note that, in the context of this specification and claims, not turning on safety lights or lampposts is also considered an operation to be performed by the device.

[0115] Therefore, the processing or evaluation of the characteristic profiles that generate sensor signal data is initially performed locally on the device, such as by applying a relatively simple algorithm running on an MCU. This algorithm aims to detect events through lightweight computation and simple logic analysis. As explained above, such an algorithm can be designed based on empirical studies of the relationship between underlying truth events and corresponding data features.

[0116] Based on the example above, using the FFT spectrum generated as a representation of spectral feature profiles, the number of spectral components with dominant values—that is, the number of peaks, the frequency binning of the peaks, and the relative values ​​of the optional dominant peaks—can be used as classification rules to compare each feature of the generated profile with its corresponding threshold, and a final decision is made by combining all the comparison results. The device then performs operations based on the final decision.

[0117] However, when Figure 1 If the result of step 12 is negative, i.e., "No (N)", then in step 14, "the operation is performed using remotely pre-processed sensor data". The sensor data is first processed at a computing device remote from the terminal device because, as in the example above, processing sensor data locally can be a challenging task for the device, as correctly interpreting sensor signal data requires more processing power than is available locally on the terminal device. Thereafter, the operation is performed by the device relying on the sensor data processed by the remote computing device.

[0118] Remote computing devices can be backend machines with higher computing power, speed, and larger storage capacity compared to local device processors. It is possible to run numerous algorithms on a backend machine to produce more reliable evaluation outputs than those possible with simple algorithms running on an MCU.

[0119] It's understandable that no single algorithm can outperform others on all problems. Strong algorithms running on remote backend machines primarily provide two functions: generating diverse and complementary features from large datasets, and using multiple sophisticated classification algorithms to vote on the final decision.

[0120] Figure 5 The schematic diagram illustrates an embodiment of a method for processing sensor data using multiple algorithms at a back-end computing device. This example illustrates the processing of data collected by a motion detector.

[0121] When considering features, data frame 41 is the basic data unit to be evaluated. Some representative features that can be used to evaluate sensor signal data are, for example:

[0122] - The arithmetic mean of all feature values ​​in the data frame;

[0123] - Standard deviation of each feature value on the data frame;

[0124] - The median in a data frame, for example, by sorting the individual feature values ​​in ascending order and returning the middle one;

[0125] - The maximum value in a data frame, for example, by sorting the individual feature values ​​in ascending order and returning the last one;

[0126] - The semaphore value region on the data frame, that is, sorting the individual feature values ​​in ascending order and returning the first feature value;

[0127] - The average sum of the squared magnitudes of each feature on the data frame;

[0128] - The entropy of the distribution of corresponding feature values ​​on the data frame;

[0129] - Iquartile range;

[0130] - Fourth-order Burg autoregressive coefficient;

[0131] - Maximum spectral component;

[0132] - Weighted average of the spectrum signal;

[0133] - Spectrum generated by continuous wavelet transform.

[0134] The number of data features listed above can be processed using several standard tools, including but not limited to Support Vector Machines (SVM), neural networks, AdaBoost, and random forests, to independently select feature combinations 42 for processing. This processing may include, for example, classifying the features using corresponding classification algorithms 43. Each classification algorithm 43 may provide a classification after the training phase. For example, in the decision phase of the corresponding algorithm, for the same data frame, if each processing algorithm determines that a target event exists, it outputs "1"; otherwise, it outputs "0". A final decision 44 is then made by weighting the votes of each classification algorithm.

[0135] The final decision 44 can be transmitted to the terminal device to execute the corresponding operation.

[0136] Instead of having decisions made directly by a remote computing device, raw sensor signal data can be preprocessed by the remote computing device and further evaluated by the device’s local processor to detect or determine specific events that enable the device to perform operations.

[0137] In other words, as disclosed above, the local processor MCU can generate feature profiles based on preprocessed data and match them with a predetermined set of feature profiles.

[0138] Those skilled in the art will appreciate that sensor signal data can be transmitted from the terminal device to the back-end remote computing device in different ways, enabling the aforementioned processing or preprocessing of the sensor signal data to be performed.

[0139] In the first approach or mode, all raw sensor data samples are sent to a back-end computing device, for example, if there are no limitations on the data transfer rate. The back-end computing device can therefore directly obtain the complete raw data samples without any further processing.

[0140] In the second approach or mode, the original sensor data samples are downsampled to form new data samples, ensuring that the transmission of these new data samples does not exceed the data transmission rate limit. For example, upon receiving new data samples, the backend computing device can reconstruct the original data samples using classic signal reconstruction techniques.

[0141] In the third approach or mode, if the computational complexity of data compression is acceptable to the MCU, the original data sample is compressed by the MCU to form a new data sample, ensuring that the transmission of the new data sample to the remote computing device does not exceed the data transmission rate limit. For example, upon receiving the new data sample, the backend computing device can reconstruct the original data sample using classic data decompression techniques.

[0142] Note that the operations performed by the device can be directly based on the results of the matching step, i.e. Figure 1 Step 12 in the process, or the result provided by the remote computing device, i.e., step 14, or the operation based on the result of an additional operational algorithm that will be applied by the device’s processor or MCU to the raw sensor signal data or the remotely preprocessed sensor signal data.

[0143] To save processing resources, the operation algorithms and data analysis algorithms used to generate feature contours can be combined into a single data processing algorithm.

[0144] Figure 6 A motion detection system 80 is schematically illustrated, comprising a terminal device or network node 50 and a backend computing device or server 60. The backend computing device 60 is arranged remotely from the terminal device 50. Those skilled in the art will appreciate that the motion detection system 80 may also include multiple terminal devices or network nodes and multiple backend computing devices.

[0145] Terminal device 50 and back-end computing device 60 include transceivers (i.e., Tx / Rx units) 51 and 61, respectively arranged for wireless 52 and 62 and / or wired 53 and 63 data exchange. Terminal device 50 and back-end computing device 60 are operatively and communicatively connected via Tx / Rx units 51 and 61, or directly and / or via network 90, as indicated by double arrows 66 and 91 respectively, for exchanging data, such as sensor signal data, between terminal device 50 and back-end computing device 60.

[0146] Communication with terminal device 50 can be performed, for example, through an intermediate network gateway device (not shown).

[0147] The terminal device 50 further includes at least one processor 54, such as a microprocessor μP or MCU, which operatively and communicatively interacts with and controls the transceiver 51 via an internal data communication and control bus 58.

[0148] At least one processor 54 may include a feature profile generation module 55 configured to generate feature profiles, such as frequency or spectral feature profiles, from sensor signal data collected 74 and stored 73 in a local data storage device 59 by at least one sensor 75, which may be part of or externally operably connected to the terminal device 50. The at least one processor 54 further includes a feature profile matching module 56, such as a frequency or spectral feature profile matching module, configured to perform an adversary evaluation based on predetermined feature profiles 76 stored in the local data storage device 59 of the terminal device 50, so as to determine whether an operation performed by the terminal device 50 is executable based on the processor 54's evaluation of the sensor signal data alone. The at least one processor 54 may also optionally include an operation module 57 configured to perform operations by the terminal device 50 directly based on sensor data or based on sensor data processed by a remote back-end computing device 60.

[0149] Terminal device 50 may be part of or operatively connected to lighting device 70, which includes lighting module 71, preferably an LED lighting module, whose operation is controlled by at least one processor 54 from or through a network gateway or by a remote control device (not shown).

[0150] The back-end computing device 60 further includes at least one processor 64 and a storage device 65, arranged to process and store sensor signal data received from the terminal device 50 using one or more algorithms 67 (such as those disclosed in conjunction with FIG. 4). The processing results of the remote back-end computing device 60 are forwarded to the terminal device 50 via the Tx / Rx unit 61 of the back-end computing device 60.

[0151] When the lighting device 70 is connected to 72 or is part of the terminal device 50, the operation of the terminal device 50 may include controlling the lighting through the lighting device, such as turning it on / off, dimming it, changing the color of the lighting module 71, etc., depending on the evaluation of the sensor signal data obtained by the sensor 75.

[0152] Specific examples are described below to illustrate how the methods of this disclosure can be used to process sensor data so as to generate accurate detection results.

[0153] When sensor data is received from a sensor such as a motion detection sensor, the opponent evaluation process performed by the device's processor is executed step by step as follows:

[0154] Step 1: Sample the sensor data at a sampling rate of approximately 1 kHz to generate data samples. To provide data frames, a sliding window is determined to have a length of 1024 sample points, and a sliding step size of 100 sample points is selected.

[0155] Step 2: Set 100 Hz, 200 Hz, and 300 Hz as background bins. For a spectrum generated from the data sample, if there are exactly three dominant peaks, and they are all in the background bins, the spectrum is classified as a background spectrum. If there is only one dominant peak, and that peak is outside the background bins, the spectrum is classified as a motion spectrum. Any other spectral patterns are evaluated and classified as difficult adversaries.

[0156] Step 3: Perform FFT on the data samples in the sliding window to obtain a 512-point spectrum.

[0157] Step 4: Find the highest value in the spectrum and make it the largest dominant peak.

[0158] Step 5: If possible, find other dominant peaks in the spectrum. The characteristics are as follows: the height of the possible peak must be no less than 40% of the maximum peak, the distance between two adjacent peaks must be at least 10 Hz, and there must be no other peaks with similar heights in the vicinity of the possible peak.

[0159] Step 6: Count the number of dominant peaks and check whether the dominant peaks are located in the background bins.

[0160] The above steps will now be explained in conjunction with the evaluation of three exemplary sensor data, where both background and motion signals are processed. Those skilled in the art will appreciate that, for example, between steps 1 and 2 above, the processor may optionally perform a type of preprocessing on the raw sensor signal data to remove noise, interference, and artifacts, thereby obtaining clean data for further processing.

[0161] Figure 7 The illustration shows the processing of sensor data from a motion detection sensor measured under good weather conditions, and represents an easy opponent. Figure 8 and Figure 9The diagram illustrates the processing of motion sensor data, presenting a formidable opponent. In all three figures, the top plot or graph represents a segment of raw sensor signal data collected by a microwave sensor in the time domain t(s), the middle plot represents the FFT background spectrum in the frequency domain f(Hz), and the bottom plot represents the motion spectrum f(Hz). In each of the background and motion spectra, the amplitude M of the peaks plotted along the vertical axis is normalized to the highest peak value. For clarity, the vertical axis of the raw sensor signal data in the top plot indicates arbitrary units of signal strength S.

[0162] exist Figure 7 In this diagram, the duration of the original sensor signal is approximately two minutes. Based on the baseline truth record, the signal in box 101 is the background signal, and the signal in box 102 is the target motion signal. The background and motion spectra shown in the middle and lower plots, respectively, are output through step 3 described above.

[0163] In the background spectrum shown in the intermediate plot, the output of step 4 shows the largest peak 104 at 200 Hz with a height or magnitude of 1.0. The output of step 5 shows two additional dominant peaks 103 and 105 at 100 Hz and 300 Hz, respectively, with heights of 0.47 and 0.77. The output of step 6 shows exactly three dominant peaks on the background sub-box.

[0164] In the motion spectrum shown in the plot below, the output of step 4 shows a maximum peak 106 at 10 Hz, with a height or amplitude of 1.0. The output of step 5 shows no other major peaks. The output of step 6 shows only one dominant peak 106, which is not on any background sub-bins. Therefore, Figure 7 The scenario depicted in the diagram represents an easy opponent.

[0165] Figure 8 The term "difficult opponent" refers to an opponent with favorable weather conditions. The duration of the raw sensor signal collected by the microwave sensor in the above plot is approximately 3 minutes. According to the baseline truth record, the signal in box 111 is the background signal, and the signal in box 112 is the motion signal. Figure 8 The background and motion spectrum shown in the middle and lower plots are output through step 3 above.

[0166] In the background spectrum shown in the intermediate plot, the output of step 4 shows a maximum peak 115 at 300 Hz with a height or magnitude of 1.0. The output of step 5 shows two additional dominant peaks 113 and 114 at 100 Hz and 200 Hz, respectively, with amplitudes of 0.97 and 0.98. The output of step 6 shows exactly three dominant peaks on the background sub-box.

[0167] In the motion spectrum shown in the plot below, the output of step 4 shows a maximum peak 116 at 55 Hz with a height of 1.0. The output of step 5 shows five other dominant peaks 117, 118, 119, 120, and 121 at 16 Hz, 100 Hz, 128 Hz, 200 Hz, and 300 Hz, with heights of 0.88, 0.68, 0.93, 0.91, and 0.85, respectively. The output of step 6 in the plot below shows six dominant peaks, and three of them, 118, 120, and 121, are on the background bins.

[0168] Based on the aforementioned competitor assessment, it is difficult to base on Figure 8 The motion spectrum shown in the plot below is used to determine the presence of a moving target event. Therefore, sensor data will need to be processed at the backend computing device or machine before the end device (i.e., the motion detector) can perform operations based on the motion detection results provided by the backend computing device.

[0169] Figure 9 The challenging counterpart was obtained under rainy conditions. The duration of the raw signal collected by the microwave sensor in the plot above is 200 seconds. According to the underlying truth record, the signal in box 131 is the background signal, and the signal in box 132 is the raindrop signal. Figure 9 The background and motion spectrum shown in the middle and lower plots are output through step 3 above.

[0170] In the intermediate plot, the output of step 4 shows the maximum peak 134 at 200 Hz with a height or magnitude of 1.0. The output of step 5 shows two other dominant peaks, 133 and 135, at 100 Hz and 300 Hz respectively, with heights of 0.65 and 0.87. The output of step 6 in the intermediate plot shows exactly three dominant peaks on the background bins.

[0171] In the plotting below, the output of step 4 shows the maximum peak 136 at 3 Hz with a height of 1.0. The output of step 5 shows three other dominant peaks 137, 138, and 139 at 100 Hz, 200 Hz, and 300 Hz, with heights of 0.81, 0.96, and 0.71, respectively. The output of step 6 in the plotting below shows the presence of four main peaks, with three of them, 137, 138, and 139, on the background bins.

[0172] Based on the adversary assessment described above, it is difficult to determine the presence of motion events based on the spectrum shown in the plot below. Therefore, sensor data will be further processed at the back-end computing device before the terminal device (i.e., the motion detector) performs operations based on the motion detection results obtained from the back-end computing device.

[0173] By performing an adversary evaluation on each sensor's data, the device's operations can be performed while balancing accuracy, efficiency, and responsiveness.

[0174] It should be understood that the embodiments of this disclosure may also be any combination of the dependent claims and / or the foregoing embodiments and their respective independent claims.

[0175] By studying the accompanying drawings, the disclosure, and the appended claims, those skilled in the art can understand and implement other variations of the disclosed examples in practice with respect to the claimed disclosure. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite articles "a" or "an" do not exclude multiple. A single processor or transceiver or other unit can perform the functions of several items listed in the claims. The mere fact that certain measures are referenced in mutually different dependent claims does not indicate that a combination of these measures cannot be used advantageously. Computer programs can be stored / distributed on suitable media, such as optical storage media or solid-state media provided with or as part of hardware, but can also be distributed in other forms, such as via the Internet or other wired or wireless telecommunications systems. Any reference numerals in the claims should not be construed as limiting their scope.

Claims

1. A method for a device including a processor to perform an operation based on sensor signal data obtained from at least one sensor, the method comprising the steps of: - The processor generates a feature profile from the sensor signal data; - The processor matches the generated feature contours with a predetermined set of feature contours available to the processor; If the generated feature contour matches at least one of the predetermined feature contour sets, the operation is performed by the device under the control of the processor based on local processing of the sensor signal data. If the generated feature contour does not match at least one of the predetermined feature contour sets, the operation is performed by the device under the control of the processor based on remote processing of the sensor signal data.

2. The method of claim 1, wherein, The step of generating a feature profile from the sensor signal data includes generating the feature profile by using a predefined data analysis algorithm that includes one of a transformation algorithm and a feature extraction algorithm.

3. The method of claim 2, wherein, The transformation algorithm is the Fast Fourier Transform (FFT) algorithm, and the generated feature profile is a spectral feature profile, which includes a set of frequency components whose amplitude exceeds a predetermined threshold. The predetermined feature profile set includes a predetermined spectral feature profile defined by the number and frequency of the frequency components that exceed the threshold.

4. The method according to any one of the preceding claims, wherein, The step of matching the generated feature contour with the predetermined set of feature contours available to the processor includes: comparing the generated feature contour with the predetermined set of feature contours available to the processor.

5. The method of any one of claims 1 to 3, wherein, The step of matching the generated feature contour with a predetermined set of feature contours available to the processor includes: determining whether the generated feature contour repeats within a predetermined time period.

6. The method of any one of claims 1 to 3, wherein, The step of performing the operation by the device using the sensor signal data includes: applying an operation algorithm to the sensor signal data, wherein, in particular, the operation algorithm includes a data analysis algorithm for generating the feature contour.

7. The method of any one of claims 1 to 3, wherein, The device further includes a transceiver, and the method includes the following steps prior to performing the operation based on remote processing of the sensor signal data: - The transceiver transmits the sensor signal data to a remote computing device, and - The transceiver receives processed sensor signal data from the remote computing device, obtained by applying at least one processing algorithm to the transmitted sensor signal data by the remote computing device.

8. The method of claim 7, wherein, The transmission of sensor signal data by the transceiver includes: transmitting a representation of the sensor signal data to the remote computing device.

9. The method according to any one of claims 1 to 3, wherein, The sensor signal data includes at least one of the following: motion data, sound data, image data, video data, temperature data, humidity data, pressure data, brightness data, chemical composition or substance data, olfactory data, and tactile data.

10. The method of any one of claims 1 to 3, wherein, The device includes lighting equipment, which includes at least one lighting module, particularly a light-emitting diode (LED), and wherein performing the operation by the device includes controlling the lighting through the at least one lighting module.

11. A device for performing an operation based on sensor signal data obtained from at least one sensor, the device comprising a processor and a transceiver, wherein, The processor and transceiver are arranged to perform the operation under the control of the processor according to the method of any one of the preceding claims.

12. The device according to claim 11, comprising at least one illumination module, in particular a light emitting diode (LED), wherein The operation performed by the device under the control of the processor includes controlling the lighting by the at least one lighting module.

13. The device according to claim 11 or 12, comprising at least one sensor arranged to provide at least one of the following: motion data, sound data, image data, video data, temperature data, humidity data, pressure data, brightness data, chemical composition or substance data, olfactory data, and tactile data.

14. A system comprising the device according to any one of claims 11, 12, or 13, and a remote back-end computing device, wherein, The remote backend computing device is configured to: - Receive sensor signal data from the device; - The received sensor signal data is processed by applying at least one processing algorithm, and - Transmit the processed sensor signal data to the device.

15. A computer program product comprising program code stored on a computer-readable medium, wherein when the program code is executed by at least one processor, the program code is arranged to perform the method according to any one of claims 1 to 10.

Images