Event masking

Abstract

An apparatus, method, and computer program are described for detecting, based on sensor data, an antecedent feature, wherein the antecedent feature is indicative of a prediction of a first event, determining an event masking action corresponding to the first event, and causing, in response to detection of the antecedent feature, performance of the event masking action for masking the first event.

Description

Technical Field

[0001] This specification relates to event masking; for example, masking events that may not be desired. Background Technology

[0002] For example, white noise can be used to mask noise or similar events. There is still a need for further development in this field. Summary of the Invention

[0003] In a first aspect, an apparatus is provided, comprising: means for detecting a prior marker based on sensor data, wherein the prior marker indicates a prediction of a first event; means for determining an event masking action corresponding to the first event; and means for causing the execution of an event masking action for masking the first event in response to the detection of the prior marker.

[0004] An apparatus is also disclosed, comprising at least one processor; and at least one memory including computer program code, the at least one memory and the computer program being configured together with the at least one processor to cause the apparatus to: detect a prior event marker based on sensor data, wherein the prior event marker indicates a prediction of a first event; determine an event masking action corresponding to the first event; and cause the execution of the event masking action for masking the first event in response to the detection of the prior event marker.

[0005] In a second aspect, a method is provided, comprising: detecting a prior marker based on sensor data, wherein the prior marker indicates a prediction of a first event; determining an event masking action corresponding to the first event; and causing the execution of an event masking action for masking the first event in response to the detection of the prior marker.

[0006] In a third aspect, a computer program is provided, including instructions for: detecting a prior marker based on sensor data, wherein the prior marker indicates a prediction of a first event; determining an event masking action corresponding to the first event; and causing the execution of an event masking action for masking the first event in response to the detection of the prior marker.

[0007] A computer-readable medium is also disclosed, having stored computer-readable instructions for: detecting a prior marker based on sensor data, wherein the prior marker indicates a prediction of a first event; determining an event masking action corresponding to the first event; and causing the execution of an event masking action to mask the first event in response to the detection of the prior marker. Attached Figure Description

[0008] The example implementation will now be described by way of example only, referring to the following schematic diagram, wherein:

[0009] Figure 1 This is a block diagram of the example system;

[0010] Figures 2 to 4 This is a sensor data graph based on an example embodiment;

[0011] Figure 5 This is a flowchart illustrating an algorithm according to an example embodiment;

[0012] Figure 6 This is a sensor data graph based on an example embodiment;

[0013] Figures 7 to 12 This is a block diagram of a system according to an example embodiment;

[0014] Figure 13 This is a block diagram of the components of a system according to an exemplary embodiment; and

[0015] Figure 14A and Figure 14B Tangible media are shown, namely a compact disc (CD) storing computer-readable code and a removable non-volatile memory unit, the computer-readable code performing operations according to the example embodiment when run by a computer. Detailed Implementation

[0016] The scope of protection sought by the various embodiments of the present invention is defined by the independent claims. Embodiments and features described in the specification that are not within the scope of the independent claims (if any) are to be interpreted as examples useful for understanding the various embodiments of the invention.

[0017] Throughout the description and accompanying drawings, the same reference numerals refer to the same elements.

[0018] Figure 1 This is a block diagram of an example system, generally indicated by reference numeral 10. System 10 includes a user equipment 1 that receives sensor data 2 from one or more sensors. At least one of the one or more sensors may be included within the user equipment 1. Alternatively or additionally, at least one of the one or more sensors may be an external sensor, for example, included within one or more external devices. Sensor data 2 may include one or more of visual data, auditory data, tactile data, motion data, radio frequency data, environmental data (e.g., temperature, humidity, etc.), etc. User equipment 1 may analyze sensor data 2 to predict the occurrence of events. This will be explained in more detail below.

[0019] In daily life, many events, such as loud noises or sudden bright lights, can cause disturbance. It may be impossible to prevent such events from occurring. Example embodiments describe how these events can be masked to, for example, reduce the perception of the event by one or more users.

[0020] Figure 2It is a sensor data graph according to an example embodiment, which is generally indicated by reference numeral 20. Figure 2 Sensor data axis 21 and time axis 23 are shown. Figure 2 It also includes a first sensor data marker 24. The first sensor data marker 24 indicates a signal corresponding to the first event 22. The first event 22 may be a disruptive event that could interfere with one or more users. The first sensor data marker 24 indicates that the first event 22 occurs at time T3. As described above, sensor data may include visual data, auditory data, tactile data, motion data, environmental data, or any other data that can be sensed by one or more sensors. For example, when the first event 22 is an auditory event, the sensor data may include an audio signal. An auditory event may be disruptive or noisy noise that could interfere with one or more users. For example, one or more users may be humans or animals (such as pets). In another example, one or more users may include devices operated based on voice commands. For example, noisy noise may cause such a voice-operated device to perform actions in response to the noisy noise (e.g., outputting a notification that it does not understand the noisy noise, or associating the noisy noise with the input command), which may be undesirable. In other examples, the first event 22 may be one or more of a visual event, an auditory event, an environmental event, a tactile event, or any other event that could interfere with one or more users.

[0021] Figure 3 This is a sensor data graph based on an example embodiment. Figure 1 Generally indicated by reference numeral 30. Figure 3 The sensor data axis 21, time axis 23, first sensor data marker 24, and first event 22 are shown, as follows: Figure 2 As shown in the image. Figure 3It also includes a second sensor data marker 31. The second sensor data marker 31 may be based on second sensor data that can be received before the first event 22. For example, the second sensor data may be received as a signal. The second sensor data received before the first event 22 may be analyzed to identify or detect one or more advance markers 32 indicating a prediction of the first event 22. For example, the second sensor data marker may include a rise in the second sensor data at T1 and a fall in the second sensor data at T2. The second sensor data marker may be included within an advance marker 32 corresponding to the first event 22. Therefore, when the second sensor data marker is identified or detected, it can be determined that the first event 22 is predicted to occur. For example, referring to time T3 (the start time of the first event 22), time T1 may be two minutes earlier than time T3, and time T2 may be one minute earlier than time T3. Therefore, the advance marker 32 may indicate that the first event 22 is predicted to occur approximately two minutes after the start (rising edge) of the second sensor data. Alternatively or additionally, the first event 22 may be predicted to occur approximately one minute after the end (falling edge) of the second sensor data. Alternatively or additionally, when the second sensor data marker 31 indicates an increase in the second sensor data and a decrease in the second sensor data within one minute after the increase, the first event 22 can be predicted to occur; and the first event 22 can be predicted to occur approximately one minute after the decrease in the second sensor data. Alternatively or additionally, the first event 22 can be predicted to occur approximately one minute and thirty seconds after the second sensor data has been high for at least thirty seconds.

[0022] Thus, the pre-marking 32 may include one or more parameters associated with sensor data from one or more sensors. Parameters may include one or more of the timing, frequency, amplitude, etc., of the sensor data and / or the signals associated with the sensor data. For example, when the sensor data includes auditory sensor data (e.g., from an auditory sensor such as a microphone), the parameters may include one or more of the timing (e.g., start or end, rising or falling edge, etc.), frequency (e.g., pitch), and amplitude (e.g., loudness) of the auditory sensor data. In another example, when the sensor data includes visual sensor data such as a light, the parameters may include one or more of the timing (e.g., the start or end of the light turning on or off, the rate of flashing, the number of flashes, etc.), frequency (e.g., the color of the light), wavelength (e.g., position on the electromagnetic spectrum; visible or invisible to the naked eye), and amplitude (e.g., the intensity or brightness of the light).

[0023] For example, the first event 22 could be an auditory event, such as a sound emitted by the heater, for example, due to the expansion of the heater's material during heating. The sound from the heater could be a disruptive noise and could disturb one or more users (e.g., if someone is sleeping). The heater's light could be turned on at a threshold time prior to the first event 22, that is, at a threshold time prior to the heater emitting the disruptive sound. For example, the threshold time could be thirty seconds. Therefore, the second sensor data marker 31 could indicate visual sensor data captured from the heater's light. The prior marker of the first event 22 (the sound from the heater) could therefore include a second sensor data marker indicating that the light was turned on thirty seconds prior to the first event 22. As described above, the prior marker 32 could include one or more parameters associated with the second sensor data (shown by the second sensor data marker 31), wherein these parameters could include one or more of timing, frequency, amplitude, etc. For example, the second sensor data marker could be defined such that the first event 22 is predicted to occur 30 seconds after the light is turned on, and when the heater's light is red (e.g., the second sensor data has a frequency (430-480 THz) and / or wavelength (700-635 nm) associated with red light), and when the light has a brightness above a threshold brightness (e.g., amplitude and / or intensity above a threshold amplitude and / or intensity). For example, if the green light is on, the second sensor data marker (associated with the red light) will not be detected, and therefore the pre-event marker will not be detected. Because the pre-event marker is not detected, the first event 22 may not be predicted to occur.

[0024] Figure 4 The above is a sensor data graph according to an example embodiment, which is generally indicated by reference numeral 40. Figure 4 The sensor data axis 21, time axis 23, first sensor data marker 24, first event 22, and second sensor data marker 31 are shown, as follows: Figure 2 As shown in the image. Figure 4It also includes a third sensor data marker 41. The second sensor data marker 31 and the third sensor data marker 41 can be based on second sensor data and third sensor data that can be received before the first event 22, respectively. For example, the second sensor data and the third signal data can be received as a second signal and a third signal, respectively. The second sensor data and the third sensor data received before the first event 22 can be analyzed to determine one or more advance markers 42 indicating a prediction of the first event 22. For example, the third sensor data marker can include a decrease in the third sensor data at T0.5 and an increase in the third sensor data at T1.5. As described above with reference to the second sensor data marker, the third sensor data marker can include one or more parameters. The second sensor data marker and the third sensor data marker can be included in the advance marker 42 corresponding to the first event 22. Therefore, when both the second sensor data marker and the third sensor data marker are detected, it can be determined that the first event 22 is predicted to occur. Alternatively or additionally, if only one of the second sensor data marker and the third sensor data marker is detected, it can be determined that the first event 22 is predicted to occur. Alternatively or otherwise, one or more combinations of second sensor data tags and third sensor data tags may be defined for one or more corresponding advance tags corresponding to the prediction of one or more corresponding events.

[0025] For example, the first event 22 could be an auditory event, such as a sound from the heater. When the heater is turned on, the heater's light may be turned on. Furthermore, the heater may only begin heating and therefore emit a sound after a threshold time following a drop in room temperature below a threshold temperature (first event 22). For example, the threshold time could be thirty seconds, and the threshold temperature could be 10 degrees Celsius. When the heater is turned on (light on) and the temperature is below 10 degrees Celsius, the heater may emit a sound (begin heating). If the heater is turned on, but the temperature is not below 10 degrees Celsius, the heater will not begin heating and therefore may not emit a sound (first event 22 does not occur). If the temperature is below 10 degrees Celsius, but the heater is not turned on (light off), the heater will not begin heating and therefore may not emit a sound (first event 22 does not occur). Therefore, the second sensor data label 31 could indicate visual sensor data captured from the heater's light, and the third sensor data label 41 could indicate temperature sensor data captured by one or more temperature sensors (e.g., included in a thermostat). Thus, the second signal data marker 31 indicates an example timing of the lamp's activation, so that the first event 22 can be predicted to occur at any time after the lamp is activated. For example, the heater can be activated at any time before the first event 22, but the first event 22 will only occur thirty seconds after the temperature drops below 10 degrees Celsius. The prior marker for the first event 22 (the sound from the heater) can therefore include the second sensor data marker (lamp activation) and the third sensor data marker (temperature dropping below 10 degrees Celsius).

[0026] As previously discussed, sensor data 2 can be collected using one or more sensors included in user device 1 and / or one or more external devices. Sensor data can be used to determine sensor data tags associated with events, as well as to determine pre-event tags. For example, one or more sensors may include a microphone or microphone array for sensing sensor data 2, including audio signals. Alternatively or additionally, one or more sensors may include a camera that can be used to sense sensor data 2, which includes visual signals associated with visual or auditory events. It is understood that one or more sensors may include any other type of sensor and are not limited to microphones or cameras.

[0027] Figure 5 It is a flowchart of an algorithm according to an example embodiment, generally indicated by reference numeral 50.

[0028] At operation 51, a prior marker is detected for the first event. The prior marker may indicate a prediction of the first event (e.g., including the nature and timing of the first event). The prior marker may include one or more of the following: auditory markers, visual markers, environmental markers (such as temperature, humidity, etc.), and tactile markers. The first event may be one or more of the following: a visual event, an auditory event, a tactile event, and an environmental event.

[0029] At operation 52, an event masking action is determined for the first event. The event masking action can be one or more of the following: visual action, auditory action, tactile action, and environmental action (such as turning on the air conditioning unit if the temperature is predicted to rise, or turning on the humidifier if the humidity is predicted to drop significantly).

[0030] At operation 53, in response to the detection of a prior marker, the determined event masking action is executed. Therefore, event masking can be implemented by algorithm 50 based on the predictive properties of the event (e.g., the timing and / or nature of the event). The use of prediction can provide a predictive masking system that does not rely on reactive masking of the event. Reactive masking can include executing masking actions(s) after the event has begun. This may result in at least the beginning of the event not being masked. In some scenarios, the beginning or initial portion of an event may be disruptive enough to disturb one or more users. In contrast, predictive masking allows for the execution of masking actions(s) before the event occurs, so that the beginning or initial portion of the event is also masked for one or more users. Thus, the event can cause minimal disturbance or interference to one or more users because the event is masked by the predictive masking actions(s).

[0031] In the example embodiment, at operation 53, user device 1 may perform an event masking action to cause the event masking action to be performed at user device 1. Alternatively or additionally, user device 1 may cause the event masking action to be performed by instructing another device to perform the event masking action. For example, user device 1 may send one or more instructions to one or more other devices to perform the event masking action. For example, the event masking action includes playing white noise to mask auditory events. User device 1 may send an instruction to a speaker device to instruct the speaker device to play white noise. The speaker device can then perform the event masking action by playing white noise.

[0032] At operation 51, a tag matching algorithm (e.g., utilizing one or more models) can be used to determine pre-tags. For example, one or more sensor data may be sensed by one or more sensors, and the sensor data may be used to determine whether the sensor data matches pre-tags (including one or more sensor data tags). Information on multiple pre-tags may be stored in a pre-tag database. Queries may be sent to the pre-tag database to determine whether the sensor data matches any pre-tag stored in the pre-tag database. Thus, pre-tags can be determined at least in part based on sensor data matching. In an example embodiment, if the sensor data is within a threshold of a pre-tag, the sensor data is considered to match a pre-tag. For example, a similarity score may be determined such that the similarity score is an indication of the degree of similarity between the sensor data and the pre-tag. If the similarity score is within a threshold similarity score, the sensor data can be considered to match a pre-tag.

[0033] At operation 52, an event masking action for masking the first event can be determined. The pre-mark database (as described above) may also include one or more events corresponding to pre-marks. When a pre-mark is determined at operation 51, the occurrence of the corresponding first event can be determined. An event masking action can be determined based on the first event so that the event masking action can mitigate, cancel, or reduce some of the effects of the first event.

[0034] In the example embodiment, an event masking action can be specified for one or more events, and multiple event masking actions can be defined for multiple events, and multiple event masking actions can be stored in a database.

[0035] The masking operation determined in operation 52 can be of the same type as the event determined in operation 51 (e.g., a visual masking action can be used when the first event is a visual event, an auditory masking action can be used when the first event is an auditory event, and so on). However, this is not essential for all example embodiments. For example, environmental events (such as changes in light levels) can be masked using either an auditory or visual action.

[0036] In one example embodiment, the first event is an auditory event, and the event masking action includes an audio suppression action. In one example, the audio suppression action is determined by sending a query to a database containing ideal audio suppression actions for the auditory event. The audio suppression action may include one or more of audio masking and audio cancellation. For example, sound masking can be performed by generating a sound that can mask the auditory event. The sound can be generated using, for example, an AI music generator, a white noise generator, a speech generator, etc. For example, one or more properties of the generated sound can be configured based on one or more properties of the auditory event so that the auditory event can be masked. In another example, audio cancellation can be performed by generating an audio output to cancel one or more detrimental sounds of the auditory event. The audio used for audio cancellation can be generated using a directional audio system, a broadcast audio system, headphones, or any other device that can be used to cancel the sound of the auditory event.

[0037] In one example, audio for audio cancellation is generated at the user's location. For instance, an auditory event might occur in a first room, while the user is in a second room. The user might be able to hear one or more sounds of the auditory event. Audio cancellation can be performed by generating audio in the second room. In another example, audio for audio cancellation is generated at the location of the auditory event. When the auditory event occurs in the first room and the user is in the second room, audio cancellation can be performed by generating audio in the first room. Performing audio cancellation at the location of the auditory event allows the suppression of audio from the auditory event for one or more nearby users located outside the second room.

[0038] Figure 6 This is a signal data diagram based on an example embodiment. Figure 1 Generally indicated by reference numeral 60. Figure 6 The signal data axis 21, time axis 23, first sensor data curve 24, and first event 22 are shown, as follows: Figure 2 As shown in the image. Figure 6 It also includes a fourth signal data marker 61. The fourth signal data marker 61 can represent a signal corresponding to, for example... Figure 5 The signal for the event masking action performed at operation 53. For example, the event masking action can begin before the first event 22 is predicted to occur, and can end after the first event 22 is predicted to end. Figure 9 The event concealment action is discussed in more detail.

[0039] Figure 7This is a block diagram of a system according to an example embodiment, generally indicated by reference numeral 70. System 70 illustrates that pre-signature 71 may include one or more of auditory sign 72, visual sign 73, environmental sign 74, and tactile sign 75.

[0040] For example, auditory marker 72 may include specific frequency components, amplitude, duration, direction of origin of the sound, sound wave pattern, or any other details that may define one or more sounds for the auditory marker. Visual marker 73 may include information about any visual changes, such as a light being turned on, the color of the light, the brightness of the light, a light being turned off, a light flashing or blinking, any physical movement of any object, etc. Environmental marker 74 may include information about any environmental changes, such as changes in temperature, humidity, air composition (e.g., the presence of smoke), odor changes, etc. Tactile marker 74 may include information about any tactile changes, such as vibration, the frequency of the vibration, the amplitude of the vibration, the duration of the vibration, the device from which the vibration originates, etc.

[0041] In one example, environmental changes can include any changes in the relevant environment, which may or may not include visual, auditory, or tactile changes.

[0042] In one example, pre-signatures may include a combination of one or more of auditory sign 72, visual sign 73, environmental sign 74, and tactile sign 75. For example, a thunderstorm may cause auditory changes (thunder), visual changes (light from lightning), environmental changes (discharge), and tactile changes (vibration). Alternatively or additionally, environmental features that include information about any environmental changes may also fall within the scope of visual, auditory, and / or tactile signs. For example, increased humidity (environmental feature) may be accompanied by fog (visual feature). Thus, any change can correspond to one or more of visual, auditory, tactile, and / or environmental signs.

[0043] In the example embodiment, the prior marker is not limited to a marker that occurs before the first event 22 occurs, so that the prior marker can correspond to sensor data received during or after the first event 22. For example, the prior marker associated with an auditory event may not necessarily be an auditory marker and could be a side-channel marker, such as a radio frequency marker, or could include sensor data from a visual microphone. In this scenario, the side-channel marker can have a speed faster than the speed of the auditory event. Therefore, the prior marker can occur during or after the sound that generates the auditory event, but the sensor data associated with the prior marker can be received before the audio signal of the auditory event is received. Because the sensor data associated with the prior marker can be received before the audio signal of the auditory event is received, event masking actions can be performed to mask, eliminate, or reduce the impact of the auditory event.

[0044] The pre-event marking 71 may include a pattern or combination of one or more of auditory marking 72, visual marking 73, environmental marking 74, and tactile marking 75.

[0045] Figure 8 This is a block diagram of a system according to an example embodiment, generally indicated by reference numeral 80. System 80 illustrates that a first event 81 (similar to first event 22) may include one or more of an auditory event 82, a visual event 83, an environmental event 84, and a tactile event 85. For example, auditory event 82 may be a noise event, such as destructive noise that may interfere with the user of user device 1 and / or one or more other users who hear destructive noise. Visual event 83 may be an event that interferes with the user or one or more users who see the event. For example, visual event 83 may be a light turning on, a light turning off, distracting movement of an object, or any other event that may be destructive. For example, environmental event 84 may be a temperature change, a humidity change, a odor change, or any other event that may interfere with the user and / or one or more other users. For example, tactile event 85 may be vibration, movement of an object, or any other destructive event that the user or one or more other users may feel.

[0046] In one example, the first event 22 can be a combination of one or more of the following: auditory event 82, visual event 83, environmental event 84, and tactile event 85. For example, a lightning event (first event 22) can be a combination of thunder (auditory event 82), flash (visual event 83), discharge (environmental event 84), and vibration (tactile event 85).

[0047] In one example, environmental events involving any change in the environment can also fall within the scope of visual, auditory, and / or tactile events. Environmental changes can include or may not include auditory, visual, or tactile changes. For example, increased humidity (an environmental event) may be accompanied by fog (a visual event). Thus, any change can correspond to one or more of visual, auditory, tactile, and / or environmental events.

[0048] Figure 9 This is a block diagram of a system according to an example embodiment, generally indicated by reference numeral 90. System 90 illustrates that an event masking action 91 may include one or more of an auditory action 92, a visual action 93, an environmental action 94, and a tactile action 95.

[0049] In the example embodiment, the event masking action can correspond to the type of event, that is, the auditory action 92 can be performed before the auditory event 82; the visual action 93 can be performed before the visual event 83; the environmental action 94 can be performed before the environmental event 84; and the tactile action 95 can be performed before the tactile event 85.

[0050] For example, auditory action 92 may include generating an audio signal (e.g., white noise or noise-reduced frequency) for masking an auditory event (first event 22).

[0051] Visual action 93 may include generating light to reduce the impact of the visual event or to create a visual barrier between the user and the visual event. For example, the first event 22 may be a flash of light in the room, and the event masking action may be adjusting the room lighting to make the user or one or more other users in the room less distracted or disturbed by the flash of light.

[0052] Environmental action 94 may include, for example, adjusting temperature, adjusting humidity, generating odor, etc. For example, the first event 22 may be an increase in external temperature above a desired level. If the first event 22 is predicted to occur (e.g., an increase in external temperature is predicted), an event masking action (such as environmental action 94) can be performed by turning on the air conditioning system (before the external temperature increases) to lower the temperature, so that the user in the room or one or more other users are not disturbed by the increase in external temperature. Similarly, the first event 22 may be a decrease in humidity. If the first event 22 is predicted to occur (e.g., a decrease in humidity is predicted, possibly below a desired level), an event masking action (such as environmental action 94) can be performed by turning on a humidifier.

[0053] Tactile action 95 may include, for example, vibration, movement, applying heat to the body, or applying cooling to the body. For example, the first event 22 may be the jolt of removing a car seat from a car seat support, which may disturb an infant (user) sitting in the car seat or wake them from sleep. The first event 22 may be predicted to occur when the car door is opened (pre-marked). Event masking actions may include tactile action 95. Tactile action 95 can be performed by gradually increasing the vibration of the car seat, such that the movement of the car seat gradually increases from a low level to a high level. This reduces the jolt when removing the car seat, minimizing disturbance to the infant and reducing the infant's perception of the jolt. In another example, the first event 22 may be a drop in room temperature that may cause the user to feel cold. The first event 22 may be predicted to occur when the room window is open and the outside temperature is below a threshold (pre-marked). Event masking actions may include tactile action 95, which may be performed by applying heat to the user's body (e.g., via an electric blanket).

[0054] In example embodiments, the event masking action may differ from the type of event. For instance, any one of the auditory action 92, visual action 93, environmental action 94, and tactile action 95 may be performed before any one of the auditory event 82, visual event 83, environmental event 84, and tactile event 85. For example, the first event 22 is an auditory event 82, such as disruptive noise. An event masking action such as visual action 93 may be performed to distract the user or one or more other users from the first event 22 (i.e., disruptive noise). In another example, the first event 22 is a visual event, such as a sudden bright light that may disturb a user (e.g., a user who is uncomfortable due to photosensitivity). If the first event 22 is predicted to occur, the event masking action may include a tactile action 95, which may include applying pressure to one or more pressure points on the user's body. For example, applying pressure to the temples of the head (e.g., via a hat or a headband including an inflatable airbag) before the user is exposed to sudden bright light may help alleviate discomfort caused by photosensitivity.

[0055] Figure 10 This is a block diagram of a system according to an example embodiment, generally indicated by reference numeral 100. System 100 includes a first model 101, which can be used to determine prior markings.

[0056] The first model 101 receives multiple sensor data and information about one or more events as input. The first model 101 can be used to determine the correlation between one or more sensor data features and one or more events. One or more sensor data features can be determined from the sensor data input.

[0057] In an example embodiment, pre-labels for one or more events can be stored in a database, such as a first model 101. The first model 101 can be trained over a long time period using multiple sensor data sets (e.g., a large dataset of events and sensor data, which can be included in the pre-labels for one or more events). In an example embodiment, the first model 101 is trained by determining the correlation between one or more sensor data features and one or more events (e.g., first event 22). One or more sensor data features can be associated with an event (e.g., first event 22) if the uniqueness or repeatability of one or more sensor data features associated with first event 22 is higher than a first threshold. If the occurrence of sensor data features typically follows first event 22, the uniqueness of one or more sensor data features associated with first event 22 is likely higher than the first threshold. If the occurrence of first event 22 typically precedes the occurrence of one or more sensor data features, the repeatability of one or more sensor data features associated with first event 22 is likely higher than the first threshold. The sensor data features associated with the first event can then be used to train the first model 101. For example, machine learning principles can be used to train the first model 101. Alternatively, unsupervised learning can be used, for example, to train the first model 101 during the use of the first model.

[0058] For example, the first event 22 could be an auditory event, such as a sound produced at the heater. The heater's light could be turned on at a threshold time prior to the occurrence of the first event 22. The turning on of the light could be a sensor data feature. A first model 101 could be trained for the first event 22 (i.e., the sound from the heater) over a training period. During the training period, the uniqueness of the sensor data feature of the light turning on could be determined based on whether a sound is produced at the heater after the light is turned on, typically (e.g., for most of the time when the heater's light is on). If it is determined that a sound is produced at the heater after the light is turned on, typically (e.g., for most of the time when the light is on), the uniqueness of the sensor data feature is likely higher than the first threshold. The repeatability of the signal data feature of the light turning on could be determined based on whether a sound produced at the heater, typically (e.g., for most of the time when a sound is produced at the heater), precedes the turning on of the light. If it is determined that a sound produced at the heater, typically (e.g., for most of the time when a sound is produced at the heater), precedes the turning on of the light, the repeatability of the signal data feature is likely higher than the first threshold. It is understood that training of the first model 101 can continue after the training period, so that the first model 101 can be improved and / or updated during normal use. Alternatively or additionally, unsupervised learning can be used to train the first model, so that the first model 101 may not need to be pre-trained. Users can begin using the device of the above example embodiment, and the first model 101 can begin learning the correlation between sensor data features and events while the device is in use. For example, the first model 101 may include a neural network. One or more parameters of the first model 101 can be updated and / or improved based on machine learning principles in unsupervised learning and / or through manual input from the user.

[0059] Figure 11This is a block diagram of a system according to an example embodiment, generally indicated by reference numeral 110. System 110 includes user equipment 121, a first device 117, a second device 118, and a third device 119. The first device 117, the second device 118, and the third device 119 can generate one or more sounds or noises that may or may cause interference to the user of user equipment 121 or one or more other users. For example, the first device 117 can generate noise, which may be a first event 112. A prior marker 111 is determined to indicate a prediction of the first event 112. The second device 118 can generate noise, which may be a second event 114. A prior marker 113 is determined to indicate a prediction of the first event 114. The third device 119 can generate noise, which may be a third event 116. A prior marker 115 is determined to indicate a prediction of the third event 116. User equipment 121 can receive sensor data from each of the pre-marked markers 111, 113, and 115, and therefore can predict the occurrence of the first event 112, the second event 114, and the third event 116. When user equipment 121 predicts the occurrence of an event, user equipment 121 can perform one or more event masking actions to mask the first event 112, the second event 114, and the third event 116, such as... Figure 12 As shown in the image.

[0060] Event masking actions can be used to mask one or more events from all users who can perceive them. For example, adjusting the light level in a room to mask a visual event in the room can provide masking for all users present in the room, and thus allow them to see the visual event. However, in some embodiments, masking can be performed only on a subset of users who can perceive the masked event. For example, it might be desirable to mask the sound of an event for everyone except a single user. An example of this might be a bedroom where two or more users are sleeping, and only one user wants to experience the event—for example, only one user wants to be woken up by an alarm clock while the others want to continue sleeping.

[0061] Figure 12 This is a block diagram of a system according to an example embodiment, generally indicated by reference numeral 120. User equipment 121 may perform an event masking action 122 to mask the second event 114 (i.e., the noise generated by the second device 118). The event masking action 122 may be an auditory action, such as, for example, generating sound using one or more speakers of user equipment 121, one or more external speakers, or one or more headphones or earphones worn by the user and / or other users. The generated sound may result in masking, cancellation, mitigation, or reduction of the impact of the noise generated by device 118 on the user or other users.

[0062] For the sake of completeness, Figure 13This is a schematic diagram of components used to implement one or more modules of the above algorithm, hereinafter collectively referred to as processing system 300. Processing system 300 may have a processor 302, a memory 304 coupled to the processor and consisting of RAM 314 and ROM 312, and optional viewer input 310 and display 318. Processing system 300 may include one or more network interfaces 308 for connecting to a network, such as a wired or wireless modem.

[0063] The processor 302 is connected to each of the other components in order to control their operation.

[0064] Memory 304 may include non-volatile memory, hard disk drive (HDD), or solid-state drive (SSD). ROM 312 of memory 304 specifically stores operating system 315 and may also store software applications 316. Processor 302 uses RAM 314 of memory 304 to temporarily store data. Operating system 315 may contain code that, when executed by the processor, implements various aspects of algorithm 50.

[0065] Processor 302 can take any suitable form. For example, it can be a microcontroller, multiple microcontrollers, a processor, or multiple processors. Processor 302 may include processor circuitry.

[0066] The processing system 300 can be a standalone computer, server, console, or its network.

[0067] In some embodiments, the processing system 300 may also be associated with external software applications. These may be applications stored on a remote server device and may run partially or exclusively on the remote server device. These applications may be referred to as cloud-hosted applications. The processing system 300 may communicate with the remote server device to utilize the software applications stored there.

[0068] Figure 14A and Figure 14B Tangible media are shown, namely a compact disc (CD) 368 storing computer-readable code and a removable storage unit 365, which, when run by a computer, can execute the methods according to the above embodiments. The removable storage unit 365 may be a memory stick, such as a USB memory stick, having internal memory 366 for storing computer-readable code. The computer system can access the memory 366 via a connector 367. The CD 368 may be a CD-ROM, DVD, or the like. Other forms of tangible storage media may be used.

[0069] Embodiments of the present invention can be implemented in software, hardware, application logic, or a combination of software, hardware, and application logic. The software, application logic, and / or hardware can reside on memory or any computer medium. In example embodiments, the application logic, software, or instruction set is maintained on any of a variety of conventional computer-readable media. In the context of this document, "memory" or "computer-readable medium" can be any non-transitory medium or apparatus that can contain, store, communicate, propagate, or transmit instructions for use by or in connection with an instruction execution system, apparatus, or device (such as a computer).

[0070] In relevant contexts, references to “computer-readable storage medium,” “computer program product,” “tangible computer program,” or “processor” or “processing circuitry” should be understood to encompass not only computers with different architectures (such as single / multiprocessor architectures and sequencer / parallel architectures) but also special-purpose circuits, such as field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), signal processing devices, and other devices. References to computer programs, instructions, code, etc., should be understood to express software used for programmable processor firmware, such as programmable content of hardware devices as instructions for a processor, or configured or configured settings for fixed-function devices, gate arrays, programmable logic devices, etc.

[0071] In this application, the term "circuit system" means all of the following: (a) a purely hardware circuit implementation (such as an implementation in analog and / or digital circuits only) and (b) a combination of circuits and software (and / or firmware), such as (if applicable): (i) a combination of (multiple) processors or (ii) portions / software of (multiple) processors (including (multiple) digital signal processors), software and (multiple) memories, which work together to enable a device such as a server to perform various functions, and (c) circuits that require software or firmware for operation (such as (multiple) microprocessors or portions of (multiple) microprocessors), even if the software or firmware does not actually exist.

[0072] If desired, the different functions discussed herein can be executed in different orders and / or simultaneously with each other. Furthermore, if desired, one or more of the aforementioned functions can be optional or can be combined. Similarly, it should be understood that... Figure 5 The flowchart is merely an example and the various operations depicted therein can be omitted, reordered, and / or combined.

[0073] It should be understood that the above-described exemplary embodiments are purely illustrative and do not limit the scope of the invention. Other variations and modifications will be apparent to those skilled in the art after reading this specification.

[0074] Furthermore, the disclosure of this application should be understood to include any novel feature or any novel combination of features or any generalization thereof that is expressly or implicitly disclosed herein, and during the proceedings of this application or any application derived therefrom, new claims may be formulated to cover any such feature and / or combination of such features.

[0075] Although various aspects of the invention are set forth in the independent claims, other aspects of the invention include other combinations of features from the described embodiments and / or dependent claims with features of the independent claims, and not only the combinations expressly set forth in the claims.

[0076] It should also be noted in this document that while various examples have been described above, these descriptions should not be considered limiting. Rather, several changes and modifications may be made without departing from the scope of the invention as defined by the appended claims.

Claims

1. A method of event masking, comprising: detecting a pre-event marker based on sensor data, wherein the pre-event marker indicates a prediction of a first event, wherein a type of the first event comprises at least one of visual, auditory, or tactile; determining an event masking action corresponding to the first event, wherein the determined event masking action is of a same type as the first event; and in response to detection of the pre-event marker, performing the event masking action at an occurrence of the first event.

2. The method of claim 1, wherein: the first event further comprises an environmental event; and the event masking action is one or more of a visual action, an auditory action, a tactile action, or an environmental action.

3. The method of any one of claims 1 and 2, wherein the pre-event marker is determined using a first model. The first model is trained based at least on a correlation between one or more sensor data markers and the first event.

4. The method of claim 3, further comprising:

5. An apparatus for event masking, comprising: means for detecting a pre-event marker based on sensor data, wherein the pre-event marker indicates a prediction of a first event, wherein a type of the first event comprises at least one of visual, auditory, or tactile; means for determining an event masking action corresponding to the first event, wherein the determined event masking action is of a same type as the first event; and in response to detection of the pre-event marker, means for performing the event masking action at an occurrence of the first event.

6. The apparatus of claim 5, wherein the pre-event marker comprises one or more of an auditory marker, a visual marker, an environmental marker, or a tactile marker.

7. The apparatus of claim 5 or 6, wherein the first event further comprises an environmental event.

8. The apparatus of claim 5, wherein the event masking action is one or more of a visual action, an auditory action, a tactile action, or an environmental action.

9. The apparatus of claim 5, wherein the first event comprises an auditory event and the event masking action comprises an audio suppression action.

10. The apparatus of claim 5, wherein the pre-event marker is determined using a first model.

11. The apparatus of claim 10, wherein the first model is a machine learning model.

12. The apparatus of claim 10 or 11, further comprising means for training the first model, wherein the means for training the first model comprises means for determining a correlation between one or more sensor data features and the first event.

13. The apparatus of claim 12, further comprising means for associating one or more sensor data features with the first event if at least one of a uniqueness and a repeatability of the one or more sensor data features is above a first threshold, wherein the first model is trained using sensor data features associated with the first event.

14. The apparatus of claim 5, wherein the apparatus comprises: ​ ​ at least one processor; and at least one memory including computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the performance of the method of claim 1.

15. A computer program comprising instructions for causing an apparatus to perform at least the following: detecting a pre-marker based on sensor data, wherein the pre-marker is indicative of a prediction of a first event, wherein a type of the first event comprises at least one of visual, auditory, or tactile; determining an event-masking action corresponding to the first event, wherein the determined event-masking action is of a same type as the first event; and in response to detection of the pre-marker, performing the event-masking action at a time of occurrence of the first event.

Images