Glass-break sound detection

Abstract

Systems, devices, methods, and non-transitory, machine-readable media for glass-break sound detection are provided. A glass-break sound detector may be developed. A glass-break sound waveform may be decomposed into sound components that may be recombined in combinations to derive a glass-break sound dataset, which may be developed based on room reverberations and noise. A model may be trained to facilitate detection of glass-break sounds using the glass-break sound dataset. The model may be configured to receive second audio data and predict whether a sound event corresponds to a glass break. A glass-break detection device may monitor sounds in a home and feed an audio stream corresponding to the sounds to a detector. The detector may be configured with the model and may detect a sound event in the audio stream. The detector may predict whether the sound event corresponds to a glass break.

Description

GLASS-BREAK SOUND DETECTIONBACKGROUND

[0001] Detecting a glass break in a window of a home or other structure when an object impacts the window may be an important part of a security plan. However, the development of accurate glass-break detection devices and systems that correctly recognize glass-break events and differentiate glass-break events from other types of events may be a complex and expensive proposition. There is a need for systems, devices, and methods that address the challenges and problems encumbering full realization of the potential for glass-break detection devices and systems in various situations. This and other needs may be addressed by the present disclosure.BRIEF SUMMARY

[0002] Certain embodiments of the present disclosure relate generally to sound detection, and more particularly to systems, devices, methods, and non-transitory, machine-readable media for glass-break sound detection.

[0003] In one aspect, a method for developing a glass-break sound detector may be performed. The method may include one or a combination of the following. First audio data corresponding to glass-break sounds may be processed. A glass-break sound waveform may be identified from the first audio data. The glass-break sound waveform may be decomposed into sound components. The sound components may be recombined in various combinations to derive a glass-break sound dataset. A glass-break sound dataset may be developed based at least in part on incorporating room reverberations and / or noise into the glass-break sound dataset. A model may be trained to facilitate detection of glass-break sounds using the developed glass-break sound dataset. The model may be configured to receive second audio data corresponding to a sound event and predict whether the sound event corresponds to a glass break. The glass-break sounds may be anechoic glass-break sounds. The glass-break sounds may be captured in an anechoic chamber. The sound components may include an initial portion, an interim portion, and an end portion. Windowing functions may be applied to each of the initial portion, the interim portion, and the end portion prior to the recombining. The model may be a machine learning model and may be, more specifically, a neural network model.

[0004] In another aspect, a system to develop a glass-break sound detector is disclosed. The system may include one or more processing devices and memory communicatively coupled with and readable by the one or more processing devices. The memory may store machine-readable instructions thereon which, when executed by the one or more processing devices, cause the one ormore processing devices to perform one or a combination of the following operations. First audio data corresponding to glass-break sounds may be processed. A glass-break sound waveform may be identified from the first audio data. The glass-break sound waveform may be decomposed into sound components. The sound components may be recombined in various combinations to derive a glass-break sound dataset. A glass-break sound dataset may be developed based at least in part on incorporating room reverberations and / or noise into the glass-break sound dataset. A model may be trained to facilitate detection of glass-break sounds using the developed glass-break sound dataset. The model may be configured to receive second audio data corresponding to a sound event and predict whether the sound event corresponds to a glass break. The glass-break sounds may be anechoic glass-break sounds. The glass-break sounds may be captured in an anechoic chamber. The sound components may include an initial portion, an interim portion, and an end portion. Windowing functions may be applied to each of the initial portion, the interim portion, and the end portion prior to the recombining. The model may be a machine learning model and may be, more specifically, a neural network model.

[0005] In another aspect, one or more non-transitory, machine-readable media are disclosed as having machine-readable instructions thereon which, when executed by one or more processing devices, cause the one or more processing devices to perform one or a combination of the following operations. First audio data corresponding to glass-break sounds may be processed. A glass-break sound waveform may be identified from the first audio data. The glass-break sound waveform may be decomposed into sound components. The sound components may be recombined in various combinations to derive a glass-break sound dataset. A glass-break sound dataset may be developed based at least in part on incorporating room reverberations and / or noise into the glass-break sound dataset. A model may be trained to facilitate detection of glass-break sounds using the developed glass-break sound dataset. The model may be configured to receive second audio data corresponding to a sound event and predict whether the sound event corresponds to a glass break. The glass-break sounds may be anechoic glass-break sounds. The glass-break sounds may be captured in an anechoic chamber. The sound components may include an initial portion, an interim portion, and an end portion. Windowing functions may be applied to each of the initial portion, the interim portion, and the end portion prior to the recombining. The model may be a machine learning model and may be, more specifically, a neural network model.

[0006] In another aspect, a method for detecting a glass-break sound may be performed. The method may include one or a combination of the following. Sounds in a home or other structure that comprises one or more windows may be monitored. An audio stream corresponding to the sounds to may be fed to a detector. The detector may be configured with a model that is trained tofacilitate detection of glass-break sounds using a glass-break sound dataset. The glass-break sound dataset may be developed based at least in part on decomposing a glass-break sound waveform into sound components and recombining the sound components in various combinations to derive the glass-break sound dataset. A sound event may be detected in the audio stream. A prediction of whether the sound event corresponds to a glass break may be determined by the detector using the model. Based at least in part on the prediction, it may be determined that the sound event corresponds to a glass break. The prediction may be compared to a threshold, where the generating the glass-break alert is based at least in part on the prediction satisfying the threshold. The model may be a machine learning model and may be, more specifically, a neural network model. A sound-pressure level of the sound event may be estimated by an energy detector of the glass-break detection device, where the determining that the sound event corresponds to a glass break is based at least in part on the sound-pressure level of the sound event. The sound-pressure level of the sound event may be compared to a threshold, where the determining that the sound event corresponds to a glass break is based at least in part on the sound-pressure level of the sound event satisfying the threshold. Audio data corresponding to the sound event may be decomposed into a set of sound components, where the prediction is based at least in part on the set of sound components. A glass-break alert may be generated consequent to the determining that the sound event corresponds to a glass break.

[0007] In another aspect, a glass-break detection device is disclosed. The glass-break detection device may include one or more processing devices and memory communicatively coupled with and readable by the one or more processing devices. The memory may store machine-readable instructions thereon which, when executed by the one or more processing devices, cause the one or more processing devices to perform one or a combination of the following operations. Sounds in a home or other structure that comprises one or more windows may be monitored. An audio stream corresponding to the sounds to may be fed to a detector. The detector may be configured with a model that is trained to facilitate detection of glass-break sounds using a glass-break sound dataset. The glass-break sound dataset may be developed based at least in part on decomposing a glassbreak sound waveform into sound components and recombining the sound components in various combinations to derive the glass-break sound dataset. A sound event may be detected in the audio stream. A prediction of whether the sound event corresponds to a glass break may be determined by the detector using the model. Based at least in part on the prediction, it may be determined that the sound event corresponds to a glass break. The prediction may be compared to a threshold, where the generating the glass-break alert is based at least in part on the prediction satisfying the threshold. The model may be a machine learning model and may be, more specifically, a neuralnetwork model. A sound-pressure level of the sound event may be estimated by an energy detector of the glass-break detection device, where the determining that the sound event corresponds to a glass break is based at least in part on the sound-pressure level of the sound event. The sound-pressure level of the sound event may be compared to a threshold, where the determining that the sound event corresponds to a glass break is based at least in part on the soundpressure level of the sound event satisfying the threshold. Audio data corresponding to the sound event may be decomposed into a set of sound components, where the prediction is based at least in part on the set of sound components. A glass-break alert may be generated consequent to the determining that the sound event corresponds to a glass break.

[0008] In another aspect, one or more non-transitory, machine-readable media are disclosed as having machine-readable instructions thereon which, when executed by one or more processing devices, cause the one or more processing devices to perform one or a combination of the following operations. Sounds in a home or other structure that comprises one or more windows may be monitored. An audio stream corresponding to the sounds to may be fed to a detector. The detector may be configured with a model that is trained to facilitate detection of glass-break sounds using a glass-break sound dataset. The glass-break sound dataset may be developed based at least in part on decomposing a glass-break sound waveform into sound components and recombining the sound components in various combinations to derive the glass-break sound dataset. A sound event may be detected in the audio stream. A prediction of whether the sound event corresponds to a glass break may be determined by the detector using the model. Based at least in part on the prediction, it may be determined that the sound event corresponds to a glass break. The prediction may be compared to a threshold, where the generating the glass-break alert is based at least in part on the prediction satisfying the threshold. The model may be a machine learning model and may be, more specifically, a neural network model. A sound-pressure level of the sound event may be estimated by an energy detector of the glass-break detection device, where the determining that the sound event corresponds to a glass break is based at least in part on the sound-pressure level of the sound event. The sound-pressure level of the sound event may be compared to a threshold, where the determining that the sound event corresponds to a glass break is based at least in part on the sound-pressure level of the sound event satisfying the threshold. Audio data corresponding to the sound event may be decomposed into a set of sound components, where the prediction is based at least in part on the set of sound components. A glass-break alert may be generated consequent to the determining that the sound event corresponds to a glass break.

[0009] Further areas of applicability of the present disclosure will become apparent from the detailed description provided hereinafter. It should be understood that the detailed description andspecific examples, while indicating various embodiments, are intended for purposes of illustration only and are not intended to necessarily limit the scope of the disclosure.BRIEF DESCRIPTION OF THE DRAWINGS[00101 A further understanding of the nature and advantages of various embodiments may be realized by reference to the following figures. In the appended figures, similar components or features may have the same reference label. Further, various components of the same type may be distinguished by following the reference label by a dash and a second label that distinguishes among the similar components. If only the first reference label is used in the specification, the description is applicable to any one of the similar components having the same first reference label irrespective of the second reference label.

[0011] FIG. 1 illustrates an example environment in which various smart-home devices may include the componentry of a glass-break detection device to perform glass-break detection, in accordance with embodiments according to the present disclosure.

[0012] FIG. 2 illustrates a simplified diagram of an example glass-break modeling system, in accordance with embodiments according to the present disclosure.

[0013] FIG. 3 illustrates example sound components of a glass-break sound waveform for an example glass-break sound event that may be identified in accordance with embodiments according to the present disclosure.

[0014] FIG. 4 illustrates a high-level block diagram illustrating an example glass-break sound segmentation process for segmenting anechoic glass-break sounds into sound components, in accordance with embodiments according to the present disclosure.

[0015] FIG. 5 illustrates a high-level block diagram illustrating an example glass-break sound augmentation process based at least in part on the sound components to generate a much larger dataset for training a glass-break sound detection model, in accordance with embodiments according to the present disclosure.

[0016] FIG. 6 illustrates a more detailed block diagram of an example combiner, in accordance with embodiments according to the present disclosure, in accordance with disclosed embodiments of the present disclosure.

[0017] FIG. 7 illustrates examples of window shapes corresponding to the window functions, in accordance with embodiments according to the present disclosure.5RECTIFIED SHEET (RULE 91) ISA / EP

[0018] FIG. 8 illustrates a block diagram of a sound-detection module, in accordance with embodiments according to the present disclosure.

[0019] FIG. 9 illustrates an example method for the sound-detection module, in accordance with embodiments according to the present disclosure.

[0020] FIG. 10 illustrates a schematic an example embodiment of a computer system that can perform various steps of the methods disclosed herein, in accordance with embodiments according to the present disclosure.DETAILED DESCRIPTION

[0021] The ensuing description provides preferred exemplary embodiment(s) only, and is not intended to limit the scope, applicability or configuration of the disclosure. Rather, the ensuing description of the preferred exemplary embodiment(s) will provide those skilled in the art with an enabling description for implementing a preferred exemplary embodiment of the disclosure. It should be understood that various changes may be made in the function and arrangement of elements without departing from the spirit and scope of the disclosure as set forth in the appended claims.

[0022] Specific details may be given in the following description to provide a thorough understanding of the embodiments. However, it will be understood by one of ordinary skill in the art that the embodiments maybe practiced without these specific details. For example, circuits may be shown in block diagrams in order not to obscure the embodiments in unnecessary detail. In other instances, well-known circuits, processes, algorithms, structures, and techniques may be shown without unnecessary detail in order to avoid obscuring the embodiments.

[0023] Also, it is noted that the embodiments may be described as a process which is depicted as a flowchart, a flow diagram, a data flow diagram, a structure diagram, or a block diagram. Although a flowchart may describe the operations as a sequential process, many of the operations can be performed in parallel or concurrently. In addition, the order of the operations may be rearranged. A process is terminated when its operations may be completed but could have additional steps not included in the figure. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, its termination corresponds to a return of the function to the calling function or the main function.

[0024] Various embodiments according to the present disclosure may provide for a novel approach for developing an improved glass-break sound detector. The glass-break sound detector may be configured with a neural -network model to detect glass-break sounds in various kinds ofroom environments. Audio data augmentation techniques, such as adding room reverb and background noise, may help make the glass-break sound detector be more robust to such variations. However, such audio data augmentation techniques may not enhance the variety of the clean (i.e., anechoic or near-anechoic data with high signal-to-noise ratios) glass-break sound data. Since obtaining clean glass-break sound data is an expensive and time-consuming endeavor, it becomes important to develop techniques to generate more clean glass-break sound data from initial clean glass-break sound data that have been collected. Various embodiments may enhance the variety of the clean glass-break sound data and may train the neural-network model to facilitate the improved glass-break sound detector. Various embodiments may further provide for improvements in adaptability and responsiveness of providing faster and maximized detection of glass breaks while minimizing or eliminating false positives caused by interference and / or similar sounds learned by the glass-break sound detector.

[0025] Various embodiments will now be discussed in greater detail with reference to the accompanying figures, beginning with FIG. 1. FIG. 1 illustrates an example environment 100 in which various smart-home devices may include the componentry of a glass-break detection device (e.g., glass-break detection device 260 in FIG. 2) to perform glass-break detection, in accordance with embodiments according to the present disclosure. Various smart-home devices may benefit from the ability to detect glass breaks. The smart home environment 100, which may correspond to a smart home environment, includes a structure 150 (e.g., a house, daycare, office building, apartment, condominium, garage, or mobile home) with various integrated and / or non-integrated devices. It will be appreciated that devices may also be integrated into or otherwise included in a smart home environment 100 that does not include an entire structure 150, such as an apartment or condominium. Further, the smart home environment 100 may control and / or be coupled to devices outside of the actual structure 150. Indeed, several devices in the smart home environment 100 need not be physically within the structure 150. It is to be appreciated that “smart home environments” may refer to smart environments for homes such as a single-family house, but the scope of the present teachings is not so limited. The present teachings are also applicable, without limitation, to duplexes, townhomes, multi-unit apartment buildings, hotels, retail stores, office buildings, industrial buildings, and more generally any living space or work space.

[0026] The depicted structure 150 includes a plurality of rooms 152, separated at least partly from each other via walls 154. The walls 154 may include interior walls or exterior walls. Each room may further include a floor 156 and a ceiling 158. Further, the structure 150 may include one or more windows 159 in one or more exterior walls, as is typical. A window 159 may be inany suitable location in the structure 150 and may be in locations other than exterior walls (e.g., skylights, within doors, within door frames, as part of storm doors or other glass doors, etc.).

[0027] Devices may be mounted on, integrated with and / or supported by a wall 154, floor 156 or ceiling 158. In some implementations, the integrated devices of the smart home environment 100 include intelligent, multi-sensing, network-connected devices that integrate seamlessly with each other in a smart home network and / or with a central server or a cloud-computing system to provide a variety of useful smart home functions. The smart home environment 100 may include one or more intelligent, multi-sensing, network-connected thermostats 102 (hereinafter referred to as “smart thermostats 102”), one or more intelligent, network-connected, multi-sensing hazard detection units 104 (hereinafter referred to as “smart hazard detectors 104”), one or more intelligent, multi-sensing, network-connected entry way interface devices 106 and 120 and one or more intelligent, multi-sensing, network-connected alarm systems 122 (hereinafter referred to as “smart alarm systems 122”). Each of these devices may have the functionality of a glass-break detection device 260 incorporated.

[0028] In some implementations, the one or more smart thermostats 102 detect ambient climate characteristics (e.g., temperature and / or humidity) and control an HVAC system 103 accordingly. For example, a respective smart thermostat 102 includes an ambient temperature sensor. A smart hazard detector may detect smoke, carbon monoxide, and / or some other hazard present in the environment. The one or more smart hazard detectors 104 may include thermal radiation sensors directed at respective heat sources (e.g., a stove, oven, other appliances, a fireplace, etc.). For example, a smart hazard detector 104 in a kitchen 153 includes a thermal radiation sensor directed at a network-connected appliance 112. A thermal radiation sensor may determine the temperature of the respective heat source (or a portion thereof) at which it is directed and may provide corresponding black-body radiation data as output.

[0029] The smart doorbell 106 and / or the smart door lock 120 may detect a person’s approach to or departure from a location (e.g., an outer door), control doorbell / door locking functionality (e.g., receive user inputs from a portable electronic device 166-1 to actuate the bolt of the smart door lock 120), announce a person’s approach or departure via audio or visual means, and / or control settings on a security system (e.g., to activate or deactivate the security system when occupants go and come). In some implementations, the smart doorbell 106 includes some or all of the components and features of the camera 118-1. In some implementations, the smart doorbell 106 includes a camera 118-1, and, therefore, is also called “doorbell camera 106” in this document. Cameras 118-1 and / or 118-2 may function as a streaming video camera and the streaming audiodevice detailed in relation to various embodiments herein. Cameras 118 may be mounted in a location, such as indoors and to a wall or can be moveable and placed on a surface, such as illustrated with camera 118-2. Various embodiments of cameras 118 may be installed indoors or outdoors. Each of these types of devices may have the functionality of a glass-break detection device 260 incorporated.

[0030] The smart alarm system 122 may detect the presence of an individual within close proximity (e.g., using built-in IR sensors), sound an alarm (e.g., through a built-in speaker, or by sending commands to one or more external speakers), and send notifications to entities or users within / outside of the smart home environment 100. In some implementations, the smart alarm system 122 may also include one or more input devices or sensors (e.g., keypad, biometric scanner, NFC transceiver, microphone) for verifying the identity of a user, and one or more output devices (e.g., display, speaker). In some implementations, the smart alarm system 122 may also be set to an armed mode, such that detection of a trigger condition or event causes the alarm to be sounded unless a disarming action is performed. Each of these devices may have the functionality of a glass-break detection device 260 incorporated.

[0031] In some implementations, the smart home environment 100 may include one or more intelligent, multi-sensing, network-connected wall switches 108 (hereinafter referred to as “smart wall switches 108”), along with one or more intelligent, multi-sensing, network-connected wall plug interfaces 110 (hereinafter referred to as “smart wall plugs 110”). The smart wall switches 108 may detect ambient lighting conditions, detect room-occupancy states, and control a power and / or dim state of one or more lights. In some instances, the smart wall switches 108 may also control a power state or speed of a fan, such as a ceiling fan. The smart wall plugs 110 may detect occupancy of a room or enclosure and control the supply of power to one or more wall plugs (e.g., such that power is not supplied to the plug if nobody is at home). Each of these types of devices may have the functionality of a glass-break detection device 260 incorporated.

[0032] In some implementations, the smart home environment 100 of FIG. 1 may include a plurality of intelligent, multi-sensing, network-connected appliances 112 (hereinafter referred to as “smart appliances 112”), such as refrigerators, stoves, ovens, televisions, washers, dryers, lights, stereos, intercom systems, wall clock, garage-door openers, floor fans, ceiling fans, wall air conditioners, pool heaters, irrigation systems, security systems, space heaters, window AC units, motorized duct vents, and so forth. Each of these devices may have the functionality of a glassbreak detection device 260 incorporated. In some implementations, when plugged in, an appliance may announce itself to the smart home network, such as by indicating what type of appliance it is,and it may automatically integrate with the controls of the smart home. Such communication by the appliance to the smart home may be facilitated by either a wired or wireless communication protocol. The smart home may also include a variety of non-communicating legacy appliances 140, such as old conventional washer / dryers, refrigerators, and the like, which may be controlled by smart wall plugs 110. The smart home environment 100 may further include a variety of partially communicating legacy appliances 142, such as infrared (“IR”) controlled wall air conditioners or other IR-controlled devices, which may be controlled by IR signals provided by the smart hazard detectors 104 or the smart wall switches 108.

[0033] In some implementations, the smart home environment 100 may include one or more network-connected cameras 118 that are configured to provide video monitoring and security in the smart home environment 100. The cameras 118 may be used to determine occupancy of the structure 150 and / or particular rooms 152 in the structure 150, and thus may act as occupancy sensors. For example, video captured by the cameras 118 may be processed to identify the presence of an occupant in the structure 150 (e.g., in a particular room 152). Specific individuals may be identified based, for example, on their appearance (e.g., height, face) and / or movement (e.g., their walk / gait). Cameras 118 may additionally include one or more sensors (e.g., IR sensors, motion detectors), input devices (e.g., microphone for capturing audio), and output devices (e.g., speaker for outputting audio). In some implementations, the cameras 118 may be configured to operate in a day mode and in a low-light mode (e.g., a night mode). In some implementations, the cameras 118 each include one or more IR illuminators for providing illumination while the camera is operating in the low-light mode. In some implementations, the cameras 118 may include one or more outdoor cameras. In some implementations, the outdoor cameras may include additional features and / or components such as weatherproofing and / or solar ray compensation. Such cameras may have the functionality of a glass-break detection device 260 incorporated.

[0034] The smart home environment 100 may additionally or alternatively include one or more other occupancy sensors (e.g., the smart doorbell 106, smart door locks 120, touch screens, IR sensors, microphones, ambient light sensors, motion detectors, smart nightlights 170, etc.). In some implementations, the smart home environment 100 includes radio-frequency identification (RFID) readers (e.g., in each room 152 or a portion thereof) that determine occupancy based on RFID tags located on or embedded in occupants. For example, RFID readers may be integrated into the smart hazard detectors 104. Each of these devices may have the functionality of a glassbreak detection device 260 incorporated.

[0035] The smart home assistant 119 may have one or more microphones that continuously listen to an ambient environment. The smart home assistant 119 may be able to respond to verbal queries posed by a user, possibly preceded by a triggering phrase. The smart home assistant 119 may stream audio and, possibly, video if a camera is integrated as part of the device, to a cloudbased server system 164 (which represents an embodiment of cloud-based host system 200 of FIG. 2). The smart home assistant 119 may be a smart device through which non-auditory discomfort alerts may be output and / or an audio stream from the streaming video camera can be output. As previously noted, the smart home assistant 119 may have the functionality of a glass-break detection device 260 incorporated.

[0036] By virtue of network connectivity, one or more of the smart-home devices of FIG. 1 may further allow a user to interact with the device even if the user is not proximate to the device. For example, a user may communicate with a device using a computer (e.g., a desktop computer, laptop computer, or tablet) or other portable electronic device 166 (e.g., a mobile phone, such as a smart phone). A webpage or application may be configured to receive communications from the user and control the device based on the communications and / or to present information about the device’s operation to the user. For example, the user may view a current set point temperature for a device (e.g., a stove) and adjust it using a computer. As another example, the user may listen to and / or view glass-break data (e.g., an audio recording of a glass break, a date and time stamp corresponding to the glass break detection, a location of the glass break, a descriptor of the glassbreak detection device 260 or the device incorporating the glass-break detection device 260 that detected the glass break, and / or the like); video data output by a camera device (e.g., cameras 118 or smart doorbell 106) for a time period before and after the glass-break detection; and / or the like. The user may be in the structure during this remote communication or outside the structure.

[0037] As discussed above, users may control smart devices in the smart home environment 100 using a network-connected computer or portable electronic device 166. In some examples, some or all of the occupants (e.g., individuals who live in the home) may register their portable electronic device 166 with the smart home environment 100. Such registration may be made at a central server to authenticate the occupant and / or the device as being associated with the home and to give permission to the occupant to use the device to control the smart devices in the home. An occupant may use their registered portable electronic device 166 to remotely control the smart devices of the home, such as when the occupant is at work or on vacation. The occupant may also use their registered device to control the smart devices when the occupant is actually located inside the home, such as when the occupant is sitting on a couch inside the home. It should be appreciated that instead of or in addition to registering portable electronic devices 166, the smarthome environment 100 may make inferences about which individuals live in the home and are therefore occupants and which portable electronic devices 166 are associated with those individuals. As such, the smart home environment may “learn” who is an occupant and permit the portable electronic devices 166 associated with those individuals to control the smart devices of the home.

[0038] In some implementations, in addition to containing processing and sensing capabilities, smart thermostat 102, smart hazard detector 104, smart doorbell 106, smart wall switch 108, smart wall plug 110, network-connected appliances 112, camera 118, smart home assistant 119, smart door lock 120, and / or smart alarm system 122 (collectively referred to as “the smart-home devices”) may be capable of data communications and information sharing with other smart devices, a central server or cloud-computing system, and / or other devices that are network- connected. Data communications may be carried out using any of a variety of custom or standard wireless protocols (e.g., IEEE 802.15.4, Wi-Fi, ZigBee, ILoWPAN, Thread, Z-Wave, Bluetooth Smart, ISA100.5A, WirelessHART, MiWi, etc.) and / or any of a variety of custom or standard wired protocols (e.g., Ethernet, HomePlug, etc ), or any other suitable communication protocol, including communication protocols not yet developed as of the filing date of this document.

[0039] In some implementations, the smart devices may serve as wireless or wired repeaters, for example, to communicate glass-break data (e.g., an alert about a glass-break detection, an audio recording of the glass break, a date and time stamp corresponding to the glass break detection, a location of the glass break, a descriptor of the glass-break detection device 260 or the device incorporating the glass-break detection device 260 that detected the glass break, and / or the like); video data output by a camera device (e.g., cameras 118 or smart doorbell 106) for a time period before and after the glass-break detection; and / or the like. In some implementations, a first one of the smart devices may communicate glass-break data with a second one of the smart devices via a wireless router. The smart devices may further communicate glass-break data with each other via a connection (e.g., network interface 160) to a network, such as the Internet. Through the Internet, the smart devices may communicate glass-break data with a cloud-based server system 164 (also called a cloud-based server system, central server system, and / or a cloud-computing system herein). The cloud-based server system 164 may be associated with a manufacturer, support entity, or service provider associated with the smart device(s). In some implementations, a user may be able to contact customer support using a smart device itself rather than needing to use other communication means, such as a telephone or Internet-connected computer. In some implementations, software updates may be automatically sent from cloud-based server system 164 to smart devices (e.g., when available, when purchased, or at routine intervals).

[0040] In some implementations, the network interface 160 may include a conventional network device (e.g., a router), and the smart home environment 100 of FIG. 1 may include a hub device 180 that is communicatively coupled to the network(s) 162 directly or via the network interface 160 and that may facilitate communication of glass-break data. The hub device 180 may be further communicatively coupled to one or more of the above intelligent, multi-sensing, network- connected devices (e.g., smart devices of the smart home environment 100). Each of these smart devices may optionally communicate with the hub device 180 using one or more radio communication networks available at least in the smart home environment 100 (e.g., ZigBee, Z- Wave, Insteon, Bluetooth, Wi-Fi and other radio communication networks). In some implementations, the hub device 180 and devices coupled with / to the hub device may be controlled and / or interacted with via an application running on a smart phone, household controller, laptop, tablet computer, game console or similar electronic device. In some implementations, a user of such controller application may view the status of the hub device or coupled smart devices, configure the hub device to interoperate with smart devices newly introduced to the home network, commission new smart devices, and adjust or view settings of connected smart devices, etc. In some implementations, the hub device may extend capabilities of low-capability smart devices to match capabilities of the highly capable smart devices of the same type, integrates functionality of multiple different device types - even across different communication protocols - and may be configured to streamline adding of new devices and commissioning of the hub device. In some implementations, hub device 180 may further include a local storage device for storing data related to, or output by, smart devices of smart home environment 100. In some implementations, the data may include one or more of: glass-break data (e.g., an alert about a glass-break detection, an audio recording of the glass break, a date and time stamp corresponding to the glass break detection, a location of the glass break, a descriptor of the glass-break detection device 260 or the device incorporating the glass-break detection device 260 that detected the glass break, and / or the like); video data output by a camera device for a time period before and after the glass-break detection; metadata output by a smart device, settings information for a smart device; usage logs for a smart device; and / or the like.

[0041] In some implementations, the smart home environment 100 may include a local storage device 190 for storing data related to, or output by, smart devices of the smart home environment 100. In some implementations, the data may include one or more of: glass-break data (e g., an audio recording of a glass break, a date and time stamp corresponding to the glass break detection, a location of the glass break, a descriptor of the glass-break detection device 260 or the device incorporating the glass-break detection device 260 that detected the glass break, and / or the like);video data output by a camera device (e.g., cameras 118 or smart doorbell 106) for a time period before and after the glass-break detection; metadata output by a smart device; settings information for a smart device; usage logs for a smart device; and / or the like. In some implementations, local storage device 190 may be communicatively coupled to one or more smart devices via a smart home network. In some implementations, local storage device 190 may be selectively coupled to one or more smart devices via a wired and / or wireless communication network. In some implementations, the local storage device 190 may be used to store glass-break data and / or video data when external network conditions are poor. For example, the local storage device 190 may be used when an encoding bitrate of cameras 118 exceeds the available bandwidth of the external network (e.g., network(s) 162). In some implementations, the local storage device 190 may temporarily store glass-break data and / or video data from one or more cameras (e.g., cameras 118) prior to transferring the video data to a server system (e.g., cloud-based server system 164).

[0042] Further included and illustrated in the exemplary smart home environment 100 of FIG. 1 are service robots 168, each configured to carry out, in an autonomous manner, any of a variety of household tasks. For some embodiments, the service robots 168 may be respectively configured to perform floor sweeping, floor washing, etc. In some embodiments, a service robot may follow a person from room to room and position itself such that the person can be monitored while in the room. The service robot may stop in a location within the room where it will likely be out of the way, but still has a relatively clear field-of-view of the room. Service robots 168 may have the functionality of smart-home device 201 incorporated such that falls can be detected. Such an arrangement may have the advantage of allowing one service robot with the functionality of a glass-break detection device 260 incorporated to monitor for glass-break throughout multiple rooms or in a large space.

[0043] Accordingly, in various embodiments, any one or a combination of the above-described smart-home devices may be configured to operate as a glass-break detection device 260. Thus, one or more glass-break detection devices 260 may be in various locations throughout the environment 100 home or other structure to monitor one or more windows within one or more rooms and / or within one or more distances from the one or more glass-break detection devices 260. For example, monitoring for glass breaks may be performed continuously in the room where smart-home device configured as a glass-break detection device 260 is located.

[0044] Various methods may be performed using the systems and embodiments of following figures. In some embodiments, a method for developing a glass-break sound detector may be performed. The method may include one or a combination of the following. First audio datacorresponding to glass-break sounds may be processed. A glass-break sound waveform may be identified from the first audio data. The glass-break sound waveform may be decomposed into sound components. The sound components may be recombined in various combinations to derive a glass-break sound dataset. A glass-break sound dataset may be developed based at least in part on incorporating room reverberations and / or noise into the glass-break sound dataset. A model may be trained to facilitate detection of glass-break sounds using the developed glass-break sound dataset. The model may be configured to receive second audio data corresponding to a sound event and predict whether the sound event corresponds to a glass break. The glass-break sounds may be anechoic glass-break sounds. The glass-break sounds may be captured in an anechoic chamber. The sound components may include an initial portion, an interim portion, and an end portion. Windowing functions may be applied to each of the initial portion, the interim portion, and the end portion prior to the recombining. The model may be a machine learning model and may be, more specifically, a neural network model.

[0045] In some embodiments, a system to develop a glass-break sound detector may include one or more processing devices and memory communicatively coupled with and readable by the one or more processing devices. The memory may store machine-readable instructions thereon which, when executed by the one or more processing devices, cause the one or more processing devices to perform one or a combination of the following operations. First audio data corresponding to glassbreak sounds may be processed. A glass-break sound waveform may be identified from the first audio data. The glass-break sound waveform may be decomposed into sound components. The sound components may be recombined in various combinations to derive a glass-break sound dataset. A glass-break sound dataset may be developed based at least in part on incorporating room reverberations and / or noise into the glass-break sound dataset. A model may be trained to facilitate detection of glass-break sounds using the developed glass-break sound dataset. The model may be configured to receive second audio data corresponding to a sound event and predict whether the sound event corresponds to a glass break. The glass-break sounds may be anechoic glass-break sounds. The glass-break sounds may be captured in an anechoic chamber. The sound components may include an initial portion, an interim portion, and an end portion. Windowing functions may be applied to each of the initial portion, the interim portion, and the end portion prior to the recombining. The model may be a machine learning model and may be, more specifically, a neural network model.

[0046] In some embodiments, one or more non-transitory, machine-readable media having machine-readable instructions thereon which, when executed by one or more processing devices, cause the one or more processing devices to perform one or a combination of the followingoperations. First audio data corresponding to glass-break sounds may be processed. A glass-break sound waveform may be identified from the first audio data. The glass-break sound waveform may be decomposed into sound components. The sound components may be recombined in various combinations to derive a glass-break sound dataset. A glass-break sound dataset may be developed based at least in part on incorporating room reverberations and / or noise into the glassbreak sound dataset. A model may be trained to facilitate detection of glass-break sounds using the developed glass-break sound dataset. The model may be configured to receive second audio data corresponding to a sound event and predict whether the sound event corresponds to a glass break. The glass-break sounds may be anechoic glass-break sounds. The glass-break sounds may be captured in an anechoic chamber. The sound components may include an initial portion, an interim portion, and an end portion. Windowing functions may be applied to each of the initial portion, the interim portion, and the end portion prior to the recombining. The model may be a machine learning model and may be, more specifically, a neural network model.

[0047] In some embodiments, a method for detecting a glass-break sound may be performed. The method may include one or a combination of the following. Sounds in a home or other structure that comprises one or more windows may be monitored. An audio stream corresponding to the sounds to may be fed to a detector. The detector may be configured with a model that is trained to facilitate detection of glass-break sounds using a glass-break sound dataset. The glassbreak sound dataset may be developed based at least in part on decomposing a glass-break sound waveform into sound components and recombining the sound components in various combinations to derive the glass-break sound dataset. A sound event may be detected in the audio stream. A prediction of whether the sound event corresponds to a glass break may be determined by the detector using the model. Based at least in part on the prediction, it may be determined that the sound event corresponds to a glass break. The prediction may be compared to a threshold, where the generating the glass-break alert is based at least in part on the prediction satisfying the threshold. The model may be a machine learning model and may be, more specifically, a neural network model. A sound-pressure level of the sound event may be estimated by an energy detector of the glass-break detection device, where the determining that the sound event corresponds to a glass break is based at least in part on the sound-pressure level of the sound event. The sound-pressure level of the sound event may be compared to a threshold, where the determining that the sound event corresponds to a glass break is based at least in part on the soundpressure level of the sound event satisfying the threshold. Audio data corresponding to the sound event may be decomposed into a set of sound components, where the prediction is based at least inpart on the set of sound components. A glass-break alert may be generated consequent to the determining that the sound event corresponds to a glass break.

[0048] In some embodiments, a glass-break detection device may include one or more processing devices and memory communicatively coupled with and readable by the one or more processing devices. The memory may store machine-readable instructions thereon which, when executed by the one or more processing devices, cause the one or more processing devices to perform one or a combination of the following operations. Sounds in a home or other structure that comprises one or more windows may be monitored. An audio stream corresponding to the sounds to may be fed to a detector. The detector may be configured with a model that is trained to facilitate detection of glass-break sounds using a glass-break sound dataset. The glass-break sound dataset may be developed based at least in part on decomposing a glass-break sound waveform into sound components and recombining the sound components in various combinations to derive the glass-break sound dataset. A sound event may be detected in the audio stream. A prediction of whether the sound event corresponds to a glass break may be determined by the detector using the model. Based at least in part on the prediction, it may be determined that the sound event corresponds to a glass break. The prediction may be compared to a threshold, where the generating the glass-break alert is based at least in part on the prediction satisfying the threshold. The model may be a machine learning model and may be, more specifically, a neural network model. A sound-pressure level of the sound event may be estimated by an energy detector of the glass-break detection device, where the determining that the sound event corresponds to a glass break is based at least in part on the sound-pressure level of the sound event. The sound-pressure level of the sound event may be compared to a threshold, where the determining that the sound event corresponds to a glass break is based at least in part on the sound-pressure level of the sound event satisfying the threshold. Audio data corresponding to the sound event may be decomposed into a set of sound components, where the prediction is based at least in part on the set of sound components. A glass-break alert may be generated consequent to the determining that the sound event corresponds to a glass break.

[0049] In some embodiments, one or more non-transitory, machine-readable media having machine-readable instructions thereon which, when executed by one or more processing devices, cause the one or more processing devices to perform one or a combination of the following operations. Sounds in a home or other structure that comprises one or more windows may be monitored. An audio stream corresponding to the sounds to may be fed to a detector. The detector may be configured with a model that is trained to facilitate detection of glass-break sounds using a glass-break sound dataset. The glass-break sound dataset may be developed based at least in parton decomposing a glass-break sound waveform into sound components and recombining the sound components in various combinations to derive the glass-break sound dataset. A sound event may be detected in the audio stream. A prediction of whether the sound event corresponds to a glass break may be determined by the detector using the model. Based at least in part on the prediction, it may be determined that the sound event corresponds to a glass break. The prediction may be compared to a threshold, where the generating the glass-break alert is based at least in part on the prediction satisfying the threshold. The model may be a machine learning model and may be, more specifically, a neural network model. A sound-pressure level of the sound event may be estimated by an energy detector of the glass-break detection device, where the determining that the sound event corresponds to a glass break is based at least in part on the sound-pressure level of the sound event. The sound-pressure level of the sound event may be compared to a threshold, where the determining that the sound event corresponds to a glass break is based at least in part on the sound-pressure level of the sound event satisfying the threshold. Audio data corresponding to the sound event may be decomposed into a set of sound components, where the prediction is based at least in part on the set of sound components. A glass-break alert may be generated consequent to the determining that the sound event corresponds to a glass break.

[0050] FIG. 2 depicts a simplified diagram of a glass-break modeling system 200, in accordance with embodiments according to the present disclosure. In some embodiments, the glass-break modeling system 200 may be included in the server system 164. The glass-break modeling system 200 may include a modeling control computer system 205 and a data storage system 210. In various embodiments, the glass-break modeling system 200 may include a set of devices configured to process, transform, encode, translate, send, receive, retrieve, detect, generate, compute, organize, categorize, qualify, store, display, present, handle, or use information and / or data suitable for the embodiments described herein. For example, computers of the glass-break modeling system 200 may be used to store software programs and data. In various embodiments, the system 200 may include, for example, software applications, machine learning and Al tools, spectral analysis software, one or more audio analyzers, and / or the like for recording audio, generating audio, segmenting audio data, automatic audio tagging, rearranging audio segments, combining audio segments, applying effects, other editing, and / or the like to facilitate the features disclosed herein.

[0051] Software implementing the systems and methods described herein may be stored on storage media in the computers, which may correspond to the server system 164. Thus, the software may be run from the storage media in the computers. In some embodiments, software implementing the systems and methods described herein may be stored on storage media of otherdevices described herein. The glass-break modeling system 200 may be implemented in or with a distributed computing and / or cloud computing environment with a plurality of servers and cloud- implemented resources. The glass-break modeling system 200 may include processing resources communicatively coupled to storage media, random access memory (RAM), read-only memory (ROM), and / or other types of memory. The glass-break modeling system 200 may include various input and output (I / O) devices, network ports, and display devices.

[0052] In various embodiments, the glass-break modeling system 200 may be containerized, cloud-native, and / or composed of one or more specialized computers, specialized server computers (including, by way of example, PC (personal computer) servers, UNIX® servers, mid-range servers, mainframe computers, rack-mounted servers, etc.), server farms, server clusters, or any other appropriate arrangement and / or combination. Accordingly, the glass-break modeling system 200 may also run on server and storage systems that are not part of the system, such that the system is portable across hardware. In various embodiments, the server system 164 may be adapted to run one or more services described herein. The glass-break modeling system 200 may run an operating system, which may correspond to a server operating system. The glass-break modeling system 200 may also run any of a variety of additional server applications and / or midtier applications, including HTTP (hypertext transport protocol) servers, FTP (file transfer protocol) servers, CGI (common gateway interface) servers, JAVA® servers, database servers, and the like. Exemplary database servers include without limitation those available from AWS, Oracle, Microsoft, Sybase, IBM (International Business Machines), and the like.

[0053] The server system 164 may include one or more system coordination servers. The system coordination servers may include any suitable type of server including, for example, a rack server, a tower server, a miniature server, a blade server, a mini rack server, a mobile server, an ultra-dense server, a super server, or the like, and may include various hardware components, for example, a motherboard, one or more processing units, memory systems, hard drives, network interfaces, power supplies, etc. System coordination servers may include one or more server farms, clusters, or any other appropriate arrangement and / or combination or computer servers. System coordination servers may operate according to stored instructions located in a memory subsystem of the servers, and may run an operating system, including any suitable server operating system and / or any other operating systems discussed herein.

[0054] The data storage system 210 may include one or more data storage servers, fde-based storage systems, block storage systems, and / or cloud object storage systems. Data storages may comprise stored data germane to the functions of the glass-break modeling system 200. In someembodiments, multiple data storages may reside on a single server, either using the same storage components of the server or using different physical storage components to assure data security and integrity between data storages. In other embodiments, each data storage may have a separate dedicated data storage server.

[0055] The data storage servers can access data that can be stored on a variety of hardware components. These hardware components can include, for example, components forming tier 0 storage, components forming tier 1 storage, components forming tier 2 storage, and / or any other tier of storage. In some embodiments, tier 0 storage refers to storage that is the fastest tier of storage in the data storage server, and particularly, the tier 0 storage is the fastest storage that is not RAM or cache memory. In some embodiments, the tier 0 memory can be embodied in solid state memory such as, for example, a solid-state drive (SSD) and / or flash memory. In some embodiments, the tier 1 storage refers to storage that is one or several higher performing systems in the memory management system, and that is relatively slower than tier 0 memory, and relatively faster than other tiers of memory. The tier 1 memory can be one or several hard disks that can be, for example, high-performance hard disks. These hard disks can be one or both of physically or communicatively connected such as, for example, by one or several fiber channels. In some embodiments, the one or several disks can be arranged into a disk storage system, and specifically can be arranged into an enterprise class disk storage system. The disk storage system can include any correct level of redundancy to protect data stored therein, and in one embodiment, the disk storage system can be made with grid architecture that creates parallelism for uniform allocation of system resources and balanced data distribution.

[0056] In some embodiments, the tier 2 storage refers to storage that includes one or several relatively lower performing systems in the memory management system, as compared to the tier 1 and tier 2 storages. Thus, tier 2 memory is relatively slower than tier 1 and tier 0 memories. Tier 2 memory can include one or several SATA-drives (e.g., Serial AT Attachment drives) or one or several NL-SATA drives. In some embodiments, the one or several hardware and / or software components of the data storage server can be arranged into one or several storage area networks (SAN), which one or several storage area networks can be one or several dedicated networks that provide access to data storage, and particularly that provides access to combined (eliminating data not useful), block level data storage. A SAN typically has its own network of storage devices that are generally not accessible through the local area network (LAN) by other devices. The SAN allows access to these devices in a manner such that these devices appear to be locally attached to the endpoint device.

[0057] The glass-break modeling system 200 may include transceiving equipment capable of transmitting and receiving data wirelessly and / or otherwise through RF, cellular, Wi-Fi, Ethernet, and / or the like connections disclosed herein. The transceiving equipment may include one or more antennas for wireless data transfer to communicate via RF, a cellular network (such as through a network such as a 5G, LTE, 4G, etc. network,) a wireless provider network, and / or a mobile operator network, such as GSM, for example without limitation, to send and receive Short Message Service (SMS) messages or Unstructured Supplementary Service Data (USSD) messages. The glass-break modeling system 200 may include one or more wireless communications interfaces, e.g., coupled to the one or more antennas, capable of communicating through the means disclosed herein. In various embodiments, the glass-break modeling system 200 may include one or more medium-specific controllers, each corresponding to a particular medium and configured to create electronic composites to transmit to glass-break detection devices 260. In some embodiments, the gateway of the glass-break modeling system 200 may include or correspond to API gateway configured to provide a service for provisioning and monitoring APIs (e.g., REST, HTTP, WebSocket, and / or the like APIs).

[0058] The glass-break modeling system 200 may include one or more ingestion engines 236, one or more modeling engines 238, and / or one or more configuration engines 240, which may be separate or integrated in various embodiments. In various embodiments, the one or more engines may correspond to executable code stored in one or more memories communicatively coupled with one or more processors. In some embodiments, the one or more engines may correspond to one or more computers of the glass-break modeling system 200 configured to perform one or more of the features in accordance with embodiments disclosed herein.

[0059] The configuration engine 240 may facilitate configuring devices as glass-break detection devices 260. In some embodiments, the device may be smart home devices disclosed herein and / or the like. The configuration engine 240 may configure and communicate firmware, applications, and / or code for other software to such devices, and may initiate installation thereof, to provide instructions and data necessary for the proper functioning of the devices as glass-break detection device 260.

[0060] The modeling engine 238 may be configured to perform the matching, modeling, learning, reasoning, and / or the like analyses features disclosed herein, including generating, training, and otherwise developing one or more models 250, which may include one or more neural -network models. In various embodiments, the modeling engine 238 may be an analysis and matching engine that employs machine learning to process the input 202 and / or data based at leastin part thereon and received from the ingestion engine 236 and develop one or more of anechoic glass-break sounds 212, sound component datasets 214, glass specifications 216, room reverberation data 218, noise data 220, augmented audio data 222, models 250, and / or the like. The modeling engine 238 may include or otherwise correspond to a learning engine that may employ an ongoing learning mode to create, update, confirm, correct, and / or refine one or more of anechoic glass-break sounds 212, sound component datasets 214, glass specifications 216, room reverberation data 218, noise data 220, augmented audio data 222, models 250, and / or the like. The modeling engine 238 may include a pattern-based reasoner may be employed to use various statistical techniques in analyzing the data, both current and historical, in order to infer particularized pattern data from the data 202. A transitive reasoner may be employed to infer relationships from a set of relationships to form the performance data. In various embodiments, the system automatically establishes and develops the particularized pattern data. In some embodiments, the modeling engine 238 may be configured to employ deep learning to process the data and derive the particularized pattern data corresponding to one or more of anechoic glassbreak sounds 212, sound component datasets 214, glass specifications 216, room reverberation data 218, noise data 220, augmented audio data 222, models 250, and / or the like.

[0061] The ingestion engine 236 may be configured to ingest glass-break audio data 202. The ingestion engine 236 may also be configured to ingest glass specifications 216 (e.g., dimension, thickness, and type specifications). The system 200 may identify glass specifications 216 for particular glass-break audio data 202 and use the glass specifications 216 to differentiate and segregate the glass-break audio data 202, sound component datasets 214, and augmented audio data 222. Accordingly, the glass-break audio data 202, sound component datasets 214, and augmented audio data 222 may be generated, stored, and / or used as a function of particular glass specifications 216.

[0062] The ingestion engine 236 may also be configured for room reverberation capture or simulation. This may correspond to room impulse response (RIR) generation to specify how glass sounds may behave in specific types of rooms with various types of reflections, scattering, and absorptions and at different distances. In some embodiments, the system 200 may receive collected, preprocessed room reverberation 206 that has been simulated or captured. In some embodiments, the modeling engine 238 may perform the simulation using acoustic modeling techniques to collect room reverberations corresponding to room impulse responses. In some embodiments, the ingestion engine 236 may capture actual room reverb data 206 to develop a collection of room reverberations corresponding to room impulse responses. The system 200 maystore the room reverberation data 218 for subsequent addition to the various glass-break sound data disclosed herein.

[0063] The ingestion engine 236 may also be configured for noise capture or simulation. In some embodiments, the system 200 may receive preprocessed noise 208 that has been simulated or captured. In some embodiments, the modeling engine 238 may perform the simulation using noise generation techniques. In some embodiments, the ingestion engine 236 may capture actual room noise 208. The system 200 may store the noise data 220 for subsequent addition to the various glass-break sound data disclosed herein.

[0064] Various embodiments may use real glass-break recordings as glass-break audio data 202. The recordings of real-world glass breaks may be taken in an anechoic chamber. In some embodiments, the ingestion engine 236 may be configured to monitor for the glass-break audio data 202. The ingestion engine 236 may, for example, be configured to monitor for the glassbreak audio data 202 as it is captured by a communicatively coupled microphone in the anechoic chamber and / or otherwise receive glass-break audio data 202 from one or more other communicatively coupled devices. The ingestion engine 236 may aggregate, process, and consolidate glass-break audio data 202 to facilitate creation, development, and / or use of anechoic glass-break sound datasets 212 and sound component datasets (e.g., thud, shatter, and fall datasets) 214 differentiated by and mapped to glass specifications 216.

[0065] The ingestion engine 236 and / or the modeling engine 238 may be configured to decompose glass-break sound waveforms from the glass-break audio data 202 into sound components. FIG. 3 is an illustration 300 of sound components 310 of a glass-break sound waveform for an example glass-break sound event that may be identified in accordance with embodiments according to the present disclosure. Each of the glass-break sound waveforms from the glass-break audio data 202 may be decomposed into particular sound components 310. A sound profile 305 illustrates the glass-break sound waveform, which may correspond to anechoic glass-break sounds. The example glass break may correspond to a window of a home or other structure being broken by way an object impacting the window.

[0066] A typical glass-break sound waveform may be decomposed into three sound components 310: the initial thud of the impact, the glass shattering, and the fall of the glass shards on the floor and, potentially, falling on and / or hitting other surfaces, such falling on a counter, hitting one or more walls, etc. Thus, as depicted, the sound components 310 may include: component A, which may correspond to an initial portion; component B, which may correspond to an interim portion; and component C, which may correspond to an end portion. The initial portion may correspond tothe initial thud. The interim portion may correspond to the glass shattering. The initial portion may correspond to glass shards falling on the floor and, potentially, falling on and / or hitting other surfaces. Typically, the initial thud may be approximately a few 10 milliseconds, the shatter may be approximately several 100 milliseconds, and the fall of the glass shards on the wall / other surfaces may be up to approximately 1-2 seconds.

[0067] FIG. 4 is a high-level block diagram illustrating a glass-break sound segmentation process 400 for segmenting anechoic glass-break sounds into sound components 310, in accordance with embodiments according to the present disclosure. In various embodiments, the ingestion engine 236 and / or the modeling engine 238 may perform the glass-break sound segmentation process 400. The segmentation process 400 may be based at least in part on executing one or more segmentation modules. As indicated by 405, the ingestion engine 236 and / or the modeling engine 238, having received an anechoic glass-break dataset 202-1 corresponding to a glass-break sound waveform, may segment the glass-break sound into a thud portion, a shatter portion, and a fall portion. As indicated by 410, the ingestion engine 236 and / or the modeling engine 238 may add labels to the thud, shatter, and fall portions. As indicated by 415, the ingestion engine 236 and / or the modeling engine 238 may save corresponding datasets 214, which may include a thud dataset 214-1, a shatter dataset 214-2, and a fall dataset 214-3.

[0068] The ingestion engine 236 and / or the modeling engine 238 may be configured to recombine the sound components 310 based at least in part on augmenting the datasets 214. FIG. 5 is a high-level block diagram illustrating a glass-break sound augmentation process 500 based at least in part on the sound components 310 to generate a much larger dataset for training a glassbreak sound detection model, in accordance with embodiments according to the present disclosure. In various embodiments, the ingestion engine 236 and / or the modeling engine 238 may perform the glass-break sound augmentation process 500. The glass-break sound augmentation process 500 may be based at least in part on executing one or more shuffling modules and / or one or more combiner modules.

[0069] As indicated by 505, the datasets 214 corresponding to the sound components 310 may be recombined in various ordered combinations to derive a much larger, comprehensive dataset corresponding to ordered, glass-break sound combinations. With these techniques, a much richer set of clean glass-break sound may be generated from a small set. The ingestion engine 236 and / or the modeling engine 238 may perform shuffling operations 510 and a combiner 515 configured to perform combining operations. For glass-break sounds from the same glass specifications 216 (e.g., dimension, thickness, and type specifications), the ingestion engine 236 and / or the modelingengine 238 may randomly select and combine the datasets 214 representing the sound components 310 to generate new glass-break sounds. The ingestion engine 236 and / or the modeling engine 238 may take a random linear combination of each of the datasets 214 representing the sound components 310 and then appropriately concatenate them. Alternatively, the ingestion engine 236 and / or the modeling engine 238 may use other approaches such as principal component analysis or wavelet analysis to get the basis components to generate a new dataset.

[0070] FIG. 6 is a more detailed block diagram of the combiner 515, in accordance with embodiments according to the present disclosure. The combiner 515 may receive sound component clips 215 generated per the shuffling operations 510 with respect to the thud dataset 214-1, the shatter dataset 214-2, and the fall dataset 214-3. For example, such clips may include a thud clip 215-1, a shatter clip 215-2, and a fall clip 215-3. The combiner 515 may apply respective time delays 605 to the thud clip 215-1, the shatter clip 215-2, and the fall clip 215-3 to appropriately delay the clips 215 to facilitate applying corresponding window functions 610 to each of the clips 215. The window functions 610 may include a left window, a middle window, and right window.

[0071] FIG. 7 is an illustration of examples of window shapes corresponding to the window functions 610, in accordance with embodiments according to the present disclosure. The window shapes illustrate the application of a left window 610-2, a middle window 610-3, and a right window 610-4 to the clips 215. The window functions 610 may be used to provide a smoothening function to avoid overlap of the clips 215. The window functions 610 may taper edges of the clip signals so, for example, the end of one signal may be tapered off and the beginning of another signal may be faded in, which, in combination may provide for a smooth transition. Such windowing 610 may provide for artifact, edge spike, and / or spectral leakage reduction / mitigation.

[0072] As illustrated in FIG. 6, the combiner 515 may apply respective magnitude variations 615 to the windowed components to increase the variations of the generated clips. As indicated by 620, the combiner 515 may combine the magnitude-varied, windowed components to generate a synthesized glass-break clip 625. Such operations of the combiner 515 may produce many synthesized glass-break clips 625 to develop the comprehensive dataset.

[0073] As indicated by 520 and 525 in FIG. 5, room reverberations and background noises of various types may be incorporated into the comprehensive dataset to create an even larger, more comprehensive environmental and glass-break sounds dataset 222. Such data may be important for configuring the glass-break sound detector to make decisions regarding identifying glass breaks in sounds detected in the short time windows in which the glass-break sound detector mustmake decisions. These techniques may also be used for generating challenging negatives such as a vase falling and breaking, a bunch of keys hitting a hard surface, a metal piece falling on the floor, etc.

[0074] As indicated by 520, the system 200 may run the synthesized glass-break clips 625 through a RIR adder to add room reverberations in order to ensure that the classifier functionality of a glass-break sound detector can detect glass breaks in different kinds of room reverberation environments and at different distances. For example, if the glass-break sound detector is close to the glass-break location, there may be low to no room reverberation that might affect the detection of the glass break by the glass-break sound detector. However, if the glass-break sound detector is relatively far away from the glass-break location (e.g., across a large room or one or more rooms away), the sounds transform into very different sounds (e g., due to room reflections, some frequency portions being diminished, some frequency portions being enhanced, and / or the like). The modeling engine 238 may, for example, apply the room reverberation using any suitable techniques, which may include, for example, finite impulse filter response filters, infinite impulse response filters, algorithmic reverb, convolution, length tailoring, early reflections, and / or the like.

[0075] As indicated by 525, the system 200 may run the reverb -enhanced glass-break clips through a noise adder to add noise in order to ensure that the classifier functionality of a glassbreak sound detector can detect glass breaks in different kinds of noise environments. The modeling engine 238 may apply the noise using any suitable techniques. Such noise may include, for example, environmental noise, white noise, Gaussian noise, impulse noise, and / or the like.

[0076] The environment-specific and glass-break-sound-optimized dataset 222 may be used to train one or more neural -network models 250 to detect glass-break sounds in various kinds of room environments and at various distances. A neural network model 250 may be any suitable type to provide the features disclosed herein. A neural network model 250 may include, for example, interconnected nodes organized into an input layer that may receive audio, one or more hidden layers that may learn patterns and abilities to classify audio as corresponding to a glass break or something else, and an output layer that may output values, labels, descriptors, and / or the like pursuant to the classifications. The training of a neural network model 250 may include feeding the neural network model 250 the augmented audio data 222 and corresponding data, such as known values, labels, descriptors, and / or the like so that the neural network model 250 may adjust its weights and biases to accurately classify known and unknown audio as corresponding to a glass break or something else.

[0077] For example, in some embodiments, the various sound-recognition features may be assigned different weights as part of the pre-trained machine learning model based on the determined relative importance for correctly identifying a glass break. The weightings may be assigned based on a training process that was performed using a training set of data that included the augmented audio data 222 indicative of a glass break and data of other sounds that are not glass breaks. The training process may have involved creating a machine learning model that can classify as many glass breaks and non-glass-breaks as accurately as possible. For instance, a training set of data that includes a large amount of data having a known classification (i.e., glass break, no glass break) may be used by the modeling engine 238, which, as disclosed herein, may be a machine learning engine. After the modeling engine 238 create, trains, and otherwise develops a neural network model 250 that accurately classifies as many of glass breaks and nonglass-breaks as possible, the neural network model 250 may be installed on a processing module of a glass-break detection device 260 such that the pre-trained machine learning model can be used on a large number of glass-break detection devices 260 being manufactured.

[0078] In some embodiments,

[0001] separate neural network models 250 may be used depending on the type of location where the glass-break detection device 260 is to be placed. For instance, different neural network models 250, that are trained separately and use different weightings, and / or biases may be used based on the type of installation location with respect to a window, such as within a certain distance, on a wall or ceiling, etc.

[0079]

[0002] In some embodiments, the neural network model 250 may be dynamic in that it can learn about situations that involved and glass breaks and non-glass-breaks after being installed in the environment in which the glass-break detection device 260 functions. For instance, if the neural network model 250 classifies a glass break with a low confidence, it may seek feedback (e.g., via application notifications, user-selectable interface options, push notifications, interactive display messages, voice messages and voice recognition, and / or the like) on whether its identification of the glass break was correct. Depending on the feedback, the neural network model 250 may be adjusted to be more accurate for future glass-break detections.

[0080] FIG. 8 is a block diagram of a sound-detection module 800, in accordance with embodiments according to the present disclosure, The sound-detection module 800 may be installed on the glass-break detection device 260. In some embodiments, the sound-detection module 800 may be configured by the system 200. Accordingly, the neural network model 250 may be on-device, as opposed to in the cloud in some embodiments. With the neural network model 250 installed locally, the glass-break detection device 260 may monitor a constant stream ofaudio 830, run it through the neural network model 250, and determine if and when there was a glass break. This may solve the technical problems of Internet connectivity being weak, interrupted, or non-existent, as well as the technical problems of having to compress and transmit sound data, latency, and privacy concerns. Even when there is no Internet connection, the glassbreak detection device 260 may still be configured to report glass-break events via cellular network.

[0081] The sound-detection module 800 may include an energy estimator 805, a feature extractor 810, a neural network detector 815, and decision logic 820 configured with thresholds 825. The neural network detector 815 may include the neural network model 250. Accordingly, the neural network model 250 may be used in conjunction with the energy estimator 805 to determine if glass-break sound is detected from audio input 830 in a given environment, such as a home environment. The combination of the energy estimator 805 with the neural network features may allow for filtering out sounds at lower sound-pressure-levels. This may solve the technical problem of sounds (e.g., glass-break sounds) coming from sound events (e.g., glass-break events) in situations where the sound event originates from a location that is too great a distance from the location of the glass-break sound detection device 260 for accurate differentiation of glass-break sounds from non-glass-break sounds. When the distances involved are too great (e.g., the glassbreak sound detection device 260 is too far away, in another room, around one or more corners, etc.), the sounds may significantly change such that the sounds are unrecognizable with a sufficient degree of confidence. Therefore, in some embodiments, the glass-break sound detection device 260 may only certify detection within a certain distance.

[0082] As depicted, the output from both the neural network detector 815 and the energy estimator 805 may go to the decision logic 820. The decision logic 820 may use the two inputs to determine if a glass-break sound is detected. Based on the sound-pressure level signals from the energy estimator 805, the decision logic 820 may filter out sounds at lower sound-pressure levels.

[0083] FIG. 9 illustrates an example method 900 for the sound-detection module 800, in accordance with embodiments according to the present disclosure. According to certain embodiments, the method 900 may begin as indicated by block 905. Teachings of the present disclosure may be implemented in a variety of configurations. As such, the order of the steps comprising the method 900 may be shuffled in any suitable manner and may depend on the implementation chosen. Moreover, while the following steps may be separated for the sake of description, it should be understood that certain steps may be performed simultaneously or substantially simultaneously.

[0084] As indicated by 905, the sound-detection module 800 may read the audio input 830. As indicated by 910, the sound-detection module 800 may extract features for the neural network detector 815. In various embodiments, this may include windowing the audio data, decomposing the audio data into sound components, fdtering out noise, and / or the like processing of the audio data in preparation for the neural network detector 815. As indicated by 915, the neural network detector 815 may make a prediction as to whether the audio data corresponds to a glass-break event or non-glass-break event. As indicated by 920, the energy estimator 805 may estimate the sound-pressure level of the audio input 830.

[0085] The decision logic 820 may determine if glass-break sound is detected is based on two prescribed thresholds 825, Tp and Te. As indicated by 925, the decision logic 820 may compare the prediction of the neural network detector 815 to the threshold Tp. The threshold TPmay be set based on precision-recall curves of the glass-break evaluation dataset 222. The precision-recall curves may define false positives and true positives. If the decision logic 820 determines that the prediction of the neural network detector 815 fails to satisfy the threshold TP, a false determination indication may be returned, as indicated by 930. Accordingly, the possibility of a glass-break event may be eliminated.

[0086] If the decision logic 820 determines that the prediction of the neural network detector 815 does satisfy the threshold TP, the decision logic 820 may compare the estimated sound-pressure level of the audio input 830 to the threshold Te, as indicated by 935. The threshold Te may be set based on the minimum sound-pressure level needed to detect a glass-break. The minimum soundpressure level may be defined for particular glass specifications. If the decision logic 820 determines that the estimated sound-pressure level of the audio input 830 fails to satisfy the threshold Te, a false determination indication may be returned, as indicated by 940. Accordingly, the possibility of a glass-break event may be eliminated. However, if the decision logic 820 determines that the estimated sound-pressure level of the audio input 830 does satisfy the threshold Te, a true determination indication may be returned, as indicated by 945.

[0087] Consequently, the glass-break sound detection device 260 may perform one or a combination of different glass-break response operations. The glass-break sound detection device 260 may generate an alert about a glass-break detection. In various embodiments, the glass-break sound detection device 260 may display such an alert on a display device of the glass-break sound detection device 260 and / or a communicatively coupled device. In various embodiments, the glass-break sound detection device 260 may send alert using one or more accounts, contacts, devices, applications, and / or the like that have been previously linked with the glass-breakdetection feature of the smart-home glass-break sound detection device 260. In some embodiments, the glass-break sound detection device 260 may alert the server system 164, which may then communicate the alert to one or more previously linked accounts, contacts, devices, applications, and / or the like.

[0088] In various embodiments, the glass-break sound detection device 260 may: communicate the alert to another device and / or account; prepare and transmit, store, and / or otherwise provide access to an audio recording of the glass break; communicate and / or store a date and time stamp corresponding to the glass break detection, a location of the glass break, a descriptor of the glassbreak detection device 260; send application notifications, push notifications, and / or audio alerts; cause one or more smart home device actions performed by the glass-break sound detection device 260 and / or a communicatively coupled smart home device, such as triggering an alarm, turning on lights; and / or the like. For example, in some embodiments, the glass-break sound detection device 260 may use auditory communication to communicate with a person that is sound-detected as present in the home. An initial auditory communication by the glass-break sound detection device 260 may be output in the form of synthesized or recorded speech with any suitable message indicating the detected glass-break event. With an alert in any form, the communication may request feedback / confirmation as to whether a glass-break actually occurred, a location of the break with respect to the glass-break sound detection device 260, and / or the like. The feedback / confirmation may be used by the neural network model 250 as part of its ongoing learning mode to improve the accuracy of its classifications.

[0089] In some embodiments, the sound-detection module 800 may dynamically adjust the thresholds 825 to adapt to circumstances. For example, in some embodiments, when the energy estimator 805 detects a high sound-pressure level (e.g., corresponding to a loud sound that exceeds one or more sound-pressure level thresholds of the energy estimator 805), the sound-detection module 800 may adjust the threshold TPof the neural network detector 815 (e.g., downward). In some embodiments, the threshold Tp may adjusted by an amount proportionate to the one or more sound-pressure level thresholds exceeded. Similarly, in some embodiments, when the neural network detector 815 predicts a glass break with a high level of certainty (e.g., that exceeds one or more certainty level thresholds of the neural network detector 815), the sound-detection module 800 may adjust the threshold Te of the energy estimator 805 (e.g., downward). In some embodiments, the threshold Te may adjusted by an amount proportionate to the one or more certainty level thresholds exceeded.

[0090] A computer system as illustrated in FIG. 10 may be incorporated as part of the previously described computerized devices, such as a glass-break detection device 260 and / or the system 200. FIG. 10 provides a schematic illustration of one embodiment of a computer system 1000 that can perform various steps of the methods provided by various embodiments. It should be noted that FIG. 10 is meant only to provide a generalized illustration of various components, any or all of which may be utilized as appropriate. FIG. 10, therefore, broadly illustrates how individual system elements may be implemented in a relatively separated or relatively more integrated manner.

[0091] The computer system 1000 is shown comprising hardware elements that can be electrically coupled via a bus 1005 (or may otherwise be in communication, as appropriate). The hardware elements may include one or more processors 1010, including without limitation one or more general -purpose processors and / or one or more special-purpose processors (such as digital signal processing chips, graphics acceleration processors, video decoders, and / or the like); one or more input devices 1015, which can include one or more microphones 1016, as well as, without limitation a mouse, a keyboard, remote control, and / or the like; and one or more output devices 1020, which can include without limitation a display device, a printer, and / or the like.

[0092] The computer system 1000 may further include (and / or be in communication with) one or more non-transitory storage devices 1025, which can comprise, without limitation, local and / or network accessible storage, and / or can include, without limitation, a disk drive, a drive array, an optical storage device, a solid-state storage device, such as a random access memory (“RAM”), and / or a read-only memory (“ROM”), which can be programmable, flash-updateable and / or the like. Such storage devices may be configured to implement any appropriate data storages, including without limitation, various file systems, database structures, and / or the like.

[0093] The computer system 1000 might also include a communications subsystem 1030, which can include without limitation a modem, a network card (wireless or wired), an infrared communication device, a wireless communication device, and / or a chipset (such as a Bluetooth™ device, an 802.11 device, a Wi-Fi device, a WiMAX device, cellular communication device, etc.), and / or the like. The communications subsystem 1030 may permit data to be exchanged with a network (such as the network described below, to name one example), other computer systems, and / or any other devices described herein. In many embodiments, the computer system 1000 will further comprise a working memory 1035, which can include a RAM or ROM device, as described above.

[0094] The computer system 1000 also can comprise software elements, shown as being currently located within the working memory 1035, including an operating system 1040, device drivers, executable libraries, and / or other code, such as one or more application programs 1045, which may comprise computer programs provided by various embodiments, and / or may be designed to implement methods, and / or configure systems, provided by other embodiments, as described herein. Merely by way of example, one or more procedures described with respect to the method(s) discussed above might be implemented as code and / or instructions executable by a computer (and / or a processor within a computer); in an aspect, then, such code and / or instructions can be used to configure and / or adapt a general purpose computer (or other device) to perform one or more operations in accordance with the described methods. For example, the working memory may include, at least in part, code for the sound-detection module 800, neural network models 250, and / or the like.

[0095] A set of these instructions and / or code might be stored on a non-transitory computer- readable storage medium, such as the non-transitory storage device(s) 1025 described above. In some cases, the storage medium might be incorporated within a computer system, such as computer system 1000. In other embodiments, the storage medium might be separate from a computer system (e.g., a removable medium, such as a compact disc), and / or provided in an installation package, such that the storage medium can be used to program, configure, and / or adapt a general purpose computer with the instructions / code stored thereon. These instructions might take the form of executable code, which is executable by the computer system 1000 and / or might take the form of source and / or installable code, which, upon compilation and / or installation on the computer system 1000 (e.g., using any of a variety of generally available compilers, installation programs, compression / decompression utilities, etc.), then takes the form of executable code.

[0096] As mentioned above, in one aspect, some embodiments may employ a computer system (such as the computer system 1000) to perform methods in accordance with various embodiments of the invention. According to a set of embodiments, some or all of the procedures of such methods are performed by the computer system 1000 in response to processor 1010 executing one or more sequences of one or more instructions (which might be incorporated into the operating system 1040 and / or other code, such as an application program 1045) contained in the working memory 1035. Such instructions may be read into the working memory 1035 from another computer-readable medium, such as one or more of the non-transitory storage device(s) 1025. Merely by way of example, execution of the sequences of instructions contained in the working memory 1035 might cause the processor(s) 1010 to perform one or more procedures of the methods described herein.

[0097] The terms “machine-readable medium,” “machine-readable media,” “computer-readable storage medium,” “computer-readable storage media,” “computer-readable medium,” “computer- readable media,” “processor-readable medium,” “processor-readable media,” and / or like terms as used herein, refer to any medium that participates in providing data that causes a machine to operate in a specific fashion. These mediums may be non-transitory. In an embodiment implemented using the computer system 1000, various computer-readable media might be involved in providing instruct ons / code to processor(s) 1010 for execution and / or might be used to store and / or carry such instructions / code. In many implementations, a computer-readable medium is a physical and / or tangible storage medium. Such a medium may take the form of a non-volatile media or volatile media. Non-volatile media include, for example, optical and / or magnetic disks, such as the non-transitory storage device(s) 1025. Volatile media include, without limitation, dynamic memory, such as the working memory 1035.

[0098] Common forms of physical and / or tangible computer-readable media include, for example, a floppy disk, a flexible disk, hard disk, magnetic tape, or any other magnetic medium, a CD-ROM, any other optical medium, any other physical medium with patterns of marks, a RAM, a PROM, EPROM, a FLASH-EPROM, any other memory chip or cartridge, or any other medium from which a computer can read instructions and / or code.

[0099] Various forms of computer-readable media may be involved in carrying one or more sequences of one or more instructions to the processor(s) 1010 for execution. Merely by way of example, the instructions may initially be carried on a magnetic disk and / or optical disc of a remote computer. A remote computer might load the instructions into its dynamic memory and send the instructions as signals over a transmission medium to be received and / or executed by the computer system 1000.

[0100] The communications subsystem 1030 (and / or components thereof) generally will receive signals, and the bus 1005 then might carry the signals (and / or the data, instructions, etc. carried by the signals) to the working memory 1035, from which the processor(s) 1010 retrieves and executes the instructions. The instructions received by the working memory 1035 may optionally be stored on a non-transitory storage device 1025 either before or after execution by the processor(s) 1010.

[0101] It should further be understood that the components of computer system 1000 can be distributed across a network. For example, some processing may be performed in one location using a first processor while other processing may be performed by another processor remote from the first processor. Other components of computer system 1000 may be similarly distributed. As such, computer system 1000 may be interpreted as a distributed computing system that performsprocessing in multiple locations. In some instances, computer system 1000 may be interpreted as a single computing device, such as a distinct laptop, desktop computer, or the like, depending on the context.

[0102] The methods, systems, and devices discussed above are examples. Various configurations may omit, substitute, or add various procedures or components as appropriate. For instance, in alternative configurations, the methods may be performed in an order different from that described, and / or various stages may be added, omitted, and / or combined. Also, features described with respect to certain configurations may be combined in various other configurations. Different aspects and elements of the configurations may be combined in a similar manner. Also, technology evolves and, thus, many of the elements are examples and do not limit the scope of the disclosure or claims.

[0103] Specific details are given in the description to provide a thorough understanding of example configurations (including implementations). However, configurations may be practiced without these specific details. For example, well-known circuits, processes, algorithms, structures, and techniques have been shown without unnecessary detail in order to avoid obscuring the configurations. This description provides example configurations only, and does not limit the scope, applicability, or configurations of the claims. Rather, the preceding description of the configurations will provide those skilled in the art with an enabling description for implementing described techniques. Various changes may be made in the function and arrangement of elements without departing from the spirit or scope of the disclosure.

[0104] Also, configurations may be described as a process which is depicted as a flow diagram or block diagram. Although each may describe the operations as a sequential process, many of the operations can be performed in parallel or concurrently. In addition, the order of the operations may be rearranged. A process may have additional steps not included in the figure. Furthermore, examples of the methods may be implemented by hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware, or microcode, the program code or code segments to perform the necessary tasks may be stored in a non-transitory computer-readable medium such as a storage medium. Processors may perform the described tasks.

[0105] Having described several example configurations, various modifications, alternative constructions, and equivalents may be used without departing from the spirit of the disclosure. For example, the above elements may be components of a larger system, wherein other rules may takeprecedence over or otherwise modify the application of the invention. Also, a number of steps may be undertaken before, during, or after the above elements are considered.

[0106] Furthermore, the example embodiments described herein may be implemented as logical operations in a computing device in a networked computing system environment. The logical operations may be implemented as: (i) a sequence of computer implemented instructions, steps, or program modules running on a computing device; and (ii) interconnected logic or hardware modules running within a computing device.

[0107] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

[0108] Also, the terms in the claims have their plain, ordinary meaning unless otherwise explicitly and clearly defined by the patentee. The indefinite articles “a” or “an,” as used in the claims, are defined herein to mean one or more than one of the element that the particular article introduces; and subsequent use of the definite article “the” is not intended to negate that meaning. Furthermore, the use of ordinal number terms, such as “first,” “second,” etc., to clarify different elements in the claims is not intended to impart a particular position in a series, or any other sequential character or order, to the elements to which the ordinal number terms have been applied.

Claims

WHAT IS CLAIMED:

1. A method for developing a glass-break sound detector, the method comprising: processing first audio data corresponding to glass-break sounds; identifying from the first audio data a glass-break sound waveform; decomposing the glass-break sound waveform into sound components; recombining the sound components in various combinations to derive a glass-break sound dataset; developing a glass-break sound dataset based at least in part on incorporating room reverberations and / or noise into the glass-break sound dataset; and training a model to facilitate detection of glass-break sounds using the developed glass-break sound dataset; wherein the model is configured to receive second audio data corresponding to a sound event and predict whether the sound event corresponds to a glass break.

2. The method for developing a glass-break sound detector as recited in claim 1, wherein the glass-break sounds are anechoic glass-break sounds.

3. The method for developing a glass-break sound detector as recited in claim 1, further comprising capturing the glass-break sounds in an anechoic chamber.

4. The method for developing a glass-break sound detector as recited in claim 1, wherein the sound components comprise an initial portion, an interim portion, and an end portion.

5. The method for developing a glass-break sound detector as recited in claim 1, further comprises applying windowing functions to each of the initial portion, the interim portion, and the end portion prior to the recombining.

6. The method for developing a glass-break sound detector as recited in claim 1, wherein the model is a machine learning model.

7. The method for developing a glass-break sound detector as recited in claim 6, wherein the model is a neural network model.

8. A method for detecting a glass-break sound, the method comprising:monitoring sounds in a home or other structure that comprises one or more windows; feeding, an audio stream corresponding to the sounds to a detector, where: the detector is configured with a model that is trained to facilitate detection of glass-break sounds using a glass-break sound dataset, and the glass-break sound dataset is developed based at least in part on decomposing a glass-break sound waveform into sound components and recombining the sound components in various combinations to derive the glassbreak sound dataset; detecting, by the detector, a sound event in the audio stream; determining, by the detector using the model, a prediction of whether the sound event corresponds to a glass break; and based at least in part on the prediction, determining that the sound event corresponds to a glass break.

9. The method for detecting a glass-break sound as recited in claim 8, further comprising comparing the prediction to a threshold, where the generating the glass-break alert is based at least in part on the prediction satisfying the threshold.

10. The method for detecting a glass-break sound as recited in claim 8, wherein the model is a machine learning model.

11. The method for detecting a glass-break sound as recited in claim 9, wherein the model is a neural network model.

12. The method for detecting a glass-break sound as recited in claim 8, further comprising: estimating a sound-pressure level of the sound event; where the determining that the sound event corresponds to a glass break is based at least in part on the sound-pressure level of the sound event.

13. The method for detecting a glass-break sound as recited in claim 12, wherein the estimating the sound-pressure level of the sound event is performed by an energy detector of a glass-break detection device.

14. The method for detecting a glass-break sound as recited in claim 8, further comprising: comparing the sound-pressure level of the sound event to a threshold, where the determining that the sound event corresponds to a glass break is based at least in part on the sound-pressure level of the sound event satisfying the threshold.

15. The method for detecting a glass-break sound as recited in claim 8, further comprising: decomposing audio data corresponding to the sound event into a set of sound components; where the prediction is based at least in part on the set of sound components.

16. The method for detecting a glass-break sound as recited in claim 8, further comprising generating a glass-break alert consequent to the determining that the sound event corresponds to a glass break.

17. A system comprising one or more processing devices and memory communicatively coupled with and readable by the one or more processing devices, where the system is configured to perform the method for developing a glass-break sound detector as recited in claim 1.

18. One or more non-transitory, machine-readable media having machine- readable instructions thereon which, when executed by one or more processing devices, cause the one or more processing devices to perform the method for developing a glass-break sound detector as recited in claim 1.

19. A glass-break detection device comprising one or more processing devices and memory communicatively coupled with and readable by the one or more processing devices, where the glass-break detection device comprises the detector and is configured to perform the method for detecting a glass-break sound as recited in claim 8.

20. One or more non-transitory, machine-readable media having machine- readable instructions thereon which, when executed by one or more processing devices, cause the one or more processing devices to perform the method for detecting a glass-break sound as recited in claim 8.

Images