VEHICLE SYSTEM FOR DETECTING ACOUSTIC FREQUENCIES

Abstract

A vehicle system includes a user device configured to generate acoustic sounds at different frequencies. The vehicle system also includes a vehicle microphone for collecting acoustic data from the user device. Additionally, the vehicle system includes a vehicle processor for storing vehicle data, including the acoustic data, and a server that is communicatively linked to the vehicle processor. According to some examples, the server is configured to determine whether the acoustic data matches one or more predefined sound frequencies.

Description

INTRODUCTION

[0001] The information provided in this section is intended to provide a general context for the disclosure. The work of the presently named inventors, to the extent described in this section, as well as those aspects of the description that do not otherwise qualify as prior art at the time of filing, are neither expressly nor implicitly acknowledged as prior art to the present disclosure.

[0002] The present disclosure generally relates to a vehicle system for detecting acoustic frequencies.

[0003] AMBER alerts are activated in the most serious cases of child abduction. The goal of an AMBER alert is to immediately mobilize the public to assist in the search for and safe rescue of a missing child. These alerts are broadcast via radio, television, traffic signals, cellphones, and other data-enabled devices. Some types of alerts, such as cellphone alerts, have a specific audible tone accompanying the alert. Additionally, the alert may contain information about the abducted child and / or a potential suspect. Furthermore, the alert may provide information about a vehicle used in the abduction of the child, including the make, model, and license plate number of the vehicle.

[0004] Vehicle telematics data for a particular vehicle may already capture one or more aspects of the preceding data, including the make, model, and / or license plate numbers of nearby vehicles. While such data is typically collected for use by the vehicle in controlling its operation and / or assisting a driver, such telematics data could provide information for use by authorities in locating an abducted child. SUMMARY

[0005] According to one configuration, a vehicle system includes a user device configured to generate acoustic sounds at different frequencies and a vehicle microphone for collecting acoustic data from the user device. The vehicle system also includes a vehicle processor for storing vehicle data including the acoustic data and a server communicatively coupled to the vehicle processor. According to some examples, the server is configured to determine whether the acoustic data matches one or more predetermined sound frequencies. The server may also be configured to digitally sample the collected acoustic data. According to some examples, the server is configured to extract the collected acoustic data into succinct features including one or more of the mel-cepstrum coefficients and the delta energy coefficients.

[0006] The server may also be configured to send a notification to a user if the acoustic data matches one or more predetermined sound frequencies. According to some examples, the server is configured to send a notification to law enforcement if the acoustic data matches one or more predetermined sound frequencies. Additionally or alternatively, the server may be configured to connect to the user device via Bluetooth® if the acoustic data collected by the vehicle microphone matches a predetermined sound frequency. A vehicle may include the vehicle system.

[0007] According to another configuration, a vehicle system includes a user device configured to generate acoustic sounds at different frequencies. The vehicle system also includes a vehicle microphone for collecting acoustic data from the user device and a vehicle camera for collecting information related to the vehicle's surroundings. Additionally, the vehicle system includes a vehicle processor for storing vehicle data, including the acoustic data and the vehicle location. A server may be communicatively coupled to the vehicle processor.

[0008] The server may be configured to determine whether the acoustic data matches a predetermined sound frequency and extract information related to the determined sound frequency. According to some examples, the server is further configured to collect data related to the vehicle's surroundings using the vehicle camera and analyze the vehicle data and the collected data to determine whether any data matches the extracted information. Additionally, the server may be configured to send a notification if the collected data matches the extracted information.

[0009] According to some examples, the server is configured to digitally sample the collected acoustic data. Additionally or alternatively, the server may be configured to extract the collected acoustic data into succinct features including one or more of the mel-cepstrum coefficients and the delta energy coefficients. Further, the server may use tokenization to analyze the collected data. According to some examples, the server is configured to send a notification to a user if the collected data matches the extracted information. According to some examples, the server is configured to send a notification to law enforcement if the collected data matches the extracted information. A vehicle may include the vehicle system.

[0010] According to yet another configuration, a vehicle system includes a user device configured to generate acoustic sounds at different frequencies and a vehicle microphone for collecting acoustic data from the user device. Additionally, the vehicle system includes a vehicle processor for storing vehicle data, including the acoustic data and vehicle event data. A server is communicatively coupled to the vehicle processor. According to some examples, the server is configured to determine whether the acoustic data collected by the vehicle microphone matches a predetermined sound frequency and to alert the vehicle occupants if the acoustic data matches the predetermined sound frequency. The vehicle event data may be used by the server to determine whether a vehicle event has occurred.The vehicle event may be one or more of a vehicle crash or a vehicle leaving the roadway. According to some examples, the server is configured to determine whether an automatic vehicle response has been implemented. The server may be configured to cancel an alert for the vehicle occupants if an automatic vehicle response has not been implemented. A vehicle may include the vehicle system. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The drawings described herein are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Fig. 1 is a perspective view of a vehicle exterior including a vehicle system according to the present disclosure; Fig. 2 a perspective view of an interior of the vehicle according to Fig. 1, including the vehicle system according to the present disclosure; Fig. 3 is an exemplary functional block diagram according to one aspect of the present disclosure; Fig. 4 is an exemplary functional block diagram according to another aspect of the present disclosure; Fig. 5 illustrates an exemplary flowchart according to another aspect of the present disclosure; Fig. 6 is an exemplary flowchart according to another aspect of the present disclosure; and Fig. 7 illustrates an exemplary flowchart according to another aspect of the present disclosure.

[0012] Corresponding reference symbols indicate corresponding parts throughout the drawings. DETAILED DESCRIPTION

[0013] Example configurations will now be described more fully with reference to the accompanying drawings. Example configurations are provided so that this disclosure will be thorough and will fully convey the scope of the disclosure to those of ordinary skill in the art. Specific details, such as examples of specific components, devices, and methods, are set forth in order to provide a thorough understanding of the configurations of the present disclosure. Those of ordinary skill in the art will appreciate that specific details need not be employed, that the example embodiments of the configurations may be embodied in many different forms, and that the specific details and example configurations should not be construed to limit the scope of the disclosure.

[0014] The terminology used herein is for the purpose of describing specific example configurations only and is not intended to be limiting. As used herein, the singular articles "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. The terms "comprises," "comprising," "including," and "having" are inclusive and therefore specify the presence of features, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, one or more other steps, one or more other operations, one or more other elements, one or more other components, and / or groups thereof.The method steps, processes, and operations described herein should not be construed as necessarily requiring their performance in the particular order discussed or illustrated, unless specifically identified as such. Additional or alternative steps may be used.

[0015] When an element or layer is described as being "on," "engaging with," "connected to," "attached to," or "coupled to" another element or layer, it may be directly on, engaging with, connected to, attached to, or coupled to the other element or layer, or there may be intervening elements or layers. Conversely, when an element is described as being "directly on," "directly engaging with," "directly connected to," "directly attached to," or "directly coupled to" another element or layer, there may be no intervening elements or layers. Other words used to describe the relationship between elements should be interpreted in a similar manner (e.g., "between" as opposed to "directly between," "adjacent" as opposed to "directly adjacent," etc.).The term “and / or” as used herein includes any combination of one or more of the associated listed elements.

[0016] The terms "first", "second", "third", etc., may be used herein to describe various elements, components, regions, layers, and / or sections. These elements, components, regions, layers, and / or sections should not be limited by these terms. These terms may only be used to distinguish one element, component, region, layer, or section from another region, layer, or section. Terms such as "first", "second", and other numerical terms do not imply a sequence or order unless clearly indicated by the context. Consequently, a first element, component, region, layer, or section coulddiscussed below may be referred to as a second element, component, region, layer, or section without departing from the teachings of the example configurations.

[0017] In this application, including the definitions below, the term "module" may be replaced with the term "circuit." The term "module" may refer to, be part of, or include an application-specific integrated circuit (ASIC); a digital, analog, or mixed analog / digital discrete circuit; a digital, analog, or mixed analog / digital integrated circuit; a combinational logic circuit; a field-programmable gate array (FPGA); a processor (shared, dedicated, or group) that executes code; a memory (shared, dedicated, or group) that stores code executed by a processor; other suitable hardware components that provide the described functionality; or a combination of some or all of the above, such as in a system on a chip.

[0018] The term "code" as used above may include software, firmware, and / or microcode, and may refer to programs, routines, functions, classes, and / or objects. The term "shared processor" includes a single processor that executes some or all of the code from multiple modules. The term "group processor" includes a processor that, in combination with additional processors, executes some or all of the code from one or more modules. The term "shared memory" includes a single memory that stores some or all of the code from multiple modules. The term "group memory" includes memory that, in combination with additional memory, stores some or all of the code from one or more modules. The term "memory" may be a subset of the term "computer-readable medium."The term "computer-readable medium" does not encompass the transitory electrical and electromagnetic signals that propagate through a medium and can therefore be considered a tangible and non-transitory storage. Non-limiting examples of non-transitory storage include a tangible computer-readable medium, including non-volatile memory, magnetic memory, and optical memory.

[0019] The devices and methods described in this application may be implemented partially or entirely by one or more computer programs executed by one or more processors. The computer programs include processor-executable instructions stored on at least one non-transitory tangible computer-readable medium. The computer programs may also include and / or rely on stored data.

[0020] A software application (i.e., a software resource) may refer to computer software that causes a computing device to perform a task. According to some examples, a software application may be referred to as an "application," an "app," or a "program." Example applications include, but are not limited to, system diagnostic applications, system management applications, system maintenance applications, word processing applications, spreadsheet applications, messaging applications, media streaming applications, social networking applications, and gaming applications.

[0021] Non-transitory memory may include physical devices used to temporarily or permanently store programs (e.g., sequences of instructions) or data (e.g., program state information) for use by a computing device. The non-transitory memory may be volatile and / or non-volatile addressable semiconductor memory. Examples of non-volatile memory include, but are not limited to, flash memory and read-only memory (ROM) / programmable read-only memory (PROM) / erasable programmable read-only memory (EPROM) / electronically erasable programmable read-only memory (EEPROM) (which, for example, is typically used for firmware, such as boot programs).Examples of volatile memory include, but are not limited to, random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), phase change memory (PCM), and disks or tapes.

[0022] These computer programs (also known as programs, software, software applications, or code) contain machine instructions for a programmable processor and may be implemented in a high-level procedural and / or object-oriented programming language and / or in an assembly / machine language. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, non-transitory computer-readable medium, device, and / or apparatus (e.g., magnetic disks, optical disks, memories, programmable logic devices (PLDs)) used to provide machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal.The term “machine-readable signal” refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0023] Various implementations of the systems and techniques described herein may be embodied in digital electronic and / or optical circuitry, integrated circuitry, specially designed ASICs (Application Specific Integrated Circuits), computer hardware, firmware, software, and / or combinations. These various implementations may include implementation in one or more computer programs executable and / or interpretable on a programmable system including at least one programmable processor, which may be a special-purpose or general-purpose processor, coupled to receive data and instructions from, and send data and instructions to, a memory system, at least one input device, and at least one output device.

[0024] The processes and logic flows described in this specification may be performed by one or more programmable processors, also referred to as data processing hardware, that execute one or more computer programs to perform functions by acting on input data and generating output. The processes and logic flows may also be performed by special-purpose logic circuitry, such as an FPGA (field-programmable gate array) or an ASIC (application-specific integrated circuit). Processors suitable for executing a computer program include, by way of example, both general-purpose and special-purpose microprocessors and any one or more processors of any type of digital computer. Generally, a processor receives instructions and data from read-only memory or random access memory, or both.The essential elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to, one or more mass storage devices for storing data, such as magnetic, magneto-optical, or optical disks, for receiving data from them, sending data to them, or both. However, a computer is not required to include such devices. Computer-readable media suitable for storing instructions and data of a computer program include all forms of non-volatile memory, media, and storage devices, including, by way of example, semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.The processor and memory may be supplemented by or incorporated into special logic circuitry.

[0025] To provide interaction with a user, one or more aspects of the disclosure may be implemented in a computer having a display device, e.g., a CRT (cathode ray tube), an LCD (liquid crystal display monitor), or a touch screen, for displaying information to the user, and optionally a keyboard and a pointing device, e.g., a mouse or trackball, through which the user can provide input to the computer. Other types of devices may also be used to provide interaction with the user; feedback provided to the user may, for example, be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback; while input from the user may be received in any form, including auditory, voice, or tactile input.Additionally, a computer may interact with a user by sending documents to and receiving documents from a device used by the user; e.g., by sending web pages to a web browser on a client user device in response to requests received from the web browser.

[0026] In the Fig. 1-7 is in Fig. 1 illustrates a vehicle 10 with a vehicle system 100. It is contemplated that the vehicle 10 is an electric vehicle (EV) and may include autonomous or semi-autonomous capabilities. Additionally or alternatively, the vehicle 10 may be a hybrid vehicle 10, including the components and capabilities of both an EV and an internal combustion engine (ICE). Additionally, the vehicle 10 may include only an internal combustion engine or other power source, if desired. According to some examples, the vehicle 10 may be a land vehicle, a watercraft, or an aircraft.

[0027] In the Fig. 1 and Fig. 2, the vehicle system 100 is shown in conjunction with a user device 300 configured to generate acoustic sounds at different frequencies. Additionally, the vehicle system 100 includes a vehicle microphone 102 for collecting acoustic data 206, a vehicle camera 104 for collecting video data, a vehicle processor 200, and a server 400 communicatively coupled to the processor 200. The user device 300 includes a display and computing hardware configured to execute commands from the vehicle processor 200 and / or the server 400. According to some examples, the user device 300 is a cellular phone or a tablet; however, it is contemplated that the user device 300 may be any device that includes the display and computing hardware necessary to execute the commands from the vehicle processor 200 and / or the server 400.User device 300 may include processing hardware and software that enables user device 300 to communicate with a cellular network. The cellular network may be a Global System for Mobile Communications (GSM) network, such as 5G, but the cellular network may also be a Code Division Multiple Access (CDMA) network, a Time Division Multiple Access (TDMA) network, or other networks.

[0028] The user device 300 is configured to generate a plurality of audible sounds. According to some examples, the audible sounds notify the user of a call or a text. However, according to some examples, the audible sounds notify the user of other events, such as an AMBER alert or an emergency call. For example, many devices are equipped with vehicle crash detection software so that if the user device 300 detects a vehicle crash, the user device 300 can automatically make an emergency call to emergency services. To notify the user that the emergency call is occurring, the user device 300 plays the audible sound. According to other examples, if an AMBER alert is broadcast by the National Center for Missing and Exploited Children, the user device 300 plays the audible sound. Additionally, according to other examples, a weather warning, such asOne or more of a Federal Emergency Management Agency (FEMA) alert, an earthquake warning, a tornado warning, and / or a tsunami warning may also be sent to the user device 300. Each sound may be unique, so that acoustic detection can identify each alert type.

[0029] Further, with reference to the Fig. 1 and Fig. 2, the vehicle system 100 also includes the vehicle microphone 102 disposed within the interior of the vehicle 10. The vehicle microphone 102 may be any type of microphone configured to transmit and / or record acoustic sounds disposed in and around the vehicle 10. According to some examples, the vehicle microphone 102 is a dynamic microphone, a condenser microphone, and / or a ribbon microphone. However, the vehicle microphone 102 may be any type of microphone if desired. The vehicle interior may also include a vehicle instrument panel disposed within the interior and configured to display words or images to the driver via a display associated with the vehicle instrument panel.

[0030] As in Fig. 1, the vehicle system 100 includes the vehicle camera 104. The vehicle camera 104 may be any type of vehicle camera 104, including, but not limited to, an interior camera, a rear camera, a front camera, a side camera, and / or a perimeter camera. The vehicle camera 104 may be one or more of monocular cameras, binocular cameras, wide-angle cameras, or the like. According to some examples, the vehicle camera 104 includes multiple cameras 104 including the same camera type or multiple different camera types. Regardless of the number, types, and locations of the vehicle cameras 104, the vehicle cameras 104 may be configured to record and / or transmit video data to the vehicle processor 200 and / or the server 400. The video data may include video data of an interior of the vehicle 10 and / or video data of an area exterior of the vehicle 10.

[0031] The vehicle processor 200 stores the vehicle data 202 of the vehicle 10. According to some examples, the vehicle data 202 includes the vehicle location 204, the acoustic data 206, the vehicle camera data 208, and / or the vehicle event data 210. The vehicle location 204 generally refers to a location of the vehicle 10. The vehicle location 204 may be obtained from a Global Positioning System (GPS), another navigation system, a user device 300, and / or a third-party application. Regardless of how the vehicle location 204 is obtained, this information may be transmitted to the vehicle processor 200. Furthermore, the vehicle location data 204 may include route data such that a route traveled by the vehicle 10 may also be transmitted to the vehicle processor 200.

[0032] The acoustic data 206 generally relates to any acoustic sounds detected by the vehicle microphone 102. The acoustic sounds may be generated by the user device 300 or may be any other sound detected by the vehicle microphone 102.

[0033] The vehicle camera data 208 generally refers to any recorded and / or transmitted video from one or more of the vehicle cameras 104. According to some examples, the vehicle camera data 208 may be data including images of license plates of nearby vehicles. Typically, the vehicle camera data 208 is collected by the vehicle cameras 104 and transmitted to the vehicle processor 200. However, it is contemplated that the camera data 208 may additionally and / or alternatively be collected by crowdsourcing camera data 208 from nearby vehicles. The data from nearby vehicles may be obtained by the vehicle processor 200 and / or the server 400 via Bluetooth®, Wi-Fi®, third-party applications, or other similar methods.

[0034] The vehicle event data 210 generally relates to whether a vehicle event, such as a vehicle crash, has occurred. The vehicle event data 210 may include, but is not limited to, vehicle speed, route, or lane information that may reflect that a vehicle event has occurred. For example, the vehicle event data 210 may include data indicating that the vehicle has left the road, which in turn may indicate that the vehicle 10 has been involved in a crash. Such data may be verified by one or more vehicle cameras 104, e.g., if the cameras 104 detect damage to the vehicle 10 and / or contact with another vehicle. The vehicle processor 200 may capture additional data regarding the vehicle that has left the road, such as:whether the vehicle 10 has returned to the road, has been off the road for an extended period of time, or is in a location that may prevent the vehicle 10 from returning to the road (i.e., near or in a body of water).

[0035] In the Fig. 1-7, the server 400 is configured as a network- and / or cloud-based system that communicates with one or both of the vehicle processor 200 and the user device 300. According to some examples, the server 400 is also configured to communicate with third-party processors 500 to collect the third-party data 502. The third-party processors 500 may include, for example, but are not limited to, vehicle processors 200 along the route entered via the user device 300. Additionally or alternatively, the third-party processors 500 may include third-party user devices 300 within the vehicles 10 along the route. The third-party data 502 is generally contemplated to include the vehicle location data 204, the vehicle event data 210, the vehicle camera data 208, and / or the acoustic data 206 from the third-party processors.According to some examples, the vehicle location data 204 is configured to transmit the vehicle location 204 of each respective vehicle 10 traveling along the route to the server 400. The vehicle event data 210 may be configured to transmit the vehicle events, including, but not limited to, the vehicle speed, lane merging, the vehicle's steering wheel angle, or other vehicle operations related to a vehicle crash or vehicle departure from the roadway, to the server 400. According to some examples, the vehicle camera data 208 includes transmitted or recorded images, which may pertain to information received in various alerts that may be transmitted to the server 400.Additionally, according to some examples, the acoustic data 206 from the vehicle microphone 102 is configured to be transmitted to the server 400 to determine whether the acoustic data 206 matches predetermined acoustic sounds.

[0036] It is also contemplated that the vehicle processor 200 may transmit some or all of the vehicle data 202 to the server 400 for further processing and / or evaluation. The vehicle processor 200 and / or the server 400 may be continuously and / or regularly updated so that the vehicle data 202 is updated in real time.

[0037] In the Fig. 3, the server 400 is configured to determine whether the acoustic data 206 matches one or more predetermined sound frequencies. According to some examples, the server 400 may be configured to digitally sample the collected acoustic data 206. Additionally, the acoustic data 206 acquired by the vehicle microphone 102 may then be de-nosed in step 500 to remove engine noise and other background noise. According to some examples, the enhanced acoustic signal is then subjected to feature extraction in step 502. For example, the server 400 may be configured to extract the collected acoustic data 206 into succinct features including one or more of the mel-cepstrum coefficients, the delta energy coefficients, the filter bank energies, the logarithmic filter bank energies, and the spectral subband centroids.Statistical algorithms including one or more of random forests, a deep neural network, and a Bayesian classifier may be used to assist in determining whether the acoustic data matches the one or more predetermined sound frequencies. The features are then input into an optimized model at step 504, where the server 400 determines whether the acoustic features match the predetermined sound of the optimized model.

[0038] The optimized model may be continuously or otherwise updated and improved. For example, in step 506, the dataset is formed, augmented, and / or grouped using various noise considerations, including, but not limited to, season, weather conditions, locations, background noise, reverberation, drone speed, or the like. The dataset is then sent to data preprocessing in step 508, where it is output as a preprocessed dataset in step 509. Additionally, according to some examples, the preprocessed dataset is input to machine learning devices for training and testing the datasets in step 510. According to some examples, the test data is input to model training in step 512, where machine learning (ML) algorithms iterate for the best model at 514 before inputting it to a candidate model in step 516.Additionally, the training data can be input directly as a candidate model in step 514. Once the model is optimized, the optimized model returns to step 504, where it can be used to detect an acoustic event, such as an AMBER alert or an emergency call.

[0039] Additionally, according to some examples, server 400 is configured to consider the data from the alert when using navigation features. For example, if the acoustic data is determined to match a predetermined sound frequency, such as in a tsunami warning, server 400 may suggest a route that avoids the area where the alert is located or suggest detour options if vehicle 10 is currently routed to be near the location of the alert.

[0040] Additionally, according to some examples, server 400 is also configured to send a notification to a user if acoustic data 206 matches one or more predetermined sound frequencies. For example, if server 400 determines that an AMBER alert has been detected, server 400 may be configured to notify the driver of the alert via audio and / or via the vehicle instrument panel.

[0041] If in the Fig. 1-7, if the server 400 determines that the acoustic data 206 matches the predetermined sound frequency, the server 400 may determine that an AMBER alert has been activated. Once the server 400 has determined that an AMBER alert has been activated, the server 400 may extract the information related to the AMBER alert. The extracted information may include information related to the suspect, including, but not limited to, vehicle information such as the make, model, color, and / or license plate number of the subject vehicle. The license plate information may include the state, the license plate number, and the license plate color. Additionally, according to some examples, tokenization may be applied to the extracted information to protect sensitive data.

[0042] As soon as in the Fig. 4, the server may use segmentation at step 600 to resolve the text contained in the alert into actionable sentences. The sentences may then undergo tokenization to protect privacy at step 602. Additionally, once tokenization has occurred, the data may undergo word embedding at step 604 to determine a feature vector. Furthermore, the data may undergo proper noun recognition (NER) at step 606 to assist in extracting entities. Additionally, the extracted entities may be processed into the alert at step 608, which data may be sent elsewhere in the server 400 at step 610 for additional processing, as described in more detail below.

[0043] Once the data from the alarm has been extracted, the server 400 is configured to collect data related to the vehicle's surroundings using the vehicle camera 104. The collected data may include, but is not limited to, the information extracted from the AMBER alarm, such as the vehicle's make, model, color, and / or license plate information described above. Additionally, according to some examples, the server 400 may be configured to analyze the vehicle data 202 and / or the collected data from the vehicle camera data 208 to determine if any data matches the information extracted from the AMBER alarm. According to some examples, the server 400 uses a weighted algorithm over each of these attributes extracted from the alarm to analyze the data. For example, the vehicle color maybe highly weighted, while the vehicle model can be weighted lower to provide a customized analysis for each alarm.

[0044] If server 400 determines that any collected data matches the extracted information, server 400 may be configured to send a notification containing the information to the driver and / or law enforcement. The notification may include one or more of an audio announcement, a visual display on the instrument panel, a notification to user device 300, a phone call, or a text message.

[0045] Additionally, according to some examples, the server 400 may be configured to notify the driver of the alarm via an audible and / or visual indication if the server 400 determines that an emergency call tone has been detected. If the server 400 determines that the acoustic data 206 collected by the vehicle microphone 102 matches a predetermined sound frequency, the server 400 may additionally or alternatively be configured to connect to the user device 300 via Bluetooth® or other wireless methods. According to some examples, the server 400 may send a notification to law enforcement, or the server 400 may be configured to terminate a duplicate emergency call (i.e.,, to terminate a call made by the vehicle 10 if a call has already been made by the user device 300, or to terminate the call made by the user device 300) when the server 400 is connected to the user device 300.

[0046] Additionally, according to some examples, server 400 may determine that an emergency call has been made if server 400 determines that acoustic data 206 matches the predetermined sound frequency and the vehicle data indicates a vehicle event, such as the vehicle leaving the roadway or a vehicle crash. If acoustic data 206 matches the predetermined sound frequency, vehicle event data 210 is further used by server 400 to determine whether a vehicle event has occurred. If a vehicle event, such as a vehicle crash, has occurred, server 400 is configured to determine whether an automatic vehicle response has been implemented. According to some examples, the automatic vehicle response may be an automatic call to emergency services through a vehicle telematics system, such as Guardian® or OnStar®, or other vehicle communication systems.If automatic vehicle response has been implemented, server 400 may additionally be configured, according to some examples, to connect to user device 300 via Bluetooth® and cancel the emergency call. Alternatively, if automatic vehicle response 10 has not been implemented, server 400 may be configured, according to some examples, to cancel the alert, allowing the emergency call from user device 300 to proceed.

[0047] In the Fig. 1-7 are in the Fig. 5-7 exemplary flowcharts of the operations of the vehicle system 100 are set forth. First, in the exemplary flowchart of the operations of the vehicle system 100 shown in Fig. 5, a push notification, such as an AMBER alert from law enforcement, is sent via the cell servers at step 700. The user device 300 receives the notification at step 702 and plays the audible sound related to the alert. The vehicle microphone 102 is configured to transmit the audible data 206 to the server 400 at step 704 for comparison with predetermined audible data 206. If a matching audible sound, such as an AMBER alert, is found, the server 400 is configured to determine the vehicle location 204 at step 708 so that only the vehicles 10 in the vicinity of the alert are activated. Next, in step 710, the vehicle location 204 is input to an artificial intelligence (AI) agent to scan the images of nearby vehicles captured by the vehicle camera 104 to determine if any images match the data from the alarm.Concurrent with step 708, server 400 is configured to extract the raw text data from the AMBER alert in step 712. Next, semantic analysis and entity extraction are performed on the raw text data in step 714, with the extracted entities also being transmitted to the AI ​​agent to scan the images captured by the vehicle cameras 104 of nearby vehicles in step 710 to determine if any images match the data from the alert. Additionally, concurrent with steps 708 and 712, in step 716, the vehicle cameras 104 are activated to transmit and / or record data and transmit that data to server 400, where features are then extracted from the images in step 718.Once the image features have been extracted, the extracted features are transmitted to the AI ​​agent to scan the images captured by the vehicle cameras 104 of nearby vehicles at step 710 to determine if any images match the data from the alert. Additionally, according to some examples, the navigation system is also activated at step 720. When planning the vehicle route at step 722, the server 400 may determine, depending on the type of particular alert, that it is safer to avoid the areas related to the alert.

[0048] Once the AI ​​agent determines at step 710 that there are matching images captured by the vehicle cameras 104, as in the AMBER alert, the server 400 may notify the user of the matching data and either prompt the user to call law enforcement or use the vehicle telematics system, such as Guardian® or OnStar®, to connect directly with law enforcement at step 724. Additionally, according to some examples, the matching images are also transmitted to a back office of the vehicle telematics system at step 726 and uploaded to a vehicle database at step 728 to enhance pattern recognition.

[0049] If server 400 determines that a matching acoustic sound has been found, server 400 may be additionally configured to connect to the back office of the vehicle telematics system at step 730 in anticipation of an emergency call from the vehicle telematics system and / or the user at step 732. If it is determined, such as at step 734, that the matching acoustic sound is a severe weather warning, server 400 may be additionally configured to notify the user of the severe weather warning and further determine whether the vehicle's route may be interrupted or otherwise delayed.

[0050] Referring now to the exemplary flow chart of operations of the vehicle system 100 in Fig. 6, when the vehicle 10 is in operation, the one or more vehicle cameras 104 are operating, transmitting data to the server 400. The server 400 may be configured to detect other vehicles 10 and / or extract images of a region of interest (ROI) in step 800. Next, the region of interest may be processed and / or segmented to determine the license plate recognition from the ROI in step 802. The license plate information is then transmitted to determine in step 804 whether the license plate information matches the information extracted from the alarm and whether the data has met a threshold to determine whether a matching vehicle has been found. In step 806, the vehicle characteristics extracted by the vehicle camera 104, such asthe color, make, or model, is processed to determine if the detected color, make, or model matches the values ​​extracted from the alarm. The color, make, and model information is then transmitted to determine if the data meets the threshold to determine if a matching vehicle has been located. Additionally, the server 400 may be configured to use analytics based on proper noun recognition to determine vehicle information in step 808. Finally, if it is determined in step 804 that the data exceeds a predetermined match threshold, the server 400 may be configured, according to some examples, to initiate a call to share the information with law enforcement in step 810.

[0051] In Fig.7, an exemplary flowchart of the operations of the vehicle system 100 is set forth. More specifically, the vehicle microphone 102 is configured in step 900 to begin listening (i.e., transmitting and / or recording) audio data. According to some examples, the vehicle microphone 102 may be activated when a user turns on the vehicle 10 or otherwise logs into the vehicle system 100. Once the vehicle microphone 102 is listening and audio data is being transmitted, the server 400 is configured to determine in step 902 whether a crash response is detected from the user device 300 or whether a response tone of the vehicle telematics system is detected in step 904. If a crash response is detected, the server 400 determines in step 906 whether automatic crash response (ACR) has been activated by the vehicle 10. If automatic crash response, such asIf an emergency call has been activated via the vehicle telematics system, such as Guardian® or OnStar®, or other telematics device, the server 400 may notify the user in step 908 to take further action, such as stopping the emergency call from the user device 300 to eliminate duplicate emergency calls. If automatic crash response is not detected, the server 400 may be configured to ask the user in step 910 if an emergency exists and offer to place an emergency call for the user via the user device 300. If a new call is connected in step 912, then the server 400 is configured to determine in step 914 whether more than one user device 300 is making an outgoing call. If there is more than one outgoing call, the server 400 is configured to notify the user in step 916 to terminate one of the calls to eliminate duplicate emergency calls.Similarly, if the new call is connected in step 912, the server 400 is configured to determine in step 918 whether a call from the vehicle telematics system is ongoing. If a call from the vehicle telematics system is ongoing, the server 400 is configured to notify the user to end the call in step 920, or the server 400 is configured to connect to the user device 300 via Bluetooth® or another method to end the dual emergency call. If the call from the vehicle telematics system is ongoing, the server 400 is additionally configured to set a flag for the vehicle telematics system account, such as Guardian® or OnStar®, to notify an advisor that multiple emergency calls are ongoing so that the advisor can notify the user and / or the appropriate emergency services.

[0052] The vehicle system 100, as described herein, includes the vehicle microphone 102, the one or more cameras 104, the vehicle processor 200, and the server 400. The vehicle processor 200 transmits acoustic data 206 from the vehicle microphone 102 to the server 400, where the server 400 is configured to determine whether the transmitted acoustic data 206 matches any predetermined acoustic data 206. According to examples where the predetermined acoustic data 206 is an AMBER alert, the vehicle system 100 may then assist law enforcement by sensing the vehicle environment and determining whether the vehicle environment is relevant to the current AMBER alert. If relevant, this information may be sent to law enforcement to assist in returning abducted children to their families.Additionally, according to the example where the predetermined acoustic data 206 is an automatic emergency call from the user device 300, the vehicle system 100 may determine whether this automatic emergency call would be a double emergency call and then prompt the user to end the call and / or connect via Bluetooth® and forcefully terminate the emergency call to prevent confusion among emergency services. As such, the vehicle system 100, as described herein, provides valuable safety enhancements for the user of the vehicle 10 and the public at large.

[0053] A number of implementations have been described. It is recognized, however, that various modifications may be made without departing from the spirit and scope of the disclosure. Accordingly, other implementations are within the scope of the following claims.

[0054] The foregoing description has been provided for purposes of illustration and description. It is not intended to be exhaustive or limiting of the disclosure. Individual elements or features of a particular configuration are generally not limited to that particular configuration, but are interchangeable and may be used in a selected configuration, even if not specifically shown or described. Furthermore, they may be varied in various ways. Such variations are not to be regarded as a departure from the disclosure, and all such modifications are intended to be included within the scope of the disclosure.

Claims

[1] Vehicle system for detecting acoustic frequencies, the vehicle system comprising: a user device configured to generate acoustic sounds at different frequencies; a vehicle microphone for collecting acoustic data from the user device; a vehicle processor for storing vehicle data containing the acoustic data; and a server communicatively coupled to the vehicle processor and configured to determine whether the acoustic data matches one or more predetermined sound frequencies. [2] The vehicle system of claim 1, wherein the server is configured to digitally sample the collected acoustic data. [3] The vehicle system of claim 2, wherein the server is configured to extract the collected acoustic data into succinct features including one or more of the mel-cepstrum coefficients and the delta energy coefficients. [4] The vehicle system of claim 1, wherein the server is configured to send a notification to a user if the acoustic data matches one or more predetermined sound frequencies. [5] The vehicle system of claim 1, wherein the server is configured to send a notification to law enforcement if the acoustic data matches one or more predetermined sound frequencies. [6] The vehicle system of claim 1, wherein the server is configured to connect to the user device via Bluetooth® if the acoustic data collected by the vehicle microphone matches a predetermined sound frequency. [7] The vehicle system of claim 1, further comprising a vehicle camera for collecting information related to the surroundings of the vehicle. [8] The vehicle system of claim 7, wherein the server is further configured to collect the data related to the surroundings of the vehicle using the vehicle camera. [9] The vehicle system of claim 8, wherein the server is further configured to extract information related to the determined sound frequency and analyze the vehicle data and the collected data from the vehicle camera to determine if any data matches the extracted information. [10] A vehicle including the vehicle system of claim 1.

Images