Vehicle system for detecting acoustic frequencies

Abstract

A vehicle system includes a user device configured to generate acoustic sounds at different frequencies. The vehicle system also includes a vehicle microphone for collecting acoustic data from the user device. Further, the vehicle system includes a vehicle processor for storing vehicle data including the acoustic data and a server communicatively coupled to the vehicle processor. In some examples, a server is configured to determine whether acoustic data matches one or more predetermined sound frequencies.

Description

Technical Field

[0001] The present invention generally relates to vehicle systems for detecting acoustic frequencies. Background Art

[0002] The information provided in this section is for the purpose of generally presenting the background of the present disclosure. To the extent described in this section, the work of the presently named inventors, as well as aspects of the description that may not qualify as prior art at the time of filing, are neither explicitly nor implicitly admitted to be prior art with respect to the present disclosure.

[0003] An AMBER Alert is activated in the most serious cases of child abduction. The purpose of an AMBER Alert is to immediately motivate the community to assist in the search and safe recovery of a missing child. These alerts are broadcast via radio, television, road signs, mobile phones, and other data-enabled devices. Some types of alerts, such as those from cellular phones, have a specific acoustic sound accompanying the alert. In addition, the alert may include information about the abducted child and / or potential suspect. In addition, the alert may provide information about the vehicle used to abduct the child, including the make, model, and license plate number of the vehicle.

[0004] Vehicle telematics data for a particular vehicle may already capture one or more aspects of the aforementioned data, including the make, model, and / or license plate number of vehicles traveling nearby. While such data is typically captured for use by the vehicle in controlling vehicle operation and / or assisting the driver, such telematics data may provide information that can be used by authorities in locating a trafficked child. Summary of the invention

[0005] In one configuration, a vehicle system includes a user device configured to generate acoustic sounds of different frequencies and a vehicle microphone for collecting acoustic data from the user device. The vehicle system also includes a vehicle processor for storing vehicle data including the acoustic data and a server communicatively coupled to the vehicle processor. In some examples, the server is configured to determine whether the acoustic data matches one or more predetermined sound frequencies. The server may also be configured to digitally sample the collected acoustic data. In some examples, the server is configured to extract the collected acoustic data into concise features including one or more of mel cepstral coefficients and delta energy coefficients.

[0006] The server may also be configured to send a notification to the user if the acoustic data matches one or more predetermined sound frequencies. In some examples, the server is configured to send a notification to law enforcement if the acoustic data matches one or more predetermined sound frequencies. Additionally or alternatively, if the acoustic data collected from the vehicle microphone matches the predetermined sound frequencies, the server may be configured to send a notification to law enforcement via Connected to a user device. A vehicle may include the vehicle system.

[0007] In another configuration, a vehicle system includes a user device configured to generate acoustic sounds of different frequencies. The vehicle system also includes a vehicle microphone for collecting acoustic data from the user device and a vehicle camera for collecting information related to the vehicle's surroundings. In addition, the vehicle system includes a vehicle processor for storing vehicle data including the acoustic data and the vehicle's location. The server can be communicatively coupled to the vehicle processor.

[0008] The server can be configured to determine whether the acoustic data matches a predetermined sound frequency and extract information related to the determined sound frequency. In some examples, the server is further configured to collect data related to the vehicle's surroundings using a vehicle camera and analyze the vehicle data and the collected data to determine whether any data matches the extracted information. In addition, the server can be configured to send a notification if the collected data matches the extracted information.

[0009] In some examples, the server is configured to digitally sample the collected acoustic data. Additionally or alternatively, the server can be configured to extract the collected acoustic data into concise features including one or more of mel cepstral coefficients and delta energy coefficients. Furthermore, the server can analyze the collected data using tokenization. In some examples, the server is configured to send a notification to a user if the collected data matches the extracted information. In some examples, the server is configured to send a notification to law enforcement if the collected data matches the extracted information. A vehicle can include the vehicle system.

[0010] In yet another configuration, a vehicle system includes a user device configured to generate acoustic sounds of different frequencies and a vehicle microphone for collecting acoustic data from the user device. In addition, the vehicle system includes a vehicle processor for storing vehicle data including acoustic data and vehicle event data. A server is communicatively coupled to the vehicle processor. In some examples, the server is configured to determine whether the acoustic data collected from the vehicle microphone matches a predetermined sound frequency, and if the acoustic data matches the predetermined sound frequency, alert the vehicle occupants. The server may use the vehicle event data to determine whether a vehicle event has occurred. The vehicle event may be one or more of a vehicle collision or a vehicle road exit. In some examples, the server is configured to determine whether an automatic vehicle response has been implemented. The server may be configured to cancel a warning to the vehicle occupants if the automatic vehicle response has not been implemented. The vehicle may include the vehicle system. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The drawings described herein are for illustrative purposes only of selected configurations and are not intended to limit the scope of the present disclosure.

[0012] Figure 1is a perspective view of the exterior of a vehicle including a vehicle system according to the present disclosure;

[0013] Figure 2 A vehicle system according to the present disclosure is provided. Figure 1 A perspective view of the interior of a vehicle;

[0014] Figure 3 is an exemplary functional block diagram according to an aspect of the present disclosure;

[0015] Figure 4 is an exemplary functional block diagram according to another aspect of the present disclosure;

[0016] Figure 5 is an exemplary flow chart according to another aspect of the present disclosure;

[0017] Figure 6 is an exemplary flow chart according to another aspect of the present disclosure; and

[0018] Figure 7 is an exemplary flow chart according to another aspect of the present disclosure.

[0019] Corresponding reference characters indicate corresponding parts throughout the several views. DETAILED DESCRIPTION

[0020] Example configurations will now be described more fully with reference to the accompanying drawings. Example configurations are provided so that the present disclosure will be thorough and will fully convey the scope of the present disclosure to those of ordinary skill in the art. Specific details, such as examples of specific components, devices, and methods, are set forth to provide a thorough understanding of the configurations of the present disclosure. It will be apparent to those of ordinary skill in the art that specific details need not be employed, that example configurations may be implemented in many different forms, and that the specific details and exemplary configurations should not be construed as limiting the scope of the present disclosure.

[0021] The terms used herein are only for describing specific exemplary configurations, not for limiting. As used herein, the singular articles "a", "an" and "the" may also be intended to include plural forms, unless the context clearly indicates otherwise. The terms "comprise", "include", "contain" and "have" are inclusive, thus specifying the presence of features, steps, operations, elements and / or parts, but do not exclude the presence or addition of one or more other features, steps, operations, elements, parts and / or groups thereof. The method steps, processes and operations described herein should not be interpreted as necessarily requiring them to be performed in the specific order discussed or shown, unless specifically identified as an execution order. Additional or alternative steps may be adopted.

[0022] When an element or layer is referred to as being "on another element or layer," "engaged to," "connected to," "attached to," or "coupled to" another element or layer, it may be directly on, engaged, connected, attached to, or coupled to another element or layer, or there may be intermediate elements or layers. Conversely, when an element is referred to as being "directly on another element or layer," "directly engaged to," "directly connected to," "directly attached to," or "directly coupled to" another element or layer, there may be no intermediate elements or layers. Other words used to describe the relationship between elements should be interpreted in a similar manner (e.g., "between" versus "directly between," "adjacent" versus "directly adjacent," etc.). As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0023] The terms first, second, third, etc. can be used here to describe various elements, components, regions, layers and / or parts. These elements, components, regions, layers and / or parts should not be limited by these terms. These terms are only used to distinguish each element, component, region, layer or part. Terms such as "first", "second" and other numerical terms do not imply order or sequence unless the context clearly indicates. Therefore, the first element, component, region, layer or part discussed below can be referred to as the second element, component, region, layer or part, without departing from the teaching of the example configuration.

[0024] In this application, including the definitions below, the term module may be replaced by the term circuit. The term "module" may refer to or be part of an application specific integrated circuit (ASIC), or include an application specific integrated circuit (ASIC); a digital, analog, or mixed analog / digital discrete circuit; a digital, analog, or mixed analog / digital integrated circuit; a combinational logic circuit; a field programmable gate array (FPGA); a processor (shared, dedicated, or group) that executes code; a memory (shared, dedicated, or group) that stores code executed by the processor; other suitable hardware components that provide the functionality; or a combination of some or all of the above, such as in a system on a chip.

[0025] The term code used above may include software, firmware and / or microcode, and may refer to a program, a routine, a function, a class and / or an object. The term shared processor includes a single processor that executes some or all of the code from multiple modules. The term group processor includes a processor that executes some or all of the code from one or more modules in combination with an additional processor. The term shared memory includes a single memory that stores some or all of the code from multiple modules. The term group memory includes a memory that stores some or all of the code from one or more modules in combination with an additional memory. The term memory may be a subset of the term computer-readable medium. The term computer-readable medium does not include transient electrical signals and electromagnetic signals propagated through the medium and can therefore be considered to be tangible non-transitory memory. Non-limiting examples of non-transitory memory include tangible computer-readable media, including non-volatile memory, magnetic memory, and optical memory.

[0026] The apparatus and methods described in this application may be implemented in part or in whole by one or more computer programs executed by one or more processors. The computer program includes processor executable instructions stored on at least one non-transitory tangible computer readable medium. The computer program may also include and / or rely on stored data.

[0027] Software applications (i.e., software resources) may refer to computer software that enables a computing device to perform tasks. In some examples, software applications may be referred to as "applications," "apps," or "programs." Example applications include, but are not limited to, system diagnostic applications, system management applications, system maintenance applications, word processing applications, spreadsheet applications, messaging applications, media streaming applications, social networking applications, and gaming applications.

[0028] Non-transitory memory can be a physical device used to temporarily or permanently store programs (e.g., instruction sequences) or data (e.g., program state information) for use by a computing device. Non-transitory memory can be volatile and / or non-volatile addressable semiconductor memory. Examples of non-volatile memory include, but are not limited to, flash memory and read-only memory (ROM) / programmable read-only memory (PROM) / erasable programmable read-only memory (EPROM) / electrically erasable programmable read-only memory (EEPROM) (e.g., commonly used for firmware, such as bootloaders). Examples of volatile memory include, but are not limited to, random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), phase change memory (PCM), and disk or tape.

[0029] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for a programmable processor and may be implemented in high-level procedural and / or object-oriented programming languages ​​and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, non-transitory computer-readable medium, apparatus, and / or device (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0030] Various implementations of the systems and techniques described herein can be realized in digital electronic and / or optical circuits, integrated circuits, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementations in one or more computer programs executable and / or interpretable on a programmable system comprising at least one programmable processor, which may be special purpose or general purpose, coupled to receive data and instructions from a storage system, and to send data and instructions to the storage system.

[0031] The processes and logic flows described in this specification can be performed by one or more programmable processors, also known as data processing hardware, executing one or more computer programs to perform functions by operating on input data and generating outputs. These processes and logic flows can also be performed by dedicated logic circuits, such as FPGAs (field programmable gate arrays) or ASICs (application-specific integrated circuits). For example, processors suitable for executing computer programs include general-purpose and special-purpose microprocessors, and any one or more processors of any kind of digital computer. Typically, the processor will receive instructions and data from a read-only memory or a random access memory or both. The basic elements of a computer are a processor for executing instructions and one or more storage devices for storing instructions and data. Typically, a computer will also include or be operably connected to one or more large-capacity storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, to receive data from the large-capacity storage device or to transmit data to it, or both. However, a computer does not need to have such a device. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and storage devices, including, for example, semiconductor memory devices, such as EPROM, EEPROM and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD ROM and DVD-ROM disks. The processor and memory may be supplemented by or incorporated in special purpose logic circuitry.

[0032] To provide interaction with a user, one or more aspects of the present disclosure may be implemented on a computer having a display device for displaying information to the user, such as a CRT (cathode ray tube), LCD (liquid crystal display) monitor, or touch screen, and an optional keyboard and pointing device, such as a mouse or trackball, through which the user can provide input to the computer. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user may be received in any form, including sound, voice, or tactile input. In addition, the computer may interact with the user by sending documents to and receiving documents from a device used by the user; for example, by sending a web page to a web browser on a user's client device in response to a request received from the web browser.

[0033] refer to Figure 1-7 , Figure 11 shows a vehicle 10 with a vehicle system 100. The vehicle 10 is envisioned as an electric vehicle 10 (EV), and may include autonomous or semi-autonomous capabilities. Additionally or alternatively, the vehicle 10 may be a hybrid vehicle 10 including EV and internal combustion engine (ICE) components and capabilities. Furthermore, if desired, the vehicle 10 may include only an ICE or other power source. In some examples, the vehicle 10 may be a land vehicle, a water vehicle, or an air vehicle.

[0034] refer to Figure 1 and Figure 2 , the vehicle system 100 is shown in conjunction with a user device 300, which is configured to generate acoustic sounds of different frequencies. In addition, the vehicle system 100 includes a vehicle microphone 102 for collecting acoustic data 206, a vehicle camera 104 for collecting video data, a vehicle processor 200, and a server 400 communicatively coupled to the vehicle processor 200. The user device 300 includes a display and data processing hardware configured to execute commands from the vehicle processor 200 and / or the server 400. In some examples, the user device 300 is a cellular phone or tablet, however, it is contemplated that the user device 300 can be any device including a display and data processing hardware required to execute commands from the vehicle processor 200 and / or the server 400. The user device 300 may include processing hardware and software that allows the user device 300 to communicate with a cellular network. The cellular network may be a global system for mobile communications (GSM) network such as 5G, however, the cellular network may also be a code division multiple access (CDMA) network, a time division multiple access (TDMA) network, or other network.

[0035] The user device 300 is configured to generate a plurality of acoustic sounds. In some examples, the acoustic sound notifies the user of a call or text. However, in some examples, the acoustic sound notifies the user of other events, such as an Amber Alert or an emergency call. For example, many devices are equipped with vehicle collision detection software so that if the user device 300 detects a vehicle collision, the user device 300 can automatically call emergency services. In order to notify the user that an emergency call is occurring, the user device 300 plays a sound. In other examples, if an Amber Alert is issued from the National Center for Missing and Exploited Children, the user device 300 plays an acoustic sound. In addition, in other examples, weather alerts, such as one or more of the Federal Emergency Management Agency (FEMA) alerts, earthquake alerts, tornado alerts, and / or tsunami alerts, may also be sent to the user device 300. Each sound may be unique so that acoustic detection can identify each alert type.

[0036] Still refer to Figure 1 and 2In the example shown, the vehicle system 100 also includes a vehicle microphone 102 disposed inside the vehicle 10. The vehicle microphone 102 can be any type of microphone configured to transmit and / or record acoustic sounds disposed in and around the vehicle 10. In some examples, the vehicle microphone 102 is a dynamic microphone, a condenser microphone, and / or a ribbon microphone. However, the vehicle microphone 102 can be any type of microphone as desired. The vehicle interior can also include a vehicle dashboard disposed inside, which can be configured to display text or images to the driver via a display associated with the vehicle dashboard.

[0037] like Figure 1 As best shown, the vehicle system 100 includes a vehicle camera 104. The vehicle camera 104 can be any type of vehicle camera 104, including but not limited to an interior camera, a rear camera, a front camera, a side camera, and / or a perimeter camera. The vehicle camera 104 can be one or more of a monocular camera, a binocular camera, a wide-angle camera, and the like. In some examples, the vehicle camera 104 includes a plurality of cameras 104, including the same type of camera or a plurality of different types of cameras. Regardless of the number, type, and location of the vehicle cameras 104, the vehicle cameras 104 can be configured to record and / or transmit video data to the vehicle processor 200 and / or the server 400. The video data can include video data of the interior of the vehicle 10 and / or video data of an area outside the vehicle 10.

[0038] The vehicle processor 200 stores vehicle data 202 for the vehicle 10. In some examples, the vehicle data 202 includes a vehicle position 204, acoustic data 206, vehicle camera data 208, and / or vehicle event data 210. The vehicle position 204 generally relates to the location of the vehicle 10. The vehicle position 204 may be obtained from a global positioning system (GPS), other navigation systems, a user device 300, and / or a third-party application. Regardless of how the vehicle position 204 is obtained, the information may be transmitted to the vehicle processor 200. In addition, the vehicle position data 204 may include route data, such that the route traveled by the vehicle 10 may also be transmitted to the vehicle processor 200.

[0039] The acoustic data 206 generally relates to any acoustic sound detected by the vehicle microphone 102. The acoustic sound may be generated by the user device 300, or may be any other sound detected by the vehicle microphone 102.

[0040] The vehicle camera data 208 generally refers to any recorded and / or transmitted video from any one or more of the vehicle cameras 104. In some examples, the vehicle camera data 208 may be data including license plate images of nearby vehicles. Typically, the vehicle camera data 208 is collected by the vehicle cameras 104 and transmitted to the vehicle processor 200. However, it is contemplated that the camera data 208 may additionally and / or alternatively be collected by crowdsourcing camera data 208 from nearby vehicles. The data from nearby vehicles may be collected by the vehicle processor 200 and / or the server 400 via third-party applications or other similar methods.

[0041] The vehicle event data 210 generally relates to whether a vehicle event, such as a vehicle collision, has occurred. The vehicle event data 210 may include, but is not limited to, vehicle speed, route, or lane information that reflects that a vehicle event has occurred. For example, the vehicle event data 210 may include data indicating that the vehicle has left the road, which in turn may indicate that the vehicle 10 has been involved in a collision. For example, if one or more vehicle cameras 104 detect damage to the vehicle 10 and / or contact with another vehicle, such data may be verified by the camera 104. The vehicle processor 200 may capture additional data related to the vehicle leaving the road, such as whether the vehicle 10 has returned to the road, has been off the road for an extended period of time, or is in a position that may prevent the vehicle 10 from returning to the road (i.e., near or in a body of water).

[0042] Reference again Figure 1-7, the server 400 is configured as a network and / or cloud-based system that communicates with one or both of the vehicle processor 200 and the user device 300. In some examples, the server 400 is also configured to communicate with a third-party processor 500 to collect third-party data 502. For example, the third-party processor 500 may include, but is not limited to, a vehicle processor 200 along a route input via the user device 300. Additionally or alternatively, the third-party processor 500 may include a third-party user device 300 within a vehicle 10 along the route. It is generally contemplated that the third-party data 502 includes vehicle position data 204, vehicle event data 210, vehicle camera data 208, and / or acoustic sound data 206 from the third-party processor. In some examples, the vehicle position data 204 is configured to transmit the vehicle position 204 of each respective vehicle 10 traveling along the route to the server 400. The vehicle event data 210 may be configured to transmit vehicle events to the server 400, including but not limited to vehicle speed, lane merging, vehicle steering wheel angle, or other vehicle operations associated with a vehicle collision or a vehicle road exit. In some examples, vehicle camera data 208 includes transmitted or recorded images that may be related to information received in various alerts that may be transmitted to server 400. Additionally, in some examples, acoustic data 206 from vehicle microphone 102 is configured to be transmitted to server 400 to determine whether acoustic data 206 matches any predetermined acoustic sounds.

[0043] It is also contemplated that the vehicle processor 200 may transmit any or all of the vehicle data 202 to the server 400 for further processing and / or evaluation. The vehicle processor 200 and / or the server 400 may be continuously and / or periodically updated so that the vehicle data 202 is updated in real time.

[0044] Reference now Figure 3In the example shown, the server 400 is configured to determine whether the acoustic data 206 matches one or more predetermined sound frequencies. In some examples, the server 400 may be configured to digitally sample the collected acoustic data 206. In addition, in step 500, the acoustic data 206 captured by the vehicle microphone 102 may then be stripped to remove engine noise and other background noise. In some examples, the enhanced acoustic signal then undergoes feature extraction in step 502. For example, the server 400 may be configured to extract the collected acoustic data 206 into one or more concise features including Mel cepstral coefficients, delta energy coefficients, filter bank energy, logarithmic filter bank energy, and spectral subband centroids. Statistical algorithms including one or more of random forests, deep neural networks, and Bayesian classifiers may be used to help determine whether the acoustic data matches one or more predetermined sound frequencies. Then, in step 504, these features are input into an optimization model, where the server 400 determines whether the acoustic features match the predetermined sound of the optimization model.

[0045] The optimized model may be continuously or otherwise updated and improved. For example, in step 506, various noise considerations, including but not limited to season, weather conditions, location, background noise, reverberation, drone speed, etc., are used to form and expand and / or group the data set. Then, in step 508, the data set is sent to data preprocessing, and in step 509, it is output as a preprocessed data set. In addition, in some examples, in step 510, the preprocessed data set is input into a machine learning device for training and testing the data set. In some examples, the test data is input into the model training in step 512, where the machine learning (ML) algorithm iterates the best model at 514 before the best model is input into the candidate model in step 516. In addition, in step 514, the training data can be directly input as a candidate model. Once the model is optimized, the optimized model returns to step 504, where it can be used to identify sound events, such as Amber Alerts or emergency calls.

[0046] Additionally, in some examples, the server 400 is configured to consider data from the alert when using the navigation feature. For example, if the acoustic data is determined to match a predetermined sound frequency such as a tsunami alert, the server 400 can suggest a route that avoids the area where the alert is located, or if the vehicle 10 current route is close to the alert location, a re-routing option can be suggested.

[0047] Additionally, in some examples, the server 400 is further configured to send a notification to the user if the acoustic data 206 matches one or a predetermined sound frequency. For example, if the server 400 determines that an Amber Alert has been detected, the server 400 can be configured to notify the driver of the alert via audio and / or via the vehicle dashboard.

[0048] Reference again Figure 1-7 In the example shown in , if the server 400 determines that the acoustic data 206 matches a predetermined sound frequency, the server 400 can determine that an Amber Alert has been activated. Once the server 400 has determined that the Amber Alert has been activated, the server 400 can extract information related to the Amber Alert. The extracted information may include information related to the suspect, including but not limited to vehicle information, such as the make, model, color and / or license plate of the vehicle. The license plate information may include the state, license plate number, and license plate color. In addition, in some examples, tokenization can be used on the extracted information to protect sensitive data.

[0049] Reference now Figure 4 In the example shown, once an Amber Alert is received, at step 600, the server 400 may use segmentation to break down the text contained in the alert into actionable sentences. The sentences may then be tokenized at step 602 to protect privacy. Further, once tokenization has occurred, at step 604, the data may undergo word embedding to determine feature vectors. Further, at step 606, the data undergoes named entity recognition (NER) to aid in extracting entities. Further, at step 608, the extracted entities may be processed into an alert, and the data may be sent to other places in the server 400 for additional processing at step 610, as described in more detail below.

[0050] Once the data from the alert is extracted, the server 400 is configured to collect data related to the vehicle's surroundings using the vehicle camera 104. The collected data may include, but is not limited to, information extracted from the Amber Alert, such as the vehicle make, model, color, and / or license plate information described above. In addition, in some examples, the server 400 may be configured to analyze the vehicle data 202 and / or data collected from the vehicle camera data 208 to determine if any data matches the information extracted from the Amber Alert. In some examples, to analyze the data, the server 400 uses a weighted algorithm for each of these attributes extracted from the alert. For example, the color of the vehicle may be assigned a higher weight, while the model of the vehicle may be assigned a lower weight, thereby providing customized analysis for each alert.

[0051] If the server 400 determines that any collected data matches the extracted information, the server 400 can be configured to send a notification with the information to the driver and / or law enforcement. The notification can include one or more of an audio notification, a visual display on the dashboard, a notification to the user device 300, or a phone call or text message.

[0052] Furthermore, in some examples, if the server 400 determines that an emergency call tone has been detected, the server 400 can be configured to notify the driver of an alert via an audio and / or visual indication. Additionally or alternatively, if the server 400 determines that the acoustic data 206 collected from the vehicle microphone 102 matches a predetermined sound frequency, the server 400 can be configured to notify the driver via an audio and / or visual indication. Or other wireless methods to connect to user device 300. In some examples, when server 400 is connected to user device 300, server 400 can send a notification to law enforcement, or server 400 can be configured to end repeated emergency calls (i.e., if user device 300 has already made a call, end the call made by vehicle 10, or end the call made by user device 300).

[0053] Additionally, in some examples, if the server 400 determines that the acoustic data 206 matches a predetermined sound frequency, and the vehicle data indicates a vehicle event such as a vehicle road exit or a vehicle collision, the server 400 may determine that an emergency call has been placed. Additionally, if the acoustic data 206 matches the predetermined frequency of the sound, the server 400 uses the vehicle event data 210 to determine whether a vehicle event has occurred. If a vehicle event such as a vehicle collision has occurred, the server 400 is configured to determine whether an automatic vehicle response has been implemented. In some examples, the automatic vehicle response may be an automatic vehicle response initiated by a vehicle telematics system such as or or other vehicle communication systems to the emergency services. In addition, in some examples, if an automatic vehicle response has been implemented, the server 400 can be configured to Connecting to the user device 300 and cancelling the emergency call. Alternatively, in some examples, if the automated vehicle 10 response is not implemented, the server 400 may be configured to cancel the alert so that the emergency call from the user device 300 continues.

[0054] Reference now Figure 1-7 ,exist Figure 5-7 An example flow chart of the operation of the vehicle system 100 is set forth in Figure 5In an example flow chart of the operation of the vehicle system 100 shown, at step 700, a push notification from law enforcement, such as an Amber Alert, is pushed through a cellular server. The user device 300 receives the notification at step 702 and plays a sound associated with the alert. At step 704, the vehicle microphone 102 is configured to transmit the acoustic data 206 to the server 400 for comparison with the predetermined acoustic data 206. If a matching acoustic sound, such as an Amber Alert, is found, the server 400 is configured to determine the vehicle location 204 at step 708 so that only vehicles 10 in the vicinity of the alert will be activated. Next, at step 710, the vehicle location 204 is input into an artificial intelligence (AI) agent to scan images captured from vehicle cameras 104 of nearby vehicles to determine if any of the images match the data from the alert. Concurrently with step 708, the server 400 is configured to extract raw text data from the Amber Alert at step 712. Next, in step 714, semantic analysis and entity extraction are performed on the raw text data, and in step 710, the extracted entities are also transmitted to the AI ​​agent to scan the images captured from the vehicle cameras 104 of nearby vehicles to determine whether any images match the data from the alert. In addition, at the same time as steps 708 and 712, the vehicle camera 104 is activated to transmit and / or record data, and the data is transmitted to the server 400 in step 716, and then image features are extracted in step 718. Once the image features are extracted, in step 710, the extracted features are transmitted to the AI ​​agent to scan the images captured from the vehicle cameras 104 of nearby vehicles to determine whether any images match the data from the alert. In addition, in some examples, the navigation system is also activated in step 720. When planning the vehicle route in step 722, depending on the determined alert type, the server 400 may determine that it is safer to avoid the area associated with the alert.

[0055] Once, at step 710, the AI ​​agent has determined that there is a matching image captured from the vehicle camera 104, such as in an Amber Alert, at step 724, the server 400 may notify the user of the matching data and either ask the user to call law enforcement, or may use a method such as or The vehicle telematics system is directly connected to the law enforcement department. In addition, in some examples, at step 726, the matching image is also transmitted to the back office of the vehicle telematics system and uploaded to the vehicle database at step 728 to improve pattern recognition.

[0056] In addition, if the server 400 determines that a matching sound is found, the server 400 can be configured to connect to the vehicle telematics system backend at step 730, anticipating an emergency call from the vehicle telematics system and / or the user at step 732. In addition, if the matching sound is determined to be a severe weather alert, for example at step 734, the server 400 can be configured to notify the user of the severe weather alert, and further determine whether the vehicle route may be interrupted or otherwise delayed.

[0057] Reference now Figure 6 , when the vehicle 10 is running, the vehicle camera 104 is running and sending data to the server 400. At step 800, the server 400 can be configured to detect other vehicles 10 and / or extract region of interest (ROI) images. Next, at step 802, the region of interest can be processed and / or segmented to determine license plate recognition based on the ROI. Then, at step 804, the license plate information is sent to determine whether the license plate information matches the information extracted from the alert and whether the data reaches a threshold to determine whether a matching vehicle has been found. At step 806, the vehicle attributes extracted from the vehicle camera 104, such as color, make, or model, are processed to determine whether the detected color, make, or model matches the value extracted from the alert. The color, make, and model information is then sent to determine whether the data reaches a threshold to determine whether a matching vehicle has been located. In addition, at step 808, the server 400 can be configured to determine vehicle information using analysis based on named entity recognition. Finally, if it is determined at step 804 that the data exceeds a predetermined matching threshold, in some examples, at step 810 , server 400 may be configured to initiate a call to share the information with law enforcement.

[0058] Reference now Figure 7 , sets forth an example flow chart of the operation of the vehicle system 100. More specifically, at step 900, the vehicle microphone 102 is configured to begin listening for (i.e., transmitting and / or recording) audio data. In some examples, the vehicle microphone 102 may be activated when a user turns on the vehicle 10 or opts into the vehicle system 100. Once the vehicle microphone 102 is listening and audio data is being sent, the server 400 is configured to determine whether a crash response from the user device 300 is detected at step 902, or whether a vehicle telematics system response tone is detected at step 904. If a crash response is detected, at step 906, the server 400 determines whether an automatic crash response (ACR) has been activated by the vehicle 10. If an automatic crash response has been activated, such as by the vehicle telematics system (e.g. or or other telematics device), then at step 908, the server 400 may notify the user to take further action, such as stopping the emergency call from the user device 300, to eliminate duplicate emergency calls. If no automatic collision response is detected, then at step 910, the server 400 may be configured to ask the user whether there is an emergency and place an emergency call for the user through the user device 300. Then, if a new call is connected at step 912, then at step 914, the server 400 is configured to determine whether more than one user device 300 is making outgoing calls. If there are more than one outgoing calls, then at step 916, the server 400 is configured to notify the user to end one of the calls to eliminate duplicate emergency calls. Similarly, if a new call is connected at step 912, the server 400 is configured to determine at step 918 whether a call from the vehicle telematics system is in progress. If a call from the vehicle telematics system is in progress, then at step 920, the server 400 is configured to notify the user to end the call, or the server 400 is configured to notify the user to end the call, or the server 400 is configured to notify the user to end the call, or to terminate the call via Or another method to connect to the user device 300 to end the repeated emergency calls. In addition, if a call from the vehicle telematics system is in progress, the server 400 is configured to send a call to the vehicle telematics system such as or The account sets a flag to notify the advisor that multiple emergency calls are in progress so that the advisor can notify the user and / or the appropriate emergency services.

[0059] The vehicle system 100 described herein includes a vehicle microphone 102, a vehicle camera 104, a vehicle processor 200, and a server 400. The vehicle processor 200 transmits acoustic data 206 from the vehicle microphone 102 to the server 400, wherein the server 400 is configured to determine whether the transmitted acoustic data 206 matches any predetermined acoustic data 206. In the example where the predetermined acoustic data 206 is an Amber Alert, the vehicle system 100 can then assist law enforcement by capturing the vehicle surroundings and determining whether the vehicle surroundings are relevant to the current Amber Alert. If so, such information can be sent to law enforcement to assist in returning abducted children to their families. Additionally, in the example where the predetermined acoustic data 206 is an automatic emergency call from a user device 300, the vehicle system 100 can determine whether the automatic emergency call is a repeat emergency call and then prompt the user to end the call and / or terminate the call via The emergency call is connected and forcefully terminated to prevent confusion of emergency services. In this way, the vehicle system 100 described herein provides valuable safety enhancements for the user of the vehicle 10 and the community at large.

[0060] Many embodiments have been described. However, it should be understood that various modifications can be made without departing from the spirit and scope of the present disclosure. Therefore, other embodiments are also within the scope of the following claims.

[0061] The foregoing description has been provided for the purpose of illustration and description. It is not intended to be exhaustive or to limit the present disclosure. Individual elements or features of a particular configuration are generally not limited to that particular configuration, but where applicable, they are interchangeable and can be used in a selected configuration even if not specifically shown or described. This can also be varied in a variety of ways. Such variations should not be considered a departure from the present disclosure, and all such modifications are intended to be included within the scope of the present disclosure.

Claims

1. A vehicle system for detecting acoustic frequencies, the vehicle system comprising: a user device configured to generate acoustic sounds of varying frequencies; A vehicle microphone to collect acoustic data from the user's device; a vehicle processor for storing vehicle data including acoustic data; as well as A server is communicatively coupled to the vehicle processor and is configured to determine whether the acoustic data matches one or more predetermined sound frequencies.

2. The vehicle system according to claim 1, wherein: The server is configured to digitally sample the collected acoustic data.

3. The vehicle system according to claim 2, wherein: The server is configured to extract the collected acoustic data into concise features including one or more of mel-cepstral coefficients and delta energy coefficients.

4. The vehicle system according to claim 1, wherein: The server is configured to send a notification to the user if the acoustic data matches one or more predetermined sound frequencies.

5. The vehicle system according to claim 1, wherein: The server is configured to send a notification to law enforcement if the acoustic data matches one or more predetermined sound frequencies.

6. The vehicle system according to claim 1, wherein: The server is configured to, if the acoustic data collected from the vehicle microphone matches a predetermined sound frequency, Connected to the user device.

7. The vehicle system of claim 1 further comprising a vehicle camera for collecting information related to the vehicle's surroundings.

8. The vehicle system according to claim 7, wherein: The server is also configured to collect data related to the vehicle's surroundings using the vehicle camera.

9. The vehicle system according to claim 8, wherein: The server is also configured to extract information related to the determined sound frequency and analyze the vehicle data and the data collected from the vehicle camera to determine whether any data matches the extracted information.

10. A vehicle comprising the vehicle system according to claim 1.

Images