Vehicle Function Control with Sensor-Based Validation

The data processing system addresses inefficiencies in network traffic by validating vehicle conditions and requests, reducing unnecessary network traffic and computational waste, thereby enhancing efficiency and response times.

JP7590512B2Active Publication Date: 2024-11-26GOOGLE LLC
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
JP2023134042
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2023-08-21
Publication Date
2024-11-26
Estimated Expiration
2037-10-03

AI Technical Summary

Technical Problem

Excessive network traffic data transmission between computing devices leads to inefficiencies, including delayed processing, complex data routing, and increased bandwidth usage, often due to invalid requests, which can degrade the quality of network responses.

Method used

A data processing system that utilizes a natural language processor to parse audio signals, validate vehicle conditions through sensor data, and generate action data structures based on request attributes, ensuring only valid requests are processed, thereby reducing unnecessary network traffic and computational waste.

Benefits of technology

The system enhances computational efficiency by preventing the processing of invalid requests, conserving bandwidth and power, and improving response times by validating vehicle conditions before executing commands.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007590512000001
    Figure 0007590512000001
  • Figure 0007590512000002
    Figure 0007590512000002
  • Figure 0007590512000003
    Figure 0007590512000003
Patent Text Reader

Abstract

To provide a data processing system to validate vehicular functions in a voice-activated computer network environment.SOLUTION: The data processing system can improve the efficiency of the network by discarding action data structures and requests that are invalid prior to transmission of the action data structures and requests over the network. The system can invalidate requests by comparing attributes of a vehicular state to attributes of a request state.SELECTED DRAWING: Figure 2
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to vehicle function control using sensor-based verification. [Background technology]

[0002] Excessive network transmissions, packet-based or otherwise, of network traffic data between computing devices may prevent the computing devices from properly processing the network traffic data, completing operations related to the network traffic data, or responding to the network traffic data in a timely manner. Excessive network transmissions of network traffic data may also complicate data routing or degrade the quality of responses if the responding computing device is over its processing capacity, thereby resulting in inefficient bandwidth utilization. A portion of the excessive network transmissions may include transmissions for requests that are not valid requests. Summary of the Invention [Means for solving the problem]

[0003] According to one aspect of the disclosure, a system for validating a vehicular function can include a data processing system. The system can include a natural language processor component executed by the data processing system. The natural language processor component can receive an input audio signal via an interface of the data processing system. The natural language processor component can parse the input audio signal to identify a request, a vehicle associated with the request, and a fulfillment interface associated with the request and the vehicle. The system can include a direct action application programming interface for generating a first action data structure based on the request. The system can include a validation engine for determining a vehicle state of a vehicle associated with the request based on a first set of sensor data. The vehicle state can include a first plurality of attributes. The validation engine can determine the request state based on the request and a second set of sensor data. The request state can include a second plurality of attributes. The validation engine can compare the second plurality of attributes of the request state to the first plurality of attributes of the vehicle state. The validation engine can send the first action data structure to a fulfillment interface to perform a function associated with the first action data structure based on one of the second plurality of attributes of the requested state matching one of the first plurality of attributes of the vehicle state.

[0004] According to one aspect of the disclosure, a method for validating a vehicle function may include receiving an input audio signal by a natural language processor component executed by a data processing system and via an interface of the data processing system. The method may include parsing the input audio signal by the natural language processor component to identify a request, a vehicle associated with the request, and a fulfillment interface associated with the request and the vehicle. The method may include generating a first action data structure by a direct action application programming interface and based on the request. The method may include determining, by a validation engine, a vehicle state of a vehicle associated with the request based on a first set of sensor data. The vehicle state may include a first plurality of attributes. The method may include determining, by the validation engine, a request state based on the request and a second set of sensor data. The request state may include a second plurality of attributes. The method may include comparing, by the validation engine, a second plurality of attributes of the request state to a first plurality of attributes of the vehicle state. The method may include sending, by the validation engine and based on a match of one of the second plurality of attributes of the requested state to one of the first plurality of attributes of the vehicle state, the first action data structure to a fulfillment interface for executing a function associated with the first action data structure.

[0005] These and other aspects and implementations are described in detail below. The above information and the following detailed description include illustrative examples of the various aspects and implementations and provide an overview or framework for understanding the nature and characteristics of the claimed aspects and implementations. The drawings provide illustration and a further understanding of the various aspects and implementations and are incorporated in and constitute a part of this specification.

[0006] The accompanying drawings are not drawn to scale. Like reference numbers and designations in the various drawings indicate like elements. For clarity, every component may not be labeled in every drawing. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 illustrates an exemplary system for authenticating and controlling vehicle functions using sensor-based verification. [Diagram 2] FIG. 2 shows a plan view of the vehicle and illustrates the interior cabin of the vehicle shown in FIG. 1. [Diagram 3] FIG. 2 illustrates an example method for verifying vehicle functionality using the example system shown in FIG. 1. [Figure 4] FIG. 1 is a block diagram of an exemplary computer system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] A more detailed description of various concepts related to, and implementations of, methods, apparatus, and systems for multi-mode transmission of packetized data in a voice activated data packet-based computer network environment follows below. The various concepts introduced above and described in more detail below may be implemented in any of a number of ways.

[0009] The present disclosure is generally directed to a data processing system for validating requests in packetized audio signals in a voice-activated computer network environment. The data processing system can improve the efficiency and effectiveness of auditory data packet transmissions over one or more computer networks, for example, by discarding or possibly invalidating requests (and their associated action data structures) that are not valid prior to transmission of the requests over the network. The solution can also improve computational efficiency by preventing the data processing system from performing further computational operations on action data structures that are not valid and should be discarded. By disabling the transmission of invalid requests, the system can reduce bandwidth utilization by not transmitting data packets carrying invalid action data structures over the network. Processing naturally spoken audio signals can be a computationally intensive task. By detecting potentially invalid requests, the system can reduce computational waste by allowing the system to skip, temporarily skip, discard, or modify potentially invalid requests.

[0010] Systems and methods described herein can include a data processing system that receives an audio input query, sometimes referred to as an audio input signal. The input audio signal can include a request to perform a function in a vehicle setting, such as opening a window or starting the car. From the input audio signal, the data processing system can identify the request. The system can generate an action data structure based on the audio input query. The action data structure can include attributes indicative of a requirement for fulfilling the request. The system can determine a vehicle state and attributes associated with the vehicle state. The system can compare and analyze attributes of the action data structure to attributes of the vehicle state to determine if the request should be fulfilled. If the attributes of the vehicle state and the attributes of the action data structure do not match, the system can invalidate or discard the action data structure (and the associated request) and prevent the fulfillment interface from fulfilling the request, which conserves network bandwidth, reduces processor utilization, and conserves power.

[0011] 1 illustrates an example system 100 for authenticating and controlling vehicle functions using sensor-based verification. The system 100 may include at least one data processing system 102, one or more client devices 128, and one or more vehicles 140.

[0012] The data processing system 102 can include an interface 104. The data processing system 102 can include a natural language processor (NLP) component 106 for parsing the audio-based input. The data processing system 102 can include an interface management component 108 for discovering and managing interfaces of other devices in the system 100. The data processing system 102 can include an audio signal generator component 110 for generating an audio-based signal. The data processing system 102 can include a direct action application programming interface (API) 112. The data processing system 102 can include a response selector component 114 for selecting a response to the audio-based input signal. The data processing system 102 can include a validation engine 116 for validating the audio-based input received by the data processing system 102. The data processing system 102 can include a data repository 118, in which the data processing system 102 can store parameters 120, policies 122, response data 124, and templates 126. The client device 128 and the vehicle 140 may include and execute instances of components of the data processing system 102. In some implementations, the client device 128 and the vehicle 140 may each include an instance of the data processing system 102.

[0013] The system 100 may also include one or more client devices 128. The client device 128 may include a sensor 130, a speaker 132, an interface 104, and a transducer 136. The client device 128 may execute an instance of the NLP component 106. The system 100 may also include one or more data providers 138. The system 100 may include one or more vehicles 140. The vehicle 140 may include a sensor 130, a speaker 132, an interface 104, and a transducer 136. The vehicle 140 may execute an instance of the NLP component 106. The components of the system 100 may communicate over a network 142. In some implementations, one or more of the client devices 128 may be located within an instance of the vehicle 140. For example, the client device 128 may be a mobile phone of a driver driving the vehicle 140. In some implementations, one or more of the client devices 128 may be remote with respect to the vehicle 140. For example, after a driver parks the vehicle 140 for work and exits, the driver's mobile phone is remote to the vehicle 140 .

[0014] Network 142 may include a computer network such as the Internet, a local area network, a wide area network, a metro area network, or other area network, an intranet, a satellite network, another computer network such as a voice or data mobile phone communication network, and combinations thereof. Network 142 may be used by data processing system 102, client device 128, and vehicle 140 to access information resources such as web pages, websites, domain names, uniform resource locators, or data providers 138. For example, data processing system 102 may access, via network 142, a data provider 138 that provides weather data for a particular location, such as a location associated with vehicle 140.

[0015] The network 142 may include, for example, a point-to-point network, a broadcast network, a wide area network, a local area network, a telecommunications network, a data communications network, a computer network, an ATM (asynchronous transfer mode) network, a SONET (synchronous optical network) network, an SDH (synchronous digital hierarchy) network, a wireless network or a wireline network, and combinations thereof. The network 142 may include a wireless link, such as an infrared channel or a satellite band. The topology of the network 142 may include a bus, a star, or a ring network topology. The network 142 may include a cellular network using any one or more protocols used to communicate between mobile devices, including Advanced Mobile Phone Protocol (AMPS), Time Division Multiple Access (TDMA), Code Division Multiple Access (CDMA), Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), or Universal Mobile Telecommunications System (UMTS). Different types of data may be transmitted over different protocols, or the same type of data may be transmitted over different protocols.

[0016] The client device 128 and the vehicle 140 each may include at least one logical device, such as a computing device having a processor for communicating with the data processing system 102 and with each other via the network 142. The client device 128 and the vehicle 140 may include an instance of any of the components described with respect to the data processing system 102. The client device 128 and the vehicle 140 may include an instance of the data processing system 102. The client device 128 may include a desktop computer, a laptop, a tablet computer, a personal digital assistant, a smartphone, a mobile device, a portable computer, a thin client computer, a virtual server, a speaker-based digital assistant, or other computing device. The vehicle 140 may include a manually or autonomous vehicle, a truck, a bus, an industrial vehicle, a motorcycle, or other transportation machine. The vehicle 140 may include one or more processors capable of executing an instance of the data processing system 102 or any of its components. The processor may be a component of the head unit of the vehicle.

[0017] The client device 128 and the vehicle 140 may include at least one sensor 130, at least one transducer 136, at least one audio driver, and at least one speaker 132. The sensor 130 may include a microphone or an audio input sensor. The sensor 130 may also include at least one of a GPS sensor, a proximity sensor, an ambient light sensor, a temperature sensor, a motion sensor, an accelerometer, or a gyroscope. The sensor may include an occupancy or weight sensor. The transducer 136 may convert the audio input into an electronic signal. The audio driver may include a script or program executed by one or more processors of the client device 128 or the vehicle 140 to control the speaker 132. The speaker 132 may render the audio signal by converting the electrical signal into audible waves.

[0018] The client device 128 and the vehicle 140 may be associated with an end user, who inputs a voice query as an input audio signal to the client device 128 or the vehicle 140 (via the sensor 130) and receives an audio output in the form of a computer-generated voice that may be provided from the data processing system 102. In response to the input audio signal, the client device 128 and the vehicle 140 may also receive an action data structure to perform a predetermined function or action. The interface 104 may receive data messages or provide data messages to the direct action API 112 of the data processing system 102 to enable communication between the components of the system 100. The client device 128 and the vehicle 140 may also include a user interface that enables a user to interact with the components of the system 100.

[0019] The data processing system 102 of the system may include at least one server having at least one processor. For example, the data processing system 102 may include multiple servers located in at least one data center or server farm. The data processing system 102 may determine a request and a trigger keyword associated with the request from the audio input signal. Based on the request and the trigger keyword, the data processing system 102 may generate or select response data. The response data may be audio-based or text-based. For example, the response data may include one or more audio files that, when rendered, provide an audio output or sound waves. The data in the response data may also be referred to as a content item. The response data may include other content (e.g., text, video, or image content) in addition to audio content.

[0020] The data processing system 102 may include multiple logically grouped servers, enabling distributed computing techniques. A logical group of servers may be referred to as a data center, a server farm, or a machine farm. The servers may be geographically distributed. A data center or machine farm may be managed as a single entity, or a machine farm may include multiple machine farms. The servers in each machine farm may be heterogeneous, with one or more of the servers or machines operating according to one or more types of operating system platforms. The data processing system 102 may include servers in a data center housed in one or more high density rack systems with associated storage systems, for example, located in an enterprise data center. In this manner, the data processing system 102 with integrated servers may improve system manageability, data security, system physical security, and system performance by placing the servers and high performance storage systems on a local high performance network. Centralization of all or part of the components of the data processing system 102, including the servers and storage systems, and combining them with advanced system management tools, allows for more efficient use of server resources, which conserves power and processing requirements and reduces bandwidth usage. Each of the components of the data processing system 102 may include at least one processing unit, server, virtual server, circuit, engine, agent, equipment, or other logic device such as a programmable logic array configured to communicate with the data repository 118 and other computing devices.

[0021] The data processing system 102 may include a data repository 118. The data repository 118 may include one or more local or distributed databases and may include a database management system. The data repository 118 may include computer data storage or memory and may store, among other data, one or more parameters 120, one or more policies 122, response data 124, and templates 126. The parameters 120, policies 122, and templates 126 may include information such as rules regarding voice-based sessions between the client device 128, the data processing system 102, and the vehicle 140. The response data 124 may include content items or associated metadata about audio output, as well as input audio messages that may be part of one or more communication sessions with the client device 128.

[0022] Applications, scripts, programs, or other components associated with data processing system 102 may be installed on client device 128 or vehicle 140. The applications may enable client device 128 or vehicle 140 to communicate input audio signals (and other data) to interface 104 of data processing system 102. The applications may enable client device 128 and vehicle 140 to drive components of client device 128 and vehicle 140 to render output audio signals.

[0023] The NLP component 106 of the data processing system can receive the input audio signal. The data processing system 102 can receive the input audio signal from a client device 128 or a vehicle 140. A first device can execute the NLP component 106, and the NLP component 106 can receive the input audio signal from a second device. For example, the vehicle 140 can execute the NLP component 106, and the input audio signal can be received from the client device 128.

[0024] The NLP component 106 can convert the input audio signal to recognized text by comparing the input audio signal to a stored representative set of audio waveforms and selecting the closest match. The representative waveforms can be generated across a large set of input audio signals. Once the input audio signal is converted to recognized text, the NLP component 106 can match the text to actions or words associated with the output audio signal, for example, via a learning phase.

[0025] From the input audio signal, the NLP component 106 can identify at least one request or at least one trigger or hot keyword corresponding to the request. The request can indicate the intent or content of the input audio signal. The trigger keyword can indicate a type of action that may be taken. For example, the NLP component 106 can parse the input audio signal to identify at least one request to open a vehicle window or to skip to the next audio file in a music playlist. The trigger keyword can include at least one word, phrase, root or partial word, or derived word that indicates an action to be taken. For example, the trigger keyword can be "go" or "ok" or "hey".

[0026] The answer selector component 114 can obtain information from a data repository 118, where the information may be stored as part of the answer data 124. The answer selector component 114 can query the data repository 118 to select or possibly identify answer phrases or content items from the answer data 124, for example.

[0027] The audio signal generator component 110 can generate or possibly obtain an output signal that includes the content item. The data processing system 102 can execute the audio signal generator component 110 to generate or create an output signal that corresponds to the content item or the request. For example, when a request is fulfilled, the signal generator component 110 can generate an audio output signal that includes the phrase "The action was completed."

[0028] The interface 104 may be a data interface or a network interface that allows the components of the system 100 to communicate with each other. The interface 104 of the data processing system 102 may provide or transmit one or more data packets including an action data structure, an audio signal, or other data to the client device 128 or the vehicle 140 via the network 142. For example, the data processing system 102 may provide an output signal to the client device 128 from the data repository 118 or from the audio signal generator 110. The data processing system 102 may also instruct the client device 128 or the vehicle 140 via the data packet transmission to perform a function indicated in the action data structure. The output signal may be obtained, generated, converted into one or more data packets (or other communication protocols), or transmitted as one or more data packets (or other communication protocols) from the data processing system 102 (or other computing device) to the client device 128 or the vehicle 140.

[0029] The direct action API 112 of the data processing system 102 can generate an action data structure based on the request, for example. The action data structure can include data or instructions for the execution of a specified action to fulfill the request. In some implementations, the action data structure can be a JSON-formatted data structure or an XML-formatted data structure.

[0030] Depending on the action specified in the request, the direct action API 112 can execute code or dialogue scripts that identify parameters needed to fulfill the request. An action data structure can be generated in response to the request. The action data structure can be included in a message sent to or received by the client device 128 or the vehicle 140. Based on the request parsed by the NLP component 106, the direct action API 112 can determine to which of the client device 128 or the vehicle 140 the message should be sent. For example, if the input audio signal includes "open a window," the NLP component 106 can identify the action word "open a window." The direct action API 112 can package the request into an action data structure for transmission to the vehicle 140. The direct action API 112 can access the vehicle ID from the response data 124 to determine which vehicle is associated with the user that generated the request. Once received, the vehicle 140 can process the action data structure and open a window of the vehicle.

[0031] The action data structure may include information for completing a request. For example, the action data structure may be an XML or JSON formatted data structure that includes attributes used in completing or possibly fulfilling the request. The attributes may include the location of the vehicle 140, the location of the client device 128, the permission level of a user associated with the client device 128, a vehicle identifier, an interface identifier, a vehicle state, or a request state. In some implementations, the request state includes one or more attributes that should be satisfied before the action is fulfilled. For example, the request state for the request "Ok, change the song" may have an attribute {requester:[authorized, passenger]} indicating that the requester should be an explicitly authorized user or a passenger in the vehicle.

[0032] The direct action API 112 can retrieve templates 126 from the repository 118 to determine which fields or attributes to include in the action data structure. The direct action API 112 can determine the required parameters and package that information in the action data structure. The direct action API 112 can retrieve content from the repository 118 to obtain information for attributes of the data structure.

[0033] The direct action API 112 can populate the fields with data from the input audio signal. The direct action API 112 can also populate the fields with data from the data provider 138, the client device 128, or the vehicle 140. In some implementations, the direct action API 112 can prompt the user for additional information when populating the fields. The templates 126 can be standardized for different types of actions, such as playing a media file through the vehicle's head unit, responding to a message, and performing an in-car function. In some implementations, the action data structure can be generated by the direct action API 112 first executed by the remote data processing system 102. The remote data processing system 102 can send the action data structure to the data processing system 102 of the vehicle 140, which can add fields and attributes to the action data structure.

[0034] The direct action API 112 can retrieve response data 124 (or parameters 120 or policies 122) from the data repository 118, as well as data received with end user consent from the client device 128, to determine location, time, user account, logistical or other information, for example, to reserve a car from a car sharing service. The response data 124 (or parameters 120 or policies 122) can be included in an action data structure. When content included in the action data structure includes end user data used for authentication, the data can be passed through a hashing function before being stored in the data repository 118.

[0035] Data processing system 102 (or its hardware, processor, or memory components) may include, interface with, or otherwise communicate with validation engine 116. Validation engine 116 may verify or otherwise determine whether the action and function associated with the action data structure should be executed or performed. When verified, validation engine 116 may allow the request to be performed. Validation engine 116 may determine whether the requester has the authority for the action to be performed, whether it is safe to perform the action, or whether another function should be performed instead of the requested function or action.

[0036] For example, in response to receiving an input audio signal of "Ok, open the trunk," the verification engine 116 may determine that the vehicle 140 is driving along a road and it is not safe to open the trunk. In another example, in response to receiving an input audio signal of "Ok, next song," the verification engine 116 may determine that the user has the authority to change songs and skip to the next song. The verification engine 116 may be executed by the client device 128, the vehicle 140, or components thereof. For example, the vehicle 140 may execute a local instance of the verification engine 116 when the vehicle 140. In some implementations, the vehicle 140 may offload computational work to the remote verification engine 116 when the verification engine 116 has a connection to the network 142 and process requests using the local instance of the verification engine 116 when the vehicle 140 does not have a connection to the network 142.

[0037] The validation engine 116 may validate the action based on data from external sensor data, such as data received from a data provider 138, sensors 130 of the client device 128, and sensors 130 of the vehicle 140. The validation engine 116 may also validate the action based on administrative privileges of the user who generated the input audio signal. The validation engine 116 may validate the action based on a vehicle state of the vehicle 140.

[0038] The validation engine 116 may determine a vehicle state of the vehicle 140 based on a set of sensor data. The vehicle state may indicate a current state of the vehicle 140 in which the request should be fulfilled. The vehicle state may indicate whether the vehicle 140 is moving, parked, or idling. The vehicle state may indicate the speed of the vehicle 140. The validation engine 116 may use a combination of sensor data to determine the vehicle state. The combination of sensor data may be used to distinguish different states that may have similar conditions. For example, different states of stopped at a red light, stopped in a parking lot, stopped in traffic, and parked in a parking lot may have some similar conditions (e.g., a vehicle speed of 0 mph), but different functions may be allowed in each of the different states. For example, it may be safe to open the trunk when the vehicle 140 is parked in a parking lot, but not safe to open the trunk when the vehicle 140 is stopped at a red light. The vehicle state may include multiple sub-states, such as a moving state, an external environment state, an internal environment state, and a device state.

[0039] The travel sub-state may include attributes such as current and average speed, current and average acceleration, drive mode status, and gear status. The exterior environment sub-state may include attributes such as exterior weather conditions (e.g., rain, sunshine, or snow), exterior temperature, exterior weather condition forecast, and exterior temperature forecast. The interior environment sub-state may include attributes such as interior temperature and interior zone temperature. The device sub-state may include attributes such as HVAC status (e.g., air conditioning on, air conditioning off, heat on, etc.), window status, lock status, truck status, door ajar status, sunroof status, heated seats status, heated steering wheel status, headlight status, and interior light status.

[0040] The validation engine 116 can determine the vehicle state based on a set of sensor data, which can include sensor data from the client device's sensors 130, sensor data from the vehicle's sensors 130, data from a data provider 138, or any combination thereof. Some of the attributes within the vehicle state can be determined from a single sensor (e.g., door ajar status) and other attributes can be determined from a combination of sensor signals (e.g., movement status can be determined from a combination of speed data and gear data).

[0041] The sensor data from the client device 128 may include location data determined by cellular tower triangulation or a GPS receiver in the client device 128, speed data determined by integrating location data, acceleration data, account data, authorization data, or any combination thereof. The sensor data may include data from one or more data providers 138. The data providers 138 may be internet-based data repositories. The data providers 138 may make their data available, for example, through an API. The data providers 138 may provide weather data, location data, event data, location consideration data, forecast data, or any combination thereof to the validation engine 116.

[0042] The validation engine 116 can determine the request state based on the request parsed from the input audio file. The validation engine 116 can determine the request state based on the second set of sensor data. The data of the first and second sets of sensor data can be received as update data signals, vectors, data arrays, scalar values, or other formats of data. The request state can include attributes under which the action of the request can be performed or which attributes are required for the request to be fulfilled. The request state can be a component of or derived from an action data structure. For example, a request "OK, open the trunk" can include a request state with attributes indicating that the trunk can only be opened when in a parked condition and when requested by the driver or an authorized passenger. For example, the request state can be {mobile state: parked, requester: authorized passenger}.

[0043] The second set of sensor data for determining the requested state may include sensor data from the first set of sensor data that the validation engine 116 used to determine the vehicle state. The second set of data may also include the location of the client device 128 (or a user associated therewith) within the vehicle 140.

[0044] The validation engine 116 can compare the request state to the vehicle state to determine whether the action of the action data structure should be executed, implemented, or further processed. For example, the vehicle state may indicate that the vehicle 140 is parked in a parking lot, the windows are closed, and the external condition is rainy (e.g., {moving state: parked, windows: closed, external condition: rain}). For example, if the input audio signal is "OK, open the windows," the validation engine 116 may determine that the request state for the request requires that the request comes from an authorized user, the driving attribute can be driving or parked, and the weather attribute must be other than rain (e.g., {requester: authorized, moving state: [driving, parked], external condition: [sunny, overcast, partly cloudy]}). Comparing the vehicle state to the request state, the validation engine 116 can determine that the external condition attribute of the vehicle state does not match (or is not an element of) the external condition attribute of the request state. Because the conditions do not match, the validation engine 116 may determine that the request should not be fulfilled because it is raining outside the vehicle 140 and windows should not be open when it is raining.

[0045] For example, when the validation engine 116 determines that a function should not be performed, the validation engine 116 can generate a request for confirmation that is presented to the user via the client device 128 or the interface 104 of the vehicle 140. The request for confirmation can ask the user whether they want to override the validation engine's decision not to perform the request. When the validation engine 116 determines that a function should not be performed, the validation engine 116 can offer to modify the action data structure. Continuing with the example above, the validation engine 116 can generate an audio-based prompt asking whether the user wants to tile the sunroof.

[0046] FIG. 2 shows a plan view of the vehicle 140 and illustrates the interior cabin of the vehicle 140. The interior cabin of the vehicle 140 can include a number of seats 200. Each of the seats 200 can include one or more sensors 130. The sensors 130 can be weight or occupancy sensors. The interior cabin of the vehicle 140 can include a number of speakers 132. The vehicle 140 can also include a head unit 202, which can execute one or more of the components described with respect to the data processing system 102 or the vehicle 140. For example, with user consent, the speaker 132 can generate a sub-audible tone 204. The sub-audible tone 204 can be a tone rendered by the speaker 132 that is not within a normal frequency range detectable by the user. The system 100 can use an audible tone in place of the sub-audible tone 204. The head unit 202 can include all of the components of the data processing system in the vehicle 140. For example, the head unit 202 may include a data processing system 102 configured as a digital audio assistant application.

[0047] The validation engine 116 can determine a relative position of the client device 128 within the vehicle 140. The position of the client device 128 can be incorporated into one or more attributes of the requested state or vehicle state. The position or location of the client device 128 is a relative position and can include different levels of resolution. For example, the relative location can be an area relative to one of the seats 200 (e.g., client device 128(1) is relative to the passenger seat 200), an area relative to a different zone (e.g., client devices 128(1) and 128(2) in a forward zone and no client device 128 in a rear zone).

[0048] The relative location of the client device 128 within the vehicle 140 may be determined using the inaudible tones 204. The relative location of the client device 128 within the vehicle 140 may also be determined based on sensor data from the sensors 130. The relative location of the client device 128 within the vehicle 140 may be determined based on a combination of the inaudible tones and sensor data from the sensors 130.

[0049] The verification engine 116 can determine the relative location of the client device 128 by determining the time difference between the transmission of the inaudible tone 204 and the receipt of the inaudible tone 204 by the client device 128. For example, the head unit 202 can cause one of the speakers 132 to generate the inaudible tone 204. The speaker 132 can generate the inaudible tone 204 at a first time. A client device 128 that is relatively closer to the speaker 132 that generated the inaudible tone 204 will receive the inaudible tone 204 before a client device 128 that is relatively farther from the speaker 132 that generated the inaudible tone 204. For example, as shown in FIG. 2, client device 128(1) will receive the inaudible tone 204 before client device 128(2).

[0050] When the client device 128 receives or, as the case may be, detects the inaudible tone 204, the client device 128 can generate an acknowledgment that the client device 128 sends to the head unit 202 or the data processing system 102. The acknowledgment can indicate a second time that the speaker 132 received the inaudible tone 204. The acknowledgment can be sent to the data processing system 102 via a cellular network, a WiFi network local to the vehicle 140, or via Bluetooth. The verification engine 116 can determine the length of time between the generation of the inaudible tone 204 and when the client device 128 received the inaudible tone 204 (e.g., the time difference between the first and second times in the example above). Different speakers 132 can generate different inaudible tones 204 at different times to allow the data processing system 102 to triangulate the location of the client device 128 within the vehicle 140.

[0051] The verification engine 116 can determine the location of the client device 128 based on data from the sensor 130. The sensor 130 can be a weight sensor that can determine whether the seat 200 is occupied. For example, the verification engine 116 can determine that the seat 200 is occupied when the sensor 130 registers a weight above a predetermined threshold. In some implementations, the verification engine 116 can use the inaudible tone 204 to determine that the client device 128 is near one of the seats 200. The verification engine 116 can verify that a user is associated with the client device 128 based on receiving sensor data that the sensor 130 of the determined seat 200 is detecting a weight. The sensor data can be used to prevent a user from accessing features that are only available to some zones or users. For example, text messaging can be disabled for a client device 128 associated with a driver but enabled for client devices 128 associated with passengers in other seats 200 of the vehicle 140. If the driver leans forward and moves the driver's client device 128 into the passenger seat 200 area, based on the inaudible tone 204, the verification engine 116 may determine that the client device 128 should be associated with the passenger seat 200 and enabled to generate requests to use a text messaging program. The verification engine 116 may receive weight data of 0 pounds from the sensor 130 in the passenger seat 200 and determine that the user is not actually in the passenger seat 200, and may prevent the driver's client device 128 from accessing the text messaging functionality. In some implementations, the verification engine 116 may update the relative location of the client device 128 in the vehicle 140 only after detecting the opening and closing of the vehicle 140 doors.

[0052] 3 illustrates an example method 300 for verifying vehicle functionality. The method 300 may include receiving an audio signal (block 302). The method 300 may include parsing the audio signal (block 304). The method 300 may include generating an action data structure (block 306). The method 300 may include determining a vehicle state (block 308). The method 300 may include determining a request state (block 310). The method 300 may include transmitting the action data structure (block 312).

[0053] The method 300 may include receiving an input audio signal (block 302). A data processing system may receive the input audio signal. For example, an NLP component executed by the data processing system may receive the input audio signal. The data processing system (and the NLP component) may be a component of a client device, a vehicle, or possibly executed by the client device, the vehicle, or may be a standalone device. A sensor, such as a microphone, in the client device or vehicle may detect the input audio signal, and the respective client device or vehicle may then transmit the input audio signal to the data processing system. For example, an application executed on the client device may detect that a user has spoken, "Ok, open the sunroof." The detected speech may be encoded into an input audio signal and transmitted to the NLP component of the data processing system or the vehicle.

[0054] The method 300 may include parsing an input audio signal (block 304). The NLP component may parse the input audio signal to identify a request in the input audio signal. The NLP component may identify a vehicle associated with the request. The NLP component may identify a fulfillment interface associated with the request and the vehicle. The fulfillment interface may be an interface of one of a client device or a vehicle that will execute an action data structure to fulfill the request of the input audio signal.

[0055] In the above example, the request may be to open the sunroof. In this example, the fulfillment interface may be an interface of a vehicle that includes a sunroof. In some implementations, the vehicle may be explicitly indicated in the input audio signal. For example, a user may assign a nickname to the user's vehicle (e.g., the user may name the user's red Toyota Camry the "red car"). When explicitly indicated, the input audio signal may be "OK, open the sunroof of my red car." In some implementations, the data processing system may determine which vehicles associated with the user's account are capable of fulfilling the action. For example, the user's red car may include a sunroof and the user's blue car may not include a sunroof. After receiving the input audio signal "Ok, open the sunroof," the data processing system may automatically select the user's red car. In some implementations, the data processing system may query the user for vehicle identification.

[0056] The method 300 may include generating an action data structure (block 306). The direct action API may generate a data structure that may be sent and processed by the client device or the vehicle to fulfill a request for an input audio signal. For example, continuing with the above example, the direct action API may generate a first action data structure for opening the sunroof of the user's car. The direct action API may generate the action data structure using a template retrieved from a data repository of the data processing system. The action data structure may include fields that are used to fulfill the request. For example, in the case of a request to open the sunroof, the action data structure may include a field (or attribute) for the vehicle ID to which the action data structure should be sent if the action data structure is approved.

[0057] The method 300 may include a step of determining a vehicle state (block 308). The vehicle state may include multiple attributes. The attributes may identify a current status of the vehicle. The vehicle state may include multiple sub-states, such as a moving state, an external environment state, an internal environment state, and a device state. The moving sub-state may include attributes such as a current and average speed, a current and average acceleration, a drive mode status, and a gear status. The external environment sub-state may include attributes such as an external weather condition (e.g., rain, sunshine, or snow), an external temperature, an external weather condition forecast, and an external temperature forecast. The internal environment sub-state may include attributes such as an internal temperature and an internal zone temperature. The device sub-state may include attributes such as an HVAC status (e.g., air conditioning on, air conditioning off, heat on, etc.), a window status, a lock status, a track status, a door ajar status, a sunroof status, a heated seat status, a heated steering wheel status, and a light status. The validation engine may determine the vehicle state using sensor data generated by the vehicle, the client device, or retrieved from a data provider. In some implementations, a validation engine running at one location can determine the vehicle state and transmit the vehicle state to a second validation engine. For example, a validation engine running at a vehicle can determine all or a portion of the vehicle state and then transmit the vehicle state to a validation engine running at a remote data processing system.

[0058] The method 300 may include a step of determining a request state (block 310). The request state may include multiple attributes. The attributes of the request state may indicate one or more conditions that should be performed or satisfied before the action associated with the action data structure is completed. The attributes (and the request state) may be components of the action data structure or may be derived from the action data structure. For example, the action data structure for a request to open the sunroof may include a field indicating under which external environmental conditions the sunroof should be opened. The validation engine may incorporate the external environmental conditions into the attributes of the request state. The request state may include user attributes. The user attributes may include a location of a client device associated with the user, or a permission state. The location may be a location in a vehicle or a geographic location (e.g., the client device is located in the user's office).

[0059] The verification engine can use the non-audible tone to determine the location of the user along with the vehicle. For example, the verification engine can cause one or more speakers in the vehicle to generate a non-audible tone. The verification engine can determine the relative location of the client device in the vehicle determining the delay between when the non-audible tone is played on the speaker and when the client device detects the non-audible tone. The relative location of the user (and the client device) in the vehicle can be determined based on the readings of a weight or occupancy sensor placed in a seat of the vehicle.

[0060] The method 300 may transmit the action data structure (block 312). The action data structure may be transmitted to a fulfillment interface configured to execute a function associated with the action data structure. In some implementations, the action data structure is transmitted to the fulfillment interface only if one or more attributes of the requested state match one or more attributes of the vehicle state. For example, the validation engine may compare attributes of the requested state, which may indicate a requirement for executing a function, with attributes of the vehicle state. For example, a request to open the sunroof may have an external environment attribute requirement {External environment: [Sunny, Overcast, Partly cloudy]}. The vehicle state may include, among other attributes, {External environment: Rain}. In this example, the validation engine may determine that the sunroof should not be opened because the requested state attribute is not an element of the vehicle state attribute and it is raining at the location of the vehicle. The validation engine may modify the action data structure when the validation engine determines that there is no match between the requested state attribute and the vehicle state attribute. For example, the validation engine may modify the action data structure to turn on the vehicle's HVAC in response to determining that the sunroof should not be opened because it is raining. The validation engine may generate a prompt to ask the user whether the user wants the action data structure to be changed. For example, the data processing system may generate a prompt saying, "I did not open the sunroof because it is raining. Would you like me to turn on the AC instead?"

[0061] The validation engine may determine that there is no match between the request state attributes and the vehicle state attributes. The data processing system may generate a notification to inform the user that the request was not completed. The notification may include the reason why the request was not completed. In some implementations, if the validation engine does not find a match, the validation engine discards the action data structure and does not send the action data structure to the fulfillment interface.

[0062] For example, when the validation engine determines that there is a match between the request state attributes and the vehicle state attributes, the validation engine can send the action data structure to a fulfillment interface to execute a function associated with the action data structure. For example, to open the sunroof, the action data structure can be sent to a vehicle's head unit, which can execute an application to control a motor that controls the position of the sunroof. In some implementations, the data processing system can send a confirmation that the function has been completed. For example, a text-based or audio-based output signal can be sent to the client device to indicate that the sunroof has been opened.

[0063] 4 is a block diagram of an exemplary computer system 400. The computer system or computing device 400 may include or be used to implement system 100, or components of system 100, such as data processing system 102. The computing system 400 includes a bus 405 or other communication components for communicating information and a processor 410 or processing circuitry coupled to the bus 405 for processing information. The computing system 400 may also include one or more processors 410 or processing circuits coupled to the bus for processing information. The computing system 400 also includes a main memory 415, such as a random access memory (RAM) or other dynamic storage device, coupled to the bus 405 for storing information and instructions to be executed by the processor 410. The main memory 415 may be or include the data repository 118. The main memory 415 may also be used to store location information, temporary variables, or other intermediate information during execution of instructions by the processor 410. Computing system 400 may further include a read only memory (ROM) 420 or other static storage device coupled to bus 405 for storing static information and instructions for processor 410. A storage device 425, such as a solid state device, magnetic disk, or optical disk, may be coupled to bus 405 for persistently storing information and instructions. Storage device 425 may include or be part of data repository 118.

[0064] The computing system 400 may be coupled via bus 405 to a display 435, such as a liquid crystal display or active matrix display, for displaying information to a user. An input device 430, such as a keyboard including alphanumeric and other keys, may be coupled to the bus 405 for communicating information and command selections to the processor 410. The input device 430 may include a touch screen display 435. The input device 430 may also include a cursor control, such as a mouse, trackball, or cursor direction keys, for communicating directional information and command selections to the processor 410 and for controlling cursor movement on the display 435. The display 435 may be part of, for example, the data processing system 102, the client computing device 128, or other components of FIG. 1.

[0065] The processes, systems, and methods described herein may be implemented by the computing system 400 in response to the processor 410 executing an arrangement of instructions contained in the main memory 415. Such instructions may be read into the main memory 415 from another computer-readable medium, such as the storage device 425. Execution of the arrangement of instructions contained in the main memory 415 causes the computing system 400 to perform the exemplary processes described herein. One or more processors in a multi-processing arrangement may also be employed to execute the instructions contained in the main memory 415. Hardwired circuitry may be used in place of or in combination with software instructions with the systems and methods described herein. The systems and methods described herein are not limited to any specific combination of hardware circuitry and software.

[0066] Although an exemplary computing system has been described in FIG. 4, the subject matter including the operations described herein may be implemented in other types of digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed herein and their structural equivalents, or in combinations of one or more of them.

[0067] In situations where the systems described herein may collect or utilize personal information about a user, the user may be given the opportunity to control whether a program or feature may collect personal information (e.g., information about the user's social network, social activities, or activities, the user's preferences, or the user's location) or to control whether or how to receive content from a content server or other data processing system that may be relevant to the user. Additionally, certain data may be anonymized in one or more ways before it is stored or used, such that personally identifiable information is removed when generating parameters. For example, the user's identity may be anonymized such that personally identifiable information cannot be determined about the user, or the user's geographic location from which location information is obtained may be generalized (such as to the city level, zip code level, or state level) such that the user's specific location cannot be determined. Thus, the user may control how information is collected about the user and used by the content server.

[0068] The subject matter and operations described herein may be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed herein and their structural equivalents, or in a combination of one or more of them. The subject matter described herein may be implemented as one or more computer programs, e.g., one or more circuits of computer program instructions, encoded on one or more computer storage media for execution by or to control the operation of a data processing device. Alternatively or additionally, the program instructions may be encoded on an artificially generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, generated to encode information for transmission to a suitable receiver device for execution by a data processing device. The computer storage medium may be or be included in a computer readable storage device, a computer readable storage substrate, a random or serial access memory array or device, or a combination of one or more of them. The computer storage medium is not a propagating signal, and the computer storage medium may be a source or destination of computer program instructions encoded in an artificially generated propagating signal. A computer storage medium may also be, or be contained within, one or more separate components or media (e.g., multiple CDs, disks, or other storage devices). The operations described herein may be implemented as operations performed by a data processing apparatus on data stored in one or more computer-readable storage devices or received from other sources.

[0069] The terms "data processing system", "computing device", "component", or "data processing apparatus" encompass various apparatuses, devices, and machines for processing data, including, by way of example, a programmable processor, a computer, a system on a chip, or a combination of the above. An apparatus may include special purpose logic circuitry, e.g., a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC). In addition to hardware, an apparatus may also include code that creates an execution environment for the computer program in question, e.g., code that constitutes a processor firmware, a protocol stack, a database management system, an operating system, a cross-platform runtime environment, a virtual machine, or one or more combinations thereof. The apparatus and execution environments may implement a variety of different computing model infrastructures, such as web services, distributed computing, and grid computing infrastructures. The components of system 100 may include or share one or more data processing apparatuses, systems, computing devices, or processors.

[0070] A computer program (also known as a program, software, software application, app, script, or code) may be written in any form of programming language, including compiled or interpreted languages, declarative or procedural languages, and may be deployed in any form, such as a standalone program or as a module, component, subroutine, object, or other unit suitable for use in a computing environment. A computer program may correspond to a file in a file system. A computer program may be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program, or in multiple cooperating files (e.g., files that store one or more modules, subprograms, or portions of code). A computer program may be deployed to run on one computer or to run on multiple computers located at one site or distributed across multiple sites and interconnected by a communication network.

[0071] The processes and logic flows described herein may be implemented by one or more programmable processors (e.g., components of data processing system 102) executing one or more computer programs to perform actions by operating on input data and generating output. The processes and logic flows may also be implemented by, and the apparatus may be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit). Devices suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including, by way of example, semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices, magnetic disks, e.g., internal hard disks or removable disks, magneto-optical disks, and CD ROM and DVD-ROM disks. The processor and memory may be supplemented by, or incorporated in, special purpose logic circuitry.

[0072] The subject matter described herein may be implemented in a computing system that includes back-end components, e.g., as a data server, or includes middleware components, e.g., an application server, or includes front-end components, e.g., a client computer having a graphical user interface or web browser through which a user can interact with an implementation of the subject matter described herein, or includes a combination of one or more such back-end, middleware, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include local area networks ("LANs") and wide area networks ("WANs"), internetworks (e.g., the Internet), and peer-to-peer networks (e.g., ad-hoc peer-to-peer networks).

[0073] A computing system such as system 100 or system 400 may include clients and servers. Clients and servers are generally remote from one another and typically interact through a communications network (e.g., network 142). The relationship of clients and servers arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. In some implementations, the server transmits data (e.g., data packets representing content items) to the client devices (e.g., for the purpose of displaying the data to and receiving user input from a user interacting with the client device). Data generated at the client device (e.g., a result of a user interaction) may be received at the server from the client device (e.g., received by data processing system 102 from client computing device 128 or vehicle 140).

[0074] Although operations are shown in a particular order in the figures, such operations are not required to be performed in the particular order shown, or in sequential order, and not all illustrated operations are required to be performed. Actions described herein may be performed in different orders.

[0075] The separation of various system components may not require separation in all implementations, and the described program components may be included in a single hardware or software product. For example, the NLP component 106 and the validation engine 116 may be part of a single component, app, or program, or a logical device having one or more processing circuits, or one or more servers of the data processing system 102.

[0076] Having now described several exemplary implementations, it is clear that the above are presented by way of example, not limitation. In particular, while many of the examples presented herein involve specific combinations of method acts or system elements, those acts and those elements may be combined in other ways to achieve the same purpose. Acts, elements, and features described with respect to one implementation are not excluded from similar roles in other implementations or implementations.

[0077] The phraseology and terminology used herein are for purposes of description and should not be considered limiting. The use herein of "including," "comprising," "having," "containing," "involving," "characterized by," "characterized in that," and variations thereof are intended to encompass the items listed thereafter, equivalents thereof, and additional items, as well as alternative implementations consisting only of the items listed thereafter. In one implementation, the systems and methods described herein consist of one, each combination of two or more of the described elements, acts, or components, or all of them.

[0078] References herein to implementations or elements or acts of systems and methods referred to in the singular may also encompass implementations that include a plurality of those elements, and references herein to implementations or elements or acts in the plural may also encompass implementations that include only a single element. References in the singular or plural do not limit the systems or methods of the present disclosure, their components, acts, or elements, to a single configuration or to multiple configurations. References to any act or element that is based on any information, act, or element may include implementations in which the act or element is based at least in part on any information, act, or element.

[0079] Any implementation disclosed herein may be combined with any other implementation or embodiment, and references to "an implementation," "some implementations," "one implementation," etc. are not necessarily mutually exclusive and indicate that a particular feature, structure, or characteristic described with respect to an implementation may be included in at least one implementation or embodiment. Such terms as used herein do not necessarily all refer to the same implementation. Any implementation may be combined with any other implementation, inclusively or exclusively, in any manner in accordance with the aspects and implementations disclosed herein.

[0080] References to "or" may be interpreted as inclusive, such that any term described using "or" may refer to either a single term, more than one term, or all of the terms described. A reference to "at least one of 'A' and 'B'" can include 'A' only, 'B' only, and both 'A' and 'B'. Such references used with "comprising" or other open terminology can include additional items.

[0081] When a reference sign follows a technical feature in a drawing, the detailed description, or any claim, the reference sign is included to enhance the comprehension of the drawing, the detailed description, and the claims, and thus, neither the reference signs nor their absence have the effect of limiting the scope of the claim element.

[0082] The systems and methods described herein may be embodied in other specific forms without departing from their characteristics. The above implementations are illustrative rather than limiting of the systems and methods described herein. The scope of the systems and methods described herein is therefore indicated by the appended claims, rather than the above description, and all changes that come within the meaning and range of equivalence of the claims are intended to be embraced therein. [Explanation of symbols]

[0083] 100 Systems 102 Data Processing System 104 Interface 106 NLP Components 108 Interface Management Components 110 Audio Signal Generator Components 112 Direct Action API 114 Response Selector Components 116 Verification Engine 118 Data Repositories 120 Parameters 122 Policy 124 Response Data 126 Templates 128 client devices 128(1) client devices 128(2) client devices 130 Sensors 132 speakers 136 Transducer 138 Data Providers 140 vehicles 142 Network 200 seats 202 Head Unit 204 Inaudible Tone 400 Computer Systems 405 Bus 410 Processor 415 Main Memory 420 Read-only memory 425 Storage Device 430 Input Devices 435 Display

Claims

1. 1. A method executed by one or more processors, comprising: receiving a user input from a user located within a vehicle, the vehicle including a plurality of zones; identifying a request to perform a vehicle function based on processing the user input; generating an action data structure based on the request to perform the vehicle function, the action data structure including actions associated with the request to perform the vehicle function; determining a given zone among the plurality of zones of the vehicle in which a client device associated with the user who provided the user input is located; determining whether the user is authorized to cause performance of the vehicle function based on the given zone in which the client device associated with the user who provided the user input is located; in response to determining that the user is authorized to cause performance of the vehicle function based on the given zone in which the client device associated with the user who provided the user input is located, causing the vehicle function to be executed based on the action data structure; in response to determining that the user is not authorized to cause performance of the vehicle function based on the given zone in which the client device associated with the user who provided the user input is located; ceasing to cause the vehicle function to be performed based on the action data structure; modifying the action data structure to include an alternative action based on a determination to cease causing the vehicle function to be performed based on the action data structure; causing the alternative actions included in the modified action data structure to be provided via the client device or a head unit of the vehicle for presentation to a user who provided the user input; A method comprising:

2. A method executed by one or more processors, comprising: receiving a user input from a user located within a vehicle, the vehicle including a plurality of zones; identifying a request to perform a vehicle function based on processing the user input; generating an action data structure based on the request to perform the vehicle function, the action data structure including actions associated with the request to perform the vehicle function; determining, from among the plurality of zones of the vehicle, a given zone in which a client device associated with the user who provided the user input is located, the step of determining the given zone in which the client device associated with the user who provided the user input is located comprises: causing a first speaker element of the vehicle to audibly emit a first inaudible tone, the first inaudible tone, when audibly emitted, to transmit to the client device: detecting the first inaudible tone via a microphone element of the client device; causing the client device to transmit a first acknowledgment to a head unit of the vehicle indicating that the client device has detected the first inaudible tone; determining the given zone in which the client device associated with the user who provided the user input is located, the given zone being based on the first notification; determining whether the user is authorized to cause performance of the vehicle function based on the given zone in which the client device associated with the user who provided the user input is located; in response to determining that the user is authorized to cause performance of the vehicle function based on the given zone in which the client device associated with the user who provided the user input is located, causing the vehicle function to be executed based on the action data structure; A method comprising:

3. determining the given zone in which the client device associated with the user who provided the user input is located, causing a second speaker element of the vehicle to audibly emit a second inaudible tone, the second inaudible tone, when audibly emitted, to communicate to the client device of the user: detecting the second inaudible tone via the microphone element of the client device; causing the client device to transmit a second acknowledgment to the head unit of the vehicle indicating that the client device detected the second inaudible tone. The method further includes the steps of: The method of claim 2 , wherein determining the given zone in which the client device associated with the user who provided the user input is located is further based on the second notification.

4. 4. The method of claim 3, wherein the first inaudible tone is audibly emitted at a first time and the second inaudible tone is audibly emitted at a second time, the second time being different from the first time.

5. A method executed by one or more processors, comprising: receiving a user input from a user located within a vehicle, the vehicle including a plurality of zones; identifying a request to perform a vehicle function based on processing the user input; generating an action data structure based on the request to perform the vehicle function, the action data structure including actions associated with the request to perform the vehicle function; determining a given zone among the plurality of zones of the vehicle in which a client device associated with the user who provided the user input is located; determining whether the user is authorized to cause performance of the vehicle function based on the given zone in which the client device associated with the user who provided the user input is located; in response to determining that the user is authorized to cause performance of the vehicle function based on the given zone in which the client device associated with the user who provided the user input is located, causing the vehicle function to be executed based on the action data structure; Including, determining the given zone in which the client device associated with the user who provided the user input is located, acquiring occupancy sensor data generated by one or more occupancy sensors of the vehicle; wherein the given zone in which the user who provided the user input is located is based on the occupancy sensor data.

6. The method of claim 1 , wherein the plurality of zones of the vehicle includes at least a driver zone, a passenger zone, and a rear passenger zone.

7. The user: When the given zone corresponds to the driver zone, the vehicle function is permitted to be triggered; The method of claim 6, wherein the given zone is not permitted to cause performance of the vehicle function when the given zone corresponds to the passenger zone or the rear seat zone.

8. The user: When the given zone corresponds to the driver zone, it is not permitted to cause the execution of the vehicle function, but The method of claim 6, wherein the given zone is permitted to cause performance of the vehicle function when the given zone corresponds to the passenger zone or the rear seat zone.

9. The user: When the given zone corresponds to the driver zone or the passenger zone, the vehicle function is allowed to be triggered; The method of claim 6, wherein when the given zone corresponds to the rear seat zone, causing the vehicle function to be not permitted to be triggered.

10. causing the vehicle function to be executed based on the action data structure, The method of claim 1 , comprising causing a head unit of the vehicle to execute the vehicle function based on the action data structure.

11. A method executed by one or more processors, comprising: receiving a user input from a user located within a vehicle, the vehicle including a plurality of zones; identifying a request to perform a vehicle function based on processing the user input; generating an action data structure based on the request to perform the vehicle function, the action data structure including actions associated with the request to perform the vehicle function; determining a given zone among the plurality of zones of the vehicle in which a client device associated with the user who provided the user input is located; determining whether the user is authorized to cause performance of the vehicle function based on the given zone in which the client device associated with the user who provided the user input is located; in response to determining that the user is authorized to cause performance of the vehicle function based on the given zone in which the client device associated with the user who provided the user input is located, causing the vehicle function to be executed based on the action data structure; Including, causing the vehicle function to be executed based on the action data structure, causing the client device to execute the vehicle function based on the action data structure.

12. A method executed by one or more processors, comprising: receiving a user input from a user located within a vehicle, the vehicle including a plurality of zones; identifying a request to perform a vehicle function based on processing the user input; generating an action data structure based on the request to perform the vehicle function, the action data structure including actions associated with the request to perform the vehicle function; determining a given zone among the plurality of zones of the vehicle in which a client device associated with the user who provided the user input is located; determining whether the user is authorized to cause performance of the vehicle function based on the given zone in which the client device associated with the user who provided the user input is located; in response to determining that the user is authorized to cause performance of the vehicle function based on the given zone in which the client device associated with the user who provided the user input is located, causing the vehicle function to be executed based on the action data structure; in response to determining that the user is not authorized to cause performance of the vehicle function based on the given zone in which the client device associated with the user who provided the user input is located, ceasing to cause the vehicle function to be executed based on the action data structure; discarding the action data structure; causing an indication that the action data structure has been discarded to be provided via the client device or a head unit of the vehicle for presentation to the user who provided the user input; A method comprising:

13. 1. A system comprising: At least one processor; A memory for recording instructions the instructions, when executed, cause the at least one processor to: receiving a user input from a user located in a vehicle, the vehicle including a plurality of zones; identifying a request to perform a vehicle function based on processing the user input; generating an action data structure based on the request to perform the vehicle function, the action data structure including actions associated with the request to perform the vehicle function; determining a given zone among the plurality of zones of the vehicle in which a client device associated with the user who provided the user input is located; determining whether the user is authorized to cause performance of the vehicle function based on the given zone in which the client device associated with the user who provided the user input is located; and in response to determining that the user is authorized to cause performance of the vehicle function based on the given zone in which the client device associated with the user who provided the user input is located, causing the vehicle function to be executed based on the action data structure; in response to determining that the user is not authorized to cause performance of the vehicle function based on the given zone in which the client device associated with the user who provided the user input is located; ceasing to cause the vehicle function to be performed based on the action data structure; and modifying the action data structure to include an alternative action based on a determination to cease causing the vehicle function to be performed based on the action data structure; causing the alternative actions included in the modified action data structure to be provided via the client device or a head unit of the vehicle for presentation to a user who provided the user input; A system that allows the user to:

14. A system comprising: At least one processor; A memory for recording instructions the instructions, when executed, cause the at least one processor to: receiving a user input from a user located in a vehicle, the vehicle including a plurality of zones; identifying a request to perform a vehicle function based on processing the user input; generating an action data structure based on the request to perform the vehicle function, the action data structure including actions associated with the request to perform the vehicle function; determining a given zone among the plurality of zones of the vehicle in which a client device associated with the user who provided the user input is located; determining whether the user is authorized to cause performance of the vehicle function based on the given zone in which the client device associated with the user who provided the user input is located; and in response to determining that the user is authorized to cause performance of the vehicle function based on the given zone in which the client device associated with the user who provided the user input is located, causing the vehicle function to be executed based on the action data structure; Let them do so, instructions for determining the given zone in which the client device associated with the user who provided the user input is located, instructions to cause a first speaker element of the vehicle to audibly emit a first inaudible tone, the first inaudible tone, when audibly emitted, to the client device: detecting the first inaudible tone via a microphone element of the client device; causing the client device to transmit a first acknowledgment to a head unit of the vehicle indicating that the client device has detected the first inaudible tone. Includes instructions, The system determining the given zone in which the client device associated with the user who provided the user input is located is based on the first notification.

15. instructions for determining the given zone in which the client device associated with the user who provided the user input is located, instructions to cause a second speaker element of the vehicle to audibly emit a second inaudible tone, the second inaudible tone, when audibly emitted, to the client device of the user; detecting the second inaudible tone via the microphone element of the client device; causing the client device to transmit a second acknowledgment to the head unit of the vehicle indicating that the client device detected the second inaudible tone. The method further includes the steps of:

15. The system of claim 14, wherein determining the given zone in which the client device associated with the user who provided the user input is located is further based on the second notification.

16. 16. The system of claim 15, wherein the first inaudible tone is audibly emitted at a first time and the second inaudible tone is audibly emitted at a second time, the second time being different from the first time.

17. instructions for determining the given zone in which the client device associated with the user who provided the user input is located, instructions for acquiring occupancy sensor data generated by one or more occupancy sensors of the vehicle; 15. The system of claim 14, further comprising: wherein the given zone in which the user who provided the user input is located is based on the occupancy sensor data.

18. A computer program that, when executed, causes at least one processor to perform a number of operations, the number of operations being: An operation of receiving a user input from a user located in a vehicle, the vehicle including a plurality of zones; identifying a request to perform a vehicle function based on processing the user input; generating, based on the request to perform the vehicle function, an action data structure including actions associated with the request to perform the vehicle function; determining a given zone among the plurality of zones of the vehicle in which a client device associated with the user who provided the user input is located; determining whether the user is authorized to cause performance of the vehicle function based on the given zone in which the client device associated with the user who provided the user input is located; in response to determining that the user is authorized to cause performance of the vehicle function based on the given zone in which the client device associated with the user who provided the user input is located, causing the vehicle function to be executed based on the action data structure; in response to determining that the user is not authorized to cause performance of the vehicle function based on the given zone in which the client device associated with the user who provided the user input is located; ceasing to cause the vehicle function to be performed based on the action data structure; and modifying the action data structure to include an alternative action based on a decision to cease causing the vehicle function to be performed based on the action data structure; causing the alternative actions included in the modified action data structure to be provided via the client device or a head unit of the vehicle for presentation to a user who provided the user input; A computer program comprising:

19. A system comprising at least one processor and a memory for recording instructions that, when executed, cause the at least one processor to perform a method described in any one of claims 2 to 5, 11, and 12.

20. A computer program which, when executed, causes at least one processor to perform a method according to any one of claims 2 to 5, 11 and 12.

Citation Information

Patent Citations

  • Vehicle

    JP2016085223A

  • Voice interaction system for vehicle

    JP2016206469A

  • Controlling a vehicle

    US9720498B2