VEHICLE COMMUNICATION

Abstract

A vehicle may detect an entity within a specified distance from a vehicle, output a first audio external to the vehicle in response to detecting the entity, classify the entity based on data collected about the entity after outputting the first audio, and output a second audio based on classifying the entity.

Description

FIELD OF TECHNOLOGY

[0001] The present disclosure discloses techniques for detecting and interacting with entities within a specified distance from a vehicle GENERAL STATE OF THE ART

[0002] Vehicles typically include sensors for collecting data about the vehicle and / or the environment surrounding the vehicle. In some examples, sensor data may be used by vehicle systems to actuate vehicle components. SUMMARY

[0003] A vehicle computer can classify an entity based on data collected by vehicle sensors. The vehicle can then operate components based on the entity's classification. For example, the vehicle can play audio, unlock doors, send a message to a remote computer, etc.

[0004] Accordingly, included in the present disclosure is a system comprising a computer including a processor and a memory, the memory having stored thereon instructions executable by the processor to: detect an entity within a specified distance from a vehicle, output a first audio external to the vehicle in response to detecting the entity, classifying the entity based on data collected about the entity after outputting the first audio, and outputting a second audio based on classifying the entity.

[0005] A vehicle feature can be enabled based on the classification of the entity.

[0006] The detection of the entity within the specified distance from the vehicle may be based on image data.

[0007] The second audio may be in response to communication from the entity.

[0008] The second audio can be selected from a stored variety of phrases.

[0009] The entity can be identified based on stored data collected before the first audio is emitted by the entity.

[0010] The entity can be classified based on a large number of images collected at different times.

[0011] The entity can be classified based on a tone of the entity.

[0012] The classification of the entity can be one of threatening or non-threatening.

[0013] The first audio may be output in a first language and the second audio may be output in a second language based on detecting the second language in an entity audio.

[0014] A first entity and a second entity may be detected within the specified distance from the vehicle, with the first entity being prioritized based on a risk assessment.

[0015] A method includes: detecting an entity within a specified distance from the vehicle, outputting a first audio external to the vehicle in response to detecting the entity, classifying the entity based on data collected about the entity after outputting the first audio, and outputting the second audio based on classifying the entity.

[0016] A vehicle feature can be enabled based on the classification of the entity.

[0017] The entity can be detected within the specified distance from the vehicle based on image data.

[0018] The second audio may be in response to communication from the entity.

[0019] The second audio can be selected from a stored variety of phrases.

[0020] The entity can be identified based on stored data collected before the first audio is emitted by the entity.

[0021] The entity can be classified based on a tone of the entity.

[0022] The classification of the entity can be one of threatening or non-threatening.

[0023] The first audio may be output in a first language and the second audio may be output in a second language based on detecting the second language in an entity audio. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a block diagram of an example vehicle. Fig. Figure 2 illustrates an example vehicle detecting an entity. Fig. 3 is a process flow diagram illustrating an example process for actuating vehicle components based on entities near the vehicle. DETAILED DESCRIPTIONExample system elements

[0024] Fig. 1 is a block diagram of a vehicle system 100 for providing digital images for vehicle operation. The vehicle 102 includes a computer 104 having memory containing instructions executable by the computer 104 to perform processes and operations, including those described herein. The computer 104 may be communicatively coupled to sensors 106, components 108, a human-machine interface (HMI) 110, and a communications module 112 within the vehicle 102 via a communications network, such as a vehicle network 114. The vehicle 102 may be any passenger or commercial vehicle, such as a car, truck, SUV, crossover, van, minivan, taxi, bus, ICE, BEV, hybrid, etc.

[0025] As stated above, the vehicle computer 104 includes a processor and memory. The memory includes one or more forms of computer-readable media and stores instructions executable by the vehicle computer 104 for performing various operations, including those disclosed herein. For example, a vehicle computer 104 may be a generic computer having a processor and memory, as described above, and / or may include an electronic control unit (ECU) or controller for a specific function or set of functions, and / or a dedicated electronic circuit including an ASIC (application-specific integrated circuit) fabricated for a specific operation, e.g., an ASIC for processing sensor data and / or communicating the sensor data.In another example, a vehicle computer 104 may include an FPGA (field-programmable gate array), which is an integrated circuit manufactured to be user-configurable. Typically, a hardware description language, such as VHDL (very high speed integrated circuit hardware description language), is used in electronic design automation to describe digital and mixed-signal systems, such as FPGAs and ASICs. For example, an ASIC is manufactured based on VHDL programming provided prior to manufacturing, whereas logical components within an FPGA may be configured based on VHDL programming (e.g., stored in memory electrically connected to the FPGA circuitry).In some examples, a combination of processor(s), ASIC(s), and / or FPGA circuits may be included in a computer 104.

[0026] The storage may be of any type (e.g., hard disk drives, solid-state drives, servers, or any volatile or non-volatile media). The storage may store the collected data sent by the sensors 106. The storage may be a separate device from the computer 104, and the computer 104 may retrieve information stored by the storage via a network 114 in the vehicle 102 (e.g., via a CAN bus, a wireless network, etc.). Alternatively or additionally, the storage may be part of the computer 104 (e.g., as a memory of the computer 104).

[0027] The vehicle computer 104 may include programming to operate one or more of vehicle braking, propulsion (e.g., controlling acceleration in the vehicle 102 by controlling one or more of an internal combustion engine, an electric motor, a hybrid engine, etc.), steering, climate control, interior and / or exterior lighting, etc., as well as to determine if and when the computer 104 should control such operations in place of a human operator.

[0028] The computer 104 may include or be communicatively coupled (e.g., via a vehicle network 114, such as a communications bus, as described in more detail below) to more than one processor (e.g., included in components 108, such as sensors 106, electronic control units (ECUs), or the like, included in the vehicle 102 for monitoring and / or controlling various vehicle components 108, e.g., a powertrain controller, a braking controller, a steering controller, etc.). The computer 104 is generally arranged for communication in a vehicle communications network 114, which may include a bus in the vehicle 102, such as a Controller Area Network (CAN) or the like, and / or other wired and / or wireless mechanisms.Alternatively or additionally, in cases where the computer 104 actually includes a plurality of devices, the vehicle communication network 114 may be used for communications between devices, which are depicted in this disclosure as the computer 104. Further, as noted below, various controllers and / or sensors 106 may provide data to the computer 104 via the vehicle communication network 114.

[0029] Via the vehicle network 114, the vehicle computer 104 may transmit messages to various devices in the vehicle 102 and / or receive messages (e.g., CAN messages) from the various devices (e.g., sensors 106, ECUs, etc.). Alternatively or additionally, in cases where the vehicle computer 104 actually includes a plurality of devices, the vehicle communications network 114 may be used for communications between devices, which are depicted in this disclosure as the vehicle computer 104. Further, as mentioned below, various controllers and / or sensors 106 may provide data to the vehicle computer 104 via the vehicle communications network 114.

[0030] The vehicle 102 typically includes a variety of sensors 106. A sensor 106 is a device that can obtain one or more measurements of one or more physical phenomena. Some sensors 106 detect internal conditions of the vehicle 102, for example, wheel speed, wheel alignment, and engine and transmission variables. Some sensors 106 detect the position or orientation of the vehicle 102, for example, global positioning system (GPS) sensors. Some sensors 106 detect objects, for example, radar sensors, laser scanner rangefinders, light detection and ranging devices (LIDAR), and image processing sensors such as cameras. Other sensors 106 detect sound, for example, dynamic or condenser microphones, piezoelectric transducers, ultrasonic sensors, acoustic emission sensors, etc.Such sensors for detecting sound are referred to in this document as audio sensors. Audio sensors detect sound waves by measuring vibrations and converting them into electrical signals.

[0031] The vehicle 102 includes a plurality of audio sensors (e.g., microphones that can detect sound inside the vehicle 102 and / or outside the vehicle 102). An audio sensor may include or be coupled to an analog-to-digital (A / D) converter to facilitate the conversion of sound waves into electrical signals.

[0032] As one example, the A / D converter may use sampling to convert sound into electrical signals. An audio sensor and / or A / D converter may be in communication with the vehicle computer 104 so that the electrical signals may be transmitted to the vehicle computer 104. One or more audio sensors may be installed or deployed in or on any suitable part of the vehicle 102 to provide detection of audio phenomena as described herein. For example, audio sensors for detecting sound emitted from or within the interior of the vehicle 102 may be provided in a vehicle instrument panel, on interior panels of a vehicle 102, the interior of the vehicle roof, etc.As another example, audio sensors for detecting sound emitted outside the vehicle 102 may be supported by the exterior of the vehicle 102, the exterior of the vehicle roof, in or on bumpers or panels of the vehicle, the vehicle hood, etc. Multiple audio sensors may be installed at different locations in or on the vehicle 102 to provide sound detection and / or to distinguish between multiple sound sources, as described herein. The audio sensors may be any suitable method for detecting sound, as described herein.

[0033] The vehicle 102 may include one or more sound output devices 116 (i.e., speakers). A speaker is a device capable of outputting sound. As one example, a speaker may include a digital-to-analog converter (D / A converter) that converts electrical signals into vibrations to generate sound at a specified frequency. As one example, the D / A converter may input the electrical signal into a piecewise constant function. A speaker may be in communication with the vehicle computer 104 to receive electrical signals to be transmitted through the speaker in the form of sound. The speaker may be any suitable type of speaker for outputting sound inside the vehicle 102 and / or outside the vehicle 102. A speaker may also operate as a microphone. That is, a device can both output and detect audio.Throughout this document, sound output devices 116 and sensors 106 are referred to as separate elements, although it is understood that the operations of a speaker and a microphone element could be performed by a single sound output device 116. The speaker may be deployed or installed in or on the vehicle 102 (e.g., in or on a surface of the vehicle 102). As one example, the speaker may be carried through the hood of the vehicle 102, so that output sound is likely to be detectable by listeners positioned in front of the vehicle 102.

[0034] Fig. Figure 2 illustrates an example scenario in which an entity 118 might be detected by a vehicle computer 104 from vehicle sensor data. The entity 118 may be any object or event detectable by the vehicle sensors 106. In various examples described herein, the entity 118 is a person, although, as further non-limiting examples, the entity 118 could be a second vehicle, a fire, etc.

[0035] As described above, entity 118 may be any object detectable from data received from vehicle sensors 106. A vehicle computer 104 could use any suitable object detection algorithm or technique to detect and / or classify an object. For example, a neural network could be trained using data collected about objects to classify objects (e.g., to distinguish people from other objects based on camera sensor data).

[0036] As another example of an entity 118, the entity 118 may be an event, such as a fire, an altercation between two or more people, etc. That is, the entity 118 may include detected events that are not objects. Any suitable technique for detecting and / or classifying objects could be used to detect events (e.g., a neural network could be trained for object detection or classification). Further, a neural network or the like could process sensor data in addition to, or as an alternative to, image data. Thus, in an example where the entity 118 is a fire, the vehicle may include sensors 106 that are thermal sensors or thermal cameras. Data collected by such sensors could include an image and / or could include a temperature of the entity 118.For example, if the temperature exceeds 160 degrees Fahrenheit and / or based on image data, a computer 104 could classify the entity 118 as a fire. As another example, if the entity is an altercation, a computer 104 could input image data and / or audio data into one or more neural networks trained to classify altercations.

[0037] The vehicle computer can identify an entity. Identifying the entity, as used herein, means determining who the entity is (e.g., determining that the entity is a particular individual). For example, an entity might be classified as a person and then identified as "Jane Doe." The vehicle computer can identify an entity using any suitable recognition technique based on stored or collected data. As an example, if the vehicle computer has data about the entity stored on it, such as an image of the entity's face, the vehicle can compare the stored image with a newly collected image of the entity's face. If the computer determines, through image recognition or any other suitable technique, that the two images match, the computer has identified the entity as the same entity from which the stored data was collected.

[0038] As in Fig. 2, a sensor 106 may be positioned in or on the vehicle 102 to provide a field of view 120 that extends relative to the vehicle body. The sensor 106 may be of any type capable of detecting an entity 118, such as a camera, a lidar, a radar, etc. In the present example, the sensor 106 is a camera. The field of view 120 may be determined by specifications of the camera (e.g., a viewing angle provided by the camera) and the pose or orientation of the camera (e.g., a pitch, roll, and yaw rate of an axis of the camera lens relative to a horizontal plane of the vehicle 102). The field of view 120 is referred to as fixed or static if the camera is installed such that its pose cannot be changed.Additionally or alternatively, the camera may be movably carried by the vehicle 102 and operable by the vehicle computer 104 so that the camera pose and / or a focal point or orientation of a lens may be changed and the field of view 120 may thereby be dynamically adjusted.

[0039] A camera may generate image data of its field of view 120. Image data from a camera typically includes a sequence of individual images from the field of view 120 (i.e., image data may be video data). The individual images may be input to the vehicle computer 104 via the vehicle network 114. The vehicle computer 104 may analyze the individual images (e.g., may utilize suitable image recognition techniques) and may make a determination as to whether the entity 118 is within a specified distance 122 from the vehicle 102, as further described below.

[0040] The vehicle computer 104 may be programmed to detect an entity 118 within a specified distance 122 from the vehicle 102. For example, the vehicle computer 104 may detect the entity 118 and determine that the entity 118 is within the specified distance 122 from the vehicle 102 based on collected data about the entity 118 (e.g., measured as a radius around a point defined in or on the vehicle 102, as a distance from an edge of a vehicle 102 as defined by a vehicle body, or based on any other suitable reference).The vehicle computer 104 may make a determination that an entity 118 is within a specified distance 122 from the vehicle 102 when it determines that the entity 118, or at least a portion of the entity 118, is detected within a distance from the vehicle 102 that is less than the specified distance 122. The detection of the entity 118 within the specified distance 122 from the vehicle 102 may be based on data collected by the sensors 106. As one example, the data may be image data. The distance may be measured from any selected point on the vehicle 102. As one example, the distance may be measured from the sensor 106 that collected the entity data. As another example, the distance may be measured from a surface of the vehicle 102 that is closest to the entity 118.

[0041] The specified distance 122 may be any suitable distance determined by the vehicle computer 104 and / or a vehicle operator. For example, the vehicle computer 104 may determine a specified distance 122 based on the vehicle 102 entering a geographic location (i.e., the vehicle computer 104 may store specified distances 122 for respective geographic locations and may retrieve a specified distance 122 for a location from memory based on data from a location sensor, e.g., a GPS sensor). A specified distance 122 (i.e., a distance from a vehicle 102 within which detection of an entity 118 triggers classification and interaction with the entity 118) may be determined based on properties or attributes of a geographic location.For example, a larger specified distance 122 of 20 meters could be selected when in geographic locations where vehicles are less likely to be parked close together (e.g., in suburban residential areas or rural locations). On the other hand, a smaller specified distance 122 of 10 meters could be selected when vehicles are likely to be parked closer together (e.g., an urban location, a parking garage or parking lot, etc.). A smaller specified distance 122 may increase the frequency with which entities are classified and audio is output, and vice versa. Alternatively or additionally, a specified distance 122 may be selected by a vehicle operator.As an example, if the vehicle operator desires the vehicle 102 to interact only with entities located near the vehicle 102, the vehicle operator may set the specified distance 122 to 1 meter. The vehicle operator may select the specified distance 122 by any suitable means, such as by providing input via a vehicle MMS 110.

[0042] The specified distance may be measured from the vehicle 102 while the vehicle 102 is stationary or while the vehicle is moving. The entity 118 may move relative to the vehicle 102 while the vehicle is stationary or while the vehicle is moving and still be detected by the computer. As an example, the vehicle 102 may be reversing in a driveway and an entity 118 may approach toward the rear of the vehicle 102. The computer may then detect the entity 118 within the specified distance.

[0043] The vehicle computer 104 may be programmed to actuate output of a first audio 124 upon detecting that an entity 118 is within the specified distance 122 from the vehicle 102. The first audio 124 may be output external to the vehicle 102 (e.g., via an external vehicle speaker) in response to detecting the entity 118. The first audio 124 may be any sound directed to be heard by the entity 118. The first audio 124 to be played may be selected by the vehicle computer 104 from the stored first audio 124 based on the classification of the entity 118, as described in more detail below. For example, the vehicle computer 104 may process various audio (e.g., sounds, words, phrases, etc.).) that could be selected based on a classification of an entity 118 and / or other factors, and / or have access to a memory that stores it. In one example, the vehicle computer 104 may store audio data (e.g., files or the like) for playback and may further store associations between entities and / or entity classifications, whereby the computer 104, after identifying and / or classifying an entity 118, may select audio data (e.g., a file) for output. This association could be stored in a table or the like. For example, the first audio 124 to be output to an owner of the vehicle according to the audio table may be "Hello" along with the owner's name. As another example, for specific entity classifications, an audio table could specify that no audio output is to be provided.

[0044] Example factors that may guide vehicle computer 104's selection of the first audio 124 may include a time of day, weather conditions (e.g., presence or absence of precipitation) and / or other environmental conditions (e.g., an ambient light level that is above or below a runtime threshold), a day of the week, a day of year, etc. As an example, if vehicle computer 104 triggers output of the first audio 124 at 9:00 a.m. on January 1, vehicle computer 104 may select the first audio 124 as "Good morning and Happy New Year."As another example, if the vehicle computer 104 triggers an output of the first audio 124 during a storm or similar severe weather event where a person is having difficulty hearing, the vehicle computer 104 may select the first audio 124 as “Please seek shelter” while increasing the volume of the first audio 124 to compensate for any ambient noise.

[0045] As in Fig. 2, entity 118 may output entity audio 126. "Entity audio" as used herein refers to a sound generated by entity 118 and detected by vehicle audio sensors. As one example, the entity 118 audio may be a request made by or an expression of distress from an entity 118 within a specified distance 122. The entity 118 audio is further described below.

[0046] The vehicle computer 104 may be programmed to classify the entity 118 based on entity audio 126, camera image data, and / or any other suitable data. The classification may include any suitable object classification technique. A vehicle computer 104 may analyze data collected about the entity 118 using a machine learning or rule-based program and assign the entity 118 a confidence score (typically a percentage indicating an estimated probability, e.g., 90%, 99%, etc.) that the entity classification is correct.

[0047] The classification for a specific entity 118 may be selected by the vehicle computer 104 from a plurality of stored classifications. The classifications may be determined to trigger respective actions by the vehicle computer 104, as described below. For example, and without limitation, the vehicle computer 104 could classify a person detected within the specified distance 122 as threatening or non-threatening.

[0048] The vehicle computer 104 can assign a risk score to the entity 118. Determining a risk score can be rule-based. That is, if specified conditions are met during the process of analyzing the data, the risk score can be increased or decreased. A specified condition, in this context, means any condition or phenomenon detectable by vehicle sensors 106 that, according to the programming in the computer 104, can influence the risk score of an entity 118. Examples of specified conditions include a speed at which an entity 118 approaches the vehicle 102, a decibel level at which the entity 118 outputs audio, a distance of the entity 118 from the vehicle 102 that is within the specified distance 122, etc.If a specified condition meets a threshold, the vehicle computer 104 may reduce or increase the risk score of the entity (e.g., moving the entity 118 below the specified distance 122 may increase the risk score, detecting audio output by an entity 118 above a decibel threshold may increase the risk score, moving the entity 118 away from the vehicle 102 may decrease the risk score, etc.).

[0049] The computer 104 could store a table or the like that assigns risk scores to specified conditions based on empirical testing or design considerations, such as how much risk can be tolerated. Further, a detected entity 118 could initially be defined with a default risk score (e.g., 50 on a scale of zero to 100, or "threatening" or "non-threatening," where the risk score is a binary determination of whether the entity 118 appears to pose a risk (i.e., is threatening) or does not appear to pose a risk (i.e., is non-threatening)). The default risk score could be a risk threshold; that is, the computer 104 could be programmed so that risk scores below the risk threshold do not trigger action, while risk scores at or above the threshold do.For example, the computer 104 could store data specifying risk ratings for entities that cannot be identified based on a distance from the vehicle 102. The computer 104 could alternatively or additionally store instructions to adjust a risk rating based on a speed at which the entity 118 is moving relative to the vehicle 102 (e.g., a positive speed (i.e., toward the vehicle 102) could increase the risk rating, whereas a negative speed (i.e., away from the vehicle 102) could decrease the risk rating). For example, the computer 104 could detect an entity 118 within the specified distance 122 and could further determine that the entity 118 is approaching the vehicle 102 at a speed of 5 meters per second. The computer 104 could then (e.g.,According to a stored table, the risk score increases by 5 points for every 1 meter per second above 1 meter per second at which entity 118 approaches. In this example, entity 118 would therefore be assigned a risk score of 20. In this example, entity 118 then has a risk score below the risk threshold. As another example of specified conditions, vehicle computer 104 could detect that entity 118 has a key or remote key fob or the like to access vehicle 102, thus reducing the entity's risk score. As another example, based on analyzing the data collected about entity 118, vehicle computer 104 may detect that entity 118 is swinging an object. Vehicle computer 104 may then increase entity 118's risk score.

[0050] Determining the risk score may further be based on stored data about the entity 118. That is, the computer 104 may assign a risk score to an entity 118 based on a stored risk score. As an example of setting a risk score by the computer 104 based on a stored risk score, a risk score assigned to an entity 118 during a previous interaction could be retrieved from storage or obtained from a remote computer 104 when the entity 118 is identified. In such an example, the initial risk score assigned to the entity 118 could be the retrieved risk score instead of the default risk score described above. The default risk score could further represent a risk threshold in such examples.In addition to the risk score being based on specified conditions being met, as mentioned above, the risk score may be increased or decreased based on a stored risk score. For example, a risk score assigned to an entity 118 during a previous interaction could be retrieved from memory or obtained from a remote computer 104 when the entity 118 is identified. In such an example, the computer 104 could assign a default risk score to the entity 118 and increase or decrease the risk score by an amount proportional to the stored risk score. For example, the current risk score of the entity 118 could be increased by 5 if the stored risk score exceeds the risk threshold by 10.

[0051] The vehicle computer 104 may be programmed to assign risk scores to multiple entities 118. That is, if more than one entity 118 is detected within the specified distance, the computer 104 may perform all of the functions described herein for each entity 118 individually. For example, if a first entity (unnumbered) is detected within the specified distance on one side of the vehicle 102 and a second entity (unnumbered) is simultaneously detected within the specified distance on a second side of the vehicle 102, the computer may separately classify the instances, assign them a risk score, identify them, etc. The computer 104 may further be programmed to prioritize a first entity over a second entity based on the first entity being assigned a higher risk score than the second entity.That is, if two entities are detected within the specified distance and the first entity has been assigned a risk rating that exceeds the risk threshold, the computer 104 may treat both entities as having a risk rating that exceeds the risk threshold. For example, two people may be within a specified distance of a vehicle 104. If the first person is calm (i.e., has a low risk rating) and the second person is agitated (i.e., has a high risk rating), the computer 104 may treat both as having a risk rating above the risk threshold and take the actions described herein.Additionally, if an entity 118 is identified by the vehicle, as described below, that entity 118 may still be assigned permissions despite the presence of an entity 118 with a risk score above the risk threshold. For example, if the first entity is identified as the vehicle owner and the second entity has a risk score above the risk threshold, the first entity will not be treated as having a risk score above the risk threshold.

[0052] As described further below, the vehicle computer 104 may actuate various vehicle components 108 (e.g., an audio output component, as described above) based on the risk assessment exceeding a threshold, for example, exceeding the default risk assessment. As an example, a risk assessment above a risk threshold would cause the vehicle 102 to lock vehicle doors, close windows, and / or send a message to a remote device, etc. Alternatively or additionally, a risk assessment below a threshold could cause the computer 104 to actuate one or more vehicle components 108.

[0053] The risk score could be determined in other ways in addition to or alternatively to initially assigning the default risk score and can be determined based on any suitable characteristic or action of entity 118. As one example, the risk score can be based on the measured decibels of the audio of entity 118. For example, entity 118 yelling may increase its risk score, whereas entity 118 speaking below a threshold decibel level may or may not reduce the entity's risk score. The risk score can alternatively or additionally be determined based on physical actions taken by entity 118 (i.e., if entity 118 swings its arms and / or holds an object, entity 118's risk score may be increased).Furthermore, the risk score may be increased or decreased based on a speed of the entity 118 measured by the sensors 106 (i.e., if the entity 118 approaches the vehicle 102 at a specified speed or more, this may increase its risk score).

[0054] As mentioned above, a risk assessment of entity 118 may be based on stored data about entity 118 (i.e., data collected or received by vehicle computer 104 prior to a present detection of entity 118). Vehicle computer 104 may store data collected about entity 118 and retrieve the stored data when entity 118 is identified as previously detected and one for which stored data is available. For example, entity 118 could be identified using known recognition techniques, such as image recognition or speech recognition. Typically, such recognition techniques utilize machine learning. Vehicle computer 104 may retrieve stored data for entity 118 and compare the stored data with new or current data once vehicle computer 104 identifies entity 118.

[0055] In one example, stored data about an entity 118 could be compared with currently collected data about the entity 118. For example, the vehicle computer 104 may determine a risk assessment of an entity based on its speech volume and body movements, as described above. This risk assessment and the factors on which it was based may then be stored by the vehicle computer 104. At a later time (e.g., hours or days later), the vehicle computer 104 may then detect an entity 118 and collect additional, current data about it. The stored data may then be retrieved if it is determined that the stored data was collected from the same entity 118, with the stored data, as well as any newly collected data, being used to make determinations regarding the entity 118.

[0056] Additionally, the vehicle computer 104 may assign or determine permissions to an entity 118 when the entity 118 is identified (or based on being unable to identify the entity 118). Examples of permissions might include "may operate the vehicle," "may access the vehicle," "may communicate with the vehicle computer 104," etc. Such permissions might be based on the preferences of the vehicle owner or based on determinations made by the vehicle computer 104. For example, the vehicle owner might assign a permission to operate a vehicle 102 to another user. Further, the vehicle computer 104 may ask the entity 118 for image data of a driver's license to confirm that the entity 118 is the acquaintance, or might use object recognition techniques to determine the identity of the acquaintance 118.

[0057] The risk rating of the entity may be increased or decreased based on data collected after the first audio 124 is played. This data may include the tone of voice used by the entity 118, the body language of the entity 118, whether the entity 118 is holding an object, etc. The vehicle computer 104 may be further programmed to assign permissions to the entity 118 based on data collected after the first audio 124 is played. As an example, the vehicle computer 104 may increase or decrease the risk rating of the entity 118 based on a tone of voice of the entity 118. "Tone," as used herein, refers to the intonations used by a person when speaking. Such intonations may be detected by the vehicle computer 104 and used to determine a risk rating and / or make classifications (i.e.,(For example, if the tone of voice of entity 118 is determined by vehicle computer 104 to be "excited," vehicle computer 104 could increase the risk rating of entity 118.) As another example, vehicle computer 104 may decrease the risk rating of entity 118 if the tone of voice of entity 118 is determined to be "questioning." Tone may be measured by analyzing audio data of the entity regarding the rate, volume, and pitch of speech. For example, the computer may store a table specifying certain tones associated with certain notes. If the computer detects that the pitch of the entity's audio is a particular note, the computer looks up the corresponding tone for that note. The table may further store a table specifying certain tones associated with speech rate and volume.The vehicle computer 104 may determine the tone of voice of the entity 118 using any suitable technique (e.g., a machine learning program trained to classify entities based on tone). For example, the vehicle computer 104 may detect through the machine learning program that a pitch of the person's voice rises at the end of a sentence, which could indicate that the person's tone is questioning, thereby lowering the risk score. As another example, the vehicle computer 104 may detect that the pitch of the person's voice remains low and harsh, which could indicate hostility, and may increase their risk score.

[0058] The vehicle computer 104 may store a permission table or the like that specifies which permissions are granted to particular entities that may be identified. For example, the user John Doe could be given full permission to access and operate the vehicle 102.

[0059] The vehicle computer 104 may be programmed to classify the entity 118 based on image data from one or more vehicle sensors 106. Classifying the entity 118 means determining a type or category of the entity 118 according to any suitable technique. For example, various techniques may be used to analyze data from cameras or other sensors 106 to determine a type or category of entity 118. A type or category of entity 118 may refer to a type of object, etc., a person, a bicycle, a vehicle, a rock, etc. Image data used for entity classification and / or other purposes (e.g., determining the behavior of the entity, such as direction and / or speed of movement) may be a plurality of images collected at different times.For example, analyzing a plurality of images collected at different times may allow the vehicle computer 104 to determine whether the entity 118 is walking toward the vehicle 102 or waving its hand.

[0060] Entity classification may be based on data collected about entity 118 after the first audio 124 is output from the vehicle speakers and then further data about or from entity 118 is detected. That is, in addition to or alternatively to collecting data about entity 118 upon detecting entity 118 within the specified distance 122 (i.e., before outputting the first audio 124), vehicle computer 104 may collect further data about entity 118 after vehicle computer 104 outputs the first audio 124 described above. In such an example, vehicle computer 104 may detect entity 118 before outputting the first audio 124, but may not classify or identify it. As one example, an entity 118 may be detected within the specified distance 122.The vehicle computer 104 may then output the first audio 124 in the form of a greeting, such as "Hello." The vehicle computer 104 may then collect further data about the entity 118 and could use the additional data to classify and / or identify the entity 118.

[0061] As described above, the vehicle computer 104 may be further programmed to assign classifications to the entity 118 based on the data collected after the first audio 124 is output. For example, the entity 118 may output audio as described above (i.e., the entity 118 may speak to the vehicle 102). The entity 118 may output an audio such as the question, "Who owns this vehicle?" The vehicle computer 104 may then collect further data about the entity 118 and classify the entity 118 as a person based on the entity's use of language. As another example, the entity 118 may output an audio, such as the dog barking, which could be used by the computer 104 to classify the entity 118 as a dog.

[0062] The vehicle computer 104 may be programmed to output a second audio based on the classification of the entity 118. The vehicle computer 104 may select the second audio from a list of sounds and phrases, as described above with respect to the first audio 124. The second audio is selected by the vehicle computer 104 based on the classification of the entity 118. For example, if the vehicle computer 104 has assigned an entity 118 a classification of "non-threatening" and has given a "vehicle owner" authorization (e.g., has identified the entity 118 as the vehicle owner), the vehicle computer 104 may output a second audio greeting the vehicle owner by name. As another example, the second audio may be in response to communication by the entity 118.If the vehicle computer 104 has assigned the entity 118 a classification of "non-threatening" and has provided authorization as a "stranger" (e.g., has identified the entity 118 as a stranger), the vehicle computer 104 may output a second audio that includes a response to a question posed by the entity 118. In other words, the second audio may be selected from the stored plurality of phrases based on a verbal response by the entity 118 to the first audio 124. As another example, if the vehicle computer 104 has assigned the entity 118 a classification of "threatening" and the entity 118 has a risk score above a risk threshold, the vehicle computer 104 may output a second audio requesting that the entity 118 move away.

[0063] The vehicle computer 104 may be programmed to output a second audio based on the entity audio 126. For example, the vehicle computer 104 may have a default language. The default language may be English in the United States, German in Germany, any language selected by the vehicle owner, etc. If the entity 118 outputs entity audio 126 in a language that is not the default language, the computer 104 may output the second audio in the language spoken by the entity 118. The vehicle computer 104 may use known speech recognition techniques to recognize the language spoken by the entity 118.

[0064] As described above, the vehicle computer 104 may be programmed to turn on a vehicle feature based on classifying the entity 118. For example, based on classifying the entity 118 as "non-threatening" and assigning permissions as "vehicle owner" (e.g., identifying the entity 118 as the vehicle owner), the vehicle computer 104 may unlock the doors, start the engine, or turn on any feature of the vehicle 102. In contrast, and as another example, if the entity 118 is classified as "non-threatening" and receives permission of "access to the vehicle permitted" (e.g., identified as a person known to the vehicle owner), the vehicle computer 104 may unlock the vehicle doors, but not necessarily start the engine. As another example, if the entity 118 is classified as "threatening" (e.g.,the entity 118 has a risk score above a risk threshold), send a communication to the vehicle owner warning the vehicle owner regarding the entity 118. Example processes

[0065] Fig.3 shows a process flow diagram of an exemplary process 300 for detecting and interacting with entities within a specified distance 122 from a vehicle 102. The process may be performed according to program instructions executed by the vehicle computer 104. The process begins at a decision block 305, where the computer 104 determines whether an entity 118 has been detected. The detection of an entity 118 is described in more detail above. If the computer 104 does not detect an entity 118 within the specified distance 122 from the vehicle 102, the process proceeds to block 370. If the vehicle computer 104 detects an entity 118 within the specified distance 122 from the vehicle 102, the process proceeds to block 310.

[0066] Next, in a block 310, the computer 104 actuates sensors 106 to collect data about the entity 118. In some implementations, actuating or turning on sensors may not be required because sensors may already be active. However, in some implementations, various sensors 106, such as camera sensors or the like, may be inactive (e.g., to conserve power while a vehicle 102 is parked), and accordingly, sensors 106 may be actuated and / or turned on when an entity 118 is detected. There may be a plurality of sensors, fewer than all available vehicle sensors, that are turned on or active at any time to monitor for an entity 118 within the specified distance 122, as described above.For example, when a vehicle 102 is in a key-off state or the like, one or more sensors may be deployed to monitor vehicle surroundings (e.g., to provide a field of view or fields of view 120 around the vehicle 102) and detect entities. Once an entity 118 is detected, the vehicle computer 104 may then actuate the remaining sensors 106 to begin collecting further data about the entity 118.

[0067] Next, in a block 315, the computer 104 compares the data collected in block 310 with stored data. As described above, the computer 104 may store data collected about an entity 118 that is to be retrieved later. The comparison of stored data with the newly collected data may begin once data collection has begun.

[0068] Next, in a block 320, the computer 104 classifies the entity 118. As described above, the classification of the entity 118 may be based on data collected from the entity 118 and on previously stored data about the entity 118.

[0069] Next, in a decision block 325, the computer 104 determines whether to output a first audio 124. The computer 104 may determine not to output a first audio 124. For example, if the entity 118 is classified as an animal or as another vehicle, the computer 104 may not output a first audio 124. Similarly, if the entity 118 is a person passing within the specified distance 122 and moving away (i.e., passing the vehicle 102), the computer 104 may not output a first audio 124.

[0070] Next, in a block 330, the computer 104 outputs the first audio 124, as described above. The vehicle computer 104 may select the first audio 124 to play from the stored first audios based on the classification and identification of the entity 118. As an example, if the entity 118 is identified by the computer 104 as a stranger asking a question, the first audio 124 may be a response to the question. If the entity 118 is identified as the owner of the vehicle 102, the first audio 124 may be a greeting for the owner.

[0071] Next, in a block 335, the computer 104 stores the data collected about the entity 118. As described above, the data may be retrieved later to identify the entity 118 and / or assign a risk score to it.

[0072] Next, in a decision block 340, the computer 104 determines whether the entity 118 has been identified. That is, an individual entity 118 (e.g., a person) may be identified using a suitable technique, as described above. As an example, if the vehicle computer 104 detects audio output from the entity 118, the vehicle computer 104 may use a speech recognition technique to compare the audio to stored audio data and then identify the entity 118. If the computer 104 determines that the stored audio data and the newly collected audio data are from the same entity 118, the computer 104 marks the entity 118 as identified. If the entity 118 has been identified, the process proceeds to block 345. Otherwise, the process proceeds to block 350.

[0073] In a block 345, the vehicle computer 104 has identified the entity 118. The vehicle computer 104 may then grant vehicle permissions to the entity 118. Permissions may be based on a permission table or the like, as described above. Furthermore, the entity 118 may be identified but have no associated permissions, in which case none may have been granted.

[0074] In decision block 350, the computer 104 previously determined that the entity 118 is not identified by the vehicle computer 104. The computer 104 then analyzes the data collected in block 310 for the specified conditions to determine the risk score of the entity 118, as described above. If the computer 104 increases the risk score of the entity to or above the risk threshold, the process proceeds to block 355. Otherwise, the process proceeds to decision block 360.

[0075] Next, in a block 355, the vehicle computer 104 may send a message to a remote device. The remote device may be any suitable computing device other than the vehicle computer 104, such as a mobile device or a smartphone, or the like. If the specified conditions in the previous block are detected (i.e., the entity's risk score exceeds the risk threshold), the vehicle computer 104 may send a message to the vehicle owner's mobile device to alert them regarding the entity 118 and / or send a message to a law enforcement device to request assistance.

[0076] Next, in a decision block 360, the computer 104 determines whether a second audio should be output. The computer 104 may determine not to output a second audio. For example, if the entity 118 was identified as a person asking a question and the first audio 124 answered the question, causing the entity 118 to move away, the computer 104 may determine not to output a second audio. As another example, if the entity 118 was identified as the vehicle owner and the vehicle owner entered the vehicle 102, the computer 104 may determine not to output a second audio. As described above, the vehicle computer 104 may output the second audio in a language of the entity 118 (e.g., Spanish if the entity audio was Spanish).As another example, if the entity 118 has been assigned a risk score above the risk threshold, the computer 104 may output a second audio, as noted below. If the computer 104 determines to output a second audio, the process proceeds to block 365. Otherwise, the process proceeds to decision block 370.

[0077] Next, in a block 365, the computer 104 actuates a vehicle component 108 to output a second audio. As described above, the vehicle computer 104 may select the second audio from a list of sounds and phrases, as described above with respect to the first audio 124. The second audio is selected by the vehicle computer 104 based on the risk assessment and / or the identification of the entity 118. That is, the second audio is selected to correspond to the risk assessment and / or the identification of the entity 118. For example, if the vehicle computer 104 has identified the entity 118 as the owner of the vehicle 102, the vehicle computer 104 may output a second audio greeting the vehicle owner by name. As another example, the second audio may be in response to communication by the entity 118.If the vehicle 102 has identified the entity 118 as a person asking a question, the vehicle computer 104 may output a second audio including a response to the question posed by the entity 118. In other words, the second audio may be selected from the stored plurality of phrases based on a verbal response by the entity 118 to the first audio 124. As another example, if the vehicle computer 104 has assigned the entity 118 a risk score above a risk threshold, as described above, the vehicle computer 104 may output a second audio requesting that the entity 118 move away.

[0078] At decision block 370, the computer 104 determines whether the process should continue. For example, once the process 300 has been initiated, the computer 104 could continue monitoring for the detection of entities 118 within the specified distance 122 of the vehicle 102 by returning to block 305. However, the process could end upon an input or event to terminate the process, such as a vehicle user beginning operation of the vehicle 102 (e.g., turning on a propulsion system such as an engine), a user providing input to terminate the process, etc. If the process 300 should continue, the process returns to block 305; otherwise, the process 300 ends.

[0079] Computing devices, such as those discussed herein, generally each include instructions executable by one or more computing devices, such as those mentioned above, and for performing blocks or steps of the processes described above. For example, the process blocks described above may be embodied as computer-executable instructions.

[0080] Computer-executable instructions may be compiled or interpreted from computer programs created using a variety of programming languages ​​and / or technologies, including, among others, either alone or in combination, Java™, C, C++, Python, Julia, SCALA, Visual Basic, Java Script, Perl, HTML, etc. In general, a processor (i.e., a microprocessor) receives instructions (i.e., from memory, a computer-readable medium, etc.) and executes those instructions, thereby performing one or more processes that include one or more of the processes described in this document. Such instructions and other data may be stored in files and transferred using a variety of computer-readable media. A file in a computing device is generally a collection of data stored on a computer-readable medium, such as a storage medium, random access memory, etc.

[0081] A computer-readable medium (also referred to as a processor-readable medium) includes any non-transitory (e.g., physical) medium involved in providing data (e.g., instructions) that can be read by a computer 104 (e.g., by a processor of a computer 104). Such a medium may take many forms, including, without limitation, non-transitory media and volatile media. Instructions may be transmitted through one or more transmission media, including fiber optics, wires, wireless communications, including internal structural elements comprising a system bus coupled to a processor of a computer 104. Common forms of computer-readable media include, for example, RAM, a PROM, an EPROM, a FLASH EEPROM, any other memory chip or storage device, or any other medium from which a computer 104 can read.

[0082] All terms used in the claims are intended to have their common and ordinary meaning as understood by one skilled in the art, unless expressly stated otherwise herein. In particular, the use of the singular articles, such as "a," "an," "the," "the," "the," etc., is to be interpreted as referring to one or more of the listed elements, unless a claim expressly limits the matter to the contrary.

[0083] In the drawings, the same reference numerals indicate the same elements. Furthermore, some or all of these elements could be changed. With respect to the media, processes, systems, methods, etc. described in this specification, it should be understood that although the steps or blocks of such processes, etc., have been described as occurring according to a certain sequence, such processes could be implemented such that the described steps are performed in a different order than the order described in this specification. Further, it should be understood that certain steps could be performed concurrently, other steps could be added, or certain steps described herein could be omitted. In other words, the descriptions of processes in this specification are provided to illustrate certain embodiments and should in no way be construed to limit the claimed invention.

[0084] The use of "in response to," "based on," and "when determined" in this document indicates a causal relationship, not just a purely temporal one. "Based on" or "in response to" can mean at least partly based on or at least partly in response to, unless explicitly stated otherwise.

[0085] Examples are contemplated in this specification. Any exemplary embodiment or exemplary feature described in this specification should not be construed as preferred or advantageous over other embodiments or features. Furthermore, exemplary embodiments described in this specification are not intended to be limiting. It should be readily understood that certain aspects of the disclosed systems and methods may be arranged and combined in a wide variety of different configurations, all of which are contemplated herein. In addition, the specific arrangements shown in the figures should not be considered limiting. It should be understood that other embodiments may include more or less of each element shown in a given figure.Additionally, some of the illustrated elements may be combined or omitted. Still further, an exemplary embodiment may include elements not illustrated in the figures.

[0086] The disclosure has been described in an illustrative manner, and it is understood that the terminology used is intended to be descriptive rather than limiting. In light of the above teachings, many modifications and variations of the present disclosure are possible, and the disclosure may be practiced otherwise than as specifically described. It is understood that the use of the terms "first" and "second" are merely identifying and do not necessarily indicate priority.

[0087] According to the present invention, a system is provided comprising a computer having a processor and a memory, the memory having instructions stored thereon that are executable by the processor to: detect an entity within a specified distance from a vehicle; output a first audio external to the vehicle in response to detecting the entity; classifying the entity based on data collected about the entity after outputting the first audio; and outputting a second audio based on classifying the entity.

[0088] According to one embodiment, the instructions further include instructions to enable a vehicle feature based on classifying the entity.

[0089] According to one embodiment, the detection of the entity within the specified distance from the vehicle is based on image data.

[0090] According to one embodiment, the second audio occurs in response to communication by the entity.

[0091] According to one embodiment, the second audio is selected from a stored plurality of phrases.

[0092] According to one embodiment, the instructions further include instructions to identify the entity based on stored data collected from the entity prior to outputting the first audio.

[0093] According to one embodiment, the instructions further include instructions to classify the entity based on a plurality of images collected at different times.

[0094] According to one embodiment, the instructions further include instructions to classify the entity based on a sound of the entity.

[0095] According to one embodiment, the classification of the entity is at least one of threatening and non-threatening.

[0096] According to one embodiment, the first audio is output in a first language and the second audio is output in a second language based on detecting the second language in an entity audio.

[0097] According to one embodiment, the invention is further characterized by detecting a first entity and a second entity within the specified distance from the vehicle, wherein the first entity is prioritized based on a risk assessment.

[0098] According to the present invention, a method includes: detecting an entity within a specified distance from a vehicle; outputting a first audio external to the vehicle in response to detecting the entity; classifying the entity based on data collected about the entity after outputting the first audio; and outputting a second audio based on classifying the entity.

[0099] In one aspect of the invention, the method includes enabling a vehicle feature based on classifying the entity.

[0100] In one aspect of the invention, detecting the entity within the specified distance from the vehicle is based on image data.

[0101] In one aspect of the invention, the second audio occurs in response to communication by the entity.

[0102] In one aspect of the invention, the second audio is selected from a stored plurality of phrases.

[0103] In one aspect of the invention, the method includes identifying the entity based on stored data collected from the entity prior to outputting the first audio.

[0104] In one aspect of the invention, the method includes classifying the entity based on a sound of the entity.

[0105] In one aspect of the invention, the classification of the entity is at least one of threatening and non-threatening.

[0106] In one aspect of the invention, the first audio is output in a first language and the second audio is output in a second language based on detecting the second language in an entity audio.

Claims

[1] Method comprising: Detecting an entity within a specified distance from a vehicle; Outputting a first audio external to the vehicle in response to detecting the entity; Classifying the entity based on data collected about the entity after the first audio is played; and Output a second audio based on the classification of the entity. [2] The method of claim 1, further comprising enabling a vehicle feature based on classifying the entity. [3] The method of claim 1, wherein detecting the entity within the specified distance from the vehicle is based on image data. [4] The method of claim 1, wherein the second audio is in response to a communication by the entity. [5] The method of claim 1, wherein the second audio is selected from a stored plurality of phrases. [6] The method of claim 1, further comprising identifying the entity based on stored data collected from the entity prior to outputting the first audio. [7] The method of claim 1, further comprising classifying the entity based on a plurality of images collected at different times. [8] The method of claim 1, further comprising classifying the entity based on a sound of the entity. [9] The method of claim 1, wherein the classification of the entity is at least one of threatening and non-threatening. [10] The method of claim 1, further comprising outputting the first audio in a first language and outputting the second audio in a second language based on detecting the second language in an entity audio. [11] The method of claim 1, further comprising detecting a first entity and a second entity within the specified distance from the vehicle, wherein the first entity is prioritized based on a risk assessment. [12] The method of claim 1, wherein the second audio is selected from a stored plurality of phrases based on a verbal response by the entity. [13] The method of claim 1, further comprising classifying the entity based on image data. [14] The method of claim 1, further comprising assigning a risk score to the entity based on image data. [15] A remote computer comprising a processor and a memory, the memory storing instructions executable by the processor to perform the method of any one of claims 1-14.

Images