System and method for enhanced vehicle visualization

Abstract

The present disclosure provides "Systems and methods for enhanced vehicle visualization." Systems and methods are provided for enhancing the visualization of a vehicle in an environment. The vehicle can use internal and external sensors to continuously monitor people in its vicinity to determine their trajectories and whether some people are currently fully engaged, and if so, how engaged they are. If the vehicle determines that a person's current trajectory is likely to result in a physical interaction between the vehicle and the person, and if the level of engagement exceeds a certain threshold, the vehicle can output a direct alert to the person. The vehicle can then continue to monitor the person's behavior to determine whether the alert resulted in a change in the person's behavior such that the likelihood of the physical interaction is reduced or eliminated. If not, the vehicle can modify the alert and keep outputting the modified alert until a change in behavior is detected.

Description

Technical Field

[0001] This disclosure relates to the field of vehicles. Specifically, embodiments of this disclosure relate to methods and systems for establishing and / or enhancing the presence and / or visualization of a vehicle for people around the vehicle. Background Technology

[0002] Compared to traditional internal combustion engine vehicles, some vehicles (such as electric vehicles or even some newer internal combustion engine-based vehicles) produce less noise. For example, electric vehicles may be quieter at low speeds, making them more difficult for people, especially those with visual impairments, to detect their approach. People often rely on the sound of approaching vehicles to judge their distance and speed. Quiet electric vehicles reduce this natural auditory cue.

[0003] People using smartphones, headphones, or other devices may be so engrossed in their surroundings that they are less likely to notice oncoming vehicles. The quietness of electric vehicles may exacerbate this problem. Without auditory or visual cues, a person who is fully absorbed in their surroundings is less likely to look up or become aware of their environment. Summary of the Invention

[0004] This disclosure describes systems and methods for improving the perception of the presence of vehicles in the environment. Specifically, embodiments of this disclosure relate to systems and methods for outputting target alerts to people near a vehicle to make those people aware of the vehicle's presence.

[0005] Embodiments of this disclosure provide a method. The method includes: determining the presence of a person in the environment surrounding a vehicle, and determining, based on the person's trajectory, that the person may be present within the interaction zone of the vehicle. The method further includes: determining that the person is currently fully attentive; determining the person's level of attentiveness; determining that the level of attentiveness exceeds a threshold; and outputting an alarm in a direction toward the person.

[0006] In another scenario, a method is provided that may include: determining an interaction zone associated with a vehicle; and determining that a person near the vehicle is likely to be present in the interaction zone in the near future. The method may further include: determining that the person is currently attentive; determining environmental conditions near the vehicle; generating an alarm based on the environmental conditions and the person's attentiveness; and outputting the alarm.

[0007] In another embodiment, a vehicle is provided, the vehicle comprising: one or more sensors; an interaction and predictive analytics unit including a controller and coupled to the one or more sensors; an alarm generation and output unit coupled to the interaction and predictive analytics unit; and a memory device coupled to the controller and storing instructions. The controller can execute the instructions, which cause the controller to use the one or more sensors to determine the presence of a person in the environment surrounding the vehicle, determine, based on the person's trajectory, that the person may be present within the interaction zone of the vehicle, determine that the person is currently attentive, determine the person's attentiveness level, determine that the attentiveness level exceeds a threshold, and cause the alarm generation and output unit to output an alarm in the direction of the person.

[0008] These and other advantages of this disclosure are provided in detail herein. Attached Figure Description

[0009] Specific embodiments are illustrated with reference to the accompanying drawings. The same reference numerals may be used to indicate similar or identical items. Various embodiments may utilize elements and / or components other than those shown in the drawings, and some elements and / or components may not be present in various embodiments. Elements and / or components in the drawings are not necessarily drawn to scale. Throughout this disclosure, singular and plural terms may be used interchangeably depending on the context.

[0010] Figure 1 An environment in which embodiments of the present disclosure may be implemented is shown.

[0011] Figure 2 A block diagram of a vehicle according to an embodiment of the present disclosure is shown.

[0012] Figure 3 Example scenarios in which embodiments of the present disclosure may be implemented are shown.

[0013] Figure 4 Scenario 400 is shown according to an embodiment of the present invention.

[0014] Figure 5 A system according to an embodiment of the present disclosure is shown.

[0015] Figure 6 A flowchart of a process according to an embodiment of the present disclosure is shown.

[0016] Figure 7 A flowchart according to another embodiment of this disclosure is shown.

[0017] Figure 8 A flowchart of yet another embodiment according to this disclosure is shown.

[0018] Figure 9 A flowchart of yet another embodiment according to this disclosure is shown.

[0019] Figure 10 A block diagram of a server according to an embodiment of the present disclosure is shown. Detailed Implementation

[0020] The present disclosure will now be described more fully with reference to the accompanying drawings, in which exemplary embodiments of the disclosure are shown, and which are not intended to be limiting.

[0021] Figure 1 An environment 100 in which embodiments of the present disclosure may be implemented is shown. Vehicle 102 may be any passenger or commercial vehicle, such as a car, truck, tanker, bus, etc. Vehicle 102 may be an autonomous vehicle or a vehicle requiring human driving. Environment 100 may also include a control server 104. Control server 104 may be part of a cloud-based computing infrastructure and may be associated with and / or include a Telematics Service Delivery Network (SDN) that provides digital data services to vehicle 102. References are made below. Figure 10 Provide details of control server 104.

[0022] Environment 100 may also include a user device 112. User device 112 may be a mobile phone, tablet, personal computer, smart key fob, etc. User device 112 may be associated with a user 110 of vehicle 102. User 110 may be the driver of vehicle 102 or a passenger in vehicle 102. User device 112 may receive information from vehicle 102 and / or control server 104. User device 112 may have a dedicated application installed thereon, which can interface with vehicle 102 to download and display various types of vehicle-generated information and other control data. In one embodiment, vehicle 102 may communicate directly with user device 112 to send and receive data without network 108 and / or server 104.

[0023] Environment 100 may also include network 108. Network 108 illustrates an example communication infrastructure in which connected devices discussed in various embodiments of this disclosure may communicate. Network 108 may be and / or include the Internet, a private network, a public network, or other configurations operating using any one or more known communication protocols such as Transmission Control Protocol / Internet Protocol (TCP / IP), Bluetooth, etc. ® Bluetooth ®Low Energy (BLE), Wi-Fi based on the IEEE 802.11 standard, Ultra Wideband (UWB), and cellular technologies such as Time Division Multiple Access (TDMA), Code Division Multiple Access (CDMA), High Speed ​​Packet Access (HSPDA), Long Term Evolution (LTE), Global System for Mobile Communications (GSM), and 5G are just a few examples.

[0024] Vehicle 102 may include multiple units, including but not limited to an automotive computer, a vehicle control unit (VCU), and a detection unit. See below for reference. Figure 2 Details of vehicle 102 are provided.

[0025] Figure 2 A block diagram of a vehicle 102 in which embodiments of the present disclosure may be implemented is shown. The vehicle 102 may include multiple units, including but not limited to an automotive computer 208, a vehicle control unit (VCU) 210, and an infotainment unit 238. The VCU 210 may include multiple electronic control units (ECUs) 214 communicating with the automotive computer 208.

[0026] In some embodiments, a user device such as a mobile phone, laptop computer, smart key fob, etc., can be configured to connect to the vehicle computer 208. The user device can communicate via one or more wireless connections, and / or via near field communication (NFC) protocol, Bluetooth, etc. ® Protocols, Wi-Fi, Ultra-Wideband (UWB), and other possible data connectivity and sharing technologies can be used to directly connect to vehicle 102.

[0027] According to this disclosure, the vehicle computer 208 can be installed anywhere in the vehicle 102. The vehicle computer 208 may be or include an electronic vehicle controller having one or more processors 202, one or more memory devices 204, and one or more transceivers 206.

[0028] Processor 202 may be configured to communicate with one or more memory devices (e.g., memory 204 and / or memory) of a corresponding computing system. Figure 2The processor 202 may communicate with one or more external databases (not shown in the diagram). The processor 202 may utilize the memory 204 to store programs and / or data in code form to perform operations according to this disclosure. The memory 204 may be a non-transitory computer-readable storage medium or memory storing vehicle control program code. The memory 204 may include any or a combination of volatile memory elements (e.g., dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), etc.) and may include any one or more non-volatile memory elements (e.g., erasable programmable read-only memory (EPROM), flash memory, electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), etc.). In some embodiments, the memory 204 may include modules 245 that may implement various embodiments of this disclosure. Modules 245 may include instructions that can be executed by the processor 202 to implement various embodiments of this disclosure.

[0029] The vehicle computer 208 may also include a transceiver 206. The transceiver 206 may be configured to receive information / input from one or more external devices or systems (e.g., user device 208, external server, etc.). Furthermore, the transceiver 206 may transmit notifications, requests, signals, etc., to external devices or systems. Additionally, the transceiver 206 may be configured to receive information / input from vehicle components (such as vehicle sensing system 232, one or more ECUs 214, etc.). Furthermore, the transceiver 206 may transmit signals (e.g., command signals) or notifications to vehicle components such as BCM 220, infotainment system 238, etc.

[0030] In some embodiments, VCU 210 may share a power and / or communication bus with vehicle computer 208 and may be configured and / or programmed to coordinate data between vehicle systems, connected servers, etc. VCU 210 may include or communicate with any combination of ECUs 214, such as BCM 220, Engine Control Module (ECM) 222, Transmission Control Module (TCM) 224, Telematics Control Unit (TCU) 226, Driver Assist Technology (DAT) Controller 228, etc. VCU 210 may also include and / or communicate with a Vehicle Sensing System (VPS) 230, which may connect to and / or control one or more vehicle sensing systems 232. The vehicle sensing system 232 may include one or more vehicle sensors, including but not limited to radio detection and ranging (RADAR or "radar") sensors configured to use radio waves to detect and locate objects inside and outside the vehicle 102, seating area latch sensors, seating area sensors, light detection and ranging ("LiDAR") sensors, door sensors, proximity sensors, temperature sensors, wheel sensors, one or more ambient weather or temperature sensors, interior and exterior cameras, steering wheel sensors, etc. Sensors as part of the vehicle sensing system 232 may be coupled to the vehicle 102 at one or more locations and in one or more configurations. For example, various sensors of the vehicle sensing system 232 may be integrated into various subsystems of the vehicle 102 (such as doors, mirrors, roof, etc.) or attached to the vehicle 102 using suitable mounting mechanisms. In some embodiments, various sensors of the vehicle sensing system 232 may be located at the front, rear, sides, top, bottom, and underside of the vehicle 102. The location of the sensors may depend on their function. For example, sensors monitoring the area beneath the vehicle can be attached to the underside of vehicle 102, while sensors monitoring areas on either side of vehicle 102 can be mounted or integrated into the doors of vehicle 102. Vehicle sensing system 232 may also include one or more road noise sensors, such as accelerometers coupled to various mechanical components and / or systems of vehicle 102. Those skilled in the art will recognize that sensors can be coupled to the vehicle in various different ways and locations besides those mentioned above.

[0031] In some embodiments, VCU 210 can control vehicle operation aspects and implement one or more instruction sets received from server 104, user device 112, or from one or more instruction sets stored in memory 204.

[0032] TCU 226 can be configured and / or programmed to provide vehicle connectivity to wireless computing systems on and outside the vehicle 102, and may include a navigation (NAV) receiver 234 for receiving and processing GPS signals, BLE ® Module (BLEM) 236, Wi-Fi transceiver, UWB transceiver and / or may be configured to be used in vehicle 102 with other systems (e.g., vehicle key fob). Figure 2 Other wireless transceivers (not shown in the image), external servers, user devices, etc., for wireless communication (including cellular communication) between computers and modules. Figure 2 (Not shown in the image). TCU 226 can communicate with ECU 214 via a wired or wireless bus. In some respects, TCU 226 can be configured to determine the real-time vehicle geolocation, for example, via NAV receiver 234.

[0033] ECU 214 can control various aspects of vehicle operation and communication using inputs from the human driver, inputs from the vehicle computer 208, and / or wireless signal inputs received from other connected devices (such as server 206) via a wireless connection.

[0034] The BCM 220 typically integrates sensors, vehicle performance indicators, and variable reactors associated with vehicle systems. It may also include a processor-based power distribution circuitry that controls functions associated with the vehicle body, such as lights, windows, safety devices, cameras, audio systems, wipers, door locks and entry controls, and various comfort controls. The BCM 220 can also operate as a gateway for bus and network interfaces to communicate with remote ECUs ( Figure 2 (Not shown in the image) Interaction.

[0035] The DAT controller 228 and / or the autonomous driving system 240 can provide Level 1 to Level 5 automated driving and driver assistance functionality, which may include features such as active parking assist, vehicle reversing assist, and / or adaptive cruise control. The DAT controller 228 can also provide various aspects of user and environmental inputs that can be used for user authentication.

[0036] In some embodiments, the vehicle computer 208 may be connected to the infotainment system 238 (or vehicle human-machine interface (HMI)). The infotainment system 238 may include a touchscreen interface portion and voice recognition features, enabling it to identify the user's biometrics based on facial recognition, voice recognition, fingerprint recognition, or other biometric methods. In other aspects, the infotainment system 238 may also be configured to receive user commands via the touchscreen interface portion and / or output or display notifications, navigation maps, etc., on the touchscreen interface portion. In some embodiments, the user device 112 may provide an HMI interface.

[0037] The computing system architecture of the automotive computer 208 and / or VCU 210 can omit certain computing modules. This should be easily understood. Figure 2 The computing environment depicted herein is an example of possible implementations according to this disclosure and should therefore not be considered restrictive or exclusive.

[0038] Vehicle 102 may include an interaction evaluation system 242. The interaction evaluation system 242 may receive data from one or more sensors of the vehicle and / or one or more sensors in the external environment where the vehicle is operating. Based on the received data, the interaction evaluation system 242 may determine whether anyone is near the vehicle who may have physical interaction with vehicle 102. This determination may be accomplished by predicting the trajectories of one or more users based on data received from the sensors. The interaction evaluation system 242 may then generate an alarm, such as a visual or auditory alarm, and send the alarm to the one or more people predicted to have physical interaction with vehicle 102. In one example, the alarm may include light output, sound output, or a message sent to the user device of one or more people that causes the user device to output an alarm. The interaction evaluation system 242 may include a memory programmed with dedicated instructions capable of performing the functions detailed above. In some embodiments, the interaction evaluation system 242 may be integrated into VCU 210.

[0039] In addition to the components mentioned above, vehicle 102 may also have numerous mechanical systems and subsystems. A chassis, frame, or unibody construction may form the backbone of vehicle 102 and support the body and other components of vehicle 102. Vehicle 102 may include an engine that converts fuel into mechanical power to propel the vehicle forward. The engine includes various components such as the engine block, pistons, valves, and spark plugs. Vehicle 102 may also include a transmission system. The transmission system transmits power from the engine to the wheels. It includes a clutch, gearbox, drive shaft, differential, and other components. The transmission adjusts the power output to suit the vehicle's speed and load. Vehicle 102 may also include a suspension system. The suspension system absorbs shocks and maintains contact between the tires and the road, providing a smooth ride. It includes components such as springs, shock absorbers, and linkages. Vehicle 102 also includes a vehicle stopping system that allows the driver to decelerate or stop vehicle 102. It includes components such as pedals, master cylinders, lines, and bushings or shoes. Vehicle 102 also includes a steering system that allows the driver to guide the vehicle. The steering system includes components such as a steering wheel, steering column, rack and pinion, and tie rods. Vehicle 102 may also include an exhaust system for removing and filtering exhaust gases produced by the engine. It includes an exhaust manifold, catalytic converter, muffler, and exhaust tailpipe, among other components. Vehicle 102 also includes a cooling system to prevent overheating of the engine and / or battery. It includes components such as a radiator, water pump, thermostat, and coolant. Vehicle 102 may also include a cooling system for storing fuel and supplying fuel to the engine. It includes a fuel tank, fuel pump, fuel filter, and fuel injectors. The electrical system of vehicle 102 powers the vehicle's electrical components. It may include a battery, alternator, starter motor, and wiring. The heating, ventilation, and air conditioning (HVAC) system controls the temperature inside vehicle 102. It includes a heater core, blower motor, and air conditioning compressor. In some embodiments, the vehicle may be an electric vehicle (EV) or a hybrid vehicle, and in either case, some of the aforementioned components will be replaced by an electric motor and a high-voltage battery. All the mechanical components working together ensure optimal operation of vehicle 102.

[0040] Figure 3An example scenario 300 in which embodiments of the present disclosure may be implemented is shown. Scenario 300 depicts a typical busy road intersection in an urban area. Scenario 300 may include a vehicle 102 traveling along the road. As the vehicle 102 travels along the road, it can continuously monitor a first zone 302 adjacent to the vehicle 102. The vehicle may use one or more external sensors to monitor the first zone 302. Multiple people may be present near the vehicle. Each of these people may be engaged in some activity and may be in various states of concentration. For example, person 306 may be crossing the road while looking away from vehicle 102. When person 308 is attempting to cross the road, he / she may be talking on his / her mobile phone. Person 310 may be standing on the sidewalk and may attempt to cross the road in the near future. People 312 and 314 may be engrossed in conversation and may attempt to cross the road while participating in the conversation. Thus, multiple people in various states of concentration may be present near the vehicle. At any given point in time, one or more of these people may enter a second zone 304. The second zone 304 represents the area near vehicle 102, where there is a high probability that one or more people may have physical contact with vehicle 102. In some embodiments, the second zone 304 is smaller than the first zone 302 and may be entirely within the first zone 302. In other cases, the second zone 304 may at least partially overlap with the first zone 302. In situations where vehicle 102 is traveling at low speed and / or is generally silent in its operation, one or more of the people present in scenario 300 may not be aware of the presence of vehicle 102. In some cases, any of the aforementioned people may inadvertently attempt to cross the road when the intersection traffic light is red, indicating that users should not cross.

[0041] Vehicle 102 continuously monitors a first zone 302 and one or more people in its vicinity. As part of the monitoring of people, the vehicle can determine the predicted trajectory of each of these people, and based on the predicted trajectory, vehicle 102 can determine whether any of these people might occupy the second zone 304 at the same time that vehicle 102 anticipates being in the second zone 304. For example, consider person 308 talking on his mobile phone and about to cross the road. In this case, vehicle 102 can determine the expected time that vehicle 102 will be in the second zone 304 based on its current speed and heading. Similarly, vehicle 102 can determine when person 308 might be in the second zone 304 based on person 308's trajectory. Since person 308 is talking on his mobile phone, vehicle 102 can also (e.g., based on the person's image data) determine that person 308 is attentive. Based on the above, the vehicle can generate an alarm and direct it to person 308 to make person 308 aware of the presence of vehicle 102. The alarm may be in the form of audio output, light output, and / or a message to the person's mobile phone. In some embodiments, the vehicle can determine the level of attention of person 308 before outputting an alarm towards person 308. If the level of attention is greater than a certain threshold, the vehicle can output an alarm. This ensures that only the person who most needs the alarm will receive it, without disturbing others near the vehicle 102. In some embodiments, the alarm can be sent to one or more users who may not be in the second zone 304. In other embodiments, the vehicle 102 can dynamically adjust the second zone 304 if there are objects on the road that may prevent a user from entering the second zone 304.

[0042] Figure 4A scenario 400 according to an embodiment of this disclosure is illustrated. In scenario 400, a person 404 is entering a road intersection, and a vehicle 102 has the right-of-way. The person 404 is busy interacting with his mobile device 406 and is unaware of the approaching vehicle 102. The vehicle 102 can detect the presence of the person 404 using data from one or more of the vehicle's sensors (such as cameras, lidar, radar, ultrasonic sensors, etc.). Subsequently, the vehicle 102's interaction evaluation system 242 can process the sensor data using machine learning and computer vision techniques (such as object detection algorithms (e.g., YOLO, SSD, or Faster R-CNN) and skeleton tracking algorithms) to detect the presence of the user 404. The vehicle 102 can then track the movement of the person 404 over time to estimate their speed and direction (e.g., using Kalman filters or optical flow techniques). Based on the determined speed and direction, the interaction evaluation system 242 can estimate the person's likely movement into area 402. For example, the interaction evaluation system 242 can use physics-based models, behavioral models, or machine learning models to predict the person's trajectory. Then, the interaction assessment system 242 can use the trajectory of person 404, as well as the planned path and current speed of vehicle 102, to determine the likelihood of physical interaction between vehicle 102 and person 404 within area 402. Based on this determination, the interaction assessment system 242 can cause vehicle 102 to output an alarm 408 to person 404.

[0043] Alarm 408 can be in the form of directional audio. For example, one or more speakers of vehicle 102 can use beamforming to output audio in the direction of the user. In this example, multiple speakers of the vehicle can emit sound waves simultaneously. The timing (phase) and amplitude of the sound waves from each speaker are adjusted such that the sound waves combine constructively in the desired direction and destructively elsewhere. Thus, vehicle 102 can focus the audio in the direction of person 404. Person 404 can then hear the sound and become aware of the presence of vehicle 102. In another example, alarm 408 can be in the form of light output. Since vehicle 102 knows the possible location of person 404 based on trajectory determination, the vehicle can activate one or more light-emitting devices to direct light in the direction of the user. This can be achieved by using directional light control technology. Based on the trajectory of person 404, interactive evaluation system 242 can cause the vehicle to operate one or more lights of the vehicle to direct one or more beams of light toward person 404. For example, vehicle 102 may include headlights or other lights mounted to gimbals or rotary actuators. Vehicle 102 can operate one or more of the motorized support and / or rotary actuator to direct light output toward person 404. In some cases, the vehicle can adjust the alarm based on environmental conditions such as ambient light, noise level, etc.

[0044] In another scenario, alarm 408 could take the form of a Bluetooth Low Energy (BLE) advertisement. A BLE advertisement is a short broadcast data packet that may include device information (e.g., name, UUID, address), optional data for pairing or interaction, and application-specific flags or payloads. Vehicle 102 can use techniques such as device filtering and / or Resolved Private Address (RPA) to direct BLE advertisement messages to human mobile device 406. Human mobile device 406 can receive BLE packets and display a message to user 404 warning of the presence of vehicle 102.

[0045] Figure 5 A system 500 according to an embodiment of the present disclosure is illustrated. In one embodiment, the system 500 may only be used in... Figure 2 The system 500 is implemented in vehicle 102. In other embodiments, the system 500 may be partially in vehicle 102 and partially in… Figure 1 The system is implemented in server 104. System 500 can be used to detect the presence of a person, determine their trajectory, determine their level of consciousness / attention, determine the likelihood of contact between the person and the vehicle, generate an alarm for the person, monitor the person's behavior after the alarm is output, and adjust the alarm based on the person's behavior and / or environmental conditions. System 500 may include multiple sensors from the vehicle and sensors external to the vehicle. Sensors may include environmental sensor 502. Environmental sensor 502 may include air quality sensors (e.g., particulate matter (PM) sensors (e.g., PM2.5, PM10), carbon dioxide (CO2) sensors, etc.), weather and atmospheric sensors (e.g., temperature sensors, humidity sensors, atmospheric pressure sensors, wind speed sensors, wind direction sensors, rain sensors (rain gauges), snow depth sensors, solar radiation sensors (solar intensity meters), UV index sensors, lightning detectors, cloud cover sensors, etc.), light and radiation sensors (e.g., light intensity sensors (illuminometers), ultraviolet (UV) sensors, infrared (IR) sensors, visible spectrum sensors, etc.), noise and vibration sensors (e.g., sound level meters, microphones for noise monitoring, vibration sensors (accelerometers, seismographs), seismometers (earthquake monitoring), etc.) and / or motion and position sensors. System 500 may also include motion sensor 504. Motion sensor 504 may include passive infrared (PIR) sensors, ultrasonic sensors, microwave sensors, radar, proximity sensors, vibration sensors, inertial sensors, optical sensors, microelectromechanical systems (MEMS) sensors, etc.

[0046] System 500 may also include a motion sensor 506. Image sensor 506 may include a charge-coupled device (CCD) sensor, a complementary metal-oxide-semiconductor (CMOS) sensor, a time-of-flight (ToF) sensor, an IR sensor, a UV sensor, a visible spectrum sensor, a lidar, etc. System 500 may also include an audio sensor 508. Audio sensor 508 may include a microphone, a MEMA microphone, a fiber optic microphone, a sound level meter, a directional microphone, an omnidirectional microphone, an ultrasonic sensor, an acoustic emission sensor, a bioacoustic sensor, etc. Sensors 502 to 508 may also include sensors located outside the vehicle in the environment surrounding the vehicle. For example, one or more infrastructure sensors present in the environment, such as cameras and / or motion sensors, traffic light sensors, etc.

[0047] System 500 may include an interaction prediction and analysis unit 510. The interaction prediction and analysis unit 510 may receive data from all different sensors 502 to 508. The interaction prediction and analysis unit 510 may use vehicle-to-infrastructure (V2X) communication protocols to receive data from sensors located in the vehicle's external environment. The interaction prediction and analysis unit 510 may use various sensor data to determine the presence of one or more people near the vehicle, determine the trajectory of one or more people, and determine the perception / attention level of one or more people. For example, sensor data may be used to monitor a person's behavior over time. If it is determined that a person has been watching their moving device for a period of time without looking elsewhere, system 500 may infer that the person is highly attentive. Thresholds for attentiveness levels may be determined based on the analysis of historical data from multiple vehicles under various conditions. The interaction prediction and analysis unit 510 may also determine the likelihood of interaction between a person and the vehicle based on the vehicle's current speed and heading, as well as the person's trajectory. If the interaction prediction and analysis unit 510 determines that physical contact between the user and the vehicle may occur in the near future, the interaction prediction and analysis unit 510 may send a message indicating the physical contact to the alarm generation and output unit 512.

[0048] The alarm generation and output unit 512 then determines the current environmental conditions and / or characteristics associated with the user to determine the type of alarm to be output. For example, if it is nighttime, the alarm generation and output unit 512 may determine that light output is optimal for alerting the user. If environmental sensors indicate that the environment is noisy, outputting an audio alarm may not be optimal, or it may be necessary to output an audio alarm with a high amplitude so that a person can hear the sound amidst the noise. In another case, if the environment is too bright, outputting light may be less effective than outputting sound. In other words, the alarm generation and output system monitors the environmental conditions outside the vehicle and dynamically adjusts the type of alarm output by the vehicle. This ensures that the optimal type of alarm is output so that the intended person can receive the alarm and take appropriate action. In some cases, the system 500 also includes a human behavior monitoring unit 514. The human behavior monitoring unit 514 can track the person who is the recipient of the alarm output by the alarm generation and output unit 512. The human behavior monitoring unit 514 detects the person's movements after an alarm has been output to determine if there has been a change in the person's behavior. In some cases, the person receiving the alarm may adjust his / her trajectory so that they move away from the vehicle's path, thereby eliminating the possibility of contact. In other cases, even after an alarm is issued, the person can continue their current trajectory. This could be because the person did not receive an alarm notification. The human behavior monitoring unit 514 provides this feedback to the alarm generation and output unit 512. The alarm generation and output unit 512 can modify the alarm and output the modified alarm. This feedback and dynamic adjustment of the alarm process can continue until the person changes their behavior. The modified alarm may include different types of alarms (e.g., light and sound), alarms output at different frequencies and / or amplitudes, alarms output at different rates, etc.

[0049] Figure 6This is a flowchart of process 600 according to an embodiment of the present disclosure. Process 600 may be performed solely by system 500 of vehicle 102 or by vehicle 102 in conjunction with server 104. At step 602, the vehicle may receive data from one or more sensors. As explained above, the one or more sensors may include sensors of the vehicle as well as sensors outside the vehicle and as part of the environment surrounding the vehicle. At step 604, the presence of one or more people near the vehicle is detected using data from the one or more sensors. Additionally, the position of each of the one or more people relative to the vehicle may be determined. Subsequently, at step 606, the trajectory of the one or more people is determined using any of the techniques described above. Based on the predicted trajectories of the one or more people and the trajectory of the vehicle, at step 608, the vehicle may also determine an area or region in which interaction between the one or more people and the vehicle may occur. Subsequently, at step 610, the vehicle may determine that a subset of one or more people may be present in the area while the vehicle may be in the area. This subset of people is the most likely to have physical interaction with the vehicle. At step 612, the vehicle may generate an alarm indicating the presence of the vehicle. As explained above, the alarm can be in the form of light output, audio output, or a message that can be received by a device associated with each person in the human subset. At step 614, the vehicle can output the alarm in the direction toward the human subset using any of the techniques described above. The human subset can then receive the alarm and change their trajectory to move away from the vehicle's path, thereby preventing potential physical interaction with the vehicle.

[0050] Figure 7 This is a flowchart of process 700 according to another embodiment of this disclosure. Process 700 may be performed solely by system 500 of vehicle 102 or by vehicle 102 in conjunction with server 104. At step 702, the vehicle may receive data from multiple sensors. The multiple sensors may include sensors of the vehicle as well as sensors located outside the vehicle and as part of the environment surrounding the vehicle. The vehicle may use the data from the sensors to determine the presence of multiple people near the vehicle. The data from the sensors may also be used to determine the predicted trajectory of each of the multiple people near the vehicle. Subsequently, based on the trajectory of each of the multiple people and the trajectory and speed of the vehicle, at step 704, the vehicle may determine the person among the multiple people who may have physical interaction with the vehicle. Additionally, the vehicle may determine whether this person is currently engrossed in something (e.g., talking on a mobile phone). Subsequently, at step 706, the vehicle may determine whether the probability / likelihood of the person having physical interaction with the vehicle is greater than a certain threshold. If the probability is less than the threshold, process 700 may return to step 702, and the vehicle may continue to monitor this person using real-time data from the sensors.

[0051] If, at step 706, it is determined that the probability of physical interaction between the person and the vehicle is greater than a threshold, the vehicle may output a directional alarm toward the person at step 708. Outputting the directional alarm makes the person aware of the vehicle's presence. In some embodiments, the vehicle may also output a directional alarm if it is determined that the person is currently attentive. At step 710, the vehicle may monitor the person's behavior after outputting the alarm to determine if the person has received the alarm. Real-time data from one or more sensors may be used to continuously monitor the person's behavior. At step 712, the vehicle may determine if the person has changed their behavior such that the probability of physical interaction with the vehicle has decreased below a threshold. For example, if the person was busy talking on their mobile device while walking, and then stopped walking and looked up / around after outputting the alarm, this could indicate that the person has received the alarm. Other behavioral changes may also indicate that the person has received the alarm. At step 714, if it is determined that the person has changed their behavior to reduce the probability of physical interaction with the vehicle, the vehicle may stop outputting the alarm.

[0052] If it is determined at step 714 that the person has not changed their behavior, the vehicle can infer that the person has not received an alarm. In this case, the vehicle can modify the alarm based on the current environmental conditions and output a modified alarm. Any of the techniques described above can be used to modify the alarm. Once the modified alarm has been output, the vehicle can monitor the person's behavior again, and process 700 can return to step 710. The process can iteratively execute steps 710, 712, and 716 until the person modifies their behavior to reduce or eliminate the probability of physical interaction with the vehicle.

[0053] Figure 8 This is a flowchart of process 800 according to yet another embodiment of the present disclosure. Process 800 may be performed solely by system 500 of vehicle 102 or by vehicle 102 in conjunction with server 104. At step 802, the system may detect the presence of one or more people in the external environment of the vehicle and near the vehicle. The system may use sensor data from sensors inside and / or outside the vehicle to detect the presence of people. The system may also determine an interaction area associated with the vehicle at step 804. In one embodiment, the interaction area may be similar to that described above. Figure 3 The described area is 304. If the vehicle is in motion, the interaction area can change continuously, and the system can determine the area in real time and continuously update the interaction area based on the vehicle's speed and direction. At step 806, the vehicle can determine that at least one of the detected people may be present in the interaction area in the near future. The system can make this determination based on the person's current trajectory and the vehicle's trajectory.

[0054] At step 808, the system can determine the current environmental conditions near the vehicle. As mentioned above, environmental conditions may include ambient light, ambient noise, the person's level of concentration, etc. Based on the current environmental conditions, the system can generate an appropriate alarm at step 810. For example, if it is very bright outside, the system may prefer to generate an audio alarm rather than a visual alarm because a person is more likely to hear an audio alarm under given conditions. In other words, given the current environmental conditions, the system will generate the alarm that is most likely to be received / perceived by a person. At step 812, the system can then output the alarm in the direction of the user. Any of the aforementioned techniques for outputting directional alarms (e.g., beamforming, directional light, BLE advertising messages, etc.) can be used to direct the alarm in the direction of the person. This also ensures that others who may be near the vehicle but are not expected to be in the interaction zone of the vehicle are not unduly disturbed by the alarm. This type of directional alarm ensures that only the most relevant person receives the message.

[0055] At step 814, the system can monitor the person's behavior after outputting an alarm to determine if there has been any change in the person's behavior. For example, after receiving an alarm, the person may change their trajectory, making it less likely that they are within the vehicle's interaction zone. If it is determined at step 814 that the person has not changed their behavior, the system can continue outputting the alarm. For example, the person may continue their previous trajectory, indicating that the person has not yet received an alarm. In one embodiment, steps 812 and 814 may be repeated several times until the person changes their behavior. If it is determined at step 814 that the person has changed their behavior, the system can then determine at step 816 whether the change in behavior could have caused the person to move away from or out of the interaction zone. For example, as determined at step 814, the person may have changed their behavior, but the change in behavior may not be due to the alarm but due to some other reason. Therefore, although the person may have changed their behavior, they may still have continued their previous trajectory, indicating that the change in behavior is unlikely to be due to the alarm. In the case where it is determined that the user's behavior change is unlikely to reduce the chance of physical interaction with the vehicle, the system can modify the alarm at step 818 and output the modified alarm. For example, the system can change the type of alarm (sound vs. light vs. BLE message), change the alarm intensity (e.g., sound amplitude or light brightness), or change the alarm frequency / rate, etc. In one embodiment, steps 812, 814, 816, and 818 can be repeated until it is determined that the person's behavioral change is caused by the alarm and that the change is likely to cause the person to be outside the interaction area. Once it is determined that the person is unlikely to be in the interaction area, the system can stop outputting the alarm at step 820.

[0056] Figure 9This is a flowchart of process 900 according to another embodiment of the present disclosure. Process 900 may be performed solely by system 500 of vehicle 102 or by vehicle 102 in conjunction with server 104. At step 902, the system may use any of the techniques described above to detect the presence of a person near the vehicle. At step 904, the system may (e.g., based on the trajectory of the person and the trajectory of the vehicle) determine whether the person is likely to have physical interaction with the vehicle in the near future. At step 906, the system may determine that the person is currently engrossed in something. This determination may be made using image data captured by the vehicle. For example, the person may be interacting with their mobile device, engrossed in a conversation with another person, wearing headphones, etc. In other cases, the person may be carrying a cane, which may indicate a physical disability that may affect the person's ability to perceive their surroundings. For example, a person carrying a white cane may indicate that the person has a visual impairment, and a person carrying a white cane with a red band or base may indicate that the person has both visual and hearing impairments, etc.

[0057] At step 908, the system can determine the level of concentration associated with the user and whether that level of concentration is greater than or less than a specific threshold. For example, it might be inconvenient to issue an alert if someone is only temporarily attentive but otherwise aware of their surroundings (e.g., someone is temporarily looking at their mobile device but otherwise aware of their surroundings). A threshold can be set for the level of concentration. For example, the threshold could be based on duration. For example, if a person is considered to be attentive for more than five seconds, the system can determine that the person's level of concentration exceeds the threshold. In other cases, the threshold could be based on a person's physical characteristics. For example, if the person is carrying one of the aforementioned canes, the system can determine that the person's level of concentration is greater than the threshold. Those skilled in the art will recognize that various other forms of thresholds can be implemented. In another example, historical data on indicators of concentration can be used to determine an appropriate threshold. For example, historical data on person-related behavioral indicators (e.g., reaction time, eye movements, body language, speech patterns, etc.), contextual cues (e.g., task execution, environmental scanning, engagement testing, etc.), environmental context (e.g., ambient noise, ambient light, etc.), facial recognition, attention tracking software data, and / or physiological indicators (e.g., heart rate variability, eye tracking, EEG activity, skin conductance, etc.) can be used to generate machine learning models. The model can then output categorical labels or continuous scores representing levels of concentration. For example, levels of concentration can be defined as focused, moderately focused, and highly focused. The machine learning model used can be a supervised model, an unsupervised model, a deep learning model, etc. The model can be deployed in vehicles to reduce deterministic waiting times.

[0058] If at step 908 it is determined that the level of concentration does not exceed the threshold, the system can continue to monitor the person and maintain real-time determination of their concentration level. If at step 908 it is determined that the person's concentration level exceeds the threshold, the system can determine the current environmental conditions at step 910. At step 912, the system can generate an alarm based on the current environmental conditions and the concentration level exceeding the threshold. Subsequently, at step 914, the system can output an alarm in the direction of the person. In some embodiments, the system can perform additional actions, such as those described above. Figure 8 The actions described in steps 814 to 820.

[0059] In some embodiments, the interactive evaluation system 242 can be enabled or disabled by the vehicle user based on time and / or driving conditions. In other embodiments, the rate of monitoring people and / or outputting alarms can be scaled up or down based on environmental conditions, local rules, etc. In some embodiments, people can subscribe to an alarm service (e.g., via an application on a mobile device). Entities can operate such an alarm service using server 104 and / or vehicle 102. In any of the above scenarios, anyone who has subscribed to such a service can receive alarms on their mobile device. If a person has subscribed to such an alarm service, they can customize the alarm service for a more personalized user experience by setting their alarm receiving preferences in their profile information. A person can register their mobile device with a service operator, and the service operator can use the mobile device information to send personalized alarms to that person (e.g., targeted BLE advertising). In some embodiments, multiple types of alarms can be sent to a person of interest simultaneously. For example, the system can output directional sound toward a person and send BLE messages to that person's mobile device. This increases the likelihood that the person will receive an alarm.

[0060] Figure 10 An example control server 1000 is depicted according to one or more exemplary embodiments of the present disclosure, on which any of one or more technologies (e.g., methods) can be performed or on which the methods described above in conjunction with vehicle 102 can be performed (e.g., Figure 1A block diagram of the control server 1000. In other embodiments, server 1000 may act as a standalone device or may be connected (e.g., networked) to other servers. In a networked deployment, server 1000 may operate as a server machine, a client machine, or both in a server-client network environment. In the example, server 1000 may act as a peer-to-peer (P2P) (or other distributed) network environment. Server 1000 may be a personal computer (PC), tablet PC, set-top box (STB), personal digital assistant (PDA), mobile phone, smart keychain, wearable computing device, network device, network router, switch, or bridge, or any machine capable of executing instructions (continuously or otherwise) specifying actions to be taken by the server (such as a base station). Furthermore, while only a single server is described, the term "server" should also be considered as including any collection of servers that individually or jointly execute a set (or more sets) of instructions for performing any one or more of the methodologies discussed herein, such as those configured for cloud computing, Software as a Service (SaaS), or other computer clusters.

[0061] The examples described herein may include logic or components, modules, or mechanisms, or may operate on logic or components, modules, or mechanisms. A module is a tangible entity (e.g., hardware) capable of performing a specified operation during operation. A module includes hardware. In the examples, the hardware may be specifically configured to perform a specific operation (e.g., hardwired). In another example, the hardware may include a configurable execution unit (e.g., transistor, circuitry, etc.) and a computer-readable medium containing instructions that configure the execution unit to perform a specific task when in operation. The configuration may occur under the guidance of the execution unit or loading mechanism. Thus, when the device is operating, the execution unit is communicatively coupled to the computer-readable medium. In this example, the execution unit may be a member of more than one module. For example, under operation, the execution unit may be configured at one point in time to implement a first module via a first instruction set, and at a second point in time to be reconfigured via a second instruction set to implement a second module.

[0062] Server (e.g., computer system) 1000 may include a hardware processor 1002 (e.g., a central processing unit (CPU), a graphics processing unit (GPU), a hardware processor core, or any combination thereof), main memory 1004, and static memory 1006, some or all of which may communicate with each other via an interconnect (e.g., a bus) 1008. Server 1000 may also include a graphics display device 1010, an alphanumeric input device 1012 (e.g., a keyboard), and a user interface (UI) navigation device 1014 (e.g., a mouse). In this example, the graphics display device 1010, the alphanumeric input device 1012, and the UI navigation device 1014 may be a touchscreen display. Server 1000 may additionally include: a storage device (i.e., a drive unit) 1016; a network interface device / transceiver 1020 coupled to an antenna; and one or more sensors 1028, such as a global positioning system (GPS) sensor, a compass, an accelerometer, or other sensors. Server 1000 may include output controller 1034, such as serial (e.g., Universal Serial Bus (USB)), parallel or other wired or wireless (e.g., infrared (IR)), near field communication (NFC) connections, to communicate with or control one or more peripheral devices (e.g., printers, card readers, etc.).

[0063] Storage device 1016 may include machine-readable medium 1022 thereon storing one or more data structures or instruction sets (e.g., software) embodied or utilized by any or more of the techniques or functions described herein. Instructions may also reside wholly or at least partially within main memory 1004, static memory 1006, or hardware processor 1002 during execution of the instructions by server 1000. In this example, one or any combination of hardware processor 1002, main memory 1004, static memory 1006, or storage device 1016 may constitute a machine-readable medium.

[0064] Although machine-readable medium 1022 is shown as a single medium, the term "machine-readable medium" can include a single medium or multiple media (e.g., a centralized or distributed database, and / or associated caches and servers) configured to store one or more instructions.

[0065] Various embodiments may be implemented wholly or partially in software and / or firmware. This software and / or firmware may take the form of instructions contained in or on a non-transitory computer-readable storage medium. Those instructions may then be read and executed by one or more processors to enable the performance of the operations described herein. The instructions may be in any suitable form, such as, but not limited to, source code, compiled code, interpreted code, executable code, static code, dynamic code, etc. Such computer-readable medium may include any tangible non-transitory medium for storing information in a form readable by one or more computers, such as, but not limited to, read-only memory (ROM); random access memory (RAM); disk storage media; optical storage media; flash memory; etc.

[0066] The term "machine-readable medium" can include any medium having the following properties: capable of storing, encoding, or transporting instructions executable by server 1000; causing server 1000 to perform any or more of the techniques disclosed herein; or capable of storing, encoding, or transporting data structures used by or associated with such instructions. Examples of non-limiting machine-readable media can include solid-state memory as well as optical and magnetic media. In examples, large-scale machine-readable media includes machine-readable media having a plurality of particles having rest masses. Specific examples of large-scale machine-readable media can include non-volatile memory, such as semiconductor memory devices (e.g., electrically programmable read-only memory (EPROM) or electrically erasable programmable read-only memory (EEPROM)) and flash memory devices; disks, such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.

[0067] Instructions can also be transmitted or received over a communication network via a transmission medium using the network interface device / transceiver 1020, utilizing any of several transport protocols (e.g., Frame Relay, Internet Protocol (IP), Transmission Control Protocol (TCP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), etc.). Exemplary communication networks may include local area networks (LANs), wide area networks (WANs), packet data networks (e.g., the Internet), mobile phone networks (e.g., cellular networks), conventional telephone (POTS) networks, wireless data networks (e.g., the IEEE 802.11 series of standards known as Wi-Fi®, the IEEE 802.16 series of standards known as WiMax®), the IEEE 802.15.4 series of standards, and peer-to-peer (P2P) networks, etc. In this example, the network interface device / transceiver 1020 may include one or more physical sockets (e.g., Ethernet sockets, coaxial sockets, or telephone sockets) or one or more antennas for connection to the communication network. In the example, the network interface device / transceiver 1020 may include multiple antennas to communicate wirelessly using at least one of the following: Single-Input Multiple-Output (SIMO) technology, Multiple-Input Multiple-Output (MIMO) technology, or Multiple-Input Single-Output (MISO) technology. The term "transmission medium" should be considered to include any intangible medium capable of storing, encoding, or transmitting instructions for execution by server 1000 and comprising digital or analog communication signals, or other intangible media used to facilitate communication of such software. In various implementations, the operations and processes described and shown above may be implemented or performed in any suitable order as needed. Additionally, in some implementations, at least a portion of the operations may be performed in parallel. Furthermore, in some implementations, fewer or more operations than those described may be performed.

[0068] It should be noted that the vehicle implements and / or performs the operations described herein in accordance with the owner's manual and safety guidelines. Additionally, any actions taken by the vehicle owner / driver based on recommendations or notices provided by the vehicle should comply with all rules specific to the vehicle's location and operation (e.g., federal, state, national, city, etc.). Recommendations or notices provided by the vehicle should be considered as advice and followed only in accordance with any rules specific to the vehicle's location and operation. In the foregoing disclosure, reference has been made to the accompanying drawings, which form part of the foregoing disclosure, illustrating specific implementations in which the present disclosure may be practiced. It should be understood that other implementations may be utilized and structural changes may be made without departing from the scope of the present disclosure. References to “an embodiment,” “embodiment,” “example embodiment,” etc., in this specification indicate that the described embodiment may include a particular feature, structure, or characteristic, but each embodiment may not necessarily include said particular feature, structure, or characteristic. Furthermore, such phrases do not necessarily refer to the same embodiment. Moreover, when features, structures, or characteristics are described in connection with embodiments, those skilled in the art will recognize such features, structures, or characteristics in conjunction with other embodiments, whether explicitly described or not.

[0069] Additionally, where appropriate, the functions described herein may be performed in one or more of the following: hardware, software, firmware, digital components, or analog components. For example, one or more application-specific integrated circuits (ASICs) may be programmed to implement one or more of the systems and programs described herein. Throughout the specification and claims, certain terms are used to refer to specific system components. As those skilled in the art will appreciate, components may be referred to by different names. This document is not intended to distinguish between components with different names but identical functions.

[0070] It should also be understood that the word “example” as used herein is intended to be non-exclusive and non-restrictive in nature. More specifically, the word “example” as used herein refers to one of several examples, and it should be understood that there is no undue emphasis or preference for any particular example described.

[0071] Computer-readable media (also known as processor-readable media) include any non-transitory (e.g., tangible) medium that contributes to providing data (e.g., instructions) that can be read by a computer (e.g., by the computer's processor). Such media can take many forms, including but not limited to non-volatile and volatile media. A computing device may include computer-executable instructions, said instructions which can be executed by one or more computing devices (such as those listed above) and stored on a computer-readable medium.

[0072] Regarding the processes, systems, methods, heuristics, etc., described herein, it should be understood that although the steps of such processes, etc., have been described as occurring in a certain ordered order, such processes can be practiced with the described steps performed in a different order than that described herein. It should also be understood that some steps may be performed simultaneously, other steps may be added, or some steps described herein may be omitted. In other words, the description of processes herein is provided for the purpose of illustrating various embodiments and should in no way be construed as limiting the claims.

[0073] Therefore, it should be understood that the above description is intended to be illustrative rather than restrictive. Many embodiments and applications beyond the examples provided will become apparent upon reading the above description. The scope should not be determined by reference to the above description, but rather by reference to the appended claims and the full scope of their equivalents. Future developments in the techniques discussed herein are anticipated and expected, and the disclosed systems and methods will be incorporated into such future embodiments. In conclusion, it should be understood that modifications and changes are possible with this application.

[0074] Unless explicitly indicated otherwise herein, all terms used in the claims are intended to be given their ordinary meaning as understood by one skilled in the art as described herein. In particular, unless the claims explicitly limit the recitation to the contrary, the use of singular articles such as “a,” “the,” or “the” should be interpreted as one or more of the elements indicated by the recitation. Unless otherwise specifically stated or otherwise understood in the context of use, conditional language such as, in particular, “can,” “may,” “may,” or “may” is generally intended to express that some embodiments may include certain features, elements, and / or steps, while other embodiments may not include certain features, elements, and / or steps. Therefore, such conditional language is generally not intended to imply that one or more embodiments require each feature, element, and / or step in any way.

[0075] According to one embodiment, the instructions further cause the controller to: determine current environmental data associated with the environment; and cause the alarm generation and output unit to generate the alarm based on the current environmental data.

[0076] According to one embodiment, the instructions further cause the controller to: determine the person's behavior after outputting the alarm; cause the alarm generation and output unit to generate a second alarm based on the behavior to generate a modified alarm; and cause the alarm generation and output unit to output the second alarm in the direction toward the person.

[0077] According to one embodiment, in order to determine that the person is currently fully engaged, the instruction further causes the controller to: receive data from the one or more sensors; and based on the data determine that the person is: interacting with a mobile device, fully engaged in a conversation, or carrying an object indicating a physical disability.

[0078] According to one embodiment, the alarm includes one or more of the following: a directional audio signal; a directional light output; or a directional Bluetooth Low Energy advertising message.

[0079] According to one embodiment, the instructions also cause the controller to: determine the person's behavior after outputting the alarm; and stop outputting the alarm based on the behavior.

Claims

1. A method comprising: Determine the presence of people in the environment surrounding the vehicle; Based on the person's trajectory, it is determined that the person may exist within the interaction area of ​​the vehicle; It is determined that the person is currently fully focused; Determine the person's level of concentration; It was determined that the level of concentration exceeded a threshold; as well as An alarm is output in the direction of the person.

2. The method of claim 1, further comprising: Determine the current environmental data associated with the environment; as well as The alert is generated based on the current environmental data.

3. The method of claim 1, further comprising: After the alarm is issued, the person's behavior is determined; Modify the alert based on the behavior to generate a second alert; as well as The second alarm is output in the direction toward the person.

4. The method of claim 1, wherein outputting the alarm in the direction toward the person comprises outputting directional audio or directional light in the direction.

5. The method of claim 1, further comprising: After the alarm is issued, the changes in the person's trajectory are determined; as well as Stop outputting the alarm.

6. The method of claim 1, wherein determining that the person is currently fully attentive further comprises one or more of the following: Use one or more sensors of the vehicle to determine the person's current behavior; or Determine the physical attributes of the person and one or more objects carried by the person.

7. The method of claim 6, wherein the current behavior includes one or more of the following: The person interacts with the mobile device; The person in question is in a conversation; or The person is looking in the direction opposite to the vehicle.

8. A method comprising: Identify the interaction area associated with the vehicle; It is determined that people near the vehicle may be present in the interaction area in the near future; It is determined that the person is currently fully focused; Determine the environmental conditions near the vehicle; An alarm is generated based on the environmental conditions and the person's complete concentration; and Output the alarm.

9. The method of claim 8, further comprising: Determine the person's level of concentration; as well as Determine that the level of concentration exceeds a threshold.

10. The method of claim 8, further comprising: Monitor the person's behavior after the alarm is issued; The alert is modified based on the behavior to generate a modified alert; as well as Output the modified alert.

11. The method of claim 8, wherein the alarm comprises one or more of the following: Directional audio signals; Directional light output; or Targeted Bluetooth Low Energy advertising messages.

12. The method of claim 8, wherein outputting the alarm further comprises one or more of the following: Output a directional audio signal in the direction toward the person; Output a directional beam of light in the direction toward the person; or Output Bluetooth Low Energy advertising messages that can be received by the person's device.

13. The method of claim 8, wherein determining that the person is fully attentive further comprises: Receive data from one or more sensors of the vehicle; as well as Based on the data, it is determined that the person is one of the following: interacting with a mobile device; In the dialogue; Or carrying objects that indicate physical disabilities.

14. The method of claim 8, wherein the environmental conditions include one or more of the following: Ambient light; Environmental noise; The density of people in the vicinity; or Weather conditions.

15. A vehicle comprising: One or more sensors; An interactive prediction and analysis unit, the interactive prediction and analysis unit including a controller and coupled to the one or more sensors; An alarm generation and output unit, which is coupled to the interactive prediction and analysis unit; as well as A memory device coupled to the controller and storing instructions that, when executed by the controller, cause the controller to: The one or more sensors are used to determine the presence of people in the environment surrounding the vehicle; Based on the person's trajectory, it is determined that the person may exist within the interaction area of ​​the vehicle; Determine the person's level of concentration; It was determined that the level of concentration exceeded a threshold; and The alarm generation and output unit outputs an alarm in the direction toward the person.

Images