Coexisting warning method and system to warn actor of nearby target while driving for collision avoidance
Patent Information
- Application Number
- JP2024138449
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-08-25
- Filing Date
- 2024-08-20
- Publication Date
- 2025-03-26
Smart Images
Figure 2025041534000001_ABST
Abstract
Description
[Technical field]
[0001] (CROSS REFERENCE TO RELATED APPLICATIONS) The present invention claims priority from Indian Application No. 202321057195, filed on August 25, 2023.
[0002] The disclosure herein relates generally to the field of vehicle driving assistance and safety control, and more specifically to a coexistence warning method and system for warning nearby target actors while driving to avoid collisions. [Background technology]
[0003] An advanced driver assistance system (ADAS) is an electronic system equipped in a host vehicle that uses advanced technology to assist the driver while driving. ADAS systems alert the driver of the host vehicle and sometimes control the host vehicle to avoid a potential collision. However, current ADAS systems are limited to only assisting the driver of the host vehicle. Therefore, current ADAS systems target only a small portion of traffic accidents, leaving the majority of accidents behind. This is because actors present on the road, such as other vehicles, pedestrians, or non-motorized vehicles, may not be equipped with an ADAS system.
[0004] Described below are specific scenarios where current ADAS systems may fail to warn or where there is very little time available to take action to avoid a collision: (a) the inability of the vehicle's sensors to detect nearby targets (non-motorized vehicles and other vehicles, including pedestrians) due to an occluded environment; (b) the sudden appearance of targets (non-motorized vehicles and other vehicles, including pedestrians); and (c) the inattentive driving of the driver of the target vehicle (other vehicle, including non-motorized vehicles). Summary of the Invention
[0005] SUMMARY OF THE DISCLOSURE Embodiments of the present disclosure represent technical improvements that are solutions to one or more of the above-mentioned technical problems identified by the inventors in conventional systems.
[0006] In one aspect, a processor-implemented method for generating a causal and sentiment-infused knowledge graph for portfolio optimization is provided. The method includes receiving one or more road condition parameters for creating a 360-degree scene recognition of a road surrounding the host vehicle, the 360-degree scene recognition of the road surrounding the host vehicle including one or more actors present on the road as the host vehicle travels on the road; estimating one or more three-dimensional (3D) scene semantics of each of the one or more actors present on the road using the 360-degree scene recognition as the host vehicle travels using localization techniques; detecting one or more priority actors that may cause a collision from the one or more actors present on the road based on the one or more three-dimensional (3D) scene semantics as the host vehicle travels using path estimation and tracking techniques; determining to generate a coexistence warning signal for the one or more priority actors based on the one or more three-dimensional (3D) scene semantics and the 360-degree scene recognition of the road surrounding using a coexistence warning trained model; and generating a coexistence warning signal for the one or more priority actors based on the decision to generate using one or more output means.
[0007] In another aspect, a causal and sentiment-infused knowledge graph for portfolio optimization is provided. The system includes a memory for storing instructions, one or more input / output (I / O) interfaces, and one or more hardware processors coupled to the memory via the one or more I / O interfaces, the one or more hardware processors receiving one or more road condition parameters to create a 360-degree scene awareness of a road surrounding a host vehicle, the 360-degree scene awareness of a road surrounding the host vehicle including one or more actors present on the road as the host vehicle travels on the road; and using localization techniques to create one or more three-dimensional (3D) images of each of the one or more actors present on the road as the host vehicle travels. D) estimating scene semantics; detecting one or more priority actors that may cause a collision from one or more actors present on the road when the host vehicle is traveling using path estimation and tracking techniques based on the one or more three-dimensional (3D) scene semantics; deciding to generate a coexistence warning signal for one or more priority actors using a coexistence warning trained model based on the one or more three-dimensional (3D) scene semantics and 360-degree scene recognition around the road; and generating a coexistence warning signal for the one or more priority actors based on the decision to generate using one or more output means.
[0008] In yet another aspect, when executed by one or more hardware processors, a method includes receiving one or more road condition parameters and creating a 360-degree scene awareness of a road surrounding a host vehicle, the 360-degree scene awareness of a road surrounding the host vehicle including one or more actors present on the road as the host vehicle travels on the road; estimating one or more three-dimensional (3D) scene semantics of each of the one or more actors present on the road as the host vehicle travels using the 360-degree scene awareness of the road surrounding using localization techniques; and generating the one or more three-dimensional (3D) scene semantics of each of the one or more actors present on the road as the host vehicle travels using path estimation and tracking techniques. One or more non-transitory machine-readable information storage media are provided, the one or more instructions including: detecting one or more prioritized actors that may cause a collision from one or more actors present on a road when a host vehicle is traveling based on semantics; determining to generate a coexistence warning signal for the one or more prioritized actors based on one or more three-dimensional (3D) scene semantics and 360-degree scene recognition around the road using a coexistence warning trained model; and generating a coexistence warning signal for the one or more prioritized actors based on the decision to generate, using one or more output means.
[0009] In one embodiment, determining one or more preferred actors that may cause a collision from one or more actors present on a road as the host vehicle travels based on one or more three-dimensional (3D) scene semantics using path estimation and tracking techniques includes detecting one or more actors of interest from one or more actors present on the road as the host vehicle travels using path estimation and tracking techniques, determining one or more predicted movements of (i) each of the one or more actors of interest and (ii) the host vehicle using the path estimation and tracking techniques, and mapping (i) the one or more predicted movements, and (ii) current movements of each of the one or more actors of interest and the host vehicle to detect one or more preferred actors that may cause a collision.
[0010] In an embodiment, determining to generate a coexistence warning signal(s) of the one or more coexistence warning signals for one or more prioritized actors based on one or more three-dimensional (3D) scene semantics and 360-degree scene recognition around the road using the coexistence warning trained model includes determining a current state of the host vehicle and the one or more prioritized actors using the 360-degree scene recognition around the road, the state being associated with a current dynamics of each of the one or more prioritized actors on the road; , determining a current action for the host vehicle and one or more prioritized actors based on a current state of the host vehicle and one or more prioritized actors, where the action describes one or more possible movements of each of the one or more prioritized actors and the host vehicle, and sending (i) the current state of the host vehicle and one or more prioritized actors, and (ii) the current action for the host vehicle and one or more prioritized actors to the coexistence warning trained model to determine whether to generate a coexistence warning signal for the one or more prioritized actors that may cause a collision.
[0011] In one embodiment, a coexistence warning trained model includes receiving a training dataset including a plurality of training samples, the training dataset being associated with one or more driving environment scenarios and including one or more training sub-datasets, each training sub-dataset being associated with a respective driving environment scenario and including one or more training samples of the plurality of training samples, each of the plurality of training samples including (i) a training 360 degree scene perception of a road surrounding for a training host vehicle, and (ii) one or more training three-dimensional (3D) scene semantics for the training host vehicle associated with a plurality of training actors; and processing each of the plurality of training samples present in each training sub-dataset to extract a plurality of processing results from the plurality of training samples. The method includes the steps of: acquiring processed training samples, each of which includes (i) a training 360-degree scene recognition of a road surrounding, and (ii) a state and an action of each training prioritized actor and a training host vehicle from one or more three-dimensional (3D) scene semantics; assigning trained coexistence warning signals of the plurality of coexistence warning signals to each of the processed training samples based on the state and action of each training prioritized actor and the training host vehicle; and training a deep reinforcement learning neural network model on each processed training sample at a time using the assigned associated trained coexistence warning signals until the plurality of processed training samples are completed, to acquire a coexistence warning trained model.
[0012] In one embodiment, training the deep reinforcement learning neural network model includes sending a state of the training host vehicle for each processed training sample to a reinforcement learning (RL) agent of the deep reinforcement learning neural network model to obtain a coexistence warning trained model, using each processed training sample at a time with an assigned associated trained coexistence warning signal, until the multiple processed training samples are completed; obtaining from the RL agent (i) a predicted action of each training preferred actor and training host vehicle for a given state of the training vehicle, and (ii) a predicted coexistence warning signal from each processed training sample; comparing, for each processed training sample, (i) the predicted action of each training preferred actor and training host vehicle, and the predicted coexistence warning signal with the assigned corresponding trained coexistence warning signal, and providing a reward to the RL agent based on the comparison; and training the reinforcement learning (RL) agent with successive processed training samples until the multiple processed training samples are completed, maximizing the reward for the RL agent.
[0013] In one embodiment, the one or more actors present on the road include one or more motorized and non-motorized vehicles, pedestrians, and animals present on the road surrounding the host vehicle.
[0014] In one embodiment, based on the type of priority actor and the driving environment scenario, coexistence warning signals are generated via horn, headlights, vehicle-to-vehicle communication, and combinations thereof.
[0015] In one embodiment, one or more road condition parameters are associated with the road modeling, and the one or more road condition parameters are received from one or more 360-degree lidars, one or more front corner radars, one or more rear corner radars, one or more cameras, one or more ultrasonic sensors, and one or more geographic road information devices, or combinations thereof, mounted on the host vehicle.
[0016] It is to be understood that both the proceeding general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention as claimed.
[0017] The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate exemplary embodiments and, together with the description, serve to explain the disclosed principles. [Brief description of the drawings]
[0018] [Figure 1] FIG. 1 is an example block diagram of a coexistence warning system for warning nearby target actors while driving for collision avoidance, according to some embodiments of the present disclosure. [Diagram 2] 2 is an example block diagram illustrating modules of the coexistence warning system of FIG. 1 for warning nearby target actors while driving for collision avoidance, according to some embodiments of the present disclosure. [Diagram 3] FIG. 1 is an example flow diagram of a processor-implemented coexistence warning method for warning nearby target actors while driving for collision avoidance, according to some embodiments of the present disclosure. [Figure 4] FIG. 1 is an example flow diagram for detecting one or more priority actors that may cause a collision from one or more actors present on a road during travel of a host vehicle based on one or more three-dimensional (3D) scene semantics using path estimation and tracking techniques according to some embodiments of the present disclosure. [Diagram 5] FIG. 13 is an example flow diagram for obtaining a coexistence warning trained model, according to some embodiments of the present disclosure. [Figure 6] FIG. 13 is an example flow diagram for training a deep reinforcement neural network model with each processed training sample at a time using the assigned corresponding training coexistence warning signals until multiple coexistence warning signals are completed to obtain a coexistence warning trained model according to some embodiments of the present disclosure. [Figure 7]According to some embodiments of the present disclosure, a coexistence warning trained model is used to determine to generate a coexistence warning signal out of one or more coexistence warning signals for one or more priority actors that may cause a collision based on one or more three-dimensional (3D) scene semantics and 360-degree scene recognition of the road surroundings.
[0019] Exemplary embodiments are described with reference to the accompanying drawings. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. For convenience, the same reference numbers will be used throughout the figures to refer to the same or similar parts. Examples and features of the disclosed principles are described herein, however, modifications, adaptations and other implementations are possible without departing from the scope of the disclosed embodiments.
[0020] Current ADAS systems are limited to only assisting the driver of the host vehicle in potential danger. Therefore, current ADAS systems do not effectively prevent accidents or provide assistance in certain scenarios, because actors on the road, such as other vehicles, pedestrians, or non-motorized vehicles, may not be equipped with ADAS systems.
[0021] The following are some scenarios in which current ADAS systems may fail to warn or have very little time to take necessary actions to avoid a collision: (a) inability of the vehicle's sensors to detect nearby targets (other vehicles, including non-motorized vehicles and pedestrians) due to an occluded environment; (b) sudden appearance of targets (other vehicles, including non-motorized vehicles and pedestrians) in the vicinity of the host vehicle; and (c) inattentive driving by the driver of the target vehicle (other vehicles, including non-motorized vehicles).
[0022] The present disclosure solves technical problems in the art by using a coexistence warning method and a system installed in a host vehicle to identify such scenarios and provide necessary warnings to target vehicles, pedestrians and animals around the host vehicle. Conventional ADAS systems issue warnings and alert the driver of the host vehicle, while current coexistence warning methods and systems automatically alert nearby target vehicles, pedestrians and animals by gaining control of the host vehicle communication device using horn, headlight light, or Dedicated Short Range Communication (DSRC) technology.
[0023] Referring now to the corresponding figures, and in particular to FIGS. 1 through 7, in which reference characters throughout the application indicate corresponding features, suitable embodiments are shown and described in the context of the following example systems and / or methods.
[0024] 1 is an example block diagram of a coexistence warning system 100 for warning nearby target actors during driving for collision avoidance, according to some embodiments of the present disclosure. In one embodiment, the system 100 includes or is in other manner in communication with one or more hardware processors 104 operably coupled to a processor 104, a communication interface device or input / output (I / O) interface 106, and one or more data storage devices or memories 102. The one or more hardware processors 104, memory 102, and I / O interface 106 may be coupled to a system bus 108 or similar mechanism.
[0025] The I / O interface 106 may include a variety of software and hardware interfaces, such as a web interface, a graphical user interface, etc. The I / O interface 106 may include a variety of software and hardware interfaces, such as interfaces for peripheral devices, such as a keyboard, a mouse, external memory, multiple sensor devices, a printer, etc. Additionally, the I / O interface 106 may enable the system 100 to communicate with other devices, such as a web server and an external database.
[0026] I / O interface 106 may facilitate communications within a variety of network and protocol types, including, for example, a local area network (LAN), a wired network such as cable, and a wireless network such as a wireless LAN (WLAN), cellular or satellite. To this end, I / O interface 106 may include one or more ports for connecting multiple computer systems to one another or to a server computer. Additionally, I / O interface 106 may include one or more ports for connecting multiple devices to one another or to a server computer.
[0027] The one or more hardware processors 104 may be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuits, and / or any device that manipulates signals based on operational instructions. Among other functions, the one or more hardware processors 104 are configured to retrieve and execute computer-readable instructions stored in the memory 102. In the context of this disclosure, the expressions "processor" and "hardware processor" may be used interchangeably. In an embodiment, the system 100 may be implemented in a variety of computing systems, such as laptop computers, portable computers, notebooks, handheld devices, workstations, mainframe computers, servers, network clouds, and the like.
[0028] The memory 102 may include any computer-readable medium known in the art, including, for example, volatile memory, such as static random access memory (SRAM) and dynamic random access memory, and / or non-volatile memory, such as read-only memory (ROM), erasable programmable ROM, flash memory, hard disks, optical disks, and magnetic tapes. In one embodiment, the memory 102 includes modules 102a and a repository 102b for storing data processed, received, and generated by one or more of the modules 102a. The modules 102a include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement abstract data types.
[0029] The modules 102a may include programs, computer readable instructions, or coded instructions that supplement the applications or functions performed by the system 100. The modules 102a may also be used as signal processors, state machines, logic circuits, and / or any other devices or components that manipulate signals based on operational instructions. Furthermore, the modules 102a may be used by hardware, by computer readable instructions executed by one or more hardware processors 104, or by a combination thereof. In an embodiment, the modules 102a may include various sub-modules (not shown in FIG. 1). Furthermore, the memory 102 may include information regarding the input / output of the system 100 and each step performed by the processor 104 of the method of the present disclosure.
[0030] The repository 102b may include a database or a data engine. Furthermore, the repository 102b may function as a database or include multiple databases, among others, to store processed, received, or generated data as a result of the execution of the multiple modules 102a. It will be noted that while the repository 102b is shown internal to the system 100, in another embodiment, the repository 102b may be implemented external to the system 100 and the repository 102b may be stored in an external database (not shown in FIG. 1 ) communicatively coupled to the system 100. The data contained in such an external database may be periodically updated. For example, data may be added to the external database, and / or existing data may be modified, and / or useless data may be removed from the external database. In one example, the data may be stored in an external system, such as a Lightweight Directory Access Protocol (LDAP) directory and a Relational Database Management System (RDBMS). In another embodiment, the data stored in the repository 102b is distributed between the system 100 and the external database.
[0031] 2 and 3 collectively, components and functions of the system 100 according to exemplary embodiments of the present disclosure are described. For example, Fig. 2 is an example block diagram illustrating a module 200 of the coexistence warning system 100 of Fig. 1 for alerting nearby target actors during driving for collision avoidance according to some embodiments of the present disclosure. In one embodiment, the coexistence warning system 100 is installed or equipped in a host vehicle.
[0032] 2, the modules 200 include a context modeling module 202, a scene semantics estimation module 204, a preferred actor detection module 206, a coexistence warning and collision avoidance module 208, and an output means 210. In one embodiment, the modules 200 are stored in the repository 102b of the system 100.
[0033] In one embodiment, the context modeling module 202 is configured to receive one or more road condition parameters and creates a 360-degree scene awareness of the road surroundings of the host vehicle. The 360-degree scene awareness of the road surroundings of the host vehicle includes one or more actors present on the road in a certain range, vicinity, or distance to the host vehicle as the host vehicle travels on the road. In one embodiment, the one or more actors present on the road include, but are not limited to, one or more motorized vehicles, one or more non-motorized vehicles, pedestrians, and animals present on the road surrounding the host vehicle.
[0034] In one embodiment, one or more road condition parameters are associated with parameters of the road surroundings as the host vehicle travels along the road approaching the one or more actors. In one embodiment, one or more road condition parameters are received from one or more 360 degree light detection and ranging (Lidars), one or more front corner radars, one or more rear corner radio detection and ranging (radars), one or more cameras, one or more ultrasonic sensors, and one or more geographic road information devices, or combinations thereof, that are installed or equipped on the host vehicle.
[0035] In one embodiment, the 360-degree scene perception of the road surroundings of the host vehicle includes one or more images and / or one or more sensor values, or a combination thereof, which are generated using one or more road condition parameters associated with the road surroundings of the host vehicle as the host vehicle moves.
[0036] The scene semantics estimation module 204 is configured to estimate one or more three-dimensional (3D) scene semantics for each of one or more actors present on the road as the host vehicle travels. The 360-degree scene perception of the host vehicle's road surroundings is analyzed using localization techniques to estimate one or more three-dimensional (3D) scene semantics for each of one or more actors present on the road as the host vehicle travels.
[0037] In one embodiment, the one or more three-dimensional (3D) scene semantics include, but are not limited to, a number of one or more motorized vehicles, a current distance between the host vehicle and each of the one or more motorized vehicles, an acceleration of each of the one or more motorized vehicles, dimensions and dynamics of each of the one or more motorized vehicles, a current direction of movement of each of the one or more motorized vehicles, a number of one or more non-motorized vehicles, a current distance between the host vehicle and each of the one or more non-motorized vehicles, an acceleration of each of the one or more non-motorized vehicles, dimensions and dynamics of each of the one or more non-motorized vehicles, a number of one or more pedestrians, a current distance between the host vehicle and each of the one or more pedestrians, a current direction of movement of each of the one or more pedestrians, a number of one or more animals, a distance between the host vehicle and each of the one or more animals, and a current direction of movement of each of the one or more animals.
[0038] In one embodiment, the preferred actor detection module 206 is configured to detect one or more preferred actors from one or more actors present on a road when the host vehicle travels. The one or more preferred actors are actors that may cause a collision with the host vehicle when the host vehicle travels on the road. The one or more preferred actors are detected by the scene semantics estimation module 204 using one or more three-dimensional (3D) scene semantics estimations. In one embodiment, the preferred actor detection module 206 utilizes path estimation and tracking techniques to detect the one or more preferred actors that may cause a collision.
[0039] In one embodiment, the coexistence warning and collision avoidance module 208 is configured to continuously determine when to generate a coexistence warning signal and to which of one or more prioritized actors to provide a coexistence warning. The coexistence warning and collision avoidance module 208 utilizes a coexistence warning trained model to determine when to generate a coexistence warning signal and to which of one or more prioritized actors to provide a coexistence warning based on the 360-degree scene perception of the road surroundings created by the context modeling module 202 and one or more three-dimensional (3D) scene semantics estimated by the scene semantics estimation module 204.
[0040] In various embodiments, the coexistence warning signal is generated via horn, headlights, inter-vehicle communication, and combinations thereof based on the type of priority actor and the driving environment scenario. For example, if the host vehicle is an automobile and the surrounding environment is daylight, the coexistence warning signal may be generated by a sounding horn. Similarly, if the host vehicle is an automobile and the surrounding environment is night (dark environment), the coexistence warning signal may be generated by a flashing headlight. Similarly, if the host vehicle is a vehicle similar to the host vehicle and both have inter-vehicle communication, the coexistence warning signal is a form of inter-vehicle communication.
[0041] In an embodiment, the output means 210 includes one or more coexistence warning signal means, such as a horn, a headlight, inter-vehicle communication, etc., mounted on the host vehicle. The output means 210 is configured to receive instructions determined by the coexistence warning and collision avoidance module 208 and generate a coexistence warning signal to one or more priority actors based on the determination by the coexistence warning and collision avoidance module 208 through one or more output means (coexistence warning signal means).
[0042] With reference to FIG. 3, components and functions of the system 100 according to an exemplary embodiment of the present disclosure are described. For example, FIG. 3 illustrates an exemplary flow diagram of a processor-implemented coexistence warning method 300 for warning nearby target actors during driving for collision avoidance according to some embodiments of the present disclosure. Although the steps of the method 400, including process steps, method steps, techniques, etc., are described in sequence, such processes, methods, and techniques can be configured to operate in other orders. In other words, any order and sequence of steps that may be described does not necessarily require that the steps be performed in that order. The steps of the processes described herein can be performed in any particular order. Furthermore, some steps can be performed simultaneously or can be performed singly or independently.
[0043] In step 302 of the method 300, the one or more hardware processors 104 of the system 100 are configured to receive one or more road condition parameters via the context modeling module 202 to create a 360-degree scene awareness of the road surroundings of the host vehicle. The one or more road condition parameters relate to parameters of the road surroundings as the host vehicle travels on the road toward the one or more actors.
[0044] In one embodiment, one or more road condition parameters are received from one or more 360 degree light detection and ranging (Lidars), one or more front corner radars, one or more rear corner radio detection and ranging (radars), one or more cameras, one or more ultrasonic sensors, and one or more geographic road information devices, or combinations thereof, installed or equipped on the host vehicle.
[0045] In one embodiment, the host vehicle's 360-degree scene perception of the road surroundings provides the host vehicle with a visual representation of its surroundings (helping it see the world around it) and allows the host vehicle to recognize and identify one or more actors (things) in its surroundings while the host vehicle is traveling. In addition to one or more actors, the host vehicle needs to see and classify traffic lights, pedestrians, road signs, sidewalks, parking lots, lanes, etc. In addition, the host vehicle also needs to know the exact distance between itself and the surrounding objects.
[0046] In one embodiment, the 360-degree scene perception of the road surroundings of the host vehicle specifically includes one or more images, one or more sensor values, or a combination thereof, which are generated using one or more road condition parameters associated with the road surroundings of the host vehicle as the host vehicle moves.
[0047] In some embodiments, one or more image processing techniques are utilized to create a 360-degree scene recognition of the road surroundings of the host vehicle from one or more road condition parameters. In some embodiments, the one or more image processing techniques may include, but are not limited to, OmniDet and SurroundNet. Furthermore, a deep learning-based object detection model is utilized to recognize one or more actors present on the road from the created 360-degree scene recognition of the road surroundings. In some embodiments, some deep learning-based object detection models include, but are not limited to, region-based convolutional neural network (R-CNN) and point cloud based networks.
[0048] At step 304 of method 300, one or more hardware processors 104 of system 100 are configured, via scene semantics estimation module 204, to use 360-degree scene recognition of the road surroundings to estimate one or more three-dimensional (3D) scene semantics for each of one or more actors present on the road as the host vehicle travels. As described, localization techniques are utilized by scene semantics estimation module 204 to estimate one or more three-dimensional (3D) scene semantics for each of the one or more actors.
[0049] In one embodiment, the one or more three-dimensional (3D) scene semantics include, but are not limited to, a number of one or more motorized vehicles, a current distance between the host vehicle and each of the one or more motorized vehicles, an acceleration of each of the one or more motorized vehicles, dimensions and dynamics of each of the one or more motorized vehicles, a current direction of movement of each of the one or more motorized vehicles, a number of one or more non-motorized vehicles, a current distance between the host vehicle and each of the one or more non-motorized vehicles, an acceleration of each of the one or more non-motorized vehicles, dimensions and dynamics of each of the one or more non-motorized vehicles, a current direction of movement of each of the one or more non-motorized vehicles, a number of one or more pedestrians, a current distance between the host vehicle and each of the one or more pedestrians, a current direction of movement of each of the one or more pedestrians, a number of one or more animals, a current distance between the host vehicle and each of the one or more animals, and a current direction of movement of each of the one or more animals.
[0050] In one embodiment, the localization technique includes one or more localization algorithms that calculate the position and orientation of the host vehicle relative to the positions of one or more actors around the road. In one embodiment, a visual odometry (VO) algorithm is utilized that works by matching keypoints in successive video frames, where the keypoints are positions and orientations. For each frame, the keypoints are used as input to a mapping algorithm. The mapping algorithm, such as simultaneous localization and mapping (SLAM), calculates the position and orientation of each actor in the current frame relative to the previous frame, helping to classify all actors on the road.
[0051] In some embodiments, deep learning may be used to improve the performance of VO algorithms and classify different actors in general. Neural networks such as PoseNet and VLocNet++ are some frameworks that use point data to estimate 3D positions and orientations. These estimated 3D positions and orientations can be used to derive one or more three-dimensional (3D) scene semantics.
[0052] In step 306 of the method 300, the one or more hardware processors 104 of the system 100 are configured, via the preferred actor detection module 206, to detect one or more preferred actors that may cause a collision from one or more actors present on a road as the host vehicle travels. The one or more preferred actors are detected using one or more three-dimensional (3D) scene semantics estimated in step 304 of the method 300. In one embodiment, the preferred actor detection module 206 uses path estimation and tracking techniques to detect the one or more preferred actors.
[0053] 4 illustrates an example flow diagram for detecting one or more preferred actors from one or more actors present on a road as a host vehicle travels based on one or more three-dimensional (3D) scene semantics using a path estimation and tracking technique according to some embodiments of the present disclosure. In one embodiment, the preferred actor detection module 206 utilizes a path estimation and tracking technique to detect one or more preferred actors.
[0054] In step 306a, using a path estimation and tracking technique, one or more actors of interest are detected from one or more actors present on the road when the host vehicle travels. In step 306b, (i) one or more predicted movements of each of the one or more actors of interest relative to the host vehicle by their movements are determined using a path estimation and tracking technique. In one embodiment, some path estimation and tracking techniques include, but are not limited to, the Pure Pursuit algorithm and the Kalman filter. Finally, in step 306c, (i) one or more predicted movements and (ii) current movements of each of the one or more actors of interest and the host vehicle are mapped to detect one or more priority actors that may cause a collision. Current movements refer to real-time current or present movements.
[0055] The priority actor detection module 206 detects one or more priority actors that are affected by a possible collision with the host vehicle. These one or more priority actors can be vehicles in blind spots or occluded areas, vehicles changing lanes, non-motorized vehicles, or pedestrians, etc. Furthermore, the priority actor detection module 206 calculates a time to collision (TTC) which is the distance between each actor and the host vehicle divided by their relative speed. The time to collision (TTC) is also used for warning calculation. In one embodiment, the priority actor detection module 206 processes outputs from other existing ADAS systems such as Adaptive Cruise Control, Emergency braking system, Blind spot warning, Rear / Front cross traffic alert, Lateral collision warning, etc. This ensures the reuse of existing algorithms and avoids repetition.
[0056] In step 308 of method 300, one or more hardware processors 104 of system 100, via coexistence warning and collision avoidance module 208, are configured to continually determine when to generate a coexistence warning signal and which of one or more priority actors may cause a collision with the host vehicle.
[0057] The coexistence warning and collision avoidance module 208 utilizes the coexistence warning trained model to determine when to generate a coexistence warning signal for which of one or more priority actors that may cause a collision with the host vehicle, based on the 360-degree scene perception of the road surroundings created by the context modeling module 202 and one or more three-dimensional (3D) scene semantics estimated by the scene semantics estimation module 204.
[0058] In various embodiments, the coexistence warning signal is generated via a horn, headlights, inter-vehicle communication, and combinations thereof equipped on the host vehicle based on the type of the priority actor and the surrounding environment scenario. For example, if the host vehicle is an automobile and the surrounding environment is daytime, the generated coexistence warning signal may be via a horn that emits an audio signal. Similarly, if the host vehicle is an automobile and the surrounding environment is night (dark environment), the generated coexistence warning signal may be via a flashing headlight. Similarly, when the host vehicle is a vehicle similar to the host vehicle, if both have inter-vehicle communication, the coexistence warning signal may be in the form of inter-vehicle communication.
[0059] FIG. 5 illustrates an example flow diagram for obtaining a coexistence warning trained model according to some embodiments of the present disclosure. As shown in FIG. 5, obtaining a coexistence warning trained model is described through steps 308b1 to 308b4. In step 308b1, a training dataset including a plurality of training samples is received. The training dataset is associated with one or more driving environment scenarios. The training dataset includes one or more training sub-datasets, each training sub-dataset being associated with a respective driving environment scenario.
[0060] In one embodiment, the one or more driving environment scenarios include, but are not limited to, (i) lane-cutting driving, (ii) driving around hairpin curves in hilly areas, (iii) abnormal / unintended lane changes by a priority actor while the host vehicle is in the blind spot, (iv) overtaking the host vehicle on narrow roads, (v) sudden close-up appearance of pedestrians / animals / other vehicles on the road, (vi) merging lane and T-junction situations, and (vii) vehicle backing up.
[0061] In one embodiment, each training sub-dataset includes one or more training samples corresponding to a driving environment scenario, each training sample including (i) a training 360-degree scene perception of a road surrounding for a training host vehicle associated with a number of training actors, and (ii) one or more training three-dimensional (3D) scene semantics for the training host vehicle.
[0062] In step 308b2, each training sample in each training sub-dataset is processed to obtain a plurality of processed training samples from the plurality of training samples. Each processed training sample includes states and actions of each training priority actor and training host vehicle defined from (i) a training 360-degree scene recognition of the road surroundings, and (ii) one or more training three-dimensional (3D) scene semantics. In one embodiment, the states and actions of each training priority actor and training host vehicle correspond to one of one or more driving environment scenarios.
[0063] In step 308b3, a training coexistence warning signal of multiple coexistence warning signals is assigned to each processed training sample based on the state and action of each training priority actor and training host vehicle. In one embodiment, the multiple coexistence warning signals are generated via horn, headlight, vehicle-to-vehicle communication, and combinations thereof based on the type of priority actor and driving environment scenario.
[0064] Table 1 shows exemplary processed training samples present in each training sub-dataset associated with exemplary states and different driving scenarios. In Table 1, states are defined with corresponding speed conditions. TIFF2025041534000002.tif162160 Table 1
[0065] In step 308b4, a deep reinforcement learning neural network model is trained on each processed training sample at a time using the corresponding assigned training coexistence warning signals obtained in step 308b2 and step 308b3, respectively. The training process is repeated until multiple processed training samples are completed to obtain a coexistence warning trained model. In one embodiment, the deep reinforcement learning neural network model is one of a list including but not limited to Q-learning and Markov decision process (MDP).
[0066] 6 illustrates an example flow diagram of training a deep reinforcement learning neural network model on each processed training sample at a time using the corresponding assigned training coexistence warning signal until multiple processed training samples are completed to obtain a coexistence warning trained model according to some embodiments of the present disclosure. As shown in FIG. 6, training a deep reinforcement learning neural network model to obtain a coexistence warning trained model is described through steps 308c1 to 308c4.
[0067] In step 308c1, the state of the training host vehicle in each processed training sample is sent to a reinforcement learning (RL) agent of the deep reinforcement learning neural network model. In step 308c2, (i) predicted motions of each training priority actor and the training host vehicle for a given state of the training host vehicle, and (ii) predicted coexistence warning signals from each processed training sample are obtained from the RL agent.
[0068] In step 308c3, (i) the predicted actions of each training preferred actor and training host vehicle from each processed training sample and (ii) the predicted coexistence warning signal and the assigned corresponding training coexistence warning signal are compared, and a reward is provided to the RL agent based on the comparison. In step 308c4, training of the reinforcement learning (RL) agent using the normal processed training samples is performed until multiple processed training samples are completed to maximize the reward for the RK agent and obtain a coexistence warned model.
[0069] 7 illustrates an example flow diagram of determining to generate a coexistence warning signal of one or more coexistence warning signals for one or more priority actors based on one or more three-dimensional (3D) scene semantics and 360-degree scene recognition of road surroundings using a coexistence warning trained model according to some embodiments of the present disclosure. As shown in FIG. 7, the determination to generate a coexistence warning signal of one or more coexistence warning signals for one or more priority actors is described through steps 308a1 to 308a3.
[0070] In step 308a1, a current state of the host vehicle and one or more preferred actors is determined using the 360-degree scene recognition of the road surroundings created in step 302 of method 300. In one embodiment, the state is associated with a current dynamics of each of the one or more preferred actors on the road. In step 308a2, a current action for the host vehicle and one or more preferred actors is determined based on the current state of the host vehicle and one or more preferred actors determined in step 308a1 using one or more three-dimensional (3D) scene semantics estimated in step 304 of method 300 and the 360-degree scene recognition of the road surroundings created in step 302 of method 300. In one embodiment, the action describes one or more possible movements of each of the one or more preferred actors and the host vehicle.
[0071] In step 308a3, the current states of the host vehicle and one or more prioritized actors, and (ii) the current actions for the host vehicle and one or more prioritized actors determined in steps 308a1 and 308a2, respectively, are sent to the coexistence warning trained model to determine whether to generate a coexistence warning signal for the one or more prioritized actors that may cause a collision. When the current states of the host vehicle and one or more prioritized actors and the current actions for the host vehicle and one or more prioritized actors are associated with one of the driving scenarios, a corresponding coexistence warning signal is generated to warn the one or more prioritized actors.
[0072] In step 310 of the method 300, the one or more hardware processors 104 of the system 100 are configured to generate, via the output means 210, using the one or more output means, a coexistence warning signal to one or more prioritized actors that may cause a collision based on the generating decision. In an embodiment, the output means 210 includes one or more coexistence warning signal means, such as a horn, a headlight, inter-vehicle communication, etc., equipped on the host vehicle. The output means 210 is configured to receive an instruction determined by the coexistence warning and collision avoidance module 208, and generate, via the one or more output means (coexistence warning signal means), a coexistence warning signal to one or more prioritized actors that may cause a collision based on the coexistence warning and collision avoidance module 208.
[0073] The disclosed method and system automatically warns nearby target vehicles, pedestrians and animals by taking control of the communication host vehicle's communication devices such as horn, headlight light or Dedicated Short Range Communication (DSRC) technology. The coexistence warning method and system is installed in the host vehicle to identify such scenarios and provide necessary warnings to target vehicles, pedestrians and animals around the host vehicle.
[0074] The coexistence warning system of the present disclosure can be implemented on top of current ADAS systems, so no separate hardware is required, reducing costs.The method and system of the present disclosure are useful for warning targets in a very short time, so many fatal road accidents can be avoided.
[0075] The embodiments of the present disclosure herein address the unsolved problem of alerting target actors on the road by a host vehicle. The coexistence alerting method and system of the present disclosure automatically alerts nearby target vehicles, pedestrians and animals by taking control of the host vehicle communications, such as horn, headlight lights, and communications with Dedicated Short Range Communications (DSRC) technology.
[0076] The description set forth herein describes the subject matter and enables any person skilled in the art to make and use the embodiments. The scope of the subject matter embodiments is defined by the claims and may include other variations that occur to those skilled in the art. Such other variations are intended to be within the scope of the claims if they contain equivalent elements with minor differences from the literal language of the claims.
[0077] It should be understood that the scope of protection extends to such a program in addition to a computer readable means having a message in itself, such computer readable storage means including program code means for the implementation of one or more steps of the method when the program is executed on any kind of device that can be programmed, including a server, a mobile device, or any kind of computer, such as, for example, a server or a personal computer, or any combination thereof. The hardware device can be any kind of device that can be programmed, such as, for example, any kind of computer, such as, for example, a server, a personal computer, or any combination thereof. The device may also include means that can be hardware means, such as, for example, an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a combination of hardware and software, such as, for example, an ASIC and an FPGA, or at least one microprocessor and at least one memory with software processing components located on it. Thus, the means can include both hardware means and software means. The embodiments of the method described herein can be implemented in hardware and software. The device can also include software means. Alternatively, the embodiments can be implemented using different hardware devices, such as, for example, multiple CPUs.
[0078] The embodiments herein may include hardware and software elements. Embodiments that may be implemented in software include, but are not limited to, firmware, resident software, microcode, etc. The functions performed by various components described herein may be implemented in other components or combinations of other components. For purposes of this description, a computer usable or computer readable medium may be any apparatus that can contain, store, communicate, propagate, or transport a program for use by or in communication with an instruction execution system, apparatus, or device.
[0079] The illustrated steps are presented to explain the illustrated exemplary embodiments, and ongoing technological developments should be expected to change the means by which a particular function is implemented. These examples are provided herein for purposes of illustration and not limitation. Moreover, boundaries of functional building blocks have been arbitrarily defined herein for convenience of description. Alternative boundaries can be defined so long as the specified functions and relationships therein are appropriately performed. Alternatives (including equivalents, extensions, variations, deviations, etc. of those described herein) will be apparent to those skilled in the art based on the teachings contained herein. Such alternatives fall within the scope of the disclosed embodiments. Also, "including," "having," "including," and other similar forms are intended to be equivalent, and any items following those words are open-ended in not implying an exhaustive listing of those items or being limited to only the listed items. It should also be noted that the singular forms "a," "an," and "the," as used in this specification and the appended claims, include plural references unless the context clearly dictates otherwise.
[0080] Furthermore, one or more computer-readable storage media can be utilized in embodiments implemented in accordance with the present disclosure. A computer-readable storage medium refers to any type of physical memory capable of storing information or data readable by a processor. Thus, a computer-readable storage medium can store instructions for execution by one or more processors, including instructions for causing a processor to perform steps or stages in accordance with the embodiments described herein. The term "computer-readable medium" should be understood to include tangible items and exclude carrier waves and transient signals, i.e., non-transitory. Examples include random access memory (RAM), read-only memory (ROM), volatile memory, non-volatile memory, hard drives, CD ROM, DVD, flash drives, disks, and any other known physical storage media.
[0081] The present disclosure and examples are intended to be exemplary only, with the principles of the disclosed embodiments being indicated by the following claims.
Claims
1. A processor implementation method (300), comprising: receiving (302) one or more road condition parameters via one or more hardware processors to create a 360 degree scene awareness of a road surrounding a host vehicle, the 360 degree scene awareness of the road surrounding the host vehicle including one or more actors present on the road as the host vehicle travels on the road; using localization techniques to estimate (304) one or more three-dimensional (3D) scene semantics of each of the one or more actors present on the road as the host vehicle travels, via the one or more hardware processors, using 360 degree scene recognition of the road surroundings; detecting (306) one or more priority actors that may cause a collision from the one or more actors present on the road as the host vehicle travels, based on the one or more three-dimensional (3D) scene semantics, via the one or more hardware processors, using path estimation and tracking techniques; determining (308) to generate a coexistence warning signal to one or more priority actors via the one or more hardware processors based on the one or more three-dimensional (3D) scene semantics and 360-degree scene recognition of the road surroundings using a coexistence warning trained model; generating (310) said coexistence warning signal via said one or more hardware processors to one or more priority actors based on said generating decision using one or more output means; A processor-implemented method (300) comprising:
2. detecting, based on the one or more three-dimensional (3D) scene semantics, one or more priority actors that may cause a collision from the one or more actors present on the road as the host vehicle travels, using a path estimation and tracking technique; Detecting one or more actors of interest from the one or more actors present on the road as the host vehicle travels using the path estimation and tracking techniques (306a); determining (306b) one or more predicted movements of (i) each of the one or more actors of interest and (ii) the host vehicle using the path estimation and tracking techniques; Mapping (306c) (i) the one or more predicted movements, and (ii) the current movement of each of the one or more actors of interest and the host vehicle to detect the one or more priority actors that may cause a collision; 2. The method of claim 1, comprising:
3. determining to generate the coexistence warning signal of the one or more coexistence warning signals for one or more priority actors based on the one or more three-dimensional (3D) scene semantics and 360-degree scene recognition of the road surroundings using the coexistence warning trained model; determining a current state of the host vehicle and the one or more priority actors using 360° scene awareness of the road surroundings, the state being related to the current dynamics of each of the one or more priority actors on the road (308a1); determining current actions for the host vehicle and the one or more preferred actors based on the current states of the host vehicle and the one or more preferred actors using the one or more three-dimensional (3D) scene semantics and 360 degree scene recognition of the road surroundings, the actions describing one or more possible movements of each of the one or more preferred actors and the host vehicle (308a2); Sending (308a3) (i) the current states of the host vehicle and the one or more prioritized actors, and (ii) the current actions for the host vehicle and the one or more prioritized actors to the coexistence warning trained model to determine whether to generate the coexistence warning signal for one or more prioritized actors that may cause a collision; 2. The method of claim 1, comprising:
4. The coexistence warning trained model is receiving a training dataset including a plurality of training samples, the training dataset being associated with one or more driving environment scenarios and including one or more training sub-datasets, each training sub-dataset being associated with a respective driving environment scenario and including one or more training samples of the plurality of training samples, each of the plurality of training samples being associated with a plurality of training actors including (i) a training 360 degree scene recognition of a road surrounding of a training host vehicle, and (ii) one or more three-dimensional (3D) scene semantics of the training host vehicle; processing each of the plurality of training samples present in each training sub-dataset to obtain a plurality of processed training samples from the plurality of training samples, each processed training sample including (i) a training 360° scene recognition of the road surroundings, and (ii) one or more three-dimensional (3D) scene semantics, and states and actions of each training priority actor and the training host vehicle (308b2); assigning (308b3) a training coexistence warning signal of a plurality of coexistence warning signals to each processed training sample based on the state and the action of each training priority actor and the training host vehicle; training (308b4) a deep reinforcement learning neural network with each processed training sample at a time using the assigned associated training coexistence warning signal until the plurality of processed training samples are completed to obtain the coexistence warning trained model; 2. The processor-implemented method of claim 1 obtained by:
5. training a deep reinforcement learning neural network with each processed training sample when using the assigned associated training coexistence warning signal until the plurality of processed training samples are completed to obtain the coexistence warning trained model; Sending the state of the training host vehicle for each processed training sample to a reinforcement learning (RL) agent of a deep reinforcement learning neural network model (308c1); Obtaining from the RL agent (i) predicted actions of each training priority actor and the training host vehicle for a given state of the training host vehicle, and (ii) predicted coexistence warning signals from each processed training sample (308c2); (i) comparing the predicted actions of each training preferred actor and the training host vehicle from each processed training sample with each training preferred actor and the host vehicle, and (ii) for each processed training sample, comparing the predicted coexistence warning signal with an assigned corresponding training coexistence warning signal, and providing a reward to the RL agent based on the comparison (308c3); performing the training of the reinforcement learning (RL) agent using successive processed training samples to maximize the reward for the RL agent until the plurality of processed training samples is completed (308c4); 4. The method of claim 3, comprising:
6. the one or more actors on the road include one or more motorized and non-motorized vehicles, pedestrians, and animals on the road around the host vehicle; The coexistence warning signal is generated by a horn, a headlight, a vehicle-to-vehicle communication, or a combination thereof based on the type of the priority actor and a driving environment scenario.
2. The method of claim 1.
7. 2. The processor-implemented method of claim 1, wherein the one or more road condition parameters are associated with road modeling, and the one or more road condition parameters are received from one or more of: one or more 360 degree light detection and ranging, one or more front corner radars, one or more rear corner radars, one or more cameras, one or more ultrasonic sensors, and one or more geographic road information devices, or combinations thereof, mounted to the host vehicle.
8. A system (100), comprising: A memory (102) for storing instructions; one or more input / output (I / O) interfaces (106); one or more hardware processors (104) coupled to the memory (102) via the one or more input / output interfaces (106), the one or more hardware processors (104) comprising: receiving one or more road condition parameters and creating a 360 degree scene awareness of a road surrounding for a host vehicle, the 360 degree scene awareness of a road surrounding for the host vehicle including one or more actors present on the road as the host vehicle travels on the road; using localization techniques to estimate one or more three-dimensional (3D) scene semantics for each of the one or more actors present on the road as the host vehicle travels, using 360 degree scene recognition around the road; detecting one or more actors that may cause a collision from the one or more actors present on the road as the host vehicle travels based on the one or more three-dimensional (3D) scene semantics using path estimation and tracking techniques; determining, using a coexistence warning trained model, to generate a coexistence warning signal for one or more priority actors based on the one or more three-dimensional (3D) scene semantics and a 360-degree scene recognition of the road surroundings; and generating said coexistence warning signal to one or more priority actors based on said decision to generate using one or more output means; One or more hardware processors (104) configured with instructions to execute A system (100).
9. The coexistence warning and collision avoidance module (102b1) via the one or more hardware processors (104): detecting one or more actors of interest from the one or more actors present on the road as the host vehicle travels using the path estimation and tracking techniques; determining one or more predicted movements of (i) each of the one or more actors of interest and (ii) the host vehicle using the path estimation and tracking techniques; mapping (i) the one or more predicted movements, and (ii) current movements of each of the one or more actors of interest and the host vehicle to detect the one or more actors that may be likely to cause a collision; 10. The system of claim 8, further configured to use path estimation and tracking techniques to detect the one or more actors that may cause a collision from the one or more actors present on the road as the host vehicle travels based on the one or more three-dimensional (3D) scene semantics.
10. The coexistence warning and collision avoidance module (102b1) via the one or more hardware processors (104): determining a current state of the host vehicle and the one or more priority actors using 360 degree scene awareness of the road surroundings, the state being related to the current dynamics of each of the one or more priority actors on the road; determining current actions for the host vehicle and the one or more preferred actors based on the current states of the host vehicle and the one or more preferred actors using the one or more three-dimensional (3D) scene semantics and 360 degree scene recognition of the road surroundings, the actions describing one or more possible movements of the one or more preferred actors and the host vehicle; (i) sending the current states of the host vehicle and the one or more prioritized actors, and (ii) the current actions of the host vehicle and the one or more prioritized actors to the coexistence warning trained model to determine whether to generate the coexistence warning signal for one or more prioritized actors that may cause a collision; The system of claim 8, configured to use the coexistence warning trained model to determine to generate the coexistence warning signal of the one or more coexistence warning signals for one or more priority actors based on the one or more three-dimensional (3D) scene semantics and 360-degree scene perception of the road surroundings.
11. The one or more hardware processors (104) receiving a training dataset including a plurality of training samples, the training dataset being associated with one or more driving environment scenarios, the plurality of training samples including one or more training samples, each of the plurality of training samples including (i) a training 360 degree scene perception of a road surrounding for a training host vehicle associated with a plurality of training actors, and (ii) one or more training three-dimensional (3D) scene semantics for the training host vehicle; sending each of the plurality of training samples present in each training sub-dataset to obtain a plurality of processed training samples from the plurality of training samples, each processed training sample including states and actions of each training priority actor and the training host vehicle from (i) a training 360-degree scene recognition of the road surroundings, and (ii) one or more training three-dimensional (3D) scene semantics; assigning a training coexistence warning signal of a plurality of coexistence warning signals to each of the processed training samples based on the state and the action of each training priority actor and the training host vehicle; training a deep reinforcement learning neural network model with each processed training sample at a time using the assigned associated training coexistence warning signal until the plurality of processed training samples are completed, thereby obtaining the coexistence warning trained model; The system of claim 8 , configured to acquire the coexistence warning trained model by:
12. The one or more hardware processors (104) perform the following steps to obtain the coexistence alert trained model: sending the state of a training host vehicle for each processed training sample to a reinforcement learning (RL) agent of the deep reinforcement learning neural network model; Obtaining from the RL agent (i) predicted actions of each training priority actor and the training host vehicle for a given state of the training host vehicle, and (ii) predicted coexistence warning signals from each processed training sample; for each processed training sample, (i) comparing the predicted actions of each training preferred actor and training host vehicle from each processed training sample with the actions of each training preferred actor and training host vehicle, and (ii) comparing the predicted coexistence warning signal with the corresponding assigned training coexistence warning signal, and providing a reward to the RL agent based on the comparison; performing the training of the reinforcement learning (RL) agent using successive processed learning samples until the plurality of processed training samples is completed, thereby maximizing the reward to the RL agent; 12. The system of claim 11, configured to train the deep reinforcement learning neural network model with each processed training sample using the assigned associated training coexistence warning signal until the plurality of processed training samples are completed by:
13. the one or more actors on the road include one or more motorized and non-motorized vehicles, pedestrians, and animals on the road around the host vehicle; The coexistence warning signal is generated by horn, headlight, vehicle-to-vehicle communication, and combinations thereof based on the type of the priority actor and a driving environment scenario. The system of claim 8.
14. 10. The system of claim 8, wherein the one or more road condition parameters are associated with road modeling, and the one or more road condition parameters are received from one or more of: one or more 360 degree light detection and ranging, one or more front corner radars, one or more rear corner radars, one or more cameras, one or more ultrasonic sensors, and one or more geographic road information devices, or combinations thereof, mounted on the host vehicle.
15. One or more non-transitory machine-readable information storage media that, when executed by one or more hardware processors, receiving one or more road condition parameters and creating a 360 degree scene awareness of a road surrounding for a host vehicle, the 360 degree scene awareness of the road surrounding for the host vehicle including one or more actors present on the road as the host vehicle travels on the road; using localization techniques to estimate one or more three-dimensional scene semantics for each of the one or more actors present on the road as the host vehicle travels using 360 degree scene recognition of the road surroundings; detecting, using a path estimation and tracking technique, one or more priority actors that may cause a collision from the one or more actors present on the road when the host vehicle is traveling based on the one or more 3D scene semantics; determining, using a coexistence warning trained model, to generate a coexistence warning signal for one or more priority actors based on the one or more three-dimensional scene semantics and a 360-degree scene recognition of a road surrounding; generating said coexistence warning signal to one or more priority actors based on said decision to generate using one or more output means; One or more non-transitory machine-readable information storage media comprising one or more instructions for performing the steps of: