Combination warning method and system for warning near target activer to avoide

Abstract

The present disclosure generally relates to a symbiotic warning method and system that warns nearby target actors to avoid a collision while driving. A current ADAS system is limited only to assist a driver of a host vehicle in coping with potential hazards. The present disclosure identifies various driving scenarios and provides necessary warnings to target vehicles, pedestrians, and animals around a host vehicle. A symbiotic warning method involves receiving one or more road context parameters to create a 360 degree scene perception of a road environment of a host vehicle with one or more actors. Then, one or more three-dimensional (3-D) scene semantics are estimated for each driving scenario in the one or more actors, and one or more key actors that cause a possible collision are detected. Further, it is determined that a symbiotic warning signal is generated to the one or more actors, and a symbiotic warning signal is generated to the one or more actors that cause the possible collision.

Description

[0001] Cross-reference to related applications

[0002] This application claims priority to Indian application No. 202321057195 filed on August 25, 2023. Technical Field

[0003] The present disclosure relates generally to the field of vehicle driver assistance and safety control, and more particularly to a symbiotic warning method and system to warn nearby target actors for collision avoidance while driving. Background Art

[0004] Advanced driver assistance systems (ADAS) are electronic systems equipped in the host vehicle that use advanced technology to assist the driver while driving. The ADAS system warns the driver of the host vehicle and sometimes controls the host vehicle to avoid potential collisions. However, current ADAS systems are limited to assisting the driver of the host vehicle. Therefore, current ADAS systems are only for small-scale road accidents, leaving large-scale accidents behind. This is because actors (such as other vehicles, pedestrians, or non-motorized vehicles) that appear on the road may not be equipped with ADAS systems.

[0005] Mentioned below are some scenarios where current ADAS systems may fail to warn, or they take too little time to take necessary actions to avoid collisions: (a) the car sensors are unable to detect nearby targets (other vehicles, including non-motorized vehicles and pedestrians) due to obscured environment, (b) targets (other vehicles, including non-motorized vehicles and pedestrians) suddenly appear near the host vehicle, and (c) inattentive driving from the driver of the target vehicle (other vehicle, including non-motorized vehicles). Summary of the invention

[0006] The embodiments of the present disclosure propose technical improvements as solutions to one or more of the above-mentioned technical problems recognized by the inventors in conventional systems.

[0007] In one aspect, a processor-implemented method for generating a causal sentiment-enriched knowledge graph for portfolio optimization is provided. The method includes the following steps: receiving one or more road context parameters to create a 360-degree scene perception of a road environment of a host vehicle, wherein the 360-degree scene perception of the road environment of the host vehicle includes one or more actors present on the road when the host vehicle is navigating on the road; using the 360-degree scene perception of the road environment, using a positioning technique to estimate one or more three-dimensional (3-D) scene semantics of each of the one or more actors present on the road when the host vehicle is navigating; using a path estimation and tracking technique based on the one or more three-dimensional (3-D) scene semantics, detecting one or more key actors present on the road that cause a possible collision from the one or more actors present on the road when the host vehicle is navigating; using a symbiotic warning training model, based on the one or more three-dimensional (3-D) scene semantics and the 360-degree scene perception of the road environment, determining to generate a symbiotic warning signal to the one or more key actors; and generating a symbiotic warning signal to the one or more key actors using one or more output devices based on the generated determination.

[0008] In another aspect, a system for generating a causal sentiment-enriched knowledge graph for portfolio optimization is provided. The system includes: a memory storing instructions; one or more input / output (I / O) interfaces; and one or more hardware processors coupled to the memory via the one or more I / O interfaces, wherein the one or more hardware processors are configured by the instructions to: receive one or more road context parameters to create a 360-degree scene perception of a road environment of a host vehicle, wherein the 360-degree scene perception of the road environment of the host vehicle includes: one or more actors appearing on a road when the host vehicle is navigating on the road; when the host vehicle is navigating, using the 360-degree scene perception of the road environment, using positioning technology to estimate one or more three-dimensional (3-D) scene semantics of each of the one or more actors appearing on the road; based on the one or more three-dimensional (3-D) scene semantics, using path estimation and tracking technology, from the one or more actors appearing on the road when the host vehicle is navigating, one or more key actors that cause a possible collision are detected; based on the one or more three-dimensional (3-D) scene semantics and the 360-degree scene perception of the road environment, using a symbiotic warning training model to determine to generate a symbiotic warning signal to the one or more key actors; and based on the generated determination, using one or more output devices to generate a symbiotic warning signal to the one or more key actors.

[0009] In yet another aspect, one or more non-transitory machine-readable storage media are provided, comprising one or more instructions that, when executed by one or more hardware processors, result in: receiving one or more road context parameters to create a 360-degree scene perception of a road surrounding environment of a host vehicle, wherein the 360-degree scene perception of the road surrounding environment of the host vehicle includes: one or more actors appearing on a road as the host vehicle navigates on the road; using the 360-degree scene perception of the road environment and positioning technology to estimate one or more three-dimensional (3-D) scene semantics of each of the one or more actors appearing on the road as the host vehicle navigates; based on the one or more three-dimensional (3-D) scene semantics, using path estimation and tracking technology, detecting one or more key actors that cause a possible collision from the one or more actors appearing on the road as the host vehicle navigates; using a symbiotic warning training model, based on the one or more three-dimensional (3-D) scene semantics and the 360-degree scene perception of the road environment, deciding to generate a symbiotic warning signal to one or more key actors; and based on the generated decision, using one or more output devices to generate a symbiotic warning signal to the one or more key actors.

[0010] In an embodiment, based on one or more three-dimensional (3-D) scene semantics, path estimation and tracking technology is used to detect one or more key actors that may cause a possible collision from one or more actors that appear on the road when the host vehicle is navigating, including: using path estimation and tracking technology to detect one or more key actors from one or more actors that appear on the road when the host vehicle is navigating; using path estimation and tracking technology to determine (i) each of the one or more key actors and (ii) one or more predicted movements of the host vehicle; and mapping each of the one or more key actors and the host vehicle's (i) one or more predicted movements and (ii) current movements to detect one or more key actors that may cause a possible collision.

[0011] In an embodiment, a symbiotic warning training model is used to determine, based on one or more three-dimensional (3-D) scene semantics and 360-degree scene perception of a road environment, whether to generate one or more symbiotic warning signals to one or more key actors, including: using the 360-degree scene perception of the road environment to determine the current state of a main vehicle and one or more key actors, wherein the state is associated with the current dynamics of each of the one or more key actors on the road; based on the current state of the main vehicle and one or more key actors, using one or more three-dimensional (3-D) scene semantics and 360-degree scene perception of the road environment to determine the current actions of the main vehicle and one or more key actors, wherein the actions describe one or more possible movements of each of the one or more key actors and the main vehicle; and (i) the current state of the main vehicle and one or more key actors, (ii) the current actions of the main vehicle and one or more key actors are transmitted to the symbiotic warning training model to determine whether to generate a symbiotic warning signal to one or more key actors that may cause a possible collision.

[0012] In an embodiment, a symbiotic warning training model is obtained by: receiving a training data set including a plurality of training samples, wherein the training data set is associated with one or more driving environment scenarios and includes one or more training sub-data sets, wherein each training sub-data set is associated with each driving environment scenario and includes one or more training samples of the plurality of training samples, and wherein each training sample of the plurality of training samples includes (i) training 360-degree scene perception of a road environment of a training host vehicle and (ii) one or more training three-dimensional (3-D) scene semantics of a training host vehicle associated with a plurality of training actors; One or more training three-dimensional (3-D) scene semantics, processing each of the multiple training samples appearing in each training sub-dataset to obtain multiple processed training samples from the multiple training samples, wherein each processed training sample includes the state and action of each training key actor and the training main vehicle; assigning a training symbiotic warning signal from a plurality of symbiotic warning signals to each processed training sample based on the state and action of each training key actor and the training main vehicle; and training a deep reinforcement learning neural network model using the assigned associated training symbiotic warning signal using each processed training sample each time until the multiple processed training samples are completed to obtain a symbiotic warning training model.

[0013] In an embodiment, a deep reinforcement learning neural network model is trained using each processed training sample each time, using the assigned associated training symbiotic warning signal, until multiple processed training samples are completed to obtain a symbiotic warning training model, including: passing the state of the training host vehicle of each processed training sample to a reinforcement learning (RL) agent of the deep reinforcement learning neural network model; obtaining from the RL agent (i) predicted actions of each training focus actor and training host vehicle for a given state of the training host vehicle, and (ii) predicted symbiotic warning signals from each processed training sample; comparing (i) the predicted actions of each training focus actor and training host vehicle from each processed training sample with the actions of each training focus actor and training host vehicle, and (ii) comparing the predicted symbiotic warning signal with the corresponding training symbiotic warning signal assigned to each processed training sample to provide a reward to the RL agent based on the comparison; and performing training of the reinforcement learning (RL) agent using continuously processed training samples until multiple processed training samples are completed to maximize the reward to the RL agent.

[0014] In an embodiment, the one or more actors present on the road include one or more motorized and non-motorized vehicles, pedestrians, and animals present on the road around the host vehicle.

[0015] In an embodiment, wherein, based on the type of key actor and the driving environment scenario, a symbiotic warning signal is generated through a horn, headlights, vehicle-to-vehicle communication, and a combination thereof.

[0016] In an embodiment, one or more road situation parameters are associated with road modeling, and the one or more road situation parameters are received from one or more of the following: one or more 360-degree lidars, one or more front corner radars, one or more rear corner radars, one or more cameras, one or more ultrasonic sensors, and one or more geographic road information devices or a combination thereof installed in the main vehicle.

[0017] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate exemplary embodiments and, together with the description, serve to explain the principles of the disclosure:

[0019] Figure 1 is an exemplary block diagram of a symbiotic warning system for warning nearby target actors to avoid collisions while driving, according to some embodiments of the present disclosure.

[0020] Figure 2 is a diagram showing some embodiments of the present disclosure Figure 1 An exemplary block diagram of modules of a symbiotic warning system for warning nearby target actors to avoid collisions while driving.

[0021] Figure 3 An exemplary flow chart of a processor-implemented symbiotic warning method for warning nearby target actors to avoid collisions while driving is shown in accordance with some embodiments of the present disclosure.

[0022] Figure 4 An exemplary flowchart is shown for detecting one or more focal actors causing a possible collision from among one or more actors present on a road as a host vehicle navigates based on one or more three-dimensional (3-D) scene semantics using path estimation and tracking techniques in accordance with some embodiments of the present disclosure.

[0023] Figure 5 An exemplary flow chart for obtaining a symbiotic warning training model according to some embodiments of the present disclosure is shown.

[0024] Figure 6 An exemplary flowchart for training a deep reinforcement learning neural network model according to some embodiments of the present disclosure is shown, each time using each processed training sample, using the assigned corresponding training symbiotic warning signal until multiple symbiotic warning signals are completed to obtain a symbiotic warning training model.

[0025] Figure 7 An exemplary flowchart is shown for using a symbiotic warning training model to determine, based on one or more three-dimensional (3-D) scene semantics and 360-degree scene perception of a road environment, generating one or more symbiotic warning signals to one or more key actors causing a possible collision, according to some embodiments of the present disclosure. DETAILED DESCRIPTION

[0026] Exemplary embodiments are described with reference to the accompanying drawings. In the drawings, the leftmost digit of a reference number identifies the drawing in which the reference number first appears. Whenever convenient, the same reference numerals are used throughout the drawings to represent the same or similar parts. Although examples and features of the disclosed principles are described herein, modifications, adaptations, and other implementations are possible without departing from the scope of the disclosed embodiments.

[0027] Current ADAS systems are limited to only assisting the driver of the host vehicle to deal with potential hazards. Therefore, current ADAS systems cannot effectively avoid accidents or provide assistance in certain scenarios. This is because actors (such as other vehicles, pedestrians, or non-motorized vehicles) appearing on the road may not be equipped with ADAS systems.

[0028] The following are some scenarios where current ADAS systems may fail to warn, or they take too little time to take necessary actions to avoid collisions: (a) the car sensors cannot detect nearby targets (other vehicles, including non-motorized vehicles and pedestrians) due to obscured environment, (b) targets (other vehicles, including non-motorized vehicles and pedestrians) suddenly appear near the host vehicle, and (c) inattentive driving from the driver of the target vehicle (other vehicles, including non-motorized vehicles).

[0029] The present disclosure solves the technical problems in the art by using a symbiotic warning method and system installed in a host vehicle to identify these scenarios and provide necessary warnings to target vehicles, pedestrians, and animals around the host vehicle. Conventional ADAS systems issue warnings to alert the driver of the host vehicle, while the present symbiotic warning method and system automatically warns nearby target vehicles, pedestrians, and animals by controlling the host vehicle's communication devices such as horns, headlight flashers, or communicating with dedicated short-range communication (DSRC) technology.

[0030] Referring now to the drawings, and more particularly to Figures 1 to 7 , wherein like reference numerals denote corresponding features throughout the drawings, preferred embodiments are shown and described in the context of the following exemplary systems and / or methods.

[0031] Figure 1 is an exemplary block diagram of a symbiotic warning system 100 for warning nearby target actors to avoid collisions while driving, according to some embodiments of the present disclosure. In embodiments, the system 100 includes or otherwise communicates with one or more hardware processors 104, a communication interface device or input / output (I / O) interface 106, and one or more data storage devices or memories 102 operably coupled to the one or more hardware processors 104. The one or more hardware processors 104, the memories 102, and the I / O interfaces 106 may be coupled to a system bus 108 or similar mechanism.

[0032] The I / O interface 106 may include various software and hardware interfaces, such as a network interface, a graphical user interface, etc. The I / O interface 106 may include various software and hardware interfaces, such as interfaces for peripheral devices such as a keyboard, a mouse, an external memory, a plurality of sensor devices, a printer, etc. Further, the I / O interface 106 may enable the system 100 to communicate with other devices such as a web server and an external database.

[0033] The I / O interface 106 can facilitate multiple communications within a variety of network and protocol types, including wired networks (e.g., local area network (LAN), cable, etc.) and wireless networks (e.g., wireless LAN (WLAN), cellular, or satellite). To this end, the I / O interface 106 may include one or more ports for connecting multiple computing systems to each other or to another server computer. Further, the I / O interface 106 may include one or more ports for connecting multiple devices to each other or to another server.

[0034] The one or more hardware processors 104 may be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuits, and / or any device that manipulates signals based on operating instructions. Among other capabilities, the one or more hardware processors 104 are configured to fetch and execute computer-readable instructions stored in the memory 102. In the context of this disclosure, the expressions 'processor' and 'hardware processor' may be used interchangeably. In embodiments, the system 100 may be implemented in a variety of computing systems, such as laptop computers, portable computers, notebook computers, handheld devices, workstations, mainframe computers, servers, network clouds, and the like.

[0035] The memory 102 may include any computer-readable medium known in the art, including, for example, volatile memory (such as static random access memory (SRAM) and dynamic random access memory (DRAM)) and / or non-volatile memory (such as read-only memory (ROM), erasable programmable ROM, flash memory, hard disk, optical disk, and magnetic tape). In an embodiment, the memory 102 includes a plurality of modules 102a and a repository 102b for storing data processed, received, and generated by one or more of the plurality of modules 102a. The plurality of modules 102a may include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types.

[0036] The plurality of modules 102a may include programs or computer readable instructions or coded instructions that supplement the applications or functions performed by the system 100. The plurality of modules 102a may also function as signal processors, state machines, logic circuits, and / or any other device or component that operates signals based on operating instructions. Further, the plurality of modules 102a may be implemented by hardware, computer readable instructions executed by one or more hardware processors 104, or a combination thereof. In an embodiment, the plurality of modules 102a may include various sub-modules ( Figure 1 Further, the memory 102 may include information related to the input / output of each step performed by the processor 104 of the system 100 and method of the present disclosure.

[0037] The repository 102b may include a database or a data engine. Further, the repository 102b may act as a database or include multiple databases for storing data processed, received, or generated due to the execution of multiple modules 102a. Although the repository 102b is shown as being internal to the system 100, it should be noted that in alternative embodiments, the repository 102b may also be implemented external to the system 100, wherein the repository 102b may be stored in an external database ( Figure 1 The data contained in such an external database may be periodically updated. For example, data may be added to the external database and / or existing data may be modified and / or non-useful data may be deleted from the external database. In one example, the data may be stored in an external system such as a Lightweight Directory Access Protocol (LDAP) directory and a relational database management system (RDBMS). In another embodiment, the data stored in the repository 102 b may be distributed between the system 100 and the external database.

[0038] Common Reference Figure 2 and Figure 3 , the components and functions of the system 100 are described according to an exemplary embodiment of the present disclosure. For example, Figure 2 Some embodiments of the present disclosure are shown in Figure 1 FIG. 1 is an exemplary block diagram of a module 200 of a symbiosis warning system 100 for warning a nearby target actor for collision avoidance while driving. In an embodiment, the symbiosis warning system 100 is installed or equipped in a host vehicle.

[0039] like Figure 2 As shown, the module 200 includes a situation modeling module 202 , a scene semantics estimation module 204 , a key actor detection module 206 , a co-occurrence warning and collision avoidance module 208 , and an output device 210 . In an embodiment, the module 200 is stored in the repository 102 b of the system 100 .

[0040] In an embodiment, the situational modeling module 202 is configured to receive one or more road situational parameters to create a 360-degree scene perception of the road surroundings of the host vehicle. The 360-degree scene perception of the road surroundings of the host vehicle includes one or more actors that appear on the road within a certain range, proximity, or distance of the host vehicle when the host vehicle navigates on the road. In an embodiment, the one or more actors that appear on the road include, but are not limited to, one or more motor vehicles, one or more non-motor vehicles, pedestrians, and animals that appear on the road around the host vehicle.

[0041] In an embodiment, the one or more road context parameters relate to parameters of the road environment when the host vehicle is traveling on a road close to one or more actors. In an embodiment, the one or more road context parameters are received from one or more of the following installed or equipped in the host vehicle: one or more 360-degree light detection and ranging (radar), one or more front corner radars, one or more rear corner radio detection and ranging (radar), one or more cameras, one or more ultrasonic sensors, and one or more geographic road information devices or a combination of the above.

[0042] In an embodiment, 360-degree scene awareness of the road environment of the host vehicle generally includes one or more images and / or one or more sensor values, or a combination thereof, generated using one or more road context parameters associated with the road environment of the host vehicle as the host vehicle moves.

[0043] The scene semantics estimation module 204 is configured to estimate one or more three-dimensional (3-D) scene semantics of each of the one or more actors present on the road when the host vehicle is navigating. The 360-degree scene perception of the road environment of the host vehicle is analyzed using positioning technology to estimate one or more three-dimensional (3-D) scene semantics of each of the one or more actors present on the road when the host vehicle is navigating.

[0044] In an embodiment, one or more three-dimensional (3-D) scene semantics include, but are not limited to: the number of one or more motor vehicles, the current distance between the main vehicle and each of the one or more motor vehicles, the acceleration of each of the one or more motor vehicles, the size and dynamic characteristics of each of the one or more motor vehicles, the current moving direction of each of the one or more motor vehicles, the number of one or more non-motor vehicles, the current distance between the main vehicle and each of the one or more non-motor vehicles, the acceleration of each of the one or more non-motor vehicles, the size and dynamic characteristics of each of the one or more non-motor vehicles, the current moving direction of each of the one or more non-motor vehicles, the number of one or more pedestrians, the current distance between the main vehicle and each of the one or more pedestrians, the current moving direction of each of the one or more pedestrians, the number of one or more animals, the current distance between the main vehicle and each of the one or more animals, and the current moving direction of each of the one or more animals.

[0045] In an embodiment, the key actor detection module 206 is configured to detect one or more key actors from one or more actors appearing on a road when the host vehicle is navigating. The one or more key actors are actors that may cause a possible collision with the host vehicle when they are navigating on the road. The one or more key actors are detected by one or more three-dimensional (3-D) scene semantics estimated by the scene semantics estimation module 204. In an embodiment, the key actor detection module 206 utilizes path estimation and tracking techniques to detect one or more key actors that may cause a possible collision.

[0046] In an embodiment, the symbiotic warning and collision avoidance module 208 is configured to continuously decide when to generate a symbiotic warning signal and to which of the one or more key actors the symbiotic warning is provided. The symbiotic warning and collision avoidance module 208 uses a symbiotic warning training model to decide when to generate a symbiotic warning signal and to which of the one or more key actors the symbiotic warning is provided based on the 360-degree scene perception of the road environment created by the situation modeling module 202 and one or more three-dimensional (3-D) scene semantics estimated by the scene semantics estimation module 204.

[0047] In various embodiments, based on the type of key actor and the driving environment scenario, a symbiotic warning signal is generated by a horn, headlights, vehicle-to-vehicle communication, and a combination thereof equipped in the host vehicle. For example, if the host vehicle is a car and the surrounding environment is daylight, a symbiotic warning signal can be generated by a sounding horn. Similarly, if the host vehicle is a car and the surrounding environment is nighttime (dark environment), a symbiotic warning signal can be generated by flashing the headlights. Similarly, when the host vehicle is a vehicle similar to the host vehicle and if both have vehicle-to-vehicle communication, the symbiotic warning signal is in the form of vehicle-to-vehicle communication.

[0048] In an embodiment, the output device 210 includes one or more symbiotic warning signal devices, such as a horn, headlights, vehicle-to-vehicle communication equipped in the host vehicle. The output device 210 is configured to receive instructions determined by the symbiotic warning and collision avoidance module 208, and generate a symbiotic warning signal to one or more key actors based on the determination of the symbiotic warning and collision avoidance module 208 through one or more output devices (symbiotic warning signal devices).

[0049] refer to Figure 3 , the components and functions of the system 100 are described according to an exemplary embodiment of the present disclosure. For example, Figure 3 An exemplary flow chart of a processor-implemented symbiotic warning method 300 for warning nearby target actors to avoid collisions while driving in accordance with some embodiments of the present disclosure is shown. Although the steps of method 300, including process steps, method steps, techniques, etc., may be described in a continuous sequence, such processes, methods, and techniques may be configured to work in alternate sequences. In other words, any order or sequence of steps that may be described does not necessarily indicate a requirement to perform the steps in that order. The processing steps herein may be performed in any practical order. Further, some steps may be performed simultaneously, or some steps may be performed separately or independently.

[0050] At step 302 of method 300, one or more hardware processors 104 of system 100 are configured to receive one or more road situation parameters to create a 360-degree scene awareness of the road surroundings of the host vehicle through situation modeling module 202. The one or more road situation parameters relate to parameters of the road environment when the host vehicle is traveling on the road in proximity to one or more actors.

[0051] In an embodiment, one or more road situation parameters are received from one or more of the following: one or more 360-degree light detection and ranging (radars), one or more front corner radars, one or more rear corner radio detection and ranging (radars), one or more cameras, one or more ultrasonic sensors, and one or more geographic road information devices or a combination of the above installed in or equipped in the main vehicle.

[0052] In an embodiment, 360-degree scene perception of the host vehicle's road environment gives a visual representation of the host vehicle's environment (helps see the world around it) and allows the host vehicle to identify and classify one or more actors (things) in the environment during navigation of the host vehicle. In addition to one or more actors, the host vehicle needs to see and classify traffic lights, pedestrians, road signs, sidewalks, parking lots, lanes, etc. In addition, the host vehicle also needs to know the precise distance between itself and the things around it.

[0053] In an embodiment, 360-degree scene awareness of the road environment of the host vehicle generally includes one or more images, one or more sensor values, or a combination thereof generated using one or more road context parameters associated with the road environment of the host vehicle as the host vehicle moves.

[0054] In an embodiment, one or more image processing techniques may be used to create a 360-degree scene perception of the road environment of the host vehicle from one or more road context parameters. In an embodiment, the one or more image processing techniques include, but are not limited to, omnidirectional detection (OmniDet) and surround network (SurroundNet). Further, a deep learning-based object detection model may be used to identify one or more actors present on the road from the created 360-degree scene perception of the road environment. In an embodiment, some deep learning-based object detection models include, but are not limited to, region-based convolutional neural networks (R-CNN) and point cloud-based networks.

[0055] At step 304 of method 300, one or more hardware processors 104 of system 100 are configured to estimate one or more three-dimensional (3-D) scene semantics of each of one or more actors present on the road when the host vehicle is navigating, using 360-degree scene perception of the road environment, through scene semantics estimation module 204. As described, scene semantics estimation module 204 employs positioning technology to estimate one or more three-dimensional (3-D) scene semantics of each of the one or more actors.

[0056] In an embodiment, one or more three-dimensional (3-D) scene semantics include, but are not limited to: the number of one or more motor vehicles, the current distance between the main vehicle and each of the one or more motor vehicles, the acceleration of each of the one or more motor vehicles, the size and dynamic characteristics of each of the one or more motor vehicles, the current moving direction of each of the one or more motor vehicles, the number of one or more non-motor vehicles, the current distance between the main vehicle and each of the one or more non-motor vehicles, the acceleration of each of the one or more non-motor vehicles, the size and dynamic characteristics of each of the one or more non-motor vehicles, the current moving direction of each of the one or more non-motor vehicles, the number of one or more pedestrians, the current distance between the main vehicle and each of the one or more pedestrians, the current moving direction of each of the one or more pedestrians, the number of one or more animals, the current distance between the main vehicle and each of the one or more animals, and the current moving direction of each of the one or more animals.

[0057] In an embodiment, the localization technology includes one or more localization algorithms that calculate the position and orientation of the host vehicle relative to the position of one or more actors in the road environment. In an embodiment, a visual odometry (VO) algorithm is employed that works by matching key points in consecutive video frames, where a key point is a position and an orientation. For each frame, the key points are used as input to a mapping algorithm. A mapping algorithm, such as simultaneous localization and mapping (SLAM), calculates the position and orientation of each actor in the current frame relative to the previous frame and helps classify all actors on the road.

[0058] In embodiments, deep learning can be used to improve the performance of VO algorithms and classify different actors. Neural networks, such as PoseNet and VLocNet++, are some architectures that use point data to estimate 3-D positions and orientations. These estimated 3-D positions and orientations can be used to derive one or more three-dimensional (3-D) scene semantics.

[0059] At step 306 of the method 300, the one or more hardware processors 104 of the system 100 are configured to detect, through the key actor detection module 206, one or more key actors that may cause a collision from among the one or more actors that appear on the road when the host vehicle is navigating. The one or more key actors are detected using the one or more three-dimensional (3-D) scene semantics estimated at step 304 of the method 300. In one embodiment, the key actor detection module 206 detects the one or more key actors using path estimation and tracking techniques.

[0060] Figure 4 An exemplary flow chart for detecting one or more key actors from one or more actors present on a road when a host vehicle is navigating using path estimation and tracking techniques based on one or more three-dimensional (3-D) scene semantics according to some embodiments of the present disclosure is shown. Figure 4 As shown, detecting one or more key actors causing a possible collision from among one or more actors is explained through steps 306a to 306c.

[0061] In step 306a, one or more actors of interest are detected from one or more actors present on the road as the host vehicle navigates using path estimation and tracking techniques. In step 306b, (i) one or more predicted movements of each of the one or more actors of interest relative to the movement of the host vehicle are determined using path estimation and tracking techniques. In an embodiment, some methods in path estimation and tracking techniques include, but are not limited to, pure tracking algorithms and Kalman filters. Finally, in step 306c, (i) one or more predicted movements and (ii) current movements of each of the one or more actors of interest and the host vehicle are mapped to detect one or more actors of interest that may cause a collision. Current movement is real-time present or current movement.

[0062] The key actor detection module 206 detects one or more key actors that will be affected by a possible collision with the host vehicle. These one or more key actors may be vehicles in blind spots or occluded areas, lane-changing vehicles, non-motorized vehicles, pedestrians, etc. Further, the key actor detection module 206 calculates the time to collision (TTC), which is the distance between each actor and the host vehicle divided by their relative speed. The time to collision (TTC) is further used for warning calculations. In an embodiment, the key actor detection module 206 processes outputs from other existing ADAS systems, such as adaptive cruise control, emergency braking system, blind spot warning, rear / front cross traffic alert, lateral collision warning, etc. This ensures that existing algorithms are reused and duplication is avoided.

[0063] At step 308 of the method 300 , the one or more hardware processors 104 of the system 100 are configured to continuously determine, via the symbiosis warning and collision avoidance module 208 , when to generate a symbiosis warning signal and to which of the one or more focus actors causing a possible collision with the host vehicle to generate the symbiosis warning signal.

[0064] The symbiotic warning and collision avoidance module 208 uses the symbiotic warning training model to decide when to generate a symbiotic warning signal and to which of the one or more focus actors causing a possible collision with the host vehicle to generate a symbiotic warning signal based on the 360-degree scene perception of the road environment created by the situational modeling module 202 and one or more three-dimensional (3-D) scene semantics estimated by the scene semantics estimation module 204.

[0065] In various embodiments, based on the type of key actor and the driving environment scenario, a symbiotic warning signal is generated by a horn, headlights, vehicle-to-vehicle communication, and a combination thereof equipped in the host vehicle. For example, if the host vehicle is a car and the surrounding environment is daylight, a symbiotic warning signal can be generated by a sounding horn. Similarly, if the host vehicle is a car and the surrounding environment is nighttime (dark environment), a symbiotic warning signal can be generated by flashing the headlights. Similarly, when the host vehicle is a vehicle similar to the host vehicle and if both have vehicle-to-vehicle communication, the symbiotic warning signal is in the form of vehicle-to-vehicle communication.

[0066] Figure 5 An exemplary flow chart for obtaining a symbiotic warning training model according to some embodiments of the present disclosure is shown. Figure 5 As shown, the symbiotic warning training model is obtained through steps 308b1 to 308b4. In step 308b1, a training data set including a plurality of training samples is received. The training data set is associated with one or more driving environment scenarios. The training data set includes one or more training sub-data sets, wherein each training sub-data set is associated with each driving environment scenario.

[0067] In an embodiment, one or more driving environment scenarios include, but are not limited to: (i) lane-cutting driving, (ii) driving through a hairpin turn on the side of a hill, (iii) an abnormal / unintentional lane change by a key actor when the host vehicle is in a blind spot, (iv) overtaking the host vehicle in a narrow lane, (v) the sudden close appearance of pedestrians / animals / other vehicles on the road, (vi) merging lanes and T-intersection situations, and (vii) vehicle reversing.

[0068] In an embodiment, each training sub-dataset includes one or more training samples corresponding to a driving environment scene. Each training sample includes (i) 360-degree scene perception of a road environment of a training host vehicle associated with a plurality of training actors, and (ii) one or more training three-dimensional (3-D) scene semantics of the training host vehicle.

[0069] At step 308b2, each training sample present in each training sub-dataset is processed to obtain a plurality of processed training samples from the plurality of training samples. Each processed training sample includes states and actions of each training focus actor and training host vehicle defined from (i) training 360-degree scene perception of the road environment and (ii) one or more training three-dimensional (3-D) scene semantics. In an embodiment, the states and actions of each training focus actor and training host vehicle correspond to one of the one or more driving environment scenarios.

[0070] At step 308b3, a training symbiotic warning signal from a plurality of symbiotic warning signals is assigned to each processed training sample based on the state and action of each training focus actor and training host vehicle. In an embodiment, the plurality of symbiotic warning signals are generated by horns, headlights, vehicle-to-vehicle communications, and combinations thereof based on the type of focus actor and the driving environment scenario.

[0071] Table 1 shows exemplary processed training samples and their exemplary states that appear in each training sub-dataset associated with different driving scenarios. In Table 1, the states are defined according to their corresponding speeds.

[0072]

[0073]

[0074]

[0075] In step 308b4, the deep reinforcement learning neural network model is trained using each processed training sample at a time, using the assigned corresponding training symbiotic warning signals obtained in steps 308b2 and 308b3, respectively. The training process is repeated until a plurality of processed training samples are completed to obtain a symbiotic warning training model. In an embodiment, the deep reinforcement learning neural network model is a model from a list including but not limited to Q-learning and Markov decision process (MDP).

[0076] Figure 6 An exemplary flow chart for training a deep reinforcement learning neural network model according to some embodiments of the present disclosure is shown, wherein each processed training sample is used each time, and the corresponding training symbiotic warning signal is used until a plurality of processed training samples are completed to obtain a symbiotic warning training model. Figure 6 As shown, training a deep reinforcement learning neural network model to obtain a symbiotic warning training model is explained through steps 308c1 to 308c4.

[0077] At step 308c1, the state of the training host vehicle at each processed training sample is passed to a reinforcement learning (RL) agent of the deep reinforcement learning neural network model. At step 308c2, (i) the predicted actions of each training focus actor and the training host vehicle for a given state of the training host vehicle, and (ii) the predicted symbiotic warning signal from each processed training sample are obtained from the RL agent.

[0078] At step 308c3, (i) the predicted actions of each training focus actor and training host vehicle from each processed training sample are compared with the actions of each training focus actor and training host vehicle, and (ii) the predicted symbiotic warning signal is compared with the corresponding training symbiotic warning signal assigned to each processed training sample to provide a reward to the RL agent based on the comparison. At step 308c4, training of the reinforcement learning (RL) agent is performed using the continuously processed training samples until the plurality of processed training samples are completed to maximize the reward to the RL agent and obtain a symbiotic warning training model.

[0079] Figure 7 An exemplary flow chart is shown for using a symbiotic warning training model to determine, based on one or more three-dimensional (3-D) scene semantics and 360-degree road surrounding scene perception, the generation of one or more symbiotic warning signals to one or more key actors using symbiotic warning training models according to some embodiments of the present disclosure. Figure 7 As shown, the decision to generate one or more symbiotic warning signals to one or more key actors is explained through steps 308a1 to 308a3.

[0080] At step 308a1, the current state of the host vehicle and one or more key actors is determined using the 360-degree scene perception of the road environment created at step 302 of method 300. In an embodiment, the state is associated with the current dynamics of each of the one or more key actors on the road. At 308a2, the current actions of the host vehicle and one or more key actors are determined based on the current state of the host vehicle and one or more key actors determined at step 308a1 using one or more three-dimensional (3-D) scene semantics estimated at step 304 of method 300 and the 360-degree scene perception of the road environment created at step 302 of method 300. In an embodiment, the actions describe one or more possible movements of each of the one or more key actors and the host vehicle.

[0081] In step 308a3, (i) the current state of the host vehicle and one or more key actors, and (ii) the current actions of the host vehicle and one or more key actors determined in steps 308a1 and 308a2, respectively, are transmitted to the symbiotic warning training model to determine whether to generate a symbiotic warning signal to the one or more key actors that may cause a possible collision. When the current state of the host vehicle and one or more key actors, and the current actions of the host vehicle and one or more key actors are associated with one of the driving scenarios, a corresponding symbiotic warning signal is generated to alert the one or more key actors.

[0082] At step 310 of method 300, one or more hardware processors 104 of system 100 are configured to generate a symbiotic warning signal to one or more key actors causing a possible collision using one or more output devices via output device 210 based on the generated decision. In an embodiment, output device 210 includes one or more symbiotic warning signal devices, such as a horn, headlights, and vehicle-to-vehicle communication equipped in the host vehicle. Output device 210 is configured to receive instructions determined by symbiotic warning and collision avoidance module 208, and generate a symbiotic warning signal to one or more key actors causing a possible collision based on the generated decision by symbiotic warning and collision avoidance module 208 via one or more output devices (symbiotic warning signal devices).

[0083] The method and system of the present disclosure automatically warns nearby target vehicles, pedestrians and animals by controlling the host vehicle communication device such as the horn, headlight flashers or communicating using dedicated short-range communication (DSRC) technology. The symbiotic warning method and system are installed in the host vehicle to recognize these scenarios and give necessary warnings to target vehicles, pedestrians and animals around the host vehicle.

[0084] The symbiotic warning system of the present disclosure can be implemented on top of current ADAS systems, eliminating the need for separate hardware and reducing costs. The method and system of the present disclosure can be used to warn a target in a very fast time, so that many road fatal accidents can be avoided.

[0085] Embodiments of the present disclosure solve the unsolved problem of alerting target actors on the road by a host vehicle. The symbiotic warning method and system of the present disclosure automatically warns nearby target vehicles, pedestrians, and animals by controlling the host vehicle communication devices (such as horn, headlight flashers) or communicating using dedicated short-range communication (DSRC) technology.

[0086] The written description describes the subject matter herein to enable any person skilled in the art to make and use the embodiments. The scope of the subject matter embodiments is defined by the claims, and may include other modifications that occur to those skilled in the art. Such other modifications are intended to fall within the scope of the claims if they have similar elements that do not differ from the literal language of the claims, or if such other modifications include equivalent elements that do not differ substantially from the literal language of the claims.

[0087] It should be understood that, in addition to the computer-readable device having the message therein, the scope of protection also extends to such a program; such a computer-readable storage device contains program code means for implementing one or more steps of the method when the program is running on a server or mobile device or any suitable programmable device. The hardware device can be any kind of programmable device, including any kind of computer such as a server or a personal computer, or any combination thereof. The device may also include at least one microprocessor and at least one memory that may be, for example, a hardware device (e.g., an application specific integrated circuit (ASIC)), a field programmable gate array (FPGA), or a combination of hardware and software devices (e.g., an ASIC and an FPGA) or having a software processing component located therein. Thus, the device may include both hardware devices and software devices. The method embodiments described herein may be implemented in hardware and software. The device may also include software devices. Alternatively, the embodiments may be implemented on different hardware devices, for example, using multiple CPUs.

[0088] Embodiments herein may include hardware and software elements. Embodiments implemented in software include, but are not limited to, firmware, resident software, microcode, etc. The functions performed by the respective components described herein may be implemented in other components or in combinations of other components. For the purposes of this description, a computer-usable or computer-readable medium may be any instrument capable of containing, storing, communicating, propagating, or transmitting a program for use by or in conjunction with an instruction execution system, instrument, or device.

[0089] The steps shown are set forth to explain the exemplary embodiments shown, and it should be anticipated that ongoing technological development will change the way in which specific functions are performed. The examples presented herein are for the purpose of illustration rather than limitation. In addition, for ease of description, the boundaries of the functional building blocks have been arbitrarily defined herein. As long as the specified functions and their relationships are properly performed, alternative boundaries can be defined. Based on the teachings contained herein, it will be clear to those skilled in the relevant art that alternatives (including those equivalents, extensions, variations, deviations, etc. described herein) are sufficient. Such alternatives fall within the scope of the disclosed embodiments. Moreover, the words "comprising", "having", "including" and "including" and other similar forms are intended to be equivalent in meaning and open-ended, because one or more of any of these words after any one does not mean an exhaustive list of such or these items, or is not meant to be limited to the listed one or more. It must also be noted that, as used herein and in the appended claims, the singular forms "one", "an" and "the" include plural referents unless the content clearly indicates otherwise.

[0090] In addition, one or more computer-readable storage media can be used to implement embodiments consistent with the present disclosure. Computer-readable storage media refers to any type of physical memory that can store information or data that can be read by a processor. Thus, a computer-readable storage medium can store instructions for being executed by one or more processors, including instructions for causing the processor to execute steps or stages consistent with the embodiments described herein. The term "computer-readable medium" should be understood to include tangible items and exclude carrier waves and transient signals, i.e., non-transient signals. Examples include random access memory (RAM), read-only memory (ROM), volatile memory, non-volatile memory, hard disk drives, CD ROMs, DVDs, flash drives, optical disks, and any other known physical storage media.

[0091] It is intended that the disclosure and examples be considered exemplary only, with the true scope of the disclosed embodiments being indicated by the following claims.

Claims

1. A processor-implemented method (300), comprising the following steps: Receiving, via one or more hardware processors, one or more road context parameters to create a 360-degree scene awareness of a road environment of a host vehicle, wherein the 360-degree scene awareness of the road environment of the host vehicle includes one or more actors present on the road as the host vehicle navigates on the road (302); estimating, via the one or more hardware processors, using 360-degree scene perception of the road environment, using localization techniques, one or more three-dimensional (3-D) scene semantics (304) of each of the one or more actors present on the road as the host vehicle navigates; detecting, via the one or more hardware processors, one or more focal actors causing a possible collision from among the one or more actors present on the road as the host vehicle navigates based on the one or more three-dimensional (3-D) scene semantics using path estimation and tracking techniques (306); Determining, via the one or more hardware processors, to generate a symbiotic warning signal to one or more key actors using a symbiotic warning training model based on the one or more three-dimensional (3-D) scene semantics and the 360-degree scene perception of the road environment (308); and A symbiosis warning signal is generated, via the one or more hardware processors, to one or more key actors using one or more output devices based on the generated decision ( 310 ).

2. The processor-implemented method of claim 1 , wherein: Based on the one or more three-dimensional (3-D) scene semantics, detecting one or more key actors causing a possible collision from among the one or more actors present on the road as the host vehicle navigates using path estimation and tracking techniques, including: detecting one or more actors of interest from among the one or more actors present on the road as the host vehicle navigates using the path estimation and tracking techniques (306a); determining one or more predicted movements (306b) of (i) each of the one or more actors of interest and (ii) the host vehicle using the path estimation and tracking techniques; and Each of the one or more actors of interest and (i) one or more predicted movements and (ii) current movements of the host vehicle are mapped to detect one or more actors of interest that could cause a possible collision ( 306 c ).

3. The processor-implemented method of claim 1 , wherein: Determining, using the symbiotic warning training model, to generate one or more symbiotic warning signals to one or more key actors based on the one or more three-dimensional (3-D) scene semantics and the 360-degree scene perception of the road environment, comprising: determining a current state of the host vehicle and the one or more key actors using 360-degree scene awareness of the road environment, wherein the state is associated with a current dynamics of each of the one or more key actors on the road (308a1); determining, based on the current states of the host vehicle and the one or more key actors, current actions of the host vehicle and the one or more key actors using the one or more three-dimensional (3-D) scene semantics and 360-degree scene perception of the road environment, wherein the actions describe one or more possible movements of each of the one or more key actors and the host vehicle (308a2); and (i) the current state of the host vehicle and the one or more key actors, and (ii) the current actions of the host vehicle and the one or more key actors are transmitted to the symbiotic warning training model to determine whether to generate a symbiotic warning signal (308a3) to the one or more key actors causing a possible collision.

4. The processor-implemented method of claim 1, obtaining the symbiosis warning training model by: A training data set including a plurality of training samples is received, wherein: The training data set is associated with one or more driving environment scenarios and includes one or more training sub-data sets, wherein each training sub-data set is associated with each driving environment scenario and includes one or more training samples of the plurality of training samples, and wherein each training sample of the plurality of training samples includes (i) training 360-degree scene perception of a road environment of a training host vehicle and (ii) one or more training three-dimensional (3-D) scene semantics (308b1) of the training host vehicle associated with a plurality of training actors; Processing each of the plurality of training samples appearing in each training sub-dataset from (i) the training 360-degree scene perception of the road environment and (ii) one or more training three-dimensional (3-D) scene semantics to obtain a plurality of processed training samples from the plurality of training samples, wherein each processed training sample includes a state and action of each training focus actor and a training subject vehicle (308b2); allocating a training symbiosis warning signal from a plurality of symbiosis warning signals to each processed training sample based on the states and the actions of each training focus actor and training host vehicle (308b3); and The deep reinforcement learning neural network model is trained using the assigned associated training symbiosis warning signal with each processed training sample at a time until the plurality of processed training samples are completed to obtain a symbiosis warning training model (308b4).

5. The processor-implemented method of claim 3, wherein: Using each processed training sample at a time, training the deep reinforcement learning neural network model using the assigned associated training symbiotic warning signal until a plurality of processed training samples are completed to obtain the symbiotic warning training model includes: Passing the state of the training host vehicle for each processed training sample to the reinforcement learning RL agent (308c1) of the deep reinforcement learning neural network model; obtaining from the RL agent (i) a predicted action for each training focus actor and training host vehicle for a given state of the training host vehicle, and (ii) a predicted symbiotic warning signal from each processed training sample (308c2); comparing (i) the predicted actions of each training focus actor and training host vehicle from each processed training sample with the actions of each training focus actor and training host vehicle, and comparing (ii) the predicted symbiotic warning signal with the corresponding training symbiotic warning signal assigned to each processed training sample to provide a reward to the RL agent based on the comparison (308c3); and Training of the reinforcement learning RL agent is performed using the successively processed training samples until the plurality of processed training samples are completed to maximize a reward for the RL agent (308c4).

6. The processor-implemented method of claim 1 , wherein: The one or more actors present on the road include one or more motorized and non-motorized vehicles, pedestrians, and animals present on the road around the host vehicle, and Based on the type of the key actor and the driving environment scenario, the symbiotic warning signal is generated through a horn, headlights, vehicle-to-vehicle communication, and a combination thereof.

7. The processor-implemented method of claim 1 , wherein: The one or more road situation parameters are associated with road modeling and are received from one or more of the following: one or more 360-degree lidars, one or more front corner radars, one or more rear corner radars, one or more cameras, one or more ultrasonic sensors, and one or more geographic road information devices or a combination thereof installed in the main vehicle.

8. A system (100), comprising: A memory (102) for storing instructions; One or more input / output I / O interfaces (106); as well as One or more hardware processors (104) are coupled to the memory (102) via one or more I / O interfaces (106), wherein the one or more hardware processors (104) are configured by the instructions to: receiving one or more road context parameters to create a 360-degree scene awareness of a road environment of a host vehicle, wherein the 360-degree scene awareness of the road environment of the host vehicle includes one or more actors present on the road as the host vehicle navigates on the road; estimating, using 360-degree scene perception of the road environment, one or more three-dimensional (3-D) scene semantics of each of the one or more actors present on the road as the host vehicle navigates, using localization techniques; detecting, based on the one or more three-dimensional (3-D) scene semantics, one or more focal actors causing a possible collision from among the one or more actors present on the road as the host vehicle navigates using path estimation and tracking techniques; Determining, using a symbiotic warning training model, to generate a symbiotic warning signal to one or more key actors based on the one or more three-dimensional (3-D) scene semantics and the 360-degree scene perception of the road environment; and Based on the generated decision, a symbiosis warning signal is generated to one or more key actors using one or more output devices.

9. The system according to claim 8, wherein: The symbiosis warning and collision avoidance module (102b1) is configured via the one or more hardware processors (104) to detect one or more focal actors causing a possible collision from among the one or more actors present on the road as the host vehicle navigates based on the one or more three-dimensional (3-D) scene semantics using path estimation and tracking techniques by: detecting one or more actors of interest from among the one or more actors present on the road as the host vehicle navigates using the path estimation and tracking techniques; determining one or more predicted movements of (i) each of the one or more actors of interest and (ii) the host vehicle using the path estimation and tracking techniques; as well as (i) one or more predicted movements and (ii) current movements of each of the one or more actors of interest and the host vehicle are mapped to detect the one or more actors of interest that could cause a possible collision.

10. The system according to claim 8, wherein: The symbiotic warning and collision avoidance module (102b1) is configured via the one or more hardware processors (104) to determine, based on the one or more three-dimensional (3-D) scene semantics and the 360-degree scene perception of the road environment, using the symbiotic warning training model to generate one or more symbiotic warning signals to one or more key actors by: determining a current state of the host vehicle and the one or more key actors using 360-degree scene awareness of the road environment, wherein the state is associated with a current dynamics of each of the one or more key actors on the road; determining, based on the current states of the host vehicle and the one or more key actors, current actions of the host vehicle and the one or more key actors using the one or more three-dimensional (3-D) scene semantics and 360-degree scene awareness of the road environment, wherein the actions describe one or more possible movements of each of the one or more key actors and the host vehicle; and (i) the current state of the host vehicle and the one or more key actors, and (ii) the current actions of the host vehicle and the one or more key actors are transmitted to the symbiotic warning training model to determine whether to generate the symbiotic warning signal to the one or more key actors causing a possible collision.

11. The system according to claim 8, wherein: The one or more hardware processors (104) are configured to obtain the symbiotic warning training model by: Receiving a training data set including a plurality of training samples, wherein the training data set is associated with one or more driving environment scenarios and includes one or more training sub-data sets, wherein each training sub-data set is associated with each driving environment scenario and includes one or more training samples of the plurality of training samples, and wherein each training sample of the plurality of training samples includes (i) training 360-degree scene perception of a road environment of a training host vehicle and (ii) one or more training three-dimensional (3-D) scene semantics of the training host vehicle associated with a plurality of training actors; Processing each of the plurality of training samples appearing in each training sub-dataset from (i) the training 360-degree scene perception of the road environment and (ii) one or more training three-dimensional (3-D) scene semantics to obtain a plurality of processed training samples from the plurality of training samples, wherein each processed training sample includes a state and action of each training focus actor and a training subject vehicle; assigning a training symbiosis warning signal from a plurality of symbiosis warning signals to each processed training sample based on the states and the actions of each training focus actor and the training host vehicle; and The deep reinforcement learning neural network model is trained using the assigned associated training symbiosis warning signal with each processed training sample at a time until the plurality of processed training samples are completed to obtain the symbiosis warning training model.

12. The system according to claim 11, wherein: The one or more hardware processors (104) are configured to train the deep reinforcement learning neural network model using the assigned associated training symbiosis warning signal with each processed training sample at a time until the plurality of processed training samples are completed to obtain the symbiosis warning training model by: passing the state of the training host vehicle for each processed training sample to the reinforcement learning RL agent of the deep reinforcement learning neural network model; obtaining from the RL agent (i) a predicted action for each training focus actor and training host vehicle for a given state of the training host vehicle, and (ii) a predicted symbiotic warning signal from each processed training sample; comparing (i) the predicted actions of each training focus actor and training host vehicle from each processed training sample with the actions of each training focus actor and training host vehicle, and comparing (ii) the predicted symbiotic warning signal with the corresponding training symbiotic warning signal assigned to each processed training sample to provide a reward to the RL agent based on the comparison; as well as Training of the reinforcement learning RL agent is performed using successively processed training samples until the plurality of processed training samples are completed to maximize a reward for the RL agent.

13. The system of claim 8, wherein: The one or more actors present on the road include one or more motorized and non-motorized vehicles, pedestrians, and animals present on the road around the host vehicle, and Based on the type of the key actor and the driving environment scenario, the symbiotic warning signal is generated through a horn, headlights, vehicle-to-vehicle communication, and a combination thereof.

14. The system according to claim 8, wherein: The one or more road situation parameters are associated with road modeling and are received from one or more of the following: one or more 360-degree lidars, one or more front corner radars, one or more rear corner radars, one or more cameras, one or more ultrasonic sensors, and one or more geographic road information devices or a combination thereof installed in the main vehicle.

15. One or more non-transitory machine-readable information storage media comprising one or more instructions that, when executed by one or more hardware processors, cause: One or more road context parameters are received to create a 360-degree scene perception of the road environment of the host vehicle, wherein: 360-degree scene awareness of the host vehicle's road environment including one or more actors present on the road as the host vehicle navigates the road; estimating, using 360-degree scene perception of the road environment, one or more three-dimensional scene semantics of each of the one or more actors present on the road as the host vehicle navigates, using localization techniques; detecting, based on the one or more three-dimensional scene semantics, one or more focal actors causing a possible collision from among the one or more actors present on the road as the host vehicle navigates using path estimation and tracking techniques; Determining, using a symbiotic warning training model, to generate a symbiotic warning signal to one or more key actors based on the one or more three-dimensional scene semantics and the 360-degree scene perception of the road environment; as well as Based on the generated decision, a symbiosis warning signal is generated to one or more key actors using one or more output devices.

Images