Method for detecting and managing iced uphill roads in autonomous vehicles

Abstract

Embodiments relate to a system wherein the system is operable to determine a road surface condition wherein the road surface condition is at least one of ice, wet, and snow; adjusting the safety distance value of the vehicle based on the road surface condition; detecting the vehicle type, speed and visible roof area of the target vehicle; determining an uphill road on which the vehicle is approaching; determining that the target vehicle is decelerating based on the real-time speed change of the target vehicle; and determining a collision avoidance action for the host vehicle to avoid collision with the target vehicle.

Description

Technical Field

[0001] The present disclosure relates to the field of vehicle safety. More specifically, the present disclosure relates to systems and methods for detecting and managing icy uphill roads in autonomous vehicles. Background Art

[0002] Driving on icy uphill roads can be dangerous and challenging due to reduced traction, increased sliding, stalling, extended braking distances and reduced visibility.

[0003] Therefore, there is a need to detect and manage icy uphill roads in autonomous vehicles, and if there is a risk of collision when driving on icy uphill roads, provide warnings and / or take actions to minimize or avoid the collision. Summary of the invention

[0004] An overview is provided below to provide a basic understanding of one or more embodiments described herein. This summary is not intended to identify key or core elements or to describe any scope of different embodiments and / or any scope of claims. The sole purpose of this summary is to present some concepts in a simplified form as a prelude to a more detailed description presented herein.

[0005] According to one embodiment, it is a system comprising a processor, a computer vision module and a control module; wherein the system is operable to determine the road surface condition through the control module of the vehicle, wherein the road surface condition is at least one of ice, wetness and snow; the control module of the vehicle adjusts the safety distance value of the vehicle based on the road surface condition; the control module of the vehicle detects the vehicle type, speed and visible roof area of ​​the target vehicle, wherein the target vehicle is the vehicle in front of the vehicle; the control module of the vehicle determines that the vehicle is approaching an uphill road; the control module of the vehicle determines that the target vehicle is decelerating based on the real-time speed change of the target vehicle; the processor predicts a collision between the vehicle and the target vehicle; the control module of the vehicle determines the collision avoidance action of the vehicle to avoid a collision with the target vehicle.

[0006] According to one embodiment, it is a method, comprising: determining, by a processor, a road surface condition, wherein the road surface condition is at least one of ice, wetness and snow; adjusting, by the processor, a safety distance value of the host vehicle based on the road surface condition; detecting, by a computer vision module, a vehicle type, a speed and a visible roof area of ​​a target vehicle, wherein the target vehicle is a vehicle in front of the host vehicle; determining, by the processor, that the host vehicle is approaching an uphill road; determining, by the processor, that the target vehicle is decelerating based on real-time speed changes of the target vehicle; predicting, by the processor, a collision between the host vehicle and the target vehicle; and determining, by the processor, a collision avoidance action for the host vehicle to avoid a collision with the target vehicle.

[0007] According to one embodiment, it is a non-transitory computer-readable medium having instructions stored thereon, which can be executed by a computer system to perform operations, the operations including: determining a road surface condition by the processor, wherein the road surface condition is at least one of ice, wetness and snow; adjusting a safety distance value of the host vehicle based on the road surface condition by the processor; detecting a vehicle type, speed and visible roof area of ​​a target vehicle through a computer vision module, wherein the target vehicle is a vehicle in front of the host vehicle; determining by the processor that the host vehicle is approaching an uphill road; determining by the processor that the target vehicle is decelerating based on real-time speed changes of the target vehicle; predicting a collision between the host vehicle and the target vehicle by the processor; and determining a collision avoidance action for the host vehicle to avoid a collision with the target vehicle by the processor.

[0008] According to a first aspect, a system is provided, comprising: a processor, a computer vision module and a control module; wherein the system is operable to: determine a road surface condition by the control module of the vehicle, wherein the road surface condition is at least one of ice, wetness and snow; adjust a safety distance value of the vehicle based on the road surface condition by the control module of the vehicle; detect a vehicle type, speed and visible roof area of ​​a target vehicle by the control module of the vehicle, wherein the target vehicle is a vehicle in front of the vehicle; determine an uphill road that the vehicle is approaching by the control module of the vehicle; determine that the target vehicle is decelerating based on the real-time speed change of the target vehicle by the control module of the vehicle; predict a collision between the vehicle and the target vehicle by the processor; and determine a collision avoidance action of the vehicle to avoid a collision with the target vehicle by the control module of the vehicle.

[0009] According to an embodiment of the first aspect, the system is operable as a component of the host vehicle.

[0010] According to an embodiment of the first aspect, the road surface condition is determined by receiving a real-time weather broadcast.

[0011] According to an embodiment of the first aspect, the road surface condition is determined from a map showing icy roads in the route.

[0012] According to an embodiment of the first aspect, the road surface condition is determined by detecting the road condition in real time via the computer vision module. According to an embodiment, the road surface condition is determined by analyzing the scattering of the emitted light beam on the road surface using a filtering technique on an image captured by the computer vision module.

[0013] According to an embodiment of the first aspect, the road surface condition is detected by using vehicle dynamics sensor data and an image of the surrounding environment in front of the host vehicle obtained by a camera, wherein the camera is a monochrome camera. According to an embodiment, a surface condition observer classifies the road condition based on an evaluation of the image from the camera and comparing the image with the vehicle dynamics sensor data. According to an embodiment, the surface condition observer further considers local weather data and derives the road surface condition, wherein the local weather data includes temperature, humidity value, rainfall, sleet, snowfall, wiper activity, and cloud data.

[0014] According to an embodiment of the first aspect, the road surface condition is detected by recognizing road signs using the computer vision module.

[0015] According to an embodiment of the first aspect, the road surface condition is detected by vehicle-to-infrastructure (V2I) communication.

[0016] According to an embodiment of the first aspect, the road surface condition is determined by determining that a neighboring vehicle has navigation difficulty.

[0017] According to an embodiment of the first aspect, the road surface condition is determined by receiving a broadcast message about the road surface condition from neighboring vehicles.

[0018] According to an embodiment of the first aspect, the road surface condition is determined by a coefficient of friction between a road on which the host vehicle is travelling and tires of the host vehicle via an accelerometer sensor.

[0019] According to an embodiment of the first aspect, the road surface condition is determined by a road condition sensor, wherein the road condition sensor is at least one of a temperature sensor, an infrared sensor, an accelerometer, a GPS sensor, and a humidity sensor.

[0020] According to an embodiment of the first aspect, the safety distance value is set to a higher value than a value in dry road surface conditions.

[0021] According to an embodiment of the first aspect, the system is further operable to determine a new route having a minimum number of icy roads.

[0022] According to an embodiment of the first aspect, the vehicle type is detected by detecting the license plate number of the target vehicle via the computer vision module. According to an embodiment, the license plate number is matched with the vehicle type by referencing a database.

[0023] According to an embodiment of the first aspect, the host vehicle is further operable to detect the vehicle type and vehicle model of the target vehicle.

[0024] According to an embodiment of the first aspect, the speed of the target vehicle is determined in real time via a computer vision module including a camera, LiDAR, GPS and radar.

[0025] According to an embodiment of the first aspect, the computer vision module also includes an artificial intelligence engine and a machine learning algorithm.

[0026] According to an embodiment of the first aspect, the speed of the target vehicle is determined in real time via radar.

[0027] According to an embodiment of the first aspect, a bounding box algorithm is used to detect the visible roof area of ​​the target vehicle. According to an embodiment, the bounding box algorithm is used in combination with a machine learning algorithm for object detection, wherein the machine learning algorithm is a convolutional neural network (CNN).

[0028] According to an embodiment of the first aspect, the uphill road is determined based on a vehicle type of the target vehicle, a speed of the target vehicle, and a visible roof area of ​​the target vehicle.

[0029] According to an embodiment of the first aspect, the uphill road is determined based on a GPS position of the target vehicle and map data to determine the slope of the uphill road.

[0030] According to an embodiment of the first aspect, the host vehicle maps the visible roof area by the vehicle type, vehicle model and the speed to determine the inclination of the uphill road.

[0031] According to one embodiment of the first aspect, the host vehicle is operable to determine the inclination of the uphill road by detecting changes in the visible roof area by taking a sequence of consecutive readings timed at predetermined intervals.

[0032] According to one embodiment of the first aspect, the host vehicle is operable to determine the inclination of the uphill road by detecting changes in the visible roof area of ​​a vehicle approaching in another lane in an opposite direction to the target vehicle by taking a sequence of consecutive readings timed at predetermined intervals.

[0033] According to an embodiment of the first aspect, the host vehicle is operable to detect the beginning of the uphill road and the end of the uphill road.

[0034] According to an embodiment of the first aspect, the host vehicle is operable to notify a driver of the host vehicle of an approaching uphill road via a notification, wherein the notification is at least one of a sound, a tactile prompt, a visual prompt and a text message.

[0035] According to an embodiment of the first aspect, the host vehicle is operable to detect a plurality of lanes on the uphill road via the computer vision module.

[0036] According to an embodiment of the first aspect, the host vehicle is operable to correlate the visible roof area based on the vehicle type and vehicle model in order to determine the inclination of the uphill road.

[0037] According to an embodiment of the first aspect, the safety distance value of the host vehicle is further increased based on the inclination of the uphill road.

[0038] According to an embodiment of the first aspect, the speed change of the target vehicle is calculated by the host vehicle based on speed data collected over time.

[0039] According to an embodiment of the first aspect, the host vehicle is operable to detect whether the target vehicle is stalling.

[0040] According to an embodiment of the first aspect, the host vehicle is operable to detect whether the target vehicle is losing traction.

[0041] According to an embodiment of the first aspect, the host vehicle is operable to predict whether a speed of the target vehicle is approaching zero.

[0042] According to one embodiment of the first aspect, the host vehicle is operable to predict via the computer vision module using an artificial intelligence algorithm whether the target vehicle will move backwards on the uphill road.

[0043] According to one embodiment of the first aspect, the host vehicle is operable to detect via the computer vision module comprising radar that the target vehicle is moving backwards on the uphill road.

[0044] According to an embodiment of the first aspect, the host vehicle is operable via a rear view camera to detect a vehicle behind the host vehicle.

[0045] According to an embodiment of the first aspect, the host vehicle detects vehicles in other lanes via LiDAR and radar.

[0046] According to one embodiment of the first aspect, the host vehicle is operable to establish communication with the target vehicle via a communication module so as to obtain information about the state of the target vehicle, wherein the state includes a speed change and an uphill road inclination of the target vehicle. According to one embodiment, the uphill road inclination is obtained using a longitudinal accelerometer sensor, GPS, an inertial measurement unit (IMU), LiDAR, a radar, and a camera. According to another embodiment, the communication between the host vehicle and the target vehicle is performed via vehicle-to-vehicle (V2V) communication. According to one embodiment, the vehicle-to-vehicle (V2V) communication is based on a wireless communication protocol using at least one of dedicated short-range communication (DSRC) and cellular vehicle-to-everything (C-V2X) technologies. According to yet another embodiment, the communication between the host vehicle and the target vehicle is performed via an Internet connection.

[0047] According to an embodiment of the first aspect, the collision avoidance action includes generating an alert in the host vehicle, wherein the alert is at least one of a text message, a visual prompt, an audio alert, a tactile prompt, and a vibration.

[0048] According to an embodiment of the first aspect, the collision avoidance action includes further increasing the safety distance value between the host vehicle and the target vehicle.

[0049] According to an embodiment of the first aspect, the collision avoidance action includes waiting until the target vehicle passes the uphill road.

[0050] According to an embodiment of the first aspect, the collision avoidance action comprises at least one of: a reverse movement initiated by the host vehicle; and a lane change initiated by the host vehicle.

[0051] According to an embodiment of the first aspect, the collision avoidance action includes broadcasting a message to vehicles behind the host vehicle that the target vehicle is sliding down the uphill road.

[0052] According to an embodiment of the first aspect, the collision avoidance action comprises broadcasting a message to neighboring vehicles, wherein the message comprises maneuvering instructions to the neighboring vehicles to avoid the imminent collision using collective and coordinated action.

[0053] According to an embodiment of the first aspect, the collision avoidance action comprises predicting the collision and alerting a driver of the host vehicle about the deployment of an airbag during the collision.

[0054] According to an embodiment of the first aspect, the collision avoidance action includes predicting the collision, alerting the driver of the host vehicle and suggesting that the driver prepare for the impact posture.

[0055] According to an embodiment of the first aspect, the collision avoidance action comprises predicting the collision, alerting the driver of the host vehicle and suggesting him to leave the host vehicle.

[0056] According to an embodiment of the first aspect, the collision avoidance action comprises alerting the target vehicle of an impending collision.

[0057] According to an embodiment of the first aspect, the collision avoidance action comprises alerting neighboring vehicles of the collision.

[0058] According to an embodiment of the first aspect, the collision avoidance action includes at least one of the following: generating a warning in the host vehicle; maintaining at least the safe distance value between the host vehicle and the target vehicle; initiating reverse movement by the host vehicle; initiating lane change by the host vehicle; warning the target vehicle of an impending collision; and broadcasting a message to neighboring vehicles, wherein the message includes instructions for maneuvering the neighboring vehicles to use collective and coordinated actions to avoid the impending collision.

[0059] According to an embodiment of the first aspect, the collision avoidance action is determined and executed in real time until the host vehicle passes the uphill road.

[0060] According to an embodiment of the first aspect, the collision avoidance maneuver is determined via a computer vision module comprising an artificial intelligence engine, wherein the artificial intelligence engine comprises a machine learning algorithm.

[0061] According to an embodiment of the first aspect, the collision avoidance action further comprises activating a camera of the computer vision module to record the surrounding environment and the host vehicle.

[0062] According to an embodiment of the first aspect, the collision avoidance action comprises alerting a third party of the impending collision. According to an embodiment, the third party is an emergency service.

[0063] According to an embodiment of the first aspect, the system is further configured for autonomously performing the collision avoidance action by the host vehicle.

[0064] According to a second aspect of the present invention, a method is provided, which includes: determining, by a processor, a road surface condition, wherein the road surface condition is at least one of ice, wetness and snow; adjusting, by the processor, a safety distance value of the host vehicle based on the road surface condition; detecting, by a computer vision module, a vehicle type, speed and visible roof area of ​​a target vehicle, wherein the target vehicle is a vehicle in front of the host vehicle; determining, by the processor, that the host vehicle is approaching an uphill road; determining, by the processor, that the target vehicle is decelerating based on real-time speed changes of the target vehicle; predicting, by the processor, a collision between the host vehicle and the target vehicle; and determining, by the processor, a collision avoidance action for the host vehicle to avoid a collision with the target vehicle.

[0065] According to an embodiment of the second aspect, the road surface condition is determined by detecting the road condition in real time via the computer vision module.

[0066] According to an embodiment of the second aspect, the uphill road is determined based on a vehicle type of the target vehicle, a speed of the target vehicle, and a visible roof area of ​​the target vehicle.

[0067] According to an embodiment of the second aspect, the collision avoidance action includes generating a warning in the host vehicle, wherein the warning is at least one of a text message, a visual prompt, an audio warning, a tactile prompt and a vibration.

[0068] According to an embodiment of the second aspect, the collision avoidance action includes further increasing the safety distance value between the host vehicle and the target vehicle.

[0069] According to an embodiment of the second aspect, the collision avoidance action includes waiting until the target vehicle passes the uphill road.

[0070] According to an embodiment of the second aspect, the collision avoidance action comprises at least one of: a reverse movement initiated by the host vehicle; and a lane change initiated by the host vehicle.

[0071] According to an embodiment of the second aspect, the collision avoidance action includes broadcasting a message to vehicles behind the host vehicle that the target vehicle is sliding down from the uphill road.

[0072] According to an embodiment of the second aspect, the collision avoidance action comprises broadcasting a message to neighboring vehicles, wherein the message comprises instructions for maneuvering of the neighboring vehicles to avoid the imminent collision using collective and coordinated action.

[0073] According to an embodiment of the second aspect, the collision avoidance action includes predicting the collision and alerting the driver of the host vehicle of the deployment of an airbag during the collision.

[0074] According to an embodiment of the second aspect, the collision avoidance action includes predicting the collision, alerting the driver of the host vehicle and suggesting that the driver prepare for the collision posture.

[0075] According to an embodiment of the second aspect, the collision avoidance action comprises predicting the collision, alerting the driver of the host vehicle and suggesting him to leave the host vehicle.

[0076] According to an embodiment of the second aspect, the collision avoidance action includes alerting the target vehicle of an impending collision.

[0077] According to an embodiment of the second aspect, the method is further configured for autonomously performing the collision avoidance action by the host vehicle.

[0078] According to a third aspect of the present invention, there is provided a non-temporary computer-readable medium having instructions stored thereon, the instructions being executable by a computer system to perform operations including: determining, by a processor, a road surface condition, wherein the road surface condition is at least one of ice, wetness and snow; adjusting, by the processor, a safety distance value of the host vehicle based on the road surface condition; detecting, by a computer vision module, a vehicle type, speed and visible roof area of ​​a target vehicle, wherein the target vehicle is a vehicle in front of the host vehicle; determining, by the processor, that the host vehicle is approaching an uphill road; determining, by the processor, that the target vehicle is decelerating based on real-time speed changes of the target vehicle; predicting, by the processor, a collision between the host vehicle and the target vehicle; and determining, by the processor, a collision avoidance action for the host vehicle to avoid a collision with the target vehicle.

[0079] According to an embodiment of the third aspect, the road surface condition is determined by detecting the road condition in real time via the computer vision module.

[0080] According to an embodiment of the third aspect, the uphill road is determined based on a vehicle type of the target vehicle, a speed of the target vehicle, and a visible roof area of ​​the target vehicle.

[0081] According to an embodiment of the third aspect, the collision avoidance action includes generating a warning in the host vehicle, wherein the warning is at least one of a text message, a visual prompt, an audio warning, a tactile prompt and a vibration.

[0082] According to an embodiment of the third aspect, the collision avoidance action includes further increasing the safety distance value between the host vehicle and the target vehicle.

[0083] According to an embodiment of the third aspect, the non-transitory computer-readable medium is further configured for autonomously performing the collision avoidance action by the host vehicle.

[0084] According to an embodiment of the third aspect, the collision avoidance action comprises at least one of: a reverse movement initiated by the host vehicle; and a lane change initiated by the host vehicle.

[0085] According to an embodiment of the third aspect, the collision avoidance action includes broadcasting a message to vehicles behind the host vehicle that the target vehicle is sliding down from the uphill road.

[0086] According to an embodiment of the third aspect, the collision avoidance action comprises broadcasting a message to neighboring vehicles, wherein the message comprises instructions for maneuvering of the neighboring vehicles to avoid the imminent collision using collective and coordinated action.

[0087] According to an embodiment of the third aspect, the collision avoidance action includes at least one of the following: generating a warning in the host vehicle; maintaining at least the safe distance value between the host vehicle and the target vehicle; initiating reverse movement by the host vehicle; initiating lane change by the host vehicle; warning the target vehicle of an impending collision; and broadcasting a message to neighboring vehicles, wherein the message includes instructions for maneuvering the neighboring vehicles to use collective and coordinated actions to avoid the impending collision. BRIEF DESCRIPTION OF THE DRAWINGS

[0088] These and other aspects of the present invention will now be described in more detail with reference to the accompanying drawings showing exemplary embodiments of the present invention, in which:

[0089] Figure 1 A scenario is shown when a vehicle is traveling on an uphill road under icy conditions according to one embodiment.

[0090] Figure 2A is an illustration of an exemplary autonomous vehicle with various sensors, actuators, and systems according to one embodiment.

[0091] Figure 2B A block diagram of electronic components of a vehicle is shown according to one embodiment.

[0092] Figure 3A A block diagram of a method for avoiding collision based on road surface conditions according to one embodiment is shown.

[0093] Figure 3B A block diagram of a system for avoiding collisions based on road surface conditions according to one embodiment is shown.

[0094] Figure 3CA block diagram of a method for avoiding collisions based on road surface conditions stored on a non-transitory computer readable medium according to one embodiment is shown.

[0095] Figure 4A Shown are portions of a vehicle detected using a bounding box approach according to one embodiment.

[0096] Figure 4B is a process flow diagram of a method for identifying and classifying one or more objects according to one embodiment of the present invention.

[0097] Figure 4C An autonomous vehicle is shown detecting the roof of another vehicle according to one embodiment.

[0098] Figure 4D An autonomous vehicle is shown continuously detecting the roof of another vehicle according to one embodiment.

[0099] Figure 4E An autonomous vehicle is shown detecting the roof of another vehicle being towed by a trailer, according to one embodiment.

[0100] Figure 4F An autonomous vehicle is shown continuously detecting the roof of another vehicle according to one embodiment.

[0101] Figure 4G An autonomous vehicle is shown detecting the type, model, and make of a vehicle according to one embodiment.

[0102] Figure 4H An autonomous vehicle is shown associating roof area with road grade based on vehicle type according to one embodiment.

[0103] Fig. 4I An autonomous vehicle is shown determining the grade of a road based on image analysis of the road ahead according to one embodiment.

[0104] Figure 4J An autonomous vehicle is shown determining the grade of a road according to one embodiment.

[0105] Figure 4K An autonomous vehicle is shown determining a start point of an uphill road according to one embodiment.

[0106] Figure 4L An autonomous vehicle is shown determining a road surface condition for an uphill road according to one embodiment.

[0107] Figure 4M A road surface sign warning of icy roads according to one embodiment is shown.

[0108] Figure 4NAn autonomous vehicle providing a warning about road surface conditions to an occupant of another autonomous vehicle is shown according to one embodiment.

[0109] Fig.4O A block diagram of a lane detection system according to one embodiment is shown.

[0110] Figure 4P An autonomous vehicle is shown detecting the end of an uphill road according to one embodiment.

[0111] Figure 4Q An autonomous vehicle is shown classifying an uphill road according to one embodiment.

[0112] Figure 4R An autonomous vehicle is shown detecting the speed of a target vehicle in real time according to one embodiment.

[0113] Figure 4S An autonomous vehicle is shown deciding on a collision avoidance action based on the speed of a target vehicle according to one embodiment.

[0114] Figure 5 A method for the vehicle to avoid a collision on an icy uphill road section according to one embodiment is shown.

[0115] Figure 6 An example is shown in which the host vehicle warns nearby vehicles that a target vehicle is sliding on an uphill section.

[0116] Figure 7 An example message of a dedicated short range communication (DSRC) message format is shown.

[0117] Fig. 8A An example message from a host vehicle to nearby / neighboring vehicles is shown according to one embodiment.

[0118] Figure 8B An example message from a target vehicle to other vehicles is shown according to one embodiment.

[0119] Figure 8C An example warning message displayed in the present vehicle is shown according to one embodiment.

[0120] Fig.9A The structure of a neural network / machine learning model with a feedback loop according to one embodiment is shown.

[0121] Fig. 9B The structure of a neural network / machine learning model with reinforcement learning according to one embodiment is shown.

[0122] Fig. 9C An example block diagram for detecting collision events using a machine learning model according to one embodiment is shown.

[0123] Fig.10 An example flow chart for detecting collision events using a machine learning model is shown according to one embodiment.

[0124] Fig.11A A block diagram of a network security module for a system and a server is shown.

[0125] Fig. 11B One embodiment of a network security module according to one embodiment of the system is shown.

[0126] Fig. 11C Another embodiment of a network security module according to one embodiment of the system is shown. DETAILED DESCRIPTION

[0127] For simplicity and clarity of illustration, the drawings illustrate the general manner of construction. The description and drawings may omit descriptions and details of well-known features and techniques to avoid unnecessarily obscuring the present disclosure. The drawings exaggerate the size of some elements relative to other elements to help improve the understanding of the embodiments of the present disclosure. The same reference numerals in different drawings represent the same elements.

[0128] Although the detailed description herein contains many specifics for the purpose of illustration, one of ordinary skill in the art will appreciate that many variations and changes to the details are considered to be included herein.

[0129] Therefore, the examples herein do not have any loss of generality to any of the claims set forth, nor do they impose limitations thereon. The terms used herein are for the purpose of describing specific embodiments only and are not limiting. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the art to which the present disclosure belongs.

[0130] As used herein, the qualifiers "a" and "an" refer to one or more than one (i.e., at least one) of the grammatical object of the qualifier. For example, "an / an element" means one element or more than one element. In addition, the use of the qualifiers "a" and "an" in this specification and the drawings is to be interpreted as meaning "one or more" unless otherwise specified or the context clearly indicates a singular form.

[0131] As used herein, the terms "example" and / or "exemplary" are meant to serve as an example, instance, or illustration. For the avoidance of doubt, such examples do not limit the subject matter described herein. Furthermore, any aspect or design described herein as "example" and / or "exemplary" is not necessarily preferred or advantageous over other aspects or designs, nor does it exclude equivalent exemplary structures and techniques known to those of ordinary skill in the art.

[0132] As used herein, the terms "first", "second", "third", etc. (if any) in the specification and claims distinguish similar elements and do not necessarily describe a specific sequence or chronological order. These terms are interchangeable where appropriate, so that the embodiments of this document can, for example, operate in a sequence different from that shown or otherwise described herein. In addition, the terms "including", "having" and any variations thereof cover non-exclusive inclusions, so that the process, method, system, article, device or equipment including a list of elements is not necessarily limited to those elements, but may include other elements that are not explicitly listed or inherent to such processes, methods, systems, articles, devices or equipment.

[0133] As used herein, the terms "left", "right", "front", "back", "top", "bottom", "upper", "lower", etc., if any, in the specification and claims are used for descriptive purposes and not necessarily for describing permanent relative positions. The terms so used are interchangeable where appropriate, such that embodiments of the apparatus, methods, and / or articles described herein are, for example, capable of operation in other orientations than those illustrated or otherwise described herein.

[0134] Unless so clearly described, any element action or instruction used in this article is not critical or necessary. In addition, the term "set" includes items (e.g., related items, unrelated items, combinations of related items and unrelated items, etc.) and can be interchanged with "one or more". In the case of only one item expected, the term "one" or similar language is used. In addition, the terms "having", "containing", "including", etc. are open terms. In addition, unless otherwise clearly stated, the phrase "based on" means "based at least in part on".

[0135] As used herein, the terms "system", "device", "unit" and / or "module" refer to different components, component parts or components of each level of structure. However, other expressions that achieve the same purpose can replace these terms.

[0136] As used herein, the terms "coupled", "coupled", etc. refer to connecting two or more elements mechanically, electrically, and / or otherwise. Two or more electrical elements may be electrically coupled together, but not mechanically or otherwise coupled together. The coupling / coupling may last for any length of time, such as permanently or semi-permanently or only for a moment. "Electrically coupled" includes all types of electrical couplings. The absence of the words "removable", "removably", etc. near the words "coupled / coupling", etc. does not mean that the coupling / coupling, etc. in question is or is not removable.

[0137] As used herein, the term "or" refers to an inclusive "or" rather than an exclusive "or." That is, unless specified otherwise or clear from the context, "X employs A or B" is intended to mean any of the natural inclusive permutations. That is, if X employs A; X employs B; or X employs both A and B, then "X employs A or B" is satisfied in any of the foregoing instances.

[0138] As used herein, two or more elements are "unitary" if they comprise the same piece of material. Two or more elements are "non-unitary" if they comprise different pieces of material.

[0139] or - (using appropriate definition)

[0140] As used herein, two or more elements or modules are "unitary" or "integrated" if they functionally operate together. Two or more elements are "non-unitary" if each element can functionally operate independently.

[0141] As used herein, the term "real time" refers to operations performed as quickly as possible when a triggering event occurs. A triggering event can include the receipt of data necessary for performing a task or otherwise processing information. Due to the inherent delays in transmission and / or computing speeds, the term "real time" encompasses operations that occur in a "near" real time or slightly delayed manner from a triggering event. In a number of embodiments, "real time" can represent real time minus the time delay for processing (e.g., determining) and / or transmitting data. A specific time delay can vary according to the type and / or amount of data, the processing speed of the hardware, the transmission capability of the communication hardware, the transmission distance, etc. However, in many embodiments, the time delay can be less than approximately one second, two seconds, five seconds, or ten seconds.

[0142] As used herein, the term "approximately" can mean within a specified or unspecified range of a specified or unspecified stated value. In some embodiments, "approximately" can mean within plus or minus ten percent of the stated value. In other embodiments, "approximately" can mean within plus or minus five percent of the stated value. In other embodiments, "approximately" can mean within plus or minus three percent of the stated value. In other embodiments, "approximately" can mean within plus or minus one percent of the stated value.

[0143] Without departing from the spirit or features of the present invention, other specific forms may embody the present invention. The described embodiments are illustrative and non-restrictive in all respects. Therefore, the appended claims rather than the description herein indicate the scope of the present invention. All variations falling within the meaning and scope of the equivalents of the claims are within their scope.

[0144] As used herein, the term "component" is broadly interpreted as hardware, firmware, and / or a combination of hardware, firmware, and software.

[0145] Digital electronic circuits or computer software, firmware or hardware, including the structures disclosed herein and their structural equivalents, or a combination of one or more of them, can implement the implementations and all functional operations described herein. The implementation can be one or more computer program products, that is, one or more modules of computer program instructions encoded on a computer-readable medium, for being executed by a data processing device or controlling the operation of a data processing device. The computer-readable medium can be a combination of a machine-readable storage device, a machine-readable storage substrate, a memory device, a combination of substances that affect a machine-readable propagation signal, or a combination of one or more of them. The term "computing system" covers all devices, devices and machines for processing data, including, for example, a programmable processor, a computer or multiple processors or computers. In addition to hardware, the device can also include code that creates an execution environment for the computer program in question, for example, code that constitutes a processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them. A propagation signal is an artificially generated signal (e.g., a machine-generated electrical signal, an optical signal, or an electromagnetic signal) that encodes information for transmission to a suitable receiver device.

[0146] The actual dedicated control hardware or software code used to implement these systems and / or methods is not limited to these implementations. Therefore, any software and any hardware can implement the systems and / or methods based on the description of this article without referring to specific software codes.

[0147] A computer program (also referred to as a program, software, software application, script, or code) is written in any suitable form of a programming language, including compiled or interpreted languages. It can be deployed in any suitable form, including a stand-alone program or a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program may be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions). A computer program may be executed on a single computer or on multiple computers located at one site or distributed across multiple sites and interconnected by a communications network.

[0148] One or more programmable processors execute one or more computer programs to perform functions by operating on input data and generating output, performing the processes and logic flows described herein. The processes and logic flows may also be performed by a dedicated logic circuit, and the device may also be implemented as a dedicated logic circuit, such as, but not limited to, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), an application specific standard product (ASSP), a system on a chip (SOC) type system, a complex programmable logic device (CPLD), etc.

[0149] Processors suitable for executing computer programs include, for example, general-purpose microprocessors and special-purpose microprocessors, and any one or more processors of digital computers of any suitable type. The processor will receive instructions and data from a read-only memory or a random access memory or both. The elements of a computer can include a processor for executing instructions and one or more memory devices for storing instructions and data. The computer will also include one or more large-capacity storage devices (such as magnetic disks, magneto-optical disks, optical disks, or solid-state disks) for storing data, or be operably coupled to receive data from the one or more large-capacity storage devices, transmit data to the one or more large-capacity storage devices, or both. However, the computer does not need to have such a device. In addition, another device, for example, a mobile phone, a personal digital assistant (PDA), a mobile audio player, a global positioning system (GPS) receiver, etc. can be embedded in the computer. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices (e.g., erasable programmable read-only memory (EPROM), electronically erasable programmable read-only memory (EEPROM), and flash memory devices), magnetic disks (e.g., internal hard disks or removable disks), magneto-optical disks (e.g., compact disk read-only memory (CD ROM) disks, digital versatile disk read-only memory (DVD-ROM) disks), and solid-state disks. Special purpose logic circuits may supplement or be incorporated in the processor and memory.

[0150] To provide interaction with a user, a computer may have a display device, such as a cathode ray tube (CRT) or liquid crystal display (LCD) monitor, for displaying information to the user, and a keyboard and pointing device, such as a mouse or trackball, through which the user can provide input to the computer. Other kinds of devices also provide interaction with the user. For example, feedback to the user may be any suitable form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and a computer may receive input from a user in any suitable form, including acoustic, voice, or tactile input.

[0151] A computing system that includes a back-end component (e.g., a data server), or includes a middleware component (e.g., an application server), or includes a front-end component (e.g., a client computer with a graphical user interface or a web browser through which a user can interact with the implementation), or any suitable combination of one or more such back-end, middleware, or front-end components can implement the implementations described herein. Any suitable form or medium of digital data communication (e.g., a communication network) can interconnect the components of the system. Examples of communication networks include local area networks (LANs) and wide area networks (WANs), such as intranets and the Internet.

[0152] A computing system may include clients and servers. The clients and servers are remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

[0153] Embodiments of the present invention may include or utilize a special or general purpose computer including computer hardware. Embodiments within the scope of the present invention may also include physical and other computer readable media for carrying or storing computer executable instructions and / or data structures. Such computer readable media can be any medium accessible to a general or special purpose computer system. The computer readable medium storing computer executable instructions is a physical storage medium. The computer readable medium carrying computer executable instructions is a transmission medium. Therefore, by way of example and not limitation, embodiments of the present invention can include at least two different types of computer readable media: a physical computer readable storage medium and a transmission computer readable medium.

[0154] Although the present embodiments described herein are with reference to specific example embodiments, it will be apparent that various modifications and changes may be made to these embodiments without departing from the broader spirit and scope of the various embodiments. For example, hardware circuits (e.g., complementary metal oxide semiconductor (CMOS) based logic circuits), firmware, software (e.g., embodied in a non-transitory machine-readable medium), or any combination of hardware, firmware, and software may enable and operate the various devices, units, and modules described herein. For example, transistors, logic gates, and circuits (e.g., application specific integrated circuits (ASICs) and / or digital signal processor (DSP) circuits) may embody various electrical structures and methods.

[0155] In addition, non-transitory machine-readable media and / or systems may embody various operations, processes, and methods disclosed herein.Accordingly, the specification and drawings are illustrative rather than restrictive.

[0156] Physical computer-readable storage media include RAM, ROM, EEPROM, CD-ROM or other optical disk storage (such as CD, DVD, etc.), magnetic disk storage or other magnetic storage devices, solid-state disks or any other media. They store the desired program code in the form of computer-executable instructions or data structures that can be accessed by general or special computers.

[0157] As used herein, the term "network" refers to one or more data links that can carry electronic data between computer systems and / or modules and / or other electronic devices. When a network or other communication connection (hardwired, wireless, or a combination of hardwired or wireless) transmits or provides information to a computer, the computer correctly regards the connection as a transmission medium. General or special-purpose computers access transmission media, wherein the transmission medium can include networks and / or data links that carry computer executable instructions or required program codes in the form of data structures. The scope of computer-readable media includes the above-mentioned combinations, which enable electronic data to be carried between computer systems and / or modules and / or other electronic devices.

[0158] Furthermore, upon reaching various computer system components, program code in the form of computer executable instructions or data structures can be automatically transferred from a transmission computer readable medium to a physical computer readable storage medium (or vice versa). For example, computer executable instructions or data structures received over a network or data link can be buffered in RAM within a network interface module (NIC) and then ultimately transferred to computer system RAM and / or a less volatile computer readable physical storage medium at the computer system. Thus, computer system components that also (or even primarily) utilize transmission media can include computer readable physical storage media.

[0159] For example, computer executable instructions include instructions and data that cause a general-purpose computer, a special-purpose computer, or a special-purpose processing device to perform a specific function or group of functions. Computer executable instructions can be, for example, binary, intermediate format instructions such as assembly language, or even source code. Although the subject matter described herein is in a language specific to structural features and / or method actions, the described features or described actions do not limit the subject matter defined in the claims. On the contrary, the features and actions described herein are example forms of implementing the claims.

[0160] Although this specification contains many details, these are not to be construed as limitations on the scope of the present disclosure or claims, but rather as descriptions of features that are specific to a particular implementation. A single implementation may implement certain features described in this specification in the context of a separate implementation. Conversely, multiple implementations may implement various features described in the context of a single implementation individually or in any suitable subcombination. In addition, although features described herein as functioning in certain combinations and even initially claimed as such, in some cases, one or more features from a claimed combination may be removed from the combination, and a claimed combination may be directed to a subcombination or a variation of a subcombination.

[0161] Similarly, although operations are depicted in the drawings herein in a particular order to achieve desired results, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desired results. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system components in the embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems may be integrated together in a single software product or packaged into multiple software products.

[0162] Even though particular combinations of features are recited in the claims and / or disclosed in the specification, these combinations are not intended to limit the disclosure of possible implementations. Other implementations fall within the scope of the claims. For example, the actions recited in the claims can be performed in a different order and still achieve the desired results. In fact, many of these features can be combined in ways not specifically recited in the claims and / or disclosed in the specification. Although each dependent claim may directly refer to only one claim, the disclosure of possible implementations includes the combination of each dependent claim with every other claim in the claim group.

[0163] In addition, a computer system including one or more processors and a computer-readable medium such as a computer memory can practice these methods. Specifically, one or more processors execute computer-executable instructions stored in the computer memory to perform various functions, such as the actions described in the embodiments.

[0164] Those skilled in the art will appreciate that the present invention can be implemented in a network computing environment with many types of computer system configurations, including personal computers, desktop computers, laptop computers, message processors, handheld devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, mobile phones, PDAs, pagers, routers, switches, etc. The present invention can also be implemented in a distributed system environment in which local and remote computer systems that are linked by a network (by a hardwired data link, a wireless data link, or by a combination of a hardwired and wireless data link) all perform tasks. In a distributed system environment, program modules can be located in both local and remote memory storage devices.

[0165] Unless otherwise indicated, the following terms and phrases shall be understood to have the following meanings.

[0166] As used herein, the term "cryptographic protocol" is also known as a security protocol or encryption protocol. It is an abstract or concrete protocol that performs security-related functions and applies cryptographic methods, usually as a sequence of cryptographic primitives. The protocol describes the use of algorithms. A sufficiently detailed protocol includes details about data structures and representations to enable multiple interoperable versions of the program.

[0167] Secure application-level data transport widely uses cryptographic protocols. Cryptographic protocols typically include at least some of the following aspects: key negotiation or establishment, entity authentication, symmetric encryption, and message authentication material construction, secure application-level data transport, non-repudiation methods, secret sharing methods, and secure multi-party computation.

[0168] Network switches use cryptographic protocols such as Secure Socket Layer (SSL) and Transport Layer Security (TLS), the successor to SSL, to secure data communications on wireless networks.

[0169] As used herein, the term "unauthorized access" means that someone uses another person's account or other methods to gain access to a website, program, server, service, or other system. For example, if someone keeps guessing passwords or usernames for accounts that do not belong to them until they gain access, this is considered unauthorized access.

[0170] As used herein, the term “IoT” stands for the Internet of Things, which describes a network of physical “things” or objects embedded with sensors, software, and other technologies for the purpose of connecting and exchanging data with other devices and systems over the Internet.

[0171] As used herein, "machine learning" refers to an algorithm that gives a computer the ability to learn without explicit programming, including algorithms that learn from data and predict data. Machine learning techniques include, but are not limited to, support vector machines, artificial neural networks (ANNs) (also referred to herein as "neural networks"), deep learning neural networks, logistic regression, discriminant analysis, random forests, linear regression, rule-based machine learning, naive Bayes, nearest neighbors, decision trees, decision tree learning, and hidden Markov, etc. For clarity, a part of the machine learning process can use algorithms such as linear regression or logistic regression. However, using linear regression or another algorithm as part of the machine learning process is different from using a spreadsheet program to perform statistical analysis such as regression. The machine learning process can continuously learn and adjust the classifier as new data becomes available, and does not rely on explicit or rule-based programming. ANN can be characterized by a feedback loop to dynamically adjust the system output when new data becomes available because it learns from the new data. In machine learning, back propagation and feedback loops are used to train artificial intelligence / machine learning (AI / ML) models to improve the accuracy and performance of the model over time.

[0172] Statistical modeling relies on discovering relationships between variables (like mathematical equations) to predict outcomes.

[0173] As used herein, the term "data mining" is the process used to transform raw data into useful information.

[0174] As used herein, the term "data acquisition" is the process of sampling signals that measure real-world physical conditions and converting the resulting samples into digital values ​​that are processed by a computer. Data acquisition systems typically convert analog waveforms into digital values ​​for processing. Components of a data acquisition system include sensors that convert physical parameters into electrical signals, signal conditioning circuits that convert sensor signals into a form that can be converted into digital values, and analog-to-digital converters that convert the conditioned sensor signals into digital values. Standalone data acquisition systems are often referred to as dataloggers.

[0175] As used herein, the term "dashboard" is a type of interface that visualizes specific key performance indicators (KPIs) for a specific goal or process. It is based on data visualization and infographics.

[0176] As used herein, a "database" is a collection of information organized so that it can be easily accessed, managed, and updated. A computer database typically contains an aggregation of data records or files.

[0177] As used herein, the term "data set" or "Dataset" is a collection of data. In the case of tabular data, a data set corresponds to one or more database tables, where each column of the table represents a specific variable and each row corresponds to a given record of the data set in question. A data set lists the value of each variable for each member of the data set, such as the height and weight of an object. Each value is called a datum. A data set can also consist of a collection of documents or files.

[0178] As used herein, a "sensor" is a device that measures physical input from its environment and converts it into data that can be interpreted by a person or machine. Most sensors are electronic and present electronic data, but some are simpler, such as a glass thermometer, and present visual data.

[0179] The term "infotainment system" or "in-vehicle infotainment system" (IVI) as used herein refers to a combination of vehicle systems for delivering entertainment and information. In one example, the information can be delivered to the driver and passengers / occupants of the vehicle through audio / video interfaces, control elements such as touch screen displays, button panels, voice commands, etc. Some of the major components of an in-vehicle infotainment system are an integrated head-unit, heads-up display (also known as head-up display), high-end digital signal processors (DSPs) and graphics processing units (GPUs) to support multiple displays, operating systems, controller area networks (CAN), low voltage differential signaling (LVDS) and other network protocol support (as required), connectivity modules, automotive sensor integration, digital instrument clusters, etc.

[0180] The term "environment" or "surroundings" as used herein refers to the surroundings and space that the vehicle is traveling in. It refers to the dynamic environment in which the vehicle is traveling, including other vehicles, obstacles, pedestrians, lane boundaries, traffic signs and signals, speed limits, potholes, snow, water, etc.

[0181] The term "autonomous mode" as used herein refers to an independent and unsupervised mode of operation.

[0182] As used herein, the term "vehicle" refers to an object used to transport people or goods. Cars, cars, trucks, buses, etc. are examples of vehicles.

[0183] As used herein, the term "autonomous vehicle," also known as self-driving vehicle, driverless vehicle, robotic vehicle, refers to a vehicle that incorporates vehicle automation, i.e., a ground vehicle that is able to sense its environment and move safely with little or no human input. Autonomous vehicles incorporate a variety of sensors to sense their surroundings, such as thermal imaging cameras, radio detection and ranging (RADAR), light detection and ranging (LIDAR), sound navigation and ranging (sonar), global positioning system (GPS), odometer, and inertial measurement unit. A control system designed for this purpose interprets the sensor information to identify the appropriate navigation path as well as obstacles and related landmarks.

[0184] The term "communication module" or "communication system" as used herein refers to a system that enables information exchange between two points. The process of transmitting and receiving information is called communication. The elements of communication include but are not limited to the transmitter of information, the channel or medium of communication, and the receiver of information.

[0185] As used herein, the term "autonomous communication" includes communication in different scenarios within a period of time with minimal supervision, and is not solely or completely based on pre-coded scenarios or pre-coded rules or pre-defined protocols. Generally speaking, autonomous communication occurs in an independent and unsupervised manner. In one embodiment, the communication module enables autonomous communication.

[0186] The term "connection" as used herein refers to a communication link. It refers to a communication channel that connects two or more devices for the purpose of data transfer. It can refer to a physical transmission medium, such as a wire, or to a logical connection through a multiplexed medium, such as a radio channel in telecommunications and a computer network. A channel is used to transmit information, such as a digital bit stream, from one or several transmitters to one or several receivers. A channel has a certain information transmission capacity, which is usually measured by its bandwidth in Hertz (Hz) or data rate in bits per second. For example, vehicle-to-vehicle (V2V) communication can wirelessly exchange information about the speed, location, and direction of surrounding vehicles.

[0187] The term "communication" as used herein refers to the transmission of information and / or data from one point to another. Communication can be with the help of electromagnetic waves. Communication is also the flow of information from one point (called the source) to another point (called the receiver). Communication includes one of the following: transmitting data, instructions, information, or a combination of data, instructions and information. Communication occurs between any two communication systems or communication units. The term communication in this article includes systems that combine other more specific types of communication, such as: V2I (vehicle to infrastructure), V2N (vehicle to network), V2V (vehicle to vehicle), V2P (vehicle to pedestrian), V2D (vehicle to device), V2G (vehicle to grid) and vehicle to everything (V2X) communication.

[0188] In addition, the communication device is configured on a computer having a communication function, and performs two-way communication with the vehicle-mounted emergency reporting device by means of a communication line through a radio station and a communication network such as a public telephone network or by means of satellite communication through a communication satellite. The communication device is suitable for communicating with the communication terminal through the communication network.

[0189] The term "vehicle-to-vehicle (V2V) communication" refers to technology that allows vehicles to broadcast and receive messages. Said messages can be omnidirectional, creating a 360-degree "awareness" of other vehicles in the vicinity. Vehicles can be equipped with appropriate software (or safety applications) that can use information from surrounding vehicles to determine potential collision threats as they progress.

[0190] As used herein, the term "Vehicle-to-Everything (V2X) communication" refers to the transmission of information from a vehicle to any entity that may affect the vehicle, and vice versa. There are two types of V2X communication technologies, depending on the underlying technology employed: cellular networks and other technologies that support direct device-to-device communication (such as Dedicated Short Range Communication (DSRC), Port Community System (PCS), wait).

[0191] The term "protocol" as used herein refers to the procedures required to initiate and maintain communications; a formal set of conventions governing the format and relative timing of message exchanges between two communicating terminals; a set of conventions governing the interaction of processes, devices and other components within a system; a set of signaling rules for passing information or commands between boards connected to a bus; a set of signaling rules for passing information between agents; a set of semantic and syntactic rules that determine the behavior of interacting entities; a set of rules and formats (semantics and syntax) that determine the communication behavior of simulated applications; a set of conventions or rules governing the interaction of processes or applications between communicating terminals; a formal set of conventions governing the format and relative timing of message exchanges between communicating terminals; a set of semantic and syntactic rules that determine the behavior of functional units in achieving meaningful communication; a set of semantic and syntactic rules for exchanging information.

[0192] As used herein, the term "communication protocol" refers to standardized communications between any two systems. An example communication protocol is the DSRC protocol. The DSRC protocol uses a specific frequency band (e.g., 5.9 GHz (gigahertz)) and a specific message format (e.g., basic safety messages, signal phase and timing, and roadside warnings) to enable communications between vehicles and infrastructure components (e.g., traffic signals and roadside sensors). DSRC is a standardized protocol whose specifications are maintained by various organizations, including the Institute of Electrical and Electronics Engineers (IEEE) and the Society of Automotive Engineers (SAE).

[0193] As used herein, the term "two-way communication" refers to the exchange of data between two components. In one example, the first component may be a vehicle and the second component may be an infrastructure that is usable through a system of hardware, software, and firmware.

[0194] The term "alert" or "alert signal" refers to a communication that draws attention. Alerts may include visual, tactile, audible alerts, and combinations of these alerts to alert the driver or passengers. These alerts allow the recipient, such as the driver or passenger, to react and respond quickly.

[0195] As used herein, the term "communicates with" refers to any coupling, connection, or interaction using signals to exchange information, messages, instructions, commands, and / or data using any system, hardware, software, protocol, or format, whether the exchange occurs wirelessly or through a wired connection.

[0196] As used herein, the term "network" refers to one or more data links capable of transmitting electronic data between computer systems and / or modules and / or other electronic devices. When a network or other communication connection (hardwired, wireless, or a combination of hardwired or wireless) transmits or provides information to a computer, the computer correctly regards the connection as a transmission medium. A general or special computer accesses a transmission medium, which can include a network and / or data link that carries a required program code in the form of a computer executable instruction or data structure. The scope of computer-readable media includes the above-mentioned combination, which enables electronic data to be transmitted between a computer system and / or module and / or other electronic devices. A network may include one or more networks or communication systems, such as the Internet, a telephone system, a satellite network, a cable television network, and various other dedicated and public networks. In addition, a connection may include a wired connection (such as an electric wire, a cable, an optical fiber line, etc.), a wireless connection, or a combination thereof. In addition, although not shown, other computers, systems, devices, and networks may also be connected to a network. A network refers to any group of devices or subsystems connected by a link that joins (directly or indirectly) a group of terminal nodes that share resources located on or provided by a network node. Computers communicate with each other using common communication protocols over digital interconnections. For example, a subsystem can include a cloud. The cloud refers to servers accessed over the internet, and the software and databases running on those servers.

[0197] The term "electronic control unit" (ECU), also known as an "electronic control module" (ECM), is generally a module that controls one or more subsystems. Here, an ECU can be installed in an automobile or other motor vehicle. It can refer to many ECUs, and can include, but is not limited to, an engine control module (ECM), a powertrain control module (PCM), a transmission control module (TCM), a brake control module (BCM) or an electronic brake control module (EBCM), a central control module (CCM), a central timing module (CTM), a general electronic module (GEM), a body control module (BCM), and a suspension control module (SCM). ECUs are sometimes collectively referred to as the vehicle's computer or the vehicle's central computer, and may include a separate computer. In one example, an electronic control unit can be an embedded system in the automotive electronics. In another example, the electronic control unit is wirelessly coupled to the automotive electronics.

[0198] The terms "non-transitory computer-readable medium" and "computer-readable medium" include a single medium or multiple media, such as a centralized or distributed database, and / or associated caches and servers that store one or more sets of instructions. Furthermore, the terms "non-transitory computer-readable medium" and "computer-readable medium" include any tangible medium that can store, encode, or carry a set of instructions for execution by a processor, which, when executed, causes a system to perform any one or more of the methods or operations disclosed herein. As used herein, the term "computer-readable medium" is expressly defined to include any type of computer-readable storage device and / or storage disk, and to exclude propagating signals.

[0199] As used herein, the term “vehicle data bus” refers to an interface to a vehicle data bus (e.g., Controller Area Network (CAN), Local Interconnect Network (LIN), Ethernet / IP, FlexRay, and Media Oriented Systems Transport (MOST)) that enables communication between vehicle onboard equipment (OBE) and other vehicle systems to support connected vehicle applications.

[0200] The term "handshaking" refers to the exchange of predetermined signals between agents connected by a communications channel to ensure that each agent is connected to the other agent (and not to an imposter). This can also include the use of passwords and codes by operators. Handshaking signals are transmitted back and forth across a communications network to establish an active connection between two stations. Hardware handshaking uses dedicated wires such as the Request to Send (RTS) and Clear to Send (CTS) lines in Recommended Standard 232 (RS-232) serial transmissions. Software handshaking sends codes such as "synchronize" (SYN) and "acknowledge" (ACK) in Transmission Control Protocol / Internet Protocol (TCP / IP) transmissions.

[0201] The term "computer vision module" or "computer vision system" allows the vehicle to "see" and interpret the world around it. The system uses a combination of cameras, sensors and other technologies such as radio detection and ranging (RADAR), light detection and ranging (LiDAR), sound navigation and ranging (sonar), global positioning system (GPS) and machine learning algorithms to collect visual data about the vehicle's surroundings in real time and analyze this data. The computer vision system is designed to perform a range of tasks, including object detection, lane detection and pedestrian recognition. It uses deep learning algorithms and other machine learning techniques to analyze visual data and make decisions about how to control the vehicle. For example, a computer vision system can use object detection algorithms to identify other vehicles, pedestrians and obstacles in the vehicle's path. It is then able to use this information to calculate the vehicle's speed and direction, adjust its trajectory to avoid collisions, and brake or accelerate as needed. It allows the vehicle to navigate safely and efficiently in a variety of driving conditions.

[0202] As used herein, the term "driver" refers to an occupant who, even if the occupant is not actually driving the vehicle, is located in the vehicle so as to take control and act as the driver of the vehicle when the vehicle control system transfers control to the occupant or driver, or when the vehicle control system is not operating in an autonomous or semi-autonomous driving mode.

[0203] As used herein, the term "host vehicle" refers to a vehicle that observes its environment in order to make decisions based on the observations.

[0204] As used herein, the term "target vehicle" refers to a vehicle of interest to the host vehicle. The target vehicle may or may not be an autonomous vehicle. It may or may not be enabled for vehicle-to-vehicle (V2V) communication.

[0205] As used herein, the term "nearby vehicle" or "neighboring vehicle" or "surrounding vehicle" refers to a vehicle anywhere near the host vehicle within the communication range of the host vehicle. It may or may not be an autonomous vehicle. It may or may not be enabled for V2V communication. In some embodiments, a neighboring vehicle may more specifically refer to a vehicle that is immediately in the next lane or behind the host vehicle.

[0206] As used herein, the term "road surface condition" refers to the physical state of the road, including its smoothness, texture, and friction. It can be affected by various factors, such as weather, traffic volume, and maintenance practices. In one embodiment, it can be classified as dry, wet, snow-covered, or icy.

[0207] The term "safety distance", also known as following distance, refers to the distance between a first vehicle and a second vehicle, where the second vehicle is located in front of the first vehicle when driving. It is the space maintained between the first vehicle and the second vehicle located in front of the first vehicle to avoid a collision in a sudden stop or emergency situation. Maintaining a safe following distance can help prevent rear-end collisions and provide enough time to react to any sudden changes in traffic conditions.

[0208] The term "uphill road" or "uphill road section" refers to a road with an ascending slope, which means that it slopes upward as it extends further.

[0209] As used herein, the term "visible roof area" refers to the roof area of ​​a target vehicle that is visible from the perspective of the host vehicle and detected by the host vehicle.

[0210] The term "precipitation" includes rain, snow, sleet, hail, ice, and freezing rain.

[0211] The term "application server" refers to a server that hosts applications or software that delivers business applications through communication protocols. The application server framework is a service layer model. It includes software components that are available to software developers through application programming interfaces. It is the system software that resides between the operating system (OS) on one side, external resources such as database management systems (DBMS), communications and Internet services on another side, and user applications on the third side.

[0212] As used herein, the term “cybersecurity” refers to the application of technologies, processes, and controls to protect systems, networks, programs, devices, and data from cyber-attacks.

[0213] As used herein, the term "cybersecurity module" refers to a module that includes the application of technologies, processes and controls to protect systems, networks, programs, devices and data from cyber attacks and threats. It is designed to reduce the risk of cyber attacks and prevent unauthorized exploitation of systems, networks and technologies. It includes but is not limited to critical infrastructure security, application security, network security, cloud security, Internet of Things (IoT) security.

[0214] The term "encrypt" as used herein refers to the use of one or more mathematical techniques and a password or "key" for decrypting information to protect digital data, and refers to the conversion of information or data into code, especially to prevent unauthorized access. It can also refer to hiding information or data by converting it into code. It can also be called a cipher, code, encipher, encoding. A simple example is to represent the alphabet with numbers - for example, "A" is "01", "B" is "02", and so on. For example, a message like "HELLO" will be encrypted as "0805121215", and this value will be transmitted to the recipient over the network.

[0215] The term "decryption" as used herein refers to the process of converting an encrypted message back to its original format. It is generally the reverse process of encryption. It decodes the encrypted information so that only authorized users can decrypt the data, as a secret key or password is required for decryption. The term can be used to describe methods of decrypting data manually or using an appropriate code or key.

[0216] The term "cybersecurity threat" as used herein refers to any possible malicious attack that attempts to illegally access data, disrupt digital operations, or destroy information. Malicious behavior includes, but is not limited to, destroying data, stealing data, or disrupting digital life. Cyber ​​threats include, but are not limited to, malware, spyware, phishing attacks, ransomware, zero-day exploits, Trojans, advanced persistent threats, wiper attacks, data manipulation, data destruction, rogue software, malicious advertising, unpatched software, computer viruses, man-in-the-middle attacks, data breaches, denial of service (DoS) attacks, and other attack vectors.

[0217] The term "hash value" as used herein can be thought of as a fingerprint of a file. The contents of a file are processed through an encryption algorithm and a unique value (i.e., a hash value) is generated to identify the contents of the file. If the content is modified in any way, the hash value will also change significantly. Example algorithms for generating hash values: Message Digest-5 (MD5) algorithm and Secure Hash Algorithm-1 (SHA1).

[0218] As used herein, the term "integrity verification" refers to verifying the accuracy and consistency of system-related files, data, etc. It can be performed using verification tools that can detect whether any critical system files have been changed, thereby enabling system administrators to look for unauthorized changes to the system. For example, data integrity corresponds to the quality of the data in a database and the level at which a user verifies the quality, integrity, and reliability of the data. Data integrity verification verifies that the data in the database is accurate and operates as expected in a given application.

[0219] As used herein, the term "alert" refers to a trigger when a system or a component in a system fails or does not perform as expected. A system may enter an alert state when a specific event occurs. An alert indicator signal is a visual signal indicating an alert state. For example, when a network security threat is detected, an alert may be issued to a system administrator via an audible alert, a message, an illuminated LED, a pop-up window, etc. The alert indicator signal may be reported downstream of the detection device to prevent adverse conditions or cascading effects.

[0220] As used herein, the term "in communication with" refers to any coupling, connection, or interaction that uses electrical signals to exchange information or data using any system, hardware, software, protocol, or format, whether the exchange occurs wirelessly or through a wired connection.

[0221] As used herein, the term "cryptographic protocol" is also referred to as a security protocol or encryption protocol. It is an abstract or concrete protocol that performs security-related functions and applies encryption methods that are usually sequences of cryptographic primitives. The protocol describes how the algorithm should be used. A sufficiently detailed protocol includes details about data structures and representations, at which point it can be used to implement multiple interoperable versions of a program. Cryptographic protocols are widely used for secure application-level data transmission. Cryptographic protocols typically include at least some of the following aspects: key negotiation or establishment, entity authentication, symmetric encryption and message authentication material construction, secure application-level data transmission, non-repudiation methods, secret sharing methods, and secure multi-party computation. Hash algorithms can be used to verify the integrity of data. Secure Sockets Layer (SSL) and Transport Layer Security (TLS) (the successor to SSL) are cryptographic protocols that network switches can use to protect data communications on the network.

[0222] As used herein, the term "network" may include the Internet, a local area network, a wide area network, or a combination thereof. A network may include one or more networks or communication systems, such as the Internet, a telephone system, a satellite network, a cable television network, and various other private and public networks. In addition, the connection may include a wired connection (such as a wire, a cable, an optical fiber line, etc.), a wireless connection, or a combination thereof. In addition, although not shown, other computers, systems, devices, and networks may also be connected to the network. A network refers to any group of devices or subsystems connected by a link that joins (directly or indirectly) a group of terminal nodes that share resources located on or provided by a network node. Computers communicate with each other using a public communication protocol through digital interconnection. For example, a subsystem may include a cloud. A cloud refers to a server accessed via the Internet, and the software and databases running on these servers.

[0223] The embodiments described herein can be directed to one or more of the systems, methods, devices and / or computer program products of any possible level of integrated technical detail. The computer program product can include a computer-readable storage medium (or medium) having computer-readable program instructions thereon, for causing a processor to perform aspects of one or more embodiments described herein. The computer-readable storage medium can be a tangible device capable of retaining and storing instructions for use by an instruction execution device. For example, a computer-readable storage medium can be, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a superconducting storage device and / or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media can also include the following: a portable computer floppy disk (diskette), a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable optical disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk (floppydisk), a mechanical encoding device, such as a punch card or a protruding structure in a groove on which instructions are recorded and / or any suitable combination of the foregoing. The computer-readable storage media used in this document is not to be construed as transient signals themselves, such as radio waves and / or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides and / or other transmission media (e.g., light pulses through optical cables), and / or electrical signals transmitted through wires.

[0224] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a corresponding computing / processing device and / or to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical transmission optical fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives computer-readable program instructions from the network, and forwards computer-readable program instructions to be stored in a computer-readable storage medium in the corresponding computing / processing device. The computer-readable program instructions for performing the operation of one or more embodiments described herein can be assembly instructions, instruction set architecture (instruction-set-architecture, referred to as ISA) instructions, machine instructions, machine dependent instructions (machine dependent instructions), microcode, firmware instructions, state setting data (state-setting data), configuration data of an integrated circuit, and / or source code and / or object code written in any combination of one or more programming languages ​​(including object-oriented programming languages, such as Smalltalk, C++, etc., and / or process programming languages, such as "C" programming languages ​​and / or similar programming languages). The computer-readable program instructions can be executed entirely on a computer, partially on a computer, as a stand-alone software package, partially on a computer and / or partially on a remote computer or entirely on a remote computer and / or server. In the latter scenario, the remote computer can be connected to the computer via any type of network, including a local area network (LAN) and / or a wide area network (WAN), and / or can be connected to an external computer (e.g., via the Internet using an Internet service provider). In one or more embodiments, electronic circuits including, for example, programmable logic circuits, field programmable gate arrays (FPGAs) and / or programmable logic arrays (PLAs) can run computer-readable program instructions to personalize the electronic circuits by utilizing the state information of the computer-readable program instructions, so as to perform aspects of one or more embodiments described herein.

[0225] Aspects of one or more embodiments described herein are described with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to one or more embodiments described herein. Each block of the flowchart and / or block diagram and the combination of the blocks in the flowchart and / or block diagram can be realized by computer-readable program instructions. These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer and / or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device can create a device (or module, means) for realizing the function / action specified in the flowchart and / or block diagram block. These computer-readable program instructions can also be stored in a computer-readable storage medium, which can guide a computer, a programmable data processing device and / or other devices to work in a specific manner, so that a computer-readable storage medium having instructions stored therein can include a product, which includes instructions that can realize various aspects of the function / action specified in the flowchart and / or block diagram block. Computer-readable program instructions can also be loaded onto a computer, other programmable data processing devices and / or other devices to cause a series of operational actions to be performed on the computer, other programmable devices and / or other devices, thereby producing a computer-implemented process, so that the instructions executed on the computer, other programmable devices and / or other devices implement the functions / actions specified in the flowchart and / or block diagram blocks.

[0226] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and / or operations of the system, computer-implemented method and / or computer program product according to one or more embodiments described herein. In this regard, each block in the flowchart or block diagram can represent a module, segment and / or part of an instruction, which includes one or more executable instructions for implementing a specified logical function. In one or more alternative implementations, the functions recorded in the block can appear in a sequence not recorded in the figure. For example, depending on the functions involved, two blocks shown in succession can be executed substantially simultaneously, and / or these blocks can sometimes be executed in reverse order. It will also be noted that each block in the block diagram and / or flowchart, and / or the combination of blocks in the block diagram and / or flowchart, can be implemented by a hardware-based dedicated system that can perform specified functions and / or actions and / or perform one or more combinations of dedicated hardware and / or computer instructions.

[0227] Although the subject matter described herein is in the general context of computer executable instructions of a computer program product running on one and / or multiple computers, those skilled in the art will recognize that one or more embodiments herein can also be implemented in conjunction with one or more other program modules. Program modules include routines, programs, components, data structures, etc. that perform specific tasks and / or implement specific abstract data types. In addition, other computer system configurations, including single-processor and / or multi-processor computer systems, microcomputing devices, mainframe computers, and computers, handheld computing devices (e.g., PDAs, phones), microprocessor-based or programmable consumer and / or industrial electronic devices, and / or similar devices, can practice the computer-implemented methods described herein. In a distributed computing environment, remote processing devices linked by a communication network can also practice the aspects shown. However, a stand-alone computer can practice one or more aspects of one or more embodiments described herein, if not all aspects. In a distributed computing environment, program modules can be located in local and remote memory storage devices.

[0228] As used in this application, the term "component", "system", "platform", "interface" and / or the like can refer to and / or can include an entity associated with a computer or an entity associated with an operating machine having one or more specific functions. The entity described herein can be hardware, a combination of hardware and software, software or software in execution. For example, a component can be, but is not limited to, a process, a processor, an object, an executable file, an execution thread, a program and / or a computer running on a processor. For example, both the application program and the server running on the server can be components. One or more components can reside in the process and / or thread of execution, and the component can be localized on a computer and / or distributed between two or more computers. In another example, each component can be operated by various computer-readable media having various data structures stored thereon. These components can communicate via local and / or remote processes, for example, according to a signal with one or more data packets (for example, data from a component interacting with another component of a local system, a distributed system via a signal and / or interacting with other systems across a network (such as the Internet)). As another example, a component can be a device having a specific function provided by a mechanical component operated by an electrical or electronic circuit, which is operated by a software and / or firmware application executed by a processor. In this case, the processor can be located inside and / or outside the device and can execute at least a portion of the software and / or firmware application. As yet another example, a component can be a device that provides a specific function without a mechanical component by an electronic component, wherein the electronic component can include a processor and / or other device to execute at least part of the software and / or firmware that gives the electronic component its function. In one aspect, the component can simulate the electronic component via a virtual machine, for example, within a cloud computing system.

[0229] As used in this subject specification, the term "processor" can refer to any computational processing unit and / or device, including but not limited to: a single-core processor; a single processor with software multi-threaded execution capability; a multi-core processor; a multi-core processor with software multi-threaded execution capability; a multi-core processor using hardware multi-threading technology; a parallel platform; and / or a parallel platform with distributed shared memory. In addition, a processor can refer to an integrated circuit, an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a field programmable gate array (FPGA), a programmable logic controller (PLC), a complex programmable logic device (CPLD), discrete gate or transistor logic, discrete hardware components, and / or any combination thereof designed to perform the functions described herein. In addition, a processor can utilize nanoscale architectures, such as but not limited to molecular-based transistors, switches, and / or gates, in order to optimize space usage and / or enhance the performance of associated devices. A combination of computational processing units can implement a processor.

[0230] As used herein, terms such as "storage," "storage," "data storage," "data storage," "database," and any other information storage component related to the operation and functionality of a component refer to a "memory component," an entity contained in a "memory," or a component that includes a memory. The memory and / or memory components described herein can be volatile memory or nonvolatile memory, or can include volatile memory and nonvolatile memory. By way of illustration and not limitation, nonvolatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable ROM (EEPROM), flash memory, and / or nonvolatile random access memory (RAM) (e.g., ferroelectric RAM (FeRAM)). Volatile memory can include RAM, which can, for example, be used as an external cache memory. By way of illustration and not limitation, RAM can have a variety of forms, such as synchronous RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), direct Rambus RAM (DRRAM), direct Rambus dynamic RAM (DRDRAM), and / or Rambus dynamic RAM (RDRAM). In addition, the memory components of the systems and / or computer-implemented methods described herein include, but are not limited to, including these and / or any other suitable types of memory.

[0231] The embodiments described herein include only examples of systems and computer-implemented methods. Of course, it is not possible to describe every conceivable combination of components and / or computer-implemented methods for purposes of describing one or more embodiments, but one of ordinary skill in the art will recognize that many further combinations and / or permutations of one or more embodiments are possible. In addition, where the terms "including," "having," "containing," and the like are used in the detailed description, claims, appendices, and / or drawings, these terms are intended to be inclusive in a manner similar to the term "comprising," as interpreted when "including" is used as a transitional word in a claim.

[0232] The description of one or more embodiments is for illustrative purposes, but is not exhaustive or limited to the embodiments described herein. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terms used herein best explain the principles of the embodiments, practical applications, and / or technical improvements over technologies found on the market, and / or enable others of ordinary skill in the art to understand the embodiments described herein.

[0233] Driving on icy uphill roads can be challenging and dangerous because it can cause the vehicle to lose traction and slide uncontrollably. Some of the problems that may occur when driving on icy uphill roads are: (i) Icy roads reduce the traction between the tires and the road surface, making it difficult to get enough traction to climb the hill. (ii) If the vehicle loses traction, the vehicle slides. It may slide backwards or sideways, increasing the risk of an accident. (iii) When driving uphill on an icy road surface, the engine may have difficulty maintaining power and stall, making it difficult to continue to climb. (iv) Icy roads can significantly increase the distance required to stop the vehicle. If the driver needs to brake suddenly when driving uphill, they may not be able to stop in time to avoid an accident. (v) Ice, snow, and fog can significantly reduce visibility, making it difficult to see the road and other vehicles clearly. To minimize these problems, autonomous vehicles must detect them on icy roads and avoid possible collisions by taking collision avoidance actions. It may include adjusting driving behavior, being vigilant (valuing) proper winter tires, and avoiding driving on steep icy uphill roads whenever possible or maintaining an appropriate safe distance when driving on steep icy uphill roads.

[0234] Figure 1 A scenario is shown when a vehicle is traveling on an uphill road in icy conditions according to one embodiment. To avoid accidents and icy roads, as a first step, in the vehicle route, it is determined how many icy roads and how many uphill icy roads will be encountered. Once the determination is made, and the vehicle itself is already on an icy road, it is automatically detected that the vehicle 102 on top may slide backwards and provide space for the vehicle 104, obtaining a warning mode in advance in the vehicle 104 before the vehicle 102 begins to descend. In one embodiment, the system avoids accidents on icy days, and more specifically in areas where there are uphill roads.

[0235] In one embodiment, the system / method finds a potential situation when two vehicles 102 and 104 are on a sloping road and there is a risk of an accident due to the type of weather conditions such as ice, sleet, snow, etc. The vehicle 102 in front of the vehicle 104 may lose some traction in the wheels and therefore move backwards and may hit the vehicle 104.

[0236] Whenever a vehicle is traveling on icy roads, such as in countries in Europe and the United States where there may be a lot of snow or ice, the icy roads will cause the vehicle to begin to slide. Figure 1 As shown, when both vehicles 102 and 104 are traveling upward, there is a certain amount of safety distance that needs to be maintained on an uphill road.

[0237] In one embodiment, if the vehicle 104 is traveling on an icy road, the safety distance will be automatically modified by the vehicle control system to a value higher than normal roads / normal days. If the vehicle is traveling from point A to point B on an uphill road in icy conditions, the safety distance is further modified to a higher value. In one embodiment, in addition to automatically modifying the safety distance, an additional warning / alert message is issued on the infotainment system on the vehicle 104 to attract the attention of the vehicle driver or occupants.

[0238] Even after the safe distance, the vehicle 104 observes that the vehicle 102 in front of the vehicle 104 is losing traction, the vehicle 104 will provide a warning and can further adjust the safe distance. For example, if the safe distance is 20 feet on a normal day, it can be set to 30 feet on an icy road, and it can be further set to 40 feet on an icy uphill road. In one embodiment, if the safe distance is not maintained, the vehicle issues an audio warning. In one embodiment, when the vehicle 102 in front of the vehicle 104 begins to slide downward, a warning is provided in the vehicle 104. In one embodiment, the icy road, the icy uphill road, the vehicle 102 in front of the vehicle 104 are detected and the vehicle 104 makes a decision to avoid the accident, and the vehicle 104 can be an autonomous vehicle.

[0239] Figure 2A is a diagram of an exemplary autonomous vehicle with various sensors, actuators, and systems according to one embodiment. The autonomous system includes various sensors, such as ultrasonic sensors, light detection and ranging sensors, radar sensors, etc., actuators, such as brake actuators, steering actuators, etc., and various subsystems, such as propulsion systems, steering systems, brake sensor systems, communication systems, etc. Figure 1Depicted as an example system; it is neither limited to the depicted system nor an exhaustive enumeration of the features of the sensors, actuators and systems / subsystems and / or autonomous vehicles. Furthermore, the illustrated vehicle should not be interpreted as limiting the arrangement of any of the depicted sensors, actuators and systems / subsystems. These sensors, actuators and systems / subsystems can be arranged to suit the purpose performed by the autonomous vehicle. Autonomous vehicles, also known as self-driving cars or driverless cars, are vehicles that can navigate and operate without human intervention. Sensors, such as cameras, light detection and ranging systems, radars, and ultrasonic sensors, enable autonomous vehicles to detect and identify objects, obstacles, and pedestrians on the road. Autonomous vehicles use advanced control systems to make real-time decisions based on sensor data and pre-programmed rules or intelligence-based decision systems. These systems control, for example, the acceleration, braking, steering, and communication of the vehicle. Navigation systems such as GPS, maps, and other location-based technologies help autonomous vehicles navigate and plan the best route to their destination. The communication systems of autonomous vehicles help them communicate with other vehicles and infrastructure (such as traffic lights and road signs) to exchange information and optimize traffic flow. Autonomous vehicles have a variety of safety features, including collision avoidance systems, emergency braking, and backup systems in case of system failure. Autonomous vehicles analyze data, recognize patterns, and improve performance over time with the help of artificial intelligence and machine learning algorithms.

[0240] In one embodiment, the radar and light detection and ranging system detects that vehicle 102 begins to back up and approaches vehicle 104. While vehicle 104 may be stationary or moving forward, vehicle 102 in front of vehicle 104 is closing the distance.

[0241] In one embodiment, vehicle 102 begins sending signals. These signals may be, "vehicle is skidding because the road is icy"; "vehicle wheels are not moving"; "vehicle has no forward momentum"; "vehicle is gaining reverse momentum". These signals alert all other vehicles around it of the current condition or state of vehicle 102. Surrounding or nearby vehicles may then maintain a greater safety distance between their vehicles and vehicle 102 than the normal safety distance.

[0242] In one embodiment, once the host vehicle determines that there is an icy road in the path, it provides an early warning that the vehicle is not suitable for driving on the icy road because the tire tread does not meet the requirements of the vehicle's surrounding environment (i.e., icy conditions). In one embodiment, the host vehicle adjusts the safety distance to avoid an accident / collision.

[0243] In one embodiment, the autonomous vehicle avoids accidents based on other vehicles and other circumstances.

[0244] In one embodiment, the vehicle will avoid icy routes and generate an alternative route to the destination. However, once the vehicle is on the road, the safety distance for icy days (the safety distance is the distance between the vehicle and another vehicle before the warning arrives) is increased from the safety distance for normal road surface conditions. If the vehicle continues to move forward on an uphill road under icy conditions, the safety distance is further increased. In one embodiment, the increase can be 2 to 5 times the safety distance maintained during dry road surface conditions. If the distance between the vehicle in front of the vehicle and the vehicle begins to close because the vehicle in front of the vehicle is losing traction and sliding downward, the vehicle will provide an early warning and warn the driver / system of the vehicle not to drive forward. In one embodiment, the control system of the vehicle can suggest corrective actions (collision avoidance actions) to the driver of the vehicle or take autonomous actions to move backward (reverse) or to the side so that the vehicle sliding downward does not affect the vehicle.

[0245] In one embodiment, the host vehicle will track the number of available lanes, nearby traffic, vehicles behind it, and vehicles beside it before deciding / suggesting corrective action.

[0246] If the host vehicle's sensors sense an icy uphill road and detect that the target vehicle is sliding or predicts that the target vehicle is about to slide, the host vehicle system generates an alert. At this time, the host vehicle must sense its surroundings, the vehicles next to it, the vehicles behind it, the number of lanes, etc. Based on the sensed surroundings, the host vehicle will decide whether to move to the side lane, reverse, stop where it is, or continue forward. In one embodiment, the host vehicle will send a communication signal / broadcast message to the vehicles around it so that the vehicles around the host vehicle can start taking corrective actions.

[0247] In one embodiment, the host vehicle will take corrective action on its own to avoid an accident. In another embodiment, the host vehicle will provide information to other vehicles so that the host vehicle and other vehicles can avoid a collision. In one embodiment, if the host vehicle itself is losing traction or sliding backwards, it will provide a warning signal and broadcast a message to surrounding vehicles. In one embodiment, the host vehicle determines how many vehicles are nearby and then begins sending signals by establishing a connection with other vehicles.

[0248] In one embodiment, when the host vehicle gives a signal to other vehicles that it has to move to the right or it is sliding down, then the vehicles around the host vehicle will move in a coordinated manner by giving way to the host vehicle and avoiding collision with the host vehicle. In one embodiment, the other vehicles accept the host vehicle instruction and they all move in a coordinated manner to avoid collision. In one embodiment, it will be a coordinated collision avoidance system.

[0249] In one embodiment, the vehicle sliding down has no control, it can go in any direction, down, sideways, etc. The vehicle may not be able to control anything, but there is some chance that other vehicles that see the vehicle can start taking action to avoid any collision. The vehicle sliding down must establish constant communication with other vehicles.

[0250] In one embodiment, when there is a situation of a predicted or impending collision, communication signals take precedence over other things in such scenarios of the vehicle. For example, on a very steep but very short uphill slope, the host vehicle is following another vehicle and the other vehicle begins to slide, then the sensitivity of the host vehicle's sensors must be increased. Signaling that occurs when there is an impending or predicted collision has priority, which means that all resources ensure that processing is completed in the highest manner in order to obtain information as quickly as possible with minimal delay. All sensors are raised, so the vehicle must decide whether it can move right, left, or backward based on the vehicles around it. In one example, if there is a vehicle behind the host vehicle and there is a vehicle next to the host vehicle, the host vehicle can only potentially send more signals to other vehicles to take corrective action, such as possibly forcing them to move into a vacant space to create space for the host vehicle.

[0251] In one embodiment, the host vehicle can prepare for collisions from other vehicles, rather than the host vehicle being responsible for collisions with other vehicles. In one embodiment, when the host vehicle or other vehicles around it begin to slide and there is a possibility of collision, the host vehicle decides various corrective actions. Such corrective actions include steering angles, braking profiles, lane changes, speed profiles, and automatically and autonomously performing maneuvers. In one embodiment, it can help the driver take corrective actions.

[0252] Figure 2B A block diagram of electronic components of a vehicle according to one embodiment is shown. In the example shown, the electronic components include an onboard computing platform 202 , a human machine interface (HMI) unit 204 , a communication module 220 , a sensor 206 , an electronic control unit (ECU) 208 , and a vehicle data bus 210 . Figure 2B It shows that Figure 2A Example architectures for some of the electronic components shown in FIG.

[0253] The onboard computing platform 202 includes a processor 212 (also referred to as a microcontroller unit or controller) and a memory 214. In the example shown, the processor 212 of the onboard computing platform 202 is configured to include a controller 212-1. In other examples, the controller 212-1 is incorporated into another ECU having its own processor and memory. The processor 212 can be any suitable processing device or set of processing devices, such as but not limited to a microprocessor, a microcontroller-based platform, an integrated circuit, one or more field programmable gate arrays (FPGAs) and / or one or more application-specific integrated circuits (ASICs). The memory 214 can be a volatile memory (e.g., a RAM including non-volatile RAM, magnetic RAM, ferroelectric RAM, etc.), a non-volatile memory (e.g., a disk memory, a flash memory, an EPROM, an EEPROM, a non-volatile solid-state memory based on a memristor, etc.), an unchangeable memory (e.g., an EPROM), a read-only memory, and / or a high-capacity storage device (e.g., a hard disk drive, a solid-state drive, etc.). In some examples, the memory 214 includes multiple types of memory, particularly volatile memory and non-volatile memory. The memory 214 is a computer-readable medium on which one or more sets of instructions can be embedded, such as software for operating the methods of the present disclosure. The instructions may embody one or more of the methods or logics described herein. For example, during the execution of the instructions, the instructions reside completely or at least partially in any one or more of the memory 214, the computer-readable medium, and / or the processor 212.

[0254] The HMI unit 204 provides an interface between the vehicle and the user. The HMI unit 204 includes digital and / or analog interfaces (e.g., input devices and output devices) to receive input from the user and display information for the user. Input devices include, for example, control knobs, dashboards, digital cameras for image capture and / or visual command recognition, touch screens, audio input devices (e.g., cockpit microphones), buttons or touch pads. Output devices may include dashboard outputs (e.g., dials, lighting devices), tactile devices, actuators, displays 216 (e.g., head-up displays, center console displays such as liquid crystal displays (LCDs), organic light-emitting diode (OLED) displays, flat panel displays, solid-state displays, etc.) and / or speakers 218. For example, the display 216, speaker 218 and / or other output devices of the HMI unit 204 are configured to issue warnings to the operator of the vehicle (e.g., the driver), such as warnings requesting manual takeover. In addition, the HMI unit 204 of the illustrated example includes hardware (e.g., processors or controllers, memory, storage devices, etc.) and software (e.g., operating systems, etc.) of the infotainment system presented via the display 216.

[0255] The sensors 206 are arranged in and / or around the vehicle to monitor the properties of the vehicle and / or the environment in which the vehicle is located. One or more of the sensors 206 can be installed to measure the properties of the surrounding outside the vehicle. In addition or alternatively, one or more of the sensors 206 can be installed inside the cabin of the vehicle or in the body of the vehicle (e.g., the engine compartment, the rim well, etc.) to measure the properties of the vehicle and / or the interior of the vehicle. For example, the sensors 206 include accelerometers, odometers, tachometers, pitch and yaw sensors, wheel speed sensors, microphones, tire pressure sensors, biometric sensors, ultrasonic sensors, infrared sensors, light detection and ranging systems (Lidar, also known as laser radar), radio detection and ranging systems (radar, also known as radar), global positioning systems (GPS), cameras and / or any other suitable type of sensors. In the example shown, the sensors 206 include distance detection sensors configured to monitor (one or more) objects located in the surrounding area of ​​the vehicle.

[0256] ECU 208 monitors and controls the subsystems of the vehicle. For example, ECU 208 is a discrete group of electronic devices that includes their own circuits (e.g., integrated circuits, microprocessors, memory, storage devices, etc.) and firmware, sensors, actuators, and / or mounting hardware. ECU 208 communicates and exchanges information via a vehicle data bus (e.g., vehicle data bus 210). In addition, ECU 208 can communicate properties (e.g., ECU status, sensor readings, control status, errors and diagnostic codes, etc.) and / or receive requests from each other. For example, a vehicle may have dozens of ECUs that are positioned in various locations around the vehicle and communicatively coupled via vehicle data bus 210.

[0257] In the illustrated example, the ECU 208 includes an autonomous unit 208-1 and a body control module 208-2. For example, the autonomous unit 208-1 is configured to perform autonomous and / or semi-autonomous driving maneuvers (e.g., defensive driving maneuvers) of the vehicle based at least in part on instructions received from the controller 212-1 and / or data collected by the sensor 206 (e.g., a distance detection sensor). In addition, the body control module 208-2 controls one or more subsystems of the entire vehicle, such as power windows, power locks, anti-theft systems, power mirrors, etc. For example, the body control module 208-2 includes circuits that drive one or more relays (e.g., to control wiper fluid, etc.), brushed direct current (DC) motors (e.g., to control power seats, power locks, power windows, wipers, etc.), stepper motors, LEDs, safety systems (e.g., seat belt pretensioners, airbags, etc.), etc.

[0258] The vehicle data bus 210 communicatively couples the communication module 220, the onboard computing platform 202, the HMI unit 204, the sensor 206, and the ECU 208. In some examples, the vehicle data bus 210 includes one or more data buses. The vehicle data bus 210 can be configured in accordance with the Controller Area Network (CAN) bus protocol defined by the International Organization for Standardization (ISO) 11898-1, the Media Oriented Systems Transport (MOST) bus protocol, the CAN Flexible Data (CAN-FD) bus protocol (ISO 11898-7), and / or the K-line bus protocol (ISO 9141 and ISO 14230-1), and / or the Ethernet bus protocol. TM It is implemented by bus protocol IEEE 802.3 (after 2002) and so on.

[0259] The communication module 220-1 is configured to communicate with other nearby communication devices. In the example shown, the communication module 220 includes a dedicated short-range communication (DSRC) module. The DSRC module includes an antenna, a radio, and software to communicate with nearby vehicles via vehicle-to-vehicle (V2V) communication, with infrastructure-based modules via vehicle-to-infrastructure (V2I) communication, and / or more generally with nearby communication devices (e.g., mobile device-based modules) via vehicle-to-everything (V2X) communication.

[0260] V2V communication allows vehicles to share information such as speed, location, direction, and other relevant data, enabling them to collaborate and coordinate their actions to improve safety, efficiency, and mobility on the road. V2V communication can be used to support a variety of applications, such as collision avoidance, lane change assistance, platooning, and traffic management. It can rely on dedicated short-range communications (DSRC) and other wireless protocols that enable fast and reliable data transmission between vehicles. V2V communication is a form of wireless communication between vehicles that allows vehicles to exchange information and coordinate with other vehicles on the road. V2V communication enables vehicles to share data about their location, speed, direction, acceleration, and braking with other nearby vehicles, which can help improve safety, reduce congestion, and make transportation systems more efficient.

[0261] V2V communications are typically based on wireless communication protocols, such as Dedicated Short Range Communications (DSRC) or Cellular Vehicle-to-Everything (C-V2X) technology. Using V2V communications, vehicles can receive information about potential hazards, such as accidents or road closures, and adjust their behavior accordingly. V2V communications can also be used to support advanced driver assistance systems (ADAS) and autonomous driving technologies, such as platooning, in which a group of vehicles use V2V communications to coordinate their movements and travel closely together.

[0262] More information about DSRC networks and how the network communicates with vehicle hardware and software is available in the USDepartment of Transportation's Core June 2011 Core System Requirements Specification (SyRS) report (available at http: / / wwwits.dot.gov / meetings / pdf / CoreSystemSESyRSRevA%20(2011-06-13).pdf). DSRC systems can be installed on vehicles and on roadside infrastructure. DSRC systems that include infrastructure information are called "roadside" systems. DSRC can be combined with other technologies, such as global positioning systems (GPS), visible light communications (VLC), cellular communications, and short-range radar to facilitate vehicles to communicate their position, speed, heading, relative position with respect to other objects, and exchange information with other vehicles or external computer systems. DSRC systems can be integrated with other systems such as mobile phones.

[0263] Currently, DSRC networks are identified by the DSRC abbreviation or name. However, other names are sometimes used, usually in connection with the Connected Vehicle program, etc. Most of these systems are either pure DSRC or variants of the IEEE 802.11 wireless standard. However, in addition to pure DSRC systems, it also means a dedicated wireless communication system covering the vehicle and the roadside infrastructure system, which is integrated with GPS and based on the IEEE802.11 protocol for wireless LAN (such as 802.11p, etc.).

[0264] Additionally or alternatively, the communication module 220-2 includes a cellular vehicle-to-everything (C-V2X) module. The C-V2X module includes hardware and software for communicating with other vehicles via V2V communication, with infrastructure-based modules via V2I communication, and / or more generally, with nearby communication devices (e.g., mobile device-based modules) via V2X communication. For example, the C-V2X module is configured to communicate with nearby devices (e.g., vehicles, roadside units, mobile devices, etc.) directly and / or via a cellular network. Currently, C-V2X communication-related standards are being developed by the 3rd Generation Partnership Project.

[0265] In addition, the communication module 220-2 is configured to communicate with an external network. For example, the communication module 220-2 includes hardware (e.g., a processor, a memory, a storage device, an antenna, etc.) and software for controlling a wired or wireless network interface. In the example shown, the communication module 220-2 includes one or more communication controllers for cellular networks (e.g., Global System for Mobile Communications (GSM), Universal Mobile Telecommunications System (UMTS), Long Term Evolution (LTE), Code Division Multiple Access (CDMA)), near field communication (NFC), and / or networks based on other standards (e.g., WiMAX (IEEE 802.16m), local area wireless networks (including IEEE 802.11a / b / g / n / ac or others), wireless Gigabit (IEEE 802.11ad), etc.). In some examples, the communication module 220-2 includes a wired or wireless interface (e.g., an auxiliary port, a universal serial bus (USB) port, The vehicle may be coupled to a mobile device (e.g., a smart phone, a wearable device, a smart watch, a tablet, etc.) to communicate with the external network via the coupled mobile device. The external network may be a public network, such as the Internet; a private network, such as an intranet; or a combination thereof, and may utilize various networking protocols now available or later developed, including but not limited to networking protocols based on TCP / IP.

[0266] In one embodiment of the system, when the target vehicle begins to move, the host vehicle updates the boundaries of the impact zone in real time.

[0267] In one embodiment of the system, the communication between the host vehicle and the target vehicle is via vehicle-to-vehicle (V2V) communication. In one embodiment of the system, the vehicle-to-vehicle (V2V) communication is based on a wireless communication protocol using at least one of dedicated short-range communication (DSRC) and cellular vehicle-to-everything (C-V2X) technology. In one embodiment of the system, the communication between the host vehicle and the target vehicle is via an Internet connection.

[0268] In one embodiment, the communication module enables autonomous communication, wherein autonomous communication includes communication during time periods with minimal supervision in different scenarios. The communication module includes hardware components, the hardware components include a vehicle gateway system, the vehicle gateway system includes a microcontroller, a transceiver, a power management integrated circuit, and an Internet of Things device capable of transmitting one of analog and digital signals through one of telephone, wired or wireless communication.

[0269] The autonomous unit 208-1 of the illustrated example is configured to perform autonomous and / or semi-autonomous driving maneuvers on the vehicle, such as defensive driving maneuvers. For example, the autonomous unit 208-1 performs autonomous and / or semi-autonomous driving maneuvers based on data collected by the sensor 206. In some examples, the autonomous unit 208-1 is configured to operate a fully autonomous system, a parking assist system, an advanced driver assistance system (ADAS), and / or other autonomous systems for the vehicle.

[0270] ADAS is configured to assist the driver in safely operating the vehicle. For example, ADAS is configured to perform adaptive cruise control, collision avoidance, lane assistance (e.g., lane centering), blind spot detection, rear-end collision warning, lane departure warning, and / or any other function that assists the driver in operating the vehicle. In order to perform driver assistance features, ADAS monitors objects (e.g., vehicles, pedestrians, traffic signals, etc.) and develops situational awareness around the vehicle. For example, ADAS utilizes data collected by sensors 206, communication module 220-1 (e.g., from other vehicles, from roadside units, etc.), communication module 220-2 from a remote server, and / or other sources to monitor nearby objects and develop situational awareness.

[0271] In addition, in the example shown, the controller (or control module) 212-1 is configured to monitor the surrounding environment of the vehicle. For example, in order to enable the autonomous unit 208-1 to perform autonomous and / or semi-autonomous driving operations, the controller 212-1 collects data collected by the sensor 206 of the vehicle. In some examples, the controller 212-1 collects location-based data via the communication module 220-1 and / or another module (e.g., a GPS receiver) to facilitate the autonomous unit 208-1 to perform autonomous and / or semi-autonomous driving operations. In addition, the controller 212-1 collects data from adjacent vehicles and / or (ii) via the communication module 220-1 and V2V communication from the roadside unit to further facilitate the autonomous unit 208-1 to perform autonomous and / or semi-autonomous driving operations.

[0272] In operation, according to one embodiment, the communication module 220-1 performs V2V communication with neighboring vehicles. For example, the communication module 220-1 collects data from the neighboring vehicle that identifies (i) whether the neighboring vehicle includes an autonomous and / or semi-autonomous system (e.g., ADAS), (ii) whether the autonomous and / or semi-autonomous system is active, (iii) whether a manual takeover request for the autonomous and / or semi-autonomous system has been issued, (iv) lane detection information of the neighboring vehicle, (v) the speed and / or acceleration of the neighboring vehicle, (vi) the (relative) position of the neighboring vehicle, (vii) the direction of travel of the neighboring vehicle, (viii) the rate of change of the steering angle of the neighboring vehicle, (ix) the size of the neighboring vehicle, (x) whether the neighboring vehicle is utilizing a stability control system (e.g., anti-lock braking, traction control, electronic stability control, etc.), and / or any other information that helps the controller 212-1 monitor the neighboring vehicle.

[0273] Based at least in part on data collected by the communication module 220-1 from the adjacent vehicle via V2V communication, the controller 212-1 is able to determine the collision probability of the adjacent vehicle. For example, the controller 212-1 determines the collision probability of the adjacent vehicle in response to identifying a manual takeover request within the data collected by the communication module 220-1 from the adjacent vehicle. Additionally or alternatively, the controller 212-1 determines the collision probability of the adjacent vehicle in response to identifying the difference between (i) the lane marker position determined by the controller 212-1 of the vehicle based on the sensor 206 and (ii) the lane marker position determined by the adjacent vehicle. In addition, in some examples, the controller 212-1 determines the collision probability of the adjacent vehicle based on data collected from other sources, such as sensors 206 (e.g., the vehicle's distance detector sensor 206-1 and / or other sensors), roadside units communicating with the communication module 220-1 via V2I communication, and / or remote servers communicating with the communication module 220-1. For example, the controller 212 - 1 determines the collision probability of the neighboring vehicle when it is determined that the neighboring vehicle has not detected a nearby object based on data collected by sensors of the vehicle and the neighboring vehicle.

[0274] In some examples, the controller 212-1 determines the collision probability based on the takeover time of the neighboring vehicle and / or the time-to-collision of the neighboring vehicle. For example, the takeover time corresponds to the duration between (1) the neighboring vehicle issuing a request for manual takeover to be performed and (2) the operator of the neighboring vehicle manually taking over control of the neighboring vehicle. The controller 212-1 is configured to determine the takeover time of the neighboring vehicle based on measured characteristics of the neighboring vehicle (e.g., speed, acceleration, size, etc.), the operator of the neighboring vehicle (e.g., measured reaction time, etc.), and / or the environment of the neighboring vehicle (e.g., road conditions, weather conditions, etc.). In addition, the time-to-collision corresponds to the time it will take for the neighboring vehicle to collide with another vehicle (e.g., a third vehicle) and / or an object (e.g., a guardrail, a highway lane divider, etc.) if the current conditions are maintained.

[0275] Additionally or alternatively, the controller 212-1 is configured to determine the time to collision of the adjacent vehicle based on the speed, acceleration, direction of travel, distance to the object, steering angle required to avoid the object, steering angle change rate, and / or other measured characteristics of the adjacent vehicle collected from the adjacent vehicle by the communication module 220-1 via V2V communication. In addition, the controller 212-1 is configured to determine the collision probability of the vehicle based on the collision probability of the adjacent vehicle.

[0276] When determining the collision probability of the adjacent vehicle and determining that the collision probability does not meet the threshold, the autonomous unit 208-1 autonomously performs (e.g., for ADAS) a defensive driving maneuver to prevent the vehicle from being involved in a collision caused by the adjacent vehicle. For example, the autonomous defensive driving maneuver includes deceleration, emergency braking, changing lanes, changing the position within the current driving lane, etc. In some examples, the autonomous unit 208-1 is configured to initiate a defensive driving maneuver before the takeover time of the adjacent vehicle has been completed. That is, the controller 212-1 is configured to cause the autonomous unit 208-1 to perform a defensive driving maneuver before the operator of the adjacent vehicle manually takes over the control of the adjacent vehicle. In addition, in some examples, the controller 212-1 issues an audio, visual, tactile and / or other warning (e.g., via the HMI unit 204) to the operator of the vehicle in response to determining that the collision probability is less than a first threshold but greater than a second threshold to request manual takeover. By issuing such a warning, the controller 212-1 enables the operator of the vehicle to safely control the vehicle before a collision with an adjacent vehicle may occur. Additionally or alternatively, the controller 212 - 1 is configured to perform other defensive measures (eg, pre-filling the brake fluid line) in response to determining that the probability of a collision is greater than a threshold (eg, a second threshold, a third threshold).

[0277] The communication module enables in-vehicle communication, communication with other vehicles, infrastructure communication, grid communication, etc. using vehicle-to-network (V2N), vehicle-to-infrastructure (V2I), vehicle-to-vehicle (V2V), vehicle-to-cloud (V2C), vehicle-to-pedestrian (V2P), vehicle-to-device (V2D), vehicle-to-grid (V2G) communication systems. The system then notifies nearby or surrounding vehicles or vehicles that communicate with the vehicle's communication module. The vehicle uses, for example, a message protocol, i.e., messages that reach other vehicles via broadcast.

[0278] In one embodiment, a connection is established between a vehicle and a nearby vehicle as a surrounding vehicle. The vehicle control system detects the nearby vehicle. The nearby vehicle is detected by exchanging handshake signals. Handshake is an automatic process for negotiating the establishment of a communication channel between entities. The processor sends a start signal through the communication channel to detect the nearby vehicle. If there is a nearby vehicle, the processor can receive a confirmation signal from the nearby vehicle. Upon receiving the confirmation signal, the processor establishes a secure connection with the nearby vehicle. The processor can receive a signal from the nearby vehicle at the communication module. The processor can also automatically determine the source of the signal. The processor communicatively connects the communication module to the nearby vehicle. The processor is then configured to send messages to and / or receive messages from the nearby vehicle. The signal received by the communication module can be analyzed to identify the source of the signal, thereby determining the location of the nearby vehicle.

[0279] In one embodiment, the system enables for two-way communication. The system sends a signal and then receives a signal / communication. In one embodiment, the communication can be permission to access to control another vehicle. In another embodiment, the communication can be incremental control communication, for example, initial control of speed up to 10 miles per hour, then a further additional 10 miles per hour, etc.

[0280] As a first step of the method according to the present disclosure, a data link between the vehicle and a nearby vehicle or any other external device is established to allow data to be exchanged between the vehicle and the nearby vehicle or any other external device in the form of a two-way communication. This can be done, for example, via a radio link or a data cable. Thus, a nearby vehicle or any other external device can receive data from the vehicle, or the vehicle can request data from a nearby vehicle or any other external device.

[0281] In one embodiment, the two-way communication includes a device for data acquisition and is designed to exchange data bidirectionally with each other. In addition, at least the vehicle includes a logical device for collecting data and arranging it to a certain protocol based on the protocol of the receiving entity.

[0282] Initially, a data link for bidirectional communication is established. The vehicle and nearby vehicles or any other external device can communicate with each other via this data link and thus request or exchange data, wherein the data link can be implemented, for example, as a cable link or a radio link.

[0283] Bidirectional communication has various advantages as described herein. In various embodiments, data is communicated and transmitted at suitable intervals (including, for example, 200 millisecond (ms) intervals, 100 ms intervals, 50 ms intervals, 20 ms intervals, 10 ms intervals, or even more frequently and / or in real time or near real time) to allow the vehicle to respond to the data or otherwise react to the data. Bidirectional IR communication can be used to facilitate data exchange.

[0284] The device for performing two-way communication for a vehicle according to an embodiment may be by means of a personal area network (PAN) modem. Therefore, a user can access an external device using the in-vehicle information terminal, and then can store, move, and delete data desired by the user.

[0285] In one embodiment, the vehicles can transmit messages via a communication link. It can use any combination of vehicle-to-vehicle (V2V), vehicle-to-everything (V2X), or vehicle-to-infrastructure (V2I) type communications. In one embodiment, it uses vehicle-to-vehicle (V2V) communications, which enables vehicles to wirelessly exchange (communicate) information such as about their speed, location, and heading.

[0286] In one embodiment, the messaging protocol includes at least one of Advanced Message Queuing Protocol (AMQP), Message Queuing Telemetry Transport (MQTT), Simple (or Streaming) Text-Oriented Messaging Protocol (STOMP), MQTT-S (and an extension of open publish / subscribe MQTT), which are heavily used in IoT-based technologies and edge networks.

[0287] In one embodiment of the system, the host vehicle is operable to establish communication with the target vehicle via the communication module to obtain information about the state of the target vehicle, wherein the state includes a speed change and an uphill road inclination of the target vehicle. In one embodiment of the system, the uphill road inclination is obtained using a longitudinal accelerometer sensor, GPS, an inertial measurement unit (IMU), LiDAR, radar, and a camera.

[0288] In one embodiment of the system, the communication between the host vehicle and the target vehicle is via vehicle-to-vehicle (V2V) communication. In one embodiment of the system, the V2V communication is based on a wireless communication protocol using at least one of dedicated short range communication (DSRC) and cellular vehicle-to-everything (C-V2X) technology. In one embodiment of the system, the communication between the host vehicle and the target vehicle is via an Internet connection.

[0289] Figure 3A A block diagram of a method for avoiding collision based on road surface conditions according to one embodiment is shown. According to one embodiment, this is a method, which includes: in step 302, a processor determines a road surface condition, wherein the road surface condition is at least one of ice, wetness and snow; in step 304, a processor adjusts a safety distance value of a host vehicle based on the road surface condition; in step 306, a computer vision module is used to detect a vehicle type, speed and visible roof area of ​​a target vehicle, wherein the target vehicle is a vehicle in front of the host vehicle; in step 308, a processor determines an uphill road that the host vehicle is approaching; in step 310, a processor determines that the target vehicle is decelerating based on a real-time speed change of the target vehicle; in step 312, a processor predicts a collision between the host vehicle and the target vehicle; in step 314, a processor determines a collision avoidance action of the host vehicle to avoid a collision with the target vehicle.

[0290] In one embodiment, the system is configured to autonomously perform the collision avoidance maneuver by the host vehicle, and wherein the host vehicle is an autonomous vehicle.

[0291] In one embodiment, the type of vehicle or vehicle type may refer to a broad category of vehicles, such as cars, trucks, vans, motorcycles, etc., while the vehicle model refers to a specific version of a specific vehicle type produced by a manufacturer with a unique name and features, such as Volvo S60, XC90, XC60, and V90.

[0292] In one embodiment of the system, the collision avoidance action includes generating an alert in the host vehicle, wherein the alert is at least one of a text message, a visual prompt, an audio alert, a tactile prompt, and a vibration. In one embodiment of the system, the collision avoidance action includes further increasing a safety distance value between the host vehicle and the target vehicle.

[0293] In one embodiment of the system, the collision avoidance action includes waiting until the target vehicle safely passes the uphill road. In one embodiment of the system, the collision avoidance action includes at least one of: initiating a reverse movement by the host vehicle; and initiating a lane change by the host vehicle.

[0294] In one embodiment of the system, the collision avoidance action includes broadcasting a message to vehicles behind the host vehicle that the target vehicle is sliding down an uphill road. In one embodiment of the system, the collision avoidance action includes broadcasting a message to neighboring vehicles, wherein the message includes maneuvering instructions for neighboring vehicles to use collective and coordinated actions to avoid the impending collision.

[0295] In one embodiment of the system, the collision avoidance action includes predicting a collision and alerting the driver of the host vehicle of the deployment of the airbag during the collision. In one embodiment of the system, the collision avoidance action includes predicting a collision, alerting the driver of the host vehicle, and advising the driver to prepare for the impact posture. In one embodiment of the system, the collision avoidance action includes predicting a collision, alerting the driver of the host vehicle, and advising him to leave the host vehicle.

[0296] In one embodiment of the system, the collision avoidance action includes alerting the target vehicle of the impending collision. In one embodiment of the system, the collision avoidance action includes alerting a neighboring vehicle of the collision.

[0297] In one embodiment of the system, the collision avoidance action includes at least one of: generating an alert in the host vehicle; maintaining at least a safe distance value between the host vehicle and the target vehicle; initiating a reverse movement by the host vehicle; initiating a lane change by the host vehicle; issuing an alert to the target vehicle about an impending collision; and broadcasting a message to neighboring vehicles, wherein the message includes maneuvering instructions for neighboring vehicles to avoid the impending collision using collective and coordinated actions. In one embodiment of the system, the collision avoidance action is determined and executed in real time until the target vehicle safely passes the uphill section.

[0298] In one embodiment of the system, the collision avoidance maneuver is determined via a computer vision module including an artificial intelligence engine, wherein the artificial intelligence engine includes a machine learning algorithm. In one embodiment of the system, the collision avoidance maneuver also includes activating a camera of the computer vision module to record the surroundings of the host vehicle.

[0299] In one embodiment of the system, the collision avoidance action comprises alerting a third party of an impending collision. In one embodiment of the system, the third party is emergency services.

[0300] Figure 3BA block diagram of a system for avoiding collisions based on road surface conditions according to one embodiment is shown. According to one embodiment, it is a system 340, which includes: a processor 342, a computer vision module 344 and a control module 346; wherein the processor 342 is operable to: in step 302, the processor determines the road surface condition, wherein the road surface condition is at least one of ice, wetness and snow; in step 304, the processor adjusts the safety distance value of the host vehicle based on the road surface condition; in step 306, the computer vision module detects the vehicle type, speed and visible roof area of ​​the target vehicle, wherein the target vehicle is the vehicle in front of the host vehicle; in step 308, the processor determines the uphill road that the host vehicle is approaching; in step 310, the processor determines that the target vehicle is decelerating based on the real-time speed change of the target vehicle; in step 312, the processor predicts a collision between the host vehicle and the target vehicle; in step 314, the processor determines a collision avoidance action for the host vehicle to avoid a collision with the target vehicle, and wherein the host vehicle is an autonomous vehicle. According to one embodiment of the system, the system is operable as a component of the host vehicle.

[0301] According to one embodiment of the system, the road surface conditions are determined by receiving real-time weather broadcasts. According to one embodiment of the system, the road surface conditions are determined by a map showing icy roads in the route.

[0302] According to one embodiment of the system, the road surface condition is determined by detecting the road condition in real time via a computer vision module. According to one embodiment of the system, the road surface condition is determined by analyzing the scattering of the emitted light beam on the road surface using a filtering technique on an image captured by the computer vision module.

[0303] According to one embodiment of the system, the road surface condition is detected by using vehicle dynamics sensor data and an image of the environment in front of the host vehicle obtained by a camera, wherein the camera is a monochrome camera.

[0304] According to one embodiment of the system, a surface condition observer classifies road conditions based on evaluating images from a camera and comparing them to vehicle dynamics sensor data.

[0305] According to one embodiment of the system, the surface condition observer also considers local weather data to derive the road surface condition, wherein the local weather data includes temperature, humidity values, rainfall, sleet, snowfall, wiper activity, and cloud data.

[0306] According to one embodiment of the system, the road surface condition is detected by identifying road signs using a computer vision module. According to one embodiment of the system, the road surface condition is detected by vehicle-to-infrastructure (V2I) communication.

[0307] According to one embodiment of the system, the road surface condition is determined by determining that a neighboring vehicle has difficulty navigating. According to one embodiment of the system, the road surface condition is determined by receiving a broadcast message from a neighboring vehicle regarding the road surface condition. According to one embodiment of the system, the road surface condition is determined by the coefficient of friction between the road on which the host vehicle is traveling and the tires of the host vehicle via an accelerometer sensor.

[0308] According to one embodiment of the system, the road surface condition is determined by a road condition sensor, wherein the road condition sensor is at least one of a temperature sensor, an infrared sensor, an accelerometer, a GPS sensor, and a humidity sensor.

[0309] Figure 3C A block diagram of a method for avoiding collisions based on road surface conditions stored on a non-transitory computer-readable medium according to one embodiment is shown. According to one embodiment, it is a non-transitory computer-readable medium 374 on which instructions executable by a computer system are stored to perform operations including: in step 302, a road surface condition is determined by a processor, wherein the road surface condition is at least one of ice, wetness, and snow; in step 304, a safety distance value of the host vehicle is adjusted by the processor based on the road surface condition; in step 306, a vehicle type, speed, and visible roof area of ​​a target vehicle are detected by a computer vision module, wherein the target vehicle is a vehicle in front of the host vehicle; in step 308, an uphill road that the host vehicle is approaching is determined by the processor; in step 310, the target vehicle is determined to be decelerating based on the real-time speed change of the target vehicle by the processor; in step 312, a collision between the host vehicle and the target vehicle is predicted by the processor; in step 314, a collision avoidance action of the host vehicle is determined by the processor to avoid a collision with the target vehicle; and wherein the host vehicle is an autonomous vehicle. A software application 376 can be stored on a computer-readable medium 374 and executed by a processor 372 of a computer system 371.

[0310] In one embodiment, a control module of the host vehicle determines a road surface condition, wherein the road surface condition is at least one of ice, wet, or snow. In one embodiment, a system of the vehicle is enabled to detect icy conditions. This may be based on: (i) weather broadcasts predicting / forecasting freezing days (ii) broadcasts of vehicles going uphill finding difficulty due to icy road surface conditions (iii) equipment on the vehicle itself.

[0311] In one embodiment, an autonomous vehicle (Av) uses a combination of sensors and algorithms to detect and respond to different road conditions, including the presence of ice on the road. Some methods of detecting ice / precipitation on the road include: (i) Temperature sensors: As with snow, temperature sensors can be used to detect when the road surface temperature drops below freezing, which is an indication that ice may be present. (ii) Light Detection and Ranging Systems (LiDAR): Light Detection and Ranging sensors use lasers to create a 3D map of the autonomous vehicle's surroundings. This can be used to detect changes in the road surface, including the presence of ice. (iii) Radar (RADAR): Radar sensors can detect the presence of ice on the road by measuring the reflectivity of the surface. Ice typically has a higher reflectivity than dry pavement, which can be detected by radar sensors. (iv) Infrared cameras: Infrared cameras can detect the temperature of the road surface and can be used to determine the presence of ice. Ice is typically cooler than dry pavement, which can be detected by infrared cameras. (v) Road Condition Databases: Some autonomous vehicles can use road condition databases to predict the presence of ice on the road. These databases can compile data collected from a variety of sources, including weather reports, road temperature sensors, and other autonomous vehicles.

[0312] In one embodiment, the detection of a snow / ice covered path of a travel surface is via subsurface scattering of light. A method for determining a snow / ice / sleet covered surface condition of a travel path, the method comprising: emitting a light beam at a surface of the travel path by a light emitting source; capturing an image of the surface in a downward direction relative to the vehicle by an image capture device mounted on a vehicle, wherein the captured image captures the emitted light beam; identifying a region of interest having the captured emitted light beam in the captured image; analyzing, by a processor, subsurface scattering of the emitted light beam on the surface in the region of interest, including converting image components to a Lab color space to separate color components from a brightness component of the captured emitted light beam, and applying feature extraction to at least one color component in the Lab color space using a filtering technique to detect edges in the region of interest, identifying a snow / ice / sleet covered surface condition in the region of interest by applying a binary conversion to the Lab color space; and generating a snow covered path signal of the travel surface in response to identifying the snow covered surface condition. Lab (L: brightness; a*: red / green value; b*: blue / yellow value) color values ​​provide a method of locating and conveying color. By isolating the effects of ambient noise, images converted from RGB to Lab color space easily capture the relevant color information of the light beams in the image for analysis. This involves converting the RGB image (i.e., red-green-blue image) to Lab color space to better detect the broad blurring pattern that varies with light as the light signal is scattered under the surface of the snow. Lab space includes various components with a dimension L for brightness and "A" and "B" for color opposition dimensions. Lab color space includes all perceptible colors, which means that its range exceeds the range of the RGB color model.

[0313] Precipitation on a driving surface creates several different problems for a vehicle or person traveling along a path of travel. For example, snow on the road reduces the coefficient of friction between the vehicle's tires and the road surface, causing vehicle stability issues. Detection of snow / ice on the road of travel is typically determined by the host vehicle sensing the snow / ice on the road using some sensing operation that occurs when the snow / ice has affected vehicle operation (e.g., detecting wheel slip). As a result, the vehicle monitors its own operating conditions (e.g., wheel slip) against dry road operating conditions to determine if snow / ice is present.

[0314] In one embodiment, when the autonomous vehicle predicts icy / snow covered / wet roads, then the autonomous vehicle will adjust for your route. In one embodiment, the autonomous vehicle further adjusts the safety distance value to a higher value than normal. For example, during normal road conditions, if the safety distance is 20 feet, then when icy roads are predicted or detected, the vehicle automatically updates the safety distance value to 40 feet. On an icy uphill road, any distance below 40 feet will generate an alert to the autonomous vehicle driver. When the autonomous vehicle is on an uphill icy road, it will further increase the safety distance to a higher value than on a flat icy road.

[0315] When the host vehicle is on an uphill road and notices that the vehicle in front of it is skidding, i.e. moving backwards, due to icy roads or any other reason, then the host vehicle automatically starts taking actions to avoid any possible collision. In one embodiment, the host vehicle can move to a side lane, or it will start backing up to avoid the vehicle skidding, i.e. moving backwards, until the situation is resolved or the host vehicle is safe.

[0316] In one embodiment, an advanced driver assistance system (ADAS) may have sensors and algorithms capable of detecting the condition of a tire, including tire pressure, tread depth, and tire type, configured to provide a warning to the driver if the tire is not suitable for the current driving conditions, including icy roads. Avs use a combination of sensors, algorithms, and machine learning to detect and respond to different driving conditions, including the condition of the tires. An autonomous vehicle may have sensors that measure tire pressure, wear, and traction, and use this data to adjust the vehicle's behavior accordingly. For example, if a tire has less traction, the autonomous vehicle can detect this and adjust its driving style to compensate for the reduced traction.

[0317] In one embodiment, sensors of the autonomous vehicle detect tire parameters and provide warnings that the tires are not suitable for the road surface conditions. In one embodiment, the autonomous vehicle will adjust driving behavior according to the road surface conditions.

[0318] In one embodiment, the road condition may not be icy even if the weather is cold. In one embodiment, a vehicle traveling on an icy road may detect icing by detecting that the friction is not as good as normal road surface conditions. In one embodiment, a vehicle already on the road reports back that the road surface condition is slippery. Based on the input of the road condition being icy, a routing system may provide an alternate route with no icy roads or preferably with minimal icy roads.

[0319] In one embodiment, the control module of the host vehicle adjusts the safety distance value of the host vehicle based on the road surface condition. Figure 1 In one embodiment, if the vehicle 104 is traveling on an icy road, the safety distance will be automatically modified by the vehicle control system to a value higher than that on normal roads / normal days. If the vehicle is traveling from point A to point B in icy conditions on an uphill road, the safety distance is further modified to a higher value. In one embodiment, in addition to automatically modifying the safety distance, an additional warning / alert message is issued on the infotainment system on the vehicle 104 to attract the attention of the vehicle driver or occupants.

[0320] Even after adjusting the safety distance, if the vehicle 104 observes that the vehicle 102 in front of the vehicle 104 is losing traction, the vehicle 104 will provide a warning and can further adjust the safety distance. For example, if the safety distance is 20 feet on a normal day, then on an icy road, it can be set to 30 feet, and on an icy uphill road, it can be further set to 40 feet. In one embodiment, the vehicle issues a warning if the safety distance is not maintained. In one embodiment, when the vehicle 102 in front of the vehicle 104 begins to slide downward, a warning is provided in the vehicle 104. In one embodiment, an icy road, an icy uphill road, a vehicle 102 in front of the vehicle 104 is detected, and the vehicle 104 begins to make decisions to avoid accidents, and the vehicle 104 can be an autonomous vehicle.

[0321] In one embodiment, the warning system alerts the user of the vehicle that there is an icy road in the vehicle's path. In one embodiment, the vehicle sets the safety distance value to a value higher than normal road conditions.

[0322] In one embodiment, the system will also consider the type of vehicle and the weight of the vehicle. On icy roads, weight and vehicle type have a great impact on stopping distance. In one embodiment, based on the weight of the host vehicle, the safe distance between the host vehicle and the target vehicle (i.e., the vehicle in front of the host vehicle) is determined. In one embodiment, based on the weight of the target vehicle, the safe distance between the host vehicle and the target vehicle (i.e., the vehicle in front of the host vehicle) is determined. In one embodiment, based on the tire tread of the host vehicle, the safe distance between the host vehicle and the target vehicle (i.e., the vehicle in front of the host vehicle) is determined. In one embodiment, based on the tire tread of the target vehicle, the safe distance between the host vehicle and the target vehicle (i.e., the vehicle in front of the host vehicle) is determined. In one embodiment, the safe distance is determined based on at least one of the road surface condition, the weight of the host vehicle, the weight of the target vehicle, the tire tread of the host vehicle, and the tire tread of the target vehicle.

[0323] In one embodiment, the control module of the host vehicle detects the vehicle type, speed, and visible roof area of ​​a target vehicle, where the target vehicle is the vehicle in front of the host vehicle. In one embodiment, the autonomous vehicle can perform vehicle detection using computer vision, drive to a destination, and can detect parts of the target vehicle using computer vision.

[0324] Figure 4A The parts of a vehicle detected using a bounding box method are shown. In one embodiment, multiple parts of a vehicle are detected using a bounding box method. For example, the results of a bounding box method for detecting a windshield 402, a headlight 404, a door 406, and a wheel 408 are shown. In one embodiment, the bounding box can be a 3D bounding box. In one embodiment, a 3D bounding box can be used to better understand and more accurately estimate the depth parameters and orientation of an object. In computer vision, a 3D bounding box provides more information than a 2D bounding box because a 3D bounding box provides information about the size, position, and orientation of an object in the 3D world, which is important for various applications in autonomous driving. By using a 3D bounding box, a computer vision system is able to more accurately detect and track objects in an environment, thereby enabling more informed decisions and better performance in real-world scenarios. In one embodiment, a computer vision system can utilize both 2D and 3D methods in a hierarchical manner to first quickly select an area of ​​interest, and then it can utilize a 3D bounding box to try to gain more insight about the object.

[0325] Figure 4Bis a process flow chart of a method for identifying and classifying one or more objects according to one embodiment of the present invention. Object detection is a computer technology related to computer vision and image processing, which deals with detecting instances of a certain class of semantic objects in digital images and videos, such as cars, bicycles, road signs, traffic lights, road infrastructure, boundary railings, etc. Object detection in autonomous vehicles also includes face detection, pedestrian detection, etc. The method can start at box 412, where the method can include receiving vehicle dynamics data from a vehicle dynamics sensor. In one embodiment, the data receiving module can communicate with the vehicle dynamics sensor of the vehicle to collect vehicle dynamics data. Specifically, the vehicle dynamics data can include but is not limited to vehicle speed data, traction data, steering speed data, steering angle data, throttle angle data, vehicle acceleration data, braking force data, wheel speed data, wheel steering angle data, yaw rate data, transmission gear data, temperature data, revolutions per minute (RPM) data, global positioning system (GPS) / differential global positioning system (DGPS) data, etc. The vehicle dynamics data can be used to identify the current position of the vehicle, its steering angle, whether it is moving or at an intersection, etc. The vehicle trip dynamics data may be used to collect data corresponding to the vehicle's ability to perform various maneuvers. The trip-related data may also include data about the route, such as the number of lanes, whether there is an uphill road, and intersection data, such as whether the intersection is wide, narrow, or an accident-prone area. In one embodiment, the data receiving module may package and store the vehicle dynamics data on a memory.

[0326] The method may proceed to block 414, where the method may include receiving image data from a vehicle camera system. In one embodiment, the data receiving module may communicate with the vehicle camera system to capture untrimmed images / video of the vehicle's surroundings and interior portions of the vehicle. The untrimmed images / video may include a 360 degree exterior and interior view of the vehicle and the vehicle's surroundings. In one embodiment, the data receiving module may package and store the image data on a memory associated with the vehicle trip dynamics data (of block 412).

[0327] The method may proceed to block 416, where the method may include receiving LiDAR data from a vehicle laser projection system. In an exemplary embodiment, a data receiving module may communicate with a vehicle laser projection system to collect LiDAR data that classifies object coordinate sets (e.g., three-dimensional LiDAR object coordinate sets) from one or more traffic-related objects located within the vehicle's surroundings. The one or more object coordinate sets may indicate the location, range, and location of the one or more objects from which reflected laser waves were reflected relative to the location / location of the vehicle. In one embodiment, the data receiving module may package and store the LiDAR data on a memory associated with the vehicle trip dynamics data (of block 412).

[0328] The method may proceed to box 418, where the method may include fusing the vehicle trip dynamics data, image data, and LiDAR data. In an exemplary embodiment, the data receiving module may communicate with the neural network processing unit to provide artificial intelligence capabilities to perform multimodal fusion. The data receiving module may utilize one or more machine learning / deep learning fusion processes to aggregate the vehicle trip dynamics data, image data, and LiDAR data stored on the memory by the data receiving module (at boxes 412, 414, 416). The fused data may be evaluated for stored vehicle dynamic parameters, image recognition parameters, and object recognition parameters that are associated with multiple vehicle motion patterns stored in the memory and used by the neural network.

[0329] In particular, the neural network processing unit may perform machine learning / deep learning to determine one or more motion patterns from the fused data based on an evaluation of the fused data relative to stored dynamic parameters, image recognition parameters, and object recognition parameters. The determination of the one or more motion patterns may occur during each trip of the vehicle, where vehicle trip dynamic data, image data, and LiDAR data are received to determine one or more driving maneuvers. In one embodiment, machine / deep learning techniques, image logic, and LiDAR logic are used to evaluate the one or more motion patterns and at least one of the position, range, and location of one or more traffic-related objects, map data, and additional vehicle dynamic data (e.g., steering angle, throttle angle, speed, acceleration, etc.) to determine one or more driving maneuvers (e.g., braking, backing, lane change, merging, etc.).

[0330] The method may proceed to block 420, where the method may include evaluating image data associated with the external environment of the vehicle and determining one or more sets of image coordinates corresponding to one or more sets of object coordinates. In one embodiment, the neural network processing unit utilizes one or more machine learning / deep learning fusion processes and image logic to determine one or more image coordinates associated with traffic-related objects located within the surrounding environment of the vehicle. As described above, such traffic-related objects may include, but are not limited to, other vehicles, traffic participants, intersections, road attributes, and roadside objects. The image coordinate set may include a two-dimensional (x, y) coordinate representing one or more pixels of an image / video, the one or more pixels corresponding to the location of one or more objects within the image / video determined by the neural network processing unit. In one embodiment, the neural network processing unit may utilize one or more machine learning / deep learning fusion processes and LiDAR logic to determine one or more sets of object coordinates that may be associated with one or more traffic-related objects. As described above, the one or more object coordinate sets may include one or more sets of three-dimensional coordinates or geocentric coordinates based on the reception of one or more reflected laser waves by the LiDAR transceiver.

[0331] The method may proceed to block 422, where the method may include evaluating image data associated with the external environment of the vehicle and determining one or more image coordinate sets corresponding to one or more object coordinate sets. In one embodiment, in determining the one or more object coordinate sets, the neural network processing unit may access a pre-programmed LiDAR coordinate map stored on a memory, which may include one or more image coordinate sets corresponding to the one or more object coordinate sets. The neural network processing unit may utilize the pre-programmed LiDAR coordinate map to determine one or more image coordinate sets of one or more traffic-related objects corresponding to the one or more object coordinate sets of one or more traffic-related objects to accurately locate a portion of an image / video that may include the one or more traffic-related objects sensed by the vehicle camera system and the vehicle laser projection system. Utilization of the pre-programmed LiDAR coordinate map may ensure that one or more traffic-related objects located within the vehicle's surroundings and present within the vehicle's path of travel are considered, such as obstacles on the path, vehicles encountered during travel, or types of intersections that are difficult to maneuver due to their geometry, etc.

[0332] The method may proceed to block 424, where the method may include identifying one or more portions of the image including the one or more image coordinate sets as object space. In one embodiment, one or more portions of the image / video representing the one or more image coordinate sets corresponding to the one or more object coordinate sets as object space are identified. In other words, portions of the image / video corresponding to the one or more object coordinate sets of the object reflecting the laser beam wave back to the LiDAR transceiver of the vehicle laser projection system are identified as object space.

[0333] The method may proceed to block 426, where the method may include calculating one or more bounding boxes around one or more traffic-related objects. In one embodiment, the method may calculate one or more bounding boxes around one or more portions of the object space to distinguish traffic-related objects and other regions of interest within the object space from other portions of the image / video that do not include the one or more traffic-related objects or do not correspond to the regions of interest. The method may additionally calculate feature data associated with the traffic-related objects determined by the neural network processing unit that are enclosed within the calculated bounding boxes. For example, the feature data of the identified vehicle object may include its size and shape, the driver, its visual turn signals, whether it is an autonomous vehicle, etc.

[0334] Once one or more bounding boxes around one or more traffic-related objects are calculated, the method may proceed to box 428, where the method may include evaluating the one or more bounding boxes and classifying the one or more traffic-related objects. In one embodiment, the method may utilize a neural network processing unit to evaluate the traffic-related objects enclosed within the one or more calculated bounding boxes. The neural network processing unit may scan the portion of the image / video enclosed within the one or more bounding boxes, and may compare the one or more traffic-related objects found within the one or more bounding boxes with a traffic-related object model that may be stored in a memory of the vehicle and utilized by the neural network. The neural network processing unit may compare the traffic-related objects found within the one or more bounding boxes, and may determine one or more traffic-related objects (e.g., vehicles, signs, lane boundaries, etc.) that may be located within the environment of the vehicle. In one embodiment, the traffic-related objects are further detected in various portions within the object. For example, windshields, headlights, door areas, license plates, tires, etc. are detected within vehicle objects.

[0335] In one embodiment, the neural network processing unit may compare the traffic-related object(s) enclosed within the one or more calculated bounding boxes with the traffic-related object model to classify the one or more objects captured within the image(s) into various categories, groups, and subgroups. Using the category, group, and subgroup classification system, large vehicles such as commercial trucks can be classified into commercial vehicles with groups such as tractor trailers, dump trucks, or box trucks, and subgroups such as flatbed trucks or refrigerated trucks. The classification system can be applied to vehicle recognition, for example, passenger cars can be classified into sedans, sport utility vehicles (SUVs), or minivans, with further subgroups such as compact SUVs or full-size SUVs.

[0336] According to one embodiment, intersections, target vehicles near intersections, vehicle types, traffic signals, lanes, lane markings, signs, etc. can be detected by a computer vision system with the help of artificial intelligence / machine learning (AI / ML) algorithms. In one embodiment, it can use a bounding box method. According to one embodiment, intersections are detected by detecting traffic lights. According to one embodiment, traffic lights are detected by at least one of a camera of a computer vision module of the host vehicle and vehicle-to-infrastructure (V2I) communication.

[0337] Figure 4C An autonomous vehicle is shown detecting the roof of another vehicle according to one embodiment. To detect the roof of a vehicle in front of the autonomous vehicle via a computer vision algorithm, the autonomous vehicle detects portions of the vehicle and detects the area of ​​the geometric shape that the roof has. Here are some examples based on the degree of tilt of the vehicle. For example, if the target vehicle is traveling at the same tilt as the angle viewed from the autonomous vehicle, a smaller area in the geometric shape is obtained. In the case where the vehicle in front of the autonomous vehicle begins to tilt, such as Figure 4C As shown in (A) to (C), the area increases. The autonomous vehicle as the host vehicle enables detection of the roof area and changes in the roof area in consecutive reading intervals. In one embodiment, the autonomous vehicle detects the roof of the other vehicle as a geometric shape, such as a rectangle. Depending on the inclination of the vehicle in front of the autonomous vehicle, the autonomous vehicle will obtain geometric shapes with different areas.

[0338] According to one embodiment, using a camera of an autonomous vehicle and using a computer and computer vision algorithms, a system or method detects the vehicle and parts of the vehicle.

[0339] In one embodiment, it can be radar and image processing that can detect the tilt of the vehicle or the positioning of the vehicle.In one embodiment, the system and / or method detects how much of the roof area in the image is captured.

[0340] Figure 4DAn autonomous vehicle is shown continuously detecting the roof of another vehicle according to one embodiment. In one embodiment, the autonomous vehicle will look for a sequence of roof areas. The autonomous vehicle will set a timer to sample the roof in sequence (e.g., every 1 or 2 seconds). Once the roof is detected and the area is obtained, the autonomous vehicle will look for changes in the roof area. Figure 4D As shown, a sequence of detecting a small area and then increasing that area will allow the autonomous vehicle to infer that the vehicle is on a sloped road. The sequence of readings is needed to avoid false positives, such as a vehicle in a tilted position being towed by a trailer.

[0341] Figure 4E An autonomous vehicle is shown continuously detecting the roof of another vehicle acting as a tow vehicle according to one embodiment. In one embodiment, the autonomous vehicle will look for a sequence of roof areas. The autonomous vehicle will set a timer to sample the roofs in sequence (e.g., every 1 or 2 seconds) to avoid false positives, such as a tilted vehicle being towed by a trailer being interpreted as a vehicle on a tilted road.

[0342] Figure 4F An autonomous vehicle is shown continuously detecting the roof of another vehicle according to one embodiment. In one embodiment, the autonomous vehicle can also look for vehicles coming in the opposite direction and observe the change in roof area. If the roof area is decreasing, then it is further certain that the road section is inclined. A change from a larger roof area to a smaller roof area for a vehicle in the opposite lane would mean that the vehicle is moving downward. In one embodiment, radar and LiDAR and computer vision systems can be used to detect the roof area, speed, movement, and direction of vehicles around the autonomous vehicle.

[0343] In one embodiment, a rear view camera, radar, and LiDAR are used via a computer vision module to determine the change in roof area of ​​a vehicle moving uphill or downhill.

[0344] In one embodiment, an autonomous vehicle measures roof areas of a first target vehicle and a second target vehicle, wherein the first target vehicle is located in front of the autonomous vehicle and is traveling in the same direction as the autonomous vehicle, and the second target vehicle is descending in a direction opposite to the direction of the autonomous vehicle.

[0345] In one embodiment, radar is used to ensure that there are no vehicles behind the ego vehicle. In one embodiment, if there is a vehicle behind the ego vehicle, the ego vehicle will detect whether it is an uphill section of the road or a flat section to make appropriate decisions about collision avoidance maneuvers. In one embodiment, detection of the roof of the vehicle is performed by the ego vehicle through front radar for the vehicle ahead and rear radar for the vehicle behind.

[0346] Figure 4GAn autonomous vehicle is shown detecting the vehicle type, model, and brand according to one embodiment. The autonomous vehicle will perform vehicle type recognition to understand how large the expected roof area of ​​the vehicle is.

[0347] In one embodiment, the control module of the host vehicle detects the license plate number of the target vehicle. License plate detection can be one of the features of the autonomous vehicle, wherein it enables the host vehicle to distinguish and identify other vehicles on the road. License plate detection can be implemented using a combination of cameras, sensors, and software algorithms. According to one embodiment of the system, the vehicle type is determined from the license plate number of the target vehicle by referencing a database.

[0348] In one embodiment, an autonomous vehicle equipped with a camera captures images of the surrounding environment, including other vehicles on the road. The images captured by the camera are processed using a computer vision system that uses image processing algorithms to identify the license plate of the target vehicle. This includes analyzing the unique shape and pattern of the license plate in the image. Once the license plate is identified, optical character recognition (OCR) software is used to extract alphanumeric characters from the license plate. This includes analyzing the specific shapes and patterns of the characters in the image and then translating these patterns into text or numbers. The extracted license plate number can then be used to find information about the vehicle in a database via a database lookup. In one embodiment, the details can include, but are not limited to, the make and model of the vehicle, the registered owner, the maximum weight, the maximum speed. In one embodiment, license plate detection can also be used to track the movement of the target vehicle.

[0349] In one embodiment, the control module of the host vehicle determines that the host vehicle is approaching an uphill road. In one embodiment, the autonomous vehicle can detect the uphill road using a combination of sensors and map positioning technology.

[0350] In one embodiment of the system, uphill roads are detected using GPS and mapping technology. The autonomous vehicle can use GPS to determine its location and use map data to determine the slope of the road at that location. This information can also be used to adjust the vehicle's speed and power output to accommodate the slope.

[0351] In one embodiment of the system, sensors such as accelerometers and gyroscopes are used to detect uphill roads. These sensors can detect changes in vehicle acceleration and orientation, which can indicate that the vehicle is traveling on a slope. By analyzing this data, the vehicle's control system can adjust the power output and torque of the vehicle's engine to maintain a consistent speed on the uphill road.

[0352] In one embodiment of the system, uphill roads are detected using computer vision in order to detect the slope of the road. By analyzing images of the road ahead, the vehicle's onboard computer is able to detect changes in the angle of the road. This data can then be used to adjust the vehicle's speed and power output accordingly.

[0353] In one embodiment of the system, an autonomous vehicle can use a combination of sensors, mapping technology, and computer vision to detect uphill roads. This information can be used to adjust speed and power output to navigate safely and efficiently.

[0354] In one embodiment of the system, the uphill road is determined based on the vehicle type of the target vehicle, the speed of the target vehicle, and the visible roof area of ​​the target vehicle. In one embodiment of the system, the uphill road is determined based on the GPS location of the target vehicle and map data to determine the slope of the uphill road. In one embodiment of the system, the host vehicle maps the visible roof area by vehicle type, vehicle model, and speed to determine the inclination of the uphill road.

[0355] In one embodiment of the system, the collision avoidance action includes waiting until the target vehicle passes an uphill road.

[0356] In one embodiment of the system, the vehicle is operable to detect changes in visible roof area to determine the inclination of an uphill road by a sequence of consecutive readings timed at predetermined intervals. In one embodiment of the system, the vehicle is operable to detect changes in visible roof area of ​​a vehicle approaching in another lane in the opposite direction of the target vehicle to determine the inclination of an uphill road by a sequence of consecutive readings timed at predetermined intervals. In one embodiment of the system, the vehicle is operable to detect the beginning of an uphill road and the end of an uphill road.

[0357] In one embodiment of the system, the host vehicle is operable to notify the driver of the host vehicle of an approaching uphill road via a notification to the driver of the host vehicle, wherein the notification is at least one of an audible, tactile prompt, a visual prompt, and a text message.

[0358] In one embodiment of the system, the host vehicle is operable to detect the number of lanes on the uphill road via a computer vision module. In one embodiment of the system, the host vehicle is operable to correlate the visible roof area based on the vehicle type and vehicle model to determine the inclination of the uphill road. In one embodiment of the system, the safety distance value of the host vehicle is further increased based on the inclination of the uphill road and the road surface conditions. In one embodiment, the safety distance value depends on the speed at which the target vehicle and the host vehicle are traveling.

[0359] In one embodiment of the system, the change in speed of the target vehicle is calculated by the vehicle based on speed data collected over time. In one embodiment of the system, the vehicle is operable to detect whether the target vehicle is stalling. In one embodiment of the system, the vehicle is operable to detect whether the target vehicle is losing traction. In one embodiment of the system, the vehicle is operable to predict whether the speed of the target vehicle is approaching zero. In one embodiment of the system, the vehicle is operable to use an artificial intelligence algorithm to predict whether the target vehicle will move backward on an uphill road via a computer vision module. In one embodiment of the system, the vehicle is operable to detect that the target vehicle is moving backward on an uphill road via a computer vision module including a radar. In one embodiment of the system, the vehicle is operable to discover vehicles behind the vehicle via a rearview camera. In one embodiment of the system, the vehicle discovers vehicles in other lanes via LiDAR and radar.

[0360] In one embodiment, there are several sensors that can be used by an autonomous vehicle to detect uphill roads. (i) A Global Positioning System (GPS) sensor can be used to detect changes in altitude by measuring the height of the vehicle. This information can be used to determine if the vehicle is traveling on an uphill road. (ii) An Inertial Measurement Unit (IMU) sensor measures the acceleration and rotation speed of the vehicle. By analyzing the data from the IMU, the autonomous vehicle can detect changes in elevation and determine if it is traveling on an uphill road. (iii) A Light Detection and Ranging (LiDAR) sensor emits laser pulses and measures the time it takes for the pulses to reflect back. Using this data, the LiDAR sensor can create a detailed 3D map of the road and detect changes in elevation, such as uphill roads. (iv) A radar sensor uses radio waves to detect the distance and speed of an object. By analyzing the data from the radar sensor, the autonomous vehicle can detect changes in elevation and determine if it is traveling on an uphill road. (v) A camera can be used to detect changes in the road surface, such as the uphill grade. By analyzing the images from the camera, the autonomous vehicle can detect changes in elevation and adjust its driving behavior accordingly.

[0361] In one embodiment, an autonomous vehicle can use a combination of sensors to detect uphill roads and adjust its driving behavior to maintain a safe distance from other vehicles.

[0362] In one embodiment, the autonomous vehicle can use the roof area of ​​other vehicles to detect approaching uphill roads.

[0363] Figure 4HFIG. 1 shows an autonomous vehicle according to one embodiment associating a roof area with a road slope based on vehicle type. In one embodiment, the autonomous vehicle can determine how steep an uphill road may be based on the model information of the target vehicle and the detected roof area. In one example, for similarly sloped roads, such as Figure 4H The roof area detected for the SUV as shown in (A) can be larger than Figure 4H (B) shows the detected roof area for a small vehicle. For the same inclination, different roof areas are detected due to the size of the vehicle. This means that for a similar detected roof area, it may mean a "very inclined" road for a small car, but it may mean "not so inclined" for a long SUV, as shown in Figure 4H As shown in (C). Figure 4H The roof areas of both vehicles in (C) are the same, however, based on the car model, the autonomous vehicle will expect a certain roof area for each vehicle type, and therefore, the uphill classification will be different.

[0364] Fig. 4I An autonomous vehicle according to one embodiment is shown determining the road inclination based on image analysis of the road ahead. In one embodiment, the autonomous vehicle can use the image of the road to detect an approaching uphill road. Fig. 4I , while analyzing the image, if the computer vision system finds no visible sky or very little visible sky, and a portion of the road is visible at the horizon, as shown in (A), then the computer vision system's algorithm can conclude that the road is very inclined. The algorithm can also mark the beginning and end of the uphill portion. If the computer vision system finds less visible sky at the horizon, as shown in (A), then the computer vision system's algorithm can conclude that the road is very inclined. Fig. 4I If the computer vision system finds more visible sky at the horizon, as shown in (B), the algorithm of the computer vision system can conclude that the road is not very inclined. The algorithm can also mark the beginning and end of the uphill portion. Fig. 4I As shown in (C), the algorithm of the computer vision system can conclude that the road is not tilted or is tilted very slightly. In one embodiment, the autonomous vehicle can also determine the downhill portion of the road by observing lanes or road signs to determine that the road suddenly disappears and then suddenly appears.

[0365] Figure 4J FIG. 1 shows an autonomous vehicle determining a road slope according to one embodiment. Based on the degree of inclination of the uphill road, the autonomous vehicle may determine a safe distance and will travel at a different speed. For example, when the autonomous vehicle determines that the uphill road inclination is small, it may not increase the safe distance or modify its speed, such as Figure 4JWhen the autonomous vehicle determines that the uphill road slope is very steep, it will maintain a larger distance and will go uphill at a faster speed to increase momentum, as shown in (A). Figure 4J As shown in (B).

[0366] Figure 4K An autonomous vehicle according to one embodiment is shown determining the start of an uphill road. According to one embodiment, the autonomous vehicle will detect where the first change in the sequence occurs, i.e., the change in the roof area of ​​the target vehicle, and the distance to the target vehicle at the location where the autonomous vehicle detects the change in roof area. The control system of the autonomous vehicle can then mark the situation as "uphill detected". Figure 4K , the point marked A shows the uphill starting point detected by the autonomous vehicle, and the distance at which it detects the uphill starting point is 100 meters.

[0367] Figure 4L An autonomous vehicle according to one embodiment is shown determining a road surface condition for an uphill road. In one embodiment, sensors available in the vehicle are used to obtain information about the grip of the road surface. In one embodiment, the autonomous vehicle adjusts the functionality of the advanced driver assistance system to adapt to the determined road surface condition. For example, automatic emergency braking must be initiated on a wet road much earlier than on a dry road to avoid an imminent collision. In one embodiment, the coefficient of friction (interaction between the tire and the road surface) is measured via the force applied by the tire to the road surface. This may also be referred to as grip. In one embodiment, the system identifies and classifies the road surface condition as dry, wet, snowy, and icy. The road surface condition is classified based on analysis of camera images of the surrounding environment in front of the autonomous vehicle and comparing it with vehicle dynamics data from the electronic stability control (ESC), including but not limited to local and regional weather data, including but not limited to temperature, wiper activity, cloud data, and tire behavior. The coefficient of friction can be derived using data from the electronic stability control (ESC), including but not limited to local and regional weather data, including but not limited to temperature, wiper activity, cloud data, and tire behavior. The camera can be a monochrome camera. A monochrome camera is short for a black and white camera. This is a camera that captures black and white or gray images. Unlike color cameras that use filter arrays or other methods to capture full-color images, monochrome cameras typically have a single image sensor that only records the brightness values ​​of the captured scene. This makes it very useful for applications such as scientific imaging, surveillance, and machine vision, where color information may not be needed or even distracting. In one embodiment, the system can also look for signs 452 or road infrastructure in the autonomous vehicle environment to determine the surface condition of the road.

[0368] Figure 4MAn autonomous vehicle according to one embodiment is shown determining the road surface condition based on a sign. The autonomous vehicle may also rely on available infrastructure to detect icy roads. Computer vision techniques are used to process images or video streams captured by a camera mounted on the vehicle. These techniques can identify the shape and color of the sign, as well as any text or symbols on it. For example, object detection algorithms can be used to locate and identify the sign in an image, while optical character recognition (OCR) techniques can be used to extract text from the image. Machine learning algorithms are then used to classify the sign according to its shape, color, and content. In addition to cameras, autonomous vehicles may also use other sensors such as LiDAR or radar to detect the sign. Light detection and ranging sensors use lasers to measure distances and create a 3D map of the environment, which can help identify the location and orientation of the sign. Radar sensors can detect the presence of an object and measure its speed, which can help detect signs that may be obscured by other objects or in poor lighting conditions. Once the sign is detected and classified, the autonomous vehicle can use this information to classify the road surface condition and make decisions about how to navigate the road ahead. For example, it can adjust its speed, safety distance value, and / or route based on the determined road surface condition.

[0369] Machine learning algorithms are used to classify signs based on their shape, color, and content. For example, a machine learning algorithm can be trained with a large dataset of signs to recognize common features, such as the shape and color of a stop sign or the text on a sign. Once the algorithm has been trained, it can be used to recognize new signs based on their features. In addition to computer vision and machine learning techniques, autonomous vehicles can also use natural language processing (NLP) algorithms to understand the meaning of text on signs. NLP algorithms can analyze text to determine the intended information, which can be used to decide how to navigate the road ahead. For example, when a vehicle detects icy or slippery road conditions, the vehicle can slow down, increase a safe distance, stop, or move with extra caution.

[0370] Figure 4N An autonomous vehicle providing a warning to its occupants about road surface conditions according to one embodiment is shown. The autonomous vehicle will detect if there is a driver / occupant in the autonomous vehicle, and if so, the autonomous vehicle will notify the driver that the autonomous vehicle is approaching an uphill slope. The notification can also include the slope category, the length of the sloped road, the road surface conditions. In one embodiment, the driver / occupant is made aware of the approaching road conditions, and if the driver likes, she / he can decide to take over the steering wheel.

[0371] In one embodiment, the autonomous vehicle will detect the number of lanes on an uphill slope. Fig.4OA block diagram of a lane detection system according to one embodiment is shown. A lane detection system for a vehicle includes a camera disposed at the vehicle so as to have a field of view in front of the vehicle. The camera captures image data. A non-vision-based sensor is disposed at the vehicle so as to have a sensing field in front of the vehicle. The non-vision-based sensor captures sensor data. A controller includes at least one processor operable to process image data captured by the camera and sensor data captured by the non-vision-based sensor. The controller detects visible lane markings drawn on a road on which the vehicle is traveling in response to processing the captured image data. In response to processing the captured sensor data, the controller detects road embedded elements disposed along the road. Based on the detected lane markings or the detected road embedded elements, the controller determines at least the lane in which the vehicle is traveling. Fig.4O A block diagram of a lane detection system according to one embodiment is shown. Lane marking data from a camera (vision-based sensor) is fused in a sensor fusion module with lane marking and object position data using non-vision sensors such as LiDAR, radar, ultrasonic sensors, etc. When lane markings are lost or covered by snow or in heavy rain at night, the system is still able to detect reliable lane positions using the output of non-vision-based sensors. Typically, an autonomous vehicle will be equipped with a set of sensors, including multiple machine vision cameras deployed at the front, sides, and rear of the vehicle, multiple radar sensors deployed at the front, sides, and rear of the vehicle, and / or multiple light detection and ranging sensors deployed at the front, sides, and rear of the vehicle. All sensors can be used to detect lanes based on regions of interest. In one embodiment, the system can use image data captured by a camera as a way to detect lane markings and determine lanes of a road (when lane markings are exposed to the camera and can be viewed by the camera), and in this case, non-vision-based sensor data (when the camera is able to detect lane markings) may or may not be used. When it is determined that the lane marking is worn or obscured or not visible, the system may utilize sensor data sensed by a non-vision based sensor to detect lane objects or elements disposed on or in the roadway.

[0372] According to one embodiment of the system, the lanes beside the host vehicle are detected via a computer vision module including radar, LiDAR, camera, GPS, and an artificial intelligence engine including a machine learning algorithm.

[0373] Sensor fusion is the process of combining data from multiple sensors to improve the accuracy, reliability, and efficiency of the information collected. It involves integrating information from different sources, such as cameras, radar, LiDAR, and other sensors to obtain a more complete and accurate picture of the environment. Sensor fusion may be able to reduce the errors and uncertainties that may be generated by using a single sensor and obtain a more comprehensive understanding of the world around us. By combining data from multiple sensors, autonomous vehicle systems are able to make more intelligent decisions and respond to changing conditions in real time. The process of sensor fusion generally involves several steps, including data acquisition, signal processing, feature extraction, data association, and estimation. Depending on the application and specific sensors used, different algorithms and techniques can be used to integrate information from multiple sensors.

[0374] According to one embodiment, AI-based lane detection / object can be used in combination with sensor fusion technology. Some algorithms suitable for lane detection include, but are not limited to: (i) Hough Transform: The Hough Transform algorithm is a classic method for lane detection that works by identifying straight lines in an image. The Hough Transform can be used to detect the edges of lane markings and estimate the position and orientation of lanes. (ii) Convolutional Neural Network (CNN): CNN is a deep learning algorithm that has been widely used for lane detection. The neural network is trained on lane data obtained from various sensors to distinguish the characteristics of lane markings, such as their color, shape, and texture. Once the network is trained, it can be used to detect and classify lanes in real time. (iii) Support Vector Machine (SVM): SVM is a machine learning algorithm that works by learning the boundaries that separate lane markings from the rest of the image. The SVM algorithm can be trained using labeled data to create a model. The model is then used to detect the position and orientation of the lane. (iv) Markov Random Field (MRF): MRF is a probabilistic graphical model that identifies the position and orientation of lane markings by modeling the relationship between adjacent pixels in an image. MRF can be combined with other algorithms such as CNN or SVM to create a more robust and accurate lane detection system.

[0375] The system may provide an output indicating one or more lanes determined (such as determined based on the processing of captured image data or the processing of captured sensor data). The output may be used to determine whether other vehicles are allowed to turn into the lane, whether the lane is narrow enough or wide enough to make a decision on the clearance for turning vehicles. The output is provided to the vehicle's driver assistance system to provide a warning or exercise attention when other vehicles turn into those identified lanes. Optionally, the output may be provided to an autonomous vehicle control system, whereby the vehicle is autonomously controlled to follow the action to maintain the lane, change lanes or reverse, etc., to avoid any collision from other vehicles. For autonomous vehicles, in certain circumstances, it may be desirable or required that the occupants of the vehicle take over the operation / control of the vehicle and drive the vehicle so that potential dangers can be avoided as long as the autonomous system relinquishes such control or driving. Such a vehicle occupant thus becomes the driver of the autonomous vehicle.

[0376] In one embodiment, the autonomous vehicle may perform a lane change and / or maintain a greater distance when the autonomous vehicle detects at least two lanes in the same direction in which it is traveling. The autonomous vehicle may perform a lane change in the event that the target vehicle (the vehicle in front of the autonomous vehicle) accidentally slides backward. However, the autonomous vehicle may perform a lane change if it is able to perform a lane change before starting the uphill segment. If the autonomous vehicle finds other vehicles traveling in two / all detected lanes, it may not make a difference if the autonomous vehicle performs a lane change. In one embodiment, the autonomous vehicle may not be able to perform a lane change due to other vehicles occupying the lane, in which case the autonomous vehicle may increase the safety distance from the other vehicles. If there is only one lane, or if all detected lanes are occupied, the autonomous vehicle may maintain a greater safety distance depending on the inclination of the uphill segment.

[0377] Figure 4P An autonomous vehicle is shown detecting the end of an uphill road according to one embodiment. The autonomous vehicle can detect the length of the uphill by detecting the end segment of the uphill using a roof area detection method. In one embodiment, it can use vehicle 442 in front of target vehicle 440. The autonomous vehicle will detect a point where the roof area of ​​vehicle 442 is no longer detected. The autonomous vehicle can interpret the point shown as point B as the end of the uphill. The length of the uphill road segment is calculated from point A and point B. The detection of point A at Figure 4K Detailed description in.

[0378] In one embodiment, the autonomous vehicle can also verify via GPS whether an uphill section of the road is straight or has curvature. In the case of curvature, the length of the uphill section can be labeled as "uncertain" or "inaccurate", and the autonomous vehicle can be further cautious when driving on such an uphill section of the road.

[0379] Figure 4Q An autonomous vehicle is shown classifying an uphill road according to one embodiment. In one embodiment, the autonomous vehicle may classify the uphill road into at least two categories. In one embodiment, there may be more than two categories. The qualitative classification is based on a quantitative assessment of the uphill segment. In one embodiment, the uphill road may be classified as "very inclined and short", and the autonomous vehicle may decide to wait until the target vehicle 440 passes the uphill segment, such as Figure 4Q In one embodiment, an uphill road may be classified as "very steep and long", and the autonomous vehicle may decide to maintain a greater distance from the target vehicle 440 before starting the uphill segment, as shown in FIG. Figure 4Q As shown in (B).

[0380] Figure 4R 4 shows the speed of the target vehicle detected in real time by the autonomous vehicle according to one embodiment. In one embodiment, the autonomous vehicle can detect the real-time speed of the target vehicle 440. When the autonomous vehicle is traveling on an uphill section, it may follow the rule of "do not increase power (or stop) when going uphill".

[0381] In one embodiment, the autonomous vehicle can detect whether the target vehicle 440 is losing traction / speed. The autonomous vehicle can detect the possibility of a collision, i.e., whether the target vehicle may move backwards and thus hit the autonomous vehicle.

[0382] Figure 4S The autonomous vehicle is shown deciding on a collision avoidance action based on the speed of a target vehicle according to one embodiment. In one embodiment, the autonomous vehicle detects that the target vehicle speed reaches 0 km / h, which means that the target vehicle may start to reverse at some point. The autonomous vehicle may then decide to perform a lane change if possible and allowed based on the number of lanes and the traffic in the lanes.

[0383] In one embodiment, the autonomous vehicle can detect the deceleration of the vehicle. It can predict at what point the target vehicle may reach zero velocity and can predict the target vehicle's backward movement.

[0384] In one embodiment, the autonomous vehicle may detect that target vehicle 440 is backing up, and it may use V2V to notify other oncoming vehicles so that the oncoming vehicles may also perform a lane change, or the oncoming vehicles may not take the uphill section until the situation is clear.

[0385] In one embodiment, the autonomous vehicle can detect that a lane change is not possible and the autonomous vehicle will just "accept" the collision and will brake. However, the notification will still be sent to other vehicles around the autonomous vehicle to prevent a pileup.

[0386] Additionally, the autonomous vehicle can submit an “icy uphill” notification to the server to alert upcoming vehicles in the path.

[0387] In one embodiment, if a collision event is not predicted, the autonomous vehicle may continue driving until the uphill slope is cleared.

[0388] The system or method is described as some examples, but they are not restrictive. The concept is not limited to these examples. Sometimes an algorithm is proposed to show that the step is possible. But of course, it is not limited to the specific object detection, lane detection or road surface condition detection algorithm proposed. These steps are also not necessarily in a strict order, so they can be interchangeable, or they can be processed in parallel.

[0389] In one embodiment, the autonomous vehicle detects the type of vehicle in front of it to assess whether the road section is uphill. If the autonomous vehicle further detects that the target vehicle, which is the vehicle in front of the autonomous vehicle, is reversing, the autonomous vehicle can brake immediately. If the detected target vehicle type is a large vehicle, such as an SUV, the autonomous vehicle can predict that although the autonomous vehicle is stopped due to the target vehicle being a large vehicle, the autonomous vehicle will also reverse after the collision. The autonomous vehicle will then begin to notify the vehicles behind the autonomous vehicle. In one embodiment, the autonomous vehicle can broadcast "A large vehicle will hit in two seconds." This message can be based on vehicle-to-vehicle communication. In one embodiment, the autonomous vehicle can display a warning in the infotainment system of the autonomous vehicle, such as "Be prepared for a collision in two seconds."

[0390] In one embodiment, the system performs icy uphill detection and increases the safety distance between the autonomous vehicle and other vehicles.

[0391] In one embodiment, the system performs corrective action to avoid the accident, ie, a collision avoidance maneuver.

[0392] In one embodiment, in the event that the autonomous vehicle predicts that an impact is about to occur based on an analysis of its surroundings, the autonomous vehicle can activate a camera so that it captures what is happening around and with the autonomous vehicle during the impact. In one embodiment, the video can be analyzed later to detect possible steps that could minimize the impact, avoid the impact, etc., for future scenarios. In one embodiment, the video can be analyzed to find out which vehicle may be at fault as evidence to provide to authorities. In one embodiment, the autonomous vehicle can ask the driver to intervene and take corrective action, or at least let the driver take over. In one embodiment, if there is a way for the driver to move away from the vehicle and let the vehicle withstand the impact, the driver can decide to do so.

[0393] In one embodiment, if the autonomous vehicle is about to be hit, the autonomous vehicle can notify the user on the screen or phone, it can make a warning noise or display flashing lights, display a flash message to attract the attention of surrounding vehicles about the impending collision.

[0394] In one embodiment, the autonomous vehicle may determine that the impact will be 5 miles per hour and will not impact the autonomous vehicle too severely. In one embodiment, if the autonomous vehicle detects that the target vehicle is sliding downward at 10-15 miles per hour and a severe impact may occur, it will actually let the occupants know that the airbags may be deployed. In one embodiment, the impact analysis also takes into account the size of the target vehicle, whether it is small or large.

[0395] In one embodiment, an autonomous vehicle may be sliding backward on an icy uphill section of road and is publishing information to other vehicles around the autonomous vehicle. In one embodiment, the autonomous vehicle may have a rear display to alert oncoming vehicles of the situation. The information may be received by a non-autonomous vehicle via wireless communication or vehicle-to-vehicle communication. In one embodiment, the information may be relayed via an internet connection based on the location of the other vehicles.

[0396] In one embodiment, the means of communication with other vehicles can be vehicle-to-vehicle communication via a broadcast message from the autonomous vehicle. In one embodiment, if other vehicles are not autonomous or do not have vehicle-to-vehicle communication enabled, they can get an alert on a phone within a short range. In one embodiment, the alert can be via flashing of the autonomous vehicle's lights, honking of the autonomous vehicle's horn, a display on the back of the autonomous vehicle that will help inform the autonomous vehicle or a target vehicle observed by the autonomous vehicle is traveling backwards and creating a possible collision situation.

[0397] In one embodiment, the autonomous vehicle displays instructions and warnings via the instrument panel that the driver or passenger can follow. If the vehicle is not an autonomous vehicle or the driver is in control of the autonomous vehicle, the driver can follow the displayed instructions, which will guide the vehicle to escape a potential collision situation.

[0398] In one embodiment, the traffic situation may include autonomous vehicles, high-tech vehicles, and low-tech vehicles.

[0399] In one embodiment, a vehicle may receive a warning message that the vehicle is sliding downhill. The receiving vehicle will take corrective action to avoid a collision situation. The warning message may include an action to avoid a collision situation.

[0400] In one embodiment, it is a method, comprising: determining, by a control module of an autonomous vehicle, freezing weather conditions by at least one of the following: receiving real-time weather broadcasts; determining icy roads in a route; detecting freezing weather conditions in real time; determining that neighboring vehicles have navigation difficulties; receiving broadcast messages about freezing weather conditions from surrounding vehicles; upon determining that the autonomous vehicle is in freezing weather conditions, the control module of the autonomous vehicle performs: adjusting a safety distance value to a value higher than that in non-freezing weather conditions; determining a new route with a minimum number of icy roads; detecting in real time the visible roof area, model, and speed of a vehicle in front of the autonomous vehicle; determining an uphill road and an inclination of the uphill road based on the visible roof area and model of the vehicle; determining a speed change of a vehicle in front of the autonomous vehicle; upon determining that the speed change of the vehicle in front of the autonomous vehicle tends to zero, the control module of the autonomous vehicle performs at least one of the following: generating a warning in the autonomous vehicle; maintaining at least a safety distance value between the autonomous vehicle and the vehicle in front of the autonomous vehicle; initiating a reverse action of the autonomous vehicle; initiating a lane change action of the autonomous vehicle; broadcasting messages to neighboring vehicles; and wherein the method is operable to detect and avoid accidents in freezing weather conditions.

[0401] In one embodiment, it is a method, including: starting an autonomous vehicle by a user; scanning a weather channel by the user to determine road conditions on a path; after understanding the road conditions, changing the route to a new path to avoid icy road conditions; when on icy road conditions, adjusting the safety distance from other vehicles to a value higher than that for non-icy road conditions; detecting an uphill road; when on an icy uphill road, further adjusting the safety distance to a value higher than that for icy road conditions; when driving on an icy road, detecting a vehicle in front of the autonomous vehicle and detecting a speed change of the vehicle; when the speed change of the vehicle tends to zero, performing at least one of the following: generating a warning in the autonomous vehicle; raising the priority of multiple sensors of the autonomous vehicle to handle icy weather conditions; after determining that there is sufficient free distance behind the autonomous vehicle, initiating a reverse action of the autonomous vehicle; determining that there is a free spot to move to a side lane; initiating a lane change action of the autonomous vehicle to the side lane; broadcasting a message about at least one of the lane change and the reverse movement of the neighboring vehicle to neighboring vehicles for collective and coordinated actions; and wherein the method is operable to detect and avoid accidents in icy weather conditions.

[0402] Figure 5A method for avoiding collision on an icy uphill section in the vehicle according to one embodiment is shown. It includes detecting the road surface condition by the vehicle in step 502. The vehicle can use a computer vision module to detect the road surface condition. It can classify the road surface condition as dry, wet, snowy or icy. Once any condition other than a dry condition is detected, the method can increase the safety distance value of the vehicle at step 504 compared to the safety distance value of a normal dry condition. In addition, at step 506, the vehicle can continue to detect whether the vehicle is approaching an uphill section. If an uphill section is detected, the method can further increase the safety distance value of the vehicle to a higher value than step 504 at step 508. At step 510, the method continues to detect the target vehicle and its speed. If the target vehicle speed is detected to be approaching zero, then at step 512, the method of the vehicle can predict that the target vehicle may slide down the uphill road and predict the possibility of a collision. Then, the method of the vehicle will determine the collision avoidance action at step 514. It can be based on observing the traffic around the vehicle. For example, the collision avoidance action may include, at step 514-1, conveying to nearby vehicles a message that the target vehicle is sliding downward, and requesting them to change lanes or switch to reverse for coordinated action in order to avoid a collision or reduce the impact of a collision. In another variation, if no traffic is observed around the vehicle, then at step 514-2, the vehicle itself may perform a collision avoidance action by changing lanes or by reverse in the same lane. In another variation, when the vehicle predicts that a collision is inevitable, at step 514-3, a warning message may be generated within the vehicle to remind the driver to prepare for the collision. Any one or a combination of steps 514-1, 514-2, and 514-3 may be performed, which may be suitable for the scenario analyzed by the vehicle.

[0403] According to one embodiment of the system, the safety distance value is set to a higher value than in dry road surface conditions. According to one embodiment of the system, the system is further operable to determine a new route having a minimum number of icy roads.

[0404] According to one embodiment of the system, the vehicle type is detected by detecting the license plate number of the target vehicle via a computer vision module. According to one embodiment of the system, the license plate number is matched to the vehicle type by referring to a database.

[0405] According to one embodiment of the system, the host vehicle is also operable to detect the vehicle type and vehicle model of the target vehicle. According to one embodiment of the system, the speed of the target vehicle is determined in real time via a computer vision module including a camera, LiDAR, GPS and radar.

[0406] According to one embodiment of the system, the computer vision module further comprises an artificial intelligence engine and a machine learning algorithm. According to one embodiment of the system, the speed of the target vehicle is determined in real time via radar.

[0407] According to one embodiment of the system, a bounding box algorithm is used to detect the visible roof area of ​​the target vehicle. According to one embodiment of the system, the bounding box algorithm is used in conjunction with a machine learning algorithm for object detection, wherein the machine learning algorithm is a convolutional neural network (CNN).

[0408] Figure 6 Shown is an example of a vehicle reminding nearby vehicles about a target vehicle sliding on an uphill section. After sensing that the target vehicle 640 in front of the vehicle 638 starts to slide, the vehicle 638 immediately reminds nearby vehicles 642 and 643 via visual warnings, sound warnings and via a message conveyed. The message may include a request for a collision avoidance action to the nearby vehicle, which may be a lane change or a reverse. It can remind them so that the nearby vehicle can take appropriate action to avoid a chain collision accident due to the warning provided by the vehicle. If there is a possibility that a possible collision may also affect the vehicle 643 in the opposite lane, the vehicle in the opposite lane 643 may also be reminded. The nearby vehicle 642 may not even go uphill when receiving the warning until the uphill section is unblocked, or may reverse so that it may not be involved in a collision, thereby reducing the accumulation (pile up) of the collision.

[0409] Figure 7 An example message of a dedicated short range communication (DSRC) message format is shown. Figure 7Shown are: (i) Basic Safety Message (BSM) format: BSM is a basic message type in DSRC, used to transmit information about the status, location, and movement of a vehicle to other vehicles in the area. In the example message, the header contains the message ID and protocol version, while the data segment contains specific information about the vehicle, such as its ID, speed, heading, and positioning. The timestamp indicates when the message was generated and can be used by other vehicles to determine the current status of the transmitting vehicle; (ii) Signal Phase and Timing (SPaT) format: SPaT messages provide information about the current state of traffic signals at intersections, including which phase is currently valid and when the signal will change. Vehicles can use this information to optimize their driving behavior and avoid collisions. In the example message, the header contains the message ID and protocol version, while the data segment includes the ID of the intersection, the state of the motion phase, and the remaining time until the next and subsequent phases; (iii) Roadside Alert (RSA) format: RSA messages are used to notify vehicles about dangers or other important information on the road ahead, such as construction zones, accidents, or detours. In the example message, the header contains the message ID and protocol version, while the data segment includes the alert type (construction), the distance to the alert, the direction of the alert, and a text message providing additional details about the alert.

[0410] According to one embodiment, there are other possible elements that can be included in each message format, for example, the BSM format may also include information about acceleration, size and weight, vehicle safety extensions including brakes, lights, wipers, transmission and powertrain, vehicle dimensions, and vehicle identity information including ID number and vehicle type.

[0411] In addition to the three examples provided previously, there are several other message formats available in the DSRC protocol. Some of these message formats include the MAP (Message Assistance for Path Provisioning) format: This message provides a map of the road network, including information about road layout, speed limits, and other relevant details; CAM (Cooperative Awareness Message) format: Similar to the BSM message format, this message provides information about the vehicle's current state, such as speed, location, and heading. CAM messages are usually sent at a higher frequency than BSM messages to support more accurate situational awareness; DENM (Decentralized Environmental Notification Message) format: This message provides information about environmental conditions that may affect driving, such as weather conditions, visibility, road surface conditions, etc.; EV (Emergency Vehicle) format: This message provides information about emergency vehicles, such as their location and direction of travel, to help other drivers safely navigate around them; TCI (Traffic Control Information) format: This message provides information about traffic conditions, such as congestion, accidents, and construction zones, to help drivers make informed decisions about their routes.

[0412] Overall, the various message formats available in the DSRC protocol support a wide range of use cases, from basic vehicle-to-vehicle communications to more complex applications such as real-time traffic management and emergency response.

[0413] Fig. 8A An example message from the host vehicle to a nearby / neighboring vehicle is shown according to one embodiment. It may include elements related to a prediction of an icy uphill section and a request message, wherein the request message may request at least one of a lane change, a back-up, so that a coordinated action can be performed. If a collision is imminent, it may also include a time to collision, otherwise it may not include a time to a possible collision. In one embodiment, the message can be a broadcast message, or a message sent to a single neighboring vehicle or a group of selected vehicles that need to execute the request. In one embodiment, the request may be for avoiding a predicted collision of a neighboring or nearby vehicle.

[0414] Figure 8B An example message from a target vehicle to other vehicles according to one embodiment is shown. In the case where the target vehicle is not an autonomous vehicle, the signal can be communicated via V2V. The host vehicle, which is an autonomous vehicle, can then further transmit the message to all nearby vehicles. In one embodiment, it can be the content of the message when the host vehicle slides or may hit other vehicles due to an icy uphill section. In one embodiment, the message may include information related to the type of vehicle, the impact speed, the time from the collision, etc. It may also include a request message to change lanes or reverse, or to keep a safe distance to avoid a collision, or a message to prepare for a collision.

[0415] Figure 8C An example warning message displayed in the host vehicle according to one embodiment is shown. In one embodiment, it may include a warning message and a graphical representation to show that the host vehicle 801 is predicting a collision due to a target vehicle 802 on an uphill section of road. It may broadcast the message to other vehicles 803 and 805 to clear the impact zone. In one embodiment, the message may also show the start and end of the uphill section of the road and the location of each vehicle.

[0416] Fig.9AThe structure of a neural network / machine learning model with a feedback loop is shown. The artificial neural network (ANN) model includes an input layer, one or more hidden layers, and an output layer. Each node or artificial neuron is connected to another node or artificial neuron and has an associated weight and threshold. If the output of any single node is above a specified threshold, the node is activated and the data is sent to the next layer of the network. Otherwise, no data is passed to the next layer of the network. A machine learning model or ANN model can be trained on a set of data to accept a request in the form of input data, predict the input data, and then provide a response. The model can learn from the data. Learning can be supervised learning and / or unsupervised learning, and can be based on different scenarios and different data sets. Supervised learning includes the logic of using at least one of a decision tree, logistic regression, and a support vector machine. Unsupervised learning includes the logic of using at least one of a k-means clustering, a hierarchical clustering, a hidden Markov model, and an association rule algorithm (apriori algorithm). The output layer can predict or detect the severity of a collision and a collision based on input data such as target vehicle parameters, host vehicle and target vehicle position and orientation.

[0417] In one embodiment, the ANN may be a deep neural network (DNN), which is a multilayer tandem neural network including an artificial neural network (ANN), a convolutional neural network (CNN), and a recurrent neural network (RNN), which is capable of identifying features from inputs, conducting expert reviews, and performing actions that require prediction, creative thinking, and analysis. In one embodiment, the ANN may be a recurrent neural network (RNN), which is a type of artificial neural network (ANN) that uses sequential data or time series data. Deep learning algorithms are typically used for ordinal or temporal problems, such as language translation, natural language processing (NLP), speech recognition, and image recognition. Like feedforward and convolutional neural networks (CNNs), recurrent neural networks learn using training data. Their difference lies in their "memory" because they obtain information from previous inputs via feedback loops to affect current inputs and outputs. The output from the output layer in the neural network model is fed back to the model through feedback. When training the model, the weight changes in the hidden layer will be adjusted to better fit the expected output. This will allow the model to provide results with far fewer errors.

[0418] A neural network features a feedback loop so that it can dynamically adjust the system output as it learns from new data. In machine learning, backpropagation and feedback loops are used to train an AI model and continuously improve it as it is used. As more incoming data is received by the model, the model has more opportunities to learn from the data. The feedback loop or backpropagation algorithm identifies inconsistencies and feeds the corrected information back into the model as input.

[0419] Even if an AI / ML model is trained very well using large labeled datasets and concept sets, after a while, the performance of the model may degrade when new, unlabeled inputs are added due to many reasons, including but not limited to concept drift, loss of recall precision due to deviations from true positives, and data drift over time. The model’s feedback loop maintains the accuracy of the AI ​​results and ensures that the model maintains its performance and improves even as new unlabeled data is ingested. A feedback loop is the process by which the predicted output of an AI model is reused to train new versions of the model.

[0420] Initially, when training an AI / ML model, some labeled samples including positive and negative examples of concepts (e.g., vehicle types, various uphill sections, roof areas of target vehicles, etc.) are used for model learning. Afterwards, the model is tested using unlabeled data. By using, for example, deep learning and neural networks, the model is then able to predict whether the desired concepts (e.g., the inclination of uphill roads for the detected roof area of ​​a given vehicle type, classification of road signs, road surface conditions, etc.) are in the unlabeled images. Each image is given a probability score, with higher scores representing higher confidence in the model's prediction. When the model gives an image a high probability score, it is automatically labeled with the predicted concept. However, in the case where the model returns a low probability score, the input may be sent to a controller (which may be a human moderator) who verifies and, if necessary, corrects the result. Human moderators can only be used in exceptional cases. The feedback loop dynamically feeds the labeled data, either automatically labeled or verified by the controller, back to the model and uses it as training data so that the system can improve its predictions in real time and dynamically.

[0421] Fig. 9BThe structure of a neural network / machine learning model with reinforcement learning is shown. The network receives feedback from an authorized networked environment. While this system is similar to supervised learning, the feedback obtained in this case is evaluative rather than instructive, meaning there is no teacher as in supervised learning. After receiving the feedback, the network performs adjustments to the weights to obtain better predictions in the future. Machine learning techniques, such as deep learning, allow the model to take labeled training data and learn to recognize those concepts in subsequent data and images. New data can be fed into the model for testing, so training is reinforced by feeding the model with data that has already been predicted. If the machine learning model has a feedback loop, learning is further reinforced by rewards for each positive example output by the system. Feedback loops ensure that the AI ​​results do not stagnate. By introducing feedback loops, the model outputs are continuously and dynamically improved with use / time.

[0422] Fig. 9C An example block diagram for detecting collision events on icy uphill roads and / or maintaining a safe distance from the host vehicle during icy uphill sections using a machine learning model is shown. The machine learning model 902 can take any data associated with the host vehicle and the target vehicle as input, and learn to identify the features of the predicted collision event within the data. The training data sample may include, for example, target vehicle information 904, such as size, dimensions, vehicle type, vehicle model, weight, length, etc. In one embodiment, the present invention relates to a system and method for real-time identification of target vehicles and their features using on-board cameras and / or other sensors. The vehicle identity and feature information are transmitted to the cloud, where the vehicle identity is associated with the available turning radius and trajectory prediction in the case of previously identified similar categories of vehicles, and a similar feature is constructed. Subsequently, this information is used to calculate / estimate the safe distance to be maintained with the host vehicle. The system and method of the present disclosure can also provide data analysis information, which can be used later to improve vehicle safety. When the host vehicle is near an icy uphill section, the sensors and cameras of the host vehicle, etc. can be used to detect target vehicle characteristics and estimate the safe distance to be maintained with the host vehicle.

[0423] In one embodiment, the training data samples may also include current context information 906 related to the surrounding environment. This may include, for example, the location of the host vehicle, the location of the target vehicle, current weather conditions, temperature, time of day, traffic conditions in the area, number of lanes, other obstacles, other vehicles on uphill sections, etc. The system may also collect and store context information 906 from devices associated with the vehicle. For example, via an application installed on the device, such as Map and location services, the system can know the vehicle details.

[0424] Real-time sensor data 908 may include, for example, video, images, audio, infrared, temperature, 3D modeling, and any other suitable type of data that captures the current state of the surroundings of the host vehicle. In one embodiment, the real-time sensor data may be processed using one or more machine learning models 902 that are trained, and the real-time features of the target vehicle may be predicted based on similar types of data. The real-time features may include, for example, the size, shape, dimensions, signal lights, etc. of the target vehicle. Information about the target vehicle may be used to estimate a safe distance to be maintained from the host vehicle. For example, currently detected sensor data and / or previously known information about the class and / or size of the target vehicle may be used to predict a safe distance to be maintained from the host vehicle.

[0425] Any of the aforementioned types of data (e.g., target vehicle data 904, context information 906, sensor data 908, or any other data) may be associated with safe distance calculations, and such associations may be automatically learned by the machine learning model 902. In one embodiment, during training, the machine learning model 902 may process training data samples (e.g., target vehicle data 904 and / or context information 906), and based on the current parameters of the machine learning model 902, detect or predict output 910, which may be a probability of collision and / or a safe distance to be maintained with the host vehicle. The detection or prediction 910 of the probability of collision and / or the safe distance to be maintained with the host vehicle may depend on training data having a label 912 associated with the training data samples 918. Predicting the probability of collision and / or the safe distance to be maintained with the host vehicle refers to predicting future events based on past and present data and most commonly by analyzing, estimating trends or data patterns. Prediction or predictive analysis employs probabilities based on data analysis and processing. Detection of a collision refers to the initiation of an event and the system detecting it. Based on how the transition of the event occurs, the predicted event may or may not turn into a collision. In one embodiment, during training, the events detected at 910 may be compared at 914 to the training data with labels 912. For example, the comparison 914 may be based on a loss function that measures the difference between the detected collision event 910 and the training data with labels 912. Based on the corresponding output of the comparison or loss function at 914, the training algorithm may update the parameters of the machine learning model 902 with the goal of minimizing the difference or loss between subsequent predictions or detections of the event 910 and the corresponding labels 912. By iteratively training in this manner, the machine learning model 902 may "learn" from different training data samples and become better at detecting various collision events similar to the collision event represented by the training labels at 912 at 910. In one embodiment, the machine learning model 902 is trained using data specific to the target type, and the model is used to detect collisions to determine collision avoidance actions. In one embodiment, the machine learning model 902 is trained using data that is common to the vehicle type and is used to detect collision probabilities and therefore determine collision avoidance actions.

[0426] Using the training data, the machine learning model 902 can be trained so that it recognizes features of input data that represent or are related to certain event types. For example, the trained machine learning model 902 can identify data features that represent the possibility of an emergency as an actionable event. Through training, the machine learning model 902 can learn to identify predictive and non-predictive features and apply appropriate weights to the features to optimize the prediction accuracy of the machine learning model 902. In an embodiment where supervised learning is used and each training data sample 918 has a label 912, the training algorithm can iteratively process each training data sample 918 (including target vehicle data 904, background information 906, and / or sensor data 908) and generate a prediction of a collision 910 based on the current parameters of the model 902. Based on the comparison 914 results, the training algorithm can adjust the parameters / configuration (e.g., weights) of the model 902 accordingly to minimize the difference between the generated prediction of the collision event 910 and the corresponding label 912. Any suitable machine learning model and training algorithm can be used, including, for example, neural networks, decision trees, clustering algorithms, and any other suitable machine learning techniques. Once trained, the machine learning model 902 can take input data associated with a target vehicle and output one or more predictions that indicate a probability of a collision based on the host vehicle's position and can suggest a safe position for the host vehicle.

[0427] Fig.10 An example flow chart for detecting a collision with the host vehicle on an icy uphill road using a machine learning model is shown. As shown at 1002, the system can receive data associated with sensor outputs from one or more sensors in the host vehicle. Any type of sensor can be used to collect data related to the target vehicle. The sensor output can be, for example, an image, video, audio, LiDAR measurement, infrared measurement, temperature measurement, GPS data, or any other information of sensor measurement or detection. In one embodiment, the sensor output can be the result of environmental information associated with the environment of the host vehicle captured by one or more sensors, which can include traffic at the location, target vehicle details, number of lanes, traffic around the host vehicle, road surface conditions, etc. The system can receive any data associated with the sensor output from the sensor, including raw sensing output and / or any derived data. In one embodiment, the system can use a machine learning model that has been trained using a training data set to process the received data and identify any action events of interest. It can receive other data 1006 from other sensors of the host vehicle, such as weather conditions, humidity, temperature, driver behavior, tire tread, tire condition, etc.

[0428] As shown at 1004, the system can extract features from the received data according to the machine learning model. The machine learning model can do this automatically based on what it has learned during the training process. In one embodiment, appropriate weights learned during the training process can be applied to these features.

[0429] At step 1008, based on the characteristics of the received data, the machine learning model can generate a score representing the likelihood or confidence that the received data is associated with a particular event type (e.g., a collision when a target vehicle of a particular vehicle type is decelerating, etc.).

[0430] As shown at 1010, the system can determine whether the score is high enough relative to a threshold or standard to warrant certain actions. If the score is not high enough, thus indicating a false positive, the system can return to step 1002 and continue to monitor subsequent incoming data. On the other hand, if the score is high enough, then at step 1012, the system can generate an appropriate alert and / or determine an appropriate action / response. In one embodiment, the system can send an alert to an appropriate recipient based on the type of event detected. For example, an alert is generated in the host vehicle and a message is sent to nearby vehicles.

[0431] In one embodiment, the system may repeat the Fig.10 In one embodiment, steps 1002 to 1012 may be performed by the system, and any combination of those steps may be performed by any other computing system (e.g., a remote network or cloud network). In one embodiment where a machine learning model is used to make such a determination, the system may transmit the trained machine learning model to a computing system in the vehicle to allow event detection to occur locally. This may be desirable because the sensor data may be too large to be transmitted to a remote system in a timely manner.

[0432] In one embodiment, the system is provided, wherein the facial expression recognition module utilizes a convolutional neural network (CNN) pre-training process; and / or wherein the machine learning (ML) algorithm includes a bounding box program around the face of the subject; and / or wherein the bounding box program utilizes a Viola-Jones detection algorithm; and / or wherein the facial recognition module utilizes a facial expression recognition (FER) algorithm. In one embodiment, one or more facial expressions currently obtained from the camera are compared to thresholds of pre-trained emergency classifications to determine when the current classification is an emergency. The use of recurrent neural network architectures comes from its ability to use past temporal information to reason about current inputs. Long short-term memories (LSTMs) provide a computationally efficient method to train these networks. For example, video sequences of different vehicle types and their trajectory information around turns can be used to train the LSTM. With this training mechanism, the model is able to predict given real-time video input when a collision is about to occur given the location of the vehicle. According to one embodiment of the system, the bounding box algorithm is used in conjunction with a machine learning algorithm for object detection, wherein the machine learning algorithm is a convolutional neural network (CNN).

[0433] In one embodiment, the system may include a network security module. In one aspect, a secure communication management (SCM) computer device for providing a secure data connection is provided. The SCM computer device includes a processor in communication with a memory. The processor is programmed to receive a first data message from a first device. The first data message is in a standardized data format. The processor is also programmed to analyze the first data message for potential network security threats. If it is determined that the first data message does not contain a network security threat, the processor is also programmed to convert the first data message into a first data format associated with a vehicle environment, and send the converted first data message into a first data format associated with a vehicle environment, and transmit the converted first data message to the communication module using a first communication protocol associated with a negotiation protocol.

[0434] According to one embodiment, secure authentication for data transmission includes: supplying a hardware-based security engine (HSE) located in a network security module, the HSE having been manufactured in a secure environment and authenticated as part of an approved network in the secure environment; using the HSE to perform asynchronous authentication, verification and encryption of data, storing user permission data and connection status data in an access control list for defining allowed data communication paths of the approved network, enabling the network security module to communicate with the access control list using other computing system entities (e.g., a communication module), using the security engine to perform asynchronous verification and encryption of data, including using a hardware-based module equipped with one or more security aspects for protecting the system to identify a user device (UD) containing credentials in hardware, wherein the security aspects include the hardware-based module communicating with a user of the user device and the HSE.

[0435] Fig.11A A block diagram of a network security module according to one embodiment is shown. In one embodiment, Fig.11A A block diagram of a network security module is shown. Data communications between the system 1100 and the server 1170 via the communication module 1112 are first verified by the information security management module 1132 before being transmitted from the system to the server or from the server to the system. The information security management module is operable to analyze data for potential network security threats, encrypt data when no network security threats are detected, and transmit the encrypted data to the system or server. The system 1100 includes a processor 1108.

[0436] In one embodiment, the network security module further includes an information security management module that provides isolation between the system and the server. Fig. 11B A flow chart of protecting data by the network security module 1130 is shown. In step 1140, the information security management module is operable to receive data from the communication module. In step 1141, the information security management module exchanges security keys at the beginning of communication between the communication module and the server. In step 1142, the information security management module receives the security key from the server. In step 1143, the information security management module authenticates the identity of the server by verifying the security key. In step 1144, the information security management module analyzes the security key for potential network security threats. In step 1145, the information security management module negotiates an encryption key between the communication module and the server. In step 1146, the information security management module receives the encrypted data. In step 1147, when no network security threat is detected, the information security management module transmits the encrypted data to the server.

[0437] In one embodiment, Fig. 11CA flow chart for protecting data through the network security module 1130 is shown. In step 1151, the information security management module is operable to: exchange security keys at the beginning of communication between the communication module and the server. In step 1152, the information security management module receives the security key from the server. In step 1153, the information security management module authenticates the identity of the server by verifying the security key. In step 1154, the information security management module analyzes the security key for potential network security threats. In step 1155, the information security management module negotiates an encryption key between the communication module and the server. In step 1156, the information security management module receives the encrypted data. In step 1157, the information security management module decrypts the encrypted data and performs an integrity check on the decrypted data. In step 1158, when no network security threat is detected, the information security management module transmits the decrypted data to the communication module.

[0438] In one embodiment, the integrity check is a hash signature verification using Secure Hash Algorithm 256 (SHA256) or a similar method. In one embodiment, the information security management module is configured to perform asynchronous authentication and verification of communications between the communication module and the server.

[0439] In one embodiment, the information security management module is configured to issue a warning if a network security threat is detected. In one embodiment, the information security management module is configured to discard the received encrypted data if the integrity check of the encrypted data fails.

[0440] In one embodiment, the information security management module is configured to verify the integrity of the decrypted data by verifying accuracy, consistency, and any possible data loss during communication through the communication module.

[0441] In one embodiment, the server is physically isolated from the system through the information security management module. When the system communicates with the server, such as Fig.11AAs shown, the system and the server are first authenticated. The system is responsible for communicating / exchanging the system's public key and the signature of the public key with the server. The system's public key and the signature of the public key are sent to the information security management module. The information security management module decrypts the signature and verifies whether the decrypted public key is consistent with the received original public key. If the decrypted public key is verified, the authentication is passed. Similarly, the system and the server authenticate the information security management module. After the authentication is passed to the information security management module, the communicating parties, the system, and the server negotiate the encryption key and integrity verification key for the data communication between the communicating parties through the authenticated asymmetric key. During the authentication process, a session ID number is transmitted, so that the key needs to be bound to the session ID number; when the system sends data to the outside, the information security gateway receives the data through the communication module, performs integrity authentication on the data, and then encrypts the data through the negotiated secret key, and finally transmits the data to the server through the communication module. When the information security management module receives data through the communication module, it first decrypts the data and verifies the integrity of the decrypted data. If the verification is passed, the data is sent out through the communication module; otherwise, the data is discarded. In one embodiment, identity authentication is achieved using an asymmetric key with a signature.

[0442] In one embodiment, the signature is implemented by a pair of asymmetric keys trusted by the information security management module and the system, where the private key is used to sign the identities of the two communicating parties, and the public key is used to verify whether the identities of the two communicating parties are signed. The signing identity includes a public key and a private key pair. In other words, the signing identity is called the common name of the certificate installed in the user's machine.

[0443] In one embodiment, both communicating parties need to authenticate their own identities through a pair of asymmetric keys, and the task responsible for communicating with the information security management module of the system is identified by a unique pair of asymmetric keys.

[0444] In one embodiment, the dynamic negotiation key is encrypted by using the Rivest-Shamir-Adleman (RSA) encryption algorithm. RSA is a public key cryptographic system widely used for secure data transmission. The negotiated key includes a data encryption key and a data integrity check key.

[0445] In one embodiment, the data encryption method is a triple data encryption algorithm (3DES) encryption algorithm. The integrity check algorithm is a hash-based message authentication code (HMAC-MD5-128) algorithm. When the data is output, the data is integrity checked and calculated, and the calculated message authentication code (MAC) value is added to the header of the data message of the value, and then the 3DES algorithm is used to encrypt the data (including the MAC of the header), and the header information of the security layer is added after the data is encrypted, and then the data is sent to the next layer for processing. In one embodiment, the next layer refers to the transport layer in the Transmission Control Protocol / Internet Protocol (TCP / IP) model.

[0446] The information security management module ensures the security, reliability and confidentiality of the communication between the system and the server through identity authentication when the communication between the two parties begins data encryption and data integrity authentication. This method is particularly suitable for embedded platforms with fewer resources and not connected to the public key infrastructure (PKI) system, and by ensuring the security and reliability of the communication between the system and the server, it can ensure that the data security on the server will not be damaged by hacker attacks under Internet conditions.

Claims

1. A system, include: Processor, computer vision module and control module; Wherein, the system is operable to: determining, by the processor, a road surface condition, wherein the road surface condition is at least one of ice, wet, and snow; The processor adjusts the safety distance value of the host vehicle based on the road surface condition; Detecting, by the computer vision module, a vehicle type, a speed, and a visible roof area of ​​a target vehicle, wherein the target vehicle is a vehicle in front of the host vehicle; determining, by the processor, that the host vehicle is approaching an uphill road; determining, by the processor, that the target vehicle is decelerating; predicting, by the processor, a collision between the host vehicle and the target vehicle; and The processor determines a collision avoidance action of the host vehicle to avoid a collision with the target vehicle.

2. The system according to claim 1, in, The system is operable as a component of the host vehicle, and the host vehicle is an autonomous vehicle.

3. The system according to claim 1, in, The road surface condition is determined in real time by analyzing the scattering of the emitted light beam on the road surface using filtering techniques on images captured by the computer vision module, and wherein the computer vision module includes a machine learning algorithm.

4. The system according to claim 1, in, The increase in the safety distance value is determined based on the weight of the target vehicle and the vehicle type of the target vehicle.

5. The system according to claim 1, in, The target vehicle's speed is determined in real time via a computer vision module including camera, LiDAR, GPS and radar.

6. The system according to claim 1, in, A bounding box algorithm is used to detect the visible roof area of ​​the target vehicle, and wherein the bounding box algorithm is used in conjunction with a machine learning algorithm, wherein the machine learning algorithm is a convolutional neural network (CNN).

7. The system according to claim 1, in, The inclination of the uphill road is determined based on a vehicle type of the target vehicle and a visible roof area of ​​the target vehicle.

8. The system of claim 1, wherein the collision avoidance action comprises generating an alert in the host vehicle, wherein the alert is at least one of a text message, a visual prompt, an audible alert, a tactile prompt, and a vibration.

9. The system according to claim 1, in, The collision avoidance action includes at least one of: a counter movement initiated by the host vehicle; and a lane change initiated by the host vehicle.

10. The system according to claim 1, in, The collision avoidance action includes broadcasting a message to vehicles behind the host vehicle that the target vehicle is sliding down the uphill road.

11. The system according to claim 1, in, The collision avoidance action includes broadcasting a message to neighboring vehicles, wherein the message includes at least one of: warning the target vehicle of an impending collision; and instructions for maneuvering of the neighboring vehicles to avoid the impending collision using collective and coordinated action.

12. The system according to claim 1, in, The system is further configured to autonomously perform the collision avoidance action by the host vehicle through the control module.

13. A method, include: determining, by a processor of the host vehicle, a road surface condition, wherein the road surface condition is at least one of ice, wet, and snow; adjusting, by the processor, a safety distance value of the host vehicle based on the road surface condition; Detecting a vehicle type, a speed, and a visible roof area of ​​a target vehicle by a computer vision module, wherein the target vehicle is a vehicle in front of the host vehicle; determining, by the processor, that the host vehicle is approaching an uphill road; determining, by the processor, that the target vehicle is decelerating; predicting, by the processor, a collision between the host vehicle and the target vehicle; and A collision avoidance maneuver for the host vehicle is determined by the processor to avoid the collision.

14. The method of claim 13, wherein the road surface condition is determined by at least one of: real-time weather forecasts, a map showing icy roads on a route, identifying road signs, determining that nearby vehicles have navigation difficulties, receiving broadcast messages from nearby vehicles, and real-time analysis of the road surface condition via the computer vision module.

15. The method according to claim 13, in, The uphill road and the inclination of the uphill road are determined based on the GPS position of the target vehicle and map data.

16. The method according to claim 13, in, The host vehicle is operable to determine the inclination of the uphill road by detecting changes in the visible roof area by taking a sequence of consecutive readings timed at predetermined intervals.

17. The method of claim 13, wherein the host vehicle is operable to notify a driver of the host vehicle that an uphill road is approaching via a notification to the driver of the host vehicle, wherein the notification is at least one of an audible, tactile prompt, a visual prompt, and a text message.

18. The method according to claim 13, in, The collision avoidance action includes predicting the collision and alerting the driver of the host vehicle of the deployment of the airbag during the collision, and advising the driver to prepare for the impact posture, and activating the camera of the computer vision module to record the surrounding environment and the host vehicle.

19. A non-transitory computer-readable medium having stored thereon instructions executable by a computer system to perform operations comprising: determining, by a processor of the host vehicle, a road surface condition, wherein the road surface condition is at least one of ice, wet, and snow; The processor adjusts the safety distance value of the host vehicle based on the road surface condition; Detecting a vehicle type, a speed, and a visible roof area of ​​a target vehicle by a computer vision module, wherein the target vehicle is a vehicle in front of the host vehicle; determining, by the processor, that the host vehicle is approaching an uphill road; determining, by the processor, that the target vehicle is decelerating; predicting, by the processor, a collision between the host vehicle and the target vehicle; and A collision avoidance maneuver for the host vehicle is determined by the processor to avoid the collision.

20. The non-transitory computer readable medium of claim 19, wherein the collision avoidance action comprises alerting a third party about an impending collision, wherein the third party is emergency services.

Images