Enhanced obstacle detection

By combining radar and camera data to detect abnormal movements and road conditions of vehicles in front, the problem of difficulty in effectively detecting road changes in the prior art is solved, driving safety is improved and collision risks are reduced.

CN114291080BActive Publication Date: 2024-11-12ARGO AI LLC
View PDF 8 Cites 0 Cited by

Patent Information

Application Number
CN202111105432.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2020-09-22
Filing Date
2021-09-22
Publication Date
2024-11-12
Estimated Expiration
2041-09-22

AI Technical Summary

Technical Problem

Existing vehicle sensor systems are difficult to effectively detect road changes and obstacles, resulting in unexpected situations that may occur when the vehicle enters or approaches a changing road, affecting driving safety.

Method used

Combining radar data and camera images, through boundary parameters and map data, abnormal movement and road conditions of the vehicle ahead are detected, and the LIDAR sensor is used to determine the nominal ground plane and expected path, and set signs to indicate abnormal road conditions, and connect via Wi-Fi or cellular. Send data to the map data center.

Benefits of technology

It improves vehicles' awareness of road changes, enhances driving safety, reduces collision risks through real-time warnings and avoids maneuvers, and improves confidence in abnormal road conditions through crowdsourcing mechanisms.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114291080B_ABST
    Figure CN114291080B_ABST
Patent Text Reader

Abstract

Devices, systems, and methods for enhanced obstacle detection are provided. A vehicle may detect an adjacent vehicle traveling on a first road in a first traffic direction near the vehicle. The vehicle may collect data associated with the adjacent vehicle using one or more sensors of the vehicle. The vehicle may determine a state of the adjacent vehicle at a geographic location of the adjacent vehicle based on the collected data. The vehicle may determine a first road anomaly occurring at a geographic location based on the state of the adjacent vehicle. The vehicle may set a first flag indicating the first road anomaly. The vehicle may determine to take a first action based on the first flag.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention generally relates to systems and methods for enhanced obstacle detection. Background Art

[0002] Some vehicles are equipped with sensor systems to collect data associated with the current and developing state of the vehicle's surroundings. The proper performance of the vehicle depends on the accuracy of the data collected by the sensors in the sensor system. The sensor system may include a radar sensor, a visual spectrum camera, a laser ranging device (LIDAR), a thermal sensor, an inertial measurement unit (IMU), or other types of sensors. The sensor system enables the vehicle to detect objects and obstacles near the vehicle and track the speed and direction of pedestrians, other vehicles, traffic lights, or similar objects in the vehicle's surroundings. However, over time, the road may experience various changes, such as defects and anomalies, which may cause the vehicle to experience these changes because these changes are unexpected before driving into or approaching them. Therefore, it is necessary to enhance the vehicle's awareness of road changes to enhance the vehicle user experience. Summary of the invention

[0003] The example embodiments described herein provide certain systems, methods, and apparatus for roadway obstacle detection.

[0004] In one or more embodiments, an enhanced obstacle detection system can help combine radar data with camera images captured by radar and vehicle cameras to assess road conditions ahead of the vehicle. The enhanced obstacle detection system can capture radar data using a radar mounted on a target vehicle to determine the distance to the vehicle ahead in the radar line of sight and the speed relative to the target vehicle. On a road traveling in the same direction, the vehicle ahead will be traveling ahead of the target vehicle. The enhanced obstacle detection system can also capture multiple camera frames at predetermined intervals to assess and estimate the position of the vehicle ahead.

[0005] In one or more embodiments, the enhanced obstacle detection system may apply boundary parameters to determine any abnormal movement of the leading vehicle in front of the target vehicle. Some boundary parameters may be based on the expected coordinates of the leading vehicle when traveling on the road. These boundary parameters may be in the form of elevation thresholds, nominal attitude / orientation thresholds, and deviation thresholds from predefined lines on the road. It should be understood that attitude is typically used when describing the 3D orientation of a vehicle, ship, or aircraft. For example, a vehicle attitude may specify the orientation of a vehicle or other object relative to the horizon, direction of motion, other objects, and the like.

[0006] In one or more embodiments, the enhanced obstacle detection system can mark the location of the vehicle ahead when the vehicle ahead moves abnormally at a certain location. The enhanced obstacle detection system can mark the location as encountering a fault or road condition. This may also occur due to weather conditions. For example, on an icy road or during heavy rain. In one or more embodiments, the enhanced obstacle detection system can use camera data to estimate the location of the vehicle ahead. For example, the vehicle ahead can be associated with a specific image defined by a pixel area. The pixel area can be determined to represent the vehicle ahead. The enhanced obstacle detection system can calculate the coordinate center on the pixel area and traverse the space in front of the target vehicle along the trajectory over a period of time. During this period of time, the camera can capture multiple frames. For example, the camera can capture approximately 1 to 100 frames per second.

[0007] In one or more embodiments, the enhanced obstacle detection system may compare the data captured by the camera and radar with a predetermined normal path on a particular road. For example, the average path of the vehicle ahead can be determined based on the geographic location of the vehicle ahead at the time. This path is then compared to the straight-line moving path of the road, or if the road is not in a straight line, it can also be compared to a spline along the straight line of the road. This can be used for vehicles ahead that move left and right or bounce up and down. The movement of the vehicle ahead at a specific geographic location can be compared with various thresholds to indicate whether there are abnormal road conditions. Some of these different thresholds may include elevation thresholds and lateral thresholds, or other deviation thresholds. For example, road subsidence can cause the vehicle ahead to bounce in front of the target vehicle. In some examples, less frequent roads with abnormal road conditions may surprise the driver if the vehicle is traveling at a speed that does not take into account abnormal conditions. However, drivers who often travel on this road may predict or quickly identify abnormal road conditions and may actively slow down or adjust their vehicle trajectory to avoid unnecessary consequences due to encountering nominal speeds and trajectories with road obstacles. In the case where the vehicle ahead is familiar with the local road, the vehicle ahead may not significantly deviate from the position or posture threshold of the abnormal road condition. In this case, the enhanced obstacle detection system can use a time threshold to determine the deviation of the leading vehicle from the nominal coordinates in the combined time-space domain. If the abnormal road condition is triggered by the movement of the leading vehicle, the speed of the leading vehicle may remain consistent and will not change with the triggering of the abnormal road condition. However, the calculated coordinate center and the instantaneous posture / orientation of the leading or adjacent vehicle may change and can be used as a metric to be compared with various thresholds. For example, the image in at least one of the camera frames captured in the front field of view of the target vehicle can change and can be used as a metric to be compared with various thresholds. In one or more embodiments, the enhanced obstacle detection system can determine when the adjacent vehicle in front moves abnormally left or right. For example, in order to avoid potholes or other road conditions, the vehicle may move left or right. The enhanced obstacle detection system can use map data to calculate the normal center of the lane in which the leading vehicle is traveling. From this, a "lane" can be defined and can be projected to indicate how various vehicles should nominally travel in that particular location. For example, if a vehicle deviates from the lane, the enhanced obstacle detection system can indicate an abnormal driving condition and can set a flag for an abnormal driving condition associated with the leading vehicle. In some cases, the sign may trigger an action, which may include displaying a warning on an output device, notifying a central server, notifying a vehicle user, or causing the vehicle to take an evasive maneuver. The notification may include audio, visual, SMS text, digital data transmission, etc. A target vehicle tracking a leading vehicle may not be at the same geographic location of the leading vehicle because the leading vehicle traveled ahead of the target vehicle traveling in the same direction. Therefore, the target vehicle cannot determine the type of road conditions ahead until it reaches that location.The enhanced obstacle detection system will help the target vehicle utilize information associated with abnormal driving conditions even if the target vehicle is not currently in the same geographic location as the leading vehicle.

[0008] In one or more embodiments, an enhanced obstacle detection system may facilitate capturing road map data. The map data may be collected from a previous baseline, which may have been previously performed during a mapping process that involves repeated exposure of a vehicle to the same road. Based on repeated exposure to the road, a map may be generated when a driver drives as close to the center of the road as possible, using a spline generated using a function representing the center of a lane. From this, a mean value associated with the center of the lane may be determined based on where most vehicles travel in a normal manner within the nominal lane center plus or minus an error range.

[0009] In one or more embodiments, the enhanced obstacle detection system can use LIDAR sensors to determine the nominal ground plane and indicate the expected normal road elevation. Using the estimated coordinates, if the leading vehicle experiences vibration for a predetermined period of time (e.g., one or two seconds), but then returns to its expected path, a flag can be set to indicate an abnormal road condition. Vibration refers to the upward rebound or vertical movement of the vehicle suspension when the vehicle suspension contacts a bump in the road or other road condition that causes the vehicle suspension to move vertically.

[0010] In one or more embodiments, the enhanced obstacle detection system may use a communication mechanism such as Wi-Fi or a cellular connection to send data associated with setting signs indicating abnormal road conditions to a map data center. The map data center will then compile a history of the times and locations where these signs were set. In some cases, the sign may trigger an action, which may include displaying a warning on an output device, notifying a central server, notifying a vehicle user, or causing the vehicle to take an evasive maneuver. The notification may include sound, visual, SMS text, digital data transmission. In one or more embodiments, the enhanced obstacle detection system may notify other neighboring vehicles that may be connected to the same network as the target vehicle to confirm whether these neighboring vehicles following their own leading vehicle are also experiencing the same behavior at that geographic location.

[0011] In one or more embodiments, the enhanced obstacle detection system can promote crowdsourcing of data associated with a specific geographic location based on setting one or more signs to indicate abnormal road conditions. As more vehicles provide information associated with the geographic location based on abnormal road conditions, the confidence level increases.

[0012] The above description is for illustration only and is not limiting. There may be many other examples, configurations, processes, etc., some of which will be described in more detail below. Example embodiments will now be described with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 An example environment of a vehicle according to one or more example embodiments of the present invention is shown.

[0014] Figure 2A-2B A schematic diagram of an enhanced obstacle detection system according to one or more example embodiments of the present invention is depicted.

[0015] Figures 3A-3C A schematic diagram of an enhanced obstacle detection system is depicted in accordance with one or more example embodiments of the present disclosure.

[0016] Figure 4 A schematic diagram of enhanced obstacle detection according to one or more example embodiments of the present disclosure is depicted.

[0017] Figure 5 is a block diagram illustrating an example of a computing device or computer system on which any one of the one or more techniques (eg, methodologies) may be performed according to one or more example embodiments of the present disclosure.

[0018] Certain implementations will be described more fully below with reference to the accompanying drawings, in which various implementations and / or aspects are shown. However, the various aspects may be implemented in many different forms and should not be construed as limited to the implementations described herein; rather, these implementations are provided so that the disclosure will be thorough and complete and will fully convey the scope of the disclosure to those skilled in the art. Like numbers in the figures refer to like elements throughout. Therefore, if a feature is used in multiple figures, the number used to identify the feature in the figure in which the feature first appears will be used in later figures. DETAILED DESCRIPTION

[0019] Sensors can be located at different locations on the autonomous vehicle. These sensors can include LIDAR sensors, stereo cameras, radar sensors, thermal sensors, inertial measurement units (IMUs), or other sensors attached to the autonomous vehicle.

[0020] Radar sensors in automotive environments have a coarser resolution than LIDAR or cameras in terms of both azimuth and elevation resolution. In some cases, radar may not even have elevation resolution. However, radar sensors are very good in both velocity resolution and range resolution. Roads may have characteristics such as topological changes that vary from one point on the road to another. These topological changes may have different elevation changes, and in some cases, there may also be obstacles such as potholes, sharp drops in the road, or other types of obstacles. These road characteristics may cause an unsuspecting vehicle operator to approach an obstacle without carefully identifying what is ahead. For example, a vehicle ahead may decelerate dramatically, which will cause the following vehicle to decelerate quickly to avoid a collision. It would be beneficial to determine a course of action based on what happened to the vehicle ahead, as an indication of what is ahead, or even based on historical information to understand the road conditions.

[0021] The example embodiments described herein provide certain systems, methods, and apparatus for roadway obstacle detection.

[0022] In one or more embodiments, an enhanced obstacle detection system can help combine radar data with camera images captured by radar and vehicle cameras to assess road conditions ahead of the vehicle. The enhanced obstacle detection system can capture radar data using a radar mounted on a target vehicle to determine the distance to the vehicle ahead in the radar line of sight and the speed relative to the target vehicle. On a road traveling in the same direction, the vehicle ahead will be traveling ahead of the target vehicle. The enhanced obstacle detection system can also capture multiple camera frames at predetermined intervals to assess and estimate the position of the vehicle ahead.

[0023] In one or more embodiments, the enhanced obstacle detection system may apply boundary parameters to determine any abnormal movement of the leading vehicle in front of the target vehicle. Some boundary parameters may be based on the expected coordinates of the leading vehicle when traveling on the road. These boundary parameters may be in the form of elevation thresholds, nominal attitude / orientation thresholds, and deviation thresholds from predefined lines on the road. It should be understood that attitude is typically used when describing the 3D orientation of a vehicle, ship, or aircraft. For example, a vehicle attitude may specify the orientation of a vehicle or other object relative to the horizon, direction of motion, other objects, and the like.

[0024] In one or more embodiments, the enhanced obstacle detection system can mark the location of the vehicle ahead when the vehicle ahead moves abnormally at a certain location. The enhanced obstacle detection system can mark the location as encountering a fault or road condition. This may also occur due to weather conditions. For example, on an icy road or during heavy rain. In one or more embodiments, the enhanced obstacle detection system can use camera data to estimate the location of the vehicle ahead. For example, the vehicle ahead can be associated with a specific image defined by a pixel area. The pixel area can be determined to represent the vehicle ahead. The enhanced obstacle detection system can calculate the coordinate center on the pixel area and traverse the space in front of the target vehicle along the trajectory over a period of time. During this period of time, the camera can capture multiple frames. For example, the camera can capture approximately 1 to 100 frames per second.

[0025] In one or more embodiments, the enhanced obstacle detection system may compare the data captured by the camera and radar with a predetermined normal path on a particular road. For example, the average path of the vehicle ahead can be determined based on the geographic location of the vehicle ahead at the time. This path is then compared to the straight-line moving path of the road, or if the road is not in a straight line, it can also be compared to a spline along the straight line of the road. This can be used for vehicles ahead that move left and right or bounce up and down. The movement of the vehicle ahead at a specific geographic location can be compared with various thresholds to indicate whether there are abnormal road conditions. Some of these different thresholds may include elevation thresholds and lateral thresholds, or other deviation thresholds. For example, road subsidence can cause the vehicle ahead to bounce in front of the target vehicle. In some examples, less frequent roads with abnormal road conditions may surprise the driver if the vehicle is traveling at a speed that does not take into account abnormal conditions. However, drivers who often travel on this road may predict or quickly identify abnormal road conditions and may actively slow down or adjust their vehicle trajectory to avoid unnecessary consequences due to encountering nominal speeds and trajectories with road obstacles. In the case where the vehicle ahead is familiar with the local road, the vehicle ahead may not significantly deviate from the position or posture threshold of the abnormal road condition. In this case, the enhanced obstacle detection system can use a time threshold to determine the deviation of the leading vehicle from the nominal coordinates in the combined time-space domain. If the abnormal road condition is triggered by the movement of the leading vehicle, the speed of the leading vehicle may remain consistent and will not change with the triggering of the abnormal road condition. However, the calculated coordinate center and the instantaneous posture / orientation of the leading or adjacent vehicle may change and can be used as a metric to be compared with various thresholds. For example, the image in at least one of the camera frames captured in the front field of view of the target vehicle can change and can be used as a metric to be compared with various thresholds. In one or more embodiments, the enhanced obstacle detection system can determine when the adjacent vehicle in front moves abnormally left or right. For example, in order to avoid potholes or other road conditions, the vehicle may move left or right. The enhanced obstacle detection system can use map data to calculate the normal center of the lane in which the leading vehicle is traveling. From this, a "lane" can be defined and can be projected to indicate how various vehicles should nominally travel in that particular location. For example, if a vehicle deviates from the lane, the enhanced obstacle detection system can indicate an abnormal driving condition and can set a flag for an abnormal driving condition associated with the leading vehicle. In some cases, the sign may trigger an action, which may include displaying a warning on an output device, notifying a central server, notifying a vehicle user, or causing the vehicle to take an evasive maneuver. The notification may include audio, visual, SMS text, digital data transmission, etc. A target vehicle tracking a leading vehicle may not be at the same geographic location of the leading vehicle because the leading vehicle traveled ahead of the target vehicle traveling in the same direction. Therefore, the target vehicle cannot determine the type of road conditions ahead until it reaches that location.The enhanced obstacle detection system will help the target vehicle utilize information associated with abnormal driving conditions even if the target vehicle is not currently in the same geographic location as the leading vehicle.

[0026] In one or more embodiments, an enhanced obstacle detection system may facilitate capturing road map data. The map data may be collected from a previous baseline, which may have been previously performed during a mapping process that involves repeated exposure of a vehicle to the same road. Based on repeated exposure to the road, a map may be generated when a driver drives as close to the center of the road as possible, using a spline generated using a function representing the center of a lane. From this, a mean value associated with the center of the lane may be determined based on where most vehicles travel in a normal manner within the nominal lane center plus or minus an error range.

[0027] In one or more embodiments, the enhanced obstacle detection system can use LIDAR sensors to determine the nominal ground plane and indicate the expected normal road elevation. Using the estimated coordinates, if the leading vehicle experiences vibration for a predetermined period of time (e.g., one or two seconds), but then returns to its expected path, a flag can be set to indicate an abnormal road condition. Vibration refers to the upward rebound or vertical movement of the vehicle suspension when the vehicle suspension contacts a bump in the road or other road condition that causes the vehicle suspension to move vertically.

[0028] In one or more embodiments, the enhanced obstacle detection system may use a communication mechanism such as Wi-Fi or a cellular connection to send data associated with setting signs indicating abnormal road conditions to a map data center. The map data center will then compile a history of the times and locations where these signs were set. In some cases, the sign may trigger an action, which may include displaying a warning on an output device, notifying a central server, notifying a vehicle user, or causing the vehicle to take an evasive maneuver. The notification may include sound, visual, SMS text, digital data transmission. In one or more embodiments, the enhanced obstacle detection system may notify other neighboring vehicles that may be connected to the same network as the target vehicle to confirm whether these neighboring vehicles following their own leading vehicle are also experiencing the same behavior at that geographic location.

[0029] In one or more embodiments, the enhanced obstacle detection system can promote crowdsourcing of data associated with a specific geographic location based on setting one or more signs to indicate abnormal road conditions. As more vehicles provide information associated with the geographic location based on abnormal road conditions, the confidence level increases.

[0030] The above description is for illustration only and is not limiting. There may be many other examples, configurations, processes, etc., some of which will be described in more detail below. Example embodiments will now be described with reference to the accompanying drawings.

[0031] Figure 1 An example environment 100 of a vehicle 102 is shown in accordance with one or more example embodiments of the present invention.

[0032] refer to Figure 1 , showing a target tracking vehicle 102 and an adjacent front vehicle 103 traveling on a road in the same direction. Vehicle 102 (or vehicle 103) may include one or more sensors 125. One or more sensors 125 may be associated with multiple cameras, transmitters, and sensors. One or more sensors 125 may be connected to vehicle 102 (e.g., at different locations on vehicle 102). In this environment 100, one or more sensors 125 are shown to include LIDAR 122, camera 124, thermal sensor 126, radar 128, inertial measurement unit (IMU), and one or more processors 132. Camera 124 may capture images of objects near and around vehicle 102. Other transmitters and sensors may send and / or receive one or more signals to detect and / or capture information associated with objects near and around target vehicle 102. For example, LIDAR 122 may emit LIDAR signals (e.g., light or electromagnetic waves), radar 128 may use radio waves to determine the distance between the vehicle and objects near the vehicle, and thermal sensor 126 may capture temperature (e.g., based on emitted and detected infrared signals or other laser signals).

[0033] In one or more embodiments, the one or more sensors 125 may include a LIDAR 122. Some examples of LIDARs include Geiger-mode LIDARs, ground-based LIDARs, large-size LIDARs, small-size LIDARs, and the like. The one or more sensors 125 may include a camera 124, such as a stereo camera 102 that can capture images near the vehicle. The one or more sensors 125 may include a thermal sensor 126, such as a thermal imaging, a thermistor, a resistance temperature detector, a thermocouple, a semiconductor, and the like. In addition, the one or more sensors 125 may include a radar 128, which may be any radar that uses radio waves to capture data from objects around the vehicle 10. The one or more sensors 125 may also include one or more processors 132. The one or more processors 132 may control the sending and receiving of signals associated with the LIDAR 122, the camera 124, the thermal sensor 126, and the radar 128. The various sensors of the one or more sensors 125 should operate according to their intended use when operating under normal conditions. However, vehicle 102 may be affected by road obstacles, such as potholes, road depressions, sinkholes, irregular terrain, fallen trees, debris, or any other obstacles that may affect the driving experience of the vehicle (e.g., vehicle 102 or vehicle 103). Under such road obstacles, vehicle 102 may cause the vehicle to shake due to, for example, the road surface sinking. This will cause sudden movements that may affect the drivability of vehicle 102 or 103. It should be understood that the above description is for illustrative purposes only and is not limiting.

[0034] Figure 2A-2B A schematic diagram of an enhanced obstacle detection system according to one or more example embodiments of the present invention is depicted.

[0035] refer to Figure 2A , a vehicle 202 that can travel on a road 203 is shown. The vehicle 202 can be similar to Figure 1 103 in the target tracking vehicle. The vehicle 202 may be a front adjacent vehicle traveling in front of a target tracking vehicle (not shown here). The vehicle 202 may serve as an early indication mechanism for the tracking vehicle to indicate conditions on the road ahead. For example, any road obstructions that the front vehicle 202 may encounter may also be encountered by the target tracking vehicle. The vehicle 202 may serve as a warning to the tracking vehicle that may be traveling behind the vehicle 202. The tracking vehicle may monitor the movement of the vehicle 202 to determine if a road anomaly has occurred that the vehicle 202 may experience.

[0036] The tracking vehicle can apply boundary parameters to determine any abnormal movement of the front vehicle 202. Some boundary parameters can be based on the expected coordinates of the front vehicle 202 when traveling on the road. These boundary parameters can be in the form of elevation thresholds, nominal attitude / orientation thresholds, and deviation thresholds from predefined lines on the road. For example, the vehicle 202 is shown to have a coordinate system with a horizontal axis and a vertical axis. For example, when the vehicle 202 is traveling forward on the road 203, the horizontal axis 210 and the vertical axis 211 are associated with the vehicle 202. The position of the longitudinal axis 211 represents the lateral movement of the vehicle 202, and the horizontal axis 210 represents the elevation movement of the vehicle 202, as seen by the tracking vehicle. Some boundary parameters on the horizontal axis 210 are regarded as upper and lower limits (e.g., upper limit 215 and lower limit 216). The upper limit 215 represents the upper elevation threshold of the vehicle 202. The lower limit 216 represents the lower elevation threshold of the vehicle 202. As the tracking vehicle follows and monitors vehicle 202, the tracking vehicle will analyze the elevation, current attitude / orientation, and coordinate system origin of vehicle 202 to determine whether it is within upper and lower limits when traveling on road 203. Horizontal axis 210 may be determined to be a certain distance from road 203. For example, as shown in FIG. 2 , horizontal axis 210 is determined to be a distance 207 from nominal road surface 203.

[0037] refer to Figure 2B , showing vehicle 202 traveling forward on nominal road surface 203, but dipping (e.g., dipping 205) in the road. Having detected that vehicle 202 has deviated from the tracked vehicle outside of boundary parameters (e.g., upper limit 215 and lower limit 216), the target tracked vehicle can mark the location of the leading vehicle, indicating the abnormal movement of the leading vehicle 202 at the geographic location of vehicle 202. The marked location serves as a warning sign for the target tracked vehicle, which may also cause the target tracked vehicle to update the central server with this road anomaly, and may even notify other vehicles in the same network as the target tracked vehicle. The target tracked vehicle may mark the geographic location as encountering a fault or road surface condition. This may also occur due to weather conditions. For example, on an icy road or during heavy rain.

[0038] exist Figure 2B In the example of FIG. 2 , vehicle 202 is shown to have sunk distance 209, which is shown to be below lower limit 216. A target tracking vehicle that has detected a deviation below lower limit 216 may mark the location for further processing or action.

[0039] The marked location is used as a warning sign for the target tracking vehicle, which may also cause the target tracking vehicle to update the central server with this road anomaly and may even notify other vehicles in the same network as the target tracking vehicle. The target tracking vehicle may mark the geographic location as encountering a fault or road condition. This may also occur due to weather conditions. For example, on icy roads or during heavy rain. The target tracking vehicle may use a communication mechanism such as Wi-Fi or a cellular connection to send data associated with setting signs indicating abnormal road conditions to a map data center. The map data center may use this data to compile a history of the time and place where these signs were set. This history will then be shared with vehicles in the network. In some cases, data collection may use crowdsourcing from other vehicles or other data sources. As more vehicles provide information associated with the geographic location based on abnormal road conditions, the confidence level increases. It should be understood that the above description is for illustrative purposes and is not meant to be limiting.

[0040] Figures 3A-3C A schematic diagram of an enhanced obstacle detection system is depicted in accordance with one or more example embodiments of the present disclosure.

[0041] refer to Figure 3A , showing two vehicles traveling in the same direction. The two vehicles are a front vehicle 303 and a target tracking vehicle 302. Vehicle 303 may be a front vehicle traveling in front of the target tracking vehicle 302. Vehicle 303 may serve as an early indication of the road ahead of the target tracking vehicle 302. For example, when the tracking vehicle 302 arrives at the same geographic location as the front vehicle 303, any road obstacles that the vehicle 303 may encounter may also be encountered by the target tracking vehicle 302. The front vehicle 303 serves as a warning to the tracking vehicle 302 that may be traveling behind the front vehicle 303. The tracking vehicle 302 may monitor the movement of the front vehicle 303 to determine whether a road anomaly that the front vehicle 303 may experience has occurred.

[0042] The target tracking vehicle 302 can apply boundary parameters to determine any abnormal movement of the front vehicle 303. Some boundary parameters can be based on the expected coordinates of the front vehicle 303 when traveling on the road. These boundary parameters can be in the form of a centerline deviation threshold from a predefined line on the road. For example, the front vehicle 303 is shown to be traveling around the centerline 301. Some boundary parameters around the centerline 301 are shown as an outward deviation threshold 310 and an inward deviation threshold 311. The outward deviation threshold 310 represents the distance that the front vehicle 303 may deviate from the centerline 301 before triggering a sign indicating abnormal behavior of the front vehicle 303. The inward deviation threshold represents how far the front vehicle 303 can deviate from the centerline 301 to the oncoming lane before triggering a sign indicating abnormal behavior of the front vehicle 303. In some cases, the sign can trigger an action, which can include displaying a warning on an output device, notifying a central server, notifying a vehicle user, or causing the vehicle to take an evasive maneuver. The notification can include sound, vision, SMS text, and digital data transmission.

[0043] As the target tracking vehicle follows and monitors the front vehicle 303, the tracking vehicle 302 may analyze the lateral movement of the front vehicle 303 to determine whether the front vehicle 303 is within the specified lateral boundaries while traveling on the road. The outward and inward deviation thresholds may be set based on information received from the central server or may be set by a network administrator to indicate the level of deviation before setting a flag.

[0044] refer to Figure 3B, showing that the front vehicle 303 has deviated outwardly to the right of the tracking vehicle 302 by a distance 306, which is shown to be greater than the outward deviation threshold 310. The target tracking vehicle 302, which has detected a deviation greater than the outward deviation threshold 310, can mark the location of the front vehicle 303 for further processing or operation. Marking the location can indicate a warning to the tracking vehicle 302 that a road anomaly has occurred at the location of the front vehicle 303. The tracking vehicle 302 can update the central server with the road anomaly, or even notify other vehicles in the same network as the tracking vehicle. The tracking vehicle 302 can mark the geographic location as experiencing a breakdown, road surface condition, or any other condition that causes the front vehicle 303 to deviate from its expected trajectory on the road at the geographic location. This may also occur due to weather conditions. For example, on icy roads or during heavy rain, the front vehicle 303 may be caused to deviate from the parameter limit. The target tracking vehicle 302 can use a communication mechanism such as Wi-Fi or a cellular connection to send data associated with setting a sign indicating an abnormal road condition to a map data center. The map data center can use this data to compile a historical record of the time and location of setting these signs. This history can be shared with vehicles in the network. In some cases, data collection may use crowdsourcing from other vehicles or other data sources. As more vehicles and other data sources provide information associated with geographic locations based on abnormal road conditions, the confidence level increases.

[0045] refer to Figure 3C, it is shown that the target vehicle 322 can detect the first adjacent vehicle 323 and the second adjacent vehicle 324 at the intersection, so that the paths of the two adjacent vehicles will intersect with the path of the target vehicle 322. The target vehicle 322 can be stationary (for example, stopped at a stop sign, a yield sign, a roundabout, or a traffic light at the intersection of the first road and the second road), but can still detect two adjacent vehicles. Although in this scene, two adjacent vehicles are shown, the presence of adjacent vehicles 323 or 324 may not occur at the same time, but they are shown mainly to illustrate various scenarios that the target vehicle 322 may encounter. For example, the adjacent vehicle 323 may encounter a significant obstacle 326 along its expected travel path 330. In this scene, the adjacent vehicle 323 is shown to exhibit a lateral deviation (lateral) (for example, a new deviation path 332) and / or a significant longitudinal deviation (for example, braking or deceleration) to avoid or prepare for the obstacle 326 found near the expected path 330. In this case, the target vehicle 322 can detect the deviation and compare the unexpected position at a specific time. For example, based on historical data or other data associated with vehicles traveling in the same direction as the adjacent vehicle 323, the target vehicle 322 may have some expectations that the adjacent vehicle 323 will travel along the center of its lane and maintain smooth acceleration or braking with a moderate error margin. However, the target vehicle 322 may determine that the adjacent vehicle 323 violates the expected spatial or temporal estimate of the nominal behavior of the given scenario, exceeding one or more thresholds. For example, if the lateral deviation exceeds a predetermined lateral threshold, or if the longitudinal deviation exceeds a predetermined longitudinal threshold, the target vehicle 322 can set a flag associated with the geographic location of the adjacent vehicle 323 and save the information locally or wirelessly to a server. This will create a historical record that can be shared with other vehicles in the network. In some cases, data collection can use crowdsourcing from other vehicles or data sources. As more vehicles or other data sources provide information associated with the geographic location given the abnormal road conditions, the confidence level increases.

[0046] Similar to the case of neighboring vehicle 323, neighboring vehicle 324 may also encounter obstacle 328 along its travel path 334. However, in this case, neighboring vehicle 324 may deviate slightly in the longitudinal and / or lateral directions. This may be done to provide additional warning and safety margin away from the obstacle. However, target vehicle 322 may determine that neighboring vehicle 324 has not violated the expected spatial or temporal estimate of nominal behavior for a given scenario that exceeds one or more thresholds. For example, if the lateral deviation is below a predetermined lateral threshold, or if the longitudinal deviation is less than a predetermined longitudinal threshold, target vehicle 322 may not set a flag associated with the geographic location of neighboring vehicle 324.

[0047] It should be understood that the above description is for illustration only and is not meant to be limiting.

[0048] Figure 4 A flow chart is shown of a process 400 for an illustrative enhanced obstacle detection system in accordance with one or more example embodiments of the present disclosure.

[0049] At block 402 , a vehicle (eg, a target vehicle) may detect a neighboring vehicle traveling on a first road in a first direction of traffic in a vicinity of the vehicle.

[0050] At block 404 , the vehicle may collect data associated with neighboring vehicles using one or more sensors of the vehicle.

[0051] At block 406 , the vehicle may determine the status of a neighboring vehicle at the neighboring vehicle's geographic location based on the collected data.

[0052] At block 408 , the vehicle may determine that a first road anomaly has occurred at the geographic location based on the status of neighboring vehicles.

[0053] At block 410 , the vehicle may set a first flag indicating a first road anomaly.

[0054] At block 412 , the vehicle may determine to take a first action based on the first indicator.

[0055] It should be understood that the above description is for illustration only and is not meant to be limiting.

[0056] Figure 5 is a block diagram illustrating an example of a computing device or computer system 500 upon which any one of the one or more techniques (eg, methodologies) may be performed according to one or more example embodiments of the present disclosure.

[0057] For example, Figure 5 The computing system 500 may represent Figure 2A , 2B and one or more processors 132 and / or one or more measuring devices 133, thereby evaluating and verifying Figure 1 The sensor in the sensor system 110. The computer system (system) includes one or more processors 502-506. The processors 502-506 may include one or more internal levels of cache (not shown) and a bus controller (e.g., bus controller 522) or bus interface (e.g., I / O interface 520) unit to interact directly with the processor bus 512. The enhanced obstacle detection device 509 may also communicate with the processors 502-506 and may be connected to the processor bus 512.

[0058] The processor bus 512, also known as a host bus or front-end bus, can be used to couple the processors 502-506 and / or the enhanced obstacle detection device 509 with the system interface 524. The system interface 524 can be connected to the processor bus 512 to interface other components of the system 500 with the processor bus 512. For example, the system interface 524 can include a memory controller 518 for interfacing the main memory 516 with the processor bus 512. The main memory 516 typically includes one or more memory cards and control circuits (not shown). The system interface 524 can also include an input / output (I / O) interface 520 to interface one or more I / O bridges 525 or I / O devices 530 with the processor bus 512. As shown, one or more I / O controllers and / or I / O devices (such as I / O controllers 528 and I / O devices 530) can be connected to the I / O bus 526.

[0059] The input / output devices 530 may also include input devices (not shown), such as alphanumeric input devices, including alphanumeric and other keys for communicating information and / or command selections to the processors 502-506 and / or the enhanced obstacle detection device 509. Another type of user input device includes a cursor control, such as a mouse, trackball, or cursor direction keys, for communicating direction information and command selections to the processors 502-506 and / or the enhanced obstacle detection device 509, and for controlling cursor movement on a display device.

[0060] The system 500 may include a dynamic storage device, referred to as a main memory 516, or a random access memory (RAM) or other computer readable device coupled to the processor bus 512 for storing information and instructions to be executed by the processors 502-506 and / or the enhanced obstacle detection device 509. The main memory 516 may also be used to store temporary variables or other intermediate information during execution of instructions by the processors 502-506 and / or the enhanced obstacle detection device 509. The system 500 may include a read-only memory (ROM) and / or other static storage device coupled to the processor bus 512 for storing static information and instructions for the processors 502-506 and / or the enhanced obstacle detection device 509. Figure 5 The system outlined in is only one possible example of a computer system that may employ or be configured according to aspects of the present disclosure.

[0061] According to one embodiment, the computer system 500 can perform the above-described techniques in response to the processor 504 executing one or more sequences of one or more instructions contained in the main memory 516. These instructions can be read into the main memory 516 from another machine-readable medium (such as a storage device). Executing the sequence of instructions contained in the main memory 516 can cause the processors 502-506 and / or the enhanced obstacle detection device 509 to perform the processing steps described herein. In alternative embodiments, circuits can be used in place of or in combination with software instructions. Therefore, embodiments of the present invention may include hardware and software components.

[0062] The processors 502-506 and / or the enhanced obstacle detection device 509 can help combine radar data with camera images captured by the radar and the vehicle camera to evaluate the road conditions in front of the vehicle. The enhanced obstacle detection system can capture radar data using a radar mounted on the target vehicle to determine the distance to the vehicle ahead in the radar line of sight and the speed relative to the target vehicle. On a road traveling in the same direction, the vehicle ahead will travel in front of the target vehicle that is tracking the vehicle ahead. The enhanced obstacle detection system can also capture multiple camera frames at predetermined intervals to evaluate and estimate the position of the vehicle ahead.

[0063] The processors 502-506 and / or the enhanced obstacle detection device 509 may apply boundary parameters to determine any abnormal movement of the leading vehicle in front of the target vehicle. Some boundary parameters may be based on the expected coordinates of the leading vehicle when traveling on the road. These boundary parameters may be in the form of elevation thresholds, attitude / orientation thresholds, and deviation thresholds from predefined lines on the road.

[0064] The processors 502-506 and / or the enhanced obstacle detection device 509 can mark the location of the leading vehicle when an abnormal movement occurs at the location of the leading vehicle. The enhanced obstacle detection system can mark the location as encountering a fault or road condition. This may also occur due to weather conditions. For example, on an icy road or during heavy rain.

[0065] Processors 502-506 and / or enhanced obstacle detection device 509 can use camera data to estimate the position of the vehicle ahead. For example, the vehicle ahead can be associated with a specific image defined by a pixel area. The pixel area can be determined to represent the vehicle ahead. The enhanced obstacle detection system can determine the coordinate center, then point the coordinate center to the pixel area and pass through the space ahead of the target vehicle along the trajectory over a period of time. During this period of time, the camera can capture multiple frames. For example, during a 10-second period of time, the camera can capture eight frames.

[0066] The processors 502-506 and / or the enhanced obstacle detection device 509 can compare the data captured by the camera and radar with a predetermined normal path on a particular road. For example, the average path of the vehicle in front can be determined based on the geographic location of the vehicle in front at that time. This path is then compared with the straight-line moving path of the road, or if the road is not in a straight line, it can also be compared with a spline along the straight line of the road. This can be used for vehicles in front that move left and right or bounce up and down. The movement of the vehicle in front at a specific geographic location can be compared with various thresholds to indicate whether there is an abnormal road condition. Some of these various thresholds may include an elevation threshold, a posture / orientation threshold, a lateral motion threshold, and / or other deviation thresholds. For example, a road sinking can cause the vehicle in front to bounce in front of the target vehicle. In some examples, if the vehicle is traveling at a speed that does not take into account the abnormal situation, a less frequent road with abnormal road conditions may surprise the driver. However, drivers who often travel on this road may recognize the abnormal road condition and may slow down so that the abnormal road condition is not triggered. Although the abnormal road condition is triggered by the movement of the adjacent vehicle in front, the speed of the vehicle in front may remain consistent and will not change with the triggering of the abnormal road condition. However, its image in at least one captured camera frame in the forward field of view of the target vehicle may vary and may be used as a metric for comparison with various thresholds.

[0067] The processors 502-506 and / or the enhanced obstacle detection device 509 can determine when the vehicle ahead moves sideways abnormally. For example, in order to avoid potholes or other road conditions, the vehicle may move left and right. The enhanced obstacle detection system can use the map data to calculate the normal center of the lane in which the vehicle ahead is traveling. From this, a "swim lane" can be defined and can be projected to indicate the normal driving mode of the vehicle at that particular location. For example, if the vehicle deviates from the swim lane, the enhanced obstacle detection system can indicate an abnormal driving condition and can set a flag for the abnormal driving condition associated with the vehicle ahead. The target vehicle tracking the vehicle ahead may not be in the same geographic location of the vehicle ahead because the vehicle ahead travels before the target vehicle traveling in the same direction. Therefore, before reaching this location, the target vehicle cannot distinguish the type of road condition ahead. The processors 502-506 and / or the enhanced obstacle detection device 509 can facilitate the object vehicle to utilize information associated with abnormal driving conditions, even if the object vehicle is not yet in the same geographic location as the vehicle ahead.

[0068] The processors 502-506 and / or the enhanced obstacle detection device 509 may capture map data of the road. The map data may be collected during a mapping process that involves repeated exposure of the vehicle to the same road. Based on repeated exposure to the road, a map may be generated when the driver drives as close to the center of the road as possible, using a spline generated using a function representing the center of the lane. From this, an average value associated with the center of the lane may be determined based on the position of most people driving in a normal manner after adding or subtracting a margin of error to the center of the lane.

[0069] The processors 502-506 and / or the enhanced obstacle detection device 509 may use LIDAR sensors, IMU sensors, and / or prior map data to indicate expected normal road elevation. Using the estimated coordinates, if the leading vehicle experiences vibration for a predetermined period of time (e.g., one or two seconds), but then returns to its expected path, a flag may be set to indicate an abnormal road condition. Vibration refers to an upward rebound or vertical movement of the vehicle suspension when the vehicle suspension contacts a bump in the road or other road condition that causes the vehicle suspension to move vertically.

[0070] Processors 502-506 and / or enhanced obstacle detection device 509 may use a communication mechanism such as Wi-Fi or a cellular connection to send data associated with setting signs indicating abnormal road conditions to a map data center. The map data center will then compile a history of when and where these signs were set. Processors 502-506 and / or enhanced obstacle detection device 509 may notify other neighboring vehicles that may be connected to the same network as the target vehicle to confirm whether these neighboring vehicles following their own front vehicle also experienced the same behavior at that geographic location.

[0071] Processors 502-506 and / or enhanced obstacle detection device 509 can facilitate crowdsourcing of data associated with a particular geographic location based on setting one or more flags to indicate abnormal road conditions. As more vehicles provide information associated with the geographic location based on abnormal road conditions, the confidence level increases.

[0072] Various embodiments may be implemented in whole or in part in software and / or firmware. The software and / or firmware may take the form of instructions contained in or on a non-transitory computer-readable storage medium. These instructions may then be read and executed by one or more processors to achieve the performance of the operations described herein. The instructions may be in any suitable form, such as, but not limited to, source code, compiled code, interpreted code, executable code, static code, dynamic code, etc. Such computer-readable media may include any tangible non-transitory medium for storing information in a form readable by one or more computers, such as, but not limited to, read-only memory (ROM); random access memory (RAM); magnetic disk storage media; optical storage media; flash memory, etc.

[0073] Machine-readable media include any mechanism for storing or transmitting information in a machine (e.g., computer) readable form (e.g., software, processing applications). Such media may take the form of, but not limited to, non-volatile media and volatile media, and may include removable data storage media, non-removable data storage media, and / or external storage devices provided with such computer program products via a wired or wireless network architecture, including one or more database management products, web server products, application server products, and / or other additional software components. Examples of removable data storage media include compact disk read-only memory (CD-ROM), digital versatile disk read-only memory (DVD-ROM), magneto-optical disk, flash drive, etc. Examples of non-removable data storage media include internal hard disks, SSDs, etc. One or more memory devices 606 (not shown) may include volatile memory (e.g., dynamic random access memory (DRAM), static random access memory (SRAM), etc.) and / or non-volatile memory (e.g., read-only memory (ROM), flash memory, etc.).

[0074] A computer program product containing mechanisms for implementing systems and methods according to the presently described techniques may reside in main memory 516, which may be referred to as a machine-readable medium. It should be understood that a machine-readable medium may include any tangible, non-transitory medium capable of storing or encoding instructions to perform any one or more operations of the present invention for execution by a machine, or capable of storing or encoding data structures and / or modules used by such instructions or associated with such instructions. A machine-readable medium may include a single medium or multiple media (e.g., a centralized or distributed database, and / or associated caches and servers) storing one or more executable instructions or data structures.

[0075] Embodiments of the present invention include various steps described in this specification. These steps can be performed by hardware components, or can be embodied in machine executable instructions, which can be used to perform these steps by a general or special processor programmed with these instructions. Alternatively, these steps can be performed by a combination of hardware, software and / or firmware.

[0076] Various modifications and additions may be made to the exemplary embodiments discussed without departing from the scope of the invention. For example, although the above-described embodiments relate to specific features, the scope of the invention also includes embodiments having different combinations of features and embodiments that do not include all of the described features. Therefore, the scope of the invention is intended to include all such substitutions, modifications and variations and all equivalents thereof.

[0077] The operations and processes described and shown above may be performed or executed in any appropriate order required in various implementations. In addition, in some implementations, at least a portion of the operations may be performed in parallel. In addition, in some implementations, less than or more than the described operations may be performed.

[0078] The word “exemplary” is used herein to mean “serving as an example, instance, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0079] As used herein, unless otherwise specified, the use of ordinal adjectives "first", "second", "third", etc. to describe a common object merely indicates that different instances of similar objects are referred to and does not imply that the objects described must be in a given order, time, space, ranking, or in any other manner.

[0080] It should be understood that the above description is for illustration only and is not meant to be limiting.

[0081] Although specific embodiments of the present invention have been described, those of ordinary skill in the art will recognize that many other modifications and alternative embodiments are within the scope of the present invention. For example, any function and / or processing capability described with respect to a particular device or component may be performed by any other device or component. In addition, although various illustrative implementations and architectures have been described according to embodiments of the present invention, those of ordinary skill in the art will appreciate that many other modifications to the illustrative implementations and architectures described herein are also within the scope of the present invention.

[0082] Although embodiments have been described using language specific to structural features and / or methodological acts, it should be understood that the disclosure is not necessarily limited to the specific features or acts described. Rather, specific features and actions are disclosed as illustrative forms of implementing embodiments. Conditional language, such as, among others, "may," "could," "might," or "may," unless otherwise expressly stated or otherwise understood in the context in which it is used, is generally intended to convey that certain embodiments may include, while other embodiments do not include, certain features, elements, and / or steps. Therefore, such conditional language does not generally mean that one or more embodiments require features, elements, and / or steps in any way, or that one or more embodiments must include a method for determining whether such features, elements, and / or steps are and / or are included or will be performed in any particular embodiment, with or without user input or prompting.

Claims

1. A vehicle comprising a processing circuit coupled to a memory, wherein the processing circuit is configured to: capturing, using an imaging system of the vehicle, a plurality of images of an environment surrounding the vehicle; analyzing the captured images to detect neighboring vehicles traveling on a first road in a first traffic direction near the vehicle; collecting data associated with the neighboring vehicles using one or more sensors of the vehicle; analyzing the collected data to determine a trajectory of the neighboring vehicle over time, the trajectory including an elevation of the neighboring vehicle; using a sensor system of the vehicle, determining a nominal ground plane, the nominal ground plane defining an expected elevation of the neighboring vehicle; comparing the determined elevation of the neighboring vehicle over time to the expected elevation of the neighboring vehicle; generating a status of the neighboring vehicle at the geographic location of the neighboring vehicle based on the collected data, the status indicating whether the neighboring vehicle has deviated from the expected elevation threshold for a predetermined period of time; setting a first flag in response to a status of the neighboring vehicle indicating that a first road anomaly has occurred at the geographic location; and A first action is performed based on the first indicator, wherein the first action includes causing the vehicle to take an evasive maneuver.

2. The vehicle according to claim 1, wherein: The first traffic direction is the same direction as the vehicle or a direction intersecting the vehicle.

3. The vehicle according to claim 1, wherein: The processing circuit is further configured to: detecting a second adjacent vehicle traveling on a second road intersecting the first road near the vehicle, wherein the vehicle is stopped at an intersection of the first road and the second road; detecting an abnormal maneuver taken by the second adjacent vehicle based on a second road abnormality; as well as setting a second flag indicating a second road abnormality based on the deviation threshold; and A second signal instructing to perform a second action is output based on the second flag.

4. The vehicle according to claim 1, wherein: The one or more sensors are one or more radar sensors, one or more cameras, one or more LIDARs or one or more inertial measurement units IMUs.

5. The vehicle according to claim 1, wherein: The first action also includes at least one of displaying a warning on an output device, notifying a central server, or notifying a user of the vehicle.

6. The vehicle according to claim 1, wherein: The processing circuit is further configured to upload data associated with the first marker to a central server.

7. The vehicle according to claim 1, wherein: The processing circuit is also configured to periodically download data related to the geographic locations of nearby road obstacles and anomalies from the central server.

8. The vehicle of claim 1, wherein: The processing circuit is further configured to display a warning or notification indicating an anomaly at the geographic location of the neighboring vehicle.

9. The vehicle of claim 1, wherein: The processing circuit is further configured to: detecting one or more other vehicles capable of receiving communications from the vehicle; and The one or more other vehicles are notified of the first road abnormality.

10. The vehicle of claim 1, wherein: The processing circuit is configured to analyze the collected data to determine the trajectory of the neighboring vehicle over time by: for each of the plurality of images, detecting a region of pixels representing the neighboring vehicle; Calculating the coordinate center of each pixel region; and following the coordinate center for a period of time.

11. The vehicle of claim 1, wherein: The processing circuit is also configured to collect a plurality of images via a camera associated with the vehicle to determine a state of the road.

12. A non-transitory computer readable medium storing computer executable instructions which, when executed by one or more processors of a target vehicle, cause execution of operations comprising: capturing, using an imaging system of the target vehicle, a plurality of images of the surroundings of the target vehicle; analyzing the captured images to detect adjacent vehicles traveling on a first road in a first traffic direction near the target vehicle; collecting data associated with the neighboring vehicles using one or more sensors of the target vehicle; analyzing the collected data to determine a trajectory of the neighboring vehicle over time, the trajectory including an elevation of the neighboring vehicle; using a sensor system of the target vehicle, determining a nominal ground plane, the nominal ground plane defining an expected elevation of the neighboring vehicle; comparing the determined elevation of the neighboring vehicle over time to the expected elevation of the neighboring vehicle; generating a status of the neighboring vehicle at the geographic location of the neighboring vehicle based on the collected data, the status indicating whether the neighboring vehicle has deviated from the expected elevation threshold for a predetermined period of time; setting a first flag in response to a status of the neighboring vehicle indicating that a first road anomaly has occurred at the geographic location; and An action is performed based on the first indicator, wherein the action includes causing the target vehicle to take an evasive maneuver.

13. The non-transitory computer readable medium of claim 12, wherein: The one or more sensors are one or more radar sensors, one or more cameras, one or more LIDARs, or one or more inertial measurement units (IMUs).

14. The non-transitory computer readable medium of claim 12, wherein: Taking action also includes at least one of displaying a warning on an output device, notifying a central server, or notifying a user of the target vehicle.

15. The non-transitory computer readable medium of claim 12, wherein: The operations also include displaying a notification to upload data associated with the first marker to a central server.

16. The non-transitory computer readable medium of claim 12, wherein: The operations also include: detecting one or more other vehicles capable of receiving communications from the target vehicle; and The one or more other vehicles are notified of the road abnormality.

17. The non-transitory computer-readable medium of claim 12, wherein the operation of analyzing the collected data to determine the trajectory of the neighboring vehicle over time comprises the following operations: for each of the plurality of images, detecting a pixel region representing the neighboring vehicle; calculating a coordinate center of each pixel region; and following the coordinate center for a period of time.

18. A method performed by a target vehicle, comprising: capturing, using an imaging system of a target vehicle, a plurality of images of an environment surrounding the target vehicle; analyzing, by one or more processors of the target vehicle, the captured images to detect a leading vehicle traveling on a road in the same direction of traffic as the target vehicle; collecting data associated with the leading vehicle; analyzing the collected data to determine a trajectory of the leading vehicle over time, the trajectory including an elevation of the leading vehicle; using a sensor system of the target vehicle, determining a nominal ground plane, the nominal ground plane defining an expected elevation of the leading vehicle; comparing the elevation of the leading vehicle determined over time to the expected elevation of the leading vehicle; generating a status of the leading vehicle at the geographic location of the leading vehicle based on the collected data, the status indicating whether the leading vehicle has deviated from the expected elevation threshold for a predetermined period of time; responsive to a status of the leading vehicle indicating that a first road anomaly has occurred at the geographic location, setting a flag; and An action is taken based on the set flag, wherein the action includes causing the target vehicle to take an evasive maneuver.

19. The method according to claim 18, wherein: Collecting data also includes collecting data associated with the leading vehicle using one or more sensors of the target vehicle.

20. The method according to claim 18, wherein: Collecting data also includes collecting historical data associated with leading vehicles traveling on the road in the same direction of traffic as the target vehicle.

21. The method according to claim 18, wherein: The actions also include at least one of displaying a warning on an output device, notifying a central server, or notifying a user of the target vehicle.

22. A target vehicle comprising a processing circuit coupled to a memory, wherein the processing circuit is configured to: capturing, using an imaging system of the target vehicle, a plurality of images of the surroundings of the target vehicle; analyzing the captured images to detect a first vehicle traveling on a first road in a first traffic direction near the target vehicle; collecting data associated with the first vehicle using one or more sensors of the target vehicle; analyzing the collected data to determine a trajectory of the first vehicle over time, the trajectory including an elevation of the first vehicle; using a sensor system of the target vehicle, determining a nominal ground plane, the nominal ground plane defining an expected elevation of the first vehicle; comparing the determined elevation of the first vehicle over time to an expected elevation of the first vehicle; responsive to the determined elevation of the first vehicle deviating from the expected elevation threshold for a predetermined period of time, setting a first flag; and A first action is performed based on the first indicator, wherein the first action includes causing the target vehicle to take an evasive maneuver.

23. The target vehicle of claim 22, wherein: The processing circuit is configured to collect data associated with the first vehicle by collecting historical data associated with the first vehicle, the first vehicle traveling on the first road in a first traffic direction near the target vehicle.

24. The target vehicle of claim 23, wherein: The processing circuit is further configured to send data associated with the first vehicle including the first marker to a data center, wherein the data center is adapted to compile historical data associated with the first vehicle based on the first marker.

25. The target vehicle of claim 22, wherein: The processing circuit is configured to analyze the collected data to determine a trajectory of the first vehicle over time by: for each of the plurality of images, detecting a region of pixels representing the first vehicle; Calculating the coordinate center of each pixel region; and following the coordinate center for a period of time.

26. A non-transitory computer readable medium storing computer executable instructions which, when executed by one or more processors of a target vehicle, cause execution of operations comprising: capturing, using an imaging system of the target vehicle, a plurality of images of the surroundings of the target vehicle; analyzing the captured images to detect a first vehicle traveling on a first road in a first traffic direction near the target vehicle; collecting data associated with the first vehicle using one or more sensors of the target vehicle; analyzing the collected data to determine a trajectory of the first vehicle over time, the trajectory including an elevation of the first vehicle; using a sensor system of the target vehicle, determining a nominal ground plane, the nominal ground plane defining an expected elevation of the first vehicle; comparing the determined elevation of the first vehicle over time to the expected elevation of the first vehicle; responsive to the determined elevation of the first vehicle deviating from the expected elevation threshold for a predetermined period of time, setting a first flag; and An action is performed based on the first indicator, wherein the action includes causing the target vehicle to take an evasive maneuver.

27. The non-transitory computer-readable medium of claim 26, wherein the operation for collecting data further comprises collecting historical data associated with a first vehicle traveling on the first road in a first traffic direction near the target vehicle.

28. The non-transitory computer readable medium of claim 27, wherein the operations further comprise: Data associated with the first vehicle including the first signature is sent to a data center, wherein the data center is adapted to compile the historical data based on the first signature.

29. The non-transitory computer readable medium of claim 26, wherein: The operation of analyzing the collected data to determine a trajectory of the first vehicle over time includes the operations of: for each of the plurality of images, detecting a region of pixels representing the first vehicle; Calculating the coordinate center of each pixel region; and following the coordinate center for a period of time.

Citation Information

Patent Citations

  • Autonomous Driving At Intersections Based On Perception Data

    CN106873580A

  • Sinkhole detection systems and methods

    CN108089233A

  • Vehicle collision avoidance

    CN108146503A

  • Vehicle line-crossing detection method based on intelligent video analysis technology

    CN108596129A

  • Server, information processing method, and non-transitory storage medium storing program

    CN111009146A