Methods and systems for crowdsourced dangerous driving detection and accident mitigation

By installing integrated sensors on vehicles to observe and generate safety scores, the problem of identifying and mitigating dangerous driving behaviors is solved, enabling real-time monitoring and early warning, and improving road safety and the effectiveness of autonomous driving.

CN118358583BActive Publication Date: 2026-11-17BLACKBERRY LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202311770049.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2023-01-19
Filing Date
2023-12-21
Publication Date
2026-11-17
Estimated Expiration
2043-12-21

AI Technical Summary

Technical Problem

Existing technologies are insufficient to effectively identify and mitigate dangerous driving behaviors of road users, leading to frequent traffic accidents.

Method used

By installing integrated sensors on vehicles to observe the behavior of external road users, a normalized safety score is generated and reported to a server. Crowdsourced data is used to improve the accuracy of the safety score, providing drivers with warnings or automatically adjusting routes to avoid danger.

Benefits of technology

It has improved road safety, reduced traffic accidents, and enhanced drivers' safety awareness and vehicles' autonomous driving capabilities through real-time monitoring and early warning mechanisms.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118358583B_ABST
    Figure CN118358583B_ABST
Patent Text Reader

Abstract

This disclosure relates to methods and systems for crowdsourced dangerous driving detection and accident mitigation. A method at a computing device for a vehicle includes: observing the behavior of external road users using the vehicle's sensors; generating a normalized safety score for the external road users at the computing device; recording identification information for the external road users; and reporting the normalized safety score and identification information to a server.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This disclosure relates to vehicle systems, and more particularly to vehicle systems collecting data about road users. Background Technology

[0002] Using the road can be risky for vehicles, pedestrians, and other road users. Other hazards include poor road design in some locations, extremely high traffic volumes at certain times of day, and the sun being directly in front of drivers at specific times.

[0003] In addition, some drivers on the road are riskier than others. These drivers may drive in ways that could cause more accidents, such as continuous emergency braking, changing lanes without signaling, obstructing adjacent lanes, speeding, and other risk factors. Attached Figure Description

[0004] This disclosure will be better understood with reference to the accompanying drawings, in which:

[0005] Figure 1 This is a block diagram illustrating an example system with an insight generator and an insight consumer;

[0006] Figure 2 This is a block diagram illustrating an example architecture for edge domains.

[0007] Figure 3 It is a process diagram used to create and report security scores.

[0008] Figure 4 This is a data flow diagram showing the process of a safety report being sent from the reporting vehicle to the server and the server processing the report.

[0009] Figure 5 This is a data flow diagram illustrating insights into consumer devices receiving reports and actions taken based on those reports.

[0010] Figure 6 This is a simplified block diagram of a computing device that can be used in embodiments of this disclosure. Detailed Implementation

[0011] This disclosure provides a method for use at a computing device in a vehicle, the method comprising: observing the behavior of external road users using sensors of the vehicle; generating a normalized safety score for the external road users at the computing device; recording identification information for the external road users; and reporting the normalized safety score and identification information to a server.

[0012] This disclosure further provides a computing device for a vehicle, the computing device including: a processor; and a communication subsystem.

[0013] The computing device is configured to: use the vehicle's sensors to observe the behavior of external road users; generate a normalized safety score for external road users; record identification information for external road users; and report the normalized safety score and identification information to a server.

[0014] This disclosure also provides a computer-readable medium for storing instruction code that, when executed by a processor of a computing device for a vehicle, causes the computing device to: observe the behavior of external road users using the vehicle's sensors; generate a normalized safety score for the external road users; record identification information for the external road users; and report the normalized safety score and identification information to a server.

[0015] This disclosure relates to the use of integrated sensors to collect profiles of vehicles using roads. Multiple vehicles equipped with such integrated sensors can then crowdsource information about road users and their locations. These profiles can be used to generate a safety score for each vehicle seen on the road.

[0016] Various driving modes can be observed, such as, but not limited to, determining whether a vehicle signals to turn or change lanes; determining whether a vehicle is speeding; determining whether a vehicle is overtaking from the right in a left-hand driving area (or overtaking from the left in a right-hand driving area); determining whether a vehicle brakes suddenly; determining whether a vehicle is stopped in its lane or turns into another lane; determining whether a vehicle has lights on; determining whether a vehicle has low-tread tires, and other factors.

[0017] Each dangerous driving mode can be weighted based on its severity to arrive at a safety score.

[0018] In some embodiments, observations of good driving can be made through integrated sensors, and such observations of good driving can be used as factors in the safety score.

[0019] Furthermore, in some cases, dangerous driving modes can be implemented based on external mitigation factors, and these mitigation factors can be used when calculating safety scores. For example, if a vehicle veers into another lane due to debris on the road, this can mitigate the negative value applied to the safety score.

[0020] In some embodiments, for simplicity, it is assumed that each vehicle has one driver. However, in other embodiments, such information can be used to score safety based on driver differentiation if driver identification or differentiation is possible for a particular vehicle. Various options for driver identification may include image processing from captured images, route / location / time-based differentiation (especially if different driving behaviors are detected), observed driver behavior, and other options.

[0021] Each vehicle equipped with an integrated sensor system can report the observed vehicle's safety score, along with the vehicle's identifier, to a web-based server. This web-based server can then match the vehicle identifier with the safety score and establish a list of vehicles typically seen on roads within a specific area.

[0022] As more vehicles and dangerous driving patterns are observed, the accuracy of safety ratings will improve over time.

[0023] In addition, the web server can provide a confidence score for each identified vehicle. In particular, the confidence score can increase as more reports for a particular vehicle are received.

[0024] Once a safety score is available, and in some cases, once the confidence score has reached a threshold, various actions can be taken to improve road safety for vehicles equipped with integrated sensor systems. Safety information provided by vehicles with integrated sensor systems can also be used to improve road safety for vehicles without integrated sensor systems.

[0025] In one embodiment, a vehicle equipped with an integrated sensor system can warn its driver that one or more vehicles in the area may be dangerous. In some cases, specific information about dangerous vehicles can be provided. For example, a display or audio warning on the vehicle can indicate to the driver a history of unsignaled actions and emergency braking by a vehicle ahead. In some cases, safety information provided by a vehicle with an integrated sensor system can help the network accumulate knowledge about dangerous vehicles in the area. The network can also warn vehicles near dangerous vehicles that do not have an integrated sensor system.

[0026] In one embodiment, the vehicle may offer driving suggestions to avoid vehicles exhibiting dangerous driving patterns. For example, the driver may be advised to move to the far right lane to avoid a speeding vehicle approaching from behind, or to "slow down to avoid overtaking the vehicle in front."

[0027] In one embodiment, if the vehicle has autonomous driving capabilities, it can take direct actions such as changing lanes, changing routes, or slowing down.

[0028] In one embodiment, the action could be to route the vehicle using a "safest route". For example, when a vehicle system requests a route from point A to point B, the driver is typically offered options such as the shortest or fastest route. In this case, the "safest route" could be calculated, for example, based on other options such as the past driving patterns of such dangerous vehicles, to avoid specific locations where accidents have occurred, areas where dangerous driving vehicles have been reported, and areas where dangerous driving vehicles might be located when the vehicle arrives at that location.

[0029] In some cases, vehicle data may be anonymized to protect privacy, and instructions to the driver may simply be to exercise extra caution, for example, knowing that a vehicle in the area may brake suddenly. Other options are possible.

[0030] These and other aspects are described below.

[0031] Vehicle software system

[0032] Modern vehicles have numerous sensors. These sensors can be distributed across various computing nodes on the vehicle, with each node capable of connecting to zero, one, or more sensor drivers. These sensor nodes can further be manufactured by different companies and operate using different operating systems. Similarly, other distributed systems can have multiple nodes that need to communicate with each other.

[0033] Sensors, or groups of sensors, can be used to generate information useful for one or more applications. In modern vehicles, information from one or more physical sensors can be processed to create “insights” that may be valuable within the system. Such one or more physical sensors and their associated processing can logically be referred to as microservices or synthetic sensors (SS). The terms microservices and synthetic sensors are used interchangeably in this document.

[0034] Specifically, integrated sensors can be specialized applications dynamically deployed within systems operating in a vehicle. Their role is to observe specific signals emitted from hardware sensors within the vehicle and dynamically synthesize insights based on these signals. An example of such integrated sensors is detecting children left behind in a heated passenger compartment. These integrated sensors would observe vehicle signals such as ignition status, passenger compartment temperature, child seat locking status, and so on.

[0035] In another example, integrated sensors can use cameras, radar, lidar, or other sensors on the vehicle to derive insights into another road user's driving safety score.

[0036] Integrated sensors can be found in other types of applications, including but not limited to medical applications, manufacturing applications, Internet of Things applications, etc., and this disclosure is not limited to vehicle applications. For illustration, vehicle applications are provided below.

[0037] Insight is the term used in this article to describe any computer-generated interpretation of basic sensor data. Insight can be as simple as data aggregation or correlation, or as complex as artificial intelligence and machine learning. For example, a temperature sensor providing high and low water level indicators for notification can be considered an "insight." For location services, geofencing is an insight. For cameras, identifying external road users can be an insight. The use of cameras and associated image analysis can generate an insight into whether a vehicle overtaking is speeding. The use of a combination of sensors, such as temperature sensors and cameras, combined with an artificial intelligence model to determine whether a car seat is occupied in a hot vehicle could be an insight. Many other examples of insights are possible.

[0038] In one embodiment, the vehicle application can be implemented as a system that provides continuous access to vehicle data and intelligent insights in a way that is familiar and accessible to the developer community. This type of environment allows cloud developers to extend their reach to the edge of the vehicle by developing integrated sensors that utilize common cloud development techniques and paradigms to derive intelligent insights from vehicle data. This environment provides continuous access to vehicle data, enabling integrated sensors to be written into and deployed across a broad vehicle infrastructure without customization.

[0039] Insights can be generated based on the processor running in the initial installation or domain, but typically need to be shared with licensed software modules running in external domains. Each can run on a different operating system.

[0040] Now for reference Figure 1 It shows example systems of various generators and consumers that provide insights. Figure 1 The embodiments described are provided for illustrative purposes only, and in some cases, there will be fewer participants in the system. In other cases, there will be more participants in the system.

[0041] exist Figure 1 In some embodiments, vehicle 100 may be equipped with a computing system and a communication system. A portion of the computing system may include domain 110, which, as described below, may have applications that consume insights. Furthermore, a portion of the computing system on vehicle 110 may include edge domain 112. In some embodiments, edge domain 112 may generate insights. However, in other cases, insights may be generated within domain 110 or consumed within edge domain 112.

[0042] exist Figure 1 In the example, vehicle 100 uses a communication system to communicate with... Figure 1The eNB 120 communicates with an access point (such as a cellular base station). The base station can communicate with a core network 130, which can then forward the communication to a cloud service provider 140 via network 132. For example, the communication network 132 can be a wide area network, such as the Internet.

[0043] In other embodiments, any technology associated with a particular cellular or wireless communication protocol may be used instead of the core network 130.

[0044] In some embodiments, cloud service 140 may provide a compilation of insights (such as insights related to other road users).

[0045] In some embodiments, cloud domain 150 can generate or consume insights. Cloud domain 150 can communicate with cloud service provider 140 via network 132, and in some cases, with other domains such as domain 112 on vehicle 100.

[0046] Furthermore, device 160 can consume insights, rather than the vehicle. Device 160 can be any computing device capable of generating or consuming such insights, and can include IoT devices, mobile devices, medical devices, vehicles or vehicle-related equipment, and other options. Device 160 can communicate via network 132 using various wired or wireless technologies, including but not limited to Ethernet, fiber optic, cellular, Wi-Fi, satellite, and other options.

[0047] Device 160 may include domain 162, which in some embodiments may consume insights. Additionally, device 160 may include edge domain 164, which in some cases may generate insights. However, in other cases, domain 162 may generate insights, while edge domain 164 may consume insights.

[0048] In addition, although Figure 1 The embodiments shown depict only two domains within vehicle 100 or device 160, but in practice, vehicle 100 or device 160 may contain only one or more domains, and this disclosure is not limited to having only two domains within any particular device. Specifically, device 160 may be used alone to generate insights, in which case it will have only a single domain. For example, device 160 may be a camera associated with an intersection, a sensor on a streetlight, or another sensor, any of which can be capable of generating insights. In other cases, device 160 may consume insights alone and again have only one domain. In other cases, device 160 or vehicle 100 may have multiple domains as well as edge domain 112.

[0049] Each domain or edge domain can have an OS-independent architecture. Specifically, Figure 1The domain shown can be implemented as various domains, where different domains can have different operating systems. For example, in a vehicle system, different vehicle manufacturers might use different operating systems. Therefore, the system architecture for the edge domain can use an abstraction layer to allow the system to be implemented on different platforms. Now refer to Figure 2 .

[0050] exist Figure 2 In some embodiments, application 210 may be part of the application layer. In some cases, the application may be user-oriented and utilize integrated sensors 220 to gain insights.

[0051] Synthetic Sensor 220 is part of the Synthetic Sensor Framework. Specifically, developers can use the Synthetic Sensor Framework to build synthetic sensors. This framework exposes a mechanism to define actions for lifecycle events, services for requesting / responding to application programming interfaces (APIs), and other actions.

[0052] Figure 2 The framework can be further defined for policy-based authentication and authorization mechanisms for requests and events. When installing / uninstalling / updating integrated sensors, the installation and update service can interact with the authorization service to update authentication and authorization policies.

[0053] Other services are also possible.

[0054] The integrated sensor 220 can communicate with the vehicle abstraction layer (VAL) 250, for example, via an interface. The VAL 250 provides an insightful, value-added interface for accessing vehicle data. It accesses sensor drivers and hardware via the hardware abstraction layer 260.

[0055] The Vehicle Abstraction Layer 250 also provides access to insights based on vehicle data. These insights are inferences based on vehicle data and can be as simple as aggregation or as complex as machine learning models. For example, the Location Insights service can provide coordinate location data as well as higher-level insights such as geofencing.

[0056] Therefore, in Figure 2In this embodiment, insights may be provided as part of an insight service 252. This type of insight service 252 controls access to vehicle data in a normalized form and provides value-added inferences. Examples may include a location service that provides coordinate location data in a consistent format, as well as insights such as geofencing. Examples may further include a seat service that provides a wealth of seat information, such as seatbelt status, weight, position, and child lock status. Another example may include a camera service that provides video streaming from external cameras and may serve functions such as dialogue and / or editing. A battery service can provide insights and access to the battery, such as charging status, consumption, estimated remaining hours, estimated range, and other data. A door service can provide an abstraction of door and door status.

[0057] Insight services typically do not interact directly with sensor hardware to access sensor data; instead, they utilize the hardware abstraction layer 260. This separation provides a clear distinction between the responsibilities of the hardware abstraction layer 260 (sensor integration and normalization of sensor data) and the vehicle abstraction layer 250 (managing access to vehicle data and providing value-added insights).

[0058] Insight Service 252 can leverage sensor data from multiple hardware abstraction layers to provide vehicle abstraction and value-added insights.

[0059] The Hardware Abstraction Layer (HAL) 260 focuses on sensor integration, sensor data normalization, and acts as a barrier between certified and uncertified software. This is accomplished through HAL services 262, each of which provides three key functions: integration into the underlying sensors; sensor data normalization; and, if necessary, providing a barrier between certified and uncertified software.

[0060] Furthermore, returning to Insight Service 252, one function of Insight Service can be to implement access control on normalized sensor data and insights. To verify the authentication and authorization of service clients, Vehicle Abstraction Layer Service can utilize Policy Service 274 and Permission Service 272 from Platform Abstraction Layer 270.

[0061] While the hardware abstraction layer discussed above describes hardware integration, the platform abstraction layer (PAL) 270 encapsulates the underlying operating system 280 and key aspects of the platform, which allow the system to be ported to different platforms in the following ways: defining a consistent interface for key platform functions; encapsulating the details of the underlying OS into OS-bound components; and supporting high-level abstractions / procedural de-abstraction beyond the capabilities of the basic OS.

[0062] The PAL interface is exposed internally to HAL and VAL, and externally to the client, enabling the extension of the functionality and portability of these components to the client.

[0063] PAL 270 can offer a variety of services. Figure 2 The embodiments illustrate permission service 272 and policy service 274. In some cases, other services may be further provided, such as lifecycle services, process services, and installation and update services.

[0064] Specifically, Policy Service 274 is responsible for maintaining policy values; performing policy queries; notifying policy changes; resolving policy priorities; and controlling policy modifications. Here, policy handling configures the behavior of various components, and each policy can have a unique name, string-encoded configuration value, and priority.

[0065] The authorization service 272 is the central authority for domain authorization. Authorization-protected services utilize the authorization service to determine whether appropriate permissions have been granted. In one embodiment, authorization system authentication is supported by user and group assignments (UIDs / GIDs) from a POSIX (Portable Operating System Interface) process. The authorization service identifies / verifies components based on the UID assigned to the component by the process in the underlying operating system. This UID is carried by the underlying operating system and is considered legitimate (accessed and managed by the underlying OS using a verified / trusted scheme).

[0066] Therefore, in Figure 2 In one embodiment, a system independent of operating system 280 and specific sensor driver 290 is provided.

[0067] Capturing traffic behavior

[0068] Now for reference Figure 3 The diagram illustrates the vehicle generating insights into the behavior of road users around it. Figure 3 In one embodiment, the process begins at block 310 and proceeds to block 320, where the computing device on the vehicle can observe the behavior of external road users.

[0069] For example, using Figure 1 and Figure 2 In one embodiment, the vehicle may have one or more sensors capable of providing information about the external environment. Such sensors may include, for example, externally facing cameras, radar, lidar, communication subsystems capable of receiving communications from other vehicles, and other options.

[0070] This type of sensor data can then be extracted and provided to the insight service 252, which the integrated sensor 220 can then use to generate insights about the sensor data. In some cases, the insight service or the integrated sensor can use image analysis to extract desired information from images / videos captured by the camera.

[0071] The observation at box 320 can be any behavior being exhibited by an external road user. For example, if an external road user is seen changing lanes or turning without using a turn signal, this could be an observation at box 320. Conversely, if a vehicle does signal to turn or change lanes, this could also be an observation at box 320.

[0072] Similarly, if a vehicle is overtaking in the wrong lane, whether it is a left-hand drive vehicle overtaking from the right or a right-hand drive vehicle overtaking from the left, this can be observed at box 320.

[0073] If emergency braking of the vehicle is observed, this could be an observation at box 320. Furthermore, if more controlled braking is observed, this could also be an observation at box 320. In some embodiments, emergency braking can be determined, for example, based on a deceleration rate greater than a threshold.

[0074] The observation at box 320 can further track whether a vehicle is staying in its lane or obstructing other lanes.

[0075] Observations at box 320 can further examine the vehicle's mechanical road performance. For example, if a vehicle has non-functional lights, this may indicate a potential safety hazard. In other cases, if a large amount of smoke is being emitted from the vehicle, this could also suggest a safety hazard. If vehicle parts are detached, such as a suspended bumper, this could also indicate a safety hazard. In some situations, tire treads can be observed, and worn tires may indicate a safety hazard.

[0076] Furthermore, this disclosure is not limited to observations of other road users, but may also include observations of road hazards or road conditions. For example, if a stop sign is obscured by a tree or bush and is not seen by vehicle sensors until the last second, the location and information about the stop sign can be recorded and reported. If there is debris or obvious potholes on the road, the location and information about the hazard can be made and reported.

[0077] In other cases, the danger may be temporary. For example, the sun may be directly in front of a reporting vehicle on a particular road at a specific time of day, or it may be reflected off a building at another time of day. In other cases, intersections may be considered more dangerous during peak hours when people are in a greater hurry. Therefore, the observation at box 320 may further include a temporal component. Furthermore, as those skilled in the art will understand, such a temporal component may vary with the seasons if related to the position of the sun.

[0078] Further observations can be made in box 320.

[0079] In addition to observing the behavior of external road users at box 320, the vehicle computing device may identify or partially identify external road users and / or locations at box 330.

[0080] In some cases, this may involve capturing images / videos of external road users for processing on a server.

[0081] In some cases, this may involve capturing images / videos and performing some processing on the vehicle's computing device. For example, processing on the vehicle's computing device may use image recognition technology to find the license plates of external vehicles and derive the license plate number from such an image.

[0082] In some cases, images can be used to identify the brand, model, color, or other distinguishing features of external road users.

[0083] In some cases, the communication subsystem can receive intelligent transportation system messages from other vehicles and use the identification information within these messages.

[0084] In some cases, only a partial identification of the license plate may be available. For example, the image may be obscured if parts of the license plate are unreadable, if the license plate is partially obscured, if the license plate is old and some numbers or letters are peeling off and therefore indistinguishable from the background, or other factors. In such cases, partial identification can be stored for external vehicles.

[0085] In some cases, vehicle identification information may not be available, but location information may be. For example, if the vehicle generating the insight is traveling through an intersection where an accident has occurred, the vehicles involved in the accident may not be able to be identified, but image processing can still allow identification of the accident and its location.

[0086] In addition, in some cases, a hazard is being reported, and in such cases, it is not necessary to identify specific road users, but rather to identify the location of the hazard.

[0087] In some cases, vehicle identification and location information can be captured.

[0088] Furthermore, while the labeling in box 330 can be vehicle-specific, in some cases it can also be driver-specific. Specifically, a vehicle can be driven by multiple drivers, and the driving characteristics of each specific driver can affect the vehicle's safety score. Therefore, in some cases, images of the vehicle's driver can be captured to further differentiate the safety profile of external road users.

[0089] Based on sensor data and observations, a safety score can be created for external road users or locations at box 340.

[0090] Based on observations, weighting factors can be applied to safety scores. For example, overtaking on a dual-lane road can be considered very unsafe and has a higher weighting factor, for example, compared to turning off taillights. Weighting factors can also be applied to positive behaviors, such as staying in the lane, braking gently, and regularly signaling to turn, among other positive behaviors.

[0091] Furthermore, the weighting factor can be adjusted based on mitigation factors of driving behavior. For example, if a driver leaves their lane to avoid an obstacle on the road, this behavior may be weighted less severely than a sharp turn caused by a driver appearing to be injured.

[0092] Therefore, weighting factors can be based on behaviors that have been shown to lead to accidents. For example, the weighting factor for the first observed action may differ from the weighting factor for the second observed action, based on the type of action identified.

[0093] In some cases, safety scores can be normalized based on the integrated sensors being used on the vehicle. For example, if all vehicles reporting to a particular server use the same integrated sensor, the output from the integrated sensor can be normalized regardless of the vehicle manufacturer, sensor manufacturer, sensor driver, etc., and thus allows observations from multiple vehicles to be combined at the server.

[0094] In other cases, standards or normalization can be created among various integrated sensors implemented on multiple vehicles, again allowing the server to utilize observations from multiple vehicles.

[0095] In other cases, the aggregated sensor data can be provided to the server along with other data, which allows the server to normalize the data based on the aggregated sensor data used to create the data.

[0096] exist Figure 3In box 350, the safety score from the integrated sensors can be reported to the server. As indicated above, the safety score can be reported along with other data, such as location and / or identification information of external road users, information about the integrated sensors used to create the safety score, and other data.

[0097] Furthermore, in some embodiments, instead of deriving a safety score in the vehicle's computing device, the report at box 350 may include information such as image capture, video, and identification information such as location, or any information derived by processing images at the vehicle. In this case, box 340 is optional, and the server may derive the safety score.

[0098] Furthermore, in some embodiments, the reporting at box 350 may not be continuous. For example, the vehicle may acquire a certain amount of information until a threshold information level is reached, or it may wait until a threshold duration is reached, and so on. When such a threshold is reached, the vehicle computing device may send a report at box 350, which in some cases may include a compilation of data since the last reporting period.

[0099] Furthermore, even when using thresholds, if an event is deemed necessary to report immediately, it can be reported immediately at box 350. For example, if an accident is observed, this could be a high-priority event that might require immediate reporting. Other options are possible.

[0100] In some cases, the vehicle may lose communication with the server, and the report at box 350 may be delayed until communication is re-established. For example, the vehicle may be driving in a remote area where there is no cellular connectivity, among other options.

[0101] In some embodiments, the report at box 350 may have security features, such as encryption, to ensure the privacy of the information. Furthermore, in some cases, to ensure the authenticity of the security report, various modules, such as integrated sensors or vehicle computing devices, may sign the report. The server may know the public key of the vehicle computing sensor or integrated sensor to verify the report's origin.

[0102] Then, the process proceeds from box 350 to box 360 and ends.

[0103] It is possible Figure 4 The embodiments further show the sending of reports and the actions taken by the server. Specifically, in Figure 4 In this process, the vehicle 410 generating the insight can communicate with the server 412. The server 412 can be any server or cloud service that can receive communications from multiple vehicles 410, and can be considered a hazardous road user insight server. As those skilled in the art will understand, the server can be from... Figure 1 Any part of the core network 130, cloud service 140, or cloud domain 150 in any embodiment.

[0104] exist Figure 4 In one embodiment, vehicle 410 may, for example, utilize Figure 3 The technique of the embodiment generates one or more reports 420 and sends the reports to server 412, as shown in message 430. Figure 3 As indicated by box 350, the transmission of the report at box 430 may include information such as location and / or vehicle identification information, may include image data, raw sensor data, video or other such information, may include information about the integrated sensors used to generate the report or information within the report, and other such information. However, this disclosure is not limited to the data types transmitted in message 430, and more or fewer data fields may be provided in message 430.

[0105] In addition, the report message 430 can be encrypted and / or signed for the purpose of verifying its authenticity.

[0106] Upon receiving message 430, server 412 may first attempt to match each report in message 430 with the identification information for location and / or road user involved in that report.

[0107] For example, in some cases, the complete license plate number can be provided within the report in message 430. In this case, the information received in the message can be associated with a vehicle having the license plate identified in box 440.

[0108] However, in some cases, only partial information may be provided within the report message 430. In such cases, server 412 may combine reports received from different vehicles to attempt to match road users with the reported information. For example, multiple reports may be received from multiple reporting vehicles. If such reporting vehicles are geographically close to each other, in some cases, partial identifiers in the reports may be correlated.

[0109] For example, if the first report includes time, location, vehicle brand, model, and color, and part of the license plate, and the second report includes a second time, a second location, and vehicle brand, model, and color, as well as a different part of the license plate, then server 412 can compare the two reports. For example, if the vehicle in the first report cannot move to the location in the second report within the time difference between the two reports, then the two vehicles may not be associated.

[0110] However, if the vehicle in the first report may have moved to the location identified in the second report during the time difference between the two reports, and if the vehicle identification features match, a possible association may exist. In this case, if there is some overlap between the license plate numbers, the two vehicle reports may belong to a single vehicle.

[0111] However, this is just one example, and other related methods will be clear to those skilled in the art.

[0112] In some cases, vehicle information stored in the database on server 412 can be correlated, for example, based on previously received reports about the vehicle.

[0113] Then Figure 4 The process proceeds to box 442, where server 412 can generate or update safety reports for road users and / or locations. Specifically, if no information exists regarding the vehicle identified in box 440, a new safety report can be created for that vehicle.

[0114] However, if information about the vehicle already exists in the database, the vehicle's safety score can be updated using information from the new report. Furthermore, in some cases, new data may be stored for later processing instead of updating the score, or new data may be stored for later processing in addition to updating the score. For example, data may be interpreted differently in more contextual situations. Additionally, in some cases, the score may not be determined until more data is received.

[0115] Furthermore, the confidence score of the safety report can be updated based on the fact that more information about the vehicle has now been received. In particular, the confidence level of whether the vehicle is safe or unsafe can be increased based on the fact that multiple reports about the vehicle have been received.

[0116] Furthermore, in some cases, the confidence score can be increased more if safety reports are received from different reporting vehicles than if a single vehicle is providing multiple reports to the same external road user.

[0117] For privacy reasons, in some cases, identification information such as license plate information may be stored as a hash on server 412.

[0118] In some embodiments, as vehicle safety information is updated, a report 450 may be provided to various reporting vehicles 410, as indicated by message 450. However, message 450 is optional.

[0119] Although Figure 4The embodiment illustrates a report from a vehicle, but in practice, the insight server 412 can further receive and process information from other sources. For example, traffic cameras at an intersection can record behavior, and information about such behavior, including processed information (if integrated sensors are associated with the traffic cameras) or raw information, can be provided to server 412. In the case of raw information, the insight server 412 can further include integrated sensors to process the data and provide a safety score consistent with the score received in report 430. This information can then be matched at box 440, and at box 442, an updated safety report is created for the road user and / or location.

[0120] In some embodiments, the information may further come from police reports, city accident statistics databases, CCTV cameras, security cameras, city road construction records, construction company databases, and other options. This type of external information can be combined with reports received from reporting vehicles to provide vehicle users with a comprehensive overview of driving hazards.

[0121] based on Figure 4 In one embodiment, multiple reporting vehicles can observe and generate safety scores for the road users and / or locations they see, and upload these results to server 412. Over time, safety scores can be created for road users typically seen on roads in a specific area. In some cases, safety scores can vary based on time of day, season, or other time references.

[0122] Safety scores can improve over time as more vehicles and dangerous driving patterns are observed.

[0123] Good driving behavior was also observed and can be fed into the safety score.

[0124] Actions based on security reports

[0125] Once the insight server possesses information about specific road users, specific locations, geographic areas, and other data, it can distribute this information to various participants in the system and take action based on the reports. (See now for reference.) Figure 5 .

[0126] exist Figure 5 In this embodiment, the consumer insight device 510 communicates with the insight server 512. In particular, the consumer insight device can be a vehicle, a mobile device for pedestrians, an infrastructure unit, or other options.

[0127] If the consumer insight device 510 is mobile, it can report its location to the insight server 512 in message 530. In other cases, message 530 can be an indication that the consumer insight device has woken up and wants to update its information message.

[0128] In some cases, message 530 may include a route already entered into the vehicle's navigation system, indicating the device's current location and its expected location at a specific time. In other cases, message 530 may only include the origin and destination of the trip.

[0129] In some cases, information 530 may include the location, speed, and direction of the journey.

[0130] Message 530 may also provide other information.

[0131] Upon receiving message 530, at box 540, the insight server 512 can create a security report for device 510. For example, the security report can be based on security information within the area surrounding device 510. This area can be set based on a threshold distance from the device 510's location. In some cases, the threshold distance can vary based on the device 510's speed, direction of travel, or other factors.

[0132] In some cases, a safety report can provide information about a specific vehicle. For example, the report might indicate that a red Honda Civic with license plate “ABC 123” is nearby and that the vehicle has a history of changing lanes without signaling.

[0133] In some cases, reports may anonymize information for privacy reasons. In such cases, the report may indicate that one or more vehicles in the area have a history of changing lanes without signaling, but without providing information about the specific vehicles.

[0134] In some cases, a report may provide information about certain locations rather than about other road users, or it may provide information about certain locations in addition to information about other road users. For example, the information may indicate that a particular intersection is dangerous. In some embodiments, such an indication may be temporary and indicate that a particular intersection is dangerous at this time of day.

[0135] In some cases, the report can suggest routes that the equipment should take to avoid dangers, hazardous areas, etc.

[0136] In some cases, if the score given to such vehicles, other road users, locations, or hazards reaches or exceeds a defined threshold, the report will only include the vehicle, other road users, location, or hazard.

[0137] In some cases, the threshold can be uniform for all road users in the system.

[0138] In some cases, the threshold can be configurable based on device 510. For example, a "risk tolerance" profile can be set on device 510 to adjust the number of reports received.

[0139] In some cases, the report will not consider vehicles whose safety score confidence level is below the threshold level.

[0140] In message 550, the report can be sent from server 512 to device 510 as information or insight.

[0141] After receiving message 550, device 510 can consume the information at box 560 and perform an action at box 570. The action at box 570 can include a variety of actions, and non-limiting examples are provided below.

[0142] In some cases, the action may be to provide an alert to the driver. This alert can use any user interface and may include, for example, one or more of the following: a display in the vehicle, audio feedback in the vehicle, or haptic feedback in the vehicle. For example, the vehicle's infotainment system may include a display that may show an alert that a dangerous driver is nearby or that the vehicle is approaching a dangerous intersection. In other cases, the display may be a heads-up display. Other options are possible.

[0143] In some cases, audio alarms may include ringing or beeping sounds indicating danger, or speech synthesis providing details of the danger.

[0144] In some situations, if a route has already been planned, the action can be to suggest alternative routes to avoid danger. This can be provided as an alternative on the navigation system. For example, navigation systems typically have options such as choosing the fastest route, the shortest route, avoiding toll roads, and so on. In this case, the navigation system can include a "safest route" option as a user choice and allow the user to select such a route. Furthermore, in some cases, the "safest route" can be adaptive and can be changed when a dangerous vehicle enters the area. Therefore, Figure 5 Data streams can be applied on a continuous basis.

[0145] In some situations, actions can be controlling the vehicle. For example, control could be applying the brakes, automatically changing lanes, or steer the vehicle onto a safer route. This may be especially true if the vehicle is autonomous, but it may also apply if the vehicle has autonomous features, such as intelligent cruise control, or the ability to exceed the driver's actions in certain situations.

[0146] Figure 5 Security in the system can be established based on the use of certificate registration to the integrated sensor system. For example, this can be implemented during device manufacturing, allowing certificates for communication within the system to be installed at that time. In some cases, a new certificate can be provided to device 510. In each case, between device 510 and server 512 (and...) Figure 4 There is a trust relationship between vehicle 410 and server 412.

[0147] Even if only a subset of vehicles or road users are part of an integrated sensor system, Figures 1 to 5 The embodiment also works. In this case, the vehicle or road user as part of the integrated sensor system will observe other road users, not just those who are part of the integrated sensor system.

[0148] The domains, network elements, cloud services, nodes, vehicle computing devices, and other computing platforms described above can be implemented using any computing device. About Figure 6 This shows a simplified diagram of a computing device. Figure 6 The computing device can be any fixed computing device or a mobile computing device.

[0149] exist Figure 6 In this embodiment, device 610 includes a processor 620 and a communication subsystem 630, wherein the processor 620 and the communication subsystem 630 cooperate to perform the methods described above. The communication subsystem 630 allows device 610 to communicate with other devices or network elements and can vary depending on the type of communication being performed. Furthermore, the communication subsystem 630 can include various communication technologies, including any wired or wireless communication technology.

[0150] Processor 620 is configured to execute programmable logic, which can be stored on device 610 along with data, and Figure 6 The example is shown as memory 632. Memory 632 can be any tangible, non-transitory computer-readable storage medium that stores instruction code, which, when executed by processor 620, causes device 610 to perform the methods of this disclosure. Computer-readable storage media can be tangible or transient / non-transitory media, such as optical storage (e.g., CD, DVD, etc.), magnetic storage (e.g., magnetic tape), flash memory drives, hard disk drives, or other storage media known in the art.

[0151] Alternatively, or in addition to memory 632, device 610 may access data or programmable logic from an external storage medium, for example, via communication subsystem 630.

[0152] exist Figure 6 In the example, one or more sensors 640 may be associated with a computing device. However, this is optional, and in some cases, the computing device 610 will not be associated with a sensor.

[0153] In one embodiment, communication between the various elements of device 610 can be achieved via internal bus 660. However, other forms of communication are possible.

[0154] The embodiments described herein are examples of structures, systems, or methods whose elements correspond to elements of the technology of this application. This written description enables those skilled in the art to make and use embodiments with alternative elements, which also correspond to elements of the technology of this application. Therefore, the intended scope of the technology of this application includes other structures, systems, or methods that are not different from the technology of this application as described herein, and further includes other structures, systems, or methods that are not substantially different from the technology of this application as described herein.

[0155] Although the operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring the execution of such operations in the specific order shown or in sequential order, or performing all the illustrated operations to achieve the desired result. In some cases, multitasking and parallel processing may be employed. Furthermore, the separation of the various system components in the above implementation should not be construed as requiring such separation in all implementations, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0156] Furthermore, technologies, systems, subsystems, and methods described and illustrated as discrete or separate in various implementations can be combined or integrated with other systems, modules, technologies, or methods. Other items shown or discussed as coupled or directly coupled or communicating with each other may be indirectly coupled or communicating through some interface, device, or intermediate component, whether electrically, mechanically, or otherwise. Other examples can be identified by those skilled in the art and can be modified, replaced, and altered.

[0157] While the detailed description above has shown, described, and pointed out the essential novel features of this disclosure applicable to various present inventions, it should be understood that those skilled in the art can make various omissions, substitutions, and changes to the form and details of the described system. Furthermore, the order of the method steps does not imply their order of appearance in the claims.

[0158] When sending / receiving messages to electronic devices, such operations may not be instantaneous or directly from a server. They can be delivered synchronously or asynchronously from servers or other computing system infrastructure that support the devices / methods / systems described herein. The foregoing steps may include, in whole or in part, synchronous / asynchronous communication to / from devices / infrastructure. Furthermore, communication from electronic devices can be to one or more endpoints on a network. These endpoints can be served by servers, distributed computing systems, stream processors, etc. Content Delivery Networks (CDNs) can also provide communication to electronic devices. For example, in addition to a typical server response, a server can provide or instruct data for the CDN to wait for the electronic device to download at a later time, such as subsequent activities of the electronic device. Therefore, data can be sent directly from servers or other infrastructure, such as distributed infrastructure or CDNs that are part of or separate from the system.

[0159] Typically, storage media can include any one or a combination of the following: semiconductor memory devices, such as dynamic random access memory or static random access memory (DRAM or SRAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), and flash memory; magnetic disks, such as fixed disks, floppy disks, and removable disks; another magnetic medium, including magnetic tape; optical media, such as optical discs (CDs) or digital video discs (DVDs); or another type of storage device. Note that the instructions discussed above can be provided on a single computer-readable or machine-readable storage medium, or alternatively, can be provided on multiple computer-readable or machine-readable storage media distributed across a large system that may have multiple nodes. Such computer-readable or machine-readable storage media are considered part of an article (or article of manufacture). An article or article of manufacture can refer to any single or multiple manufactured components. One or more storage media can be located in a machine that executes the machine-readable instructions, or at a remote site from which machine-readable instructions can be downloaded via a network for execution.

[0160] In the foregoing description, numerous details have been set forth to provide an understanding of the subject matter disclosed herein. However, these implementations can be practiced without some of these details. Other implementations may include modifications and variations to the details discussed above. The appended claims are intended to cover such modifications and variations.

Claims

1. A method for detecting dangerous driving, the method comprising: Using the vehicle's sensors, sensor data related to the behavior of external road users is obtained; The sensor data is provided to an abstraction layer between the computing device used in the vehicle and the sensors of the vehicle to generate abstract data; The abstract data is provided to the integrated sensors to generate a normalized safety score for the external road user, wherein the normalized safety score is determined based on one or more dangerous driving modes; Record identification information for the external road users; Report the normalized security score and identification information to the server; and The server is also configured to: Receive a first report from the vehicle and a second report from another reporting vehicle; In response to the fact that the external road user has moved from a first location in the first report to a second location in the second report within the time difference between the first report and the second report, and that some identification information in the first report and the second report overlaps, it is determined that the first report and the second report belong to the same external road user; as well as Increase the confidence level of the normalized safety score associated with the external road users.

2. The method of claim 1, wherein the report comprises: Before sending the report to the server, a threshold number of normalized safety scores for multiple external road users is obtained.

3. The method of claim 1, wherein the report comprises: Wait for a threshold time period from sending the first report to sending another report.

4. The method of claim 1, wherein the behavior includes at least one of the following: whether the external road user signals a turn or lane change; whether the external road user is speeding; whether the external road user overtakes on the wrong side; whether the external road user brakes beyond a threshold braking rate; whether the external road user turns into another lane; whether the external road user has lights on; whether the external road user is emitting smoke; whether the external road user's tire tread is above a threshold level; whether the external road user has parts partially detached from the external road user; and whether the external road user obeys traffic rules.

5. The method of claim 1, wherein the identification information includes a photograph or video of the external road user.

6. The method of claim 1, wherein the identification information includes image processing performed by the computing device to obtain at least one of the following: a license plate number for the external road user; a brand; a model; a color; and a marking.

7. The method according to claim 1, further comprising: At the computing device, a security report is received from the server; as well as Perform the action at the computing device.

8. The method of claim 7, wherein the action comprises at least one of: providing a notification to a user interface regarding a hazardous road user; and Vehicle control is automatically performed at the vehicle location.

9. The method of claim 7, wherein the safety report includes areas to be avoided, the areas to be avoided being based on road users in the areas whose safety scores are below a safety threshold level and whose confidence levels are above a confidence threshold level.

10. A system for detecting dangerous driving, the system comprising: A computing device for at least one vehicle, the computing device comprising: Processor; and Communication subsystem The computing device is configured as follows: Using the vehicle's sensors, sensor data relating to the behavior of external road users is obtained; The sensor data is provided to an abstraction layer between the computing device and the vehicle's sensors to generate abstract data; The abstract data is provided to the integrated sensors to generate a normalized safety score for the external road user, wherein the normalized safety score is determined based on one or more dangerous driving modes; Record identification information for the external road users; and Report the normalized security score and identification information to the server; The server is configured as follows: Receive a first report from the vehicle and a second report from another reporting vehicle; In response to the external road user having moved from a first location in the first report to a second location in the second report within the time difference between the first report and the second report, and the overlap of some identification information in the first report and the second report, it is determined that the first report and the second report belong to the same external road user; and Increase the confidence level of the normalized safety score associated with the external road users.

11. The system of claim 10, wherein the computing device is configured to report by obtaining a threshold number of normalized safety scores for a plurality of external road users before sending a report to the server.

12. The system of claim 10, wherein the computing device is configured to report by waiting for a threshold time period from sending the first report to sending another report.

13. The system of claim 10, wherein the behavior includes at least one of the following: whether the external road user signals a turn or lane change; whether the external road user is speeding; whether the external road user overtakes on the wrong side; whether the external road user brakes beyond a threshold braking rate; whether the external road user turns into another lane; whether the external road user has lights on; whether the external road user is emitting smoke; whether the external road user's tire tread is above a threshold level; whether the external road user has parts partially detached from the external road user; and whether the external road user obeys traffic rules.

14. The system of claim 10, wherein the identification information includes image processing performed by the computing device to obtain at least one of the following: a license plate number for the external road user; a brand; a model; a color; and a marking.

15. The system of claim 10, wherein the computing device is further configured to: Receive security reports from the server; and Perform the action at the computing device.

16. The system of claim 15, wherein the action includes at least one of: providing a notification to a user interface about a dangerous road user; and automatically performing vehicle manipulation at the vehicle.

17. The system of claim 15, wherein the safety report includes areas to be avoided, the areas to be avoided being based on a safety score below a safety threshold level and a confidence level above a confidence threshold level being in the areas.

18. A non-transitory computer-readable medium for storing instruction code, said instruction code, when executed by a system, causing the system to: Using the vehicle's sensors, sensor data related to the behavior of external road users is obtained; The sensor data is provided to an abstraction layer between the computing device used in the vehicle and the sensors of the vehicle to generate abstract data; The abstract data is provided to the integrated sensors to generate a normalized safety score for the external road user, wherein the normalized safety score is determined based on one or more dangerous driving modes; Record identification information for the external road users; and Report the normalized security score and identification information to the server; and The server is also configured to: Receive a first report from the vehicle and a second report from another reporting vehicle; In response to the fact that the external road user has moved from a first location in the first report to a second location in the second report within the time difference between the first report and the second report, and that some identification information in the first report and the second report overlaps, it is determined that the first report and the second report belong to the same external road user; as well as Increase the confidence level of the normalized safety score associated with the external road users.

Citation Information

Patent Citations

  • Method and system for traffic behavior detection and warnings

    US20200342753A1