Safe driving system generating map points
The method uses GPS and AI to detect and alert drivers of approaching hazards, reducing distraction and enhancing safety at traffic lights, school zones, and railroad crossings.
Patent Information
- Application Number
- PCT/US2025/041801
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2024-08-13
- Filing Date
- 2025-08-13
- Publication Date
- 2026-02-19
AI Technical Summary
Driver distraction, particularly from using mobile phones during driving, significantly increases the risk of accidents, with existing hands-free systems offering little safety improvement.
A computer-implemented method using GPS, camera systems, and AI to detect approaching points of interest like traffic lights, school zones, and railroad crossings, providing timely alerts and integrating with vehicle navigation systems to enhance safety.
Reduces driver distraction by offering timely and contextually relevant alerts, improving safety at critical driving locations, even during hands-free voice calls.
Smart Images

Figure US2025041801_19022026_PF_FP_ABST
Abstract
Description
SAFE DRIVING SYSTEM GENERATING MAP POINTSCROSS-REFERENCE TO RELATED APPLICATIONS
[0001] This application claims the benefit of U.S. provisional application Serial No. 63 / 682,485 filed August 13, 2024, the disclosure of which is hereby incorporated in its entirety by reference herein.TECHNICAL FIELD
[0002] In at least one aspect, the invention relates to a warning system to assist a driver to pay attention to road conditions.BACKGROUND
[0003] In today’s crowded driving environment, distraction is a growing threat and a major contributor to crashes. In 2025, its impact is especially pronounced in several states. New Mexico leads by a wide margin, with 39.86% of all traffic fatalities involving distracted driving, far above the national average of about 8%. Other states with elevated shares include New Jersey (26.48%), Kansas (24.18%), Hawaii (20.72%), Kentucky (17.46%), Louisiana (16.90%), Washington (13.46%), Texas (11.09%), Idaho (15.46%), and Wyoming (8.47%), each at or above the national rate. These disparities point to regional differences likely driven by variation in state laws, enforcement, and public awareness. Although some drivers use hands free devices to mitigate distraction, evidence indicates the risk of a crash increases roughly fourfold while talking on a mobile phone, and hands free use, whether the phone is in a mount or receptacle or integrated into a vehicle’s infotainment system, offers no meaningful safety advantage.
[0004] A significant solution can be found U.S. Pat. Nos. 7,308,247 and 7,986,934, both titled "Cellular Telephone Safety System" and U.S. Pat. No. 9,568,334 titled “Safe Driving System Generating Map Points, each describing a system for detecting when a traffic signal is near. The systems detect when a wireless communication device is in an active voice mode and issues an alarm which can be used to warn that the traffic light is being approached. Such a system may or may notissue an alarm only when the traffic light is red or is calculated to be red by the time the vehicle reaches the intersection. Such a system can also be used to generate a warning when the vehicle is approaching a school zone or a railway crossing. The teachings of U.S. Pat. Nos. 7,308,247, 7,986,934, and 9,568,334, and the references cited therein are incorporated by reference herein. In the latter patent, vehicle GPS navigation systems and mobile GPS navigation systems display an icon of the traffic light on a map generated by the system. Such systems can display icons showing locations on the map of various other points of interest (POIs) chosen by the user, such as ATMs, restaurants, fire stations, police stations, emergency rooms, and the like.SUMMARY
[0005] In at least one aspect, a computer-implemented method for improving safety for use with a navigation map system in a vehicle is provided. The method includes a step of obtaining a location and / or pose for vehicle from one or more positioning sources. The method also includes a step of accessing data indicative of a point of interest (POI) location or receiving image data sufficient to estimate a relative position of the POI. The method also includes a step of determining whether the POI 16 is within a predetermined proximity to the vehicle based on at least one of the vehicle location / pose and / or the estimated relative position. Optionally, a determination is made if vehicle is moving above a configurable speed threshold and / or is approaching the POI. Image data is received by vision-processing module from a camera system. Typically, the vision-processing module processes the image data to detect the POI and estimate its relative position as part of a proximity determination. For example, the proximity determination can be an estimated distance between the vehicle and the POI or an estimation of the time for the vehicle to reach the POI. In some variations, the image data (and any available SPAT updates) are time-stamped and time-aligned within a synchronization tolerance to ensure temporally consistent fusion. An alert, and in particular, an audible alert is provided when the (optionally fused) proximity determination is within a predetermined distance and any approach / speed condition is satisfied.
[0006] In another aspect, the present application can be referred to as a distracted driver alert application (“DDA Application”), which enhances the foregoing systems. One enhancement is thedownloading of position data monolithically (e.g., all the data within a predetermined radius), so that the device will not be required to poll the data server continuously. In one variation, the DDA Application maintains a local cache / database of the locations of traffic lights or other points of interest (POIs) on the device so that alerts can be generated immediately. The DDA Application can query the user to update the local cache / database of POIs each time it starts up, or it can automatically request smaller incremental updates during operation. In another variation, information about a traffic signal is generated by the traffic light itself, or by a nearby signal generator. Information about the location of the traffic light (MAP data) is generated as well as information about the signal phasing and timing (SPAT), the former being static, the latter being dynamic. MAP and SPAT data can also be obtained from commercial or governmental sources. In some variations, SPAT data is pre-processed on a remote server to validate formatting, integrity, and source reliability, then transmitted to the vehicle or mobile device in compact form for final real-time processing and fiision. In latency-sensitive configurations — such as when SPAT must be fused with camera detections at sub-second intervals — SPAT data is delivered via low-latency channels (e.g., direct cellular V2X or DSRC) to an onboard Al module or mobile application, where the fusion occurs locally to meet 100 ms update requirements. The server may also provide supplemental predictive SPAT data for non-real-time analytics and historical modeling, which is distinct from the live SPAT stream used for immediate alerting. The technology of the DDA Application can save lives. In certain variations, the DDA Application and driver distraction alert system are designed to enhance safety at traffic lights, school zones, and railroad crossings by combining GPS for determining vehicle position relative to geofenced POIs, cameras for capturing real-time roadway imagery, and, in some variations, LiDAR for providing depth and range measurements to validate camera detections and enhance accuracy in low-visibility conditions. When deployed on a mobile device, only lightweight processing (such as GPS-based proximity checks, alert triggering based on pre-fused data, and displaying notifications) is performed locally. Computationally intensive Al or neural network processing of camera and / or LiDAR inputs is performed on dedicated in-vehicle hardware (e.g., an onboard Al module integrated into the infotainment or ADAS system) or on a connected edge-compute unit. An artificial intelligence (Al) module fuses data from GPS, cameras, and optionally LiDAR to identify and classify traffic lights, stop signs, and other control devices with high accuracy, even in challenging conditions. The Almodule may also apply geofencing technology to create virtual boundaries around school zones and railroad crossings. A notification module provides visual and audible alerts to the driver as they approach these areas, with alerts triggered by the Al module. An integration module interfaces with the vehicle’s navigation system to provide real-time information about school zones, railroad crossings, and traffic lights along the route. Additionally, a voice assistant module offers verbal reminders about these areas, while a contextual awareness module analyzes the driver’s conversation context and driving conditions to deliver relevant alerts. A customization module allows the driver to personalize the types of alerts they receive based on their preferences and driving habits. In some variations, Traffic Light and Stop Sign Control functionality may automatically slow or stop the vehicle when appropriate, particularly during a hands-free voice call, and then allow the vehicle to proceed when safe or commanded by the driver. The DDA Application can also wirelessly communicate the signal phase and timing of a light and provide a “Countdown to Green” voice warning.
[0007] In another aspect, a software development kit (SDK) is provided in which data loading is isolated into the SDK — and only data loading is part of the SDK. Different users of the invention can all use the same SDK. Thus, the SDK will contact a server and load the POI data. In mobile- device-only configurations, the SDK operates solely to load and locally store POI data for lightweight alert logic. In vehicle-integrated configurations, the SDK may interface with the in-vehicle Al module for high-performance processing of sensor data.
[0008] In another aspect, the Driver Distraction Alert system comprises a roadside unit (RSU) processor (AI-500-085) that interfaces with the traffic signal controller by an edge device to receive SPAT messages. The RSU is located sufficiently close to the traffic light to be able to interface with it. An edge device comprises hardware that controls dataflow between two networks. The RSU processor transmits messages via the cellular network to the Server. The driver receives information via an application in their vehicle either directly over the cellular network or connected via Bluetooth. The system activates while on a hands-free voice call at traffic lights but can also be used to warn of school zones and railroad crossings. In a refinement, SPAT data is received at update intervals of approximately 100 ms and is buffered locally so that it can be temporally aligned with camera and,where present, LiDAR sensor data. Camera frames and LiDAR sweeps are timestamped, and the fusion module selects the most recent SPAT sample within a defined synchronization window (e.g., ±50 ms) to ensure that control-state data and visual detections correspond to the same real-world moment. This synchronization reduces false positives and improves accuracy in low-latency scenarios.
[0009] In another aspect, a driver distraction alert system is designed to enhance safety at traffic lights, school zones, and railroad crossings. It includes an artificial intelligence (Al) module that uses geofencing technology to create virtual boundaries around school zones and railroad crossings. A notification module provides visual and audible alerts to the driver as they approach these areas, with alerts triggered by the Al module. The system also features an integration module that interfaces with the vehicle's navigation system to provide real-time information about school zones, railroad crossings, and traffic lights along the route. Additionally, a voice assistant module offers verbal reminders about these areas, while a contextual awareness module analyzes the driver’s conversation context and driving conditions to deliver relevant alerts. Finally, a customization module allows the driver to personalize the types of alerts they receive based on their preferences and driving habits. In a refinement, when the system uses SPAT data in combination with Al- or neural network- based visual detection from cameras and / or LiDAR, all sensor inputs are synchronized using a common clock or timestamping protocol. SPAT updates (100 ms) are queued and matched to the nearest camera frame and LiDAR point cloud within a defined time tolerance, ensuring that the fusion process compares temporally consistent data. This timing alignment is critical for real-time decision- making and is maintained whether processing occurs in-vehicle or on an attached mobile device with hardware acceleration.
[0010] In another aspect, the methods set forth herein reduce driver distraction.
[0011] The foregoing summary is illustrative only and is not intended to be in any way limiting. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features will become apparent by reference to the drawings and the following detailed description.BRIEF DESCRIPTION OF THE DRAWINGS
[0012] For a further understanding of the nature, objects, and advantages of the present disclosure, reference should be made to the following detailed description, read in conjunction with the following drawings, wherein like reference numerals denote like elements and wherein:
[0013] Figure 1 A is a schematic of a system implements a computer-implemented method for improving safety in a vehicle.
[0014] Figure IB is a navigation map with POI overlays.
[0015] Figure 1 C is a navigation map with POI overlays.
[0016] Figure 2 shows a schematic of a system for improving safety for use with a navigation GPS map system; and
[0017] Figure 3 shows a schematic of an Al system for improving safety for use with a navigation GPS map system.
[0018] Figure 4 is a user interface diagram of an embodiment of the system;
[0019] Figure 5 is a flow chart showing the startup in an embodiment of the system; Figure 3 is a flow chart showing the steps in an alert generated by an embodiment of the system;
[0020] Figure 6 is a flow chart showing the steps in an alert generated by an embodiment of the system;
[0021] Figure 7 is a diagram showing in an overlap of first and second rectangular (here, square) maps when data is requested when moving through the first map to the second map;
[0022] Figure 8 is a diagram showing how a subscriber to the system obtains Map and SPAT
[0023] Figure 9 depicts how an embodiment of the system encodes and decodes data, using JavaScript Object Notation (JSON);
[0025] Figure 11 is a flow chart describing SPAT flow of the system
[0026] Figure 12 is the code to generate a token id in an SDK functionality in an embodiment of the invention;
[0027] Figure 13 is the code to get the POI file name for the traffic signal and railway crossing and school zone in North and South American;
[0028] Figure 14 is the code to get the POI file name for the traffic signal and railway crossing and school zone except North and South American; and10029] Figure 15 shows a module to generate Android Archive (AAR) files that can be included in any android project;DETAILED DESCRIPTION
[0030] Reference will now be made in detail to presently preferred embodiments and methods of the present invention, which constitute the best modes of practicing the invention presently known to the inventors. The Figures are not necessarily to scale. However, it is to be understood that the disclosed embodiments are merely exemplary of the invention that may be embodied in various and alternative forms. Therefore, specific details disclosed herein are not to be interpreted as limiting, but merely as a representative basis for any aspect of the invention and / or as a representative basis for teaching one skilled in the art to variously employ the present invention.
[0031] It is also to be understood that this invention is not limited to the specific embodiments and methods described below, as specific components and / or conditions may, of course, vary. Furthermore, the terminology used herein is used only for the purpose of describing particular embodiments of the present invention and is not intended to be limiting in any way.
[0032] It must also be noted that, as used in the specification and the appended claims, the singular form "a," "an," and "the" comprise plural referents unless the context clearly indicates otherwise. For example, reference to a component in the singular is intended to comprise a plurality of components.
[0033] The term “comprising” is synonymous with “including,” “having,” “containing,” or “characterized by.” These terms are inclusive and open-ended and do not exclude additional, unrecited elements or method steps.
[0034] The phrase “consisting of’ excludes any element, step, or ingredient not specified in the claim. When this phrase appears in a clause of the body of a claim, rather than immediately following the preamble, it limits only the element set forth in that clause; other elements are not excluded from the claim as a whole.
[0035] The phrase “consisting essentially of’ limits the scope of a claim to the specified materials or steps, plus those that do not materially affect the basic and novel characteristic(s) of the claimed subject matter.
[0036] With respect to the terms “comprising,” “consisting of,” and “consisting essentially of,” where one of these three terms is used herein, the presently disclosed and claimed subject matter can include the use of either of the other two terms.
[0037] It should also be appreciated that integer ranges explicitly include all intervening integers. For example, the integer range 1-10 explicitly includes 1, 2, 3, 4, 5, 6, 7, 8, 9, and 10. Similarly, the range 1 to 100 includes 1, 2, 3, 4. . . . 97, 98, 99, 100. Similarly, when any range is called for, intervening numbers that are increments of the difference between the upper limit and the lowerlimit divided by 10 can be taken as alternative upper or lower limits. For example, if the range is 1.1. to 2.1 the following numbers 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9, and 2.0 can be selected as lower or upper limits.
[0038] When referring to a numerical quantity, in a refinement, the term “less than” includes a lower non-included limit that is 5 percent of the number indicated after “less than.” A lower nonincludes limit means that the numerical quantity being described is greater than the value indicated as a lower non-included limited. For example, “less than 20” includes a lower non-included limit of 1 in a refinement. Therefore, this refinement of “less than 20” includes a range between 1 and 20. In another refinement, the term “less than” includes a lower non-included limit that is, in increasing order of preference, 20 percent, 10 percent, 5 percent, 1 percent, or 0 percent of the number indicated after “less than.”
[0039] With respect to electrical devices, the term “connected to” means that the electrical components referred to as connected to are in electrical communication. In a refinement, “connected to” means that the electrical components referred to as connected to are directly wired to each other. In another refinement, “connected to” means that the electrical components communicate wirelessly or by a combination of wired and wirelessly connected components. In another refinement, “connected to” means that one or more additional electrical components are interposed between the electrical components referred to as connected to with an electrical signal from an originating component being processed (e.g., filtered, amplified, modulated, rectified, attenuated, summed, subtracted, etc.) before being received to the component connected thereto.
[0040] The term “electrical communication” means that an electrical signal is either directly or indirectly sent from an originating electronic device to a receiving electrical device. Indirect electrical communication can involve processing of the electrical signal, including but not limited to, filtering of the signal, amplification of the signal, rectification of the signal, modulation of the signal, attenuation of the signal, adding of the signal with another signal, subtracting the signal from another signal, subtracting another signal from the signal, and the like. Electrical communication can be accomplished with wired components, wirelessly connected components, or a combination thereof.
[0041] The term “communication” with respect to software means the modules exchange information.
[0042] The term “one or more” means “at least one” and the term “at least one” means “one or more.” The terms “one or more” and “at least one” include “plurality” as a subset.
[0043] The term “substantially,” “generally,” or “about” may be used herein to describe disclosed or claimed embodiments. The term “substantially” may modify a value or relative characteristic disclosed or claimed in the present disclosure. In such instances, “substantially” may signify that the value or relative characteristic it modifies is within ± 0%, 0.1%, 0.5%, 1%, 2%, 3%, 4%, 5% or 10% of the value or relative characteristic.
[0044] It should also be appreciated that any given signal that has a non-zero average value for voltage or current includes a DC signal (that may have been or is combined with an AC signal). Therefore, for such a signal, the term “DC” refers to the component not varying with time and the term “AC” refers to the time-varying component. Appropriate filtering can be used to recover the AC signal or the DC signal.
[0045] The term “electronic component” refers is any physical entity in an electronic device or system used to affect electron states, electron flow, or the electric fields associated with the electrons. Examples of electronic components include, but are not limited to, capacitors, inductors, resistors, thyristors, diodes, transistors, etc. Electronic components can be passive or active.
[0046] The term “electronic device” or “system” refers to a physical entity formed from one or more electronic components to perform a predetermined function on an electrical signal.
[0047] It should be appreciated that in any figures for electronic devices, a series of electronic components connected by lines (e.g., wires) indicates that such electronic components are in electrical communication with each other. Moreover, when lines directed connect one electronic component to another, these electronic components can be connected to each other as defined above.
[0048] The processes, methods, or algorithms disclosed herein can be deliverable to / implemented by a processing device, controller, or computer, which can include any existing programmable electronic control unit or dedicated electronic control unit. Similarly, the processes, methods, or algorithms can be stored as data and instructions executable by a controller or computer in many forms including, but not limited to, information permanently stored on non-writable storage media such as ROM devices and information alterably stored on writeable storage media such as floppy disks, magnetic tapes, CDs, RAM devices, and other magnetic and optical media. The processes, methods, or algorithms can also be implemented in a software executable object. Alternatively, the processes, methods, or algorithms can be embodied in whole or in part using suitable hardware components, such as Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), state machines, controllers or other hardware components or devices, or a combination of hardware, software and firmware components.
[0049] As with reference to the Figures, the same reference numerals may be used herein to refer to the same parameters and components or their similar modifications and alternatives. For purposes of description herein, the directional terms “upper,” “lower,” “right, ” “left, ” “rear, ” “front, ” “vertical, ” “horizontal, ” and derivatives thereof shall relate to the present disclosure as oriented in Figures 1 and 4. However, it is to be understood that the present disclosure may assume various alternative orientations, except where expressly specified to the contrary. It is also to be understood that the specific devices and processes illustrated in the drawings and described in the following specification are simply exemplary embodiments of the inventive concepts defined in the appended claims. Hence, specific dimensions and other physical characteristics relating to the embodiments disclosed herein are not to be considered as limiting, unless the claims expressly state otherwise. The drawings referenced herein are schematic and associated views thereof are not necessarily drawn to scale.
[0050] Today’s drivers are spending more time in their cars than ever before. They eat, talk on the phone, put on makeup, and do their hair, but when is it too much? As described in the Background of the Invention, numerous studies have found that driver distraction can be a major factor in traffic accidents and calls out cell phone use, specifically, as one of the major distractions. Some drivers usehands free cell phone devices to mitigate this distraction, but as previously indicated the risk of having an accident increases fourfold while talking on a mobile phone and that using a hands-free phone is not any safer.
[0051] With reference to Figure 1 A, a computer-implemented method for improving safety for use with a navigation map system in a vehicle is provided. The method includes a step of obtaining a location and / or pose for vehicle 10 (e.g., the vehicle location) from one or more positioning sources 12. In a refinement, the one or more positioning sources comprise a global navigation satellite system (GNSS) receiver and optionally an inertial measurement unit (IMU). The method also includes a step of accessing data indicative of a point of interest (POI) location 14 or receiving image data sufficient to estimate a relative position of the POI 16. The method also includes a step of determining whether the POI 16 is within a predetermined proximity to the vehicle based on at least one of the vehicle location / pose and / or the estimated relative position. Optionally, a determination is made if vehicle 10 is moving above a configurable speed threshold and / or is approaching the POI. Image data is received by vision-processing module 22 from a camera system 20. Typically, the vision-processing module 22 processes the image data to detect the POI 16 and estimate its relative position as part of a proximity determination. For example, the proximity determination can be an estimated distance between the vehicle and the POI 16 or an estimation of the time for the vehicle to reach the POI 16. In some variations, the image data (and any available SPAT updates) are time-stamped and time-aligned within a synchronization tolerance to ensure temporally consistent fusion. An alert, and in particular, an audible alert is provided when the (optionally fused) proximity determination is within a predetermined distance and any approach / speed condition is satisfied. In a refinement, the method reduces driver distraction , and in particular during an active hand free voice call. Typical, a navigation map is displayed on navigation map system 18.
[0052] In another aspect, the audible alert and / or the navigation map is provided routed through a cell phone and / or an infotainment system such that a hands-free call optionally flow through the infotainment system.
[0053] Figures IB and 1C each show a navigation map with a standard basemap. The map includes overlays for driving-relevant points of interest. These points of interest include traffic-signal icons at intersections. They also include clusters of school / crosswalk warning icons around sensitive zones. A location marker indicates the vehicle's position. A floating signal-status widget displays the current or expected state of a nearby signal when data are available. This widget features a three-light stack and a timer. A top toolbar offers settings and layer / filter controls. Typical zoom buttons appear on the map. In the distracted-driver system, this screen serves as the human-machine interface. GNSS identifies relevant POIs ahead. The system may optionally use camera or SPAT inputs for this identification. The app estimates proximity or time-to-arrival. It issues audible alerts when thresholds are met. This allows the driver to keep eyes on the road. The icons show what can trigger alerts. The status widget reflects the signal state or countdown. The system may announce this information.
[0054] In another aspect, the vision-processing module 22 is a program running on a computing device 24 (e.g., a vehicle electronic control unit (ECU), an advanced driver-assistance system (ADAS) domain controller, an infotainment head unit, or any other computing device) configured to receive the image data from a camera system 20, to detect the POI 16, and to estimate its relative position as a proximity determination. Alternatively, the vision-processing module 22 is a hardware component, such as an ASIC, FPGA, or GPU accelerator integrated within a vehicle ECU or domain controller, configured to receive the image data from a camera system 20, to detect the POI 16, and to estimate its relative position as a proximity determination.
[0055] In another aspect, the method further includes map-matching the vehicle location to a road segment, wherein determining whether the vehicle is moving involves comparing a vehicle speed to a speed threshold, wherein determining whether the POI is within the predetermined proximity comprises computing a time-to-arrival based on the vehicle speed and a range estimate to the POI and comparing the time-to-arrival to a threshold, and wherein the predetermined proximity is adaptively varied based on at least one of weather, ambient light level, vehicle speed, road class, or a quality of the position estimate.
[0056] In another aspect, one or more Advanced Driver- Assistance Systems (ADAS) sensors 26 are used for at least one of (i) detecting the POI, (ii) estimating a range, bearing, or time-to-arrival to the POI, (iii) determining whether the vehicle is moving based on vehicle-motion signals, and (iv) validating the proximity determination, the ADAS sensors 26 include at least one of a camera 20, a LiDAR sensor 261, a millimeter-wave radar sensor 262, an ultrasonic sensor 263, an inertial measurement unit 264, a wheel-speed sensor 265, a steering-angle sensor 266, or a yaw-rate sensor 267. In a refinement outputs from at least two ADAS sensors are processed to produce the proximity determination, with range and bearing to the POI obtained from a LiDAR sensor or a millimeter- wave radar sensor when available and otherwise derived from vision-based depth, and wherein individual sensor contributions are selected, weighted, or suppressed based on ambient conditions or a sensor self-test result. In this regard, a camera captures images of the scene so software can read lanes, signs, lights, pedestrians, and vehicle shapes; it’s great for classification and lane geometry. A LiDAR sensor fires laser pulses to build a precise 3D map, giving exact distance and shape of nearby objects, day or night. A millimeter-wave radar sensor emits radio waves to measure an object’s range, angle, and relative speed (Doppler), working reliably in rain, fog, and darkness. An ultrasonic sensor measures very short-range distance (centimeters to a few meters), ideal for parking and curb detection. An inertial measurement unit (IMU) combines accelerometers and gyroscopes to track the car’s linear acceleration and rotation, stabilizing estimates between GPS fixes. A wheel-speed sensor reports each wheel’s rotational speed for vehicle speed, ABS / traction control, and slip detection. A steering-angle sensor tells how much the front wheels are turned, revealing the driver’s intended path for lane keeping and path prediction. A yaw-rate sensor measures how quickly the vehicle is rotating around its vertical axis, crucial for stability control and comparing where the car is pointing vs. where it’s actually going.
[0057] In another aspect, receiving image data comprises receiving frames from a forwardfacing monocular camera, and processing the image data comprises executing an anti-flicker routine configured to detect a state of a traffic signal, wherein the POI is selected from the group consisting of a traffic signal, a stop sign, a pedestrian crosswalk, a railroad crossing, a school zone, and a work zone, and further comprising receiving infrastructure message data indicative of signal phase and timing (SPAT) for the POI and fusing the infrastructure message data with an output of the vision-processing module, the fusing comprising temporally synchronizing the infrastructure message data and the image data (e.g. within about ±50 milliseconds) and buffering the infrastructure message data to compensate for network latency, and further comprising computing a confidence score for the POI detection.
[0058] In another aspect, the audible alert is provided together with a vibration or other tactile alert. In a refinement, the audible alert and / or the vibration is provided by a user’s cell phone. In another aspect, a vehicle accelerometer system 28 is configured to initiate the audible alert. In a refinement, the vehicle accelerometer system 28 is configured to slow down the vehicle while a voice call is active. In another refinement, the audible alert includes a sound indicative of characteristics of the point of interest.
[0059] In another aspect, the method further includes steps of determining that the vehicle is moving faster than ten miles per hour; determining, from sensors in a wireless communication device or the vehicle, an acceleration or a change in motion; and allowing the vehicle to be commanded to slow down via a wireless network during an active voice call.]0060| In another aspect, the method further includes steps of determining whether a wireless communication device is in an active voice mode and, responsive to a positive determination, displaying an icon of the point of interest. In a refinement, the method further includes a step of displaying an icon of a phone only when the wireless communication device is in an active voice mode. In a further refinement, in addition to displaying an icon of the point of interest, an audible alarm is sounded. In still a further refinement, a navigation map displays a line from a present vehicle location to the point of interest.
[0061] In another aspect, the point of interest is a traffic light and the icon is an icon of a traffic light. In a refinement, the method further includes a step of determining whether the traffic light is yellow or red, or predicting that it will be yellow or red when the vehicle reaches the traffic light, and displaying the icon only if the determination is positive.
[0062] In another aspect, the point of interest is a school zone and the icon is an icon of a school.
[0063] In another aspect, the point of interest is a railway crossing and the icon is an icon of a railway crossing barrier.
[0064] In another aspect, the method further includes a step of receiving distance measurements from a LiDAR sensor mounted to the vehicle and estimating a range and bearing to the point of interest. In a refinement, the method further includes time-stamping LiDAR measurements and synchronizing them with camera image data and / or SPAT data within a synchronization tolerance window. In a refinement, the method further includes fusing camera-based detection with a LiDAR- derived range and / or bearing to determine proximity to the point of interest. Typically, the fusing weights LiDAR more heavily under low-illumination, glare, fog, precipitation, or other reduced- visibility conditions. In another refinement, the method further includes transforming LiDAR points from a vehicle coordinate frame into geographic coordinates using a vehicle pose estimate and storing resulting longitude / latitude in a same format as a point-of-interest database.
[0065] In another aspect, the system employs LiDAR as a fallback primary sensing input whenever the camera system is unavailable, saturated, or below a configured quality threshold. LiDAR returns are time-stamped and processed to estimate range and bearing to the point of interest and to maintain a temporally filtered track that bridges short-term occlusions using successive scans. A confidence score is computed from LiDAR signal quality, consistency across frames, and agreement with vehicle pose, and the audible alert is issued only when the score exceeds a threshold. The LiDAR sensor may comprise a two-dimensional scanning LiDAR or a three-dimensional LiDAR. In certain refinements, LiDAR-derived intersection geometry is cross-validated against SPAT data identifying the intersection to confirm the point of interest prior to issuing the audible alert.
[0066] In another aspect, a method and system are designed to improve safety for navigation GPS systems used in vehicles where a wireless communication device, like a cell phone, is in operation. In a refinement, this method is computer-implemented. This system can provide alerts aboutapproaching points of interest, such as traffic lights, school zones, and railway crossings. First, the method involves determining the current location of the vehicle and the GPS location of the point of interest. It assesses whether the vehicle is within a predetermined distance from the point of interest and whether the vehicle is in motion. In a refinement, it is also determined if the wireless communication device is in an active voice mode whereupon an icon of the point of interest is displayed only if such determination is positive. When the vehicle approaches a point of interest, an audible alert is provided, which can be accompanied by a vibration or another tactile alert. In a refinement, the audible alert includes a sound that indicates characteristics of the point of interest. For example, the clickety-clack or bell sound can be played, indicating an approaching railroad crossing. This can be accompanied by a voice saying, “Railroad crossing.” Similarly, an alarm sound for an approaching traffic light can be played accompanied by a voice saying “traffic light.” In some cases, a voice counting down when a traffic light will turn green or red can be voiced.
[0067] In another aspect, the alert and / or vibration and / or other tactile alert can be generated by the user's cell phone. In a variation, the system can utilize the vehicle’s accelerometer to trigger the alert and, if necessary, prompt the vehicle to slow down, especially during voice calls.
[0068] In another aspect, the method can also be configured to determine if the vehicle is traveling faster than ten miles per hour and if sensors in the wireless communication device or the vehicle detect acceleration changes, allowing the vehicle to slow down via the wireless network while on an active voice call. Additionally, when the wireless communication device is in active voice mode, an icon of the point of interest is displayed, alongside an icon of a phone. An audible alarm is also sounded. The map system displays a line from the vehicle's current location to the point of interest, with specific icons representing each point of interest: a traffic light, a school, or a railway crossing barrier. For traffic lights, the system can determine the light’s current state — whether it is yellow or red — or predict its state when the vehicle reaches it. The icon for the traffic light is displayed only if this assessment is positive, ensuring the driver is aware of the traffic conditions ahead.
[0069] Figure 2 provides a schematic system 30 in which the vehicle’s accelerometer 32 is in electrical communication with the electronic control unit (ECU) 34. ECU 34 controls and is inelectrical communication with the wireless interface 36, which can send alerts and other signals to a user’s wireless device 38. In a refinement, wireless interface 36 is integrated into ECU 34.
[0070] In another aspect, advanced Al technology and / or neural network-based technology is integrated to enhance safety at traffic lights, school zones, and railroad crossings, especially when a driver is on a voice call. Al provides timely alerts and reminders to ensure the driver's attention remains focused on the road, thereby reducing the risk of accidents in these critical areas. The Driver Distraction Alert (DDA) system leverages Al-powered geofencing technology to create virtual boundaries around designated school zones and railroad crossings. In some refinements, the Al module includes a convolutional neural network (CNN)-based computer vision engine to detect and classify traffic lights and stop signs from camera input. When the vehicle approaches these areas, the Al system triggers alerts to notify the driver of upcoming hazards. This functionality remains active even during voice calls, ensuring the driver is continually aware of potential dangers. To further enhance driver awareness, the Al system provides both visual and audible alerts, displaying warnings on the vehicle's dashboard or heads-up display and playing alerts through the vehicle's speakers. These alerts ensure that the driver receives clear notifications about the presence of school zones, railroad crossings, and traffic lights, even when engaged in a conversation. Al integration with the vehicle's navigation system allows for real-time information about school zones, railroad crossings, and traffic lights along the route, offering proactive alerts to ensure the driver is aware of upcoming hazards well in advance. This real-time data integration helps the driver take necessary precautions, such as slowing down or preparing to stop. Al-powered voice assistants in the DDA system provide verbal reminders about the presence of school zones, railroad crossings, and traffic lights, maintaining the driver's focus on the road and preparing them for necessary actions, such as slowing down or stopping, thereby enhancing overall road safety. Al algorithms analyze the context of the driver's conversation and driving conditions to deliver contextually relevant alerts. If both SPAT-derived traffic signal state data and neural network-based visual detection are available, the system may perform data fusion to validate results or select the most reliable source. For example, if the driver is on a voice call and approaching a school zone, the Al system prioritizes delivering alerts to direct the driver's attention to the road. This contextual awareness ensures that alerts are timely and relevant to the current driving scenario.Additionally, the DDA system allows drivers to customize the types of alerts they receive based on their preferences and driving habits, ensuring that alerts are effective and not overly distracting, thereby enhancing the driver's overall experience and safety. In a refinement, the audible alert includes a sound that indicates characteristics of the point of interest. For example, the clickety-clack or bell sound can be played, indicating an approaching railroad crossing. This can be accompanied by a voice saying, “Railroad crossing.” Similarly, an alarm sound for an approaching traffic light can be played accompanied by a voice saying “traffic light.” In some cases, a voice counting down when a traffic light will turn green or red can be voiced.
[0071] While using Al and / or neural network-based computer vision to determine if a traffic light is about to change from red to green during a voice call can be potentially dangerous and should be approached with caution, Al can be used responsibly to enhance safety. Al can detect and recognize traffic lights in real time using a camera mounted on the car, a technology commonly employed in autonomous vehicles. A trained neural network, such as a convolutional neural network (CNN), processes camera images to identify the state of the light (red, yellow, green) and track its changes over time. Al algorithms analyze images of traffic lights to determine their current state (red, yellow, green) and predict when they might change using computer vision techniques. Al models can be trained on extensive datasets of traffic light images to learn patterns and make predictions about changes based on time of day, traffic patterns, and more. In some variations, this visual detection output is cross-checked with SPAT-based traffic light state data, and discrepancies may trigger a validation flag or fallback to the more reliable source. When the Al predicts a light change, it can alert the driver through a safe channel, such as visual or auditory alerts on the car's dashboard. It is crucial to ensure that using Al in this context does not compromise safety. Focusing on driving and obeying traffic rules should always be prioritized to ensure safety. For those interested in exploring Al applications in driving scenarios, advanced driver-assistance systems (ADAS) are designed to enhance driver safety rather than distract from it.
[0072] Creating and maintaining safe systems for generating map points can benefit significantly from artificial intelligence (Al) in several ways. Al-powered drones and autonomous vehicles can efficiently collect geographical and environmental data, reducing the need for humanintervention. Al can combine data from various sensors, such as LiDAR, cameras, and GPS, to create a comprehensive and accurate map. Machine learning algorithms can identity and correct errors or inconsistencies in raw data, ensuring higher quality inputs for map generation. Computer vision algorithms can analyze images to identify and classify objects like roads, buildings, and natural features, essential for creating detailed maps. Al can detect patterns and anomalies in geographical data, such as changes in terrain or new construction, which might be missed by traditional methods. Al can process data in real-time, allowing for continuous updating of maps as new data arrives, which is crucial for applications requiring up-to-date information, like autonomous driving. Al can also predict changes in the environment, such as traffic patterns and weather impacts, and update the map accordingly. Al algorithms perform continuous checks to ensure the accuracy and reliability of map data, identifying potential errors before they cause problems. Al can assess the safety of routes and areas, flagging potentially dangerous zones, such as high-crime areas and zones prone to natural disasters. Al can tailor maps to individual users’ needs, providing customized routes and points of interest based on user preferences and behavior. Al can enhance user interaction with the map system through advanced voice commands and augmented reality (AR) overlays. Al can calculate the most efficient routes, taking into account real-time traffic data, road conditions, and other variables. Al can optimize the use of resources, such as battery life for drones and processing power for servers, to maintain system efficiency. Al can implement robust security measures to protect sensitive geographical data from cyber threats and anonymize data to protect users’ privacy while still providing accurate map information. Al can integrate map systems with the Internet of Things (loT) devices, such as smart traffic lights and connected vehicles, for a more cohesive infrastructure. Al can also enhance map applications with AR, providing users with real-time, interactive visual information overlaid on their physical surroundings. By integrating Al into the patent for a safe system generating map points, the technology can achieve greater accuracy, efficiency, and safety, ultimately providing a more reliable and user-friendly mapping solution.
[0073] Referring to Figure 3, a schematic of a driver distraction alert Al system integrating Al is provided. The system can be designed to enhance safety at traffic lights, school zones, and railroad crossings by incorporating a comprehensive array of components and features. In one refinement, theAl system is implemented in software. In another refinement, the Al system is implemented with a hardware component. Al system 40 includes an artificial intelligence (Al) module 42 configured to apply geofencing technology to create virtual boundaries around school zones and railroad crossings. The Al module 42 may further include a neural network-based visual recognition subsystem, such as a convolutional neural network (CNN), trained to detect and classify traffic lights and stop signs from camera input. This visual recognition output may be fused with SPAT-derived traffic signal state data to validate results, resolve discrepancies, and select the most reliable source before presenting information to the driver. Al module 42 recognizes road signs, detects pedestrians, and identifies changes in traffic conditions using advanced computer vision techniques. Additionally, it analyzes historical traffic data and driver behavior to predict when the driver is likely to encounter a traffic light, adjusting the route or driving behavior accordingly. In the event of an emergency or if the driver fails to respond to alerts, the Al module 42 can take over control of the vehicle or contact emergency services. Moreover, it monitors real-time data from vehicle sensors to detect signs of driver distraction, such as drowsiness, inattention, or phone usage, and uses facial recognition technology to monitor the driver's face for signs of drowsiness, fatigue, or distraction. By analyzing driver behavior patterns over time, the Al module 42 can identify deviations that indicate distraction and trigger appropriate alerts. Furthermore, it integrates with other vehicle safety features, such as collision avoidance systems or lane departure warning systems, to provide a comprehensive safety net for the driver. The Al module 42 also facilitates integration with smart city infrastructure, receiving real-time updates on traffic light status, road closures, and other pertinent information. Finally, it offers personalization by learning from the driver’s behavior over time, adjusting alert settings, notification preferences, and response strategies to align with the driver’s unique needs and driving style. The following provides a pseudocode example of Al Module 42 using geofencing technology to detect and create virtual boundaries around school zones and railroad crossings:Module AI Module:Define geofence zones as a list of geofence areas with coordinates Function InitializeGeofences():For each zone in geofence zones:Create virtual boundary using GPS coordinatesFunction DetectZone(location):For each zone in geofence zones:If location is within zone boundary:Return zoneReturn None
[0074] In another aspect, system 40 can include a notification module 44 configured to provide visual and audible alerts to a driver approaching school zones and railroad crossings, with alerts triggered by the Al module 42. Typically, notification module 44 communicates with Al module 42. When traffic light status is involved, notification module 44 receives fused results from the Al module 22, which combines SPAT-derived data with neural network-based visual detection to produce the most reliable state information before issuing an alert. Notification module 44 is fiirther configured to display visual warnings on a vehicle's dashboard or heads-up display and play audible alerts through a vehicle's speakers. It offers proactive alerts, ensuring the driver is aware of upcoming hazards well in advance. The following is an example of pseudo-code for notification module 44 which provides visual and audible alerts when the driver approaches specific zones:Module Notification Module:Function TriggerAlert(zone, alertType):If alertType is "Visual":Display visual alert on the vehicle's dashboardElse if alertType is "Audio":Play audible alert through vehicle's speakersFunction AlertDriver(location): zone = AI Module.DetectZone(location) If zone is not None:TriggerAlert(zone, "Visual") Trigger Alert(zone, "Audio")
[0075] In another aspect, a voice assistant module 46 communicates with Al module 42 and is configured to provide verbal reminders about the presence of school zones, railroad crossings, and traffic lights. It also enables hands-free operation of the vehicle's systems, allowing the driver to stay focused on the road while managing calls, navigation, and other tasks. Additionally, the voice assistant module 46 provides verbal reminders that help the driver prepare for necessary actions, such as slowing down or stopping, and analyzes the driver's voice commands and responses to detect signs of distraction or drowsiness, triggering alerts based on unusual or slurred speech patterns. For traffic light alerts, the voice assistant module 46 uses the Al module’s fused SPAT / visual recognition results to announce the most reliable current or predicted light state, including countdowns to green or red when applicable. The following is an example of pseudo-code for the voice assistant module 46, which offers verbal reminders about upcoming zones and traffic lights:Module Voice Assistant Module:Function ProvideVerbalReminder(zoneType) :If zoneType is "School Zone":Speak "Approaching school zone. Please slow down."Else if zoneType is "Railroad Crossing":Speak "Approaching railroad crossing. Please be cautious."Else if zoneType is "Traffic Light":Speak "Approaching traffic light. Be prepared to stop."Function Announce Alerts(location): zone = AI Module.DetectZone(location)If zone is not None:ProvideVerbalReminder(zone.type)
[0076] In another aspect, integration module 48 is in communication with Al module 42 and is designed to interface with a vehicle's navigation system to provide real-time information about school zones, railroad crossings, and traffic lights along a route. For traffic lights, the integration module 48 receives the Al module’s fused SPAT and neural network visual recognition outputs and delivers that combined, validated information to the navigation system for display. Integration module 48 offers proactive alerts to ensure the driver is aware of upcoming hazards well in advance. The following is an example of pseudo-code for the integration module 48, which interfaces with the vehicle’s navigation system to provide real-time information:Module Integration Module:Function ConnectToNavigationSystem():Access vehicle's navigation systemLoad map data with school zones, railroad crossings, and traffic lightsFunction ProvideRealTimeInfo(route):For each point in route:If point is a school zone or railroad crossing:Notify navigation system of upcoming alertFunction UpdateNavigationWithTrafficLights(location):Check for traffic light data at current locationIf traffic light is found:Notify driver via navigation system
[0077] In another aspect, contextual awareness module 50 in communication with Al module 22 analyzes a driver’s conversation context and driving conditions to deliver contextually relevant alerts, prioritizing those that are timely and relevant to the current driving scenario. When a traffic light is relevant to the context, the module uses fused SPAT-derived data and Al / neural network- based camera recognition from the Al module 42 to ensure the most reliable light state is factored intothe alert. The following is an example of pseudo-code for contextual awareness module 50, which analyzes conversation context and driving conditions to deliver relevant alerts:Module Contextual Awareness Module:Define driver conversations as a log of recent driver interactionsDefine driving conditions as current weather, traffic, and road conditionsFunction AnalyzeContext(conversation) :Extract keywords from conversationDetermine if keywords match safety-related topicsFunction AnalyzeDrivingConditions() :Retrieve current weather dataCheck traffic updates Assess road conditionsFunction DeliverContextualAlert():If AnalyzeContext(driver conversations) matches alert criteria:Trigger custom alert based on contextIf AnalyzeDrivingConditions() indicates hazard: Trigger safety alert based on driving conditions
[0078] In another aspect, a customization module 52 in communication with Al module 42 can be included to allow the driver to customize the types of alerts received based on preferences and driving habits. Customization module 52 ensures that alerts are effective and not overly distracting, enhancing the driver's overall experience and safety. In a refinement, customization settings for traffic light alerts can specify whether to prioritize SPAT data, Al / neural network visual detection, or a fused reliability score from the Al module 42. Finally, the Al module 42 includes predictive maintenance features to monitor vehicle performance and predict potential mechanical issues before they becomecritical, contributing to the overall reliability and safety of the vehicle. The following is an example of pseudo-code for customization module 52, which allows drivers to customize alerts based on preferences and driving habits:Module Customization Module:Define driver_preferences as a list of preferred alert settingsFunction LoadDriverPreferencesQ:Retrieve saved preferences from driver profileFunction CustomizeAlertSettings(alertType, preference):Update driver_preferences with new settings for alertTypeFunction ApplyCustomization():LoadDriverPreferencesQFor each preference in driver_preferences:Adjust Notification Module, Voice Assistant Module, and Contextual Awareness Module settings based on preference
[0079] The following is an example of pseudo-code illustrating a main system that coordinates the modules to operate as a cohesive driver distraction alert system:Module DriverDistractionAlertSystem:Initialize AI ModuleInitialize Notification ModuleInitialize Integration ModuleInitialize Voice Assistant ModuleInitialize Contextual Awareness ModuleInitialize Customization ModuleFunction StartSystem():Integration_Module.ConnectToNavigationSystem() Customization_Module.ApplyCustomization() Continuously monitor vehicle's locationWhile vehicle is in motion: current location = GetVehicleLocationQ Notification Module.AlertDriver(current location) Voice_Assistant_Module.AnnounceAlerts(current_location) Contextual_Awareness_Module.DeliverContextualAlert()
[0080] In summary, the driver distraction alert system is composed of several interconnected modules that work together to enhance driving safety. The Al Module establishes virtual boundaries using geofencing technology to detect when a vehicle enters a specified zone, such as school zones or railroad crossings. For traffic lights, the Al module fuses SPAT-derived data with Al / neural network- based camera detection results to determine the most reliable current or predicted state. Upon detection, the Notification Module triggers visual and audible alerts to warn the driver of the approaching zone. The system integrates seamlessly with the vehicle's navigation system through the Integration Module, which provides real-time updates on zones and traffic lights along the route. The Voice Assistant Module complements these alerts by delivering verbal notifications to the driver about upcoming zones and traffic lights. To enhance the relevance of alerts, the Contextual Awareness Module analyzes the driver’s conversation context and current driving conditions, ensuring that notifications are contextually appropriate. Drivers can further tailor the alert system to their needs through the Customization Module, which allows them to set preferences based on their habits and requirements. Finally, the Main System coordinates all these modules, providing a comprehensive and adaptive driver distraction alert system that enhances safety and driver awareness.
[0081] Details of a product implementing various features of the system of figures 1 A, 2, and 3 are set forth below.
[0082] Product Overview. The invention is a mobile alert system that uses a global positioning system (GPS) chip lodged inside cell phones, or in the automobile itself, to alert drivers that they are approaching a traffic light and, in some variations, integrates cameras for capturing real-time roadway imagery and LiDAR for providing depth and range measurements, with an onboard artificial intelligence (Al) module fusing data from GPS, cameras, and LiDAR to detect and classify traffic lights, stop signs, and other control devices in real time, even in low-visibility conditions. This is accomplished by an early warning system designed to assist drivers engaged in a phone call to pay attention to road conditions. The system will alert the driver by playing a distinctive sound when the following conditions are met:1) The DDA Application has been activated by the user.2) a voice call is in progress (e.g., for mobile phones) alternatively, in an variation, the system is always on.3) the device is moving faster than a predefined velocity (e.g., 10 MPH)4) the device is approaching a “point of interest” (POI)5) The arrival time at the POI being approached is calculated to be within e.g., 150 meters.
[0083] The system uses location-based services on the above-mentioned chip - typically a combination of GPS enhanced with network augmentation to track the device’s coordinates and, in some variations, integrates data from cameras and optional LiDAR sensors processed by an onboard artificial intelligence (Al) module to visually detect and classify points of interest in real time, supplementing GPS-based detection. Points of interest, which will generate an alert, may include stop lights, school zones, construction zones, traffic accident areas, etc. The sound is played on the output speaker currently in use, e.g., earpiece / speaker / headset / hands-free kit. The DDA Application provides a minimal user interface to allow the user to enable / disable the DDA Application. If required, the user interface can display a “splash” screen that provides copyright information, and / or a legal disclaimer.
[0084] If the system requires explicit user permission to grant the DDA Application access to positioning data, the DDA Application also provides the user interface for this. If the device provides a standard user interface for this purpose, the DDA Application will use it. The DDA Application provides the ability for the user to download coordinate data monolithically so that the device will notbe required to poll the data server continuously. For example, the data can be obtained within a radius of from 10 miles to 500-miles or more, Aside from the minimal user interface described above the DDA Application will run in the background and not be visible at all. The system relies on position data that is updated from a remote server and stored on the device, to maximize responsiveness, and in some variations supplements this position data with real-time imagery from cameras and optional LiDAR sensors processed by an onboard artificial intelligence (Al) module, enabling immediate detection and classification of points of interest without relying solely on preloaded coordinate data. The system updates local data through an application programming interface (API) provided by the device operating system, or installed GPS interface.
[0085] Hardware Requirements. The hardware requirements for the system are:(1) A mobile device capable of retrieving real-time geographical positional data (i.e., GPS positioning) and, in some variations, receiving and processing imagery from integrated cameras and optional LiDAR sensors through an onboard artificial intelligence (Al) module.(2) A programmable device, which can run a background process.(3) A device with the ability to simultaneously play an audio warning.(4) For data updates the device should be able to retrieve data from an internet server.(5) The device must be able to do these things during a voice call.
[0086] Architectural Overview. The system uses a client-server program. The client side of the system will run on the user’s mobile device, or on the above-mentioned chip, to retrieve location informatio[ from the device operating system, retrieve point of interest data, process real-time imagery from cameras and, in some variations, depth data from optional LiDAR sensors through an onboard artificial intelligence (Al) module to detect and classify points of interest, and trigger alerts on the device according to the rules described above, in an event-driven manner. Events are conditions that occur at unspecified, or asynchronous, times - such as reception of a GPS update, receipt of AI- processed camera or LiDAR detection results, or reception of data from the remote server. Since a location-based application is naturally event driven, the DDA Application is designed to work in this manner.
[0087] The DDA Application will maintain a local cache / database of POIs on the device so that alerts can be generated immediately. The DDA Application can query the user to update the local cache / database of POIs each time the DDA Application starts up, or it can automatically request smaller incremental updates during operation if the device supports it. Since companies may charge the user to download data according to a data plan, and a data connection may not always be available, the DDA Application should support downloading a large chunk of data at one time to store locally. The user should also be able to configure the program to update data automatically. In Al- and / or neural network-enabled variations, the local cache / database may also store dynamically detected POIs generated in real time by Al analysis of camera and, optionally, LiDAR data, along with associated classification labels, confidence scores, and timestamps. These Al-detected POIs can be merged with static map data so that both sources contribute to alerts. The Server program will respond to positional requests from the client by sending the requested points of interest.
[0088] User Interface. The user interface of the DDA Application is that portion of the DDAApplication which the users sees when they start the DDA Application, and with which they interact directly. The major functionality of the user interface is to allow the user to “activate” and “deactivate” the DDA Application and to update the local data store. After the DDA Application is initially activated, it will continue to run in “background mode” and restart automatically in background mode each time the device is restarted. The user will only need to run the user interface to modify their settings or deactivate the program. In Al- and / or neural network-enabled variations, the user interface may also present options for enabling or disabling Al-based detection modes, adjusting camera and optional LiDAR sensitivity, reviewing recently detected dynamic POIs with associated confidence scores, and prioritizing certain types of Al-detected alerts (e.g., traffic lights, school zones, railroad crossings). These controls allow the user to fine-tune Al and sensor integration according to personal preferences and driving conditions.
[0089] Splash Screen and Legal Disclaimer. Figure 4 is a diagram of a user interface of an variation of the system. Following a brief display of a splash screen, the system displays a legal disclaimer which the user must accept to proceed. The disclaimer informs the user that ultimately it is their responsibility to drive safely and that limitations of the technology may cause the device to notplay an alert when an alert would have been expected to play. This could happen because of not having a connection to the GPS satellites, not having the data of the POI in the database, interference with another program (i.e., contention for the GPS or audio resources), location outside of map boundaries, or failure to manually update data when automatic updates are disabled. It can also state that it is possible that even if the alert sounds, they may not hear it due to other distractions. For AI- and / or neural network-enabled variations, the disclaimer may further inform the user that Al-based camera analysis and optional LiDAR sensing are subject to environmental and technical limitations, including but not limited to poor visibility, adverse weather, occluded objects, sensor misalignment, or misclassification errors. The disclaimer can note that AI / LiDAR detections are supplemental to, and not a replacement for, safe driving practices. The display of the legal disclaimer will have a scrollbar so that the user can view the entire text.
[0090] Main Menu. The main menu will provide the user with the ability to activate or deactivate the DDA Application, as well as updating the local data store, modifying the sound display options, and configuring the device for automatic updates or manual updates. When the DDA Application is “Active” alerts will be generated regardless of the user interface being visible or not. If the user chooses to “deactivate” the DDA Application, a confirmation dialog will first be displayed. If the users confirms the deactivation, an audio confirmation of deactivation will play, the “activate” menu option will be enabled, and the deactivate menu item will be grayed out. In Al- and / or neural network-enabled variations, the main menu may also include options to enable or disable Al-based detection modes, adjust camera and optional LiDAR integration settings, view a log of recent AI- detected POIs along with confidence scores, and prioritize which Al-detected events will trigger alerts. These settings allow the user to customize AI / LiDAR behavior to their driving environment and personal preferences.
[0091] Data Updates. Data updates are new POIs that become available after the current data set has been downloaded to the device (e.g., a new traffic light gets added within the current city) and supplement the existing local data. If the device is unable to download an update, alerts will continue to happen using the POIs already on the device. POIs are to be discarded only If the amount of data received would overflow the local storage. Ideally, data would be retrieved from a data serverautomatically; however, this may not always be possible for various reasons: the device may not be able to establish a data connection in some locations, platform certification requirements may require the user to authorize and enable data downloads, another application may be using the data connection (device specific), etc. Additionally, a data plan may charge the user to establish a data connection and an additional fee for the size of the download; consequently, on devices on which data can be dynamically downloaded, this option should be configurable by the user. The amount of data that gets downloaded may be limited by the configured size of the local data storage (e.g., a 1MB data file). If the amount of data received would overflow the local storage, the points furthest from the device are discarded as new points are inserted. If the device leaves the current data boundaries, it can be configured to play a warning tone. In Al- and / or neural network-enabled variations, data updates may also include updated Al training models, detection parameters, and LiDAR calibration files to improve the accuracy of visual and spatial recognition of POIs. Such updates can be delivered alongside POI coordinate data so that AI / LiDAR capabilities evolve with changing traffic infrastructure.
[0092] New local data should be provided in updates that occur several times per year, as POI data becomes available. It is recommended that data updates be built into the DDA Application and that the user downloads the update as an update to the entire DDA Application. This technique greatly simplifies the data update for the DDA Application by delegating the update processes to the device platform (i.e., manufacturer, operating system, and wireless carriers - e.g., Google Play Store, Apple App Store, Verizon VCast App Store, and the like). Users will automatically be notified of updates when a new version of the DDA Application is uploaded and will be able to download it when they update their DDA Applications typically when they have a reliable connection. For AI / LiDAR- enhanced versions, these platform-managed updates may also push optimized Al model weights, improved neural network architectures, and enhanced sensor fusion algorithms so that the DDA Application continually benefits from state-of-the-art recognition and alerting performance.
[0093] Manual Updating. If the device is configured for Manual Updates (automatic updates disabled), the DDA Application will retrieve as much data as practical to store in the local storage. It can do this by requesting data from the server in a predetermined radius: e.g., 200 miles around current location. To update data, the device must be able to retrieve the device position, as well as connect toan internet data server. If it is not able to do these, it should display an error message with the option to “try again,” or cancel. An animation will play to show the user that data is being updated. If the amount of data that will be downloaded can be determined, the DDA Application should display a progress bar. If not, the DDA Application should display a looping animation - e.g., an animating hourglass. For Al- and / or neural network-enabled variations, manual updates may also retrieve the latest Al detection models, visual recognition parameters, and optional LiDAR mapping data for the selected radius. This ensures that manual updates not only refresh static POI coordinate data but also enhance AI / LiDAR performance for the user’s specific geographic region.
[0094] Automatic Updating, In automatic mode, the DDA Application will request updates from the server as the device approaches the boundaries of the current local data - e.g., when the device is within 10 miles of current local data boundary. In Al- and / or neural network-enabled variations, automatic updates may also retrieve updated Al models, refined image-recognition datasets, and optional LiDAR mapping information for nearby regions to ensure optimal performance in detecting and classifying POIs.
[0095] Options. The options provided to the user are minimal. They will be able to enable automatic data updates and to select the sound for the audible alert - a female voice, a male voice, or a chirp (default). The user will be able to hear the currently selected alert sound by pressing a play sound button. The user will also be able to set the warning sound for when they are leaving the map boundary so that they will know not to expect any more alerts. The drop list will have an option called “<no sound>” in case the user does not wish to receive alerts for these options. For Al- and / or neural network-enabled variations, the options menu may further allow the user to toggle Al-based POI recognition, enable or disable camera and optional LiDAR input, and adjust the sensitivity or confidence threshold for Al-triggered alerts.
[0096] About. An “About” screen will display the DDA Application name, copyright, version information as required, and reference the company website. In Al-enhanced versions, the About screen may also display the Al model version and last update date for transparency and user awareness.
[0097] Error Messages. To interfere with the user as little as possible error messages will only be displayed while the user interface is displayed - i.e., no error messages will be displayed by the background process. AI / LiDAR functionality is active, diagnostic error messages related to camera feed loss, LiDAR sensor unavailability, or Al model loading errors may be optionally enabled for advanced users.
[0098] PDA Application Startup. Referring to Figures 4, 5 and 6, When the DDA Application starts up it will play an audible sound to indicate to the user whether the DDA Application is active. Once activated alerts will sound regardless of whether the user interface is visible. If the DDA Application was not started explicitly by the user, the DDA Application will go directly to background mode. Referring to the single asterisk in Figure 6, removal of POIs which are behind the device is only necessary if database retrieval uses rectangular optimization. Referring to the double asterisk in Figure 6, optionally, if the nearest POI is selected so that the appropriate sound type can be played for the type of POI, e.g., if a different sound is played for crosswalks or railroad crossings. If the DDA Application has been configured to automatically update data, it will send a server request out to retrieve updated POI data. In Al- and / or neural network-enabled variations, startup routines may also include loading and initializing Al models, calibrating camera and optional LiDAR sensors, and verifying geofenced POI data alignment with Al detection parameters.
[0099] Main Background Processing Loop. The background processing loop is the evaluation routine that decides whether to play an alert. It should be reevaluated each time the position of the device changes. For devices that do not receive frequent GPS updates, the device can predict the position in between GPS updates. In some variations, the loop also incorporates results from an onboard artificial intelligence (Al) and / or neural network module that processes real-time imagery from cameras and, optionally, depth data from LiDAR sensors to detect and classify points of interest, with these detections triggering reevaluation events in parallel with GPS updates. The “Last Played” POI is remembered each time that an alert is played to avoid repeatedly playing the alert for the same traffic light. Once the conditions that would cause the alert to be played for that light have changed, this information is cleared so that the alert can be replayed for that POL
[0100] Event Processes. Events are conditions that occur at unspecified (or asynchronous) times. The DDA Application receives control of the device, handles the event, and then returns control of the device to the operating system - i.e., the DDA Application is designed as an event-driven program. Typical events are reception of the location data in response to a previous position request, reception of POI data from the server, and timer interruptions that the DDA Application has requested. In some variations, additional events include reception of detection results from an onboard artificial intelligence (Al) and / or neural network module that processes real-time imagery from cameras and, optionally, depth data from LiDAR sensors to detect and classify points of interest, with such detections treated as event triggers in the same manner as GPS or server data updates. The Main Event handlers are the routines that execute whenever new POI data is received, or new GPS location data is received, or on a timer. Following is Pseudo Code for the main events received during operation of the program.
[0101] OnPOIDataReceived . This event handler is called when POI data is received from the server in response to a “get” data query. Reception of a new POI is inserted into both the local data and the in-memory data - called the ActivePOItree:OnPOIDataReceivedInsertNewDataIntoLocalDatabase();InsertNewDataIntoActivePOITree()RequestGPSPosition();**ProcessCameraAndOptionalLiDARDataWithAIorNeuralNetworkModule();** EvaluatelfShouldPlay Alert() ;In some variations, after updating the POI data, the handler invokes an onboard artificial intelligence (Al) and / or neural network module to process real-time imagery from cameras and, optionally, depth data from LiDAR sensors, enabling the system to detect and classify points of interest not yet present in the received POI data before proceeding to the alert evaluation step.
[0102] OnGPSDataReceived. This event is called by the Operating System in response to a request for the GPS position of the local device. Depending on the API for the mobile device, a request for a GPS position may be delivered asynchronously. When the position is finally received, the program can “prune” the active POI tree, that is, it can discard POI data that is far enough away from the device that it is no longer relevant. This pruning will reduce the memory usage of the program. The program may also choose to prune the persistent data set based on a larger threshold. In some variations, updated GPS position is also fiised with detections from an onboard artificial intelligence (Al) and / or neural network module processing camera imagery and, optionally, LiDAR depth, to refine POI relevance before alert evaluation.OnGPSDataReceived()UpdatePositionVariables()Prune ActivePOITree() ;**ProcessCameraAndOptionalLiDARDataWithAIorNeuralNetworkModule();*** *UpdateSensorFusionStateWithGPSAndAIDetections(); * * EvaluateIfShouldPlayAlert( );The fusion step aligns the latest GPS fix with Al-based detections (e.g., traffic lights, stop signs) so that subsequent alert decisions reflect both map-based proximity and real-time visual confirmation.
[0103] On Timer. The DDA Application should set the location change to ensure that the alert evaluation happens on a regular basis. The location depends on how reliable and regular the GPS notifications are from the local device. Please see the section on Calculations and Helper Functions below. In some variations, the timer event also triggers processing by an onboard artificial intelligence (Al) and / or neural network module that analyzes real-time camera imagery and, optionally, LiDAR depth data to detect and classify points of interest, allowing alerts to be generated even if no new GPS update has been received.OnLocationChanged()RequestGPSPositon();PredictCurrentPosition() ;**ProcessCameraAndOptionalLiDARDataWithAIorNeuralNetworkModule();** EvaluateIfShouldPlayAlert( );
[0104] Client-Side Active POIs. The client will contain an in-memory data structure which contains POIs that are physically near the location of the device. POIs will be removed from the data structure when the distance to the device exceeds a defined distance (e.g., 10 miles). Devices may operate in two selectable modes: (i) Full Dataset Mode, where pruning applies only to the in-memory “active set” and globally stored POIs are not deleted due to distance; and (ii) Regional / Pruned Mode, where both persistent and in-memory sets may be pruned according to storage limits. This distance can be altered dynamically if insufficient memory is available for the desired outer range distance. In some variations, the in-memory POI list may also include dynamically detected POIs generated by an onboard Al / neural-network module processing real-time camera imagery (and, optionally, LiDAR depth data), allowing transient or unmapped POIs to be tracked alongside server- or map-based POI.
[0105] As the device moves, it will request data updates for new POIs within a given distance of the device (e.g., 5 miles) from the Local POI database manager. The Local POI Manager maintains the local database which is persisted on the device file system. Additionally, the Local POI Manager will request data updates from a remote / intemet server as required (see section on Server-Side Design) and as configured by the user (see section on Data Updates). In Full Dataset Mode, remote updates are optional and limited to corrections; in Regional / Pruned Mode, remote updates populate and refresh regional tiles as the vehicle moves. In Al-enabled variations, updates from the Local POI Manager may be supplemented or replaced by real-time POI detections from the Al and / or neural network module, ensuring that relevant POIs are identified even before map or server updates occur.
[0106] Both the local storage and the remote storage are optional, although at least one is required. This allows the device to be configured to use only local storage, only remote storage, or both. This design allows data to be preloaded (or downloaded through a data cable) on devices which do not support an internet connection. This also permits the software to work on devices which do not have any local persistent storage. Devices that store the data locally can also be updated with new data as it becomes available. In some variations, dynamically detected POIs from the Al and / or neural network module may be stored temporarily in local memory for rapid re-access, with optional LiDAR- confirmed detections prioritized for persistence. To reconcile storage constraints, the system supports two operating profiles: (i) Full Dataset Mode (ample local storage; worldwide POIs preloaded or cached by region), and (ii) Regional / Pruned Mode (constrained storage; only nearby / regional tiles persisted and pruned as needed). Pruning logic, eviction policy, and on-the-move tile loading apply only to Regional / Pruned Mode.
[0107] Triggering Conditions. For any point, the trigger condition will be that the device is approaching the POI, exceeding the threshold speed, and that the device is calculated to reach the POI in less than a threshold time (e.g., speed > 10 mph and ETA < 10 seconds). Bool bPlay Alert = Current Velocity > VelocityTrigger && IsApproaching(NearestPOI) && TimeToPOI(NearestPOI) < TimeThreshold. In some variations, the NearestPOI may be determined not only from stored map or server data but also from real-time detections generated by an onboard artificial intelligence (Al) and / or neural network module processing imagery from cameras and, optionally, LiDAR depth data, allowing transient or unmapped POIs to be included in the trigger evaluation. When selecting the TargetPOI, we will weed out POIs which are behind the device, that way we will stop playing sound after we pass a POI even if it is still the closest. All sensor inputs used for fusion (SPAT packets, GPS poses, camera detections) are synchronized by timestamp with a configurable tolerance (e.g., ±50 ms); late or stale inputs beyond this window are discarded to maintain deterministic alert timing.
[0108] Playing the alert sound. WWhen an alert is played, the POI that generated the alert should be recorded as well as the time the alert was played. In some variations, the recorded POI may originate from either stored map / server data or from a real-time detection generated by an onboard artificial intelligence (Al) and / or neural network module processing camera imagery and, optionally,LiDAR depth data. When a request to play an alert is made, if the alert type is the same as the currently playing type, then the alert sound will not be interrupted.
[0109] To avoid replaying an alert multiple times for the same POI, the DDA Application should remember the last POI played until one of the following conditions occurs:1. The POI is behind the device (e.g., the user has passed the traffic light).2. A timeout for that POI has passed (e.g., 1 minute).3. The DDA Application restarts. For Al- or neural network-detected POIs, these same conditions apply, with optional LiDAR confirmation data stored alongside the detection to improve reliability in repeated detection scenarios. These conditions will allow the DDA Application to handle users that travel back and forth along routes that only have a single POI.
[0110] User Notification Sounds.Device approaching POI - beeping soundDevice in alert mode - voice notificationDevice removed from alert mode - notification that will not be receiving further warnings.Device leaving map boundaries - notification that will not be alertedDevice entering map boundaries - notification that will start playing alertsDDA Application Started up - notification sound - may be a tune / beep / or voice alert.In Al- and / or neural network-enabled variations, notifications may also indicate when a POI has been detected in real time from camera imagery and, optionally, LiDAR depth data, including distinguishing sounds or voice alerts for dynamically detected versus preloaded POIs.
[0111] Client-Side POI Manager. Storage of the POIs on the client device will be logically divided into two portions: the “in memory” POIs, and the persistent storage (database). An application object called CPOIManager will keep track of all the POIs on the client device. It will cache a set of the nearest POIs used by the main DDA Application as well as manage the persistently stored POIs. It will automatically request new data from the server as the device reaches the current map boundaries (if so configured) and can purge POIs from the device when they exceed a specific distance. In somevariations, CPOIManager will also store and manage dynamically detected POIs generated by an onboard artificial intelligence (Al) and / or neural network module processing real-time camera imagery and, optionally, LiDAR depth data, allowing these detections to be treated equivalently to server- provided POIs.
[0112] The in-memory POIs set is essentially a cache of the nearest points. CPOIManager will organize the in-memory data using a Point Quadtree data structure. This will allow the Client DDA Application to quickly find the closest POIs in the tree. The in-memory footprint can be configured to contain a maximum number of entries. The furthest POIs in the QuadTree can be automatically removed as new data is inserted once the maximum is reached. In Al-enabled and / or neural network variations, dynamically detected POIs may be inserted into the QuadTree in real time, with optional LiDAR confirmation improving confidence scores used in POI prioritization.
[0113] The POI data will be locally persisted on the device in a data file. The data should be spatially organized and may utilize some form of compression - see the section entitled Local Database File Format below. The size of the local file store will also be configurable. This will allow the POI list to be read at startup time even if there is no internet connection to a remote server available. For devices which do not have internet connection, the client-side database will contain all the data available to the DDA Application. For additional coverage, the user will need to download additional map data. In variations with Al and / or neural network detection, locally detected POIs may be written to this file with metadata indicating their source and detection confidence, enabling persistence and later revalidation against updated map or server data.
[0114] AI-Detected POIs (Ephemeral Layer). Al-detected features (e.g., temporary work zones) are kept in an EphemeralDetections layer with a time-to-live (TTL) and confidence score and are not merged into the static POI store unless corroborated by (a) repeated detections above a confidence threshold, (b) external data (e.g., municipal feeds), or (c) user / admin verification. This avoids uncontrolled growth of the static POI dataset while still enabling timely alerts for transient conditions. Ephemeral entries expire automatically on TTL or when contradicted by authoritative data.
[0115] Data Format. Latitude and Longitude are used as decimal degrees. For example, we will use 47.5° rather than 47 degrees, 30 minutes. The natural representation of this is a floating-point number. However, many mobile devices do not have floating point support, and so we can support the decimal degrees using fixed point number representation. To encapsulate for devices that don’t support floating point, the data type will be called FLOAT32. In variations incorporating onboard artificial intelligence (Al) and / or neural networks with optional LiDAR, additional metadata such as detection confidence scores, POI classification type, and optional depth measurements may also be stored alongside the latitude and longitude coordinates in FLOAT32-compatible or companion data structures to maintain uniformity across POI sources.
[0116] For fixed point representation, FLOAT32 will be encoded as 10.22 to allow + / - 360 degrees to be encoded. In a 32-bit integer this means that 10 bits will encode the integer portion and 22 bits will encode the decimal portion. This will provide approximately 0.00000024 degrees of accuracy (1.0 / (2Λ22)). In Seattle (Latitude 47°):1° of Latitude is approximately 111.325 km * 1000 m / km * 0.00000024 = 2.67 m1° of Longitude is approximately 1° of Latitude * Cos(Lat). ~= 2.67 m * cos(47°) = 1.82 m.When data is received from remote sources, it will be converted to FLOAT32 for application consistency. For vision-derived POIs, image-plane detections are mapped into geographic coordinates via calibrated camera intrinsics / extrinsics and ego-pose; the resulting lon / lat are stored in FLOAT32, and accompanying metadata (e.g., camera frame transform, fusion timestamp ns, and quality / confidence values) is persisted for traceability and alignment with SPAT / POI records. For AI- or neural network generated POIs, the same coordinate conversion will be applied, ensuring that dynamically detected locations are stored in the same precision and format as map- or server-based POIs, with optional LiDAR-derived elevation or range values normalized for integration into the active POI database.
[0117] Data Structures. The representation of data structures are generically presented as C structures, which are easily converted to the equivalent for platform specific programming. A Coordinate is a location on the globe and contains two elements. Lon = Longitude, and Lat = Latitude. Typically, a coordinate is called a Point and is abbreviated as “pt.” In variations incorporating artificialintelligence (Al) and / or neural networks with optional LiDAR, a Coordinate structure may also be associated with supplemental metadata such as detection confidence, classification source (map, server, or Al detection), and optional LiDAR-derived range or elevation data. struct CoordinateFLOAT32 Ion, lat;**UINT8 sourceType; / / e.g., map=0, server=l, AI=2****FLOAT32 confidence; / / Al detection confidence (0-1)****FLOAT32 lidarRange; / / Optional LiDAR distance measurement**
[0118] The list of active POIs will be stored in a dynamic “point quadtree” data structure. Each node in the data structure will contain four pointers: pNorthEast, pSouthEast, pSouthWest, and pNorthWest, as well as the type of the POI, so the appropriate alert sound can be played. The root node for the data structure may be stored in a global variable or as an DDA Application class member variable. In Al-enabled variations, additional fields may store attributes such as Al-derived classification labels or LiDAR confirmation status for the POI. struct QNodeCoordinate pt; ePOIType eType; / / 8 bits**char classificationLabel
[0032] ; / / Optional AI / NN classification****UINT8 lidarConfirmed; / / 0 = no, 1 = yes** struct QNode* pNE; / / points which are North East of this point struct QNode* pSE; / / points which are South East of this point struct QNode* pSW; / / points which are South West of this point struct QNode* pNW; / / points which are North West of this point
[0119] As GPS location updates are received. The DDA Application should store and update the current and previous positions and calculate the current velocity and acceleration. This will allow the program to predict the future position of the device - see section on Position Prediction below. For Al- and / or neural network-enabled variations, this structure may also store fused sensor data outputs, such as Al-detected POI proximity or LiDAR-derived obstacle distances. struct Position{Coordinate pt;UINT32 timeReceived;FLOAT32 velocity;FLOAT32 acceleration;**FLOAT32 aiDetectedPOIDistance; / / meters to nearest Al-detected POI**
[0120] The DDA Application should also keep track of the POI that activates an alert and at what time it has been played. This will be used when deciding whether to play an alert again. In AI- enabled variations, this record may include whether the POI was confirmed by Al-only, LiDAR-only, or both. struct Alertitem{Coordinate Position; / / Lon, Lat of POI that triggered alert ePOIType eType;UINT32 time Activated;**UINT8 detectionMode; / / 0=map / server, l=AI-only, 2=AI+LiDAR**
[0121] As new nodes are received from the server, they will be inserted into the local quadtree. And nodes that exceed the outer range will be pruned from the tree to conserve memory. On insertioninto a non-empty tree, the new point should be inserted at a random level in the tree, rotating the sub trees as required. At each level, the probability of inserting should be calculated as 1 / n, where n is the number of items already in the tree. If inserting at a non-node level, a standard root insertion process, which rotates the tree, should be used. Note: that since the tree is a quadtree, the rotation must happen for two axis - the NE - SW axis, and the NW - SE Axis. This randomization tends to create a more balanced tree and protect against “worst case” insertion, i.e., when the data is inserted in a sequential order, without explicitly having to balance it. In Al-enabled variations, dynamically detected POIs can be inserted into the quadtree in real time, with optional LiDAR confirmation improving the ranking or priority within the tree for faster retrieval during alert evaluation.
[0122] Local Database File Format. Any GIS spatial file format may be used for local persistent storage. All that is required is that the client can read and write the file using random file access. Below are described a simple file format, and a more complex alternative. In variations incorporating artificial intelligence (Al) and / or neural networks with optional LiDAR, the local database file format may also include additional fields for each POI entry, such as detection source type (map, server, Al), Al classification label, detection confidence score, and optional LiDAR- derived range or elevation data. These additional fields can be appended to or embedded within the chosen GIS format, ensuring backward compatibility while allowing enhanced data from real-time sensing to be stored and retrieved consistently with static POI data.
[0123] Simple File Format. For local persistent storage, the simple file format used will be a serialization of the quadtree data structure. Immediately following the file header, there will be a file data node. All the information in the node will be identical to the data in the quadtree with exception that memory pointers will be replaced with Record indices. The file offset of the record can be calculated simply as:FilePos = FileHeader.OffsetData + index * sizeof(FileNodeRecord).Struct FileHeader int MagicNumber; / / used to help identify the file format e.g., 0xAD0G15DADAint SizeFileHeader; / / size of this structure int NumRecords; int MaxRecords; int SizeOfRecord; / / size of FileNodeRecord int FileOffsetRoot; / / index of the quad tree root node int FileOffsetFreeList; / / 0 if no free nodes available int OffsetData / / start of data - used to convert index values to file offsetsNode records will not be deleted from the file, but rather just moved to the free list. This will allow fast insertions and deletions and a fixed size local data file. Once the free list is empty, inserting a new record will first delete the furthest POI from the current location and then reuse that location. Index 0 is reserved as an “invalid” index. struct FileNodeRecord{FLOAT32 Lat,lon; ePOIType eType; / / 8 bits bool bPersist; / / should have lifetime? int pNE; / / Next node NE of this one (indexed) int pSE; / / Next node SE of this one (indexed) int pSW; / / Next node SW of this one (indexed) int pNW; / / Next node NW of this one (indexed)The Free List will use the same structure, but only use pNE as the pointer to the next free index. Index 0 is reserved as an Invalid index - i.e., a NULL pointer. In Al-enabled or neural network variations, the free list retains AI / LiDAR metadata fields so that newly inserted POIs (whether server-provided or AI / NN-detected) retain consistent record layouts in the file for compatibility and quick reuse.[0124[ R-Tree file format. A more efficient (and complex) data structure that is well suited to spatial access methods is the R-Tree. With an R-Tree, nearby POIs will be grouped and can thus savespace in the file system. This data structure will also work well with updating data as the user device moves beyond existing map boundaries. In variations incorporating artificial intelligence (Al) and / or neural networks with optional LiDAR, each R-Tree node may also store supplemental metadata such as detection source type (map, server, Al, AI+LiDAR), Al classification label, detection confidence score, and optional LiDAR-derived range or elevation data. These fields enable spatial grouping to leverage both static and dynamically detected POIs, with AI / LiDAR-enhanced nodes potentially prioritized for faster access during real-time alert evaluation.
[0125] Since data for the entire world will be shipped with the DDA Application, the local database file format can be simplified. For example, each regional map file can have a single line that states the bounding rectangle, and the number of POIs of each type. E.g., the following example indicates that this map data file is bounded from -122.87809,45.75770 upper left (NW) coordinate to lower right coordinate (SE) -104.99686, 39.74014, and contains 12000 type 0 (traffic signals), 112000 type 1 (school zones), and 15000 type 2 (Railroad crossings) POIs.-122.87809,45.75770,-104.99686,39.74014, 12000,11200,15000-122.87809,45.75770-122.66130,45.58112-122.41392,45.50169-122.68535,45.52647-122.41249,45.51905-122.56681,45.47931-122.77382,45.58771-122.69334,45.58735-122.69309,45.55647In AI / LiDAR-enabled variations, these records may also be augmented — either inline or in a parallel metadata index — with fields such as detection source (0=map, l=server, 2= Al, 3= AI+LiDAR), Al confidence scores, LiDAR-measured distances, and classification labels. The metadata index can be aligned with the main coordinate list to allow rapid binary loading while preserving backwardcompatibility with simpler, static-only formats. The map can be preprocessed into binary form such that the load can happen very quickly without having to parse the data. Additionally, the map files can be individually compressed to save space if required.
[0126] Man Boundaries. The client-side POI manager (CPOIManager) can keep track of the known data boundaries so that the client DDA Application will know if the user has crossed outside the map boundaries and can alert the user appropriately. If an internet connection is available and the user has enabled automatic data updates, CPOIManager will send queries out to the server requesting more data as the user approaches the map boundaries. Note, however, if all the world data is available on the server, it will not be necessary to handle updates from a server. The map boundaries can be preprocessed for optimal use by the POI Manager. In Al- and / or neural network-enabled variations, crossing a map boundary may also trigger real-time Al detection mode using camera and optional LiDAR data until new static POI map data is loaded, ensuring uninterrupted hazard and traffic signal detection.
[0127] The POI Manager will have an index list of the individual regional maps, and their bounding areas. As the device moves, the local POI manager will be able to load the appropriate maps into memory. Each map should be loaded into its own Active POI quad tree such that the entire tree can be discarded when that region is no longer necessary. The maps should be sized such that at least two maps can be in memory at once. In AI / LiDAR-enabled variations, each regional map index entry may also store metadata indicating whether supplemental Al-detected POIs exist for that region, along with average detection confidence scores, to prioritize certain maps for earlier preloading. struct MapBoundaries{Coordinate ptNW;Coordinate ptSE;POITree* pPOIQuadTree; / / null if not currently loaded **UINT8 hasAIMetadata; / / 0 = no, 1 = yes** **FLOAT32 avgAIConfidence; / / average Al confidence for POIs in region**MapBoundaries Mapindex [NUM MAPS];
[0128] A query from the DDA Application to the local POI manager will return a result that includes the set of unique POIs from all the POI trees which contain the current point. If regions are not rectangular, regional bounding boxes could have significant overlap. If the amount of memory usage due to this is significant, the regions can easily be preprocessed into smaller sub regions with less overlap. For AI / LiDAR-enhanced systems, overlapping regions may be merged or split dynamically based on real-time Al detection density and priority scoring, ensuring that high- confidence hazard zones remain active in memory even if static map boundaries would otherwise unload them.
[0129] Referring to Figure 7, In order to simplify processing, the POI manager will keep track of the current map data as occurring within a square. The size of the square should be as large as possible but will be limited by the amount of data the device can store. There will be a border width defined that is some size such that the DDA Application will have time to download new data before the device reaches the perimeter of the map. When data for the new region arrives, the POIs from the previous map that are not within the new map boundaries will be discarded. In Al- and / or neural network-enabled variations, this process may also include preloading region-specific Al model optimizations, visual detection parameters, and optional LiDAR point cloud data for the new map region to ensure seamless POI recognition and alert generation during the transition between maps.
[0130] In Figure 7, the device / user is traveling South East. The current data set is called Map A. When the user reaches the border of Map A (dotted line), the device will request data for Map B. Then when data for Map B arrives, the device will discard data from map A that is not inside Map B. For AI / LiDAR-enabled versions, the transition from Map A to Map B may also trigger recalibration of Al recognition thresholds and optional LiDAR alignment to the updated geospatial dataset, ensuring consistent detection accuracy across regional boundaries.
[0131] Server-Side Design. The basic server-side DDA Application is a Webservice REST API which runs on a web server and will respond to queries for POI data from client DDA Applications by querying a database for POIs in a specified rectangle. The data in the database will be contained in a table with all the GPS coordinates for the traffic lights and other POIs as well as the type of POI (e.g., Traffic light, school crossing, railway crossing). A database with spatial support (or Spatial Extensions) which has spatial indexing should be used to optimize lookups. In certain variations, the server may also host Al and / or neural network models trained to enhance POI recognition, contextual alert prioritization, and predictive alert timing, as well as store and serve optional LiDAR mapping datasets for improved environmental accuracy in client devices that support LiDAR. The REST API may further be extended to deliver Al model updates and LiDAR data selectively, based on the client’s geographic region and hardware capabilities.
[0132] An appropriate server(s) should be selected and hosted in a highly reliable data center. A backup server should also be run in a separate data center, and the client DDA Application should be configured to use the backup server when it is not able to contact the primary server. The IPs of the servers can be changed by modifying the DNS information for the web address of the server. For AI / LiDAR-enabled configurations, redundancy should also apply to model and dataset hosting, ensuring that both primary and backup servers can provide the most recent Al model parameters and LiDAR map updates without service interruption.
[0133] Server API. The server will respond to standard HTTP “get” requests. This allows the actual server implementation to use a standard server-side solution (e.g., PHP or Java scripting) and permits flexibility and ease of upgrading or changing the server-side solution. In Al-enabled variations, the API may additionally expose endpoints for retrieving trained Al and / or neural network model parameters, Al inference results for POI relevance or prioritization, and, if supported by the client device, LiDAR-enhanced spatial data sets. These endpoints can be versioned to ensure backward compatibility with older client software.
[0134] The rectangle is specified by its central point, and by a radius, the radius will be specified with separate values for longitude and latitude. The radius will be specified in the sameFLOAT32 degrees values (converted to ASCII) and must be positive (see section on Data Format). The URI provided to the “get” request will encode the location and radius. Data requests with rectangle boundaries which are out of range are returned a basic error message, or for increased security, no response at all. The server should also set a limit on the maximum rectangle size to avoid sending down too much data. When AI / LiDAR features are enabled, the request parameters can also specify whether Al-prioritized POIs or LiDAR-enhanced POI coordinates are to be included in the response, allowing the server to tailor data delivery to the client’s capabilities and settings.
[0135] The resultant POIs will be returned in a packet which specifies, the number of POIs in the packet, and a bounding rectangle of all the points - specified using the North West and South East comers. It will then contain a series of data blocks that first state the type of the POI, and number of points followed by the actual data. In Al-enhanced implementations, each POI record may optionally include an Al-generated confidence score, a predicted relevance ranking, or supplemental LiDAR- derived attributes (e.g., 3D elevation or precise structure outlines) to assist the client application in rendering and prioritizing alerts.Sample Returned Data Packet:
[0136] To reduce the data download size and time, which can also reduce user cost; it is recommended that the data be delivered as binary data that contains the coordinates already inFLOAT32 format. Sending in binary is more efficient than sending the data as text since generally the data size will be smaller and the client-side processing will be simpler. If doing this the MIME type can be set to “application / octet-stream”. If so required, the data can also be compressed if the client and server match compression schemes and the client has decompression software. When AI / LiDAR features are used, the binary packet format may be extended to include Al confidence scores, LiDAR point cloud segments, or other metadata in compact binary form to minimize additional bandwidth requirements.
[0137] Open-Source Solutions. Various open-source solutions can be used to set up a custom data server — various configurations are presented below. However, with delegation of the data update to the device platform, a custom server is not required. The user interface can be simplified to remove any mention of data updates. In certain variations, open-source computer vision frameworks and neural network models may also be leveraged in place of, or in combination with, server-based POI data. For example, camera-based recognition software — built on open-source libraries — can process real-time images to detect traffic lights, stop signs, and other roadway features locally on the device. This approach can reduce dependency on continuous server connectivity, as both POI updates and traffic control detection can be supported through freely available, modifiable codebases.
[0138] Linux + Apache + MySQL + PHP, This is a common open-source configuration commonly known as “LAMP”. It is well suited to high demand server-side applications. Additionally, many web hosting companies provide this as a standard hosted configuration. This would allow the solution provider to take advantage of the server hosting and maintenance by a third party. MySQL provides spatial functionality that is suited to supporting our POI data. The coordinates should be entered in the database as Point data which is of the Geometry class.
[0139] A REST API written in PHP will receive the request from the client DDA Application, look up the data in the MySQL database, format the data to be returned and return it via http. PHP can manipulate and return the data in binary form by setting the content type with the “headerQ” function and then using the echo command to send the data:Header( "Content-type: DDA Application / octet-stream" ); echo $data;
[0140] *nix + Apache + Resin + MySQL, A Java springboot based Webservice API can be designed which responds to queries from the client DDA Application and looks up the POIs in a MySQL database. The Java applet can use a ServletOutputStream to output the data in a binary format: final ServletOutputStream out = res.getOutputStream(); res.setContentType(application / octet-streambytef] buf = new byte[bufsize];I I read data from database... and store in buf byte array out. Write(buf, o, datalength);
[0141] *nix + Apache + PostgreSQL + PostGIS + PHP. PostgreSQL DBMS is a powerful open-source relational database. PostGIS is also open source which “spatially enables” PostgreSQL by adding support for geographic objects, with the express purpose of allowing it to be a backend spatial database. PHP contains native support for PostgreSQL and thus provides a natural interface and is sited as “an excellent interface” by PostgreSQL. Any of the various *nixs (Linux, FreeBSD, etc) provide support for these programs.
[0142] Third Party Solutions. GPS data is available both as a separate product which can be installed directly by the customer or is available from a source called Open Street Data to which the customer’s POI data can be overlaid and can serve as the backend of a POI server solution. OpenStreetMap is headquartered in Cambridge England, though they may seek new headquarters in Europe. The database is hosted by the OpenStreetMap Foundation, a non-profit organization registered in England and Wales and is funded mostly via donations. OpenStreetMap is freely licensed under the Open Database License.
[0143] POI Data. A key component of the DDA Application is the GPS location data for traffic lights and other POIs. Populating the server database is thus a key consideration.
[0144] Calculations and Helper Functions. Helper functions encapsulate common decision code. Explanations of calculations behind their implementation is provided below:isApproaching(LonLat& POI, LonLat & CurPos, LonLat& Velocity) isBehind(LonLat& POI, LonLat & CurPos, LonLat& Velocity) DistanceInKm(LonLat & pl, LonLat & p2)VelocityKmPH(LonLat & pl, LonLat & p2, FLOAT deltatime) Distance(LonLatVector &Velocity, FLOAT time); / / result in Kilometers TimeToPOI(LonLat& POI, LonLat & CurPos, LonLat& Velocity);
[0145] Distance Calculations. Distances can be calculated using the spherical law of cosines: d = arcos (sin(lati)*sin(lat2) + cos(latf)!8‘cos(lat2)!8!cos (lom-lom ))*MeanRadi osOfEarthIf the device operating system does not provide trigonometric functions, or the trigonometric functions are expensive, then the DDA Application can maintain a lookup table to optimize the trigonometric function lookups.
[0146] Alternatively, distances can be calculated as: dLat = (p2.1at -pl.lat) ^LatitudeScalar; dLon::::(p2.1on - pl. Ion) * LongitadeScalar; d = Sqrt (dLat*dLat + dLon*dLon);For “nearest point” calculations, d squared can be used so the square root does not need to be calculated. For consistency and ease of computation all distances are calculated in kilometers (to). To convert kilometers to miles, multiply by 0.6213712. To convert kilometers to nautical miles, multiply by 0.5399568.
[0147] Degrees to Distance Conversion (Scalars). A simplification for various calculations is to “flatten” the globe and treat Latitude and Longitude as a planar orthogonal coordinate system ratherthan use angular coordinates. This is a valid approximation when distances are short and the device is not near one of the poles. Since rings of latitude are constant, the latitude scalar is essentially constant:LatitudeScalarKm = Circumference of Earth / 360 degrees ~= 40075 Km / 360 ~= 111.325 Km / degree.
[0148] Longitude varies based on latitude, and is easily calculated: LongitudeScalarKm = cos (latitude) * LatitudeScalarThe longitude scalar can be recalculated occasionally.
[0149] Velocity Calculations. Typically, the velocity will be provided by the GPS system. If can also be determined by recording the received positions and the time differences of when they are received. The velocity is then calculated by the formula VelocityVector = dPos / dTime. That is the change in position divided by the elapsed time. For position prediction it is helpful to maintain the velocity as a vector with longitude and latitude components. These correspond to angular velocities. To convert these units to KmH, for our velocity threshold test, we use the formula:KmPH = Sqrt ( (Vel.Lat * LatitudeScalarKm)A2 + (Vel.lon * LongitudeScalarKm)A2). If we are just looking at the velocity to decide if we are moving fast enough, we can save the Sqrt calculation by using Velocity squared.
[0150] Bearing Calculation, The bearing (or heading) is the clockwise angle from North (East = 90 degrees). Typically this will be provided by the GPS interface. If it is not, then the DDA Application can compare the new position to the previous position each time an update is received and calculate the current bearing using the formula:0 = atan2(sin(dLon)*cos(lat2), cos(latl )*sin(lat2)-sin(latl )*cos(lat2)*cos(dLon) )Where dLon = lon2 - Lonl .
[0151] IsBehind. The IsBehind function is used to Prune POIs which the device has already passed since the distance to them no longer matters. It can be simply evaluated using two different methods: 1) If the devices bearing (or heading) is known, calculate the bearing from the current position to the POL If the absolute value of the difference between the bearings is greater than 90 degrees, then the POI is behind the device. See section titled Bearing Calculation above. 2) Generate a vector from the current location to the POI. Evaluate the dot product of this vector with the vector along the direction of travel, vecDirection. If the result is less than zero the point is behind:VecDirection is usually in the same direction as VecVelocity, however, VecVelocity can be zero, while VecDirection should never be allowed to be zero:VecDevice = VecDirection;VecPOI = PosPOI - PosDevice;IsBehind = (VecPoi.Lon * VecDevice.Lon + VecPOI.Lat * VecPOI.lat) < 0;
[0152] Position Prediction. Position Prediction is used for two things: 1) It is used to calculate the future position of the device so the DDA Application can retrieve POIs that are ahead of the device. 2) If updates from the GPS are not as frequent as required, the DDA Application can predict the current position on shorter intervals during timer events.
[0153] The DDA Application should utilize velocity and acceleration, to get the most accurate predictions. The latitude and longitude components will be calculated separately, so we can account for acceleration in each direction - e.g., acceleration in a curve.
[0154] Velocity and Acceleration can be calculated over short time intervals using just the known positions and the time at which the positions were received. Each time a positional update is received, the position and time are recorded. Only the difference in time from the previous positional update is required, so the DDA Application should use the highest precision time available - typically this is a system clock with millisecond resolution.
[0155] By recording the current and previous positions, the DDA Application can calculate the Current Velocity, by comparing the Current and Previous Velocity, the DDA Application can calculate the current average Acceleration:OnGPSPositionData() :PositionPrev = Positioncurrent;TimePrev = TimeCurrent;*DeltaTime = TimeCurrent - TimePrev;PositionNew = GPSPosition;VelocityNew = (PositionNew - PositionPrev) / DeltaTime; Acceleration = (VelocityNew - VelocityPrev) / DeltaTime;Distance = Vi acceleration * velocity * timeNewLon = Positioncurrent. Ion + Distance.lon;NewLat = PositionCurrent. lat + Distance.lat;Zero delta time should be tested before calculating the new velocity and acceleration.
[0156] Local Data is important to the operation of the DDA Application. The Local Data stored on the device is recommended to be as large as will fit on the device. Currently, it is now practical to store data for most of the entire world on the device. Unreliability of data connections has been demonstrated to adversely affect the operation of the DDA Application, so the primary recommendation is that the DDA Application ship with the entire world POI data (compressed by region if necessary). This will also have the usability benefit that the DDA Application will be able to work immediately without a data update.
[0157] SPAT (Signal Process and Timing) - Introduction. Referring to Figure 8, a series of messages are described that are part of what is commonly known as Cooperative Intelligent Transport Systems or CITS, which are a series of international standards available to all road authorities and vehicle manufacturers. In some variations, SPAT data is augmented with real-time Al- or neural network-based camera perception that visually identifies and classifies the state of traffic lights and stop signs. This dual-source approach — SPAT plus camera-based Al — provides redundancy and allows the system to operate when either source is unavailable or inconsistent.
[0158] There are 2 types of data that are used in transmission: 1) map data and 2) SPAT data. The standards for SPAT and map data are defined in a standard known as SAE J2735. Both map and SPAT messages are collected in traffic light intersections and are transmitted every lOOmS (l / 10th of a second) to all vehicles through data providers. The data can be identified as being sent ftom a specific intersection. In addition to SPAT messages from infrastructure, the DDA Application may determine the state of traffic lights using an onboard camera system and a neural network-based traffic signal recognition module, which processes images in real time to identify and classify the state of traffic lights.
[0159] Data Provider. Data providers are institutions that receive the map and the SPAT data from the traffic light intersections. These data providers in turn provide the data to companies that build automated car systems based on the map and the SPAT data. The data provider usually streams the data continuously to the companies that are subscribers that data. The DDA Application may receive such SPAT data while simultaneously processing camera imagery, and may fuse the results from both sources to improve detection accuracy and resilience in varying environmental conditions.
[0160] This is done in the following way1) The companies that subscribe to the provider setup a server.2) The subscriber company opens a particular User Diagram Protocol (UDP) port number to receive the data. (The UDP is part of the Internet Protocol suite, referred to as UDP / IP suite and is a cooectionless protocol, sothere is no need to establish a connection prior to data transfer)3) The provider sends map and SPAT data to the subscriber via a server port every 100 ms.4) In parallel, the in-vehicle Al- or neural network-based camera perception module continuously monitors the environment for visual indicators such as illuminated traffic signal lights or stop signs. This visual state information is timestamped and synchronized with incoming SPAT data for real-time cross-verification and confidence scoring.
[0161] Data Fusion. The fusion of SPAT and camera / neural network detection allows the DDAApplication to verify traffic signal states, reduce false detections, and anticipate phase changes earlier than by using either source alone. This hybrid approach ensures continued functionality in cases of network loss, GPS dropouts, or visual obstructions.
[0162] Server Implementation. In accordance with an embodiment of the invention, one can subscribe to the map and SPAT Data ftom a Data provider. One can set up an Amazon web cloud server and open a specific UDP port to receive map and SPAT data in Hex format. In addition, the system can integrate camera and Al- or neural network-based traffic light and stop sign recognition on the vehicle side, enabling detection of signal states even when SPAT data is unavailable or delayed.
[0163] Server Technology, The server comprises of the following technologies:— Cloud Provider: Amazon web Services— Operating System: Linux— Web Server: Apache— Programming Languages1) Python Libraries ( Pycrate)2) Python - Custom JSON Parser developed in Python3) C - Custom Parser developed in C for lane calculation4) PHP -RESTful webservices for consumption of Mobile App- addlocation.php5) CurlCron Job to detect and restart the Python code for JSONparsing
[0164] Conversion of Hex Data. Figures 8 and 9 depict how an embodiment of the system encodes and decodes data using JavaScript Object Notation (JSON). The system uses the Pycrate library to convert the hex data format to text data format. Pycrate is a Python library for manipulating various digital formats. It provides basically a runtime for encoding and decoding data structures, including CSN.1 and ASN.1 Additionally, it features a 3G and LTE mobile core network. The Pycratelibrary internally uses the SAE J2735 library to convert the Hex values to text values in JSON( JavaScript Object Notation) Format. When integrated with camera and Al- or neural network-based detection, the vehicle’s onboard perception module can cross-check SPAT-reported states against visual detection in real time.
[0165] JSON Parser. The output of the Pycrate library is a JSON bases text file. The system uses a custom written parser to parse the relevant sections of the map and SPAT data. Where integrated with Al- or neural network-based camera modules, the parser can additionally merge camera-detected traffic signal states into the parsed SPAT records, with rules to handle agreement, disagreement, or missing data. The map data is used to identify the intersections that are targeted for the technology proof of concept. The Parser then parses the SPAT data and forms a file called the “inputMAPSPaT.txt” file which has this string of the following format:SPSSRSSXSRSSSRSSSPSSSSSXPSSSPSSSSRSSRSSSFor e.g., in the string above, the second character is a “P” which corresponds to a protected movement (green signal) for the Signal Group 2 which controls southbound through lanes. The Al- or neural network-based system can perform similar classification visually, enabling fallback detection and validation of SPAT -derived states.
[0166] Figure 9 depicts how an embodiment of the system decodes data from an autonomous system number identifier (ASN) library (ASnlC), using JSON, to a SPAT parser, asnl c being a free, open-source compiler of ASN.l specifications into C source code and wherein an autonomous system number (ASN) is a globally unique identifier. The different states of the signal are represented as different letters:unavailable = N dark = D stop-then = proceed = T stop-and-remain = S pre-movement = M protected-movement-allowed = P permissive-movement-allowed = R protected-clearance = C permitted clearance = E caution-conflicting-traffic = LThe Al- or neural network-based vision module can map these same states to visually detected conditions, creating a synchronized data layer for redundancy.
[0167] Getting the Lat / Long Location. The server is set to subscribe to and receive the provider data once in 100 ms. However, the Server has no idea on the Latitude or the Longitude of the car. This is how the car’s position is passed on to the server:1) The Mobile App calls a webservice deployed on the server and passes on the Lat / long values of the Vehicle.2) The Server receives the Lat / long values of the Vehicle and stores it in the “latlong.txt fie” in the same directory of the Parser.
[0168] Finding the lane information. The next step in the process is to find the lane information and the signal information. A custom parser that parses the output of the JSON Parser is stored in the “inputMAPSPaT.txt” file. This Parser can be written in C language and it reads the “inputMAPSPaT.txt” file, parses the string data and writes into the output file. This parser finds the following based on the 8 Character string written in the “inputMAPSPaT.txt” for a particular intersection. It finds the following:1) Lane information2) Signal information - Red / Green / Y ellow3) Direction Allowed - Straight, Left of Right4) Count Down to Green Signal5) If available, Al- or neural network-based camera detection results that independently classify the current traffic light state, which can be used to confirm or override the SPAT-derived state if reliability thresholds are met.The Output File is called as “outputMAPSPaT.txt”. The “outputMAPSPaT.txt” file consists of just 3 characters separated by a comma:!) First Character - The Direction - Straight . Left of Right(2 Second Character - The Signal - Red / Green / Y ellow3) Third Character - The “Count Down to Green” AlertThe “outputMAPSPaT.txt” file may look like “N,R,0” or “S,Y,1” or S,G,0.First parameter:S - straight.L - left.R - right.Second parameter:R - red.Y - yellow.G - green.Third parameter:1 -Countdown to Green - positive.The Al- or neural network-based perception module can independently detect and timestamp these same parameters from live camera feed to corroborate the server’s “outputMAPSPaT.txt” results.
[0169] Cron Job. Cron is a utility program that lets users input commands for scheduling tasks repeatedly at a specific time. The Cron Job is written in CURL (standing for Client UR) and runs every 2 minutes. The Functions of the Cron Job are as follows:1) Check if the JSON parser written in Python is running2) Restart the JSON parser written in Python if it is NOT running.The Cron Job ensures that the server parsing is continuous and is never stopped.
[0170] Deciphering the Output File and the Lat / Long Location. The next step in the process is to find the lane information and the signal information and the vehicle position. A parser that parses the output of the JSON Parser is stored in the “inputMAPSPaT.txt” file. It also parses the Latlong.txt file and based on the Lat / Long it calculates the Lane information. After calculation of the Lane information, the Parser moves ahead and reads the ‘‘inputMAPSPaT.txt’’ file and reads the 8-character string that corresponds to that traffic light. Based on the lane information and the 8-character string, the parser forms the output values and writes them to an outputMAPSPaT.txf’file.
[0171] SPAT Flow. Figure 11 is a flow chart describing the SPAT flow. The SPAT flow application, whether in the user’s mobile device, or on the above-mentioned chip, functions to read and display data based on the map and SPAT data, following the following logic:1) SPAT will continuously monitor the Lat / long of the mobile device (and hence the vehicle).2) When the Vehicle is near any traffic Light the DDA Application checks alerts the user and sends the Traffic Light audible and visual alert. Simultaneously, the onboard Al- or neural network-based camera system detects the current signal state from the live video stream.3) The DDA Application then checks if the speed of the vehicle is 0 Mph and if the Lat / Long position is among the traffic lights that the Data provider provides data.4) If the speed of the vehicle is 0 Mph, and if the Lat / Long position is among the traffic lights that the Data provider provides data, the DDA Application understands that the vehicle is at a traffic light that has SPAT values streamed to the server.5) The DDA Application sends the Lat / Long position to the server every second to the Addlocation.PHP webservice on the server6) As discussed in the paragraph above, the Lat / Long position is stored in the “latlong.txt fie” in the same directory of the Parser by the Addlocation.PHP webservice.7) The Addlocation.PHP webservice on the server then proceeds to call the Custom parser that get the lane information.8) The custom parser than gets the Lane information, reads the 8-character strings in the “inputMAPSPaT.txt” file that correspond to the particular Lat / Long and based on the Lat / Long on the “latlong.txt fie” in the same directory of the Parser, it outputs the values in the “outputMAPSPaT.txt”9) The addlocation.PHP webservice then reads the “outputMAPSPaT.txt” file and returns the values to the DDA Application.10) The DDA Application then reads the output of the addlocation.PHP and based on the parameters, displays the Red / Green / Y ellow lights. If the SPAT-derived state differs from the Al- or neural network-based visual detection, the system initiates a validation process to reconcile the discrepancy and select the most reliable result.11) For systems with Al- or neural network-enabled camera modules, the DDA Application concurrently receives visual detections of traffic light or stop sign states, compares these with the SPAT-derived data, and applies a sensor fusion algorithm to determine and present the most accurate state information to the driver.12) The DDA Application continues to server webservice addlocation.PHP every second and display the latest traffic light information on the App.13) If the third parameter of the “outputMAPSPaT.txt” file returns the value “1”, then the sever sends a voice alert “Countdown to Green” based on a “Countdown to Green” flag set in the server, and starts a countdown, usually sent 5-6 seconds before the light turns green to alert the driver to stop any distracting activities (like texting) and get back on focus.
[0172] SDK. In an embodiment of the invention, a software development kit (SDK) is provided to transfer software to a vehicle GPS navigation system wherein data loading is isolated into the SDK. Different users of the invention can all use the same SDK. Thus, the SDK will contact aserver and load the point of interest (POI) data. In accordance with the invention, country codes and information about latitude and longitude locations of the POIs are provided as string values. The DDA Application uses the SDK for authentication, data loading, and alerts. The SDK does not implement alert generation, Al / camera perception, SPAT parsing, or sensor-fusion; those functions reside in the application or in-vehicle modules. For avoidance of doubt, the SDK is limited to POI data transport / storage and related authentication in all configurations. When integrated with Al- or neural network-based camera perception modules, the SDK can also facilitate the transfer and synchronization of visual detection results — such as the state of traffic lights or stop signs — alongside SPAT-derived data, enabling redundancy and real-time cross- verification.
[0173] SDK Functionalities. More specifically, the SDK functions to authenticate the client, initialize and integrate an API, and make a POI class SFK in the following steps:— Get Country Code using current location latitude , longitude— Get Access Token— Get POI Files using latitude, longitude— Get POI Files using country codes— Receive Al- or neural network-based detection results from onboard vehicle cameras and integrate them with SPAT data for enhanced traffic signal and stop sign state determination. Alternatively, in a refinement, the SDK provides optional data-loading progress / state callbacks; the SDK performs no Al / camera processing, SPAT handling, alert logic, or sensor fusion.
[0174] Flow of the DDA Application. The DDA Application:1. Creates a map with setting options.2. Calls the Authentication method of SDK.3. Gets the current latitude / longitude of the Vehicle.4. Provides current latitude / longitude to SDK and get traffic, school, and rail crossing files for that country / region.5. Reads the data (latitude and longitude) from files received from the API through the SDK.6. Gets latitude, longitude for traffic signal, rail crossing, school zone and show on map.7. Periodically queries the SDK for updated POI data based on current location (e.g., every 30 seconds or when approaching map boundaries).8. Invokes the application's fiision / alert module every second using current Lat / Long to evaluate proximity to POIs retrieved from SDK, fuse with SPAT / camera data if available, and determine if alerts should trigger.9. Shows the alert box with sound for traffic lights, railway crossings, and school zones when the application's alert logic determines notification is warranted.10. Receives and processes Al- or neural network-based camera detection results for traffic lights and stop signs, and displays visual and audible alerts when these detections confirm or supplement SPAT-based information.
[0175] SDK Methods- Get Token ID, Figure 12 is the code to generate a token Id. The token Id mainly used as a parameter while call other APIs from SDK methods. Put device id and api key as parameters. There are no return value while use this method. The device id and api key should be string values.
[0176] SDK Methods- GetPOI with Latitude and Longitude. This method is only for an American user. Figure 13 is the code to get the POI file name for the traffic signal and railway crossing and school zone in North and South America. We need to pass latitude, longitude, and apiKey,accessToken parameters in this method which should be a string value. There are no return values in this method. In some variations, this method may also trigger an Al- or neural network- based camera module to perform real-time visual detection of nearby traffic lights or stop signs, enabling the SDK to provide both location-based POI data and vision-based state information for improved accuracy.
[0177] SDK Methods - GetPOI with country code. Figure 14 is the code to get the POI file name for the traffic signal and railway crossing and school zone except North and South America. We need to pass latitude, longitude, countryCode, apiKey,accessToken in this method. Latitude, longitude, apiKey,accessToken parameter should be a string value. There are no return value while using this method. Similar to
[0141] , in certain variations, this method can also integrate Al- or neural network-based visual detection for traffic control devices to supplement POI data with live camera-derived information.
[0178] SDK Methods - Get Current tokenid. Below is the code used to get current token id. There are no input parameters for this method, currenttokenid is the return value of this method. The return value is a string value.Public static string getCurrenttokenid() { return currenttokenid;
[0179] SDK Methods - GetPOIfile size. Below is the code to get the size of the POI files ( example: some countries provide traffic file only, some countries provide traffic and school file path only, so we can get count of path files through this method). There are no input parameters for this method. The return value is an integer value. public static string getPoifilesize() { return poifilesize;
[0180] SDK Methods - GetFirstFile name. Below is the code to get the first POI file name. There are no input parameters for this method, fileone is the return value of this method. The return value is a string value. public static String getFileoneQ { return fileone;
[0181] SDK Methods - GetFirstFile Type. Below is the code to get the first POI file type (i.e., 1. Traffic light, 2. Schools, 3, railroad crossing). There are no input parameters for this method, fileonetype is the return value of this method. The return value is an integer value. public static int getFileonetype() { return fileonetype;
[0182] SDK Methods - GetSecondFile name. Below is the code to get the second POI file name. There are no input parameters for this method. Filetwo is the return value of this method. The return value is a string value. public static int getFiletwo() { return filetwo;
[0183] SDK Methods - GetSecondFile Type. Below is the code to get the second POI file type (i.e., 1. Traffic light, 2. Schools, 3, railroad crossing). There are no input parameters for this method, filetwotype is the return value of this method. The return value is an integer value. public static int getFiletwotype() { return filetwotype;
[0184] SDK Methods - GetThirdFile name. Below is the code to get the third POI file name. There are no input parameters for this method, filethree is the return value of this method. The return value is a string value.public static int getFilethree() { return filethree;
[0185] SDK Methods - GetThirdFile Type. Below is the code to get the third POI file type (i.e., 1. Traffic light, 2. Schools, 3, railroad crossing). There are no input parameters for this method, filetthreeype is the return value of this method. The return value is an integer value. public static int getFiletwotype() { return filethreetype;
[0186] Libraryyl - Android Library. This module generates AAR files which can be included Any android project. Figure 15 is an Android library used in an embodiment of this invention and shows menu items for a module to which to add AAR dependency. In certain variations, the Android library also includes APIs to receive Al- or neural network-based camera input for traffic light and stop sign recognition, allowing the application to cross-check SPAT and POI-based state data with real-time vision-based results.
[0187] While exemplary embodiments are described above, it is not intended that these embodiments describe all possible forms of the invention. Rather, the words used in the specification are words of description rather than limitation, and it is understood that various changes may be made without departing from the spirit and scope of the invention. Additionally, the features of various implementing embodiments may be combined to form further embodiments of the invention.
Claims
WHAT IS CLAIMED IS:
1. A computer-implemented method for improving safety for use with a navigation map system in a vehicle, the method comprising: obtaining vehicle location from one or more positioning sources; accessing data indicative of a point of interest (POI) location or receiving image data sufficient to estimate a relative position of the POI; determining whether the POI is within a predetermined proximity to the vehicle based on at least one of the vehicle location and the estimated relative position; determining whether the vehicle is moving; receiving image data from a camera system and processing the image data with a vision-processing module to detect the POI and estimate its relative position as a proximity determination; and providing an audible alert when the proximity determination is within predetermined distance such that driver distraction is reduced during an active hand free voice call.
2. The method of claim 1, wherein the vision-processing module is an artificial intelligence (Al) vision-processing module or neural network module that processes the image data to detect the point of interest and / or estimate its relative position with respect to the vehicle.
3. The method of claim 1, wherein one or more Advanced Driver- Assistance Systems (ADAS) sensors are used for at least one of (i) detecting the POI, (ii) estimating a range, bearing, or time-to-arrival to the POI, (iii) determining whether the vehicle is moving based on vehicle-motion signals, and (iv) validating the proximity determination, the ADAS sensors comprising at least one of a camera, a LiDAR sensor, a millimeter-wave radar sensor, an ultrasonic sensor, an inertial measurement unit, a wheel-speed sensor, a steering-angle sensor, or a yaw-rate sensor.
4. The method of claim 3, wherein outputs from at least two ADAS sensors are processed to produce the proximity determination, with range and bearing to the POI obtained from a LiDAR sensor or a millimeter-wave radar sensor when available and otherwise derived from vision-baseddepth, and wherein individual sensor contributions are selected, weighted, or suppressed based on ambient conditions or a sensor self-test result.
5. The method of claim 1 , wherein receiving image data comprises receiving frames from a forward-facing monocular camera, and processing the image data comprises executing an anti-flicker routine configured to detect a state of a traffic signal, wherein the POI is selected from the group consisting of a traffic signal, a stop sign, a pedestrian crosswalk, a railroad crossing, a school zone, and a work zone, and further comprising receiving infrastructure message data indicative of signal phase and timing (SPAT) for the POI and fusing the infrastructure message data with an output of the vision-processing module, the fusing comprising temporally synchronizing the infrastructure message data and the image data and buffering the infrastructure message data to compensate for network latency, and further comprising computing a confidence score for the POI detection.
6. The method of claim 1, wherein the one or more positioning sources comprise a global navigation satellite system (GNSS) receiver and optionally an inertial measurement unit (IMU).
7. The method of claim 1 , wherein a vehicle accelerometer system is configured to initiate the audible alert.
8. The method of claim 1 , wherein the audible alert and / or the navigation map is provided routed through a cell phone and / or an infotainment system such that a hands-free call optionally flow through the infotainment system.
9. The method of claim 1 , wherein the audible alert comprises one or more of: (i) a distinct tone pattern associated with a type of the point of interest, (ii) a spoken phrase naming the point of interest or identifiable with the point of interest, and (iii) a chime sequence mapped to the type of the point of interest.
10. The method of claim 8, wherein the point of interest is selected from the group consisting of a traffic signal, a school zone, a railroad crossing, and combinations thereof.
11. The method of claim 8, wherein the audible alert provides a spoken phrase indicated that a traffic signal will turn from green or yellow to red or from red to green or wherein audible alerts provides a spoken phrase or chime indicating that alert mode is activated.
12. The method of claim 1, further comprising: determining that the vehicle is moving faster than ten miles per hour; determining, from sensors in a wireless communication device or the vehicle, an acceleration or a change in motion; and allowing the vehicle to be commanded to slow down via a wireless network during an active voice call.
13. The method of claim 1, further comprising determining whether a wireless communication device is in an active voice mode and, responsive to a positive determination, displaying an icon of the point of interest.
14. The method of claim 13, further comprising displaying an icon of a phone only when the wireless communication device is in an active voice mode.
15. The method of claim 1, wherein, in addition to displaying an icon of the point of interest, an audible alarm is sounded.
16. The method of claim 15, wherein a navigation map displays a line from a present vehicle location to the point of interest.
17. The method of claim 15, wherein the point of interest is a traffic light and the icon is an icon of a traffic light.
18. The method of claim 17 further comprising the step of determining if the traffic light is a yellow light or a red light or is calculated to be yellow or red by the time the vehicle reaches the traffic light, and displaying said icon only if such determination is positive.
19. The method of claim 15, wherein the point of interest is a school zone and the icon is an icon of a school.
20. The method of claim 15, wherein the point of interest is a railway crossing and the icon is an icon of a railway crossing barrier.
21. The method of claim 1, wherein the audible alert includes a sound indicative of characteristics of the point of interest.
22. The method of claim 1, farther comprising receiving distance measurements from a LiDAR sensor mounted to the vehicle and estimating a range and bearing to the point of interest.
23. The method of claim 22, farther comprising time-stamping LiDAR measurements and synchronizing them with camera image data and / or SPAT data within a synchronization tolerance window.
24. The method of claim 22, farther comprising fusing camera-based detection with a LiDAR-derived range and / or bearing to determine proximity to the point of interest.
25. The method of claim 24, wherein fusing weights LiDAR more heavily under low- illumination, glare, fog, precipitation, or other reduced- visibility conditions.
26. The method of claim 22, wherein LiDAR measurements are used as a fallback primary input when the camera system is unavailable, saturated, or below a quality threshold.
27. The method of claim 23, further comprising transforming LiDAR points from a vehicle coordinate frame into geographic coordinates using a vehicle pose estimate and storing resulting longitude / latitude in a same format as a point-of-interest database.
28. The method of claim 23, further comprising computing a confidence score from LiDAR returns and issuing the audible alert only when a score exceeds a threshold.
29. The method of claim 23, wherein the LiDAR sensor comprises at least one of a two- dimensional scanning LiDAR or a three-dimensional LiDAR.
30. The method of claim 25, further comprising maintaining a temporally filtered track of the point of interest to bridge short-term occlusions based on successive LiDAR measurements.
31. The method of claim 25, further comprising cross-validating LiDAR-derived geometry of an intersection structure with SPAT data identifying an intersection to confirm the point of interest prior to issuing the audible alert.
32. A method for improving safety for use with a navigation GPS map system that can display a current location of a vehicle that contains a navigation system on a map generated by the system and in which a wireless communication device is being used in the vehicle, the method comprising:(a) determining the current location of the vehicle;(b) determining a GPS location of a point of interest;(c) determining if the vehicle is within a predetermined distance from a point of interest;(d) determining if the vehicle is moving;(e) determining if the wireless communication device is in an active voice mode whereupon an icon of the point of interest is displayed only if such determination is positive; and(f) determining if the vehicle is within a predetermined distance of a point of interest such that an audible alert is provided when the vehicle is within the predetermined distance of a point of interest.
33. The method of claim 32, wherein one or more Advanced Driver-Assistance Systems (ADAS) sensors are used for at least one of (i) detecting the POI, (ii) estimating a range, bearing, or time-to-arrival to the POI, (iii) determining whether the vehicle is moving based on vehicle-motion signals, and (iv) validating a proximity determination, the ADAS sensors comprising at least one of a camera, a LiDAR sensor, a millimeter-wave radar sensor, an ultrasonic sensor, an inertial measurement unit, a wheel-speed sensor, a steering-angle sensor, or a yaw-rate sensor.
34. The method of claim 33, wherein a vehicle accelerometer system is configured to initiate the audible alert.
35. The method of claim 34, wherein the vehicle accelerometer system is configured to slow down while on a voice call.
36. The method of claim 33 further comprising: determining if the vehicle is moving faster than ten miles per hour; determining if sensors in the wireless communication device or the vehicle measures acceleration or changes in an object over time; and allowing the vehicle to slow down via a wireless network from an active voice call.
37. The method of claim 33 further comprising determining if the wireless communication device is in an active voice mode whereupon an icon of the point of interest is displayed only if such determination is positive.
38. The method of claim 33 further including a step of displaying an icon of a phone only if a determination that the wireless communication device is in an active voice mode is positive.
39. The method of claim 33 in which in addition to an icon of the point of interest being displayed, an audible alarm is sounded.
40. The method of claim 33 in which the map displays a line from a present location to a displayed point of interest.
41. The method of claim 33 wherein the point of interest is a traffic light and the icon is an icon of a traffic light.
42. The method of claim 41 further including a step of determining if the traffic light is a yellow light or a red light or is calculated to be yellow or red when the vehicle reaches the traffic light, and displaying said icon only if such determination is positive.
43. The method of claim 33 wherein the point of interest is a school zone and the icon is an icon of a school.
44. The method of claim 33 wherein the point of interest is a railway crossing and the icon is an icon of a railway crossing barrier.
45. The method of claim 33 wherein the audible alert includes a sound that indicates charascteristics of the point of interest.
46. A driver distraction alert system for enhancing safety at traffic lights, school zones, and railroad crossings, comprising: an artificial intelligence (Al) module configured to apply geofencing technology to create virtual boundaries around school zones and railroad crossings; a notification module configured to provide visual and audible alerts to a driver approaching said school zones and railroad crossings, the audible alerts being triggered by the Al module; an integration module configured to interface with a vehicle's navigation system to provide real-time information about school zones, railroad crossings, and traffic lights along a route; a voice assistant module configured to provide verbal reminders about the presence of school zones, railroad crossings, and traffic lights; a contextual awareness module configured to analyze a driver’s conversation context and driving conditions to deliver contextually relevant alerts; and a customization module configured to allow the driver to customize types of alerts received based on preferences and driving habits.
47. The system of claim 46, wherein the Al module is further configured to recognize road signs, detect pedestrians, and identify changes in traffic conditions using advanced computer vision techniques.
48. The system of claim 46, wherein the notification module is further configured to display visual warnings on a vehicle's dashboard or heads-up display and play audible alerts through a vehicle's speakers.
49. The system of claim 46, wherein the integration module is further configured to offer proactive alerts, ensuring the driver is aware of upcoming hazards well in advance.
50. The system of claim 46, wherein the voice assistant module is further configured to provide verbal reminders that help a driver prepare for necessary actions, such as slowing down or stopping.
51. The system of claim 46, wherein the contextual awareness module is further configured to prioritize delivering alerts that are timely and relevant to a current driving scenario.
52. The system of claim 46, wherein the customization module is further configured to ensure that alerts are effective and not overly distracting, enhancing a driver's overall experience and safety.
53. The system of claim 46, wherein the Al module is further configured to analyze historical traffic data and driver behavior to predict when the driver is likely to encounter a traffic light, adjusting a route or driving behavior accordingly.
54. The system of claim 46, wherein the voice assistant module is further configured to enable hands-free operation of a vehicle's systems, allowing the driver to stay focused on the road while managing calls, navigation, and other tasks.
55. The system of claim 46, wherein the Al module is further configured to take over control of a vehicle or contact emergency services in case of an emergency or if the driver fails to respond to alerts.
56. The system of claim 46, wherein the Al module is further configured to monitor realtime data from vehicle sensors to detect signs of driver distraction, such as drowsiness, inattention, or phone usage.
57. The system of claim 46, wherein the Al module is further configured to analyze driver behavior patterns over time to identify deviations indicating distraction and trigger alerts.
58. The system of claim 46, wherein the Al module is further configured to use facial recognition technology to monitor a driver's face and detect signs of drowsiness, fatigue, or distraction.
59. The system of claim 46, wherein the Al module is further configured to analyze a driver's voice commands and responses to detect signs of distraction or drowsiness, triggering alerts based on unusual or slurred speech patterns.
60. The system of claim 46, wherein the Al module is further configured to integrate with other vehicle safety features, such as collision avoidance systems or lane departure warning systems, to provide a comprehensive safety net for the driver.
61. The system of claim 46, wherein the Al module is further configured to utilize predictive maintenance features to monitor vehicle performance and predict potential mechanical issues before they become critical.
62. The system of claim 46, wherein the Al module is further configured to facilitate integration with smart city infrastructure, receiving real-time updates on traffic light status, road closures, and other pertinent information.
63. The system of claim 46, wherein one or more Advanced Driver- Assistance Systems (ADAS) sensors are used for at least one of (i) detecting the POI, (ii) estimating a range, bearing, ortime-to-arrival to the POI, (iii) determining whether the vehicle is moving based on vehicle-motion signals, and (iv) validating a proximity determination, the ADAS sensors comprising at least one of a camera, a LiDAR sensor, a millimeter-wave radar sensor, an ultrasonic sensor, an inertial measurement unit, a wheel-speed sensor, a steering-angle sensor, or a yaw-rate sensor.
Citation Information
Patent Citations
Driver fatigue detection method based on neural network
CN110119676A
Vehecle speed control system within thd school zone and vehecle speed control method
KR1020130115041A
Intelligent information conversion for automatic driving
US20220219731A1
Systems and methods for safe and reliable autonomous vehicles
US20240045426A1
Safe driving system generating map points
WO2018125128A1