Navigation system with traffic status detection mechanism and operation method thereof

Abstract

A navigation system includes: a control circuit configured to: generate a video clip by parsing intervals of a sensor data stream of a travel area; analyze the video clip submitted to a trained deep learning model, including identifying a traffic flow estimate; access location coordinates for calculating a distance to an intersection; generate a traffic flow state by fusing a corrected speed, the traffic flow estimate, and the distance to the intersection; merge vehicle maneuvering instructions into the traffic flow state for maneuvering through the travel area; and a communication circuit coupled to the control circuit, the communication circuit configured to: transmit the traffic flow state for display on a device.

Description

Technical Field

[0001] Embodiments of the present invention generally relate to navigation systems, and more particularly to systems having a traffic status detection mechanism. Background Art

[0002] Historically major metropolitan areas have been characterized by their traffic jams. While traffic jams were limited to the start of work, lunch, and the end of the workday, this has now changed. In cities like Seoul, Tokyo, Paris, London, New York, and Los Angeles, traffic jams never end. This results in a loss of resources and productivity due to wasted time and consumed fuel. The increase in the number of cars has spread traffic jams far beyond city limits and into the suburbs. Current traffic monitoring systems are based on monitoring the speed of vehicles detected by electromagnetic sensors embedded in high-traffic highways, but are not used on city streets.

[0003] As the number of vehicles on the road increases, relying on route planning based on speeds detected on highways that can change instantly becomes unreliable. Some attempts to monitor traffic flow have gone into car telematics systems for speed detection, but the speed at which a vehicle is driving may not reflect the actual traffic flow. If a vehicle is mechanically damaged due to engine failure or tire pressure loss, the traffic flow may be completely distorted.

[0004] Therefore, there remains a need for a navigation system with a traffic state detection mechanism. Finding answers to these questions is increasingly critical given the increasing commercial competitive pressures, as well as growing consumer expectations and diminishing opportunities for meaningful product differentiation in the marketplace. Additionally, the need to reduce costs, improve efficiency and performance, and meet competitive pressures adds even greater urgency to the critical necessity of finding answers to these questions.

[0005] Solutions to these problems have been long sought but prior developments have not taught or suggested any solutions and, thus, solutions to these problems have long eluded those skilled in the art. Summary of the invention

[0006] An embodiment of the present invention provides a method for operating a navigation system, comprising: generating a video clip by parsing a sensor data stream over a period of time in a travel area; analyzing the video clip submitted to a trained deep learning model, including identifying traffic flow estimates; accessing location coordinates for calculating the distance to an intersection; generating a traffic flow state by fusing corrected speed, traffic flow estimates, and distance to an intersection; incorporating vehicle maneuvering instructions into the traffic flow state for maneuvering through the travel area; and transmitting the traffic flow state for display on a device.

[0007] An embodiment of the present invention provides a navigation system, including a control circuit, which is configured to: generate a video clip by parsing a sensor data stream over a period of time in a travel area; analyze the video clip submitted to a trained deep learning model, including identifying traffic flow estimates; access location coordinates for calculating the distance to an intersection; generate a traffic flow state by fusing a corrected speed, the traffic flow estimate and the distance to the intersection; merge vehicle maneuvering instructions into the traffic flow state for maneuvering through the travel area; and a communication circuit, which is coupled to the control circuit, the communication circuit being configured to: transmit the traffic flow state for display on a device.

[0008] Embodiments of the present invention provide a non-transitory computer-readable medium including instructions for a navigation system, including: generating a video clip by parsing a sensor data stream of a travel area over a period of time; analyzing the video clip submitted to a trained deep learning model, including identifying lane lines and vehicles; accessing location coordinates for calculating real-world coordinates; generating a traffic flow state by fusing corrected speeds, traffic views, and real-world coordinates; incorporating vehicle maneuvering instructions into the traffic flow state for maneuvering through the travel area; and transmitting the traffic flow state for display on a device.

[0009] Certain embodiments of the present invention have other steps or elements in addition to the above steps or elements, or have other steps or elements to replace the above steps or elements. These steps or elements will become clear to those skilled in the art by reading the following detailed description when referring to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 is a block diagram of a navigation system with a traffic status detection mechanism in an embodiment of the present invention.

[0011] Figure 2 is an example of a top view of a first device configured to provide a traffic status detection mechanism in an embodiment.

[0012] Figure 3 is an exemplary top view of the travel area processed by the navigation system.

[0013] Figure 4 is an exemplary functional block diagram of a traffic status detection mechanism of a navigation system.

[0014] Figure 5 is an exemplary block diagram of a navigation system in an embodiment.

[0015] Figure 6 is an exemplary operational flow chart of a navigation system.

[0016] Figure 7The figure is a flow chart of an operating method of a navigation system in an embodiment of the present invention. DETAILED DESCRIPTION

[0017] The following embodiments can accurately identify lanes and objects, which enables vehicle movement control for operating or controlling the physical movement of the vehicle. The vehicle movement control can be based on a driver-assisted or autonomous vehicle driving process, which is safe and reliable due to the accuracy of lane and object detection.

[0018] Vehicle movement control may also be based on accurately identifying traffic conditions in real time to ensure that driver-assisted or autonomous vehicle maneuvers can be completed without risk of damaging the vehicle or any adjacent objects or property.

[0019] The following embodiments are described in sufficient detail to enable those skilled in the art to make and use the invention. It is understood that other embodiments will be apparent based on this disclosure, and that the system, process or mechanical changes may be made without departing from the scope of the embodiments of the invention.

[0020] In the following description, many specific details are given to provide a thorough understanding of the present invention. However, it is apparent that the present invention can be practiced without these specific details and will be clear. In order to avoid obscuring the embodiments of the present invention, some well-known circuits, system configurations and processing steps are not disclosed in detail.

[0021] The drawings showing embodiments of the system are semi-diagrammatic and not to scale, and in particular, some dimensions are for clarity of presentation and are shown exaggerated in the drawings. Similarly, although the views in the drawings generally show similar orientations for ease of description, such depictions in the drawings are arbitrary for the most part. Generally, the present invention can be operated in any orientation. The embodiments of the various components are for ease of description and are not intended to have any other meaning or provide limitations to the embodiments of the present invention.

[0022] Those skilled in the art will appreciate that the format in which the navigation information is expressed is not critical to some embodiments of the present invention. For example, in some embodiments, the navigation information is presented in the format of (X, Y, Z); wherein X, Y, and Z are three coordinates defining the geographic location (i.e., the location of the vehicle).

[0023] Depending on the context in which the term "module" is used, in the present invention, the term "module" referred to herein may include or be implemented as or include software running on dedicated hardware, hardware, or a combination thereof. For example, software may be machine code, firmware, embedded code, and application software. Software may also include functions, calls to functions, code blocks, or a combination thereof.

[0024] Furthermore, for example, hardware may be a gate, circuit, processor, computer, integrated circuit core, memory device, pressure sensor, inertial sensor, micro-electromechanical system (MEMS), passive device, physical non-transitory storage medium including instructions for executing software functions, a portion thereof, or a combination thereof, to control one or more of the hardware units or circuits. Furthermore, if "unit" is written in the system claim section below, "unit" is deemed to include hardware circuits for the purpose and scope of the system claim.

[0025] The units in the following description of the embodiments may be coupled or attached to each other as described or shown. The coupling or attachment may be direct or indirect, with no or intermediate items between the coupled or attached modules or units. The coupling or attachment may be by physical contact or by communication (such as wireless communication) between the modules or units.

[0026] The term "fused" as used in the specification and claims has the meaning of combining, connecting, merging or integrating as appropriate to the use of the term.

[0027] It should also be understood that the nouns or elements in the embodiments may be described as singular instances. It should be understood that the use of the singular is not limited to the singular, but the use of the singular may apply to multiple instances of any particular noun or element in the present application. Many instances may be the same or similar or may be different.

[0028] Reference now Figure 1 , which shows a block diagram of a navigation system 100 with a traffic status detection mechanism in an embodiment of the present invention. The navigation system 100 may include a first device 102 (eg, a client or a server), which is connected to a second device 106 (eg, a client or a server).

[0029] The navigation system 100 may include a system for identifying traffic status based on fusion of multiple sources to coordinate and quickly identify current traffic status to assist in making lane change or route change decisions. The first device 102 may communicate with the second device 106 via a network 104 (eg, a wireless or wired network).

[0030] For example, the first device 102 may be any of a variety of computing devices (e.g., a cellular phone, a personal digital assistant, a laptop computer, a wearable device, an Internet of Things (IoT) device, an automotive telematics navigation system, or other multi-function device). In addition, for example, the first device 102 may include a device or subsystem, an autonomous or self-manipulating vehicle or object, a driver-assisted vehicle, a remotely controlled vehicle or object, or a combination thereof.

[0031] The first device 102 can be coupled directly or indirectly to the network 104 to communicate with the second device 106, or can be a standalone device. The first device 102 can further be standalone or incorporated with a vehicle (eg, a car, truck, bus, or motorcycle).

[0032] For illustrative purposes, the navigation system 100 is described with the first device 102 as a mobile computing device, although it should be understood that the first device 102 can be different types of devices. For example, the first device 102 can also be a non-mobile computing device (such as a server, a server cluster, or a desktop computer).

[0033] The second device 106 can be any of a variety of centralized or decentralized computing devices. For example, the second device 106 can be a computer, a grid computing resource, a virtualized computer resource, a cloud computing resource, a router, a switch, a peer-to-peer distributed computing device, or a combination thereof.

[0034] The second device 106 can be centralized in a single room, distributed across different rooms, distributed across different geographic locations, embedded within a telecommunications network. The second device 106 can be coupled to the network 104 to communicate with the first device 102. The second device 106 can also be a client type device as described for the first device 102.

[0035] For illustrative purposes, the navigation system 100 is described with the second device 106 as a mobile computing device, although it should be understood that the second device 106 can be a different type of computing device. For example, the second device 106 can also be a mobile computing device (such as a laptop computer, another client device, a wearable device, or a different type of client device).

[0036] Furthermore, for illustrative purposes, the navigation system 100 is described with the second device 106 as a computing device, although it should be understood that the second device 106 can be a different type of device. Furthermore, for illustrative purposes, the navigation system 100 is shown with the second device 106 and the first device 102 as endpoints of the network 104, although it should be understood that the navigation system 100 can include different divisions between the first device 102, the second device 106, and the network 104. For example, the first device 102, the second device 106, or a combination thereof can also serve as part of the network 104.

[0037] The network 104 can span and represent various networks. For example, the network 104 can include wireless communication, wired communication, light, ultrasound, or a combination thereof. Satellite communication, cellular communication, Bluetooth, infrared data association standard (IrDA), wireless fidelity (WiFi), and world microwave access interoperability (WiMAX) are examples of wireless communications that can be included in the communication path. Ethernet, digital subscriber line (DSL), fiber to the home (FTTH), and plain old telephone service (POTS) are examples of wired communications that can be included in the network 104. In addition, the network 104 can traverse multiple network topologies and distances. For example, the network 104 can include a direct connection, a personal area network (PAN), a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), or a combination thereof.

[0038] The navigation system 100 may provide additional features that are not available in prior art navigation systems. The first device 102 may be coupled to an optical sensor 110 and a sensor data local storage 108. The optical sensor 110 is a sensor (e.g., a monocular camera set positioned on the first device 102) that is configured to monitor, observe, record, or a combination thereof, the surrounding environment of the first device 102. The sensor data local storage 108 provides a non-transitory storage medium including a video clip unit 113 to store data captured by the optical sensor 110. The video clip unit 113 may include two sets of memory in the sensor data local storage 108 that are capable of continuously storing clips of a fixed length (such as 3 seconds) of captured video frames. One of the clips may be stored and processed while the next clip is loaded into the memory. Once processed, the clips may be serially added to the sensor data local storage 108.

[0039] For example, the sensor data local storage device 108 can be implemented in a variety of ways, such as a non-volatile storage device (e.g., a hard drive, a solid-state storage device (SSD), a flash memory card, or a combination thereof) that is capable of parsing the sensor data stream into segments of finite length (e.g., 3 second segments).

[0040] The first device 102 may be coupled to a traffic estimation module 115 (e.g., a pixel evaluation module), which may be implemented in dedicated hardware, full hardware, or a combination thereof in software execution, which is configured to analyze the photographic scene to identify vehicle traffic in front of and around the first device 102. The traffic estimation module 115 may parse the sensor data stream 111 including sample frames of video data to identify an estimated traffic state of the scene captured by the optical sensor 110. During the training process, the first device 102 may upload the sensor data stream 111 to the second device 106 for further analysis or to generate an updated version of the multi-layer neural network 116 to improve the detection of the estimated traffic state for traffic state detection.

[0041] The traffic estimation module 115 may include a multi-layer neural network 116, a traffic classifier module 118, and a frame sampler module 120, which may generate an estimate of the traffic flow state 109. The multi-layer neural network 116 may be a software or hardware module capable of performing matrix math operations on input sample frames of scan data within the sensor data stream 111 in order to identify an estimated traffic state detected by the optical sensor 110.

[0042] The traffic classifier module 118 may be a software or hardware module that is capable of determining the presence of a vehicle based on a grouping of sample frames passed to the multi-layer neural network 116, which may be stored in the first device 102 or the second device 106. The traffic classifier module 118 may receive input from the multi-layer neural network 116, which has a powerful ability to extract complex features from the sample frames. The sensor data stream 111 may be analyzed by submitting the scan data portion of the sensor data stream 111 to the multi-layer neural network 116. It should be understood that other portions of the sensor data stream 111, including time, real-world position, and non-intrinsic parameters of the optical sensor 110, may be stored in the first device 102 or the second device 106 for subsequent operations.

[0043] The frame sampler module 120 may be a software or hardware module capable of selecting samples of frames presented by the sensor data stream 111. The frame sampler module 120 may include a set of updateable parameters for identifying how many sampled frames and which sampled frames are presented to the multi-layer neural network 116. The analysis results of the frame sampler module 120 may be stored in a storage circuit of the first device 102 or the second device 106. It should be understood that the frame sampler module 120 may change the sampling strategy when some fluctuations occur in the sensor data stream 111 representing different scenes.

[0044] The traffic classifier module 118 may output an estimate of a traffic flow state 109, which includes a reference indicator provided by the first device 102 indicating whether the analyzed frame represents a congested state, a slow state, or a smooth traffic state. The traffic flow state 109 may provide a maneuver instruction to the vehicle to increase speed, decrease speed, change lanes, or change position within a lane.

[0045] The sensor data local storage 108 can be coupled to the optical sensor 110 to store the sensor data stream 111 returned from the second device 106 during the training period and the adjustments to the multi-layer neural network 116. The first device 102 can assemble frames for the optical sensor 110 to generate the sensor data stream 111 for analysis. The sensor data stream 111 can provide information captured by the optical sensor 110 and recorded in the sensor data local storage 108. The first device 102 can transmit the traffic state update 121 to the second device 106 via the network 104. The traffic state update 121 can be a compilation of the traffic flow state 109 determined by the first device 102 within a fixed time period (e.g., one minute). The second device 106 can distribute the traffic state update 121 to other users who are approaching the location identified in the traffic state update 121 so that they can make adjustments to avoid potential danger.

[0046] The navigation system 100 can be operated by a user 112. The user 112 may include a person or entity that accesses or utilizes the navigation system 100 or a device therein. For example, the user 112 may include a person, a service, or a combination thereof that owns or operates the first device 102. In addition, for example, the user 112 may access or utilize the second device 106 through the first device 102, a service, or a combination thereof.

[0047] The navigation system 100 may further process direct user input 114 from the user 112. The direct user input 114 may include a request for navigation assistance, location of a point of interest, parking assistance, restaurant assistance, lodging assistance, location of a gas station, event reservation, or a combination thereof. The direct user input 114 may be provided directly to or on the first device 102 by or from the user 112. The direct user input 114 may include input or stimulus directly for or related to the corresponding software, application, feature, or a combination thereof.

[0048] The navigation system 100 can implement one or more embodiments without direct user input 114. The navigation system 100 can further implement one or more embodiments using direct user input 114 independently thereof. The direct user input 114 can include a speed increase, a speed decrease, a position change within a lane, or a change of lane as prompted by the user 112.

[0049] The second device 106 may periodically receive traffic status updates 121 from the first device 102, or receive the traffic status updates when requesting services for route planning or identification or points of interest along the current road. The second device 106 may distribute the traffic status updates 121 to other users who are close to the location identified by the traffic status updates 121 from the first device 102.

[0050] The second device 106 can analyze the traffic status update 121 and generate an improvement of the multi-layer neural network 116 for use by the first device 102 during the training process. As an example, the second device 106 can apply the traffic status update 121 to the map activity manager 122. The map activity manager 122 can parse the traffic status update 121 to update the background map database 124, the real-time traffic model 126, and the hazard warning model 128. The background map database 124 may include a graphical display of roads, highways, and intersections in a given area. The real-time traffic model 126 can apply the traffic status update 121 received from the first device 102 to the background map database 124 to convey the current road conditions to other users of the navigation system 100. The hazard warning model 128 can apply the traffic status update 121 to the background map database 124 to indicate areas representing dangerous conditions, including traffic accidents, severe congestion, road construction, slow traffic, speeding traffic, or situations that increase the risk of accidents.

[0051] The navigation system 100 has been found to reliably identify the traffic flow state 109 in order to provide real-time updates of traffic conditions to the first device 102. The traffic flow state 109 may be normalized over a fixed time period to generate a traffic state update 121. By sending the traffic state update 121 to the second device 106, the communication information may be distributed to other users of the navigation system 100 for route planning, traffic or accident warnings, lane selection warnings, construction warnings, etc. The navigation system 100 may improve the safety of the first device 102 by providing real-time traffic updates, lane suggestions, alternative routes, or a combination thereof.

[0052] Reference now Figure 2 , which shows the Figure 1 An example top plan view of a vehicle of the navigation system 100 that can include or interact with the first device 102 .

[0053] The first device 102 may be an object or machine for transporting people or goods, which can automatically manipulate or operate the object or machine. The first device 102 may include a Figure 1 The first device 102 may include a vehicle accessed by the user 112 for controlling, manipulating, operating, or a combination thereof. For example, the first device 102 may include a car, a truck, a cart, a drone, or a combination thereof.

[0054] There is no corresponding operation or movement Figure 1The first device 102 may be further controlled or manipulated without direct user input 114. For example, the first device 102 may include an autonomous vehicle, or a vehicle with automatic manipulation features (e.g., smart cruise control or preventive braking). The first device 102 may include a smart cruise control feature that can set and adjust the travel speed of the first device 102 without direct user input 114. For another example, the first device 102 may be controlled or manipulated by the navigation system 100, including the navigation system 100 controlling or setting the cruising speed, lane position, or other physical manipulation or movement of the first device 102.

[0055] The navigation system 100 can further utilize traffic flow status 109 from one or more vehicles or devices. Figure 1 The traffic status update 121 may include information about traffic conditions observed by the first device 102. The traffic status update 121 may be an indication of current conditions of traffic in an area in which the first device 102 is traveling.

[0056] The traffic status update 121 may also include information describing or representing the surroundings or environment of the device or vehicle reporting the traffic status update 121. For example, the traffic status update 121 may include speed data, navigation data, traffic data, or a combination thereof.

[0057] The first device 102 or other vehicle interfaced with the navigation system 100 may include devices, circuits, one or more specific sensors (e.g., environmental sensors 210), or a combination thereof, for providing auxiliary or additional information to the user 112 controlling, manipulating, or operating the first device 102. The first device 102 or any other vehicle may include vehicle communication circuitry 204, vehicle control circuitry 206, vehicle storage circuitry 208, other interfaces, or a combination thereof.

[0058] The vehicle storage circuit 208 may include a functional unit or circuit that is integrated into the corresponding first device 102 and is configured to store and recall information. The vehicle storage circuit 208 may be a volatile memory, a non-volatile memory, an internal memory, an external memory, or a combination thereof. For example, the vehicle storage circuit 208 may be a non-volatile storage device (e.g., a non-volatile random access memory (NVRAM), a flash memory, a disk storage device), or a volatile storage device (e.g., a static random access memory (SRAM)).

[0059] The vehicle storage circuit 208 may store vehicle software, other relevant data (eg, input information, information from sensors, processing results, information predetermined or preloaded by the navigation system 100 or vehicle manufacturer, or a combination thereof).

[0060] The vehicle control circuit 206 may include functional units or circuits that are integrated into the first device 102 and configured to execute or implement instructions. The vehicle control circuit 206 may execute or implement vehicle software to provide intelligence corresponding to the vehicle, the navigation system 100, or a combination thereof.

[0061] The vehicle control circuit 206 can be implemented in a variety of different ways. For example, the vehicle control circuit 206 can be a processor, an application specific integrated circuit (ASIC), an embedded processor, a microprocessor, hardware control logic, a hardware finite state machine (FSM), a digital signal processor (DSP), or a combination thereof. As a more specific example, the vehicle control circuit 206 can include an engine control unit, one or more central processing units, or a combination thereof.

[0062] The vehicle communication circuit 204 may include a functional unit or circuit that is integrated into a corresponding vehicle (e.g., the first device 102, another vehicle, or a combination thereof). The vehicle communication circuit 204 may be configured to enable external communications to and from the corresponding vehicle. For example, the vehicle communication circuit 204 may allow the first device 102 to communicate with the corresponding vehicle. Figure 1 The second device 106 communicates with the second device 106.

[0063] The vehicle communication circuit 204 may also serve as a communication hub that allows the corresponding control vehicle to act as Figure 1 The vehicle communication circuit 204 is a part of the network 104 and is not limited to being an endpoint or terminal circuit of the network 104. The vehicle communication circuit 204 may include active and passive components (such as microelectronics or antennas) for interacting with the network 104. For example, the vehicle communication circuit 204 may include a modem, a transmitter, a receiver, a port, a connector, or a combination thereof for wired communication, wireless communication, or a combination thereof.

[0064] The vehicle communication circuit 204 may be coupled to the network 104 to directly send or receive information between the vehicle communication circuit 204 and the second device 106 as an endpoint of the communication, such as for direct line-of-sight communication or peer-to-peer communication. The vehicle communication circuit 204 may also be coupled to the network 104 to send or receive information through a server or another intermediate device between the endpoints of the communication.

[0065] The first device 102 or other vehicle may further include various interfaces. The first device 102 may include one or more interfaces for interaction or internal communication between functional units or circuits of the first device 102. For example, the first device 102 may include one or more interfaces (e.g., drivers, firmware, wire connections or buses, protocols, or a combination thereof) for the vehicle storage circuit 208, the vehicle control circuit 206, or a combination thereof.

[0066] The first device 102 or other vehicle may further include one or more interfaces for interacting with an occupant, operator or driver, passenger, or combination thereof relative to the corresponding vehicle. For example, the first device 102 or other vehicle may include a user interface 212 (e.g., a screen or touch screen, a speaker, a microphone, a keyboard or other input device, a dashboard, or a combination thereof) including an input or output device or circuit.

[0067] The first device 102 may further include one or more interfaces and switches or actuators for physically controlling movable components of the first device 102. For example, the first device 102 may include the one or more interfaces and control mechanisms to physically perform and control the manipulation of the first device 102, such as for autonomous driving, smart cruise control, or manipulation features.

[0068] Functional units or circuits in the first device 102 may operate individually and independently of other functional units or circuits. The first device 102 may operate individually and independently of the network 104, the second device 106, other devices or vehicles, or a combination thereof.

[0069] The above functional units or circuits may be implemented in hardware. For example, one or more of the functional units or circuits may be implemented using gates, circuits, processors, computers, integrated circuits, integrated circuit cores, pressure sensors, inertial sensors, micro-electromechanical systems (MEMS), passive devices, physical non-transitory storage media containing instructions for executing software functions, a portion thereof, or a combination thereof.

[0070] The environmental sensors 210 are each a device or circuit for detecting or identifying the environment of the corresponding vehicle. The environmental sensors 210 can detect, identify, determine (or a combination thereof) such as the state, surrounding environment, or movement of the corresponding vehicle. The environmental sensors 210 can detect, identify, determine (or a combination thereof) the environment inside the cabin of the corresponding vehicle, the environment outside and around the corresponding vehicle, or a combination thereof. The environmental sensors 210 can be implemented by the first device 102.

[0071] For example, the environmental sensor 210 may include a user interface 212, an optical sensor 214, a radar sensor 216, a position-motion sensor 218, or a combination thereof. The user interface 212 may include a projector, a video screen, a touch screen, a speaker, or any combination thereof. The user interface 212 may display Figure 1 traffic flow status 109, planned routes, lane suggestions, speed warnings, vehicle system alerts and combinations thereof.

[0072] The optical sensor 110 may include a sensor for detecting or determining visual information representing the exterior and surrounding environment of the corresponding vehicle. The optical sensor 110 may include a camera attached to or integrated with the corresponding vehicle or device. For example, the optical sensor 110 may include a camera (such as a forward-facing camera, a video camera, a rear-view or reverse camera, a side-view or blind spot camera, or a combination thereof). In addition, for example, the optical sensor 110 may include an infrared sensor, a night vision camera, or a night vision sensor.

[0073] The optical sensor 110 may further include a camera on the first device 102 or another user device of the user 112 connected to and interacting with the vehicle. The optical sensor 110 may also include a cabin camera for detecting or determining visual information of the vehicle or the interior of the cabin of the vehicle.

[0074] Radar sensor 216 may include an object detection system, device, or circuit. Radar sensor 216 may determine or identify the presence of an object or target (such as an obstacle or another vehicle) outside the corresponding device or vehicle, the relative position or distance between the object or target and the corresponding device or vehicle, or a combination thereof.

[0075] The radar sensor 216 can utilize radio waves to determine or identify the presence of an object or target, a relative position or distance relative to the first device 102 or other corresponding devices or vehicles, or a combination thereof. For example, the radar sensor 216 can include a proximity sensor or warning system, such as for an area that is geographically or physically relative to the front, rear, adjacent, or side of the first device 102, or a combination thereof.

[0076] The position-motion sensor 218 may be a sensor for identifying or calculating the geographic location of the corresponding vehicle or device, determining the movement or speed of the corresponding vehicle or device, or a combination thereof. The position-motion sensor 218 may include an accelerometer, a speedometer, a global positioning system (GPS) receiver or device, a gyroscope, or a compass, or a combination thereof. The first device 102 may include an environmental sensor 210 that is different from or in addition to the position-motion sensor 218. The position-motion sensor 218 may provide a gyroscope rate of change for monitoring rotation and speed from a speedometer.

[0077] The navigation system 100 can use one or more of the optical sensors 110 corresponding to one or more devices, one or more vehicles, or a combination thereof to generate a traffic flow state 109 describing or representing information about the environment surrounding the corresponding device or vehicle. The traffic flow state 109 can be further processed by the vehicle control circuit 206, stored in the vehicle storage circuit 208, transmitted to another device or vehicle via the vehicle communication circuit 204, or a combination thereof.

[0078] As a more specific example, the vehicle communication circuit 204 , the vehicle control circuit 206 , the vehicle storage circuit 208 , the optical sensor 110 , one or more interfaces, or a combination thereof may be included in the first device 102 or constitute the first device.

[0079] The navigation system 100 may utilize the traffic flow status 109 from the device, the vehicle, or a combination thereof to dynamically determine and map traffic and road conditions in a geographic area and vehicles, pedestrians, objects, or a combination thereof within the geographic area. The navigation system 100 may further utilize the traffic flow status 109 to provide information to other vehicles approaching or planning to enter the area of ​​the first device 102. As a more specific example, the navigation system 100 may use the traffic flow status 109 to dynamically locate and map vehicles on the road and provide forward-looking traffic information. The navigation system 100 may further utilize the traffic flow status 109 to control the movement of the first device 102 at a lane level granularity.

[0080] The navigation system 100 can provide the vehicle movement control 228 as a suggestion to the user 112 to steer or operate the first device 102. Details regarding utilization and processing of the traffic flow status 109 are discussed below.

[0081] The navigation system 100 can process and generate a vehicle movement control 228 for controlling or manipulating the first device 102. The vehicle movement control 228 is an instruction, signal, procedure, method, mechanism, or combination thereof that directs or controls the physical movement or travel of the first device 102.

[0082] The navigation system 100 can transmit the traffic status updates 121 from the first device 102 to the second device 106 at fixed intervals (e.g., one-minute intervals). For an illustrative example, the navigation system 100 enables the second device 106 to communicate the traffic status updates 121 from the first device 102 to other vehicles approaching or planning to enter the area reported by the first device 102.

[0083] Continuing with this example, the navigation system 100 can use the traffic flow status 109 generated or provided from the first device 102 without user input 114. The navigation system 100 can utilize the traffic flow status 109 to provide information, assist maneuvers, control maneuvers, or a combination thereof for other vehicles near the first device 102.

[0084] Continuing with this example, the navigation system 100 can transmit the traffic status update 121 to other devices or vehicles via the second device 106, or directly to other devices or vehicles, such as for a peer-to-peer communication system. The navigation system 100 can transmit the traffic status update 121 to inform other devices or vehicles of the location or status of the first device 102 itself, about other vehicles detected and identified around the first device 102, or a combination thereof.

[0085] As a more specific example, the navigation system 100 can use the traffic flow state 109 to generate the vehicle movement control 228, such as for steering, braking, setting or adjusting travel speed, auxiliary control, or a combination thereof. Details regarding the processing of the vehicle movement control 228 are discussed below.

[0086] Reference now Figure 3 , in which an example top view of a travel area 301 processed by the navigation system 100 is shown. The travel area 301 may include travel lanes, including lane 1 302, lane 2 304, lane 306, and lane 4 308. For example, the first device 102 may be located in lane 1 302 and close to a first other vehicle 310 (e.g., a slower vehicle) in lane 1 302. The first device 102 may detect that the traffic flow state is a congested traffic state 316, but the presence of an intersection 311 directly in front of the first other vehicle 310 may change the detection of the traffic flow state 109 to a slow traffic state 321 because the first other vehicle 310 is entering a turning state 319 of the intersection 311, so the traffic flow state 109 may change based on the presence of the intersection 311. If the planned route of the first device 102 does not intend to leave the highway in a short distance and there is no intersection nearby, this situation may be reported as a traffic flow state 109 classified as congested traffic.

[0087] Continuing with this example, if the first other vehicle 310 and the second other vehicle 312 are near or completely stopped in the adjacent driving lane 313, the presence of the second other vehicle 312 near the lane line 314 in the No. 2 lane 304 may be interpreted as the traffic flow state 109 being a congested traffic state 316. The detection of the congested traffic state 316 may cause the first device 102 to change lanes to the left to the No. 3 lane 306, which will provide the traffic flow state 109 of a smooth traffic state 317 because there are no other vehicles within 100 meters in the No. 3 lane 306. It should be understood that the adjacent driving lanes 313 are shown as the No. 1 lane 302 and the No. 2 lane 304 only as examples, and any number or combination of the No. 1 lane 302, the No. 2 lane 304, the No. 3 lane 306, and the No. 4 lane 308 divided by the lane line 314 are considered as adjacent lanes 313. It should also be understood that the navigation system 100 is not required to identify or distinguish lane 1 302 , lane 2 304 , lane 3 306 , and lane 4 308 , and they are identified only for ease of description.

[0088] In another example, the first device 102 may encounter a traffic accident 318 in lane 4 308, which will cause the traffic flow state 109 to be interpreted as a congested traffic state 316. When other vehicles 320 caught behind the traffic accident 318 merge from lane 4 308 to lane 3 306, the traffic flow state 109 for the other vehicles 320 may be interpreted as a turning state 319. When the other vehicles 320 pass the traffic accident 318, they may slow down to less than half of the posted speed limit provided by the background map database 124 in order to view the damage, which may create a slow traffic state 321. It should be understood that the examples provided are for ease of explanation and are not to be used on a limiting basis, as other classifications of traffic flow states 109 are contemplated.

[0089] It has been found that the detection of the traffic flow state 109 can cause the first device 102 to respond by adjusting the first device 102 Figure 1 The first device 102 may control or display instructions to the user 112 to maintain a safe position in the lane or change the position to a different lane. The first device 102 may reduce speed, increase speed, change lanes, or a combination thereof, in order to safely proceed through the travel area 301. By reporting the traffic status update 121 to the Figure 1 The second device 106 can send a preparation warning to other users who enter or plan to enter the travel area 301 of the first device 102. Those skilled in the art will appreciate that early warnings of congested traffic conditions 316 or traffic accidents 318 can improve the safety of the first device 102 and other vehicles 320 traveling through the travel area 301.

[0090] Reference now Figure 4 , which shows Figure 1 The example block diagram of the multi-information fusion mechanism 401 of the navigation system 100 depicts a multi-information fusion model 402 that can fuse the corrected speed 404, the traffic state estimation 406, and the distance 408 to the intersection. The multi-information fusion model 402 can be software running on dedicated hardware or a dedicated hardware device, which can combine the corrected speed 404, the traffic state estimation 406, and the distance 408 to the intersection in a three-second interval to provide the traffic flow state 109.

[0091] The corrected speed 404 is provided by the filter correction unit 410, which monitors Figure 1 The speed sensor 412 of the first device 102 and the Figure 2 The filter correction unit 410 may be software running on dedicated hardware or dedicated hardware circuits. The filter correction unit 410 may detect when the first device 102 is in Figure 3The output of the speed sensor 412 of the first device 102 decreases due to the turn, which can be interpreted as Figure 3 The filter correction unit 410 monitors the gyro angular rate sensor 414. If the gyro angular rate sensor 414 exceeds five degrees per second, the filter correction unit 410 identifies Figure 3 The turning state 319 warns the multi-information fusion model 402 to ignore the traffic state estimate 406 for the duration of the turning state 319.

[0092] The traffic state estimate 406 may be provided by a traffic classifier unit 416, which may be software running on specific hardware or dedicated hardware circuitry. The traffic classifier unit 416 may receive an analysis of a deep learning model 418 based on a series of frames submitted by the video clip unit 113. The traffic classifier unit 416 may record the findings of the deep learning model 418 based on sampling a plurality of frames captured in the video clip 420 to provide the traffic state estimate 406.

[0093] The deep learning model 418 may be software running on specific hardware or dedicated hardware circuits. The deep learning model 418 is a neural network of weighted interconnections and decision nodes that can analyze sample frames of a video clip 420 received from a video clip unit 113 that buffers data from the optical sensor 110. During the training process of the deep learning model 418, the weighted interconnections between the decision nodes may be adjusted to allow identification of estimated traffic found in the video clip 420. The optical sensor 110 may provide the sensor data stream 111 to the video clip unit 113 for segmentation and buffering, which produces a video clip 420 of a fixed duration (e.g., three seconds). The deep learning model 418 may be trained or updated to more efficiently identify traffic states from the video clip 420.

[0094] The multi-information fusion model 402 may receive the distance 408 of the first device 102 to the intersection from the position-mobility sensor 218. The position coordinate unit 424 may compare the background map database 124 with the position from the position-mobility sensor 218 at short intervals (such as one second intervals) as the distance 408 of the first device 102 to the intersection. The multi-information fusion model 402 may perform analysis based on the distance 408 to the intersection and the traffic state estimation 406 to identify the vehicle that is close to the intersection. Figure 3 The intersection of 311 Figure 3 The slow traffic condition 321.

[0095] For example, when the first device 102 approaches the intersection 311, the traffic flow state 109 can be easily mistaken for the congested traffic state 316. In order to evaluate the impact of the intersection 311, the background map database 124 and the location coordinates 424 can be utilized to define the traffic flow state 109. The distance to the intersection 408 can provide the distance to the intersection 311 in the direction that the first device 102 is traveling. Assuming that the traffic flow state 109 is identified as the congested traffic state 316, but the distance from the first device 102 to the intersection 311 meets a certain threshold (such as 100 meters), the traffic flow state 109 can be reset to Figure 3 The slow traffic condition 321.

[0096] The traffic flow analysis unit 426 may monitor the traffic flow state 109 for a fixed period of time, such as one minute, to generate the traffic state update 121. The traffic state update 121 may be a compilation of fixed intervals of the traffic flow state 109 presented. The traffic flow analysis unit 426 may determine whether a particular state or condition is being sent to the Figure 1 The traffic flow analysis unit 426 may perform trend analysis by weighting the second half of the fixed time interval more than the first half, or it may weight the congested traffic state 321, Figure 3 Slow traffic conditions 316 and Figure 3 The number of occurrences of the smooth traffic state 317 is counted. For example, a process (such as packaging) of the traffic flow state 109 within a fixed time interval may report the congested traffic state 316, the smooth traffic state 317, or the slow traffic state 321 with the highest count. The traffic flow state 109 may include the congested traffic state 316, the smooth traffic state 317, or the slow traffic state 321 as a majority count or as a weighted average.

[0097] It has been found that the multi-information fusion mechanism 401 of the navigation system 100 can provide the first device 102 Figure 3 The short duration of the fixed time interval (such as one minute) allows the traffic status update 121 to provide real-time analysis of the traffic conditions experienced by the first device 102 and other vehicles 320. The traffic status update 121 can identify temporary or medium-term congestion (e.g., Figure 3By sending the traffic status update 121 to the second device 106, the background map database 124 can be updated and verified. The second device 106 can forward the traffic status update 121 to other vehicles 320 in communication with the navigation system 100. This communication can provide a safer and more efficient travel path through the travel area 301 acquired by the first device 102.

[0098] Now refer to Figure 5 , which shows an example block diagram of a navigation system 100 in an embodiment. The navigation system 100 may include a first device 102, a network 104, and a second device 106. The first device 102 may send information to the second device 106 via the network 104 in a first device transmission 508. The second device 106 may send information to the first device 102 or the second device 106 via the network 104 in a second device transmission 510. Figure 3 320 other vehicles.

[0099] For illustrative purposes, the navigation system 100 is shown with the first device 102 as a client device, although it is understood that the navigation system 100 can include the first device 102 as a different type of device. For example, the first device 102 can be a server including the first display interface 530 coupled to the user interface 212.

[0100] In addition, for illustrative purposes, the navigation system 100 is shown with the second device 106 as a server, although it should be understood that the navigation system 100 may include the second device 106 as a different type of device. For example, the second device 106 may be a client device. For example, the navigation system 100 may be fully implemented on the first device 102. The second device 106 may provide Figure 1 Training and enhancement of a multi-layer neural network 116.

[0101] In addition, for the purpose of illustration, the navigation system 100 is shown with interaction between the first device 102 and the second device 106. However, it should be understood that the first device 102 can be part or all of an autonomous vehicle, an intelligent vehicle, or a combination thereof. Similarly, the second device 106 can similarly interact with the first device 102 representing an autonomous vehicle, an intelligent vehicle, or a combination thereof.

[0102] For simplicity of description in this embodiment of the invention, the first device 102 will be described as a client device and the second device 106 will be described as a server device. Embodiments of the invention are not limited to this choice of type of device. This choice is an example of an embodiment of the invention.

[0103] The first device 102 may include a first control circuit 512, a first storage circuit 514, a first communication circuit 516, a first interface circuit 518, and a first positioning circuit 520. The first control circuit 512 may include a first control interface 522. The first control circuit 512 may execute a first software 526 to provide intelligence of the navigation system 100.

[0104] The first control circuit 512 can be implemented in many different ways. For example, the first control circuit 512 can be a processor, an application specific integrated circuit (ASIC), an embedded processor, a microprocessor, a hardware control logic, a hardware finite state machine (FSM), a digital signal processor (DSP), or a combination thereof. The first control interface 522 can be used for communication between the first control circuit 512 and other functional units or circuits in the first device 102. The first control interface 522 can also be used for communication outside the first device 102.

[0105] The first control interface 522 can receive information from other functional units / circuitry or from external sources, or can transmit information to other functional units / circuitry or to external destinations. The external sources and the external destinations refer to sources and destinations external to the first device 102 .

[0106] The first control interface 522 can be implemented in different ways and can include different implementations, depending on which functional units / circuits or external units / circuits are interacting with the first control interface 522. For example, the first control interface 522 can be implemented with a pressure sensor, an inertial sensor, a micro-electromechanical system (MEMS), an optical circuit, a waveguide, a wireless circuit, a wired circuit, or a combination thereof.

[0107] The first storage circuit 514 may store the first software 526. The first storage circuit 514 may also store related information (eg, data representing input images, data representing previously presented images, sound files, or a combination thereof).

[0108] The first storage circuit 514 may be a volatile memory, a non-volatile memory, an internal memory, an external memory, or a combination thereof. For example, the first storage circuit 514 may be a non-volatile storage device (e.g., a non-volatile random access memory (NVRAM), a flash memory, a disk storage device), or a volatile storage device (e.g., a static random access memory (SRAM)).

[0109] The first storage circuit 514 may include a first storage interface 524. The first storage interface 524 may be used to connect the first storage circuit 514 to other functional units or circuits in the first device 102 (eg, Figure 1 The first storage interface 524 can also be used for communication outside the first device 102.

[0110] The first storage interface 524 can receive information from other functional units / circuits or from external sources, or can transmit information to other functional units / circuits or external destinations. The external sources and the external destinations refer to sources and destinations that are external to the first device 102. The first storage interface 524 can receive input from the traffic estimation module 115 and receive source data.

[0111] The first storage interface 524 can include different implementations depending on which functional units / circuits or external units / circuits are being interfaced with the first storage circuit 514. The first storage interface 524 can be implemented with techniques and technologies similar to the implementation of the first control interface 522.

[0112] The first communication circuit 516 can enable external communications to and from the first device 102. For example, the first communication circuit 516 can allow the first device 102 to communicate with the second device 106 and the network 104.

[0113] The first communication circuit 516 may also serve as a communication hub that allows the first device 102 to function as part of the network 104 and is not limited to being an endpoint or terminal circuit for the network 104. The first communication circuit 516 may include active and passive components (such as microelectronics or antennas) for interacting with the network 104.

[0114] The first communication circuit 516 may include a first communication interface 528. The first communication interface 528 may be used for communication between the first communication circuit 516 and other functional units or circuits in the first device 102. The first communication interface 528 may receive information from the second device 106 for distribution to other functional units / circuits, or may transmit information to other functional units or circuits.

[0115] The first communication interface 528 can include different implementations depending on which functional units or circuits are being interfaced with the first communication circuit 516. The first communication interface 528 can be implemented with techniques and technologies similar to the implementation of the first control interface 522.

[0116] The first interface circuit 518 allows Figure 1The user 112 interfaces and interacts with the first device 102. The first interface circuit 518 may include input devices and output devices. Examples of input devices of the first interface circuit 518 may include a keypad, a touchpad, soft keys, a keyboard, a microphone, an infrared sensor for receiving remote signals, an optical sensor 110, or any combination thereof to provide data and communication input. As an example, the optical sensor 110 may be connected to the first interface circuit 518 via a wired or wireless connection. The first interface circuit 518 may pass input from the optical sensor 110 to the first control circuit 512 for processing and storage. During the training of the traffic estimation module 115, the first communication interface 528 may pass input from the optical sensor 110, the location of the optical sensor 110, and external parameters of the optical sensor 110 to the second device 106 to enhance Figure 1 Multilayer neural network 116, Figure 1 object identifier 118, and Figure 1 The accuracy and reliability of the frame sampler module 120.

[0117] The first interface circuit 518 may include a first display interface 530. The first display interface 530 may include an output device. The first display interface 530 may be coupled to the user interface 212, including a projector, a video screen, a touch screen, a speaker, a microphone, a keyboard, and combinations thereof.

[0118] The first control circuit 512 can operate the first interface circuit 518 to display information generated by the navigation system 100 and receive information from Figure 1 The first control circuit 512 may also execute the first software 526 for other functions of the navigation system 100, including receiving positioning information from the first positioning circuit 520. The first control circuit 512 may also execute the first software 526 for interacting with the network 104 via the first communication circuit 516. The first control unit 512 may operate Figure 4 Multi-information fusion model 402.

[0119] The first control circuit 512 can operate the first interface circuit 518 to collect data from the optical sensor 110. The first control circuit 512 can also receive positioning information from the first positioning circuit 520. The first control circuit 512 can operate the traffic estimation module 115 to deliver the traffic flow state 109 for display on the user interface 212, and generate control guidance instructions and maneuvering instructions 554 to autonomously drive or assist in driving the first device 102. The maneuvering instructions 554 may include increasing speed, reducing speed, lane change suggestions, lane boundary warnings, and traffic avoidance alerts. The maneuvering instructions 554 can be generated based on the location of other vehicles 320, the content of the traffic flow state 109, and the occupancy of the adjacent lanes 313.

[0120] For example, the first positioning circuit 520 may generate a current heading, current acceleration, and current speed of the first device 102 by comparing the background map database 124 with the current heading, current acceleration, and current speed of the first device 102. Figure 4 The first positioning circuit 520 may be implemented in many ways. For example, the first positioning circuit 520 may be used as at least a part of a global positioning system, an inertial navigation system, a cellular tower positioning system, a gyroscope, or any combination thereof. In addition, for example, the first positioning circuit 520 may utilize components such as an accelerometer, a gyroscope, or a global positioning system (GPS) receiver.

[0121] The first positioning circuit 520 may include a first positioning interface 532. The first positioning interface 532 may be used for communication between the first positioning circuit 520 and other functional units or circuits in the first device 102 (including the optical sensor 110).

[0122] The first location interface 532 can receive information from other functional units / circuitry or from external sources, or can transmit information to other functional units / circuitry or to external destinations. The external sources and the external destinations refer to sources and destinations external to the first device 102 .

[0123] The first positioning interface 532 can include different implementations depending on which functional units / circuits or external units / circuits are being interfaced with the first positioning circuit 520. The first positioning interface 532 can be implemented with techniques and technologies similar to the implementation of the first control circuit 512.

[0124] The second device 106 may be optimized for implementing embodiments of the present invention in a multi-device embodiment with the first device 102. The second device 106 may provide additional or higher performance processing capabilities than the first device 102. The second device 106 may include a second control circuit 534, a second communication circuit 536, a second user interface 538, and a second storage circuit 546.

[0125] The second user interface 538 allows an operator (not shown) to interface and interact with the second device 106. The second user interface 538 may include an input device and an output device. Examples of input devices of the second user interface 538 may include a keypad, a touch pad, soft keys, a keyboard, a microphone, or any combination thereof, to provide data and communication input. Examples of output devices of the second user interface 538 may include a second display interface 540. The second display interface 540 may include a display, a projector, a video screen, a speaker, or any combination thereof.

[0126] During the training process, the second control circuit 534 may receive traffic status updates 121 via the second communication circuit 536. The second control circuit 534 may receive traffic status updates 121 via the second communication circuit 536. Figure 4 The second control circuit 534 may verify that the content of the traffic status update 121 indeed represents the traffic flow identified by the location of the first device 102 based on the traffic status estimate 406 of the optical sensor 110, the distance 408 to the intersection of the optical sensor 110, and the external parameters of the optical sensor 110. Once the traffic status update 121 has been verified, the second control circuit 534 may pass the traffic status update 121 to the map activity manager 122 via the second storage circuit 546 for processing and further distribution.

[0127] The second control circuit 534 can execute the second software 542 to provide the intelligence of the second device 106 of the navigation system 100. The second software 542 can operate in conjunction with the first software 526. Compared to the first control circuit 512, the second control circuit 534 can provide additional capabilities.

[0128] The second control circuit 534 can operate the second user interface 538 to display information. The second control circuit 534 can also execute the second software 542 for other functions of the navigation system 100, including operating the second communication circuit 536 to communicate with the first device 102 through the network 104.

[0129] The second control circuit 534 can be implemented in many different ways. For example, the second control circuit 534 can be a processor, an embedded processor, a microprocessor, hardware control logic, a hardware finite state machine (FSM), a digital signal processor (DSP), or a combination thereof.

[0130] The second control circuit 534 may include a second control interface 544. The second control interface 544 may be used for communication between the second control circuit 534 and other functional units or circuits in the second device 106. The second control interface 544 may also be used for communication outside the second device 106.

[0131] The second control interface 544 can receive information from the other functional units / circuits or from external sources, or can transmit information to the other functional units / circuits or to external destinations. The external sources and the external destinations refer to sources and destinations external to the second device 106 .

[0132] The second control interface 544 can be implemented in different ways and can include different implementations, depending on which functional units / circuits or external units / circuits are being interfaced with the second control interface 544. For example, the second control interface 544 can be implemented with a pressure sensor, an inertial sensor, a micro-electromechanical system (MEMS), an optical circuit, a waveguide, a wireless circuit, a wired circuit, or a combination thereof.

[0133] The second storage circuit 546 can store the second software 542. The second storage circuit 546 can also store information (e.g., data representing an input image, data representing a previously presented image, a sound file, or a combination thereof). The second storage circuit 546 can be sized to provide additional storage capacity to supplement the first storage circuit 514. During the training process, the second storage circuit 546 can receive traffic status updates 121 for the travel area 301 of the first device 102. The second storage circuit 546 can pass the traffic status updates 121 to incorporate the updates into the background map database 124, real-time traffic model 126, and hazard warning model 128 sent in real time by the first device 102. The traffic status updates 121 can then be passed through the second communication circuit 536 for transmission to Figure 3 other vehicles 320 to assist in lane selection, route planning, or alternative route planning.

[0134] For purposes of illustration, the second storage circuit 546 is shown as a single element, but it is understood that the second storage circuit 546 can be a distribution of storage elements. In addition, for purposes of illustration, the navigation system 100 is shown with the second storage circuit 546 as a single hierarchical storage system, although it is understood that the navigation system 100 can include the second storage circuit 546 in different configurations. For example, the second storage circuit 546 can be formed with different storage technologies forming a memory hierarchy system that includes different levels of cache, main memory, rotating media, or offline storage.

[0135] The second storage circuit 546 may be a controller of a volatile memory, a nonvolatile memory, an internal memory, an external memory, or a combination thereof. For example, the second storage circuit 546 may be a controller of a nonvolatile storage device such as a nonvolatile random access memory (NVRAM), a flash memory, a disk storage device, or a volatile storage device such as a static random access memory (SRAM).

[0136] The second storage interface 548 can receive information from the other functional units / circuits or from external sources, or can transmit information to the other functional units / circuits or to external destinations. The external sources and the external destinations refer to sources and destinations external to the second device 106 .

[0137] The second storage interface 548 may include different implementations depending on which functional units / circuits or external units / circuits are being interfaced with the second storage circuit 546. The second storage interface 548 may be implemented with similar techniques and technologies as the implementation of the second control interface 544. The second storage circuit 546 may receive real-world updates of the background map database 124 from receiving traffic status updates 121 from the traffic estimation module 115 in the first device 102.

[0138] The second communication circuit 536 can enable external communications to and from the second device 106. For example, the second communication circuit 536 can allow the second device 106 to communicate with the first device 102 via the network 104. For example, the second device 106 can provide the traffic status update 121 to other vehicles 320 in or near the travel area 301 of the first device 102.

[0139] The second communication circuit 536 may also serve as a communication hub that allows the second device 106 to function as part of the network 104 and is not limited to being an endpoint or terminal unit or circuit of the network 104. The second communication circuit 536 may include active and passive components (e.g., microelectronics or antennas) for interacting with the network 104.

[0140] The second communication circuit 536 may include a second communication interface 550. The second communication interface 550 may be used for communication between the second communication circuit 536 and other functional units or circuits in the second device 106. The second communication interface 550 may receive information from other functional units / circuits, or may transmit information to other functional units or circuits.

[0141] The second communication interface 550 can include different implementations depending on which functional units or circuits are being interfaced with the second communication circuit 536. The second communication interface 550 can be implemented with techniques and technologies similar to the implementation of the second control interface 544.

[0142] During the training process, the first communication circuit 516 can be coupled with the network 104 to send the traffic status update 121 in the first device transmission 508 to the second device 106. The second device 106 can receive information from the first device transmission 508 of the network 104 in the second communication circuit 536.

[0143] The second communication circuit 536 can be coupled to the network 104 to send information to the first device 102, including updates to the traffic estimation module 115 in the second device transmission 510. The first device 102 can receive information from the second device transmission 510 of the network 104 in the first communication circuit 516. The navigation system 100 can be executed by the first control circuit 512, the second control circuit 534, or a combination thereof. For illustrative purposes, the second device 106 is shown as having a partition including a second user interface 538, a second storage circuit 546, a second control circuit 534, and a second communication circuit 536, but it should be understood that the second device 106 may include different partitions. For example, the second software 542 can be partitioned differently so that some or all of its functions can be in the second control circuit 534 and the second communication circuit 536. In addition, the second device 106 may include other functional units or circuits, which are not shown for clarity. Figure 5middle.

[0144] The functional units or circuits in the first device 102 can work individually and independently of other functional units or circuits. The first device 102 can work individually and independently of the second device 106 and the network 104.

[0145] The functional units or circuits in the second device 106 can work individually and independently of other functional units or circuits. The second device 106 can work individually and independently of the first device 102 and the network 104.

[0146] The above functional units or circuits may be implemented in hardware. For example, one or more of the functional units or circuits may be implemented using a gate array, an application specific integrated circuit (ASIC), a circuit, a processor, a computer, an integrated circuit, an integrated circuit core, a pressure sensor, an inertial sensor, a micro-electromechanical system (MEMS), a passive device, a physical non-transitory memory medium containing instructions for executing software functions, a portion thereof, or a combination thereof.

[0147] For illustrative purposes, the navigation system 100 is described by the operation of the first device 102 and the second device 106. It should be understood that the first device 102 and the second device 106 can operate any modules and functions of the navigation system 100.

[0148] By way of further example, the first device 102 may be an autonomous vehicle or a driver assistance vehicle. The first interface circuit 518 may be operable from Figure 1 The optical sensor 110 receives input for compiling traffic status updates 121. The traffic status updates 121 can be generated by the first control circuit 512 based on the optical sensor 110 and the first positioning circuit 520. The traffic status updates 121 can be sent to the second device 106 via the first communication circuit 516 and the network 104 for processing by the second control circuit 534 and the map activity manager 122.

[0149] It has been discovered that the second device 106 can receive traffic status updates 121 from the first device 102, which provide updates to the map activity manager 122. As an example, the second control circuit 534 can verify the traffic status updates 121 and pass the information to the map activity manager 122 for analysis. When the map activity manager 122 has analyzed the data and updated the background map database 124, the real-time traffic model 126, and the hazard warning model 128, the second control unit 534 can generate safety warnings and lane suggestions for other vehicles 320 traveling in the travel area 301 of the first device 102. Real-time distribution of traffic warnings and lane suggestions can be achieved by minimizing backups and drawing around Figure 3 traffic accidents 318 alternative routes to improve safety, reduce delays and save energy.

[0150] Reference now Figure 6 , which shows an embodiment of the present invention Figure 1 The example operation flow chart 601 of the navigation system 100. The example operation flow chart 601 of the navigation system 100 depicts a start block 602 indicating Figure 1 The first device 102 has accessed Figure 3 The process proceeds to the recording image frame 604, where the first device 102 Figure 1 The optical sensor 110 records the travel area 301 . Figure 5 The first control circuit 512 can be accessed by Figure 2 The first positioning circuit 520 is incorporated into Figure 1 The first control circuit 512 may identify the current global position from the sensor data stream 111 of the first device 102. It should be appreciated that the current global position may indicate the actual position of the optical sensor 110 of the first device 102. The first control circuit 512 may generate external parameters of the optical sensor 110 to facilitate analysis of the sensor data stream 111.

[0151] The process proceeds to the Provide Analysis Image block 606, where the sensor data stream 111 is segmented into Figure 4 The video clip 420 and the first device 102 Figure 4 It should be appreciated that the second device 106 may have more data processing capabilities and speed than the first device 102, so for training purposes, the processing of the sensor data stream 111 may be performed by the second device 106, and the analysis may be performed by the first device 102 after the deep learning model 418 has been trained.

[0152] The process proceeds to identify traffic flow estimation block 608. Here, the deep learning model 418 can analyze the traffic flow from Figure 4 The deep learning model 418 may process the images obtained from the video clip 420 to generate a traffic flow estimate 406. Figure 2 The multi-information fusion model 402 provides traffic flow estimation.

[0153] The process then proceeds to generate traffic flow state block 610. The multi-information fusion model 402 can be used Figure 4 The distance to the intersection is 408 and Figure 4 The corrected speed 404 is used to define the traffic flow estimate 406. If the corrected speed 404 detects Figure 3If a turn state 319 is detected, the traffic flow estimate 406 is ignored for the duration of the turn state 319. If a turn state 319 is not detected, but the distance to the intersection 408 indicates Figure 3 If the intersection 311 is very close, the traffic flow estimation 406 can be modified from the congested traffic state 316 to the slow traffic state 321.

[0154] The flow then proceeds to the generate traffic state update block 612. The traffic flow analysis unit 426 can monitor the occurrence of the congested traffic state 316, the smooth traffic state 317, or the slow traffic state 321 to determine the significant state. The traffic flow analysis unit 426 can count the occurrence of the congested traffic state 316, the smooth traffic state 317, or the slow traffic state 321 to determine which state provides a greater number of events during a fixed period (e.g., one minute). The congested traffic state 316, the smooth traffic state 317, or the slow traffic state 321 representing the largest number will be reflected as the traffic state update 121 throughout the cycle.

[0155] The process may then proceed to the distribute traffic status update block 614. The first control circuit 512 may assemble the traffic flow status 109 and generate Figure 5 A set of manipulation instructions 554 for the first device 102 to proceed through Figure 3 The first control circuit 512 may combine the traffic flow state 109 and the set of maneuvering instructions 554 for display on the user interface 212. The first control circuit 512 may send the traffic flow state 109 and the maneuvering instructions 554 to the user interface 212. Figure 5 The first display interface 530 is used to present on the user interface 212. The first control circuit 512 can also be Figure 1 The network 104 transmits the traffic flow state 109 to Figure 1 The second device 106 is used as a traffic status update 212 for distribution to Figure 3 320 other vehicles.

[0156] The process proceeds to end 616, where the first device 102 may generate a traffic status update 121 and present it on Figure 2 212 of the user interface 212. It should be appreciated that the user interface 212 may display the sensor data stream 111 of the travel area 301, and the traffic status update 121 may be presented as a highlighted overlay, or the traffic status update 121 may be presented separately on the user interface 212. The first device 102 may extract the maneuvering instructions 554 from the traffic flow state 109 to enable the driver assistance or autonomous driving features of the first device 102 to safely proceed through the travel area 301.

[0157] Now refer to Figure 7 , which shows an embodiment of the present invention Figure 1 Flowchart of method 700 of operation of navigation system 100 of the present invention. Method 700 includes: in box 702, generating a video clip by parsing a sensor data stream of a period of time of a travel area; in box 704, analyzing the video clip submitted to the trained deep learning model, including identifying a traffic flow estimate; in box 706, accessing the location coordinates used to calculate the distance to the intersection; in box 708, generating a traffic flow state by fusing the corrected speed, the traffic flow estimate, and the distance to the intersection; in box 710, merging the vehicle maneuvering instructions into the traffic flow state for maneuvering through the travel area; and in box 712, transmitting the traffic flow state for display on the device.

[0158] The resulting method, process, apparatus, device, product and / or system is straightforward, cost-effective, uncomplicated, highly versatile, accurate, sensitive and effective, and may be implemented by adapting known components for readily available, efficient and economical manufacture, application and utilization. Another important aspect of the embodiments of the present invention is that it valuable supports and serves the historical trend of reducing costs, simplifying systems and improving performance.

[0159] Accordingly, these and other valuable aspects of embodiments of the present invention advance the state of the technology to at least the next level.

[0160] Although the present invention has been described in conjunction with a specific best mode, it should be understood that, based on the foregoing description, many substitutions, modifications and variations will be clear to those skilled in the art. Therefore, the present invention is intended to include all such substitutions, modifications and variations that fall within the scope of the included claims. All contents set forth herein or shown in the accompanying drawings should be interpreted as illustrative rather than restrictive.

Claims

1. A navigation system, comprising: The control circuit is configured as follows: Generate video clips by parsing a stream of sensor data over a period of time in the travel area; analyzing said video clip submitted to an already trained deep learning model, Includes identification of traffic flow estimation; Access the location coordinates for calculating the distance to the intersection; generating a traffic flow state by fusing the corrected speed, the traffic flow estimate, and the distance to the intersection; incorporating vehicle maneuver instructions into the traffic flow state for maneuvering through the travel area, wherein the traffic flow state is ignored during a detected turn condition; as well as The communication circuit, coupled to the control circuit, is configured to: The traffic flow status is transmitted for display on a device.

2. The system according to claim 1, wherein: The control circuit is configured to: generating the corrected velocity by submitting the velocity and the gyroscope angular rate to a filtering / correction unit; and The turning state is detected when the speed is decreasing and the gyro angular rate exceeds 5 degrees per second.

3. The system according to claim 1, wherein: The control circuit is configured to load the traffic flow estimation in a multi-information fusion model to detect a clear traffic state when a distance to a vehicle in the same lane is 100 meters or more.

4. The system according to claim 1, wherein: The control circuit is configured to detect a slow traffic condition when the multi-information fusion model identifies other vehicles traveling at a speed less than half of a posted speed limit provided by the background map database.

5. The system according to claim 1, wherein: The control circuit is configured to detect a congested traffic state when the multi-information fusion model recognizes that a vehicle traveling in an adjacent lane is about to approach or completely stop.

6. The system according to claim 1, wherein: The control circuit is configured to generate a traffic state update by compiling the traffic flow state over a fixed time period, including counting the occurrence of a congested traffic state, a slow traffic state, and a smooth traffic state.

7. The system according to claim 1, wherein: The control circuitry is configured to generate the vehicle maneuver command based on the position of the vehicle and the occupancy of adjacent lanes.

8. The system according to claim 1, wherein: The communication circuit is configured to send traffic status updates through a network to update a background map database, a real-time traffic model, and a hazard warning model.

9. A method for operating a navigation system, comprising: Generate video clips by parsing a stream of sensor data over a period of time in the travel area Edit; Analyze video clips submitted to trained deep learning models, including identifying traffic flow estimates; Access the location coordinates for calculating the distance to the intersection; generating a traffic flow state by fusing the corrected speed, the traffic flow estimate, and the distance to the intersection; incorporating vehicle maneuver instructions into the traffic flow state for maneuvering through the travel area, wherein the traffic flow state is ignored during a detected turn condition; as well as The traffic flow status is transmitted for display on a device.

10. The method according to claim 9, further comprising: generating a corrected velocity by submitting the velocity and the gyroscope angular rate to a filtering / correction unit; as well as The turning state is detected when the speed is decreasing and the gyro angular rate exceeds 5 degrees per second. 11 . The method of claim 9 , further comprising loading the traffic flow estimation in a multi-information fusion model to detect a clear traffic state when a distance to a vehicle in the same lane is 100 meters or more. 12 . The method of claim 9 , further comprising detecting a slow traffic state when the multi-information fusion model identifies other vehicles traveling at a speed less than half of a posted speed limit provided by a background map database.

13. The method according to claim 9, further comprising detecting a congested traffic state when the multi-information fusion model recognizes that a vehicle traveling in an adjacent lane is about to approach or completely stop.

14. The method of claim 9, further comprising generating a traffic state update by compiling the traffic flow state over a fixed time period, including counting occurrences of a jammed traffic state, a slow traffic state, and a free traffic state.

Images