System and method for providing real-time route view

Abstract

Provides a real-time route view along the suggested navigation route. [Solution] The system includes a user device computer and a server computer that communicate electronically. The user device computer is programmed to identify candidate navigation routes, send requests to the server computer for real-time images taken along the candidate navigation routes, receive one or more real-time images from the server computer, and display the received one or more real-time images. In response to receiving a request from the user device computer for real-time images taken along the candidate navigation routes, the server computer is programmed to collect one or more real-time images along the candidate navigation routes and send the captured one or more real-time images to the user device computer.

Description

Technical Field

[0003] ,

[0001] This specification generally relates to apparatuses and methods for providing real-time route views, and more particularly, to apparatuses and methods for providing real-time route views of candidate navigation routes.

Background Art

[0002] Many vehicles and smartphones include navigation systems such as a Global Positioning System (GPS) that store or access digital maps. These systems can generate candidate navigation routes from a location to a destination, and can display the candidate navigation routes to a user of the system, for example, as a highlighted route along a displayed map. The user can select one of the candidate navigation routes, for example, for guidance to the destination.

Summary of the Invention

[0003] In one embodiment, a system for providing a real-time route view includes a user device computer and a server computer that communicates electronically with the user device computer. The user device computer is programmed to identify candidate navigation routes. The user device computer is programmed to send requests to the server computer for real-time images taken along the candidate navigation routes. The user device computer is programmed to receive one or more real-time images from the server computer. The user device computer is programmed to display the received one or more real-time images. The server computer is programmed to collect one or more real-time images along the candidate navigation routes in response to receiving requests for real-time images taken along the candidate navigation routes. The server computer is programmed to send the captured one or more real-time images to the user device computer.

[0004] In another embodiment, a user device for providing a real-time route view includes a computer having a processor and memory for storing machine-readable instructions executable by the processor. The machine-readable instructions include instructions for identifying candidate navigation routes. The machine-readable instructions include instructions for sending requests for real-time images taken along the candidate navigation routes. The machine-readable instructions include instructions for receiving one or more real-time images taken along the candidate navigation routes. The machine-readable instructions include instructions for displaying one or more received real-time images.

[0005] In yet another embodiment, a method for providing a real-time route view includes identifying a candidate navigation route; sending a request for real-time images taken along the candidate navigation route; receiving one or more real-time images taken along the candidate navigation route; and displaying one or more received real-time images.

[0006] These and additional features provided by the embodiments described herein will be better understood by referring to the following detailed description in conjunction with the drawings. [Brief explanation of the drawing]

[0007] [Figure 1] Figure 1 schematically depicts a block diagram of a system providing a real-time route view according to one or more embodiments shown and described herein. [Figure 2] Figure 2 schematically illustrates a flowchart of the process for providing a real-time route view using the system of Figure 1, according to one or more embodiments shown and described herein. [Figure 3] Figure 3 schematically illustrates a user device for the system of Figure 1, which displays real-time images taken along a proposed navigation route, according to one or more embodiments shown and described herein. [Modes for carrying out the invention]

[0008] The embodiments shown in the drawings are essentially illustrative and illustrative, and are not intended to limit the subject matter defined by the claims. Reading them in conjunction with the following drawings will allow for a better understanding of the detailed description of the following exemplary embodiments, where similar structures are denoted by the same reference numerals.

[0009] Embodiments described herein relate to systems and methods for providing a real-time route view along a candidate navigation route. The system generally includes a user device computer and a server computer, which are programmed to identify candidate navigation routes and to request, receive, and display one or more real-time images captured along the candidate navigation routes. Various embodiments of the system and methods are described in more detail herein. Wherever possible, the same reference numerals are used throughout the drawings to indicate the same or similar parts.

[0010] Unless expressly indicated otherwise, no method described herein is intended to be construed as requiring the steps to be performed in a particular order, nor as requiring any particular orientation for any device. Therefore, no method claim actually specifies the order in which the steps should be followed, nor does any device claim actually specify the order or orientation of individual components, and where the claim or specification does not expressly state that the steps are limited to a particular order, or where no particular order or orientation for the components of a device is described, no order or orientation is intended to be presumed in any way. This applies to any possible implicit grounds for interpretation, including logical matters relating to the arrangement of steps, the flow of operations, the order of components, or the orientation of components, plain meaning derived from grammatical structure or punctuation, and the number or type of embodiments described in the specification.

[0011] As used herein, the singular forms "a," "an," and "the" are interpreted as including the plural unless the context clearly indicates otherwise. For example, a reference to a component includes embodiments having two or more such components unless the context clearly indicates otherwise.

[0012] Referring here to Figure 1, a system 20 for providing a real-time route view according to one or more embodiments described herein is schematically shown. The system 20 generally includes a user device computer 22 and a server computer 24 that communicates electronically with the user device computer 22. The user device computer 22 is programmed to identify candidate navigation routes 26 ( schematically shown in Figure 3). The user device computer 22 is programmed to send a request to the server computer 24 for real-time images 28 (also schematically shown in Figure 3) to be taken along the candidate navigation route 26. In response to receiving a request for real-time images 28 from the user device computer 22, the server computer 24 is programmed to take one or more real-time images 28 along the candidate navigation route 26. The server computer 24 is programmed to send one or more of the taken real-time images 28 to the user device computer 22. The user device computer 22 is programmed to receive one or more real-time images 28 from the server computer 24. The user device computer 22 is programmed to display the received one or more real-time images 28.

[0013] A candidate navigation route 26 is one possible route along streets, roads, highways and equivalents along which the vehicle will navigate. A candidate navigation route 26 specifies a particular section of streets, highways, roads and equivalents, for example, leading to a specific destination or traversing a specific geographical area. A candidate navigation route 26 includes information specifying the direction of travel along various sections of the candidate navigation route 26, for example, north, south, east, west, etc.

[0014] There may be multiple candidate navigation routes 26. Such candidate navigation routes 26 have unique characteristics. For example, one candidate navigation route 26 may be the shortest distance to the destination, another may be estimated to take the shortest time to reach the destination, another may pass through specific points of interest (e.g., historical landmarks, geographical viewpoints, etc.), another may avoid specific types of roads (e.g., routes that do not include main roads and highway sections), and another may travel only through areas with population density or traffic density below a threshold, but not limited to these.

[0015] A real-time image 28 is a still image or a series of images, such as a video, which is captured approximately simultaneously with the display and / or request for such a real-time image 28. If such a still image or video is captured within a short threshold time before display, the displayed still image or video is considered a real-time image 28. Examples include, but are not limited to, still images or videos displayed within 5 minutes of being captured. A displayed still image or video is also considered a real-time image 28 if it is captured after the request for such a still image or video, i.e., if it was not stored before the request for such a still image or video.

[0016] By capturing real-time images 28 along the candidate navigation route 26, users who receive such images 28 can see what they will see in real time along the candidate navigation route 26, which will help them make a better decision on whether or not they want to travel along the candidate navigation route 26. The real-time images 28 captured along the candidate navigation may indicate to the user that the candidate navigation route 26 is desirable for the user. For example, the user may decide whether the candidate navigation route 26 has good scenery for them enough to convince them to forgo traveling along other candidate navigation routes 26 that are faster but have inferior scenery.

[0017] <Communication Network> The communication network 32 is included in the system 20 and can facilitate electronic communication between, for example, a user device computer 22, a server computer 24, one or more remote vehicles 30, one or more infrastructure cameras (not shown) located along a candidate navigation route 26, etc. The communication network 32 may include, but is not limited to, any one or more different types of communication networks, such as a cable network, a public network (e.g., the Internet), a private network (e.g., a Frame Relay network), a wireless network, a cellular network, a telephone network (e.g., a public switched telephone network), or any other suitable private or public packet-switched or circuit-switched network. The communication network 32 may have any suitable communication range associated with it, which may include, for example, a global network (e.g., the Internet), a metropolitan area network (MAN), a wide area network (WAN), a local area network (LAN), or a personal area network (PAN). In addition, the communication network 32 may include, but is not limited to, communication links and associated network devices (e.g., link layer switches, routers, etc.) for transmitting network traffic over any suitable type of medium, including coaxial cable, twisted pair wire (e.g., twisted pair copper wire), optical fiber, hybrid fiber coaxial (HFC) medium, microwave medium, radio frequency communication medium, satellite communication medium, or any combination thereof.

[0018] <User Device> System 20 includes a user device 34 that can be operated by the user to perform various functions described herein. Exemplary user devices 34 include, but are not limited to, smartphones and tablets, personal computers, vehicle navigation systems and vehicle infotainment systems, and their equivalents.

[0019] The user device 34 includes a user device computer 22, a navigation unit 36, a communication unit 38, and / or one or more user interfaces 40. The navigation unit 36, the communication unit 38, and / or one or more user interfaces 40 are communicatively coupled to the user device computer 22 via a wired and / or wireless electronic communication structure, including, for example, a communication bus, a controller area network (CAN) bus, a local interconnection network (LIN) bus, or equivalents thereof. As used herein, the term “communicatively coupled” means that the coupled components are able to exchange data signals with each other, for example, electrical signals over a conductive medium, electromagnetic signals over air, optical signals over an optical waveguide, and equivalents thereof.

[0020] The user device computer 22 is any combination of devices or components comprising a processor and non-temporary computer-readable memory. The processor is any device capable of executing a machine-readable instruction set stored in non-temporary computer-readable memory. Therefore, the processor is an electrical control unit, an integrated circuit, a microchip, a computer, or other computing device.

[0021] Non-temporary computer-readable memory includes RAM, ROM, flash memory, hard drives, or any non-temporary memory device capable of storing machine-readable instructions, such that machine-readable instructions can be accessed and executed by the processor. The machine-readable instruction set includes, for example, machine code executed directly by the processor 132, or logic or algorithms written in any programming language of any generation (e.g., 1GL, 2GL, 3GL, 4GL, or 5GL), such as assembly language, object-oriented programming language (OOP), scripting language, microcode, etc., which are compiled or assembled into machine-readable instructions and stored in non-temporary computer-readable memory. Alternatively, the machine-readable instruction set may be written in a hardware description language (HDL), such as logic implemented via either a field-programmable gate array (FPGA) configuration or an application-specific integrated circuit (ASIC), or their equivalents. Thus, the functions described herein can be implemented in any conventional computer programming language as pre-programmed hardware elements or as a combination of hardware and software components.

[0022] The navigation unit 36 ​​is capable of generating location information and / or direction of travel information indicating the position of the user device 34 by receiving one or more GPS signals from one or more GPS satellites. The navigation unit 36 ​​includes a GPS device, an electronic compass, or equivalents thereof, which is communicably coupled to the user device computer 22 of the user device 34. The navigation unit 36 ​​is configured to generate direction of travel information, for example, based on an electronic compass. The GPS signals communicated to the user device computer 22 include location information, which includes National Marine Electronics Association (NMEA) messages, latitude and longitude datasets, addresses, names of known places based on a location database, or equivalents thereof. In addition, the navigation unit 36 ​​is interchangeable with other systems capable of generating output indicating a position. Examples include local positioning systems that provide position based on cellular signals and broadcast towers, or radio signal detection devices that can triangulate position by radio signals received from one or more radio signal antennas.

[0023] The navigation unit 36 ​​includes a digital map containing geographical information. The digital map may be dynamic in that the geographical information contained therein can be updated. The digital database is stored locally (i.e., a local digital map stored in the memory of the navigation unit 36) or uploaded from a remote location (i.e., a cloud-based digital map communicating with the navigation unit 36), and as a result, the geographical information is time-dependent and is updated according to the geographical information stored by the cloud-based digital map communicating with the navigation unit 36, the server computer 24, or equivalents thereof.

[0024] The communication unit 38 communicatively couples the user device 34 to the communication network 32. The communication unit 38 includes, for example, an antenna, a modem, a LAN port, a Wi-Fi card, a WiMax card, Bluetooth (registered trademark) hardware, IrDA hardware, wireless USB hardware, Z-Wave hardware, ZigBee (registered trademark) hardware, mobile communication hardware, short-range wireless communication hardware, satellite communication hardware, cellular network communication hardware, and / or any wired or wireless hardware for communicating with other networks and / or devices, but is not limited thereto. The cellular network includes, for example, a Global System for Mobile Communications (GSM) network, a 3G Universal Mobile Telecommunication System (UMTS) network, a 4G Long-Term Evolution (LTE) network, a 5G network, and equivalents thereof, but is not limited thereto.

[0025] The user interface 40 enables the user to interact with the user device 34. The user interface 40 includes one or more input devices that enable the user to input information into the user device computer 22. For example, the input device is a keyboard, a mouse, a joystick, a touch screen, a remote control, a pointing device, a video input device, an audio input device, a tactile feedback device, and / or equivalents thereof, but is not limited thereto. The user interface 40 includes one or more output devices that enable the user to receive information from the user device computer 22. For example, the output device is a display, a speaker, and / or equivalents thereof, but is not limited thereto. The display includes, for example, any medium capable of transmitting an optical output such as a cathode ray tube, a light-emitting diode, a liquid crystal display, a plasma display, or equivalents thereof. Further, in addition to providing optical information, the display may be a touch screen that detects the presence and position of tactile input on the surface of the display or adjacent to the display.

[0026] The user device computer 22 is programmed to identify one or more candidate navigation routes 26, i.e., the memory stores machine-readable instructions executable by the processor. The user device computer 22 identifies a plurality of candidate navigation routes 26, such as, for example, a second candidate navigation route 26, a third candidate navigation route 26, etc. Each of the candidate navigation routes 26 has overlapping characteristics and unique characteristics with respect to each other. For example, one candidate navigation route 26 avoids highways and navigates to the destination along the coast, and another candidate navigation route 26 avoids highways and navigates to the destination through a forest area.

[0027] The user device computer 22 is programmed to identify candidate navigation routes 26 based on user preferences. User preferences include information identifying one or more of a final destination, an intermediate destination, the type of roads to travel on and / or avoid, the desired population density and / or traffic density, one or more desired visual characteristics (e.g., specific plants, landscapes, points of interest, landmarks, etc.), the urgency of arrival at the destination (e.g., as soon as possible, within a certain time, or not urgent), and the like.

[0028] The user provides one or more user preferences to the user device computer 22 via one or more of the user interfaces 40. For example, the user provides tactile input to a touch screen, voice input to a microphone, actuation input using buttons, etc. The input indicates one or more of the user's preferences. For example, the user selects from a plurality of preferences displayed on the touch screen, types one or more user preferences into a keyboard or other text input device displayed on the touch screen, provides oral instructions including one or more user preferences, or interacts with the user interface 40 to specify user preferences. In response to such user input, the user interface 40 transmits information identifying the user preferences input by the user, and the user device computer 22 receives that information.

[0029] The user device computer 22 may work with the server computer 24 to determine one or more user preferences. The user device computer 22 can determine one or more user preferences based on, but is not limited to, calendar data, location data, information collected from emails, smartphone applications and websites and their equivalents, historical data, traffic data, data received from the server computer 24, etc., and this includes candidate navigation routes 26 and prior art for identifying user preferences.

[0030] The user device computer 22 identifies one or more candidate navigation routes 26 in response to a user request that includes one or more user preferences, such as one or more received from one or more of the user interfaces 40. The user device computer 22 can receive user requests and identify one or more candidate navigation routes 26 using large-scale language models (LLMs) and machine learning (MLs). For example, the user device computer 22 can receive user requests from a user input device as natural language voice commands. Such commands could be, for example, "Assistant, how do I get from point A to point B while seeing the most beautiful flowers? I'm not in a hurry right now, but I want to avoid winding roads and see lots of flowers." The user device computer 22, for example, in cooperation with a server computer 24, uses conventional techniques to analyze such commands with LLMs and MLs to identify destination B and specific user preferences, such as country roads that are generally straight and pass through geographical areas where flowers are likely to be seen.

[0031] The user device computer 22 is programmed to send requests for real-time images 28 taken along one or more of the identified candidate navigation routes 26, for example, a request for real-time images 28 along the first candidate navigation route 26, another request for real-time images 28 along the second candidate navigation route 26, and so on. The user device computer 22 sends requests for real-time images 28 to the server computer 24, for example, via the communication network 32. The server computer 24 receives the requests for real-time images 28, for example, as will be described later, and sends requests for real-time images 28 to one or more remote vehicles 30, one or more infrastructure cameras, etc., located along the candidate navigation route 26. The request for real-time images 28 identifies the candidate navigation route 26 and includes the direction of travel along the candidate navigation route 26. The direction of travel along the navigation route is the direction in which the vehicle travels along the section of the candidate navigation route 26 when following the candidate navigation route 26, i.e., when navigating the candidate navigation route 26, for example, north, south, east, west, etc. By identifying the direction of travel along the candidate navigation route 26, it becomes possible to capture real-time images 28 that better reproduce the images that may be encountered when traveling along the candidate navigation route 26. The user device computer 22 sends a request for real-time images 28 after identifying the candidate navigation route 26, for example, as will be described later, and / or the user device computer 22 sends a request for real-time images 28 in response to input from the user to the user interface 40.

[0032] Infrastructure cameras are generally fixed in place and can capture and transmit images, for example, in response to a request. Infrastructure cameras generally include a camera, such as camera 42 described later, and a communication unit, such as communication units 38 and 46 described herein. Infrastructure cameras may include a computer to facilitate the operation of the camera and communication unit. Infrastructure cameras are fixed to, for example, buildings, posts, bridges, etc., but are not limited to these. Exemplary infrastructure cameras include traffic surveillance cameras, security cameras, doorbell cameras, etc.

[0033] The user device computer 22 is programmed to receive one or more real-time images 28 taken along the candidate navigation route 26 by one or more remote vehicles 30, one or more infrastructure cameras, etc., located along the first candidate navigation route 26, the second candidate navigation route 26, etc., and traveling and / or facing the direction of travel of each candidate navigation route 26. For example, the user device 34 receives the real-time images 28 via the communication network 32. The real-time images 28 are received, for example, from a server computer 24 that acts as an intermediary between the user device 34 and one or more remote vehicles 30, one or more infrastructure cameras, etc., located along the candidate navigation route 26. The received real-time images 28 include information that identifies the location and direction of travel and / or direction of photography along the candidate navigation route 26. The received real-time images 28 also include information about one or more objects identified in the real-time images 28. The objects are identified by the user device computer 22 and / or the server computer 24, as described later.

[0034] The user device computer 22 is programmed to identify one or more objects in the real-time image 28. The user device computer 22 can identify one or more objects by utilizing feature descriptor algorithms or image descriptor algorithms such as, for example, Scale-Invariant Feature Transformation ("SIFT"), Fast-Fast Robust Feature Transformation ("SURF"), Gradient Direction Histogram ("HOG"), Generalized Search Tree ("GIST"), Fast Retinal Keypoint ("FREAK"), and Binary Robust Invariant Scalable Keypoint ("BRISK") and equivalents. The user device computer 22 can identify objects as, for example, a specific type of plant, a specific landmark or geographical feature, a specific building or architectural structure, a pedestrian, a vehicle, etc. The user device computer 22 can also identify one or more objects in cooperation with the server computer 24.

[0035] One or more identified objects relate to one or more of the user's preferences. In other words, one or more identified objects are categorically associated with one or more of the user's preferences. For example, if a user preference indicates that the user likes scenic routes, the identified objects could be a specific type of plant, a specific landmark, or a specific geographical feature.

[0036] The user device computer 22 is programmed to display one or more received real-time images 28. To display the real-time images 28, the user device computer 22 transmits information identifying the still images and / or moving images of the real-time images 28 to the display screen of the user interface 40, such as a touchscreen display. Upon receiving such information, the user interface 40 displays the still images and / or moving images of the real-time images 28. The user device computer 22 displays real-time images 28 taken along multiple candidate navigation routes 26, for example, real-time images 28 taken along the first candidate navigation route 26, real-time images 28 taken along the second candidate navigation route 26, and so on.

[0037] The user device computer 22 is programmed to present information about identified objects. The computer presents information about identified objects while simultaneously displaying one or more received real-time images 28. For example, the user device computer 22 presents information about identified objects audibly through the speaker of the user interface 40 while the real-time image 28 containing the object is visually presented on the display of the user interface 40. Alternatively, the computer may visually present information about identified objects through the speaker of the user interface 40, for example, as a text overlay on the real-time image 28 containing the object and presented on the display of the user interface 40. The user device computer 22 collects information about identified objects for one or more computers located remotely from the user device 34, such as a server computer 24. The user device 34 may utilize artificial intelligence such as ChatGPT®, OpenAI, Claude, Google Gemini, Microsoft Copilot, Perplexit, or equivalents thereof to collect information about identified objects. The information about identified objects relates to one or more of the user's preferences. For example, when a user preference indicates that the user likes a floral landscape, the information about the identified object may include, but is not limited to, the color, name, and / or flowering stage of the flowers identified in the real-time image 28.

[0038] The user device computer 22 is programmed to request a selection of a first candidate navigation route 26, a second candidate navigation route 26, etc. The user device computer 22 sends a request to one or more of the user interfaces 40. For example, the user device computer 22 may instruct a speaker to output an audio message requesting user input on whether the first candidate navigation route 26, the second candidate navigation route 26, etc. are acceptable to the user of the user device 34. In another example, the user device computer 22 may instruct an electronic display to visually project the request for user input on whether the first candidate navigation route 26, the second candidate navigation route 26, etc. are acceptable to the user of the user device 34.

[0039] The user device computer 22 is further programmed to receive user input indicating approval of one of the candidate navigation routes 26, after displaying received real-time images 28, for example, taken along the candidate navigation routes 26. The user provides user input by interacting with the user interface 40, for example by pressing a button, touching a specific location on the touchscreen, or providing a voice command.

[0040] The user device computer 22 is further programmed to navigate an approved navigation route, for example, from the current location of the user device 34 determined by the navigation unit 36. The user device computer 22 navigates the approved navigation route, for example, by activating the display of the user interface 40 to provide a map and visual guidance showing how to proceed along the approved navigation route, and / or by activating the speaker of the user interface 40 to provide voice turn-by-turn guidance. As another example, the user device computer 22 may provide instructions indicating the approved navigation route to the autonomous vehicle system, for example, when the user device 34 is integrated into or connected to an autonomous vehicle. The user device computer 22 can navigate the approved navigation route in response to user input.

[0041] <Server Computer> The server computer 24 is any combination of devices or components comprising a processor and non-temporary computer-readable memory. The processor is any device capable of executing a machine-readable instruction set stored in non-temporary computer-readable memory. Thus, the processor is an electrical control unit, an integrated circuit, a microchip, a computer, or other computing device. While the embodiment depicted in Figure 1 includes a single computer, other embodiments may include more than one computer coupled together in a communicative manner.

[0042] Non-temporary computer-readable memory includes RAM, ROM, flash memory, hard drives, or any non-temporary memory device capable of storing machine-readable instructions, such that machine-readable instructions can be accessed and executed by the processor. The machine-readable instruction set includes, for example, machine code executed directly by the processor 132, or logic or algorithms written in any programming language of any generation (e.g., 1GL, 2GL, 3GL, 4GL, or 5GL), such as assembly language, object-oriented programming language (OOP), scripting language, microcode, etc., which are compiled or assembled into machine-readable instructions and stored in non-temporary computer-readable memory. Alternatively, the machine-readable instruction set may be written in a hardware description language (HDL), such as logic implemented via either a field-programmable gate array (FPGA) configuration or an application-specific integrated circuit (ASIC), or their equivalents. Thus, the functions described herein are implemented in any conventional computer programming language, either as pre-programmed hardware elements or as a combination of hardware and software components.

[0043] The server computer 24 includes, but is not limited to, an antenna, modem, LAN port, Wi-Fi card, WiMAX card, mobile communication hardware, near-field communication hardware, satellite communication hardware, and / or any wired or wireless hardware for communicating with other networks and / or devices. The server computer 24 communicates with the user device computer 22 of the user device 34 and one or more remote vehicles 30, one or more infrastructure cameras, etc., for example via the communication network 32.

[0044] The server computer 24 is programmed to identify one or more candidate navigation routes 26, that is, the memory of the server computer 24 stores machine-readable instructions that can be executed by the processor of the server computer 24. The server computer 24 can identify candidate navigation routes 26 based on information received from the user device 34, user requests, calendar data, location data, information collected from emails and smartphone applications and websites and their equivalents, historical data, traffic data, etc., and this includes prior art for identifying candidate navigation routes 26 and user preferences, as discussed herein.

[0045] The server computer 24 is programmed to collect one or more real-time images 28 from one or more remote vehicles 30, one or more infrastructure cameras, etc., located along the first candidate navigation route 26, the second candidate navigation route 26, etc. In response to receiving a request for real-time images 28 from the user device computer 22, the server computer 24 collects one or more real-time images 28 from one or more remote vehicles 30, one or more infrastructure cameras, etc., located along the candidate navigation route 26.

[0046] For example, but not limited to, the server computer 24 sends a request for real-time images 28, i.e., a request for one or more real-time images 28, to one or more remote vehicles 30, one or more infrastructure cameras, etc., located along the candidate navigation route 26 and traveling and / or facing the direction of travel specified in the request for real-time images 28. In order to send the request, the server computer 24 identifies one or more remote vehicles 30, one or more infrastructure cameras, etc., located along the candidate navigation route 26 and traveling and / or facing the specified direction of travel. The server computer 24 sends the request for real-time images 28 to the identified one or more remote vehicles 30, one or more infrastructure cameras, etc., located along the candidate navigation route 26, for example, via the communication network 32.

[0047] The server computer 24 identifies one or more remote vehicles 30, one or more infrastructure cameras, etc., located along the candidate navigation route 26 and traveling and / or facing the specified direction of travel, based on information in a dynamic database stored in the server computer 24, which includes information about the general current position, direction of travel, and / or orientation of multiple remote vehicles 30, one or more infrastructure cameras, etc. The server computer 24 may also identify one or more remote vehicles 30, one or more infrastructure cameras, etc., located along the candidate navigation route 26, including the direction of travel specified in a request for real-time images 28 received from the user device computer 22, by comparing the candidate navigation route 26, including the direction of travel specified in the request, with information about the general current position, direction of travel, and / or orientation of multiple remote vehicles 30, one or more infrastructure cameras, etc., stored in the dynamic database, and by comparing it with the candidate navigation route 26, including the direction of travel specified in the request for real-time images 28 received from the user device computer 22, and by comparing it with the information about the general current position, direction of travel, and / or orientation of multiple remote vehicles 30, one or more infrastructure cameras, etc., stored in the dynamic database.

[0048] The server computer 24 stores information about the general current position and direction of travel of multiple remote vehicles 30 in a dynamic database based on information received from each of the remote vehicles 30. The received information includes, for example, a unique identifier that identifies a particular remote vehicle 30, information about the general current position of a particular remote vehicle 30, information about the direction of travel of a particular remote vehicle 30, and information about a planned navigation route or equivalent for a particular remote vehicle 30. The server computer 24 receives such information from the remote vehicles 30 generally continuously, at regular intervals (for example, every 20 seconds), or intermittently. The server computer 24 stores information about the general current position and direction of travel of multiple remote vehicles 30 in the dynamic database, calculated, for example, by extrapolation or equivalent, based on the information in the dynamic database, and / or calculated based on information received from the remote vehicles 30, for example, a planned navigation route for a remote vehicle 30.

[0049] The server computer 24 receives real-time images 28 taken along a candidate navigation route 26 from a remote vehicle 30, one or more infrastructure cameras, etc., for example via a communication network 32. The received real-time images 28 include information that identifies the location, direction of travel, and / or the orientation of the remote vehicle 30 or infrastructure camera that took the image. The server computer 24 is programmed to identify one or more objects in the received real-time images 28, for example as described above for the user device computer 22. The one or more identified objects relate to one or more of the user's preferences. The server computer 24 collects information about the identified objects, for example as described above for the user device computer 22. The server computer 24 transmits one or more captured real-time images 28 to the user device computer 22, for example via the communication network 32. Along with the real-time images 28, the server computer 24 transmits information that identifies the identified objects and information about the identified objects to the user device computer 22.

[0050] <Remotely operated vehicle> The remote vehicle 30 is a separate vehicle from the user device 34, and is, for example, located a certain distance away from the user device 34. For example, the remote vehicle 30 is located at least 1 / 4 mile away from the user device 34, but is not limited to this. The remote vehicle 30 is a passenger car, a van, a sports utility vehicle, and / or other type of vehicle.

[0051] Each remote vehicle 30 includes one or more cameras 42, a navigation unit 44, a communication unit 46, and a vehicle computer 48. The vehicle computer 48 communicates electronically with other components of the remote vehicle 30, such as the cameras 42, the communication unit 46, the navigation unit 44, etc. For example, the cameras 42, the communication unit 46, and / or the navigation unit 44 are communicably coupled to the vehicle computer 48 via a wired and / or wireless electronic communication structure, including, but not limited to, a communication bus, a controller area network (CAN) bus, a local interconnection network (LIN) bus, or equivalents thereof.

[0052] The camera 42 detects light and generates electronic information that identifies an image based on the detected light. Each camera 42 includes, for example, an active pixel sensor (CMOS sensor) or an equivalent thereof. The camera 42 is oriented to capture, for example, still images and / or video from outside the remote vehicle 30. For example, but not limited to, the camera 42 is oriented to face forward, backward, and / or to the side of the remote vehicle 30 incorporating such camera 42.

[0053] The navigation unit 44 determines the location of the remote vehicle 30, the direction of travel of the remote vehicle 30, and a navigation route or equivalent planned for the remote vehicle 30. The navigation unit 44 includes a GPS device, an electronic compass, or equivalents thereof, which are communicably coupled to the vehicle computer 48. The navigation unit 44 can generate location information and / or direction of travel information indicating the location of the remote vehicle 30 by receiving one or more GPS signals from one or more GPS satellites. The navigation unit 36 ​​is configured to generate direction of travel information, for example, based on an electronic compass. The GPS signals communicated to the vehicle computer 48 include location information including National Marine Electronics Association (NMEA) messages, latitude and longitude datasets, addresses, names of known places based on a location database, or equivalents thereof. In addition, the navigation unit 44 is interchangeable with other systems capable of generating location-indicating outputs. Examples include local positioning systems that provide location based on cellular signals and broadcast towers, or radio signal detection devices capable of triangulating location by radio signals received from one or more radio signal antennas. The navigation unit 44 includes a digital map containing geographical information. The digital map may be dynamic.

[0054] The communication unit 46 connects the remote vehicle 30 to the communication network 32 in a communicative manner. The communication unit includes, but is not limited to, an antenna, a modem, a LAN port, a Wi-Fi card, a WiMAX card, mobile communication hardware, near-field communication hardware, satellite communication hardware, cellular network communication hardware, and / or any wired or wireless hardware for communicating with other networks and / or devices. The cellular network includes, but is not limited to, a Global System for Mobile Communications (GSM) network, a 3G Universal Mobile Telecommunications System (UMTS) network, a 4G Long-Term Evolution (LTE) network, a 5G network, and equivalents thereof.

[0055] The vehicle computer 48 is any combination of devices or components comprising a processor and non-temporary computer-readable memory. The processor is any device capable of executing a machine-readable instruction set stored in non-temporary computer-readable memory. Therefore, the processor is an electrical control unit, an integrated circuit, a microchip, a computer, or other computing device.

[0056] Non-temporary computer-readable memory includes RAM, ROM, flash memory, hard drives, or any non-temporary memory device capable of storing machine-readable instructions, such that machine-readable instructions can be accessed and executed by the processor. The machine-readable instruction set includes, for example, machine code executed directly by the processor 132, or logic or algorithms written in any programming language of any generation (e.g., 1GL, 2GL, 3GL, 4GL, or 5GL), such as assembly language, object-oriented programming language (OOP), scripting language, microcode, etc., which are compiled or assembled into machine-readable instructions and stored in non-temporary computer-readable memory. Alternatively, the machine-readable instruction set may be written in a hardware description language (HDL), such as logic implemented via either a field-programmable gate array (FPGA) configuration or an application-specific integrated circuit (ASIC), or their equivalents. Thus, the functions described herein are implemented in any conventional computer programming language, either as pre-programmed hardware elements or as a combination of hardware and software components.

[0057] The vehicle computer 48 is programmed to collect one or more real-time images 28, that is, the memory of the vehicle computer 48 stores machine-readable instructions that can be executed by the processor of the vehicle computer 48. The real-time images 28 may be captured while traveling along a candidate navigation route 26 in the direction of travel. For example, in response to receiving a real-time image 28 request from the server computer 24, the vehicle computer 48 commands the camera 42 to capture an image almost simultaneously, i.e., within the minimum threshold time. As another example, the vehicle computer 48 may command the camera 42 to capture an image at a specified interval, for example, every 5 seconds.

[0058] The vehicle computer 48 transmits the captured real-time image 28 to the server computer 24, for example, via the communication network 32, almost simultaneously with the capture of the image. Along with the captured real-time image, the vehicle computer 48 transmits information that identifies the location and direction of travel of the remote vehicle 30.

[0059] <Processing> Figure 2 is a process flow diagram illustrating an exemplary process 200 for providing a user with a selection of candidate navigation routes 26 based on real-time images 28 taken along the candidate navigation routes 26. Executable machine-readable instructions for performing the steps of process 200 are stored in the memory of the user device computer 22 of the user device 34, the memory of the server computer 24, and / or the memory of the vehicle computer 48 of the remote vehicle 30, the memory of one or more infrastructure cameras, etc.

[0060] Process 200 begins in block 205, where a user request is received by the user device 34. For example, the user provides the user request as input to the user interface 40 and / or in a manner described herein.

[0061] Next, in block 210, the user device 34 identifies one or more candidate navigation routes 26 in response to and based on a user request received, for example, in block 205. The user device 34 identifies one or more candidate navigation routes 26 in cooperation with the server computer 24 and / or in a manner described herein.

[0062] Next, in block 215, the user device 34 sends a request to the server computer 24 for real-time images 28 taken along one or more candidate navigation routes 26, for example via the communication network 32 in the manner described herein.

[0063] In block 220, the server computer 24 identifies one or more remote vehicles 30, one or more infrastructure cameras, etc., located along one or more candidate navigation routes 26 and traveling and / or facing the direction of travel specified in the request received in block 215, as described herein, for example.

[0064] Next, in block 225, the server computer 24 transmits a request for real-time images 28 to one or more remote vehicles 30, one or more infrastructure cameras, etc., located along one or more candidate navigation routes 26 and traveling and / or facing the direction of travel identified in block 220, for example via a communication network 32 in the manner described herein.

[0065] Next, in block 230, in response to receiving a request for real-time images 28 transmitted in the manner described herein, for example in block 225, one or more remote vehicles 30 located along one or more candidate navigation routes 26 and traveling and / or facing the direction of travel, one or more infrastructure cameras, etc., capture real-time images 28.

[0066] In block 235, one or more of the remote vehicles 30, or one or more infrastructure cameras, transmit real-time images 28 captured in block 230 to the server computer 24, for example, via the communication network 32.

[0067] In block 240, the server computer 24 receives, for example, real-time images 28 that were captured by one or more remote vehicles 30, one or more infrastructure cameras, etc., located along the candidate navigation route 26, and transmitted in block 235, via a communication network 32.

[0068] In block 245, the server computer 24 identifies objects in the real-time image 28 received in block 245, as described herein, for example.

[0069] In block 250, the server computer 24 obtains or collects information about the objects identified in block 245, for example, as described herein.

[0070] In block 255, the server computer 24 transmits, for example, the real-time image 28 received in block 240 and information about the object collected in block 250 to the user device 34 via the communication network 32.

[0071] In block 260, the user device 34 receives the real-time image 28 and information about the object transmitted in block 255.

[0072] Next, in block 265, as shown in Figure 3 and discussed herein, for example, the user device 34 displays the real-time image 28 received in block 260 along with one or more candidate navigation routes 26 from which the real-time image 28 was taken.

[0073] In block 270, as discussed herein, the user device 34 displays, for example, a real-time image 28 in block 265, while simultaneously presenting information about an object received in block 260.

[0074] Next, in block 275, the user device 34 receives, for example, a user input via the user interface 40 in a manner described herein, indicating approval of a candidate navigation route 26 displayed in block 265, or a user input indicating approval of a specific one of the candidate navigation routes 26 displayed in block 265.

[0075] Next, in block 280, the user device 34 navigates along an approved route, in response to receiving approval for a candidate navigation route 26 or approval for a specific one of the candidate navigation routes 26, as described herein, for example.

[0076] From the above, it is understood that this specification describes a system and method for providing a real-time route view along a candidate navigation route by identifying a candidate navigation route, requesting one or more real-time images taken along the candidate navigation route, receiving the real-time images, and displaying the real-time images.

[0077] While specific embodiments have been illustrated and described herein, it should be understood that various other changes and modifications can be made without departing from the scope of the subject matter described in the claims. Furthermore, while various aspects of the subject matter described in the claims have been described herein, it is not necessary to use such aspects in combination. Therefore, the appended claims are intended to cover all changes and modifications within the scope of the subject matter described in the claims.

Claims

1. A system for providing real-time route views, User device computer and A server computer that communicates electronically with the aforementioned user device computer. Equipped with, The aforementioned user device computer is Identify candidate navigation routes, A request for real-time images taken along the candidate navigation route is sent to the server computer. One or more real-time images are received from the server computer. Programmed to display one or more of the received real-time images, The aforementioned server computer In response to receiving the request from the user device computer for real-time images to be captured along the candidate navigation route, collect one or more real-time images along the candidate navigation route. A system programmed to transmit one or more captured real-time images to the user device computer.

2. The system according to claim 1, wherein the user device computer is further programmed to identify one or more objects in the real-time image and to present information about the identified objects.

3. The system according to claim 2, wherein the user device computer is further programmed to display the received one or more real-time images and simultaneously present the information relating to the identified object audibly.

4. The system according to claim 1, wherein the user device computer, after displaying the received one or more real-time images, receives user input indicating approval of the candidate navigation route, and is further programmed to navigate the approved navigation route in response to receiving the user input.

5. The system according to claim 1, wherein the user device computer is programmed to identify a second candidate navigation route, send a request for real-time images along the second candidate navigation route, receive and display one or more real-time images taken along the second candidate navigation route, and request a selection of the candidate navigation route or the second candidate navigation route.

6. The system according to claim 1, wherein the server computer is further programmed to identify one or more remote vehicles located along the candidate navigation route, to send a request for real-time images to the identified one or more remote vehicles located along the candidate navigation route, and to receive real-time images from the identified one or more remote vehicles located along the candidate navigation route.

7. The system according to claim 1, wherein the candidate navigation route includes information that identifies the direction of travel, and the request for real-time images taken along the candidate navigation route includes the direction of travel.

8. The system according to claim 7, wherein one or more real-time images are captured by one or more remote vehicles traveling along the candidate navigation route in the direction of travel.

9. A user device for providing a real-time route view, A user device computer comprising a processor and memory for storing machine-readable instructions, The aforementioned machine-readable instruction is: Identify candidate navigation routes, A request is sent for real-time images taken along the aforementioned candidate navigation route. One or more real-time images taken along the aforementioned candidate navigation route are received. A user device that can be operated by the processor to display one or more of the aforementioned received real-time images.

10. The user device according to claim 9, wherein the machine-readable instruction includes an instruction for identifying one or more objects in the real-time image and for presenting information about the one or more identified objects.

11. The user device according to claim 10, wherein the machine-readable command includes a command to display one or more received real-time images and simultaneously present the information relating to the identified object in audio.

12. The user device according to claim 10, wherein the machine-readable instruction includes an instruction that identifies the candidate navigation route based on user preferences, and the one or more identified objects relate to the user preferences.

13. The user device according to claim 9, wherein the machine-readable command includes a command to receive user input indicating approval of the candidate navigation route after displaying the received one or more real-time images, and to navigate the approved navigation route in response to the receipt of the user input.

14. The user device according to claim 9, wherein the machine-readable instruction includes an instruction to identify a second candidate navigation route, send a request for real-time images along the second candidate navigation route, receive and display one or more real-time images taken along the second candidate navigation route, and request a selection of the candidate navigation route or the second candidate navigation route.

15. The user device according to claim 9, wherein the candidate navigation route includes information that identifies the direction of travel, and the received real-time images are captured by one or more remote vehicles traveling along the candidate navigation route in the direction of travel.

16. A method for providing a real-time route view, Identifying candidate navigation routes, Sending a request for real-time images taken along the aforementioned candidate navigation route, The system receives one or more real-time images taken along the aforementioned candidate navigation route, Displaying one or more of the aforementioned received real-time images. Methods that include...

17. The method according to claim 16, further comprising: taking one or more real-time images along the candidate navigation route in response to receiving the request for real-time images to be taken along the candidate navigation route; and transmitting the one or more real-time images that have been taken.

18. The method according to claim 17, wherein the request for the real-time images taken along the candidate navigation route includes the direction of travel, and the one or more real-time images are taken from one or more remote vehicles traveling along the candidate navigation route in the direction of travel.

19. The method according to claim 16, further comprising identifying one or more objects in the real-time image, displaying the received one or more real-time images, and simultaneously presenting information about the identified object.

20. The method according to claim 16, further comprising identifying a second candidate navigation route, transmitting a request for real-time images along the second candidate navigation route, receiving and displaying one or more real-time images taken along the second candidate navigation route, and requesting a selection of the candidate navigation route or the second candidate navigation route.

Images